Unix/Linux command that expands a compressed file to standard output.
Questions tagged [zcat]
80 questions
2
votes
2 answers
Using mmap with popen
I need to read in and process a bunch of ~40mb gzipped text files, and I need it done fast and with minimal i/o overhead (as the volumes are used by others as well). The fastest way I've found thus for this task looks like this:
def…
Nik
- 315
- 2
- 8
2
votes
3 answers
zcat to grep with file name
ls -ltr|grep 'Mar 4'| awk '{print $9 }'|xargs zcat -fq |grep 12345
I'm now using this command to list the records that contain my numeric string how can i alos get this command to print the name of the file the strings were found in?
thanks
matt123
2
votes
0 answers
Why does zcat throw "invalid compressed data" when decompressing a file compressed with GzipOutputStream?
In C++, I use the GzipOutputStream to compress the data, which is a big string. At first, I have to convert the string to a proto message, like this:
syntax = "proto3";
package Serverlog_Compress;
message ServerlogMessage {
string data =…
user12957385
- 31
- 2
2
votes
2 answers
Using zcat in windows
I have a big XML file which was splited in unix.
I got a readme file saying:
zcat FILE.xml.part1.gz File.xml.part2.gz > File.xml
should connect them into one parsable XML file.
The separation is guaranteed to be on a byte boundary, but not an XML…
Bjorkson
- 601
- 2
- 8
- 12
2
votes
1 answer
Unix: Last command - How do I execute a sub command for file input?
I am trying to read an on old last file, but when I use the command below
it does not work as expected. Is there a way I can do it in one line?
last -f `zcat /var/log/wtmp.1.gz`
I want to be able to read the file without decompressing the file.
The…
caaruiz
- 79
- 1
- 9
2
votes
1 answer
Bash script pass find files to awk [Mac]
I have some gzipped log files in a directory and they look like this:
log/day_1_time_log_1.log.gz
...
log/day_1_time_log_100.log.gz
log/day_1_location_log_1.log.gz
...
log/day_1_location_log_100.log.gz
I would like to take the 4th column (some json…
Boxuan
- 4,937
- 6
- 37
- 73
2
votes
0 answers
Python not reading/extracting gzip file because...compression is too high?
This is code I'm working on to search for patterns in a tar.gz then output the files found out.
Earlier I had a question similar to this, but I've figured out the problem- when I pass in a tar.gz file that has been compressed to level 9, python…
Skorpius
- 2,135
- 2
- 26
- 31
2
votes
1 answer
How to test a directory of files for gzip and uncompress gzipped files in Python using zcat?
I'm in my 2nd week of Python and I'm stuck on a directory of zipped/unzipped logfiles, which I need to parse and process.
Currently I'm doing this:
import os
import sys
import operator
import zipfile
import zlib
import gzip
import subprocess
if…
frequent
- 27,643
- 59
- 181
- 333
1
vote
1 answer
How to run paste <(zcat f1.gz) <(zcat f2.gz) using python's subprocess?
I am trying to run paste <(zcat f1.gz) <(zcat f2.gz) using subprocess.
Here's what I have done so far:
ps1 = subprocess.Popen(('zcat', 'f1.gz'), stdout=subprocess.PIPE)
ps2 = subprocess.Popen(('zcat', 'f2.gz'), stdout=subprocess.PIPE)
ps3 =…
Fatemeh Ashari
- 13
- 2
1
vote
0 answers
using zcat to pass a gzipped file to a tool with options
A question like this was asked here, but never answered.
I need to pass fastq files as options to a tool that does not accept gzipped inputs. Is there really no option other than to unzip every one of them?
It fails when I pass the gzipped…
Jess
- 186
- 3
- 13
1
vote
1 answer
zcat skip streaming first line while reading from a compressed file
I am using the zcat command and streaming a .tar.gz file through a piping operation. I want to skip the first line while zcat reads the file. How is it possible?
Pujan Paudel
- 71
- 1
- 7
1
vote
0 answers
Problems finding terms in multiple .gz files recursively
We have various archive files in a folder, and I'm trying to recursively search through these for a particular string. I've found lots of examples of various ways to do this, but I've not been successful. This is from my latest iteration:
find .…
agsYAV49
- 11
- 2
1
vote
1 answer
How to use zgrep to display all words of a x size from a wordlist?
I want to display all the words from my wordlist who start with a w and are 9 letters long. Yesterday I learnt a bit more on how to use zgrep so I came with :
zgrep '\(^w\)\(^.........$\)' a.gz
But this doesn't work and I think it's because I don't…
J.erome
- 688
- 7
- 26
1
vote
1 answer
How can I specify a regex in a zgrep/zcat command?
I want to find in a list of words, every words with a least 3 times the same letter in it. To achieve that I did .*(\w).*\1.*\1.*\1.* and you can test it here http://www.regexplanet.com/advanced/java/index.html but I don't know how to put it in my…
J.erome
- 688
- 7
- 26
1
vote
1 answer
how to use zcat without the warning when using pipe
I'm trying to silent zcat warning through the option -q or 2>/dev/null
so far nothing is working. I keep getting the same warning when a file name is missing.
I'm looping through 100s of compressed files to extract a specific data. The idea is if…
Driven
- 161
- 1
- 9