1
vote
2answers
119 views

How to split a file and compress directly?

I've a 100GB file and I want to split into 100 of 1GB file each (by line break) e.g. split --bytes=1024M /path/to/input /path/to/output For the 100 files generated, I want to apply gzip/zip to ...
0
votes
6answers
1k views

How to run command on parts of input file

I have ~ 40GB file, and a filter command that for some reason breaks when I try to run it on the file (even when passed via pipe). But. It doesn't fail when I split input file into many small files, ...