Questions tagged [compression]
Decrease the size of data, typically to preserve space or reduce network bandwidth required while transmitting files. It can be applied to files, folders, disks, etc.
420
questions
133
votes
7
answers
418k
views
Does Windows have a built-in ZIP command for the command line?
Since Windows Explorer (since at least Windows XP) has some basic support for ZIP files, it seems like there should be a command-line equivalent, but I can't seem to find any sign of one.
Does ...
104
votes
11
answers
139k
views
How to zip/unzip files in Powershell?
Is there a one-liner that will zip/unzip files (*.zip) in PowerShell?
84
votes
2
answers
108k
views
How can I zip/compress a symlink?
Is it possible and how can I zip a symlink from a linux shell?
52
votes
12
answers
66k
views
Unzipping files that are flying in through a pipe
Can I make unzip or any similar programs work on the standard output? The situation is I'm downloading a zip file, which is supposed to be unzipped on the fly.
Related issue: How do I pipe a ...
51
votes
5
answers
81k
views
Is it possible to enable http compression for requests?
I see lots of information about enabling http compression for server responses but what about for incoming requests. Wouldn't it make sense for the browsers to compress large form posts before ...
32
votes
5
answers
33k
views
tar: How to create a tar file with arbitrary leading directories w/o 'cd'ing to parent dir
Say I have a directory of files at
/home/user1/dir1
and I want to create a tar with only "dir1" as the leading directory:
/dir1/file1
/dir1/file2
I know I can first cd to the directory
cd /home/...
30
votes
2
answers
58k
views
Transparent compression filesystem in conjunction with ext4
I am trying to test a project that needs compressed storage with use of the ext4 file system since the application I use relies on ext4 features.
Are there any production/stable solutions out there ...
27
votes
5
answers
64k
views
How to unzip files bigger than 4GB?
I'm using RHEL 5.6 and unzip-5.52-3.el5. I'm trying to unzip a big file, but I get the error:
unzip -o test.zip -d unzip/
error: Zip file too big (greater than 4294959102 bytes)
Archive: test.zip
...
24
votes
8
answers
48k
views
How to check if Apache compression is working?
I just added the following to my Apache config file:
AddOutputFilterByType DEFLATE text/html text/plain text/xml
How do I check if it is actually working? Nothing on the browser tells me if the page ...
20
votes
4
answers
24k
views
rsync --compress-level: which compression levels can be used?
Rsync has command line arguments for compression:
-z, --compress compress file data during the transfer
--compress-level=NUM explicitly set compression level
What does --compress-...
19
votes
6
answers
34k
views
Are there other options to unzip a file in Ubuntu besides "unzip"? [closed]
My sysadmin is unreachable right now, and I have a zipped file on the server that I would like to unzip...however, we don't currently have zip and unzip installed, and I don't have root access to ...
19
votes
14
answers
25k
views
Best compression for ZFS send/recv
I'm sending incremental ZFS snapshots over a point-to-point T1 line and we're to a point where a day's worth of snapshots can barely make it over the wire before the next backup starts. Our send/recv ...
19
votes
5
answers
37k
views
Unzip from stdin to stdout - funzip, python
The goal is to read a zip file from stdin and uncompress to stdout.
Funzip works and is the solution I am looking for, the zip contains a single file, unfortunately funzip fails when the compressed ...
18
votes
5
answers
46k
views
For large files compress first then transfer or rsync -z? which would be fastest?
I have a ton of relativity small data files but they take up about 50 GB and I need them transferred to a different machine. I was trying to think of the most efficient way to do this.
Thoughts I had ...
17
votes
3
answers
15k
views
How to list content from a tar file without recursion?
I've a tar (gz, bzip) file and want to see its content, but not recursively.
This is: The "first level" in the file.
I already know how to see content:
tar -ztf file.tar.gz
But it's recursive!!
...
17
votes
2
answers
6k
views
How can I list the content of a zip archive, but only the first level?
I have a big zip file and I want to know what it's contain. I know I can run:
zipinfo file.zip
but the output is too verbose and there are a lot of files in the sub-directories.
I want to see a ...
16
votes
3
answers
55k
views
How to use GZip to create a .zip file
Is it possible to use GZip to create a .zip file? I've been Googling and can't find anything that leads me to believe this is possible
16
votes
1
answer
72k
views
How do you create a qcow2 file that is small yet commodious on a Linux server?
I believe that there are qcow2 files that displace a small amount of gigabytes on a server (as verified with a df -h on the physical server) yet allow for lots of space when you log into the virt-...
15
votes
3
answers
41k
views
Content-Length not sent when gzip compression enabled in Apache?
I would really appreciate some help understanding this Apache behaviour.
I am communicating to PHP from an iPhone Objective-C app in application/json. Gzip compression is enabled on the server, and ...
15
votes
3
answers
25k
views
Store file in zip archive with different name (linux command shell)
In a linux command line, you zip a file by:
zip -mqj archive.zip file.txt
Now, I need to store 'file.txt' as 'file2.txt' in 'archive.zip', without renaming the file before zipping. When unzipped, ...
15
votes
4
answers
1k
views
HTTP Compression on IIS 6.0 (Windows Server 2003)
There are many things you have to do/consider when you want to enable HTTP compression on IIS 6.0 (Windows Server 2003).
Can somebody please provide a comprehensive list of the actions you have to ...
13
votes
2
answers
6k
views
Is it good practice to use NTFS Compression on IIS Log folders?
Is it a good practice to use NTFS Compression on a IIS Log folders and files?
I was able to go down from 20GB to 7GB by doing this. The IIS logs are per day, and have an average size of 20MB but some ...
12
votes
4
answers
1k
views
How to use the full LTO-2 Tape backup capacity of 200/400GB?
I have an Ultrium 448 tape drive, LTO-2 tapes (200/400GB) and I'm using HP Data Protector as the backup client. The system is set to format and overwrite all tapes but I never manage to get more that ~...
12
votes
5
answers
13k
views
How do you set bzip2 block size when using tar?
I am using tar to backup a linux server to tape. I am using the -j option to compress the file with bzip2, however I can't see a way to adjust the block size options for bzip2 from tar. The default ...
12
votes
8
answers
43k
views
Can't send to compressed (zipped) folder
I'm IT Technician at a Secondary School and our Network Manager is away on long term leave. Currently our student's aren't able to send to - compressed folder. When trying the error 'File not found ...
12
votes
2
answers
2k
views
Is there a compression format that supports tail operations?
I'm looking for a compression format that supports being tailed. Meaning you dont have to read the entire file to get the last X uncompressed bytes.
Is this possible with any of the formats like bzip2,...
11
votes
6
answers
2k
views
Tips on efficiently storing 25TB+ worth million files in filesystem
Say you are confronted with 25 TB worth uncompressed log files and you have at your disposal an array of 20 commodity boxes with collective free storage capacity of 25 TB.
How would you store these ?...
11
votes
4
answers
7k
views
Why is IIS 7 is ignoring certain (but not all) MIME types for compression? Giving error: DYNAMIC_COMPRESSION_NOT_SUCCESS - Reason 12
So, I'm a bit of an IIS7 n00b but I've used most of the old IIS systems going back to 3. I'm trying to turn on dynamic compression and it's working, mostly. It doesn't work for my ADO.Net Data ...
10
votes
6
answers
7k
views
File size with zfs compression
I usually estimate the size of a whole directory tree using du -ks $DIRECTOY_TREE_ROOT, but this method cannot be used when zfs compression is on.
The total displayed by ls -l is ok for a single ...
10
votes
9
answers
3k
views
On a modern system, will using disk compression give me better overall performance?
It seems that CPU increases have outpaced disk speed for a while. Assuming a desktop or laptop with modern dual core Intel/AMD CPU and a single average SATA disk, would doing compression on most all ...
10
votes
6
answers
12k
views
Does gunzip work in memory or does it write to disk?
We have our log files gzipped to save space. Normally we keep them compressed and just do
gunzip -c file.gz | grep 'test'
to find important information but we're wondering if it's quicker to keep ...
10
votes
2
answers
6k
views
Apache2 and logrotate: delaycompress needed?
I am currently looking at the file size of my Apache logs as they became huge. In my logrotate configuration, I have delaycompress enabled. Does Apache really need this (as the logrotate documentation ...
9
votes
28
answers
25k
views
What is the most reliable way to send a zip file via email? [closed]
I often have clients email zip files to me only to find out that their corporate firewall has stripped the attached file. What is the most straight-forward way to send a compressed file through email ...
9
votes
2
answers
22k
views
How can I enable GZIP compression in Jetty?
I have a Jetty server which does not have compression enabled (I tested this here). How can I enable compression?
9
votes
2
answers
15k
views
SSH compression
I want max compression on my ssh tunnel cos I'm on a dialup line ;)
At the man page can one can read:
CompressionLevel
.....The meaning of the values is the same as in gzip(1). ...
9
votes
3
answers
13k
views
Need help with some IIS7 web.config compression settings
I'm trying to configure my IIS7 compression settings in my web.config file. I'm trying to enable HTTP 1.0 requests to be gzip.
MSDN has all the info about it here.
Is it possible to have this config ...
8
votes
7
answers
7k
views
Which archive/compression format?
Zip, Rar, 7z, Gzip, BZip2, Tar etc. I'm hearing 7z is the flavor of the month, why? Is it best for all situations or are there better choices for specific situations.
Or maybe the actual file ...
8
votes
5
answers
18k
views
Should I use rsync compression over a gigabit LAN?
I normally use rsync's -z option to enable compression when transferring files over the internet. However, if I am on my own (idle) gigabit LAN, do I still want compression? Or will it be faster ...
8
votes
3
answers
15k
views
create zip based on contents of a file list
I created a file with diff files diff-files.txt it has 6000+ file paths in it. Now I want want to create a zip based on those files.
I know I can zip multiple files with:
zip diffedfiles.zip file1 ...
8
votes
5
answers
2k
views
Does ZFS cache Compressed or Uncompressed data in a ZFS file-system with compression turned on?
ZFS supports file-system compression and it also caches frequently or recently accessed data.
If a system has lots of CPU but the underlying data storage system is slow. It is possible that ZFS would ...
8
votes
4
answers
3k
views
How do I enable HTTP compression under apache2?
How do I configure Apache2 to support HTTP compression?
8
votes
2
answers
12k
views
View/Find all compressed files on the server?
I need to find all compressed files/folders regardless of file format on a Windows Server 2003 machine.
Search options do not provide this capability.
Is there a way to list/view all compressed ...
8
votes
4
answers
1k
views
Join large overlapping files
I am trying to recover a (MySQL) database from a crashed disk. There are a number of recent dumps, which are corrupted bz2 files. Since the database does not change often, the dumps should be nearly ...
8
votes
3
answers
3k
views
Is it generally better to compress content on the proxy server or the app server?
We're using an F5 for load balancing and SSL proxying. Behind it we're serving up java applications with Tomcat instances. These are fairly small applications - hundreds of concurrent users.
I'd ...
8
votes
1
answer
7k
views
NTFS Compression with Deduplication?
I have installed the Data Deduplication role on Windows Server 2016. Per Microsoft, it's using NTFS compression to help save space:
Data deduplication stores more data in less physical space. It ...
8
votes
3
answers
9k
views
IIS 7.5 ignoring HttpCompression settings in web.config
I'm trying to enable dynamic compression for the mime type application/json.
In applicationHost.config, I've made the following change:
<section name="httpCompression" allowDefinition="...
7
votes
10
answers
15k
views
Can I compress an encrypted file?
I have an encrypted file (AES Symmetric encryption).For backup purposes and to save disk space, Can I compress (lossless) the file without worrying about messing up the decryption? If so, can you ...
7
votes
7
answers
34k
views
How to "really" reduce the size of KVM VMs images?
I've spent many hours searching the web about how to shrink KVM virtual disk images, especially for Windows guests, with no luck.
All I've found is to zeroize the VM free space, defrag the virtual ...
7
votes
3
answers
15k
views
ssh can't set compression level to more, than 6
When I do (I've tried various ways):
ssh -v -C -o CompressionLevel=9 user@ip
ssh -v -C -o 'CompressionLevel=9' user@ip
ssh -v -o 'Compression=yes' -o 'CompressionLevel=9' user@ip
ssh -v -o '...
7
votes
4
answers
8k
views
silent 7za compression
Without doing stdout redirection. Is it possible to have a silent run of 7za?