Questions tagged [compression]
Decrease the size of data, typically to preserve space or reduce network bandwidth required while transmitting files. It can be applied to files, folders, disks, etc.
421
questions
-1
votes
0
answers
16
views
Recursively xv compress each file in multiple subdirectories?
I have multiple directories and subdirectories, with files inside.
I would like to recursively compress each file in these (sub)directories with xv (xz -9ze -T0) which are not .xv files yet, without ...
- 99
1
vote
1
answer
58
views
Does "btrfs filesystem defrag -c" (compress option) force compression?
BTRFS with compression enabled uses a heuristic not to compress files that are not well compressible. Does compressing existing files with "btrfs filesystem defrag -c" also use the heuristic,...
- 1,643
0
votes
0
answers
29
views
Re-compressing XZ kernel module?
I have a kernel module produced by kernel option CONFIG_MODULE_COMPRESS_XZ. When I manually decompress the file with unxz and re-compress with xz, the new .ko.xz file differs from the original .ko.xz ...
- 51
1
vote
0
answers
28
views
zfs send to NAS with PIGZ compression slowness
The following command that I use to backup used to take 5 minutes to complete and since last week it is taking around 25 minutes. This is after restarting a switch stack the server is connected to. ...
- 11
2
votes
1
answer
198
views
Ways to make deployment ready pre-compressed brotli files on windows server
I am dynamically generating HTML files, which I use for deployment.
This is what I use to make these HTML file:
function writeFile()
Dim objStream
Set objStream = CreateObject("ADODB....
- 264
1
vote
2
answers
78
views
Nginx performance issues
I made a website using React and connected to Strapi. But I need to keep some background videos mainly on the home page.
The main banner has 4 videos and the background of the page is also a video.
...
1
vote
2
answers
259
views
Will disk compression impact the performance of a MySQL database?
I've recently setup a TrueNAS storage server and hosted on it is a MySQL database. It's about 3GB; I'm not too concerned with how much space it's utilising.
By default, TrueNAS enables LZ4 compression ...
- 113
1
vote
1
answer
292
views
Cloudflare uncompressing, then re-compressing response bodies
I'm using Node.js/express and if I set the headers
Content-Type: application/json
Content-Encoding: gzip
and send gzipped compressed bodies of JSON data, this works as expected - when not behind ...
- 11
0
votes
1
answer
48
views
Debian: Simple way to compress files wanted
I often need to compress archives in Linux. is there a simpler way instead of always building such complicated commands?
tar zcvf /tmp/mybackup.tar.gz /home/important
- 595
0
votes
0
answers
2k
views
is there a faster way to extract a large file tar in lz4 format?
I have a 7 TB file that is tared with lz4 compression. It takes about 8 hours to extract. is there a faster way to do this?
This is the command I use:
lz4 -d /mnt/tmp/7TBFile.lz4 | tar -xvf -
2
votes
0
answers
534
views
Are servers allowed to add a Content-Encoding header based on filename, if there is no compression on-the-fly?
Problem
Suppose we have a compressed archive on disk, e.g. file.tar.gz, which should be served as-is.
The file is served with Content-Type: application/gzip, but for some reason the server also adds a ...
- 137
1
vote
1
answer
200
views
Compressing rarely queried historical data in postgres
In short: compress multiple historical DB records to bigger, compressed records and temporarily uncompress it to the original records for easier querying, and clean that up when querying is done. ...
- 161
-1
votes
1
answer
396
views
MySQL JDBC compression in transit
Good morning,
I am looking to set my application connection to MySQL to compress data in transit.
I read MySQL doc. but it's not cleat to me if I need to do something on the client side, on the server ...
- 781
0
votes
1
answer
549
views
Returning compressed data when client omits accept-encoding header
In the HTTP 1.1 Standard, it says that "If no Accept-Encoding field is in the request, any content-coding is considered acceptable by the user agent."
Which means that a server can, for ...
- 227
1
vote
1
answer
338
views
Compress input files from rsync before writing to disk
I need to load big text files from another machine through a 10Gb link, files been creating by external/closed-source software that i don't have access to change (for example to make it to compress ...
- 111
0
votes
2
answers
1k
views
scp with -C flag does not perform compression
The scp man page shows that the -C flag can be used for compressing a file on the fly while performing a remote copy -
$ man scp | grep "\-C"
-C Compression enable. Passes the -C ...
- 123
0
votes
1
answer
16
views
Changes in compression rate
Every month I usually compress all my database dumps into a rar file.
I was very happy with the compression rate, as each compressed file was usually about 30MB, with a compression rate of 99%.
Since ...
- 235
0
votes
1
answer
615
views
Stratis filesystem will not mount after a reboot because the disk is full
It concerns a rhel8 server that has two disks (sda and sdb), and only on the sdb disk a Stratis filesystem has been created. After a restart of the server, no more mount point appears on the sdb disk. ...
0
votes
0
answers
402
views
community.windows.win_unzip shows Filename contains relative paths which would extract outside the destination error message
I'm trying to unzip a zip file in a remote server
This file was download first from jFrog repository
The following task should handle the unzip:
- name: Unzip Config.zip file
community.windows....
- 87
0
votes
0
answers
312
views
Transfer Compression with Cloudfront and nginx SSI
Is there any way to enable GZIP / Brotli compression on Cloudfront if the origin is a nginx server with SSI (Server Side Includes) enabled?
The issue seems that nginx SSI drops the Content-Length ...
- 141
0
votes
1
answer
170
views
ZFS Transfer extremely bursty
I am testing a new ZFS configuration with z-std for log storage and storage of highly compressible files.
The Array is tested on a 5 drive raidz-1 in a virtual machine on my PC which has direct access ...
- 11
5
votes
1
answer
481
views
Unable to compress 18GB file on Ubuntu Linux 18.04
It never happened to me before, but I'm unable to do a simple task such as compressing a 18.5 GB file on Ubuntu Linux 18.04 with any of the popular compressing tools such as gzip, bzip2 and 7z. All of ...
- 181
0
votes
1
answer
1k
views
How do I make a compressed disk image (qcow2 and/or squashfs) from a disk directly?
I have a disk attached to my server and I want to make the disk image of it downloadable on the same server. The disk itself is 500GB but I don't want to have a 500GB disk image for download. I'd like ...
2
votes
2
answers
1k
views
Cannot turn off SSLCompression in Apache?
My Apache is 2.4.46 and is using Openssl version 1.1.1f
I have set the directive SSLCompression Off. Even if i enable it, it says SSL Compression is not supported, which is good I guess.
But, when I ...
user549144
0
votes
2
answers
434
views
Tool to create pre-compressed [Brotli] files
I am trying to pre-generate compressed brotli files in designated directories for static html, css and js files.
I found a rust tool here – https://github.com/neosmart/static-compress
Besides this, ...
- 21
0
votes
1
answer
865
views
Preserve Cloudfront brotli compression through a nginx reverse_proxy
My question is similar to Nginx reverse-proxy to a cloudfront distribution and preserve gzip compression but especially regarding the new brotli compression algorithm that is supposed to perform ...
1
vote
0
answers
542
views
btrfs compress existing data with snapshots
I'm runninga a btrfs raid1 on 5 8 TB disks, where primarily urbackup stores backups to. Urbackup uses snapshots extensively. compsize produces the following output:
Processed 106161732 files, 57335496 ...
- 111
0
votes
0
answers
71
views
Opera Mini Compression Technology
I'm trying to find the technology used in Opera Mini's browser. I can't find anything related to this, so I would like to know if anyone knows whether there exists a paper on this or if there is ...
- 101
1
vote
1
answer
6k
views
How to install nginx-module-brotli?
I have a server with ubuntu 20.04
But I am unable to install nginx-module-brotli for my nginx server.
Here are the commands I use to install Brotli for Nginx :
$ sudo apt install git gcc cmake ...
- 39
0
votes
1
answer
162
views
How to syncronize XML files and store them as .gz at destination using rsync
I'm using rsync -z to backup a large amount of XML files. The source is a huge machine with thousands of GBs available on disk, but the destination is not that powerful.
What I want to do is to ...
4
votes
1
answer
2k
views
Compress old emails with dovecot
How can I configure dovecot to compress/archive old emails (say 6 months old) with the users still being able to read them, to save some precious space on the server?
- 201
0
votes
0
answers
157
views
tar | pigz dies if run under cron
I run a kind of backup script using tar joint with pigz for compression.
The problem is that
tar -zcf /file.tar.gz /folder
runs under cron.
While
tar -I pigz -cvf /file.tar.gz /folder
nor
tar ...
- 151
0
votes
0
answers
259
views
Nginx serve another file when possible
I currently compile my assets with Webpack's BrotliPlugin, which creates a separate file, brotli-encoded, i.e.:
style.css
style.cssb
Here's what I want to achieve:
Consider the HTML page requests ...
- 101
1
vote
0
answers
181
views
haproxy compression type regex/wildcard
As stated in haproxy docs, we can use "compression type" with list of mime types.
In my case, there is REST API backend, and the compression should be applied to multiple vendor-specific versioned ...
- 483
2
votes
2
answers
1k
views
Extracting single file from large tar.gz file
I have an incredibly large tarball. I'd extract several files out of many thousands within the archive. I'm on CentOS 6.10 running GPFS 4.2.3. I've seen from this answer that pigz is useful in ...
0
votes
1
answer
71
views
How do I find out why compact command appears to hang on certain files?
After extending the hard disk sizes on a Windows 2008 server I am trying to revert the compressing of a lot of files on the hard drives.
I use the command compact /u /s /i /f /q for it.
But it ...
- 1
0
votes
2
answers
621
views
How to untar a very large file with missing space
I have reinstalled my server and before that i did a backed up.
My backup is a tar file without special compress option (meaning that the size of the tar file is approximately the same than the size ...
3
votes
1
answer
4k
views
How to install the brotli nginx module properly on debian
I'm trying to setup brotly compression on a nginx/1.10.3 server running on Debian 9.5 Stretch / Linux 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u5 (on a
lightsail instance). I used the following ...
- 133
0
votes
1
answer
518
views
Script isn't served compressed from IIS
I have IIS 10 running on Windows 2016 (I see exactly same behavior on Windows 2019). I enabled static compression. I can see that the browser (Chrome) sends accept-encoding: gzip, deflate, br. I can ...
- 543
1
vote
1
answer
689
views
Apache mod_deflate MIME type settings
CPanel gives me options about what MIME content types to compress with Apache's mod_deflate, as shown below.
Are there disadvantages to having a long list of MIME types (eg. slower server response)? ...
- 293
0
votes
1
answer
2k
views
Can not run Brotli on my Nginx server
I installed Brotli on Ubuntu 18.04 with Nginx.
Here is the procedure I followed :
$ sudo apt-add-repository ppa:hda-me/nginx-stable
$ sudo apt update
$ sudo apt install brotli nginx nginx-module-...
- 57
4
votes
2
answers
10k
views
How to install Brotli on nginx and Ubuntu 18.04?
I installed Brotli on Ubuntu 18.04 with Nginx.
Here is the procedure I followed :
$ sudo apt-add-repository ppa:hda-me/nginx-stable
$ sudo apt update
$ sudo apt install brotli nginx nginx-module-...
- 57
5
votes
1
answer
2k
views
What kind of algorithm is used in LTO tape hardware compression?
LTO tape drives, from their very first generation, offer hardware compression that, theoretically, allows a maximum of 2 - 2.5x the rated data capacity of each cartridge to be stored, with only a ...
- 1,417
1
vote
1
answer
146
views
How to enable IIS to always precompress / pregzip a certain file?
I have a 4mb js file (gzipped version is 900kb). The file is like this: main.328479fdsdf.js (so it has a hash based on the content).
IIS seems to compress my big js file only if is accessed 2 times ...
- 111
0
votes
1
answer
493
views
Use of gzip with multiple, sequential nginx proxies
Consider the following server setup on a network:
In this example, both of the proxy servers are Nginx servers.
If the aim is to reduce load time for the client, should compression be used by the ...
- 39
4
votes
2
answers
479
views
Are LTO Hardware compression and encryption standardized?
I'm about to setup a backup using an LTO-4 Drive. I know that drives are meant to be compatible to any tape within the past 3 generations (so I should be able to read an LTO-4 tape using an LTO-4, 5 ...
- 4,050
1
vote
2
answers
3k
views
Modern transparent folder compression in Ubuntu 18.04
I'm looking for a way to transparently compress files in a directory, that will be shared over SMB.
I have a server, running Ubuntu 18.04.1 LTS, and I use it for file storage. Mainly what I want to ...
1
vote
0
answers
524
views
Why do i get gzip css and js with full size instead of compressed size
I am doing some tests with gzip, I have manually compressed the CSS and JS with gzip. The size of the JS has dropped from 33kb to 7kb.
The problem is when I look in the network tab in Chrome or ...
- 111
-2
votes
1
answer
418
views
openvpn optimizations for Low bandwidth (250mb a month)
I am using openwrt lede with a lte stick 250-300mb a month depending on the unit so preferably hitting 250 or lower would be better for the entire system these are the clients the server doesn't have ...
3
votes
1
answer
201
views
Does ZFS, using default settings, use more physical space than ext4 for the same amount of data?
I'm currently considering a switch from ext4 to ZFS and am having trouble finding information about whether or not it uses more or less space to store the same amount of data as ext4. My understanding ...
- 503