Questions tagged [disk-cache]
Disk-cache: reserved area in RAM to speed-up disk access
56
questions
2
votes
0
answers
161
views
Cache writes with cachefilesd (non-shared mode) on Linux
I'm using cachefilesd as a read-cache for an NFS share. This is simple to set up and does what is says.
However, I would like to also cache written files. I.e., if I write some.file, and then I read ...
- 41
5
votes
2
answers
961
views
Data corruption when disk write caching is enabled
I've recently learned that enabling disk write caching can significantly improve system performance. However, I'm concerned about the potential risks of data corruption or loss in case of a sudden ...
- 50.6k
26
votes
3
answers
41k
views
Linux not freeing large disk cache when memory demand goes up
Running Ubuntu on a 2.6.31-302 x86-64 kernel. The overall problem is that I have memory in the 'cached' category that keeps on going up and will not be freed or used even when our application needs it....
- 121
1
vote
0
answers
320
views
How to make two squid servers share the same disk cache?
I'm trying to setup squid proxy in a kubernetes environment as a caching server with 3 replicas. I wanted to know whether all three replicas can share the same disk storage, or I'll have to assign ...
- 21
3
votes
1
answer
2k
views
On a system with 64GB mem the Linux Buffer run full while copying with dd to dev null and io stops till manual drop_caches
I am running a server with linux software raid 10. It is a dual CPU system with 64GB Ram. 2x16GB dimms related to each of the CPUs. I want to use dd to backup kvm virtual machines and run into a ...
CommunityBot
- 1
2
votes
2
answers
2k
views
Why would diskspd perform better without cache?
We are currently investigating high disk latency on a Windows server 2012 r2 that run as an SQL server. It is a virtual machine under VMware and the datastore of the faulty disk is linked with a very ...
CommunityBot
- 1
2
votes
2
answers
2k
views
How to increase the disk write buffer for rsync on Mac OS X?
I have to copy some TB of data from one external drive to another (mostly ~400 MB files).
The source is NTFS-formatted USB2 and the best read speed I can get from it is 7MB/s.
The target is HFS+ USB3 ...
CommunityBot
- 1
4
votes
1
answer
2k
views
Linux - Disk / Filesystem as a LRU Cache (with transparent expiry)
I have a case where I'd want to use local disk as a LRU cache for (hot) files from a separate web service (something like S3). If the file doesn't exist on disk, the file is read over the internet, ...
CommunityBot
- 1
0
votes
1
answer
3k
views
Google Cloud Function warning "OpenBLAS WARNING - could not determine the L2 cache size on this system"
I get the warning
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
in the logs of the "LOGS" tab of the Cloud Function.
There is already a Q/A on this ...
google-cloud-platform
8
votes
3
answers
25k
views
Use Varnish Cache only on-disk
The situation is:
I' m building a PHP application and need http caching.
Varnish is great and lots of people tell me that Varnish stores the cached data in RAM. I want it to cache on hard disk.
Is ...
- 4,514
5
votes
4
answers
18k
views
Should I disable write caching on my Windows 2008 VM?
I have a Windows Server 2008 x64 Standard virtual machine that runs on a machine with a hardware RAID controller, a Perc 6/i, which has a battery on-board.
Doing everything I can for additional ...
- 141
7
votes
3
answers
10k
views
Why does mdadm write unusably slow when mounted synchronously?
I have a 6 disk raid6 mdadm array I'd like to benchmark writes to:
root@ubuntu:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid6 sda[0] sdf[5] sde[4] sdd[3] sdc[2] sdb[...
- 2,816
0
votes
1
answer
136
views
Run similar disk intensive tasks parallel or serial on regular Linux?
Scenario
I'm on a regular Ubuntu 18.04 LTS with ext4 Filesystems.
I'm using restic to backup my machines. Restic is a backup tool that support different backends (local, sftp, aws, gcs, ..) but only ...
- 285
1
vote
1
answer
1k
views
Linux /proc/sys/vm/drop_caches in guest VMs
Question:
Would it ever be a good idea to disable page cache inside guest VM's and instead rely on the ZFS ARC (and SSD based L2ARC) of the host?
Context:
I'm asking since I'm running a Proxmox ...
- 50.6k
14
votes
2
answers
3k
views
How to setup an NFS server that caches a network share?
User data is stored on two fairly large (>1 PB) OpenStack Swift storage clusters. Let them be Cluster A and Cluster B.
In addition, there are several PoPs that need to interact with that data. ...
CommunityBot
- 1
0
votes
0
answers
268
views
How can I disable caching for the volume in Windows 2012?
When I try to disable disk cache on Windows Server 2012, I get "windows could not change the write-caching setting for the device" error.
How do I disable the disk cache?
- 112
3
votes
1
answer
4k
views
How to ensure HDD Write Caching is disabled on MS Storage Spaces
I have a Storage Spaces pool on a server that is not currently power protected. It consists only of mechanical hard drives.
I want to compare performance of write caching enabled vs disabled so I can ...
CommunityBot
- 1
10
votes
2
answers
4k
views
Are bcache and/or dm-cache considered stable for production in 2016?
I would like to use linux SSD caching (dm-cache or bcache) with Debian Jessie production servers. (kernel 3.16)
My question: Are dm-cache and bcache modules reliables in linux 3.16 ? Do I need to ...
- 103
1
vote
0
answers
159
views
Debian max_ratio setting for device not working
My debian 8 machine has 2 drives. Sometimes I move/copy files from drive A to drive B. Drive B is slow, so the file system cache is full causing the system to be slow/unresponsive when copying large ...
- 83
0
votes
2
answers
709
views
MySQL slow, but speed up after drop disk caches
I have very busy Linux MySQL server with huge database.
Server usually work on load average 2 or less.
The server have software raid (mdadm) that is currently rebuilding.
Several times today, MySQL ...
- 545
0
votes
1
answer
284
views
Questions about Varnish file storage engine
I have Varnish cache in front of some websites with the following storage engine configuration:
Storage: file
Size: 130GB
The server where it is installed has the following configuration:
RAM: 7.5 ...
- 171
1
vote
3
answers
359
views
Confused by write cache flags
I'm looking to maximise performance of my hard drive by enabling write caching. I understand the risks in doing this, but believe them to be insignificant for how I will use it.
On the HD Properties ...
- 18.3k
7
votes
2
answers
6k
views
Windows Server virtual disk cache settings
In Windows under Azure or Hyper-V, the Disk Policy write cache settings for a virtual disk will always appear to be enabled from the server configuration, regardless of the actual status of write ...
- 9,126
2
votes
2
answers
662
views
Active process swapping despite free memory
For some reason our system starts swapping (actively used memory) at around 60GB used memory. (see edit below, it seems I/O and disk cache usage, even of previously run processes has an effect)
...
- 96
1
vote
1
answer
2k
views
Windows Server 2012 write caching
From MSDN:
By default, Windows caches file data that is read from disks and written to disks. [..] Caching is managed per file object. The policy of delaying the writing of the data to the file and ...
- 54k
1
vote
0
answers
932
views
Disable nginx fastcgi_cache when disk is full
When there is no disk space, nginx is still able to work correctly.
It can serve static files, and it can pass the request to a fastcgi server (such as PHP), and it can serve the dynamic response to ...
- 299
1
vote
1
answer
2k
views
What is the "right way" to combine sync and drop_caches?
When I need to do echo 3 > /proc/sys/vm/drop_caches with a sync beforehand, is it better to use
sync && echo 3 > /proc/sys/vm/drop_caches or
sync ; echo 3 > /proc/sys/vm/drop_caches....
- 38
2
votes
1
answer
3k
views
Apache when to use mod_cache? disk_cache vs memory_cache?
I'm making use of Apache 2.4.10 on Ubuntu 15.04 and my entire server runs on a SSD and is virtualised in Hyper-V.
When should I enable mod cache in Apache 2.4?
Currently I have pretty good TTFB for ...
4
votes
1
answer
5k
views
Disable file cache for mount point
I ran into a bug similar to this one:
RHEL NFS client returns NULL bytes when reading a growing file
So my solution was to check for \0 Bytes and reread the file. Problem here is, that the wrong ...
CommunityBot
- 1
2
votes
3
answers
5k
views
High disk I/O activity in CentOS server
I have about 16 websites in a CentOS dedicated, and I am having some problems on high traffic hours, it seems to be a high disk I/O activity causing a general slowdown.
I've installed atop and this ...
CommunityBot
- 1
0
votes
2
answers
50
views
8x slower cat for the same files on the same disk but different directory
I'm having an issue where I have 8x slower access to a set of files as compared to the same files in another directory on a linux machine.
The filesystem is a 36TB RAID-5 filesystem exported from a ...
- 50.6k
2
votes
3
answers
1k
views
Would the data on the RAID Card Cache and Drive Cache be lost when a kernel panic happens?
Is a BBU necessary when you have A+B power?
Would the data on the RAID Card Cache and Drive Cache be lost when a kernel panic happens and you are forced to do a cold reset?
CommunityBot
- 1
2
votes
1
answer
400
views
Linux filesystem page cache - hot or not
How can you tell how much of the filesystem cache is hot (frequently accessed) or not?
The use case would be determining when you are approaching the limits of your memory but before you actually hit ...
- 1,065
0
votes
1
answer
958
views
Can high disk cache usage lead to external fragmentation and ultimately to page allocation failures?
We're getting page allocation failures on a Linux server (3.2 kernel). We've been told the problem is directly related to the high use of the memory for cache; this would lead to external ...
CommunityBot
- 1
5
votes
1
answer
3k
views
Linux read disk cache and NFS
I am interested in the impact of the read disk cache on accessing a file through NFS. Lets say I have a file on NFS. I access it (do "cat myfile") from a linux host "A". If there's enough RAM a file ...
- 86
2
votes
1
answer
5k
views
Why is mongod not using all available RAM?
We have an mongod instance runnning on a VM, and it doesn't seem to be using all available memory. It's page-faulting significantly more than usual, and the system's performance has been significantly ...
CommunityBot
- 1
3
votes
4
answers
5k
views
Would a PHP application benefit from being served from a RAM drive?
I am in charge of hosting a PHP application that is large and slow, but easy to scale. The application is entirely static, with writable disk storage needed. We've profiled the application, and the ...
- 156
0
votes
1
answer
406
views
Linux not freeing the file buffer when we execute rm or > (redirection) operator
I have one router which has embedded-linux with kernel version 2.4.22 running on it. I know its very old version but i have to deal with it. Please take a look at the free command output when i run it ...
- 246
0
votes
1
answer
949
views
nginx proxy_cache or Memcached
I currently use nginx's proxy_cache for file-based caching in front of three Django applications (Apache/mod_wsgi). For some locations/URLs, I disabled proxy_cache and used Memcached (per view) within ...
CommunityBot
- 1
2
votes
1
answer
947
views
ram disk mirroring part of physical disk
What I would like to do/have is:
Some kind of application/driver that creates a RAM drive/partition that is actually assigned a drive letter (eg. D:). This drive letter should be based on/mirroring a ...
- 8,196
4
votes
2
answers
4k
views
Write caching in linux
I have read a lot on serverfault and searched on google about write caching, but still cant find the answer.
I have an HP ProLiant DL380 G5 with HW raid with 512 MB Battery backed cache. I use debian ...
CommunityBot
- 1
1
vote
1
answer
621
views
Apache mod cache mem/disk saves the cache for 10 seconds and than re-cache
I have this set up in my 000-default:
<VirtualHost 192.168.1.10:80>
ServerName live.website.com
DocumentRoot /var/www/live.website.com
SetOutputFilter INFLATE;proxy-html;DEFLATE
...
- 512
14
votes
2
answers
3k
views
Safety of write cache on SATA drives with barriers
I've been reading lately about write caching, NCQ, firmware bugs, barriers, etc regarding SATA drives, and I'm not sure what's the best setting that would make my data safe in case of a power failure.
...
- 3,387
3
votes
1
answer
96
views
varnish delete least requested files
I see that varnish can be configured to set -smalloc or -sfile with a certain size.
I want to set a file cache of 1G such that least requested files are deleted first when cache is full. Is this ...
- 1,796
1
vote
0
answers
288
views
Disk cache (write back) disables automatically
some days ago I discovered (using Munin) that my server had high iowait times.
The reason was that write cache was disabled. I enabled it using hdparm -W1 /dev/sda but after some time (ramdonly) it ...
- 11
5
votes
1
answer
1k
views
Solaris 10: Drop disk cache? (to benchmark app)
Is there a command to drop the ZFS and UFS disk caches? (Preferably just one folder hierarchy.)
I'm trying to tuning the performance settings of an application that read/writes data from a large ...
- 8,911
2
votes
1
answer
125
views
CentOS VM only using a third of available memory
I have a VM slice running CentOS, on Parallels.
Here is the output of free -m
[root@server ~]# free -m
total used free shared buffers cached
Mem: 960 ...
CommunityBot
- 1
7
votes
4
answers
12k
views
Disable all disk caching for apache2 on linux
For benchmarking purposes I want to force Apache 2 to load each requested file from disk instead of loading it from a cache in memory. From what I have read doing a sync followed by
echo 3 > /proc/...
CommunityBot
- 1
2
votes
1
answer
1k
views
How Does The Linux Disk Cache Work With iSCSI and GFS2?
I'm planning to share a GFS2 filesystem via iSCSI to my web servers.
Are the files read by my web servers cached locally using the linux disk cache stored in memory or are they only cached on the ...
CommunityBot
- 1
1
vote
1
answer
16k
views
What is read cache in "[sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA"?
I'm trying to figure out what the read cache part of the following message refers to:
[sda] Write cache: enabled, read cache: enabled, doesn't support DPO
or FUA
Is it "IDE drive's read-...
- 69.1k