A file system (or filesystem) is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device(s) which contain it.

learn more… | top users | synonyms (1)

116
votes
5answers
34k views

LVM dangers and caveats

I have recently started using LVM on some servers for hard drives larger than 1 TB. They're useful, expandable and quite easy to install. However, I could not find any data about the dangers and ...
44
votes
3answers
5k views

What is the difference between a soft (symbolic) link and a hard link?

I hear that you can now create soft links in Vista too. So, what is the difference between a soft (symbolic) link and a hard link on UNIX/Linux/Vista? Are there advantages of using one over the ...
39
votes
4answers
18k views

Drawbacks of mounting a filesystem with noatime?

Having every file be updated just when accessing them sounds like a waste. What's the catch with mounting a file system with the noatime option. What kind of applications/servers relies on the access ...
37
votes
7answers
30k views

What is the best Linux filesystem for MySQL (InnoDB)?

I tried to look for benchmark on the performances of various filesystems with MySQL InnoDB but couldn't find any. My database workload is the typical web-based OLTP, about 90% read, 10% write. Random ...
34
votes
12answers
24k views

ZFS vs XFS

We're considering building a ~16TB storage server. At the moment, we're considering both ZFS and XFS as filesystem. What are the advantages, disadvantages? What do we have to look for? Is there a ...
29
votes
6answers
3k views

What are .DS_Store files and why does OSX leave them on Windows shares?

Whenever I access windows shares from OSX 10.5 it leaves .DS_Store files on the remote filesystem. What are they used for, and are they necessary, and can they be prevented from being created?
29
votes
1answer
6k views

Why are my XFS filesystems suddenly consuming more space and full of sparse files?

I've run XFS filesystems as data/growth partitions for nearly 10 years across various Linux servers. I've noticed a strange phenomenon with recent CentOS/RHEL servers running version 6.2+. Stable ...
28
votes
6answers
14k views

filesystem for millions of small files

Which Linux filesystem would you choose for best speed in the following scenario: a hundred million files ~2k file size in average >95% read access pretty random access high concurrency (>100 ...
27
votes
7answers
47k views

df in linux not showing correct free space after file removal

I have file servers which are used to store files. Files might reside there for a week, or for a year. Unfortunately, when I remove files from the server, df command doesnt reflect the freed up space, ...
26
votes
1answer
1k views

Why does deleting a big file take longer?

My understanding is that running rm on a file simply unlinks it, marking the space as free in the filesystem. It should then follow that deleting one file always takes roughly the same amount of time ...
25
votes
6answers
22k views

What happens if I delete lost+found

When you create some Linux filesystems like ext3 a 'lost+found' directory is created. According to this files will be placed there if files were damaged from some kind of system crash. What happens ...
25
votes
3answers
9k views

The XFS filesystem is broken in RHEL/CentOS 6.x - What can I do about it?

Recent versions of RHEL/CentOS (EL6) brought some interesting changes to the XFS filesystem I've depended on heavily for over a decade. I spent part of last summer chasing down an XFS sparse file ...
24
votes
12answers
4k views

Storing and backing up 10 million files on Linux

I run a website where about 10 million files (book covers) are stored in 3 levels of subdirectories, ranging [0-f]: 0/0/0/ 0/0/1/ ... f/f/f/ This leads to around 2400 files per directory, which is ...
23
votes
7answers
3k views

Do I need to defrag unix filesystems?

I know Windows file systems are very sensitive to fragmentation and Unix such as UFS, ZFS, ext3, or ext4 are not as sensitive. But, is there any reason to defragment these filesystems at all? If so, ...
20
votes
7answers
20k views

Why can't you fsck a mounted partition?

It's well-known that you should never fsck a mounted partition. I can understand how this could easily lead to corruption if the filesystem is written to by fsck (e.g., the -a option is used), but why ...
20
votes
5answers
2k views

Turning off atime on a filesystem

I am setting up a mongoDB replica set and one of the first things I am suppose to do is turn off atime on the file system. After researching this a bit, I am not opposed to doing this, but I have to ...
20
votes
9answers
16k views

Maximum number of files in one ext3 directory while still getting acceptable performance?

I have an application writing to an ext3 directory which over time has grown to roughly three million files. Needless to say, reading the file listing of this directory is unbearably slow. I don't ...
20
votes
7answers
18k views

How to workaround the NTFS Move/Copy design flaw?

As anyone that has dealt with file server permissions is aware, NTFS has an interesting design feature/flaw known as the Move/Copy problem. As explained in this MS KB article, the permissions for a ...
20
votes
4answers
28k views

Linux/Windows/Unix/… file names: Which characters are allowed? Which are unescaped?

Which characters are allowed and which of them must be escaped on the command line in different operating systems?
18
votes
10answers
14k views

Linux - What directories should I exclude when backing up a server?

I'm backing up a linux server and storing it on some other server. I started off with a simple rsync -aPh --del server.example.com:/ /mnt/backup Then someone pointed out that I shouldn't back up ...
18
votes
9answers
19k views

Best way to compare (diff) a full directory structure?

What's the best way to compare directory structures? I have a backup utility which uses rsync. I want to tell the exact differences (in terms of file sizes and last-changed dates) between the source ...
18
votes
5answers
38k views

How do I determine the block size of an ext3 partition on Linux?

How do I determine the block size of an ext3 partition on Linux?
18
votes
3answers
22k views

What is the maximum number of files a file system can contain?

Given the current structure of a directory entry on a ext4 file system on Ubuntu, what is the maximum number of files a file system can contain? What is the general method of calculating the maximum ...
18
votes
10answers
56k views

How to tell what computer has a file open on a network share?

Environment: Windows XP sp3, Windows 2003 server Problem: We have several dozen kiosk machines each with the same logon name who occasionally and briefly a file on a share. The rate is several ...
17
votes
6answers
3k views

Which filesystem for large LVM of disks (8 TB)?

I have a Linux server with many 2 TB disks, all currently in a LVM resulting in about 10 TB of space. I use all this space on an ext4 partition, and currently have about 8,8 TB of data. Problem is, I ...
17
votes
7answers
10k views

du vs. df difference [duplicate]

I have a fileserver where df reports 94% of / full. But according to du, much less is used: # df -h / Filesystem Size Used Avail Use% Mounted on /dev/sda3 270G 240G 17G ...
17
votes
1answer
1k views

In Linux, why isn't superuser root's folder under /home?

You can create a user that has privileges like root, and it's home directory will fall under /home/username. Why does root get its own folder at the top level of the file system? Is this just ...
17
votes
2answers
13k views

What is the sticky bit in UNIX file systems? When is it used?

What is the sticky bit in a UNIX file system? As an admin when and how would you use it?
17
votes
13answers
2k views

How can I visualize hard disk space with millions of files?

We have a hard disk that is 600 Gigs and nearly full. It's been filled up with 18,501,765 files (mostly small 19k images) and 7,142,132 folders. It's very difficult to find out where exactly all the ...
16
votes
10answers
9k views

Distributed Storage Filesystem - Which one/Is there a ready to use product?

With Hadoop and CouchDB all over in Blogs and related news what's a distributed-fault-tolerant storage (engine) that actually works. CouchDB doesn't actually have any distribution features built-in, ...
16
votes
2answers
7k views

Production-ready, highly reliable filesystems on Linux: ext4 ext3 XFS or JFS (or ZFS)?

The last real question I've seen on this topic is from about two years ago (is ext4 ready for production usage). In the intervening time, how has ext4 improved? XFS, JFS, and ext3 are the standby ...
15
votes
3answers
5k views

To fsck or not fsck after 180 days

By default after 180 days or some number of mounts, most Linux filesystems force a file system check (fsck). Of course this can be turned off using, for example, tune2fs -c 0 -i 0 on ext2 or ext3. On ...
14
votes
8answers
6k views

Doing an rm -rf on a massive directory tree takes hours

We're using rsnapshot for backups. It keeps lots of snapshots of the backuped up file, but it does delete old ones. This is good. However it's taking about 7 hours to do a rm -rf on a massive ...
14
votes
4answers
920 views

How to store data on a machine whose power gets cut at random

I have a virtual machine (Debian) running on a physical machine host. The virtual machine acts as a buffer for data that it frequently receives over the local network (the period for this data is ...
14
votes
7answers
8k views

Copy permissions to identical tree on linux / unix

i have a tree of files with correct permission. then i have a (filewise) identical tree (with different file contents tough) with wrong permissions. how can i transfer the permissions layout from ...
14
votes
7answers
12k views

What is the best filesystem for insert performance on PostgreSQL?

I'm curious if anyone out there has done any experimentation or comparisons between file systems and database performance. On Linux, I'm wondering what is the optimal file system for a postgres ...
14
votes
13answers
226k views

Error 0x80070570: How to delete corrupted and unreadable file?

On my system partition I have a folder which cannot be deleted. This folder has been created by restoring files from an Acronis TrueImage backup. The error message says: Error 0x80070570: The ...
14
votes
6answers
2k views

Is the F.U.D. about ext4 justified? Or would it be safe to use in some production systems?

I am wondering if ext4 is safe to use on my servers. But I've heard so much FUD about it that I am concerned. Our system could lose some data, and it would not be too big a deal. Even a full days ...
14
votes
3answers
620 views

Ensuring a repeatable directory ordering in linux

I run a hosted continuous integration company, and we run our customers' code on Linux. Each time we run the code, we run it in a separate virtual machine. A frequent problem that arises is that a ...
14
votes
3answers
388 views

In Linux, is there a way of seeing which user updated a file?

Does Linux record who last changed a file (rather than created it)? If so, how do I find this out? If not, is there some way of monitoring file(s)?
14
votes
3answers
12k views

GlusterFS vs Ceph, which is better for production use for the moment? [closed]

I am evaluating GlusterFS and Ceph, seems Gluster is FUSE based which means it may be not as fast as Ceph. But looks like Gluster got a very friendly control panel and is ease to use. Ceph was ...
13
votes
7answers
3k views

Is btrfs production ready?

I've seen a lot of information on btrfs lately. I have been considering ext4 for my next filesystem, but am tempted by btrfs instead. How widely used is btrfs? What are the pro's and con's?
13
votes
2answers
1k views

Does btrfs have an efficient way to compare snapshots?

While diffing mounted snapshots would work, it sounds like it could be horribly slow in many cases. Is there btrfs specific functionality for diffing snapshots? (I was unable to find any in the docs) ...
12
votes
6answers
13k views

What's the maxium number of files a Unix folder can hold?

What's the maxium number of files a Unix folder can hold? I think it will be the same as the number of files.
12
votes
7answers
6k views

ZFS alternative for Linux?

I'm running OpenSolaris with ZFS for my main fileserver. I originally went with ZFS because I heard so many awesome things about it: Automatic disk spanning (zpools) Software RAID (RAID-Z) Automatic ...
12
votes
6answers
9k views

How to compare two directories (including sub directories) for differences?

How can I compare two directories with sub dirs to see where is the difference?
12
votes
1answer
451 views

ext4 listing of files very slow in one specific directory that contained lots of files before

Background I had a small logrotate misshap... Logrotate would rotate the archived logs by misstake causing a quadratic growth of files in my /var/log/. And by the time I caught wind that something ...
12
votes
2answers
416 views

Performance impact of running different filesystems on a single Linux server

The book "HBase: The definitive guide" states that Installing different filesystems on a single server is not recommended. This can have adverse effects on performance as the kernel may have to ...
12
votes
2answers
962 views

Safety of write cache on SATA drives with barriers

I've been reading lately about write caching, NCQ, firmware bugs, barriers, etc regarding SATA drives, and I'm not sure what's the best setting that would make my data safe in case of a power failure. ...
11
votes
4answers
825 views

last time a filesystem was checked?

What command can tell me the last time that a filesystem was fsck'd? A date would be nice, but I'd settle for the mount count since the last fsck. I've looked all around for this in fsck*, lsattr, ...