Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [filesystems]

A file system (or filesystem) is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device(s) which contain it.

185
votes
4answers
68k views

LVM dangers and caveats

I have recently started using LVM on some servers for hard drives larger than 1 TB. They're useful, expandable and quite easy to install. However, I could not find any data about the dangers and ...
128
votes
11answers
17k views

FreeBSD: Directory called ^C (really!) - how to remove?

I made a typo: $ history 169 9:34 la /usr/local/etc/ 170 9:35 sudo mkdir ^C 171 9:36 sudo mkdir /usr/local/etc/dnsmasq.d Now I have a file that is called ^C (ctrl+C)!! When I use ls I just ...
120
votes
11answers
216k views

df in linux not showing correct free space after file removal

I have file servers which are used to store files. Files might reside there for a week, or for a year. Unfortunately, when I remove files from the server, df command doesn't reflect the freed up space....
101
votes
4answers
179k views

SSL Certificate Location on UNIX/Linux

Is there any standard or convention for where SSL certificates and associated private keys should go on the UNIX/Linux filesystem? Thanks.
61
votes
4answers
30k views

Drawbacks of mounting a filesystem with noatime?

Having every file be updated just when accessing them sounds like a waste. What's the catch with mounting a file system with the noatime option. What kind of applications/servers relies on the access ...
59
votes
1answer
18k views

Why are my XFS filesystems suddenly consuming more space and full of sparse files?

I've run XFS filesystems as data/growth partitions for nearly 10 years across various Linux servers. I've noticed a strange phenomenon with recent CentOS/RHEL servers running version 6.2+. Stable ...
58
votes
3answers
8k views

What is the difference between a soft (symbolic) link and a hard link?

I hear that you can now create soft links in Vista too. So, what is the difference between a soft (symbolic) link and a hard link on UNIX/Linux/Vista? Are there advantages of using one over the other?...
57
votes
12answers
54k views

ZFS vs XFS

We're considering building a ~16TB storage server. At the moment, we're considering both ZFS and XFS as filesystem. What are the advantages, disadvantages? What do we have to look for? Is there a ...
48
votes
7answers
47k views

What is the best Linux filesystem for MySQL (InnoDB)?

I tried to look for benchmark on the performances of various filesystems with MySQL InnoDB but couldn't find any. My database workload is the typical web-based OLTP, about 90% read, 10% write. Random ...
45
votes
7answers
7k views

Removing an (apparently) infinitely recursive folder

Somehow, one of our old Server 2008 (not R2) boxes has developed a seemingly infinitely-recursing folder. This is playing havock with our backups, as the backup agent tries to recurse down into the ...
42
votes
4answers
87k views

Linux/Windows/Unix/… file names: Which characters are allowed? Which are unescaped?

Which characters are allowed and which of them must be escaped on the command line in different operating systems?
42
votes
7answers
26k views

filesystem for millions of small files

Which Linux filesystem would you choose for best speed in the following scenario: a hundred million files ~2k file size in average 95% read access pretty random access high concurrency (>100 ...
41
votes
7answers
43k views

Why can't you fsck a mounted partition?

It's well-known that you should never fsck a mounted partition. I can understand how this could easily lead to corruption if the filesystem is written to by fsck (e.g., the -a option is used), but why ...
40
votes
8answers
78k views

How do I determine the block size of an ext3 partition on Linux?

How do I determine the block size of an ext3 partition on Linux?
38
votes
10answers
54k views

Question marks showing in ls of directory. IO errors too

Has anyone seen this before? I've got a raid 5 mounted on my server and for whatever reason it started showing this: jason@box2:/mnt/raid1/cra$ ls -alh ls: cannot access ...
37
votes
7answers
162k views

How to extend an ext4 partition and filesystem?

I have a 400GB disk with a 320GB ext4 partition. I would like to grow the ext4 partition to use the left space (80GB of free space). +--------------------------------+--------+ | ext4 ...
37
votes
3answers
7k views

When is fsck dangerous?

Recently I've seen the root filesystem of a machine in a remote datacenter get remounted read-only, as a result of consistency issues. On reboot, this error was shown: UNEXPECTED INCONSISTENCY: RUN ...
36
votes
7answers
38k views

What happens if I delete lost+found

When you create some Linux filesystems like ext3 a 'lost+found' directory is created. According to this files will be placed there if files were damaged from some kind of system crash. What happens ...
35
votes
14answers
41k views

Linux - What directories should I exclude when backing up a server?

I'm backing up a Linux server and storing it on another server. I began with a simple rsync -aPh --del server.example.com:/ /mnt/backup Then someone pointed out that I shouldn't back up /proc, ...
35
votes
2answers
3k views

Unable to write to disk but disk isn't full

I am using Ubuntu 12.04 and can't write to any file, even as root, or do any other operation that requires writing. Neither can any process that needs to write, so they're all failing. df says I've ...
35
votes
2answers
57k views

.nfsXXXX files appearing, what are those?

I have an application running (on RHEL5) that streams data onto an NFS share. Recently, I saw a lot of .nfsXXXX... (xxx being a hexadecimal number) appearing in its working directory, where the ...
33
votes
7answers
37k views

du vs. df difference [duplicate]

I have a fileserver where df reports 94% of / full. But according to du, much less is used: # df -h / Filesystem Size Used Avail Use% Mounted on /dev/sda3 270G 240G 17G 94%...
33
votes
3answers
59k views

What is the maximum number of files a file system can contain?

Given the current structure of a directory entry on a ext4 file system on Ubuntu, what is the maximum number of files a file system can contain? What is the general method of calculating the maximum ...
31
votes
8answers
41k views

How to workaround the NTFS Move/Copy design flaw?

As anyone that has dealt with file server permissions is aware, NTFS has an interesting design feature/flaw known as the Move/Copy problem. As explained in this MS KB article, the permissions for a ...
30
votes
11answers
19k views

Distributed Storage Filesystem - Which one/Is there a ready to use product?

With Hadoop and CouchDB all over in Blogs and related news what's a distributed-fault-tolerant storage (engine) that actually works. CouchDB doesn't actually have any distribution features built-in, ...
30
votes
1answer
3k views

Why does deleting a big file take longer?

My understanding is that running rm on a file simply unlinks it, marking the space as free in the filesystem. It should then follow that deleting one file always takes roughly the same amount of time (...
29
votes
9answers
37k views

Best way to compare (diff) a full directory structure?

What's the best way to compare directory structures? I have a backup utility which uses rsync. I want to tell the exact differences (in terms of file sizes and last-changed dates) between the source ...
29
votes
6answers
4k views

What are .DS_Store files and why does OSX leave them on Windows shares?

Whenever I access windows shares from OSX 10.5 it leaves .DS_Store files on the remote filesystem. What are they used for, and are they necessary, and can they be prevented from being created?
28
votes
5answers
5k views

What are the performance implications for millions of files in a modern file system?

Let's say we're using ext4 (with dir_index enabled) to host around 3M files (with an average of 750KB size) and we need to decide what folder scheme we're going to use. In the first solution, we ...
27
votes
5answers
4k views

Turning off atime on a filesystem

I am setting up a mongoDB replica set and one of the first things I am suppose to do is turn off atime on the file system. After researching this a bit, I am not opposed to doing this, but I have to ...
27
votes
3answers
17k views

The XFS filesystem is broken in RHEL/CentOS 6.x - What can I do about it?

Recent versions of RHEL/CentOS (EL6) brought some interesting changes to the XFS filesystem I've depended on heavily for over a decade. I spent part of last summer chasing down an XFS sparse file ...
26
votes
2answers
6k views

ZFS Data Loss Scenarios

I'm looking toward building a largish ZFS Pool (150TB+), and I'd like to hear people experiences about data loss scenarios due to failed hardware, in particular, distinguishing between instances where ...
26
votes
2answers
43k views

Transparent compression filesystem in conjunction with ext4

I am trying to test a project that needs compressed storage with use of the ext4 file system since the application I use relies on ext4 features. Are there any production/stable solutions out there ...
25
votes
4answers
48k views

is there such a thing as a 'quick' format for ext4?

in Windows it's very fast to format with NTFS. I have a low powered Linux machine, with little RAM. formatting a 2TB volume to ext4 takes a long long time is there anything I can do to speed up ...
25
votes
12answers
7k views

Storing and backing up 10 million files on Linux

I run a website where about 10 million files (book covers) are stored in 3 levels of subdirectories, ranging [0-f]: 0/0/0/ 0/0/1/ ... f/f/f/ This leads to around 2400 files per directory, which is ...
25
votes
9answers
22k views

Maximum number of files in one ext3 directory while still getting acceptable performance?

I have an application writing to an ext3 directory which over time has grown to roughly three million files. Needless to say, reading the file listing of this directory is unbearably slow. I don't ...
24
votes
2answers
5k views

In Linux, why isn't superuser root's folder under /home?

You can create a user that has privileges like root, and it's home directory will fall under /home/username. Why does root get its own folder at the top level of the file system? Is this just ...
24
votes
7answers
4k views

Do I need to defrag unix filesystems?

I know Windows file systems are very sensitive to fragmentation and Unix such as UFS, ZFS, ext3, or ext4 are not as sensitive. But, is there any reason to defragment these filesystems at all? If so, ...
23
votes
8answers
30k views

Need to fix file permissions in a user's home directory

Does anyone have a tool or script that will recursively correct the file permissions on a directory? On an Ubuntu Linux machine, a bunch of files were copied to a USB disk with full 777 permissions (...
22
votes
2answers
15k views

See what files are being written to (like iotop but for files, not processes)

I have a Linux server, and it's doing a lot of disk io. There is one process that is doing most of the disk writes. I found this out with iotop, which shows the processes that are doing disk I/O. Is ...
22
votes
4answers
19k views

Mount Docker container contents in host file system

I want to be able to inspect the contents of a Docker container (read-only). An elegant way of doing this would be to mount the container's contents in a directory. I'm talking about mounting the ...
21
votes
7answers
6k views

Which filesystem for large LVM of disks (8 TB)?

I have a Linux server with many 2 TB disks, all currently in a LVM resulting in about 10 TB of space. I use all this space on an ext4 partition, and currently have about 8,8 TB of data. Problem is, I ...
21
votes
7answers
17k views

Copy permissions to identical tree on linux / unix

i have a tree of files with correct permission. then i have a (filewise) identical tree (with different file contents tough) with wrong permissions. how can i transfer the permissions layout from ...
21
votes
2answers
17k views

What is the sticky bit in UNIX file systems? When is it used?

What is the sticky bit in a UNIX file system? As an admin when and how would you use it?
21
votes
2answers
14k views

How to track all files a process opens in its lifetime

I am aware of using lsof for checking the files currently accessed by a process. Does there exist a way to see all files that an application opens in its lifetime?
21
votes
10answers
88k views

How to tell what computer has a file open on a network share?

Environment: Windows XP sp3, Windows 2003 server Problem: We have several dozen kiosk machines each with the same logon name who occasionally and briefly a file on a share. The rate is several ...
21
votes
3answers
7k views

Does btrfs have an efficient way to compare snapshots?

While diffing mounted snapshots would work, it sounds like it could be horribly slow in many cases. Is there btrfs specific functionality for diffing snapshots? (I was unable to find any in the docs)
20
votes
8answers
15k views

Doing an rm -rf on a massive directory tree takes hours

We're using rsnapshot for backups. It keeps lots of snapshots of the backuped up file, but it does delete old ones. This is good. However it's taking about 7 hours to do a rm -rf on a massive ...
20
votes
6answers
30k views

What's the maxium number of files a Unix folder can hold?

What's the maxium number of files a Unix folder can hold? I think it will be the same as the number of files.
20
votes
4answers
20k views

Performance difference between ramfs and tmpfs

I need to setup an in memory storage system for around 10 GB of data, consisting of many 100 kb single files(images). There will be lots of reads and fairly periodic writes(adding new files, deleting ...