0
votes
0answers
20 views

AIX JFS2 I/O performance problems

I have a fresh installation of AIX 7.1. I noticed that JFS2 file system with logging enabled (log=/dev/hd8) is dramatically slower than one with log=NULL. Simple zip archive extraction takes 5 minutes ...
0
votes
2answers
101 views

FS for many small files, mostly reads?

I need a direction since all the blogs/benchs seems to contradict. Goal: Having a lot of small files on FS in a directory tree fashion similar to this AB/CD/EF... Two size groups : G1 : 10-20 kB G2 ...
11
votes
1answer
275 views

ext4 listing of files very slow in one specific directory that contained lots of files before

Background I had a small logrotate misshap... Logrotate would rotate the archived logs by misstake causing a quadratic growth of files in my /var/log/. And by the time I caught wind that something ...
1
vote
4answers
375 views

Slower file create on faster server. Why?

I have thwo servers. The old one is 24GB RAM, i7-950 @ 3,0Ghz, 2x 1,5TB SATA HDD (ST31500341AS). The new is 32GB RAM, i7-3770 @ 3,4GHz, 2x 3,0TB SATA HDD (ST3000DM001-9YN166). Now I noiticed, ...
1
vote
1answer
72 views

Is writing/reading speed affected by the name of the files?

How can I find out, or does anybody know, if is faster storing/reading a file with Spanish/Greek/Cyrilic/etc... characters like mi-foto-españa-oíóáaç.jpg than mi-foto-espana-oioaac.jpg ?
2
votes
1answer
72 views

Why would disk reads be heavier than writes with a simple java app that creates many small files?

I've got a very simple java 7 app that reads from a proprietary file format (Hadoop sequencefile) sitting on disk xvdb and creates millions of 2-20kb files on disk xvdf. ByteBuffer byteBuffer = ...
-1
votes
1answer
97 views

Recommended resources for server tuning and security checklist for production Linux server deployment [closed]

There are many questions on server tuning and security checklist are being closed. So, I am not asking for the exact detail. Please point me to the resources you find it is helpful (no matter web ...
12
votes
2answers
398 views

Performance impact of running different filesystems on a single Linux server

The book "HBase: The definitive guide" states that Installing different filesystems on a single server is not recommended. This can have adverse effects on performance as the kernel may have to ...
-5
votes
1answer
195 views

CFQ vs Deadline - performance benefits [closed]

Running CPanel and wondering what would be the best for web hosting? deadline is more balanced over CFQ right so would allow for more I/O activities right
-1
votes
1answer
131 views

Linux user-space vs kernel -space file system performance

Is there a performance impact from using a user space file system vs a kernel space filesystem? I ask because I have a friend that refuses to use user space file systems because of a performance ...
0
votes
2answers
3k views

How much resources should backup tools be using? ( issues with clBackup )

My hosting provider makes backups of our servers using a tool called clBackup which pushes files over the network. Below is one ps output we took and reported to our provider: top - 10:06:24 up 25 ...
10
votes
10answers
694 views

How does the number of subdirectories impact drive read / write performance on Linux?

I've got an EXT3 formatted drive on a Linux CentOS server. This is a web app data drive and contains a directory for every user account ( there are 25,000 users ). Each folder contains files that ...
2
votes
5answers
3k views

Performance gains of putting Windows pagefile on separate partition (same disk)?

I am making a drive image that will be deployed to several computers, and I'm tweaking it for performance. It is a Windows XP image. Considering there is only one hard drive, will the machine gain ...
5
votes
5answers
623 views

Linux: how many disk I/O does it take to read a file? How to minimize it?

According to this paper on Facebook's Haystack: "Because of how the NAS appliances manage directory metadata, placing thousands of files in a directory was extremely inefficient as the ...
1
vote
2answers
188 views

Benchmarking disk performance on large-memory systems

I need to produce some comparative filesystem performance metrics for our virtual hosting environment. Specifically, I need to look at the difference between NFS and iSCSI and at the performance ...
2
votes
4answers
312 views

50,000 file in one directory, what is my best option?

I'm forced to have this directory structure /var/www/$WEBSITE/$DIR1/$DIR2/$FILES for each of those $FILES, there is approx 50,000 XHTML pages. I'm running Cherokee, which has new front-end caching ...
1
vote
3answers
163 views

ext2 dump/restore problem

I am running a mailserver with maildir storage. This means that quite a lot of files are created and I have just run out of inodes. AFAIK there is no magic command to increase number of inodes on ext# ...
10
votes
2answers
1k views

How does Linux handle concurrent disk IO?

When a Linux server is serving many concurrent requests to read many different files, does it: Seek to File_1, read the entire file, then seek to File_2, read the entire file, then seek to File_3, ...
4
votes
3answers
2k views

How can I find out where a file is physically located on the disk (block numbers)?

This is an obscure question, I know. I'm trying to do some performance testing of some disks on a Linux box. I'm getting some inconsistent results, running the same test on the same disk. I know ...
0
votes
1answer
260 views

Appropriate Linux (ubuntu 10.04) FS type for rapid FS management many small files and directories [duplicate]

Possible Duplicate: Linux's best filesystem to work with 10000's of files without overloading the system I/O I have an 240gb image store of approximately 1.5 million entries. About ...
0
votes
1answer
61 views

Store data on File Server or SQL database

We have a solution which includes storage of PDF documents. Currently we have 50,000 documents that are 0.6 MB each (all documents same size) We currently store these documents in varbinary(max) ...
3
votes
2answers
167 views

Measuring disk seeks under windows

I would like to measure the number of disk seeks that are made by a process under Windows. If I open Task Manager, I can see a column named "I/O - reads". Is this the equivalent of disk seeks? Can I ...
1
vote
1answer
90 views

Configure Windows filesystem for developer builds

I'm putting together a Windows box that will act as a continuous integration build server; we'll mostly build C# stuff using MSBuild, but I think this question applies to any compiled language on ...
1
vote
1answer
160 views

Erratic write patterns with linux md RAID1

I'm experiencing strange write performance patterns on a RAID1 array of SATA disks. My kernel is vanilla 2.6.37 and my server has 32G RAM. I'm copying a 32G file from an XFS filesystem on a SAS drive ...
2
votes
2answers
462 views

10tb file storage design question on media website

I have a fairly busy media website where MP3 audio files are uploaded by members and streamed / downloaded from 2 Windows servers which are load balanced at the moment...both servers simply mirror ...
3
votes
3answers
1k views

Linux's best filesystem to work with 10000's of files without overloading the system I/O

It is known that certain AMD64 Linuxes are subject of being unresponsive under heavy disk I/O (see Gentoo forums: AMD64 system slow/unresponsive during disk access (Part 2)), unfortunately have such ...
7
votes
6answers
2k views

What's the fastest filesystem for developer builds?

I'm putting together a Linux box that will act as a continuous integration build server; we'll mostly build Java stuff, but I think this question applies to any compiled language. What filesystem and ...
8
votes
1answer
4k views

See what files are being written to (like iotop but for files, not processes)

I have a Linux server, and it's doing a lot of disk io. There is one process that is doing most of the disk writes. I found this out with iotop, which shows the processes that are doing disk I/O. Is ...
4
votes
4answers
451 views

Suggestions for recommended file system for Linux server

There are many file systems for a Linux server administrator to choose from. Also, it is not always an easy task to figure out the appropriate layout and optimal file system for your needs. Can you ...
2
votes
1answer
544 views

Recommended technology for layered disk cache in linux

I originally asked this on superuser but it occurs to me it's more more of a server topic. I've just bought a 6-core Phenom with 16G of RAM. I use it primarily for compiling and video encoding (and ...
0
votes
3answers
254 views

Linux kernel multiple file system drivers and performance

I am not sure how Linux kernel handles multiple file system drivers and if an inode lookup for each given mounting point in file system interferes with other mounting points, or is there something ...
1
vote
2answers
3k views

How much performance is gained from setting NtfsDisable8dot3NameCreation = 1?

You can tune NTFS with different parameters in the registry and an TechNet articles states that you increase file performance by setting NtfsDisable8dot3NameCreation to 1 in the registry. In real ...
4
votes
1answer
2k views

Does increasing the journal size improve performance for ext4 filesystems?

There is an article on linux-mag which says that increasing the size of the journal on ext4 filesystems actually improves filesystem performance for very large partitions. I'm wondering if anyone ...
1
vote
2answers
1k views

JFS: long fsck time on large filesystem?

There was a power failure recently that took down one of my servers. On reboot, the main storage filesystem - JFS on a 7TB (9x1TB RAID6) filesystem - needed an fsck before mounting read-write. After I ...
5
votes
2answers
422 views

Disk fragmentation when dealing with many small files

On a daily basis we generate about 3.4 Million small jpeg files. We also delete about 3.4 Million 90 day old images. To date, we've dealt with this content by storing the images in a hierarchical ...
14
votes
3answers
12k views

GlusterFS vs Ceph, which is better for production use for the moment? [closed]

I am evaluating GlusterFS and Ceph, seems Gluster is FUSE based which means it may be not as fast as Ceph. But looks like Gluster got a very friendly control panel and is ease to use. Ceph was ...
4
votes
5answers
4k views

Tracing slow Java I/O in Solaris 10

We have a Java application which is significantly slower in Solaris 10 server than in Windows PC. We have profiled (-Xprof) the application and observed that UnixFileSystem.getBooleanAttributes0 ...
7
votes
1answer
2k views

Best practices for choosing fs block size on ext3/4?

I never really pay block size much attention but obviously there can be benefits to choosing something other than the default. I am looking for a good "best practices" paper on choosing block size. ...
2
votes
6answers
1k views

Does a file system lose performance as it fills up?

The context of the question is a Windows computer (so the filesystem in question is NTFS) that is filling up with data that can probably be removed. But I don't know whether it's worth the time to ...
7
votes
4answers
3k views

Using SSD as disk cache

Is there software for Linux to use an SSD as disk cache? I believe that Sun does something like this with ZFS, though not sure. A quick search provides nothing suitable. The goal would be to put ...
1
vote
2answers
1k views

NTFS journal on different (SSD) disk

Some advanced file systems, like ZFS have the option of using SSD disks as a second level cache for the file system or for the journal. Is this possible with NTFS?
2
votes
1answer
1k views

Optimal Directory Depth vs Number of Files in a Directory for ext3?

To access files on ext3, and (if dir_index is not being used) what is the optimal directory depth vs the number of files per directory? Does file size effect this? The total number of files might be ...
2
votes
7answers
729 views

how to track down permanent disk accesses on linux

I have a machine that runs few services, but I don't believe they are responsible for sometimes heavy disk access on this server (massive head moving which I can hear, after long periods of almost no ...
13
votes
8answers
5k views

Doing an rm -rf on a massive directory tree takes hours

We're using rsnapshot for backups. It keeps lots of snapshots of the backuped up file, but it does delete old ones. This is good. However it's taking about 7 hours to do a rm -rf on a massive ...
3
votes
4answers
7k views

Windows file server performance tuning

I seek your opinion and tips/suggestions in optimizing the performance of a new Windows file server I'm building. I've inherited a Dell NF500 storage server (basically Dell 2950 with Windows 2k3 ...
0
votes
4answers
187 views

Moving files within an ext4 filesystem?

I'd like to decrease the access-time for some files by moving them to the beginning of the fs. Task 1: Clear a certain block range at the beginning of the fs (moving existing files to free space ...
3
votes
2answers
2k views

Advices for filesystem choice and tuning on SSD?

What is your favourite filesystem on SSD disks, and why? I'm asking because on my Acer Aspire One with 8 GB SSD with ReiserFS it takes ages to boot after an unclean shutdown, so I'm probably missing ...
2
votes
4answers
2k views

What is the best filesystem for Maildir folders storage?

Projected volume size is around 1TB,. Filesystem should perform fast when dealing with LOTS of files in every folder. Snapshots support would be great, for backup purpose, but not compulsory. It ...
14
votes
7answers
11k views

What is the best filesystem for insert performance on PostgreSQL?

I'm curious if anyone out there has done any experimentation or comparisons between file systems and database performance. On Linux, I'm wondering what is the optimal file system for a postgres ...
10
votes
7answers
4k views

Best file system for external (USB) hard drives/USB memory media

Almost all external USB hard drive and USB memory media/sticks come today preformatted with FAT32. As far as I know the only advantage of the file system is the compatibility among OSes and devices. ...