5
votes
0answers
42 views

Fast distributed filesystem for a large amounts of data with metadata in database

My project uses several processing machines and one storage machine. Currently storage organized with a MSSQL filetable shared folder. Every file in storage have some metadata in database. Processing ...
0
votes
3answers
108 views

File system benchmark that generates I/O from distribution of file sizes [closed]

I am looking for a file system benchmarking tool to test CephFS. This benchmarking tool has to: Generate a set of files (call it fileset) following a particular distribution, e.g., gamma, empirical, ...
3
votes
1answer
146 views

Choice of distributed filesystem for intensive reads and writes

I have a series of servers (HP ProLiant, 34 servers) each of which with 500 G of hard drive space. These servers are part of a computational cluster which runs processes that roughly fall into two ...
1
vote
3answers
170 views

Distributed filesystem for storing temporary files

We have a cluster of web servers that currently stores temporary files (that have been uploaded by users) on a central shared filesystem. The limitation being that anytime the server containing the ...
1
vote
0answers
83 views

How do I speed up “ls -l” on a network file system (NFS)?

I have a NFS volume mounted in the following way: theHostname.com:/path/on/theHostname on /local/path type nfs (rw,nosuid,addr=10.0.0.54) The /local/path directory has 30_000 files. When I do a ls ...
0
votes
1answer
81 views

In search of a distributed file-level filesystem [closed]

I'm looking for a distributed, fault-tolerant file-level (not block-level, preferably) filesystem for use in a small Linux-based HA cluster of sorts. The files that are stored will be fairly small ...
4
votes
2answers
206 views

Distributed storage [closed]

At my University Department we are about to upgrade the computers of our student lab (about 25-30 machines). The machines will be running Linux. One thing about the new machines is that they have ...
1
vote
3answers
540 views

failsafe RAM drive solution [closed]

I'm looking for a production solution to create a RAM drive that will be safely synchronized with HDD. I have a piece of custom software with heavy I/O load (this is some kind of proprietary ...
3
votes
2answers
685 views

Linux filesystem or CDN for millions of files with replication

Please tell me solution for this scenario: several millions files, located in one directory ("img/8898f6152a0ecd7997a68631768fb72e9ac2efe1_1.jpg") ~80k file size in average 90% random read access ...
0
votes
4answers
404 views

Distributed filesystem - confused

I am new to this concept. :) Can I use a distributed filesystem in as a way to have redundancy for web applications? Server A and Server B would have Apache/mySql installed on the distributed ...
0
votes
1answer
1k views

Is it possible to set the cache size for fuse and/or sshfs?

I can't seem to find something in the documentations. Is it possible to set the cache size for fuse and/or sshfs and if so, how?
3
votes
2answers
409 views

Distributed, Parallel, Fault-tolerant File System with high throughput

I am looking for DFS (distributed file system) that is fault tolerant and easy to maintain. I will have tons (100M+) of small files (from 1K to 500K). Files will be located in some directories what ...
1
vote
1answer
601 views

How Does DFS Replication in Windows 2008 Handle Write/Delete in Rapid Succession

We'd like to replicate a mail drop folder between two servers; however, as files are written to the source folder, a timer job on that server is also regularly processing and deleting them, meaning ...
1
vote
1answer
517 views

Clustering filesystem for small files

I'm looking for a distributed filesystem which I could use for storing lots of small files (<1MB usually). What I want to get is: 2 servers which have the fs mounted themselves and mirror the ...
4
votes
3answers
100 views

How can I tie togeather extra space on Macintosh desktops with a distributed filesystem?

I have access to a bunch of Mac desktops, the hard drives of which are under-utilized. I want to set up a distributed filesystem to gang them together into one large virtual volume. The server has to ...
14
votes
3answers
12k views

GlusterFS vs Ceph, which is better for production use for the moment? [closed]

I am evaluating GlusterFS and Ceph, seems Gluster is FUSE based which means it may be not as fast as Ceph. But looks like Gluster got a very friendly control panel and is ease to use. Ceph was ...
1
vote
2answers
248 views

file systems - theoretical

I have some theoretical questions which I'm not clear with, so if someone can explain I would be very grateful, it's related to file systems. What I want to ask, in a distributed system, if we have ...
0
votes
1answer
191 views

Distributed storage with UNC share?

What approach would you take to build a massive storage, using multiple PC's running Linux OS? The recipe I would like to use sounds like this: take 10 PCs equipped with six 2TB disks, install your ...