The tag has no wiki summary.

learn more… | top users | synonyms

3
votes
0answers
21 views

Is there a way to connect a remote folder with local cache on OSX or Windows workstation?

A company with 50 graphic designers are working on the same file server. They are using mainly Indesign. A typical project is a 60 pages indesign document with 1.5GB of linked files (something like ...
-1
votes
0answers
24 views

Distributed fs which is shrinkable. Ceph? [on hold]

I am new to distributed filesystems and I am looking for a solution in a non-business environment. I need at least two features which are absolutely necessary for me: The filesystem has to be used ...
0
votes
0answers
192 views

Why is GlusterFS so slow here?

We've set up a mirroring pair of GlusterFS servers. No special tuning, whatever came "out of the box" with GlusterFS-3.5.1 in the official RHEL6 RPM, that's what we have. The cluster works, but the ...
6
votes
0answers
60 views

Fast distributed filesystem for a large amounts of data with metadata in database [closed]

My project uses several processing machines and one storage machine. Currently storage organized with a MSSQL filetable shared folder. Every file in storage have some metadata in database. Processing ...
1
vote
2answers
53 views

Setting up a multi-host SAN

I'm looking for some advice on setting-up multi-host I/O access on our SAN: I have a blade enclosure (PowerEdge1000e) containing an Equallogic PS-M4110 storage blade with a single RAID6 volume ...
1
vote
2answers
155 views

Simple, decentralized, automatically synced, local cached file share for linux server

I wonder if someone could recommend me some solution to sharing files between a small amount of linux VPS (virtual private servers). Basically I have a bunch of linux VPS to manage, and I want to ...
3
votes
2answers
130 views

what distributed file system for a two-node failover setup?

I'm trying to set up a redundant setup consisting of two servers that have everything redundant: the database (MySQL master-master in active/passive mode) the file system (distributed/replicated) ...
1
vote
1answer
439 views

In a distributed filesystem like MooseFS or XtreemFS, should individual nodes expose “raw-er” storage, or LVM'd storage?

When preparing an infrastructure to utilize a distributed storage system like MooseFS or XtreemFS, how should individual nodes present storage to the rest of the environment? Is it better for ...
0
votes
2answers
202 views

Scale-out distributed storage with snapshots

I'm aware there are many similar questions around, several with great answers. I still haven't quite found what I'm looking for though: a distributed, scale-out FS that supports snapshots. Gluster ...
0
votes
2answers
38 views

Distribute all services between 3 VPS [closed]

i just rent 3 vps servers with this: 4GB RAM 150GB hard disk 2 CPU 6 ip address so we have vps1, vps2 and vps3 and i'm thinking distribute the services like this: VPS1 - frontend + DNS Server + ...
0
votes
3answers
138 views

File system benchmark that generates I/O from distribution of file sizes [closed]

I am looking for a file system benchmarking tool to test CephFS. This benchmarking tool has to: Generate a set of files (call it fileset) following a particular distribution, e.g., gamma, empirical, ...
1
vote
6answers
634 views

Distributed file system with local disk cache

My server infrastructure is growing fast and I decided to create a distributed storage cluster. I've been looking for a proper filesystem for this task which meet my requirement, but none of them ...
1
vote
2answers
119 views

Distributed storage and computing [closed]

Dear Serverfault community, After researching a number of distributed file systems for deployment in a production environment with the main purpose of performing both batch and real-time distributed ...
3
votes
1answer
165 views

Choice of distributed filesystem for intensive reads and writes

I have a series of servers (HP ProLiant, 34 servers) each of which with 500 G of hard drive space. These servers are part of a computational cluster which runs processes that roughly fall into two ...
2
votes
1answer
144 views

Fault-tolerant network access to redundant storage in a heterogeneous environment (including Windows)?

We are facing a design "challenge" where we are required to set up a storage solution with the following properties: What we need HA a scalable storage backend offline/disconnected operation on ...
1
vote
3answers
248 views

Distributed filesystem for storing temporary files

We have a cluster of web servers that currently stores temporary files (that have been uploaded by users) on a central shared filesystem. The limitation being that anytime the server containing the ...
1
vote
0answers
102 views

How do I speed up “ls -l” on a network file system (NFS)?

I have a NFS volume mounted in the following way: theHostname.com:/path/on/theHostname on /local/path type nfs (rw,nosuid,addr=10.0.0.54) The /local/path directory has 30_000 files. When I do a ls ...
2
votes
0answers
195 views

NFS showing files in directory, but can't open or stat

I'm using a network of Linux (Debian Squeeze on kernel 2.6.32) machines, sharing files using NFS (v3). The scenario is that a process running on client A will create a file through NFS on file server ...
2
votes
1answer
38 views

distributing access across multiple disks

I have a rather large application that requires simultaneous access to rather a lot (300+) of large files, and accesses them by reading them all at the same time. If all the files are put on 1 disk ...
0
votes
1answer
74 views

Migration from ext3 to Mogilefs

I have gigs of images on ext3 file system and I want to move them to Mogilefs on a live running site. In your opinion what would be the best approach?
0
votes
1answer
85 views

In search of a distributed file-level filesystem [closed]

I'm looking for a distributed, fault-tolerant file-level (not block-level, preferably) filesystem for use in a small Linux-based HA cluster of sorts. The files that are stored will be fairly small ...
2
votes
1answer
247 views

Are snapshots and clones filesystem-wide in ZFS-backed Lustre clusters

My goal is to find a distributed filesystem on Linux that supports ZFS-like lightweight snapshots and snapshot clones. This StackOverflow question expresses what I'm looking for pretty well. I'm ...
4
votes
2answers
212 views

Distributed storage [closed]

At my University Department we are about to upgrade the computers of our student lab (about 25-30 machines). The machines will be running Linux. One thing about the new machines is that they have ...
6
votes
3answers
6k views

Are ZFS clustered filesystems possible?

Is it possible to create a ZFS cluster? Or would you need to go the seemingly ugly (to me at least) route of ZFS with UFS formatted pools governed by GlusterFS? The idea is to see if it is possible ...
0
votes
1answer
103 views

SQl Server sharding

Can SQL Server be sharded for distributed data processing? Or are there any limitations that prevent relational databases from being sharded ?
1
vote
3answers
594 views

failsafe RAM drive solution [closed]

I'm looking for a production solution to create a RAM drive that will be safely synchronized with HDD. I have a piece of custom software with heavy I/O load (this is some kind of proprietary ...
1
vote
1answer
158 views

DFS - Stop sync of large folder that has since been removed

We have a site to site DFSR on Windows Server 2008 R2 that has been running perfectly between site A to site B until someone dumped a 20GB folder. This has overwhelmed the upload and make the internet ...
1
vote
1answer
460 views

Distributed filesystem for automated offline data mirroring [closed]

I'd like to achieve the following setup: Every time I connect my laptop to a local network, my partition gets automatically mirrored to a partition on my local server. I only want to mirror what has ...
3
votes
1answer
240 views

Are there any serverless distributed filesystems?

Similar to Is there a Distributed SAN/Storage System out there? or Distributed redundant storage solution?, except with a couple key differences (and two years newer). Context: I have a small Rocks ...
3
votes
1answer
996 views

Umount stale glusterfs partition

I am using glusterfs on several Ubuntu servers: two of them are running glusterfs servers in replication mode. Without any clear error, the glusterfs partition became stale and the system shows this ...
-1
votes
1answer
103 views

Availability of big files on multiple servers

I have to handle many (1'000 - 30'000) big files ranging from 200MB up to 2GB. The demand for these files is variable (0 - 300 downloads / file). This is why a single file must saved on 2 or more ...
5
votes
1answer
122 views

deduplicating and indexing directories of images across 150 linux machines

I have a client with 150 Linux servers spread about various cloud services and physical data-centres. Much of this infrastructure is acquired projects/teams and pre-existing servers/installs. The ...
3
votes
1answer
119 views

Is distributed storage on an internal network possible?

I've been thinking about all the workstations I have on my shop floor (about 50) and the wasted drive space each of them has. For instance, my machines only use about 30G to 40G of local storage, yet ...
3
votes
1answer
985 views

How to install ceph on EC2 Amazon Linux AMI

I want to test Ceph (a distributed network storage and file system) on some EC2 hosts which is derived from Amazon Linux AMI (amzn-ami-2011.09.2.x86_64-ebs). The kernel version is 3.2 and btrfs is ...
0
votes
1answer
797 views

GlusterFS - Unable to find hostname error

I have installed GlusterFS 3.2.6 on several servers running on CentOS 6.2. I don't have working hostnames, as it's a test setup, so I have just added the servers' IPs to the /etc/hosts file. The ...
3
votes
2answers
238 views

Distributed filesystem across a slow link

I have an image in my head where a link is too slow to realize the real-time transfer of files, but fast enough to catch up every day. What I'd like to see is a master <-> master setup where when I ...
1
vote
2answers
1k views

linux distributed file system over WAN advice

I have a fairly simple (not really) requirement but I've looked at a few solutions and can't find a good solution. I've got a Red Hat EL 6 server environment in my co-location and office, and some ...
5
votes
2answers
794 views

HW/SW Design: 2 Petabyte of storage

Disclaimer Yes I'm asking you to design a system for me :) I'm tasked to design a system to store about 10 TB / day with a retention time of 180 days. My first approach would be to go with GlusterFS ...
1
vote
1answer
1k views

GlusterFS vs Mogile vs Others?

I have been looking at GlusterFS and MogileFS (Ceph and BTRFS are off the cards for me, as they don't seem production ready yet). I am developing a web app for file sharing, that will store and ...
3
votes
1answer
1k views

GlusterFS vs Ceph, which is better for production use in 2012? [closed]

This is the same question that was asked here, but it's been almost two years since. Meanwhile Ceph has seen constant development (361 kernel commits) and btrfs, in my opinion, is on the verge of ...
-1
votes
1answer
57 views

When looking for a good network connection speed, why does “bandwidth” not tell the whole story?

While I was listening in on an interesting conversation about distributed systems, I heard people talk about how overhead and "gap" were two important characteristics of distributed systems, and how ...
3
votes
2answers
744 views

Linux filesystem or CDN for millions of files with replication

Please tell me solution for this scenario: several millions files, located in one directory ("img/8898f6152a0ecd7997a68631768fb72e9ac2efe1_1.jpg") ~80k file size in average 90% random read access ...
0
votes
4answers
407 views

Distributed filesystem - confused

I am new to this concept. :) Can I use a distributed filesystem in as a way to have redundancy for web applications? Server A and Server B would have Apache/mySql installed on the distributed ...
1
vote
3answers
395 views

Moving Away from NFS

I've been working on moving more and more services away from a reliance on NFS, and I'm wondering what others have done with this issue. I know there are distributed file systems out there, and I ...
16
votes
1answer
9k views

Why doesn't SSHFS let me look into a mounted directory?

I use SSHFS to mount a directory on a remote server. There is a user xxx on client and server. UID and GID are identical on both boxes. I use sshfs -o kernel_cache -o auto_cache -o reconnect -o ...
0
votes
1answer
1k views

Is it possible to set the cache size for fuse and/or sshfs?

I can't seem to find something in the documentations. Is it possible to set the cache size for fuse and/or sshfs and if so, how?
2
votes
4answers
851 views

Is Samba the right choice for me?

we have a distributed application which uses large amounts of content (all kind of files). There are several servers that need to access the content. Right now the content is stored on each server ...
2
votes
1answer
1k views

Setting up a Rails 2.3.x app on EC2 for easy scalability

I'm running a simple rails stack on a single dedicated machine. We're reaching our full capacity and have absolutely no setup for scaling, just one app on one machine. I did some research and came up ...
3
votes
2answers
415 views

Distributed, Parallel, Fault-tolerant File System with high throughput

I am looking for DFS (distributed file system) that is fault tolerant and easy to maintain. I will have tons (100M+) of small files (from 1K to 500K). Files will be located in some directories what ...
4
votes
1answer
547 views

Recommendations for distributed processing/distributed storage systems

At my organization we have a processing and storage system spread across two dozen linux machines that handles over a petabyte of data. The system right now is very ad-hoc; processing automation and ...