Questions tagged [distributed-filesystems]
The distributed-filesystems tag has no usage guidance.
128
questions
0
votes
0
answers
29
views
Erasure coding error detection ability
I would like to know in an (2+1) Erasure coding storage setup, can it detect corruption when one node is lost, and there is a silent data corruption from HDD on one of the remaining node.
I am ...
- 23
2
votes
2
answers
307
views
Distributed SSD cache for network storage
Our small computing cluster has 3 computing nodes and 1 file server. The file server has around 70TB data to be accessed by the 3 computing nodes. All of the 3 computing nodes have 3 4TB SSDs which ...
- 21
1
vote
1
answer
54
views
Is there a remote filesystem for Linux that's encrypted and resilient against unreliable servers?
I've tried many combinations of sshfs and SMB, with autofs and/or manual mounts in a Cron job, and I've always had freezing / long timeout issues when servers become unavailable or weren't available ...
- 11
0
votes
0
answers
24
views
Seeking Advice re. IPFS Clustering - Overhead, Performance, Design Guidance
I’m about to rebuild a server cluster - currently set up for high reliability & availability - RAID, disk mirroring, automatic failover, all that good stuff.
I’ve been thinking about replacing the ...
0
votes
0
answers
17
views
transparent management of scratch fs on cluster node
I have a cluster made of 5 node, the node 1 host a big file-system exported via NFS to other nodes.
The other nodes have a fast nvme disks used for scratch.
I would like to encourage users to work on ...
- 101
-2
votes
1
answer
59
views
Are there any distributed transactional filesystem implementations that I can actually use?
I've been searching for distributed transactional filesystems so I don't have to make my own subpar implementation.
I've only come across two:
Warp Transactional Filesystem - The website is no longer ...
- 121
0
votes
0
answers
88
views
Finding the number of blocks given replication goal in Google FIle System?
A cluster contains 1500 machines each having 500GB disk capacity. Calculate approximate number of chunk servers, blocks and total available size if default chunk replica is 3 and 5 respectively?
We ...
- 1
0
votes
0
answers
63
views
Independent distributed FS for Linux with ACL
I have two Linux servers in two locations connected by VPN (over WAN - 100/100 Mb/s). On each of these servers is Samba DC (the same domain) and Samba File Server.
Samba FS are configured to store ACL ...
- 1
1
vote
1
answer
2k
views
Docker Swarm Distributed File System with local storage
I'm looking for a distributed file system solution/network file system which can be used in the following scenario:
I have a Docker Swarm with many hosts, but each host is essentially self-contained ...
0
votes
0
answers
138
views
Replicating filesystem Samba and Windows
I'm looking for a way to create a replicating filesystem between Samba and Windows. I thought that DFSR would be the answer but Samba doesn't support SYSVOL replication. I also looked at syncthing ...
- 101
1
vote
0
answers
21
views
What's the simplest way of "resetting" multiple linux clients to restore the default state of a fresh install?
We have 6 linux clients (mint) which are used for training. Courses might run anywhere from a few days to multiple weeks.
They want a "one button press", as in very quick and easy, way of ...
- 11
2
votes
0
answers
34
views
Lustre glitch: latency of minutes
Using a HPC lustre filesystem, we occasionally experience glitchiness where even simply opening a terminal and typing "ls" can take minutes to return. That is, any process that involves the ...
- 121
-3
votes
1
answer
314
views
What file systems are dominantly used in data centers? [closed]
I would like to know what file systems are typically used in data centers and how do their performance compare to ext3/4 file system?
0
votes
1
answer
69
views
How to strictly serialize replication of one folder using Windows DFS replication?
I have two locations A+B and need to process one special directory between those branches using Windows DFS replication in the following way: All writes into that directory of A need to be pulled by B....
1
vote
3
answers
311
views
Can I use RAM as Cache for a secondary storage device
I have 10TB~ RAID 5 array and two NVME 500GB on RAID 0
and they in different systems.
what I want is to make a stick of RAM like 16GB or something to act as a cache or buffer to give me faster access ...
- 11
1
vote
0
answers
72
views
Heavy network usage withg GlusterFS FUSE client
I have a simple GlusterFS 7.2 setup with 3 servers:
data1 and data2 contribute several bricks (4 each) to a distributed replicated GlusterFS volume.
host1 uses the GlusterFS FUSE client to mount the ...
- 111
0
votes
0
answers
69
views
Any Open Source Linux Distributed Storage Options Which be Deployed inside Rented KVM VPSes Without Additional Virtual Disks?
I am trying to figure out a scalable and HA solution for storing live data under production workloads like DBs, web contents, etc.. Which can be installed and setup on standard rented Linux KVM VPSes. ...
- 344
0
votes
1
answer
159
views
GlusterFS - Define Failover Cluster
I am currently planning the storage infrastructure for our new application. Since performance is a big and important item on our list, I want to use SSDs for the production environment. We sell a ...
- 1
1
vote
1
answer
91
views
Ridiculous Number of Files in a Directory
My employer acquired a company with one particular piece of software that stores a lot of PDF and PNG file files in one directory. When I first replicated it from AWS, there were about 11.5 million ...
- 51
0
votes
1
answer
149
views
Change HDFS replication factor
I've changed replication factor from 3 to 2 for some directories with command:
hdfs dfs -setrep -R 2 /path/to/dir
but my HDFS free space still the same. Should I do something else to free my disks?
1
vote
1
answer
707
views
Can kubernetes cluster nodes be geologically spread out?
Me and a couple of friends are thinking of setting up a kubernetes cluster where our homeservers will act as the nodes.
As our nodes will be spread out between our apartments I am worried this will ...
1
vote
1
answer
695
views
AWS EFS - file delete and recreate not detected programmatically for 25 to 30 seconds
I am observing a very large delay in EFS detecting that a file has been re-created i.e. deleted and then created again.
In this simple test example, I have a single file that gets deleted and ...
- 121
1
vote
1
answer
326
views
Multiple disks acting as one filesystem, while still allowing each disk to be accessed alone
I have a fairly large dataset (~160TB) that need to be delivered to a client every so often. This dataset consists of fairly large files, usually between 2Gb and 20Gb each. They exist on a BeeGFS ...
- 535
5
votes
3
answers
4k
views
What differences are between a NAS, a shared disk file system on a SAN, and a distributed filesystem?
https://en.wikipedia.org/wiki/Clustered_file_system#Network-attached_storage says
Network-attached storage (NAS) provides both storage and a file
system, like a shared disk file system on top of a ...
- 1,477
1
vote
2
answers
1k
views
Does a distributed filesystem have to consist of multiple filesystems located on different computers?
Does the concept of a distributed filesystem require a distributed filesystem to consist of multiple filesystems located on different computers?
Does an NFS filesystem consist of only one filesystem? ...
- 1,477
1
vote
2
answers
152
views
Do distributed file systems partition a file across nodes?
In HDFS, a file is partitioned into blocks which are distributed across several nodes.
I am wondering if that is also true in the following distributed file systems: NFS, Andrew File systems (and ...
- 1,477
0
votes
1
answer
229
views
How can I create or access a remote file as if it were local?
In general, I would like to run local programs on remote files provided to me via a distributed file system (e.g. samba, NFS or whichever you recommend), and I want to access the remote files in a ...
- 1,477
6
votes
0
answers
904
views
How does S3FS (or any other S3 FUSE filsystem) compare to AWS FSx for Lustre + S3
I remember trying s3fs a year back, trying to use some S3 bucket as a FUSE filesystem. I remember it being quite laggy, especially when coupled with git operations on it (an oblivious system architect ...
- 235
1
vote
2
answers
605
views
K3S arm64 distributed files systems
I deployed a K3S Cluster on arm64 and I need to have a DFS directly on-premise, It will be better to have it running on the kubernetes stack.
Regarding DFS choices there are some options, but...:
...
- 113
0
votes
0
answers
24
views
Will DFRS support a lot of data over a WAN?
I have a server in our home location that houses many of our engineers. This location has Comcast fiber 100 up and down. Internally, the network here is much faster than that. This location has Server ...
- 75
0
votes
1
answer
1k
views
glusterfs client mount point fails with 'Client quorum is not met'
Assume a basic replica 3 arbiter 1 configuration, glusterfs-server 4.0.2 and glusterfs-client 4.0.2.
glusterfs-client is installed on Ubuntu 18.04.
In order to verify that write / read ops are ...
- 101
3
votes
1
answer
384
views
Does Perkeep (camlistore) have built in protection from bitrot?
Does Perkeep (PKA Camlistore) offer protection from silent corruption (eg. bitrot) of the data in its current design like what is offered in ZFS. If it does, how well does it fare when compared to ZFS?...
- 299
1
vote
2
answers
4k
views
DFS Namespace unavailable at login
Edit: I was not fully aware of what was going on with my issue, but it has since been fixed. See my answer below.
Background
We are having some issues with DFS namespaces in our environment after a ...
- 111
2
votes
1
answer
198
views
Which distributed filesystem will actually fit my needs?
Hear me out
I have seen the question asked (in different forms) here, here, and perhaps the best one I found was here, but I do not think this is a duplicate because quite some time has past since ...
- 123
1
vote
0
answers
210
views
Is it practical to keep home directories on OpenAFS?
I am a sysadmin by trade, and I do what I do at work at home as well for fun. I have a Gentoo Linux laptop, Raspberry Pis running Raspian, a Gentoo server, ARM devices running Debian and have various ...
- 978
0
votes
2
answers
734
views
Networked In-Memory FileSystem w/ zram + XFS + GlusterFS - Best Practices
A few months ago I used XFS formatted zram devices strung together with GlusterFS to create a distributed / networked / replicated in-memory filesystem on a few bare metal servers (running RHEL 7.2).
...
- 1
1
vote
2
answers
1k
views
Asynchronous file replication over long distances, with buffering
I'm looking for a filesystem that can replicate over long distances, and tolerate being offline for extended periods of time by using a local buffer, which should be a disk buffer, to queue up changes ...
- 111
2
votes
1
answer
1k
views
How to make GlusterFS mount point highly available?
It's true that one can mount a GlusterFS volume on any brick and the FUSE client will automatically discover the other bricks and connect to them as well. If the initial brick fails the client will ...
- 7,648
0
votes
1
answer
53
views
GlusterFS server logic
I quite don't understand about GlusterFS probes. See, i add all servers to /etc/hosts file and i do this for each server. Lets say i have 100 servers and later i added 100 more. I cannot imagine ...
- 144
6
votes
1
answer
19k
views
GlusterFS Transport endpoint not connected from time to time
I'm using GlusterFS 3.7.9, currently on a single server with 4 bricks.
Each brick has 4TB and the volume is set up as distribute only.
The volume is mounted on a secondary server and I use it for ...
- 315
4
votes
2
answers
848
views
GlusterFS alternative for file upload website
I have a few file upload websites, with files ranging from hundreds of kilobytes to a few gigabytes.
Currently I have all files in a distribute-replicate Gluster volume on a few servers.
My ...
- 315
2
votes
1
answer
357
views
Glusterd dead but cluster still running
We are currently running a three node cluster, on Gluster 3.6.4.
On one of our nodes we noticed that the glusterd daemon is dead.
But the glusterfsd daemons are still running, and we believe clients ...
- 183
1
vote
0
answers
21
views
Is it possible to configure hdfs in a federation mode and in an HA mode in the same time?
I don't understand if it possible to configure HDFS in both modes in the same time. Does it make sense? Can somebody show a simple configuration of HDFS in both modes? (nameNode1, nameNode2, ...
- 733
1
vote
1
answer
312
views
How to safely store data and DBs on Kubernetes (or similar) on DigitalOcean
It boils down to setting up a private cloud on something like Digital Oceans versus another hosting solution like GCE or AWS. The latter have redundant data stores that can be safely used as volumes, ...
- 401
0
votes
1
answer
1k
views
New files on Linux Samba share: Now you don't see me, now you do
When a file is uploaded to our server, we have to do some work on it to allow or reject it.
Some files need to be opened in a Windows box (OpenXml Sdk performance reasons).
Our web server runs in a ...
- 101
2
votes
1
answer
96
views
distributed storage solutions [closed]
I've got some desktop PCs available in my company since the last user hardware renewal and I would like to re-use them for a demo plaform. They are powerfull enought to emulate a few servers. I ...
- 121
1
vote
0
answers
850
views
Glusterd won't start after node crash
We had a 2 node cluster running GlusterFS.
It's a 4 volume cluster, replica 2, one brick per server.
Dead server: nnfs02
Live server with all data: nnfs03
New server: nnfs01
After nnfs02 died, we ...
- 93
6
votes
2
answers
650
views
What is reasonable storage failover time that most OS (VM) can tolerate?
I have a GlusterFS 2 node 2 replica setup. I am planning to use it as OpenStack instance store, in which the VM disk image is stored.
From my tests, if the GlusterFS node which the hypervisor ...
- 953
4
votes
1
answer
1k
views
Distributed mirrored filesystem under FreeBSD
Can someone share their experience in building a distributed mirrored filesystem between multiple FreeBSD machines? I. e. we have two (three, four...) servers and special partition "part1" mounted on ...
0
votes
1
answer
407
views
Expanding/Growing OCFS2 forever
I have a kvm environment where a shared disk mounted by 2 VMs through rbd ceph. What I would like to know that in the future how is it possible to expand OCFS2 filesystem what I planning to use on top ...
- 29