I run multiple instances per machine of a large (~10GB) closed-source service under linux. Is there a solution to somehow have common files between the installations not take much space? (nearly all of the installs are common to all the installations). I thought about making a big hardlink tree, but each install runs its own auto-updater, so things could end up inconsistent.
Tell me more
×
Server Fault is a question and answer site for
professional system and network administrators. It's 100% free, no registration required.
|
|
Sounds like you need a filesystem with deduplication support. If there are no native filesystems with dedup support, you could consider hosting your data via NFS and placing it on say, ZFS, so each new copy doesn't take up as much additional space. |
|||||||||||
|
|
This would almost entirely depend on how your software is architected. Assuming you have access to the source code and programmers behind the program, you could talk to the devs and see if there's a way to share common data. But if each program is using its own database or something like that, you're out of luck. Otherwise you could store the data on some kind of NAS or SAN and keep the common information in a common location. If the program isn't made for it, though, I'd definitely not risk it. You'll corrupt things and create more problems for yourself. |
|||
|
|
|
I have an application that is I think might be similar. I run ~30 instances of a particular app each running on it's own VM under vsphere. Each server shares the same vmdk file for the application data store (~12 GB) which is configured as independant/non-persistant (which causes data only be persistant across reboots, and not shutdowns). symlinks are in place pointing the application to the data files on the non-persistant volume. Application state and the data we need to be persistant is written to another separate volume. When an update is released to the data set and assuming we can't shutdown the VMs (or don't have a maintenance window), we simply push the update to the non-persistant store as the changes are non-persistant and vm-specific (at least until the next shutdown). And during the next available change window, we can then finally update the backing vmdk. We've done this live a couple times in dev with hot-removing/hot-adding disks with the VM never going down, but, that's probably not something we could do in production. Obviously, this only works if you have some way of virtualizing your application, and some way of controlling how your application updates. |
|||
|
|
