In October, 2018, AWS expanded the capabilities of the network technology underpinning EFS so that it now natively works across managed VPN connections and cross-region VPC peering, without resorting to the proxy workaround detailed below.
It depends on how you set up your EC2/NFS instance - does it have Highly Available capabilities like fail over in case of node failure? Do you monitor the disk space and extend it if you're running short?
EFS is highly available and has virtually unlimited storage. The EFS storage is more expensive than your instance EBS storage but you get a managed, ...
"Pretty similar" in the regard that Amazon Linux 2 is based om RHEL/CentOS with the same package management system with rpm and yum commands and the same default locations for configuration files and such.
But as far as I know there is no guaranteed or implied binary compatibility for packages between those two distributions and for many components ...
A regular Linux NFS server would do the trick with the following combination of /etc/exportfs options:
Citing man 5 exports:
all_squash - Map all uids and gids to the anonymous user.
anonuid and anongid - These options explicitly set the uid and gid of the anonymous account.
With Amazon EFS you'll need locally ...
To mount an EFS (nfs) volume on Amazon AWS do
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 volumeID.efs.zone.amazonaws.com:/ /your/mountpoint
You could also add an "automount" option on /etc/fstab , so it is persistend after a reboot (add it as last line)
volumeID.efs.zone.amazonaws.com:/ /your/mountpoint nfs4 ...
EFS runs on a system of IO credits, these credits are generated constantly throughout the day based on the amount of space you are using in EFS.
Every time you read or write to an EFS volume you consume these credits. If you have no IO credits in your balance, reads and writes will wait until you do.
I think it's somewhat likely, you've managed to use up ...
We had similar issues and found a solution for it. We were able to narrow the issue down to SAMBA failing to report EFS size in time. More specifically, samba fails to execute sys_get_nfs4_quota(), timing out in about 60 sec.
To overcome this issue, we added a custom script to samba to report 8 Exabytes instantly without trying to calculate the size. Given ...
I'm having similar slow access via samba share mounted on Windows. I'm getting good performance (at least fraction of a second access when navigating directories etc) on the EC2. Windows takes minutes to become accessible from the "my computer"/"this pc" view of the mounted drives - it does seem fairly performant once i'm within the directory though.
So - ...
Modify the lunch configuration to restart docker service right after mounting EFS. Then only ECS will use the mounted EFS as volume. Otherwise it will use the original directory (mount will be ignored).
echo ECS_CLUSTER=prodcluster >> /etc/ecs/ecs.config
sudo yum install -y nfs-utils
sudo stop ecs
sudo mkdir /home/ec2-user/...
I'm going to speculate that your EFS filesystem is very small.
If so, you are overrunning the allowed throughput so aggressively that you are getting timeouts.
Size + Performance are tightly coupled in EFS -- larger filesystems are allocated more credits and thus able to sustain more I/O.
The only way to make a small EFS deployment faster is to create ...
Try to execute /tmp/mount-efs.sh script manually while connected via SSH to your instance and check if it returns any error.
Most likely the issue is in this line which failed to mount /mnt/efs directory to EFS:
mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 $EFS_NAME:/ /mnt/efs || true
Here's what I'd do:
Create a "golden image" AMI that has everything place as of now. Ideally that would be set up using a combination of CloudFormation and Opsworks.
Set up AWS Code Commit to store your source code
Set up AWS Code Deploy to deploy updated source code to your instances. This means you don't have to rebuild the AMI for every source code ...
I believe the audit daemon is what you are looking for:
Once you get your rules setup aureport -m gives you reporting on modifications to whatever file or directories you configure.
You'll also want to be mindful of disk space because audit logs can build up pretty quickly. You'll want to either run jobs to ship those ...
I would be cautious of the S3fs recommendation. S3 isn't designed to be a file system and has some limitations when an application expects it to be. S3 is eventually consistent, meaning an update may or may not be reflected in a timely fashion. S3 has no concept of partial changes, so using a log file as an example, if you're appending or otherwise altering, ...
I then open up port 1194 for incoming UDP connections on the server I want to mount EFS to.
UDP port 1194 is the IANA assigned port for OpenVPN, so it's not clear how you concluded this would be needed.
EFS uses only TCP port 2049. The security group associated with the EFS filesystem's mount target needs to allow access from your instance to TCP/2049.
So you've got this t2.micro in an Auto Scaling Group (ASG) I assume?
And this ASG is configured to scale up/down based on average CPU load?
You overload it with some large ZIP file manipulation, run out of CPU Credits, CloudWatch notices the average CPU load goes above the threshold and starts a new instance. As expected.
That takes the average CPU load ...
Alternatively if you don't want to mount the new EFS on the existing server you can mount it only to the new one and copy the files over network.
[root@old-instance ~] # rsync -PrvaSHz /efs new-instance:/efs
(assuming the EFS volume is mounted as /efs on both sides)
Hope that helps :)
This seems too simple... I'm not sure I've fully understood your question.
Mount both EFS file systems to a single instance, likely your current instance.
Use a copy command to copy the files to the new EFS file system.
Disconnect the new file system from your instance
Mount the new EFS file system to the new instance
Here's some commands that have ...
EFS performance depends on how much data you have on the volume. The more you store the higher is the performance. That’s probably why with just 1.3GB it’s slow.
You can however pay for EFS provisioned IOPS that will increase the performance for an extra cost.
Alternatively you can simply store a couple of big files (e.g. 10x 50GB) to increase the volume-...
Unfortunately, no. inotify works by hooking into the kernels 'write' code. As remote filesystems have the ability for other things to write to them, without the kernel being aware of it, inotify explicitly does not work for them.
For more information, there's a useful overview here:
EBS is block, EFS is file, S3 is object.
SG are fancy local caches and gateways to access the remote storage with familiar protocols.
SG still requires the other storage systems. If AWS is like a non-cloud remote storage array, SG is the filer and VTL options that speak file and tape.
How busy is your website? For a small / medium business WordPress website served from EFS should be good enough.
Start with that and see how you go. If you find it's too slow you've got a number of options:
Make sure you use EC2 instance types with high and consistent network throughput. EFS performance depends on the instance network performance so the ...
Why is source code on your server? Is it PHP or an interpreted language? If it's a compiled language keep your source code off the servers.
Best practice I think would be keep your code in source control, such as AWS CodeCommit, then build / deploy with CodeBuild and CodePipeline. You then have two main options for deployment if you want auto scaling:
umask 002 /the/directory
That will set the directory itself and the directories inside to have 775 permissions and the files to have 664 permissions.
The directory (and subdirectories) will have execute permissions for owner, group, and others in order to allow them to be traversed. The files will have read-write for the owner, read-write for ...
It seems like you need to create a new PVC for a new Pod to use an existing PV.
If you delete PV.Spec.ClaimRef reference in PV settings, any new PVC with appropriate setup could use it. Or if you want to assign PV to the exact PVC, you can fill PV.Spec.ClaimRef with the name of PVC and then create PVC with that name.
Don't forget to use Retain policy for ...
You can mount S3 as a file system using S3fs. I'm not sure if it's production ready.
A better alternative could be running backup software on your instance that does incremental backup to S3. I use Restic for this, but note that it's not up to v1.0 yet, and it's missing compression but does de-duplication.
A better option I use for critical data is to use ...
Storage Gateway will work, but you mind find it to be overkill for your purposes.
If all you are doing is backing up data you might be better served by setting up a nightly cron job to run the aws s3 sync /path/to/data s3://bucket/folder/ command to sync up whatever data you need directly to S3 without going through a mounted file share intermediary.
The way NFS operates is that is wants UIDs/GIDs synchronized across all participants. Newer versions support mappings (with idmapd) of local deviations, but that's pretty much it.
Aside from working with ACLs, I fear that you are out of luck.
You have a number of options, but what's best depends on your requirements, but you haven't given enough detail to recommend one option over another. I probably haven't covered every option.
You can set up an NFS server on an EC2 instance. This uses an EBS volume and shares it over NFS. There will be plenty of tutorials online to do this (one, ...