Tag Info

Hot answers tagged

84

An AMI is, as you note, a machine image. It's a total snapshot of a system stored as an image that can be launched as an instance. We'll get back to AMI's in a second. Lets look at EBS. Your other two items are sub-items of this. EBS is a virtual block device. You can think of it as a hard drive, although it's really a bunch of software magic to link into ...


28

When you attach an EBS volume, you specify the device to attach it as. Under linux, these devices are /dev/xvd* - and are symlinked to /dev/sd* In the AWS console, you can see your EBS volumes, what instances they are attached to, and the device each volume is attached as: You can achieve the same thing from the CLI tools. Set the necessary environment ...


26

If the root file system is ext3 or ext4, then run: sudo resize2fs /dev/xvda1 If the root file system is xfs (less common), then run: sudo xfs_growfs / You can omit "sudo" if you are logged in as root. These commands should be run while the system is running and the file system is mounted. It's standard for EBS volumes to not contain a partition ...


23

Like others have said, EBS--Elastic Block Storage. I am using it myself now that it is released to the general public. It is better than S3 on multiple points: EBS are fast. Faster than even the local mounts, according to Amazon. EBS mounts as proper devices. Unlike S3, which you'll need custom S3 oject access logic in your code, or middleware ...


22

You're safe to use the volume once you have triggered the snapshot, even if it's still in a pending state according to AWS - see this post. If you're taking a snapshot for the first time, it probably will take a while as it has to make a full copy to the region-wide S3 bucket, but remember, it's incremental after the first one has been stored so should be a ...


22

I had the same question as you, so I worked out how to do it. First, I did this from the Ubuntu 32-bit EBS-backed ami from the US-East region, other OS's or images may work differently. However, I suspect that you should be ok, as long as you are using an ext* filesystem. It might work on other filesystems, but you'll have to figure out how to resize them ...


15

Yes, EBS is fault tolerant on the back end, but EBS failures do occur and in unexpected ways. What you don't see is the type of failure that most of us are used to - drive goes bad and just fails outright. The most frequent failure is a huge and unpredictable increase in latency which can make your application unresponsive. With RAID1 or RAID 10 sets, you ...


13

It would help if you posted your my.cnf and whether you are using InnoDB or MyISAM tables and whether you are a read-heavy or write-heavy workload. Otherwise, we are just making guesses. Here are mine: First, I would look and make sure that your queries are properly indexed. High I/O on MySQL databases is either caused by extremely high concurrency, a ...


12

EBS is charged hourly and it's against what you allocate. The snapshots are supposed to only be what you've used within the allocation. For 100GB it'll get rounded up to a whole hour's usage and assuming the East or West region you should warn your bank about the incoming charge of about $0.02.


11

Try the following link. Add a config file in your root inside .ebextensions directory. Then add this. files: "/etc/httpd/conf.d/vhost.conf": mode: "000644" owner: root group: root encoding: plain content: | NameVirtualHost *:80 <VirtualHost *:80> DocumentRoot "/var/app/current/" <Directory ...


11

On a default UNIX-like system, cron has a minimal environment defined. Usually HOME, SHELL, LOGNAME are defined and PATH is set to /bin. You have two solutions: enter the full path of your script. ie: /home/me/bin/ec2-automate-backup.sh alter the PATH environment variable. If you want to receive email from the cron daemon define the MAILTO variable. ...


10

RTFM - I found my answer in the Amazon EC2 Instance Store docs. Instances that use Amazon EBS for the root device do not, by default, have instance store available at boot time. Also, you can't attach instance store volumes after you've launched an instance. Therefore, if you want your Amazon EBS-backed instance to use instance store volumes, you ...


10

As you said, this isn't exactly an apples to apples comparison (in addition there is agreement already, that decent data backup procedures must be in place for both, so I'm not going to address this). Therefore the question cannot be answered as such, rather one should be aware of the architectural details of each offering and apply those in respect to a ...


10

Just to add a little to the consensus that is EBS faster... An important point to understand here is that instance storage speed is unpredictable - Amazon makes no guarantees about it. This information comes from the book Cloud Application Architectures by George Reese (which seems pretty good from what I've skimmed it so far.) It includes a table titled ...


10

Here's a complete set of commands that you should run once you've attached the EBS volume to your EC2 instance: mkfs -t ext3 /dev/sdf # (NOTE - if you created your EBS volume from a snapshot of another volume and want to preserve its data, skip this step) echo "/dev/sdf /testName ext3 noatime 0 0" >> /etc/fstab mkdir /testName mount /testName ...


10

Make sure the EBS-drive has the name /dev/sda1 in the EC2-Console.


8

Your approach sounds very good - but I can think of a possible way to improve it. To reduce the impact of data loss since the last backup, and EBS volume failure (unlikely, but still possible) you could store your data on a separate EBS volume than your system files, and back up the data volume more frequently than the system volume. With your current ...


8

What does it provide? Quite a lot, if you're not a DBA. Automated snapshots to the second for the previous five minutes (configurable up to 8 days) DB snapshots are automatically scheduled, managed, and stored HA without having to think about it? Sign me up! As for your additional questions; the RDS DB is persistant, and it's equivalent to setting up ...


8

Amazon does not currently provide a method to report on the storage used by snapshots other than the total usage and cost numbers for all snapshots in the account. Snapshots of the same or related volumes can share storage (where blocks have not changed between snapshots) so this makes it difficult to define the size of a single snapshot. A new snapshot ...


8

One thing to remember is that not all data needs to be permanent. Instance store provides a cost effective solution for dealing with temporary data. Let me provide a few examples. The most obvious is swap space. If you want to allocate a few GB of swap space, a file on an instance store device is perfect - no cost to the I/O operations, and the data does ...


7

Amazon has not provided a simple command line interface to copy EBS boot AMIs between regions like they did with S3 based AMIs. In order to copy an EBS boot AMI, you need to start instances in both regions and copy the EBS snapshot from one region to the other using a temporary EBS volume attached to each instance. I recommend using rsync with appropriate ...


7

It all depends on how much you want the data gone. If you just want to avoid having to sit through a lengthy rm, then a reformat (as suggested by Sysadmin1138) would do the trick. If you'd prefer not to have to worry (too much) about the data still being accessable, then being an EBS volume you can just unattach it and attach a new one, and let Amazon ...


7

Some people have been able to attach 40 EBS devices to the same instance using RAID (see http://alestic.com/2009/06/ec2-ebs-raid) You could probably attach more than 40, but be aware of the device names limitations, since according to EC2 documentation these are the only devices available for external EBS drives on Linux instances running xvd drivers: ...


7

There's nothing broken, so there's not really anything to "fix". This change (/dev/sdX to /dev/xvdX) was made in recent kernel versions (perhaps the 3.x tree? I can't recall specifically at the moment). EC2 hasn't yet changed their terminology to reflect the change in device naming conventions, as that would confuse all those who are running older kernels. ...


7

We ran up roughly $1000 in AWS charges in a systematic exploration of this and related questions, and posted an extensive analysis. For random (small) reads, we found "local" (EC2 instance) storage and EBS to have similar performance, but local storage has a better latency tail. For sequential (bulk) reads, EBS is faster on m1.small, but local is faster on ...


7

Amazon does not recommend RAID1 (which is part of RAID10). See "Amazon EBS Volume Durability" in http://aws.amazon.com/ebs/ where they state: "Because Amazon EBS servers are replicated within a single Availability Zone, mirroring data across multiple Amazon EBS volumes in the same Availability Zone will not significantly improve volume durability." ...


7

The EC2 monitoring documentation suggests a method of doing this by installing the Monitoring Scripts for Amazon EC2 Instances and then configuring a cron job to gather the data and reprt back.


6

The amount of time it takes EC2 to build a snapshot is directly proportional to the number of modified blocks that volume has seen since the last snapshot. In the case of your journal, that sees a lot of write/delete operations, so it's not surprising that it would take a long time to snap. Keep in mind that these are truly block devices. They know nothing ...


6

Is S3 a proper choice to keep live linux user home directories? Amazon has announced their EFS service, which is exactly what you need for this. Either use EFS or roll your own NFS server for home directories.


6

The CPU can and will be used for other processes, provided there is at least one process that is ready to receive CPU time. There's the rub - you can have an I/O-bound system with every process waiting for I/O to complete, and as there is nothing waiting for CPU time, there is no reason to schedule (and utilize) CPU for anything other than the kernel's ...



Only top voted, non community-wiki answers of a minimum length are eligible