Hot answers tagged

370

This can now be done trivially with just the official AWS command line client: aws s3 ls --summarize --human-readable --recursive s3://bucket-name/ Official documentation This also accepts path prefixes if you don't want to count the entire bucket: aws s3 ls --summarize --human-readable --recursive s3://bucket-name/directory


179

The AWS CLI now supports the --query parameter which takes a JMESPath expressions. This means you can sum the size values given by list-objects using sum(Contents[].Size) and count like length(Contents[]). This can be be run using the official AWS CLI as below and was introduced in Feb 2014 aws s3api list-objects --bucket BUCKETNAME --output json --...


144

AWS Console: As of 28th of July 2015 you can get this information via CloudWatch. If you want a GUI, go to the CloudWatch console: (Choose Region > ) Metrics > S3 AWS CLI Command: This is much quicker than some of the other commands posted here, as it does not query the size of each file individually to calculate the sum. aws cloudwatch get-metric-...


60

Assuming that you are using the convention of putting the name of the instance in a tag with the key of "Name" (this is what the AWS Console does when you enter a name), then you can use the --filters option to list those instances with aws-cli: aws ec2 describe-instances --filters 'Name=tag:Name,Values=dev-server-*' If you just wanted the instance ids of ...


48

You cannot download a SSL certificate from ACM.


40

I was struggling with this, too, but I found an answer over here https://stackoverflow.com/a/17162973/1750869 that helped resolve this issue for me. Reposting answer below. You don't have to open permissions to everyone. Use the below Bucket policies on source and destination for copying from a bucket in one account to another using an IAM user Bucket to ...


25

You need to escape the backslashes in order to format the answer correctly. aws ec2 describe-instances --query 'Reservations[].Instances[].[PrivateIpAddress,Tags[?Key==`Name`].Value[]]' --output text | sed '$!N;s/\n/ /' So this is the actual command you want: $ aws ec2 describe-instances --filters Name=vpc-id,Values=vpc-ac973bc9 --query 'Reservations[]....


22

If you download a usage report, you can graph the daily values for the TimedStorage-ByteHrs field. If you want that number in GiB, just divide by 1024 * 1024 * 1024 * 24 (that's GiB-hours for a 24-hour cycle). If you want the number in bytes, just divide by 24 and graph away.


20

Using the official AWS s3 command line tools: aws s3 ls s3://bucket/folder --recursive | awk 'BEGIN {total=0}{total+=$3}END{print total/1024/1024" MB"}' This is a better command, just add the following 3 parameters --summarize --human-readable --recursive after aws s3 ls. --summarize is not required though gives a nice touch on the total size. aws s3 ls ...


18

sudo pip install --upgrade awscli


17

As of 28 March 2017, you can specify tags for instances (and attached volumes) as part of the run-instances command. Example: aws ec2 run-instances --image-id ami-abc12345 --count 1 \ --instance-type t2.micro --key-name MyKeyPair \ --subnet-id subnet-6e7f829e \ --tag-specifications 'ResourceType=instance,Tags=[{Key=webserver,Value=production}]' '...


14

Found on Use filter "launch-time" to find all instances newer than X date? using JMESPath query: aws ec2 describe-instances --query 'Reservations[].Instances[?LaunchTime>=`2015-03-01`][].{id: InstanceId, type: InstanceType, launched: LaunchTime}'


14

Note: While this can be used to fetch the (already public) certificate, it does not download the private key necessary to use that certificate outside Amazon's systems. It is now possible to download the certificate with the following command: aws acm list-certificates / aws acm get-certificate Ref:http://docs.aws.amazon.com/cli/latest/reference/acm/get-...


14

Of course it's possible! Let's assume you've got your Login account credentials in ~/.aws/credentials, probably something like this: ~ $ cat ~/.aws/credentials [customer-login] aws_access_key_id = AKIABCDEFGHJKLMNOPQR aws_secret_access_key = ZxCvBnMaSdFgHjKlQwErTyUiOp All you need to do is to add another profile to ~/.aws/credentials that will use the ...


13

When I faced the same issue it turned out that AWS required server-side encryption to be enabled. So the following command worked successfully for me: aws s3 cp test.txt s3://my-s3-bucket --sse AES256


13

s4cmd is the fastest way I've found (a command-line utility written in Python): pip install s4cmd Now to calculate the entire bucket size using multiple threads: s4cmd du -r s3://bucket-name


13

The cause of your ListObjects error is that you assigned permission to access the contents of your bucket (arn:aws:s3:::bucket/*) but you did not give permissions to the bucket itself (arn:aws:s3:::bucket). The ListObjects command requires access to the bucket. To test this, I did the following: Used two AWS accounts: Account A, Account B Created bucket-a ...


13

If you know how to fix the source of problem (for example: change application settings in Beanstalk Environment variables or deploy fixed version of your application), then: Go to page Auto Scaling Group, choose your region, find Auto Scaling Group by Beanstalk Environment ID (like e-abcd12345). In Details tab, push "edit", set Desired, Min and Max to 0. ...


11

aws ec2 describe-instances --filter Name=ip-address,Values=IP_1,..IP_N Should do what you need. use the filter name of private-ip-address to select using private address in your VPC. Pipe through something like jq -r '.Reservations[].Instances[] | .InstanceId, .PublicIpAddress' if you want the corresponding InstanceID


10

I finally succeed by using ~/.aws/config [default] region = us-east-1 output = json ~/.aws/credentials [adminuser] // <- here changes to adminuser aws_access_key_id = ************ aws_secret_access_key = ************ region = us-east-1 // <- note this line


10

You can use CloudFormer to export your existing AWS infrastructure into CloudFormation template. Alternatively you can use terraforming ruby gem to export your infrastructure as Terraform files.


9

Try this aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId,Tags[?Key==`Name`].Value|[0],State.Name,PrivateIpAddress,PublicIpAddress]' --output text | column -t


9

The > operator only redirects stdout ("standard output"), or "file descriptor 1". Error messages are usually printed on a different file descriptor, 2, stderr, ("standard error"). On your terminal screen, you are seeing both stdout and stderr. The > operator is really more just a shortcut for 1>, and again, only redirects stdout. The 2> ...


9

The 2 different classes are about different levels of abstraction. Client classes are low level wrappers around each API action. ie. AuthorizeSecurityGroupIngress Resource classes are object oriented, you instantiate an object to represent the group and interact with it that way. It provides a higher level of abstraction that decouples you from the ...


9

From Listing and Filtering Using the CLI and API: You can also use wildcards with the filter values. An asterisk (*) matches zero or more characters, and a question mark (?) matches exactly one character. List any RHEL 7.x HVM GA images aws ec2 describe-images --filters Name=name,Values=RHEL-7.?_HVM_GA*


8

You may want to ensure that your IAM policy has "vpc-flow-logs.amazonaws.com" defined as a trust entity. You can check (and set) the trust entity using the AWS Management Console under the IAM/Roles section and searching for your IAM Role name.


8

Here's the explanation on how you can perform the upgrade. http://docs.aws.amazon.com/cli/latest/userguide/installing.html Here's what did the job for me: To install the AWS CLI using the bundled installer Download the AWS CLI Bundled Installer using wget or curl. Unzip the package. Run the install executable. On Linux and OS X, here are the three ...


8

I'm not sure if you can point two ALBs to a single TargetGroup. However since you mentioned AutoScaling you can have the ASG instances automatically registered in more than one TargetGroups, therefore your approach with two target groups will work just as well with AutoScaling. Hope that helps. UPDATE #1 The OP was able to find this link to AWS docs on ...


8

Notice the constraint in the docs when using --db-instance-indentifier: --db-instance-identifier (string) The user-supplied instance identifier. If this parameter is specified, information from only the specific DB instance is returned. This parameter isn't case-sensitive. Constraints: - If supplied, must match the identifier of an existing ...


7

The above answers are OK, but my favorite of the same is; aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId,State.Name,InstanceType,PrivateIpAddress,PublicIpAddress,Tags[?Key==`Name`].Value[]]' --output json | tr -d '\n[] "' | perl -pe 's/i-/\ni-/g' | tr ',' '\t' | sed -e 's/null/None/g' | grep '^i-' | column -t in fact, one can ...


Only top voted, non community-wiki answers of a minimum length are eligible