7

I tried to recover some data from an ebs volume on which I accidentally ran wipefs on.

I used PhotoRec (http://www.cgsecurity.org/wiki/PhotoRec)... and it got my files back but also a ton of other files that did not belong to me.

It got images, text files, code etc... They all were valid data, from not from my account.

That lead me to ask... when I delete an EBS volume, I guess my data is in the clear usable by someone else?

1

https://d0.awsstatic.com/whitepapers/aws-security-whitepaper.pdf describes Amazon's published process for dealing w/EBS. Two quotes seem relevant:

Amazon EBS volumes are presented to you as raw unformatted block devices that have been wiped prior to being made available

but also

An EBS snapshot is a block-level view of an entire EBS volume. Note that data that is not visible through the file system on the volume, such as files that have been deleted, may be present in the EBS snapshot.

The most likely case is that you are creating your volume from a snapshot that has deleted data on it.

I tried to reproduce your scenario in us-east-1 with new PIOPS, gp2, and magnetic volumes and was not able to recover any data.

That said, you can further safeguard your EBS data by making use of KMS encrypted volumes.

13

From the AWS documentation

The physical block storage used by deleted EBS volumes is overwritten with zeroes before it is allocated to another account.

From an AWS rep on their forums.

I can confirm that when any customer volume is terminated (be it EBS or an instance storage volume) it is completely wiped before being made available for use by other customers.

If this is genuine and you really have someone else's data you need to get in touch with AWS. Extraordinary claims require extraordinary evidence.

TLDR; I did two sets of tests and was unable to reproduce the results that @stevelandiss produced.

Update - test one

I tried this out myself. Here's what I did and my results.

TLDR; could not reproduce.

0) I allocated an m3.medium spot instance, with gp2 and io1 (provisioned IOPS) volumes, 10GB each. I used the standard Ubuntu 16.04 AMI (ami-b7a114d7). Note that I could not mount as /dev/xvdb as the OP suggested, AWS forced me to use longer names like /dev/xvdba which makes me slightly suspicious.

1) I installed photorec/testdisk

apt-get install testdisk

2) I used lsblk to look at the volumes available

lsblk
NAME    MAJ:MIN   RM SIZE RO TYPE MOUNTPOINT
xvda    202:0      0   8G  0 disk
└─xvda1 202:1      0   8G  0 part /
xvdba   202:13312  0  10G  0 disk
xvdbb   202:13568  0  10G  0 disk
xvdca   202:19968  0   4G  0 disk
  1. I tried to mount the disks just to check, but of course they have no file system so it failed

    mount /dev/xvdba /gp2/ mount: wrong fs type, bad option, bad superblock on /dev/xvdba, missing codepage or helper program, or other error

    In some cases useful info is found in syslog - try dmesg | tail or so.

3) I made file systems on each device

mkfs -t ext4 /dev/xvdba
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 2621440 4k blocks and 655360 inodes
Filesystem UUID: e32b2ed1-a0f8-49df-895d-c56b9802a009
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

root@ip-11-0-2-184:/home/ubuntu# mkfs -t ext4 /dev/xvdbb
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 2621440 4k blocks and 655360 inodes
Filesystem UUID: 4f1f7c75-bbce-4887-aac7-02e197a36c89
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

4) I mounted the disks

mount /dev/xvdba /gp2/
mount /dev/xvdbb /pio/

5) I ran photorec on each volume

photorec /dev/xvdba

GP2

Photorec results on new AWS GP2 volume

IO1 provisioned IOPS

Photorec results on new AWS IO1volume

As you can see no files were found. If @stevelandiss can point out what he did differently I can try again to reproduce. For example he didn't mention any mounting, and he used a different device name. I'll try again without mounting a few minutes, but I want to save this update so I don't lose it.

Update - test two

This time I did much the same, but I didn't create a file system or mount the disk. This is closer to what to what @stevelandiss did. This made no difference, no files were recovered.

GP2

GP2 Photorec on new AWS volume

IO1 provisioned IOPS

IO1 Photorec on new AWS volume

  • 2
    Steps to repro (1) Launch a Linux instance (I used Ubuntu)... (2) allocate any PIOPS EBS volume to the instance... Call it /dev/xvdb (3) Install photorec... sudo apt-get install testdisk (4) run photorec /dev/xvdb. – steve landiss Jan 26 '17 at 2:36
  • 4
    @stevelandiss I just tried creating a new volume and hexdump-ing it, all I got was zeros. – jrdn Jan 26 '17 at 4:15
  • 4
    If no-one else tries it I'll give it a shot in the next couple of days. I'd appreciate if you could edit your question to give clear unambiguous steps to reproduce. – Tim Jan 26 '17 at 8:04
  • 3
    no mean to offend but, i'd bet 100 euros, that this is a layer-8 problem. – lauc.exon.nod Jan 26 '17 at 20:19
  • 2
    @lauc.exon.nod it could also be a PEBKAC issue – Tim Jan 26 '17 at 20:24
5

from the wipefs man pages:

wipefs does not erase the filesystem itself nor any other data from the device

  • 1
    Understood... still this means that when ai get an EBS volume, it's possible to see someone else's data... – steve landiss Jan 26 '17 at 2:08
  • 9
    @stevelandiss You need to contact AWS support with your findings. – jscott Jan 26 '17 at 2:47
  • 5
    Ouch, put down "wipefs" as the number one improperly named command ever. :-) – Brian Knoblauch Jan 26 '17 at 13:28
2

we need more information about the volume. How did you create it? Are you 100% sure that no one else created it but you?

AWS does not share how they designed the technology, so i am guessing as a NetApp certified storage guy. EBS Volumes are abstraction layers, built on RAID groups. I doubt it will just one single disk. So everytime you provision a volume, you will(would) be getting chuncks from different physical devices. That makes it very unlikely for you to get complete files.

Give us more information how you provisioned the volume. But i am guessing you are making a mistake at some point. Or else this would be a huge security violation on AWS about such a basic feature.

here is the test i made, i create a new volume, a new instance. attached the volume to the instance and then ran photoRec test. i found 0 files as expected.

PhotoRec 7.0, Data Recovery Utility, April 2015
Christophe GRENIER <grenier@cgsecurity.org>
http://www.cgsecurity.org

Disk /dev/xvdf - 1073 MB / 1024 MiB (RO)
 Partition                  Start        End    Size in sectors
P Unknown                  0   0  1   130 138  8    2097152


0 files saved in /home/ec2-user/testdisk-7.0/recup_dir directory.
Recovery completed.

do you have any other IAM users in your account? maybe they attached that volume to their instances and used that way.

  • 2
    Perhaps the volume was created from a snapshot? d0.awsstatic.com/whitepapers/aws-security-whitepaper.pdf says 'An EBS snapshot is a block-level view of an entire EBS volume. Note that data that is not visible through the file system on the volume, such as files that have been deleted, may be present in the EBS snapshot.' – Jason Martin Jan 26 '17 at 16:05

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.