35
votes

What is the most damage (of whatever kind) that you have ever caused with a single mistaken/mistyped/misguided command line? I deleted a production system database by mistake a while back, for example, but I was lucky (i.e. backed-up) and there was no permanent data loss, lost money, property damage etc.

Most importantly (for votes), what do you do to make sure it will not ever happen again?

7
  • 9
    Since this is a poll, it should be CW
    – Eddie
    May 6, 2009 at 2:00
  • Then why aren't polls CW by default?
    – Luke
    May 6, 2009 at 4:19
  • 4
    How should they know it's a poll progmatically?
    – Mikeage
    May 6, 2009 at 8:14
  • 1
    If the mechanism is the same as SO's, even if you can edit a question, you can't change it to CW. Only the OP or a moderator can do that.
    – chaos
    May 6, 2009 at 14:09
  • 9
    Good idea - post your most destructive mistake publicly on the internet for all future employers to see! :) May 13, 2009 at 19:29

81 Answers 81

1 2
3
0
votes

As root : rm ./* on a Tru64 machine. Yes, in "/". Yes, on a production machine.

I did not do this - some other jerk did.

The (longish) story of recovery is here: h t t p : / / slashdot.org/~Noryungi/journal/212909

What did I do? Nothing. Another, even bigger idiot (CTO) insisted all his programmers needed to have access to the root password. (sigh)

As a normal user : rm -fv ./* in my /home on a Linux machine. Pressed Ctrl+C fast enough to prevent huge damages. Thanked Cthulhu for backups for days afterward.

0
votes

The rm -rf * command seems to be a pretty common problem. I had network mounted drives though and the command deleted the contents of all of them too. Once that was done it deleted the Unix operating system. UGH!

0
votes

Today I had a weird file I was unable to delete so I went in the terminal. The file was begining with a " ? " so I did :

rm ?*

Which erased every files except folders. I then remembered that " ? " mean any single characters so it was like I did :

rm *

0
votes

The most recent amusing mishap was a colleague installing a new Solaris 10 zone. The first time he'd randomly selected the swap device and the zone installed and ran quite happily. The second time he picked the device that happened to be the root drive...

The box didn't come up again. Oh, and it was a production system.

0
votes

I was working for an Internet marketing company as the systems / database administrator. I had given a fellow employee (now good friend) read/write access to a web advertising system we were building - a sort of DoubleClick.net platform style thing but just for us, so we could manage all our ads across various networks. We were writing a new reporting module to gather data on something or other and he was playing around and did the classic SQL newbie mistake:

update advertisements set url='something' 

Without the where clause. So that afternoon, when the boss came in and asked "why are all of our banner ads pointing at the Tragically Hip??" I really had no idea... the Hip being a popular Canadian band, my colleague had basically sent tens of thousands of dollars worth of ad space that we had purchased on behalf of our clients to the website of his favorite band on accident. We were both able to keep our jobs, though groveling was involved.

I got really good with Netsaint (the predecessor of Nagios) after that, and learned not to give SQL newbies access to production.

0
votes

A colleague decided to be funny and did:

net send  /DOMAIN *my_username* Hey How You Going!

Unfortunately, instead of ME getting the message, the whole floor gets a popup window saying:

my_username Hey How You Going

Luckily messenger stuff is blocked at the floor router, although a manager did run out to abuse me for "wasting company resources!"

0
votes

A friend of mine works on a classifieds site running on Windows / IIS. The site had lots of image files stored on their web server's disks that were no longer referenced from their database so he decided to write a quick cleanup script. He wrote a small program to generate the script, something like this:

List<string> imagesInFileSystem = Directory.GetFiles(imageDirectory);

foreach (string file in databaseImages) {
    if (!imagesInFileSystem.Contains(file)) {
        Console.Out.WriteLine("del " + file);
    }
}

And it generate the following output:

del c:\images\image1.jpg
del c:\images\image2.jpg
...
etc

Which looked pretty good. However when he ran the program most of the images for the site were deleted! He forgot one thing: the server's filesystem is case insensitive but his check against the database wasn't.

The sales guys had to spend the next couple of days calling up people to get them to upload the files again.

0
votes

Needed to extract a file from a tar file. No problem:

tar cvf filename.tar somefile

Just reinstalled SunOS4, restored local files from backup, and was cleaning up, removing the restored files from their temporary location when I typed rm -rf /usr and wondered why it was taking so long... I got to do it all over again.

0
votes
net send /domain "Hello World"

Oh sh!+, this isn't the lab domain.....

1
  • 1
    Hah. Once upon a time, in a poorly configured college network, a friend of mine found out that, if you logged into a netware machine using the machine login that was bound to a mac, it didn't set your permissions correctly, and you could do university wide broadcasts...He set up a program that would do one every hour or so, but screwed up his count zero, so it did one after an hour, and then another 30 minutes later, and then 15, then 7...I wandered in to work to find the student techs trying to find out how a mac that they'd ripped out of the wall was sending constant broadcasts. Good times. Mar 17, 2011 at 14:57
0
votes

A lot of people have mentioned rm -rf. I'd like to point out that it isn't the trickiest command. The one you really have to worry about is sudo rm -rf. After deleting my entire Python site-packages directory, I now refer to that command as the "nuclear option".

0
votes

I'll start with the standard disclaimer: It was many years ago, I had just started, I was a noob, honest!

I was trying to debug something on a Cisco router (don't even remember what it was), but I just couldn't figure out what exactly I needed. If you've ever tried the debug command in IOS, you'll know what I mean. Every command has a ton of subcommands, arguments, options, etc.

Finally, I came up with a brilliant idea! I'll just "debug all" and then strip out the useless stuff. (I can hear the groans from the other people who've tried this).

Long story short: it killed the router. It just happened to be the core router for the ISP I was working for. 30,000 people suddenly had no internet access.

Yes, it was dumb. Yes, the ISP should not have been letting newbies run around on core routers unsupervised. Yes, we should have had hot standby. Etc, etc. :)

0
votes

I had my home directory mounted into a chroot in loop mode to do some work. The chroot was taking too much space, so I had to remove it, and I just did

rm -rf /path/to/the/chroot

It too a bit longer than I thought before I realized it was removing the documents from my loop-mounted homedir altogether...

0
votes

Good stuff here. I've done a LOT Of them. To avoid the missing WHERE clause in SQL, I now use: alias mysql='mysql --i-am-a-dummy'

An interesting one was while updating a remote server: mv /home /home_old && mv /home_new /home mv /var /var_old && mv /var_new /var mv /lib /lib_old && mv /lib_new /lib

It seems that Linux stops running (or booting) when /usr/lib dissappears. Remote servers now have either an IMPI card, or a serial console lnk to another server, plus a boot CD in the drive.

0
votes

rm -f /bin

instead of

rm -f ./bin

as root (of course) on an old FreeBSD system. Luckily all the commands in /usr/local/bin and /usr/bin sufficed to copy binaries from the install CD /bin and unhose the system.

0
votes

When trying to clean my home folder of temp/backup files:

Run this:

rm -rf *~

# OOPS! DON'T RUN THESE:
rm -rf *  # OOPS! Deleted all files in $PWD
rm -rf ~  # OOPS! Deleted all files in $HOME
rm -rf ~* # OOPS! Deleted all users' home folders!!!
0
votes

I've done a rm -rf * .o but the best I've seen is:

userdel -r root

It was done by a novice though and he was learning.

0
votes

Intending to clean up the build directory, I removed all the source code for a project with:

rm * .o

Since then, first thing in my projects is a clean target in the Makefile

0
votes

Removed my whole home directory when trying to clean a floppy disk with the command:

rm -rf * /mnt/floppy
0
votes

The worst I've seen:

In /var/www// Instead of removing the folder with [rm -r var]

He removed the complete /var folder with [rm -r /var] One slash can do a lot :/

So yeh, that guy had fun trying to get backups... what only made it worse because shutting down the server and doing a recovery would just get everything back upto the last little update, the backups were only done once a week

0
votes

I was doing a log parse for a mail server using grep, so I typed:

grep -r event_string /var/log > output

Unfortunately, I was in /var/log when I typed it, so every time it matched the string it wrote the string to a file that it was in the directory it was checking...When it got to that file, the file became a space devouring beast that could not be fed. Filled up the whole machine, and, since it was a mail server, caused a pretty pressing problem.

-3
votes

Anyone else read this and thought "damn, I done most of those"?

How about "alias rm rm -i" in your bashrc for root? then you can't as easily zap everything?

Also keep a backup script in ~ for the production db at work, so easy to run, I always do it before I log in with write privs, and if i am just running a bespoke report, use a ro user!

3
  • 1
    -f overrides -i, FYI.
    – MikeyB
    Jun 25, 2009 at 17:02
  • - thanks! Just best stay away from the ubiquitous f - think folk use it when it is not required..
    – Tom Newton
    Jun 25, 2009 at 21:39
  • Version control systems that make store their metadata in multiple read-only files train users to use rm -rf... Sep 28, 2010 at 22:44
1 2
3

Not the answer you're looking for? Browse other questions tagged .