2

Is there a smart way of deleting old files from the hdfs /tmp directory? (Just to make sure, I am not talking about the unix FS /tmp)

2
  • 1
    Define "smart".
    – mgorven
    Aug 9, 2012 at 21:38
  • 1
    I can parse the output of -lsr but I dont consider it too smart. Also, I cant list anything by date :(
    – Istvan
    Aug 10, 2012 at 20:05

2 Answers 2

2

hadoop fs -stat "%Y" "/path/*" Will output timestamps of everything in /path/. Use that along with a cut off as to what you consider too young and you can have this clean up in a shell script kicked off by cron.

This might be smarter then parsing other things outputted by hadoop fs.

0

Here's (the source code of) a small tool that does the job: https://github.com/mag-/hdfs-cleanup/

I might write one on my own (or port the given one to Python) so I don't need to create a build chain for Golang in my company.

And one more for Ruby users: https://github.com/nmilford/clean-hadoop-tmp

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .