186 reputation
11
bio website
location
age
visits member for 3 years, 8 months
seen Jul 3 at 18:58

Sep
19
comment Quickest way to transfer 55GB of images to new server
How can you store the compressed file(s) with rsync?
Aug
10
comment How to fix Hadoop HDFS cluster with missing blocks after one node was reinstalled?
Most files had replication=3 (the default), some larger datasets had replication=2, and it's possible some users used other replication (e.g. rep=1), but I'm not sure. With rep=2, HDFS should hopefully put that data on different servers, so I'm hoping that someone used rep=1 for some non-important data, and that's what the missing blocks are. I'll email the users to check
Jan
23
comment Bash Loop - How to stop the loop when I press Control-C inside a command?
Additional tip: if you're in a nested loop, and want to completely terminate, use break 2 (or replace "2" with the number of nested loops you want to terminate).
Jan
17
comment Performance impact of running different filesystems on a single Linux server
Maybe try emailing/tweeting the author.. let's get his/her input!
Jun
22
comment How much power supply do I need for my server, and could a shortage be causing my odd crashing?
In my case, the 5400's were not being used by Hadoop (more for long term less-accessed data). I'll look into sar and the IO wait time. Thanks
Jun
20
comment How much power supply do I need for my server, and could a shortage be causing my odd crashing?
Understandable.. I added more details on my particular hardware. You make a good point on using a power meter. Can you think of any other measures I can do now for determining if power is maxing out vs. some other system issue (e.g. system logs, etc)?
Jun
19
comment Hadoop: Blacklisted tasktracker
I just noticed that there are two separate settings for defining hosts (dfs.hosts and mapred.hosts). I never figured out how to restore blacklisted tasktrackers for a job, but I was trying "dfs.hosts", which in hindsight, doesn't make sense (I should have tried mapred.hosts and then the restart). If Roberto or David have insight, please share :)
Jun
13
comment Hadoop: Blacklisted tasktracker
Any idea if this should work if I put the setting in hdfds-site.xml while the job is running? I tried this, refreshed, then restarted task tracker on slave, but it is still not doing work, and shows in web interface as blocked.
Nov
11
comment Are the Hadoop {start|stop}-all.sh scripts typically used w/ cloudera 3 clusters, and if so, how?
I didn't see anything about starting/stopping the slave nodes from one server, which is what I'm looking for. Currently I can already use the etc/init.d scripts to start/stop locally. Let me know if I missed something please
Sep
17
comment Most secure way to issue commands on ubuntu cluster with sudo'ing user?
Puppet's new to me, but it looks like a promising solution. If you got any favorite how-to's (outside of their documentation) or ways you've used, please let me know. Thanks!
Sep
13
comment Are the Hadoop {start|stop}-all.sh scripts typically used w/ cloudera 3 clusters, and if so, how?
I got a tumble-weed badge for this bad girl.. who dares answer my question! please dare
Jul
14
comment How can I track down a memory leak with wsgi, django, php and apache2?
Great information. I'll try this, and reply back with my findings. Thanks
Apr
15
comment How can I serve a static landing page if my server is down?
FYI for everyone: I spoke with Rackspace, and they do not have a feature to load a static HTML page when all instances are offline. I went ahead and requested this feature, so if others like this idea, please vote it up on Rackspace: feedback.rackspacecloud.com/forums/71021-product-feedback/… I also requested the same feature from AWS, so add your two cents there as well: forums.aws.amazon.com/thread.jspa?threadID=64591
Apr
12
comment How can I serve a static landing page if my server is down?
I should've stated that I have a willingness to bring up an additional server for this problem, although I'd like to keep it at a small instance or maybe one of Rackspace's new load balancers (which I need to research a bit on first). My current main server's needs are as Medium instance
Mar
30
comment How can I control log messages for Apache (specifically, auth_mysql module & WSGI)?
Any idea on how to control this on a individual module level?