Programming enthusiast. Current interest Java, Hadoop (MapReduce & Cluster configuration). Here to learn from you great people.

I also like to play Guitar and Sing.

Dec
13
comment I cannot see the running applications in hadoop 2.5.2 (yarn)
Just found this: stackoverflow.com/questions/27174593/…
Dec
13
answered I cannot see the running applications in hadoop 2.5.2 (yarn)
Oct
14
comment How to implement sort in hadoop?
@user2961121: I think, there is: hadoop.apache.org/docs/r2.3.0/api/org/apache/hadoop/io/…
Oct
14
comment Elastic Map Reduce External Jars
Can you show the command that was used?
Oct
11
awarded Revival
Oct
10
comment Oozie custom asynchronous action
@gappc: My doubt is that somehow the AtomicBoolean variable is not updated or something. Please remove the check method statements from the start() method and then check if the log messages you are doing inside check() are visible. Also, log the value of finished.get(). Would have been much better if you could test it using a JUnit test case.
Oct
7
comment Hadoop application hangs due to container token and NM token error
Try putting federica-VirtualBox in /etc/hosts file
Oct
7
revised MultitpleOutputFormat - Hadoop
changing no to number to make more sense
Oct
7
answered Oozie custom asynchronous action
Oct
3
comment Oozie custom asynchronous action
Can you show how you have implemented the check() and initActionType() methods and how are you implementing the call() method in the Callable?
Oct
3
comment How do I control which tasks run on which hosts?
I can understand that 'w' is kind of a powerful node (used for postgres and all as you have mentioned), but I think, it won't be helpful in a performance perspective if you don't intent to use it as a datanode and still run a tasktracker there. Check if your cluster performs better if you remove the tasktracker service from there (might be a tasktracker(w) to datanode(0/1/2/3) communication issue?Not sure though). Anyways, it just my thought as to why your '1wwww' hangs and '0321w' succeeds.
Oct
3
comment Avro ClassCastException when running crunch job on hadoop 2
if you can post the MR driver code, it would help us to figure out the problem faster.
Oct
3
comment Avro ClassCastException when running crunch job on hadoop 2
I think, we don't need the '=' after the -libjar, if you are using that. One more thing I noticed, if you are using 'input' as a property key, you must have a space between -D and 'input'.
Oct
3
comment Why does NameNode HA use JournalNode instead of Zookeeper istself?
if you consider say a periodic checkpoint of 1hr produces mere 50MB of edit transactions, I guess, it's still too much for Zookeeper to store in a ZNode. In the above architecture (by what I can infer) is just to monitor the two namenodes so that when one goes down, the other can take up the duty. Zookeeper is not used for any storing purpose it seems. In the architecture, I also don't see the checkpoint node that does a FSImage and Edit log merge. Could it be that the Journal nodes' doing the duty?
Oct
3
comment Avro ClassCastException when running crunch job on hadoop 2
Interesting. It not needed but what if you push your jar with the -libjar option too?
Oct
1
comment Why does NameNode HA use JournalNode instead of Zookeeper istself?
Do you know to what size Edit logs can grow?
Sep
30
comment Instanceof correct usage in java
Anyway he one has to match up with something or the other. Keeping enum would probably be advantageous and clean. That's just my view.
Sep
30
comment Instanceof correct usage in java
You can do away with that too and instead return a String or something.
Sep
30
revised Instanceof correct usage in java
added 86 characters in body
Sep
30
answered Instanceof correct usage in java
1 2 3 4 5