Software developer mainly working with Java and associated projects: Hadoop, Oozie, Tomcat, JAXRS, Spring, Maven
|
3m
|
|
answered | Hadoop pseudo distributed mode - Datanode and tasktracker not starting |
|
9h
|
|
comment |
Hadoop pseudo distributed mode - Datanode and tasktracker not starting Can you post the contents of your /etc/hosts file
|
|
12h
|
|
revised |
Processing syslog output to csv with python Formatted log |
|
12h
|
|
comment |
Can Hadoop's Shuffle/Sort (or partition) phase be customized to perform a graph traversal? Sure, just chain multiple jobs together |
|
15h
|
|
comment |
Can Hadoop's Shuffle/Sort (or partition) phase be customized to perform a graph traversal? @JimPrahn the partitioner cannot seek, it's presented a K,V pair and asked which reducer it should be sent to. You could retain state in either the mapper or the paritioner, but you'll still have problems if the state exceeds the available memory, or you're in more than a single mapper |
|
1d
|
|
comment |
Hadoop: cannot set default FileSystem as HDFS in core-site.xml Try dumping the value of conf.get('fs.default.name') to sysout and seeing
|
|
1d
|
|
comment |
Hadoop: cannot set default FileSystem as HDFS in core-site.xml Yes but if you're launching your app with java myjar.jar ... rather than hadoop jar myjar.jar ... then the core-site.xml may not be on the classpath, and hence won't be loaded. Try and output the value of fs.default.name in your code after you create the conf object.
|
|
1d
|
|
comment |
Hadoop: cannot set default FileSystem as HDFS in core-site.xml How are you running / launching your job? Can you post more code? |
|
1d
|
|
answered | HDFS command line put throwing an exception |
|
1d
|
|
comment |
Can Hadoop's Shuffle/Sort (or partition) phase be customized to perform a graph traversal? Well you could create a custom partitioner that compares the two vertex IDs and return the hash of lesser of the two, but then you won't get both edges if you have [1,3] and [3,4] for example. There really isn't a way around this other than emitting the edges twice |
|
1d
|
|
comment |
Processing of heterogeneous documents XML in Hadoop Do the two (threee, four?) different document types have their types encoded into the filename (doc1.lib.xml and doc2.doc.xml), or are they intermixed in the same xml document? |
|
1d
|
|
comment |
Watching the value of function in an AngularJS directive +1, but for reference i had to return scope.editable()
|
|
1d
|
|
accepted | Watching the value of function in an AngularJS directive |
|
1d
|
|
asked | Watching the value of function in an AngularJS directive |
|
1d
|
|
answered | Can Hadoop's Shuffle/Sort (or partition) phase be customized to perform a graph traversal? |
|
1d
|
|
comment |
Processing of heterogeneous documents XML in Hadoop for doc1 and doc2 is lib and doc the XML document root element, or are there multiple instances of these elements that make up the document? |
|
1d
|
|
comment |
Hadoop JobRunner do not launch job Check the logs for the name node and data node(s) for any exception messages |
|
1d
|
|
comment |
how to start sort and reduce in hadoop before shuffle completes for all mappers? Well the sort and reduce can't start until all mapper input are in |
|
2d
|
|
answered | how to start sort and reduce in hadoop before shuffle completes for all mappers? |
|
2d
|
|
answered | Running jar file map reduce without Hdfs |
