IT Architect, Data Scientist, ArchiDevOps, NoSQL, Hadoop, MongoDB, etc.
|
May
12 |
|
answered | Hadoop HDFS - Keep many part files or concat? |
|
May
1 |
|
awarded | Teacher |
|
Apr
25 |
|
awarded | Popular Question |
|
Mar
15 |
|
answered | Can I use 100 Mbps Network Switch for Hadoop mini cluster setup? |
|
Mar
14 |
|
comment |
Loading json with varying schema into PIG Thanks, this works. |
|
Mar
14 |
|
accepted | Loading json with varying schema into PIG |
|
Mar
13 |
|
asked | Loading json with varying schema into PIG |
|
Mar
6 |
|
awarded | Caucus |
|
Feb
26 |
|
awarded | Popular Question |
|
Feb
24 |
|
comment |
What is the most effective way to find distinct of columns in hadoop No, If you have 10000 different values for D then these will result in 10000 different keys (the exact distincts you are looking for). These 10000 can be distributed over 10000 reducer tasks. The straggler effect depend on the maximum number of different values you have for a single distinct key. If you then still have such an effect you could consider using a combiner (partially reduce within the mapper task). |
|
Feb
24 |
|
answered | What is the most effective way to find distinct of columns in hadoop |
|
Feb
13 |
|
asked | Replay a logfile at "realtime" speed? |
|
Feb
8 |
|
accepted | How to add buildpath options for eclipse from maven |
|
Feb
8 |
|
asked | How to add buildpath options for eclipse from maven |
|
Jan
30 |
|
comment |
hadoop distinct count of a field ArrayLists don't scale into Hadoop. |
|
Jan
30 |
|
answered | hadoop distinct count of a field |
|
Jan
28 |
|
comment |
Chaining multiple MapReduce jobs in Hadoop Your answer is about how to join these jobs in terms of execution. The original question was about the best datastructures. So your answer is not relevant for this specific question. |
|
Jan
18 |
|
comment |
Hadoop secondary sort for values. Sorting, gorping loose values I can't tell if you implemented everything correctly. I assume you implemented the correct compareTo method for your key and value classes? If not then it will do a byte by byte compare. |
|
Jan
17 |
|
answered | Hadoop secondary sort for values. Sorting, gorping loose values |
|
Jan
8 |
|
awarded | Good Question |
