Mike Pone

United States

Age: 38

Big Data Engineer working in Spark, Pig, HBASE on the Cloudera stack. I also have experience in Enterprise Integration using OSGI, Apache Camel and Apache Active MQ. In my spare time, I chase my 5 kids around the house and take evening classes towards a Master's in Software Engineering.

Mar
3
reviewed Approve suggested edit on Access 2010 linked ODBC table missing column issue
Mar
3
reviewed Reject suggested edit on How do you compare two version Strings in Java?
Mar
3
reviewed Approve suggested edit on Result code 0 in POST in Swagger UI
Feb
9
reviewed Approve suggested edit on i'm confused about django statice files during deployment
Feb
9
awarded Famous Question
Feb
8
accepted How to load multiline JSON in spark with Java
Feb
4
reviewed Approve suggested edit on I have written a program to login into yahoomail. but its giving me an error
Feb
4
reviewed Approve suggested edit on Difference between ClassLoader.getSystemResourceAsStream and getClass().getResourceAsStream()
Feb
3
answered How to load multiline JSON in spark with Java
Feb
3
comment How to load multiline JSON in spark with Java
This requires fiddling with the original source which I was hoping not to have to do.
Feb
2
revised How to load multiline JSON in spark with Java
added example
Feb
2
comment How to load multiline JSON in spark with Java
@Dennis your definition of multiline is accurate. There are more than one json objects in this file
Feb
2
comment How to load multiline JSON in spark with Java
I have seen that if you have multiline JSON files, you will instead have to load the whole file and then parse each file. Does that mean parsing the file as JSON using some library and then getting those JSON records into an RDD somehow?
Feb
2
asked How to load multiline JSON in spark with Java
Dec
30
awarded Yearling
Dec
13
awarded Famous Question
Dec
1
accepted How to create RDD from within Task?
Dec
1
comment How to create RDD from within Task?
That answers my original question well, now a followup question. I am using JavaSparkContext.wholeTextFiles to read individual files from a directory. I need each of those files as their own JavaRDD<STring> to feed into spark ml algorithms, but wholeTextFiles returns JavaPairRDD<String, String> of (filename, content) and I want JavaPairRDD<String, JavaRDD<String>>. I thought it would be super easy to just create a new RDD in a map function, but alas, no... Any ideas on how to approach this?
Dec
1
asked How to create RDD from within Task?
Nov
23
awarded Constituent
1 2 3 4 5