Eran Medan

United States

linkedin.com/in/eranmedan

Age: 37

Tech Lead at NICE Actimize

  • Java, Scala, JavaScript, and all that is web
  • Data mining and fraud detection
May
20
awarded Popular Question
May
20
comment How can I calculate exact median with Apache Spark?
p.s. I think that there are faster algorithms for finding median that don't require full sorting (en.wikipedia.org/wiki/Selection_algorithm)
May
20
awarded Notable Question
May
18
accepted When joining vertices, am I forced to use MEMORY_ONLY caching?
May
16
accepted Where does the Rust 1.0 Mac OS X pkg installer install to?
May
16
asked Where does the Rust 1.0 Mac OS X pkg installer install to?
May
14
answered When joining vertices, am I forced to use MEMORY_ONLY caching?
May
13
asked When joining vertices, am I forced to use MEMORY_ONLY caching?
May
12
awarded Famous Question
May
12
comment Serialization and Custom Spark RDD Class
Actually, to be accurate it's Original Post / Original Poster (source: en.wikipedia.org/wiki/Internet_forum#Post)
May
11
comment Performance profiling and tuning in Apache Spark
My main issue is that I don't know if I'm over caching or under caching. I would like to know if there is a tool that does spark profiling. I think I covered the usual suspects, but still think it's too slow. My question is beyond the usual suspects, just like a standalone JVM has JProfiler etc, what is the equivalent profiling tool for a distributed system such as Spark?
May
10
awarded Supporter
May
10
awarded Autobiographer
May
7
awarded Student
May
7
asked Performance profiling and tuning in Apache Spark
May
7
awarded Autobiographer
May
6
comment Algorithm to find whether there is a path (any path) above length X between two vertices
How about using Eppstein's k-shortest path algorithm? static.aminer.org/pdf/PDF/001/059/121/… sourceforge.net/p/mulavito/git/ci/… It seems to support "minimum total length." for a path, which means I can ask for k=1 paths, with minimum total length X. (it allows cycles, but I think it's ok for me)
May
6
comment Algorithm to find whether there is a path (any path) above length X between two vertices
How about using Eppstein's k-shortest paths? I read that it allows for a minimal path length, and we can have K be set to 1, will that be effective? cs.stackexchange.com/questions/18849/…
May
6
comment Algorithm to find whether there is a path (any path) above length X between two vertices
What I'm currently doing is using KShortestPaths (a JGraphT class that provides the K shortest paths between two vertices, ranked by distance) - what I'm currently doing is getting it (K limited to some low value to avoid running forever). And if I find any path there above length X, I return true. I wonder what is a good way to modify it, so that it will stop once it finds a path above such a length. but I guess this is more of a stackoverflow question...
May
6
asked Algorithm to find whether there is a path (any path) above length X between two vertices
1 2 3 4 5