Building data pipelines and analytical systems at massive scale. My experience lies in distributed systems, focusing on data driven large-scale systems (10.000+ nodes).
For the highly concurrent world my choice of development environment is Erlang(BEAM) and Clojure (JVM). Using functional languages that supports thousands of lightweight threads communicating with message passing and having inverted concurrency control enables low latency and high throughput with thread safe software.
Storing data at scale has been an interesting subject to me, I am familiar with the RCFile whitepaper and the more recent publication about ORC and Parquet. I have been using columnar stores beside the classical row oriented stores (SQL servers) and key-value stores (Riak, Couchbase).
Analysis of large datasets is sometimes challenging. Using caching and sampling and few other techniques makes it possible to query these sets. I am familiar with few query engines (Hive, PrestoDB, Tez).
Q&A for system and network administrators
Q&A for professional and enthusiast programmers
Q&A for users of Linux, FreeBSD and other Un*x-like operating systems
Q&A for information security professionals
Q&A for people interested in statistics, machine learning, data analysis, data mining, and data visualization
Q&A for power users of Apple hardware and software
Q&A for computer enthusiasts and power users
Q&A for peer programmer code reviews
Q&A for programming puzzle enthusiasts and code golfers
Q&A for theoretical computer scientists and researchers in related fields
Q&A for linguists, etymologists, and serious English language enthusiasts
Q&A for musicians, students, and enthusiasts
Q&A for professional programmers interested in conceptual questions about software development
Q&A for database professionals who wish to improve their database skills and learn from others in the community
Q&A for apps, scripts, and development with the Stack Exchange API
Q&A for professional, enthusiast and amateur photographers
Q&A for pro webmasters
Q&A for power users of web applications
Q&A for meta-discussion of the Stack Exchange family of Q&A websites
Q&A for electronics and electrical engineering professionals, students, and enthusiasts
Q&A for Data science professionals, Machine Learning specialists, and those interested in learning more about the field
Q&A for people studying math at any level and professionals in related fields