On May 16th, Conversant hosted the Apache Kafka DC Meetup at our Ashburn, VA office, with the topic “High Speed Data Visualization - Kafka meets Elastic Search”. The presenter, Ryan Jepsen is a member of Conversant’ s Engineering team that utilizes data pipelines and intra-day data processing through Kafka, Spark, and various Hadoop ecosystem components. A strong Spark & Scala enthusiast, Ryan is passionate about data-driven application development and learning the latest and greatest technology to engineer lightning-fast and fault tolerant data pipelines that support real-time decision making and analytics.
The presentation included a live demo and a working session on high speed, near real-time data visualization using Kafka, Spark Streaming, Elastic Search, and Kibana. Ryan used Jupyter notebook to show the ease and flexibility of interactive stream development and prototyping.
The source code used in the presentation is available on GitHub.