Loading…
May 2-4, 2018 - Copenhagen, Denmark
Click Here For Information & Registration
View analytic
Wednesday, May 2 • 15:40 - 16:15
Scalable Monitoring Using Prometheus with Apache Spark - Diane Feddema & Zak Hassan, Red Hat Inc (Intermediate Skill Level) (Slides Attached)

Sign up or log in to save this to your schedule and see who's attending!

Tweet Share
Feedback form is now closed.
As spark applications move to a containerized environment, there are many questions about how to best configure server systems in the container world. In this talk we will demonstrate a set of tools to better monitor performance and identify optimal configuration settings. We will demonstrate how Prometheus, a project that is now part of the Cloud Native Computing Foundation (CNCF), can be applied to monitor and archive system performance data in a containerized spark environment. In our examples, we will gather spark metric output through Prometheus and present the data with Grafana dashboards. We will use our examples to demonstrate how performance can be enhanced through different tuned configuration settings. Our demo will show how to configure settings across the cluster as well as within each node.
Speakers
avatar for Diane Feddema

Diane Feddema

Principal Software Engineer, Red Hat Inc
Diane Feddema is a principal software engineer at Red Hat Inc Canada, Emerging Technologies Group. Diane is currently focused on developing and applying Data Science and Machine Learning techniques for performance analysis, automating these analyses and displaying data in novel w... Read More →
avatar for Zak Hassan

Zak Hassan

Software Engineer, Red Hat Inc
Zak Hassan is a Software Engineer on the Emerging Technology Group working on Data Science and Machine Learning. Zak previously worked as a Software Consultant in the financial services and insurance industry, building end-to-end software solutions for clients.

KubeCon Spark+Prometheus.pptx (4) pdf
Wednesday May 2, 2018 15:40 - 16:15
C1-M0
  Observability, Intermediate