Loading…
May 2-4, 2018 - Copenhagen, Denmark
Click Here For Information & Registration
Back To Schedule
Wednesday, May 2 • 15:40 - 16:15
Scalable Monitoring Using Prometheus with Apache Spark - Diane Feddema & Zak Hassan, Red Hat Inc (Intermediate Skill Level) (Slides Attached)

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Feedback form is now closed.
As spark applications move to a containerized environment, there are many questions about how to best configure server systems in the container world. In this talk we will demonstrate a set of tools to better monitor performance and identify optimal configuration settings. We will demonstrate how Prometheus, a project that is now part of the Cloud Native Computing Foundation (CNCF), can be applied to monitor and archive system performance data in a containerized spark environment. In our examples, we will gather spark metric output through Prometheus and present the data with Grafana dashboards. We will use our examples to demonstrate how performance can be enhanced through different tuned configuration settings. Our demo will show how to configure settings across the cluster as well as within each node.
Speakers
avatar for Diane Feddema

Diane Feddema

Principal Software Engineer, Red Hat
Diane Feddema is a principal software engineer at Red Hat Inc Canada, in the AI Center of Excellence. Diane is currently focused on developing and applying machine learning techniques for performance analysis using hardware accelerators, automating these analyses and displaying data... Read More →
avatar for Zak Hassan

Zak Hassan

Senior Software Engineer - AI/ML CoE, CTO Office, Red Hat Inc.
Currently focused on developing analytics platform on OpenShift and leveraging Open Source ML Frameworks: Apache Spark, Tensorflow and more. Designing high performance and scalable ML platform that exposes metrics through cloud-native technology: Prometheus and Kubernetes.

KubeCon Spark+Prometheus.pptx (4) pdf
Wednesday May 2, 2018 15:40 - 16:15
C1-M0
  Observability, Intermediate