Stream Dashboard Design

Data Visualization that Drives Decisions

timeline

August - October 2023

role

Interaction Designer

deliverables

Global and Local Dashboard

tools

Axure RP, GDocs, GSheets

context

Stream is a real-time data integration and streaming platform that enables organizations to collect, process, and deliver data in real-time, allowing for faster decision-making and improved operational efficiency.

The interaction design class at SJSU was approached by Stream to redesign the health monitoring dashboard. Due to confidentiality agreements, the company's name has been altered. However, the project was real, and the insights and outcomes described in this case study are accurate. This design has been presented to representatives of the company for feedback.

Project Summary

problem

Stream data engineers face challenges in monitoring the health of data pipelines due to a lack of a comprehensive and user-friendly solution.

process

We start by understanding user needs and their problems. This helps us define design goals that focus on building the most valuable features. By prioritizing these features, we ensure our solution effectively solves user problems.

solution

Explore the prototype →

The Problem

Thoroughly analyzing the brief and conducting in-depth Q&A sessions with data engineers.

Stream's platform lacks a comprehensive and user-friendly data pipeline health monitoring solution for production environments.

This leads to inefficiencies, potential data loss, and difficulties in troubleshooting issues for data engineers.

How do we know this?

from the given brief

There's no central system to track the health of individual applications and the overall Stream environment.

Currently, there is no single dashboard that provides a comprehensive view of the system's health, making it difficult to proactively identify and address issues. This can lead to potential data loss, performance degradation, and operational inefficiencies.

from Q&A session with data engineers

Data engineers need a more granular view of app performance.

They require information on read/write rates, lag, and resource consumption.

Monitoring multiple pipelines is crucial.

A central view of all pipelines is necessary for efficient problem identification.

Proactive alerts are essential.

Users need to be notified of critical issues or anomalies.

Drill-down capabilities are required.

The ability to zoom in on specific metrics or pipelines for deeper analysis is important.

Clear visualizations are essential.

Data should be presented in a way that is easy to understand and interpret

Who are our users?

Alex, 35

Senior Data Engineer

Responsible for overseeing the design, development, and maintenance of complex data pipelines.

Needs and Motivations

Alex wants to streamline his workflow and minimize the time spent on routine tasks. He aims to identify and address potential issues before they impact the system's performance. Alex values the ability to make informed decisions based on real-time data insights.

Sarah, 28

Data Engineer

Responsible for assisting in the design, development, and maintenance of data pipelines.

Needs and Motivations

Sarah wants to gain experience and improve her skills as a data engineer. She needs a user-friendly dashboard that provides clear instructions and explanations. Sarah appreciates having a tool that can help her identify and resolve issues independently.

Goals

What do we want to achieve with the design?

Conceptual Model

Our design goals help us narrow down the features to tackle in our conceptual model.

By understanding how users interact with objects and their features, we can prioritize the most important features of our product. This helps us create an organized and user-friendly system that meets their needs.

Object Action Matrix

Prioritization Matrix

View full conceptual model →

The Design Solution

real-time monitoring

We need to constantly track the health of our data pipelines to see how they're performing right now.

Show real-time and projected data so users can make decisions about potential bottleneck.

Provide clear indicators to system status allowing for efficient detection of issues.

performance metrics & issue detection

We need to track key numbers (KPIs) to understand how well our pipelines are working and identify areas for improvement. The system should be able to spot potential problems in the pipeline early on, so we can fix them before they cause bigger issues.

Graphs and charts to represent system health metrics in an easy-to-understand manner.

Displays detailed information about individual applications, including status, node locations, and performance metrics, allowing users to monitor multiple pipelines simultaneously.

notification system

We need a system that immediately alerts the right people when there's a problem, so we can respond quickly and minimize downtime.

Alert log to notify users of any changes in the process. Alerts use color to indicate level of priority.

All activity within a specific app is logged to ensure proper documentation of the process occurs.

Final thoughts

Designing effective dashboards can be challenging. Semantic Interaction Design (SID) helped me streamline this process. By focusing on how users interact with data, I learned to create dashboards that are not only visually appealing but also intuitive and easy to use.

SID emphasizes defining clear concepts and actions within the dashboard. This ensures the design aligns with user expectations and minimizes confusion.

To further improve this design, I would have benefited from observing how data engineers actually use the dashboard. This would have allowed me to gather valuable insights and make the dashboard even more effective for their needs.

I would like to extend my deepest gratitude to my mentor, Daniel Rosenberg for guiding me and providing feedback every step of the way.

Page updated

Google Sites

Report abuse