We’re always happy to hear from our passengers, transit enthusiasts, and other transit agencies through the contact form on the MBTA Back on Track Dashboard. One frequent question is, “The dashboard looks so nice! How did you make it?” First, thank you! Creating the dashboard was a group effort requiring a number of moving pieces. Our main goal was to present data in an accessible way to a larger audience—in other words, we wanted to make our data easy to understand for people who may not be transit gurus. At times, this required a bit of creative and collaborative thinking. Let us show you a behind-the-scenes look at all the steps that went into creating our dashboard.
What Do You Need to Build a Dashboard?
We are sometimes asked if we used a vendor for the dashboard. While we hired vendors for various distinct pieces of the project (for example, a graphic design firm created the look & feel), there is no one vendor or software package that handles all aspects of dashboard creation. Each of the following steps were handled by a different team of experts, including in-house staff, contractors, and vendors.
Data Sourcing
The dashboard is possible only because of the prior existence and maintenance of robust data feeds. No new data sources were created specifically for the purpose of the dashboard. Rather, we piggyback on existing data feeds that had been set up for other purposes, primarily operational. For example, our subway reliability data rely on information from our MBTA performance database and ODx model.
Data Analysis & Metric Development
In-house data analysts designed and developed high-level metrics to summarize the key takeaways for each data set. Our challenge was to create metrics that were easy to understand by turning large amounts of raw data into one meaningful number or chart that could be:
• Representative of something concrete—a real world example that is easy to explain and understand, even if you’re not a transit professional
• Recalculated on the fly by the software as new data comes in
• Sensitive enough to respond to day-to-day or month-to-month fluctuations
• Specific—but not misleadingly so
• As accurate as possible
Many of the metrics the MBTA already reports in other places. For example, we report ridership to the National Transit Database and the reliability metrics were developed as part of our Service Delivery Policy (link) and are reported in our annual performance report Tracker (link).
For more information about how we calculated our metrics, see our posts on Subway Reliability, Ridership, and Customer Satisfaction.
Data Management
Data analysis can be done with a static dataset, but an automated dashboard requires datasets that update themselves with as little day-to-day human intervention as possible.
The multiple data sources that power the dashboard live in all different places and come in all different formats. The data that eventually makes it into the dashboard may come from a database, an API, or a document (e.g. a CSV). The data management team collects all of these sources into a single centralized SQL database for use by the dashboard. In addition to melding different data formats, the data pulls also require a certain amount of interpretation and aggregation to improve performance and remove personally identifiable information.
Graphic Design
The look & feel of the dashboard is a huge part of its success. We were lucky to have the guidance of an excellent graphic design firm who created a fun, fresh, eye-catching visual language, style guide, and color palette on some specific constraints from our end (for example, using bold shades of red, orange, blue, or green could confuse people into thinking we’re referring to one of the subway lines.) Our designers also created our explanatory infographics and helped us to refine our data visualizations in a collaborative and iterative process.
If you’re in the process of developing visuals for a dashboard, we recommend revisiting the following questions as often as possible:
• What story is your data trying to tell? What is the key takeaway that your audience should receive from this visualization? Different answers point toward different types of charts or images (relative size, progress toward a goal, change over time, etc.).
• How can you simplify this visualization? (Data analysts are always tempted to show too much at once!)
• How are principles of visual design working for or against this visualization?
A dashboard poses an additional challenge to the last question. Because our data is automated, we don’t necessarily have the luxury of reworking images and charts until they are visually pleasing, and we can’t control all the little details. A stacked bar chart might look great if all the series are approximately the same size, but it may begin to look awkward if one series becomes a lot larger or smaller. In an automated dashboard, it’s important to test visualizations with a variety of data conditions to ensure that the chart is still legible (and pretty!) no matter what data comes in. The main visual challenge of the dashboard is making an automated chart look intentionally designed.
Web Development
You have data; you have a design spec. The final step is to bring these together in implementation.
At this point, it is possible to use a business intelligence tool, such as Tableau or Birst, to quickly create pre-designed visualizations. However, doing so would impose design limitations; we would not have been able to fully implement our unique design. We opted to develop the tool in-house.
Our IT team, responsible for the MBTA website, was able to set aside and support server space for our tool. With the server space and all the above pieces in place, the initial creation of the dashboard took one full-time software developer about 3 weeks of solid coding plus an additional couple of weeks of QA and iterative revision.
The site uses a PHP backend which queries the database created in the “Data Management” step above and outputs a JSON API containing all the information needed for the frontend. The frontend was created in Angular.js, a Javascript framework, using the Highcharts charting framework for most of the charts and graphs.
Maintenance and Improvement
Data management is the biggest ongoing lift for maintenance of the dashboard. The ability to fully automate the dashboard greatly depends on the nature of the original data sources. Because ‘;Reliability’; relies on data sources that are already automated, we’re able to update it daily. Other data sources are uploaded manually after cleaning and checking, such as our financial data which is provided by our finance department to our data management team on a monthly basis.
The dashboard software itself rarely requires maintenance, but we are always planning improvements. For example, on October 10, we released a minor update which significantly improves the data download page, offering new data sets and improved speed and performance. We’re planning future updates that will add new metrics and features, so stay tuned!