This is the 1 Bus in Boston.

NextBus predictions in pictures

The series of histograms in this section, based on NextBus predictions collected for bus trips between April 4th, 2014 and May 7th, 2014, illustrate a few of the most prominent empirical trends between bus predictions and actual arrival times. Histograms are truncated to show the most common range of errors. In each histogram, “error” represents the difference between a bus’s predicted arrival time and its actual arrival time rounded down to the nearest integer. For example, consider a NextBus prediction of five minutes. An error of zero indicates that the bus arrived as expected, five to six minutes after the original prediction. A error of one minute, for contrast, indicates that the bus was one minute late relative to the original prediction and actually arrived some time between six and seven minutes.

The first set of histograms, shown above, tracks the performance of one-minute predictions for six key stops along the 1 Bus’s southbound route from Cambridge to Boston. For the route’s originating stop at Holyoke Gate, over 90% of one-minute predictions were correct. Given that the MBTA often stages its buses at originating stops a few minutes prior to departures, this high accuracy rate is not surprising. However, NextBus has more difficulty further down the route, and the error rates increase for all other stops. Prediction accuracy reaches a minimum of just over 50% at Newbury St. (perhaps due to greater uncertainty about traffic flows near the Mass. Ave. bridge and on- and off-ramps for the Mass Turnpike), and then improves slightly for the remainder of the 1 Bus’s trip toward Dudley.

The potential impact of less predictable traffic flows on NextBus’s predictions at the middle of the 1 Bus’s route suggest two ideas to explore. First, the magnitude of NextBus’s prediction errors may depend on actual prediction times as long predictions are subject to more unexpected changes in traffic than short predictions. Second, and related to the previous point, as a bus travels along its route, predictions for stops that are further in the route are subject to greater uncertainty due to longer time windows for a bus to accumulate or make up delay (compare the histogram above at the 1 Bus’s originating stop versus its Newbury St. histogram). Consequently, longer predictions and predictions for stops further along a bus route may exhibit larger errors than either shorter predictions or predictions for stops near the beginning of a route. The next set of charts explores these potential relationships between accuracy, prediction length, and distance from the originating stop. The charts shown below display the median error for each stop and prediction, grouped into two-minute bins. NextBus predictions ranging from 0 to 119 seconds were placed in the first bin; predictions ranging from 120 seconds to 239 seconds were placed in the second bin; and so forth. For the analysis month, each prediction bin contains between 2000 to 4000 predictions.

As the 1 Bus travels south from Harvard, NextBus’s prediction performance noticeably worsens with increasing prediction time and distance from the originating stop. At the originating stop, predictions and median errors show that 1 Bus tends to leave on schedule, and predictions of all lengths are quite accurate. However, for stops at Central Square, MIT, Newbury St., Mass. Ave. Station, and Dudley, longer prediction times from NextBus consistently exhibit larger errors. For the same stop, shorter prediction times are correct more often than longer prediction times. Also, errors tend to occur in only one direction. The positive median errors suggest that when predictions are wrong, buses are more likely to arrive later than predicted, not earlier.

In addition to the correlation between increasing prediction times and increasing prediction errors for each stop, the charts above also reveal a correlation exists between the distance of a stop from the originating stop and the magnitude of NextBus’s prediction errors. The differences between predictions for Central Square, which appears relatively early in the 1 Bus’s southbound route, and for Dudley, the southbound route’s terminating stop, illustrate this trend. For both stops, predictions of zero or one minute tend to be accurate. However, looking at a prediction of 20 minutes for Central and Dudley, the prediction for Central has a relatively small median error of approximately 1 minute, while the prediction for Dudley has a relatively large median error of approximately 4 minutes. The farther the bus stop along the route, the more likely NextBus’s predictions are to exhibit larger errors.

Practically speaking, regular bus riders are likely to be aware of these patterns. People that board at Harvard quickly learn that buses generally depart exactly as predicted, while riders across the river might expect to have a few more minutes to run and catch a bus. The rarity of early bus arrivals leads to a simple heuristic for “optimizing” bus wait times that finds its roots in common sense: determine how long it takes to walk to the nearest bus stop and then head out the door with just enough time to catch the next bus. Probabilistically, this “greedy” approach works wonderfully for maximizing sleep time in the morning for risk-neutral bus riders. Unfortunately, as the first set of histograms clearly show, every so often the bus arrives just ever-so-slightly earlier than expected. The devil is in the tail events.

Time-of-day considerations

The previous analysis aggregated all predictions for one month, lumping predictions from different hours, weekdays, and weekends together. However, given the regularity of traffic patterns, does NextBus tend to give more accurate predictions during some parts of the day and less accurate predictions during others? For example, do predictions tend to be worse during busy commute times in the morning and afternoon? To explore how NextBus performs throughout the day, the graphs below show NextBus’s errors for a range of predictions during weekdays, broken down by hour. As the MBTA does not operate the 1 Bus between 2AM and 5AM, those hours have no corresponding graphs. The predictions shown below are for the 1 Bus’s southbound stop at Newbury St; on average, most of the one-minute prediction bins used to create these graphs had between 30 to 100 prediction samples over the analysis month.

A few interesting patterns appear in the series of charts above that support the idea that NextBus’s prediction accuracy also depends on the time of day. As noted before, short predictions—predictions between zero and two minutes—remain reasonably accurate throughout the day. In the morning, from 5AM to 7AM, buses tend to arrive earlier than predicted. During the morning rush hour, from 8AM to 9AM, longer predictions are less likely to be accurate. This trend is more noticeable in the early afternoon, between 2PM and 6PM, when predictions are most unreliable and median errors increase to as much as 10 minutes. As the evening continues, NextBus’s prediction accuracy slowly improves. These hourly differences reveal quite a bit of useful detail about NextBus’s predictions for commuters: buses tend to arrive earlier than predicted in the early morning and later than predicted during traditionally busy traffic times. Not all predictions, even for the same stop and the same relative arrival time, are equally reliable.

Thus far, the charts shown have explored three independent ideas: first, that longer predictions are positively correlated with larger errors; second, that longer distances from the originating stop are positively correlated with larger errors; and third, that predictions exhibit larger errors during busy traffic times. The series of charts below explores these three factors in combination by taking the previous hour-by-hour breakdown of predictions for Newbury St. and superimposing predictions for the remaining five stops of interest. From this series, the three trends are striking. Predictions for the originating stop at Harvard are consistently better than other stops, while predictions for Dudley, the farthest stop, frequently exhibit the largest errors. Short predictions for all stops are better than long predictions, and prediction errors are largest during busy traffic periods in the afternoon and early evening. In aggregate, these trends suggest that NextBus may be able to correct at least some of these systematic offsets with modifications to their prediction algorithm.