Reconstructing transit usage from Presto records

Public transit in Toronto area can now be paid with Presto fare cards. As part of an online registration functionality, one can access their card activity history. I was curious to what extent it would be possible to use that data to automatically reconstruct trips I’ve taken. I was most interested in distance travelled and mode (bus, subway, streetcar, train).

Useful data available is:

  1. Time of tap
  2. Transit agency operating the vehicle or station tapped
  3. Location of the tap – with some limitations, sometimes inaccurate
  4. Amount paid

The data isn’t quite perfect for my purposes. There are a few problems:

  1. Taps are not required everywhere
    1. Most trips do not require tapping out when leaving, including when leaving the subway. In many cases, a 2-stop trip can look the same as a 12-stop trip.
    2. Particularly in City of Toronto there are many in-station transfers for transferring between subway and buses that do not require taps
  2. The location of the tap is sometimes wrong (perhaps due to malfunctioning GPS)
  3. Sometimes a location is recorded specifically (“Square One GO Bus Terminal”), sometimes generically (“Zone 20”)
  4. Taps can take a few hours or days to arrive in online history (particularly the bus card readers seem to be uploaded nightly), though it does come in eventually
  5. There is a “discount” field which doesn’t actually show discounts as publicized in official fare schedules and is, as far as I can tell, useless

(Presto online interface also has a “Transit Usage Report” view, but it seems to only include fare payments, and none of the free transfers. As I understand it, it’s used for claiming tax credits.)

The exports are straightforward enough: it’s a simple CSV file. It is in reverse chronological order (latest taps first), but that’s easy to reverse.

Given the data, here’s what’s possible:

  1. Calculating the minimum number of trips taken, by only counting trips that had a fare charged
  2. Calculating the minimum number of trips involving the subway, by only counting entering subways with no in-station bus or streetcar transfers (these are mostly downtown Toronto)
  3. Estimating, with fairly high probability, at least one of modes of transport (bus, streetcar, subway, train) involved in a trip
  4. Estimating, with better than chance probability but not close to perfect, the number of logical “trips” taken and some of the modes involved

Here are some less obvious samples of actual data, commented to note what I was actually doing. I switched the order to earlier taps first for easier reading.

"Date","Transit Agency","Location","Type ","Service Class","Discount","Amount","Balance"
"11/2/2018 11:31:00 AM","Go Transit","Exhibition GO Station Rail","Fare Payment","","$0.00","$5.30","$24.70"
"11/2/2018 11:44:00 AM","Go Transit","Union Station Rail","Fare Payment","","$0.59","($0.59)","$25.29"
"11/2/2018 11:48:05 AM","Toronto Transit Commission","UNION STATION","Fare Payment","Regular","$0.00","$1.50","$23.79"

A trip from Exhibition GO station, on commuter rail, going one stop, then transferring to subway. I ultimately exited the subway after 4 stops at College.

"11/2/2018 2:52:43 PM","Toronto Transit Commission","15639","Fare Payment","Regular","$0.00","$3.00","$20.79"

This is a streetcar trip. The 15639 is a stop code (King Street at Bay Street), specified on TTC website (for some numbered stops it might be necessary to switch to “All stops view”), I think it also appears in published GTFS data. Most TTC stops have names specified, but a few are code only.

For reconstruction purposes, only the westbound 504 line is scheduled to serve this stop – normally with streetcars, but can be replaced with buses during weekend construction, which would not appear in data.

"11/4/2018 9:08:43 AM","Toronto Transit Commission","King St West At Shaw St","Fare Payment","Regular","$0.00","$3.00","$17.79"
"11/4/2018 9:20:04 AM","Toronto Transit Commission","ST ANDREW STATION","Transfer","Regular","$0.00","$0.00","$0.00"

A streetcar trip, transferring to subway. I ultimately exited the subway at Yorkdale which is about 10 km away. The streetcar trip is possible to guess, the length of the subway trip is unknown.

"11/4/2018 6:57:23 PM","Toronto Transit Commission","0","Fare Payment","Regular","$0.00","$3.00","$14.79"
"11/4/2018 7:23:25 PM","Toronto Transit Commission","15641","Transfer","Regular","$0.00","$0.00","$0.00"

Entered the subway at Yorkdale (for whatever reason it showed as “0” for location) then transferred to streetcar at stop 15641.

"11/6/2018 5:47:34 PM","Toronto Transit Commission","Atlantic Ave At King St West","Fare Payment","Regular","$0.00","$3.00","$35.79"

This was a bus and subway trip. I actually boarded the bus one stop further (laggy GPS), at a stop served by both bus and streetcar. I then transferred to subway using tap-less transfers which are available at some subway stations, and rode the subway for about 10 km. Unfortunately none of this is trackable from the card data – it would have looked the same if I exited a bus after a kilometre.

"Date","Transit Agency","Location","Type ","Service Class","Discount","Amount","Balance"
"11/7/2018 9:20:00 AM","Go Transit","Exhibition GO Station Rail","Fare Payment","","$0.00","$5.30","$27.49"
"11/7/2018 9:20:00 AM","Go Transit","Exhibition GO Station Rail","Fare Payment","","$0.00","($5.30)","$32.79"
"11/7/2018 9:20:00 AM","Go Transit","Exhibition GO Station Rail","Fare Payment","","$0.00","$5.30","$27.49"
"11/7/2018 10:22:00 AM","Go Transit","Burlington GO Station Rail","Fare Payment","","$1.26","$4.69","$22.80"

Commuter rail trip from Exhibition to Burlington. I cancelled the first tap because I thought it went wrong. The sequence here is tap, cancel tap, tap, then tap off at Burlington.

"11/18/2018 9:39:00 AM","Go Transit","Exhibition GO Station Rail","Fare Payment","","$0.00","$5.30","$26.50"
"11/18/2018 10:11:00 AM","Go Transit","Port Credit GO Station Rail","Fare Payment","","$0.77","$0.88","$25.62"

Finally a normal trip with well-defined beginning and end. The GO regional/commuter transit system requires tap-out, so distances can be estimated fairly accurately and mode of transport can be determined. Fare was $6.18 – I have no idea what the $0.77 “discount” is supposed to mean.

"11/23/2018 12:36:50 PM","Toronto Transit Commission","Dufferin St At King St West","Fare Payment","Regular","$0.00","$3.00","$22.62"
"11/23/2018 1:50:06 PM","Toronto Transit Commission","Caledonia Rd At Cartwright Ave","Transfer","Regular","$0.00","$0.00","$0.00"
"11/23/2018 2:43:59 PM","Toronto Transit Commission","King St West At Spencer Ave","Fare Payment","Regular","$0.00","$3.00","$19.62"

This is round-trip bus journey, about 10 km there and 10 km back. The tap at 1:50 pm is the beginning of journey back. The tap at 2:43 pm is subsequent transfer to a streetcar for a few stops. The transfer between second bus and streetcar is indirect and I had to walk a bit to make this connection. Note that fare charging doesn’t line up with logical trip organization due to allowed two-hour free transfer.

"11/24/2018 3:38:15 PM","Toronto Transit Commission","0","Fare Payment","Regular","$0.00","$3.00","$16.62"

This was actually a streetcar trip, boarding at King Street at Spadina Avenue. This is impossible to tell from the data, which has only unrecognized location and shows the same as the trip starting at Yorkdale subway on November 4th.

"11/29/2018 11:59:48 AM","Toronto Transit Commission","ST ANDREW STATION","Fare Payment","Regular","$0.00","$3.00","$13.62"

Subway-only trip, for about 15 km. I changed subway lines at Sheppard-Yonge (no tap), exited at Leslie Station (no tap).

"11/29/2018 3:14:05 PM","Toronto Transit Commission","LESLIE STATION","Fare Payment","Regular","$0.00","$3.00","$10.62"
"11/29/2018 3:55:03 PM","Toronto Transit Commission","15637","Transfer","Regular","$0.00","$0.00","$0.00"

Subway trip then streetcar. Stop number 15637 is King Street at Yonge Street, so with high probability I was on subway from Leslie station to King station. However technically it would also be possible to get a very similar tap history by exiting the subway at St Clair, in-station transfer to a streetcar, riding it to St Clair West, in-station transferring back to subway, then riding from St Clair West to King. So it’s all probabilities (less or more likely as indicated also by tap time stamps).

"12/4/2018 6:52:35 PM","Toronto Transit Commission","King St West At Shaw St","Fare Payment","Regular","$0.00","$3.00","$31.62"
"12/4/2018 8:19:57 PM","Toronto Transit Commission","Distillery Loop","Transfer","Regular","$0.00","$0.00","$0.00"

This was two actual trips on streetcars, first to Distillery Loop and then back to Shaw Street, but they were charged as one due to a 2-hour transfer policy. I can’t come up with a feasible way to reliably compute these actual trips.

"Date","Transit Agency","Location","Type ","Service Class","Discount","Amount","Balance"
"12/8/2018 11:35:39 AM","Toronto Transit Commission","King St West At Niagara St","Fare Payment","Regular","$0.00","$3.00","$28.62"
"12/8/2018 11:57:00 AM","Go Transit","Toronto - UnionStationBusTerminal At Ba","Fare Payment","","$0.00","$5.30","$23.32"
"12/8/2018 12:47:00 PM","Go Transit","Mississauga - SquareOneGOBusTerminal At","Fare Payment","","$0.77","($0.62)","$23.94"
"12/8/2018 1:06:00 PM","Go Transit","Zone 20","Fare Payment","","$0.00","$5.30","$18.64"
"12/8/2018 3:11:00 PM","Go Transit","Waterloo - UniversityofWaterloo At","Fare Payment","","$1.15","$3.85","$14.79"

This was a trip on city bus replacing streetcar (impossible to tell from data), then regional bus, transferring at Square One (which is GO fare zone 20), then finishing the bus journey in Waterloo.

"12/8/2018 7:04:00 PM","Go Transit","Zone 27","Fare Payment","","$0.00","$5.30","$9.49"
"12/8/2018 8:35:00 PM","Go Transit","Mississauga - SquareOneGOBusTerminal At","Load Amount","","$0.00","$30.00","$39.49"
"12/8/2018 8:35:00 PM","Go Transit","Mississauga - SquareOneGOBusTerminal At","Fare Payment","","$1.75","$8.55","$30.94"
"12/8/2018 8:58:00 PM","Go Transit","Mississauga - SquareOneGOBusTerminal At","Fare Payment","","$0.00","$5.30","$25.64"
"12/8/2018 9:37:00 PM","Go Transit","Toronto - UnionStationBusTerminal At Ba","Fare Payment","","$0.18","($3.83)","$29.47"
"12/8/2018 9:48:18 PM","Toronto Transit Commission","15637","Fare Payment","Regular","$0.00","$1.50","$27.97"

This was a trip on bus, from Kitchener bus terminal (which is GO fare zone 27), transferring at Square One (with automatic top-up, configured on this card to keep balance above $6), going to Toronto Union Station on regional bus, then a bus replacing streetcar from King and Yonge.

"12/13/2018 7:15:41 PM","Toronto Transit Commission","Leslie St At Commissioners St North Side","Fare Payment","Regular","$0.00","$3.00","$24.97"

This was a streetcar trip, from King Street at Shaw Street to Broadview station. It appears the streetcar had a malfunctioning GPS or GPS-to-Presto link.

"12/13/2018 10:54:22 PM","Toronto Transit Commission","BROADVIEW STATION","Fare Payment","Regular","$0.00","$3.00","$21.97"
"12/13/2018 11:24:56 PM","Toronto Transit Commission","Leslie St At Lake Shore Blvd East","Transfer","Regular","$0.00","$0.00","$0.00"

This was a streetcar trip, from the Broadview subway station streetcar loop (access controlled as part of the subway station); the streetcar short-turned at Church Street; we got on another one at King and Church – it also had a malfunctioning GPS as we were nowhere near Leslie Street. This is basically impossible to reconstruct accurately from the data given.

"Date","Transit Agency","Location","Type ","Service Class","Discount","Amount","Balance"
"12/15/2018 2:34:00 PM","Mississauga Transit","1-City Centre Transit Terminal","Fare Payment","","$0.00","$3.00","$12.79"
"12/15/2018 3:06:00 PM","Go Transit","Port Credit GO Station Rail","Fare Payment","","$0.00","$5.30","$7.49"
"12/15/2018 3:31:00 PM","Go Transit","Exhibition GO Station Rail","Load Amount","","$0.00","$30.00","$37.49"
"12/15/2018 3:31:00 PM","Go Transit","Exhibition GO Station Rail","Fare Payment","","$0.77","($1.32)","$38.81"

This was a trip from Square One to Exhibition GO on Mississauga bus (from Square One to Port Credit) and regional GO train (from Port Credit to Exhibition). Oddly enough, auto top-up activated on the tap-out, despite the tap-out actually increasing my balance.

Fare from Port Credit to Exhibition is $6.18, but when transferring from Mississauga Transit to GO, there is a $2.20 discount. Officially the discount is on the Mississauga fare, so that it should be $0.80 and the GO fare is unchanged, but the card records it the other way, since the card can’t know that the discount should apply before I do the GO trip. GO fare is then recorded as $3.98 instead.

"12/21/2018 11:18:34 PM","Union Pearson Express","Union Pearson Express Pearson Station","Fare Payment","","$0.00","$9.25","$26.56"
"12/21/2018 11:29:53 PM","Union Pearson Express","Union Pearson Express Bloor Station","Fare Inspection","","$0.00","$0.00","$0.00"
"12/21/2018 11:45:25 PM","Union Pearson Express","Union Pearson Express Bloor Station","Fare Payment","","$0.00","($4.23)","$30.79"
"12/21/2018 11:49:21 PM","Toronto Transit Commission","DUNDAS WEST STATION","Fare Payment","Regular","$0.00","$1.50","$29.29"

Trip from Pearson Airport on the UP Express train, including a card inspection. Transferring to TTC at Dundas West station, with $1.50 transfer discount. I continued on a streetcar, though this is not visible from data – I could have taken the subway and in-station transfer bus almost anywhere in Toronto.

"12/27/2018 1:16:43 PM","Toronto Transit Commission","15645","Fare Payment","Regular","$0.00","$3.00","$7.70"
"12/27/2018 1:22:15 PM","Toronto Transit Commission","King St West At Bathurst St","Transfer","Regular","$0.00","$0.00","$0.00"

Trip on King streetcar boarding at Peter Street, followed by a transfer to another King streetcar as the first one went out of service at Bathurst. The data might have looked much the same if I went in the opposite direction instead, as on December 4th.