A discussion of the different types of transit data and mobile application developer’s perspective on open data and transit data formats. For the raw Powerpoint with animations, see http://bit.ly/TransITech-Open-Transit-Data.
Center for Urban Transportation Research | University of South FloridaOpen Transit Data –A Developer’s PerspectiveSean J. Barbeau, Ph.D.
2Overview• Why Open Data?• Anatomy of Transit Data Sharing• Being Developer-Friendly
3WHY OPEN DATA?
4What is open data?• Transit data that is sharedwith the public– Typically shared viawebsite/FTP site/webservices• No login should be required(may use API key)– Should be updatedregularly, with any changesin schedule/routes/stops
5Open [Data Architecture Source]• Open architectures mostly focus on:– Standards within an agency’s software/hardware systems– Interconnectivity with other government systems• Open source means software source code is available• Open data is the sharing of data with external publicparties3rd partydevelopersOPEN DATATransit AgencyTransit Vehicle AVL ServerSchedule System
6Why is open data important?• Allows public to contributeservices that are cost/time-prohibitive for the public sector– e.g., many mobile platforms• Vendors are unpredictable– Some agencies have shared dataonly with Google– When Apple dropped GoogleMaps, iPhone users lost transitdirections– Apple relied on 3rd party apps tofill the gap – only possible if opendata was available
7Why is open data important(to developers)?• Developers want to createinnovative apps that meet aneed!– Some are monetized, some arenot• If you don’t provide opendata, developers will oftenimprovise– …via website scraping, etc.– Prone to breaking– Not beneficial to agency orrider
9Two Types of Open Data1. Static– e.g., Transit schedules / routes /stops– Change only a few times a year2. Real-time– e.g., Estimated arrival times/vehicle positions/servicealerts– Can change every few seconds
10Two Magnitudes of Open DataA. “Fire hose”– A dump of the complete state of thetransit system– Not directly suitable for mobile devices• Static -> All transit schedules/routes/stops• Real-time -> All estimated arrivals/vehiclepositions/service alertsB. “Faucet”– Precise subset of transit data– Suitable for mobile devices• Static -> “Stop ID 10 is served by Route 5”• Real-time -> “It is 2 minutes until Route 5bus arrives at Stop ID 10”
11Transit Data Flow ArchitectureProducerAggregator/FilterConsumerOpen Data(“Faucet”)Open Data(“Fire hose”)
12ProducerAggregator/FilterCommonly-used “fire hose” formatsOpen Data(“Firehose”)General TransitFeed Spec. (GTFS)GTFS-realtime- static - realtimeService Interface for Realtime Information (SIRI)Transit CommunicationsInterface Profiles (TCIP)ProducerAggregator/FilterConsumerGTFS/GTFS-realtime format - http://goo.gl/tmwv8SIRI format – http://goo.gl/VnpyvTCIP format - http://goo.gl/vd6kY
13Transit Data Flow ArchitectureProducerAggregator/FilterConsumerOpen Data(“Fire hose”)Open Data(“Faucet”)
14Aggregator/FilterConsumerCommon “faucet” formats still emergingOpen Data(“faucet”)Vendor/Agency-specific formatsVendor/Agency-specific formats- static - realtimeSIRI(REST/JSON format)OneBusAway APIOneBusAway APIProducerAggregator/FilterConsumerVendor/Agency formats - http://goo.gl/NtNJ0OneBusAway format - http://goo.gl/XXJyNSIRI REST format - http://goo.gl/0PctT
15Example – Google TransitBARTVehicles/ServersGoogleServersGoogle TransitMobile AppStatic - GTFSRealtime - GTFS-realtime Open to Public
16Example – HART in Tampa, FLHARTVehicles/ServersUSF ServerUSFOneBusAwayServerOneBusAway3rd Partymobile appsStatic & Real-time - OneBusAway APIStatic – GTFS (HART)Realtime - GTFS-realtime (USF) More at http://goo.gl/iqHD2
17Example – MTA BusTime in NYMTAVehiclesMTABusTimeServers3rd PartyMobileAppsStatic - OneBusAway APIReal-time - SIRI REST API for mobileStatic - GTFS
18Successful Open Data Formats Are…• Organic– Created and improved by the people actuallyproducing and consuming the data• Open– Open process for evolution– Data/documentation not hidden behind log-ins• Easy-to-use for app developers– Is documentation simple to understand?– Are there existing open-source software tools?
19General Transit Feed Specification(GTFS)• Created by TriMet and Google in 2005• Has become a de facto standard world-wide forstatic transit schedule/route/stop dataGTFS data consists of multiple text files GTFS data powers Google Transitand other apps
20General Transit Feed Specification(GTFS)• Over 500 agencies worldwide have transit data in GTFSformat– 49 of top 50 largest U.S. transit agencies share GTFS data,over 227 worldwide– At least 20 Canadian agencies share open data• Most agencies created GTFS data for Google Transit– But, GTFS is open-data format used by web/mobile apps,OpenTripPlanner, OneBusAway, etc.• See “GTFS Data Exchange” for list of agencies with GTFSdata– http://www.gtfs-data-exchange.com/– Or, ask your local agency City-Go-Round, http://www.citygoround.org/, Dec. 4, 2012 For more GTFS info and references, see paper co-authored by Sean Barbeau and Aaron Antrim – “The Many Uses of GTFS Data” - http://goo.gl/asR96
21BEING DEVELOPER-FRIENDLYPromoting app development with open data
22Create a relationship with developers• Open your GTFS data, and shareon GTFS-Data-Exchange!– GTFS data should not bepassword or login protected• Share real-time data too(national list pending)• Create a “Developer page” withaccess to resources (e.g., GTFSlicense, data)• Create developer emaillist/group forannouncements/Q&A/collaboration• Announce resources on “TransitDevelopers” groupHART Developer page -http://www.gohart.org/developers/ Transit Developers group, https://groups.google.com/forum/?fromgroups#!forum/transit-developers, Dec. 4th, 2012
25257.0212.6816.7619.379.4417.7718.6224.010510152025304 15 30 60BatteryLife(hours)Interval Between Wireless Transmissions (s)Using HTTP Increases Battery Life by 28% on Avg.JAX-RPC HTTP-POSTSOAPSOAP vs. HTTPMotorola i580 phone -See http://goo.gl/hq6nE for details
2602000400060008000100001200014000160001800020000Min. Max. Avg. 50th percentile68th percentile95th percentileStd dev.ElapsedTime(ms)JSONXMLXML vs. JSON Parsing Time –Samsung Galaxy S3• ~4.3 times longer toparse the firstresponse using XML• First response time iscritical for mobileapps, sinceapplication state isoften destroyed whenuser multitasks(checks email, etc.) ontheir phone-Using Jackson 2.1.2-Using MTA SIRI REST API StopMonitoring-See http://goo.gl/EhYSl for details
28Conclusions• Open data (e.g., GTFS) makes transit apps possible• Understand open [data vs. architecture vs. source]• Understand the differences in data:– Static vs. real-time– “Fire hose” vs. “Faucet”• Understand that certain formats are more appropriatethan others for certain situations (e.g., mobile)• Being developer-friendly encourages mobile appdevelopment!
29Thanks!Sean J. Barbeau, Ph.D.email@example.comPrincipal Mobile Software Architect for R&DCenter for Urban Transportation ResearchUniversity of South FloridaFor more GTFS info and references, see paper co-authored by Sean Barbeau and Aaron Antrim – “The ManyUses of GTFS Data” - http://goo.gl/asR96