TFL (Actually Not) Off with API Data

UPDATE: I took another look at it, and it appears that TFL is giving its northing as 1,000,000-REAL_NORTHING (source), where REAL_NORTHING is the standard definition of northing based on the OSGB36 grid. I hadn’t noticed that beforehand. Maybe they do it to make all northings 6 digits?

The rest of the post I wrote on Tuesday is below:

I blogged yesterday about being unable to use the user’s current location in the Journey Planner API. I have managed to get around this issue by reverse-geocoding the user’s location (latitude/longitude) to an address, which works. The problem, however, wasn’t the reverse-geocoding (which is straightforward). The problem was that TFL did not recognize a lot of the addresses that were reverse-geocoded. After many hours of work, I built a small feature that clarifies the user’s inputs, in the same way that TFL does when a user gives an incomplete address. This feature works, but is dependent on TFL’s website staying the same (which will probably be the case), as it uses TFL’s “suggestions” to suggest more specific parameters.

For the moment, that has been taken care of. Unfortunately, I’ve encountered a new issue: eastings and northings. Eastings and northings are a form of coordinate system involving measured distances from a certain origin (in the east and north directions) along the Earth’s surface. It’s sort of like a grid map. For some reason, TFL prefers to use these over latitudes and longitudes, which is unfortunate for the rest of the world and for iPhones.

Having spent a long time understanding eastings and northings, as well as looking up ways to convert between the two programatically, I have managed to overcome that difficulty. I have a working piece of code that converts from northing/easting to latitude/longitude. However, it seems that TFL’s data for northings on its Journey Planner API is plain wrong. For example, for the postcode NW3 3HJ, TFL’s API gives the northing as 815532. However, using http://www.nearby.org.uk/coord.cgi?p=NW3+3HJ, I found the actual northing to be 184468. That’s a huge difference!

Somehow, the northings are correct in the rest of TFL’s feeds, just not in the journey planner API online. This is frustrating since I cannot accurately show a map between two points if I’m not given the right coordinates. If anybody could provide suggestions on how to correct this false northing, please let me know by commenting on this post. If someone from TFL (or a fellow developer) sees this, please let me know what I’m missing. Is the data supposed to be this far off?

Until next time (probably in 2012!)

Laksh Bhasin

Difficulties with Journey Planner and Countdown

UPDATE: In V2.1, I fixed the issues with using the user’s location. I simply used Google’s reverse geocoding service to generate postcodes or addresses. I was afraid of this since it is not very accurate, but it’ll have to do for now. Also, I’ve decided to go ahead and implement Countdown, in spite of the problems it might bring. The rest of my post from December 26 is below:

I’ve been playing around with the Journey Planner API recently and although it has its flaws (it gets confused between stops of the same name; a problem Bus Buddy had until recently), it does its job well. Effectively, it can allow for location-to-location planning. Bus Buddy currently only allows for stop-to-stop planning, since location-to-location planning is algorithmically very complex (due to the sheer number of footpaths possible). It would be a big step forward if the app could accomplish online planning through this API.

Unfortunately, I have reached a stumbling block. TFL’s online Journey Planner API is designed for websites with a front-end IP address. This is unfortunate for iPhone developers like me, who don’t have the servers to assist 100,000 users at a time. Luckily, I found a way around this, by using journeyplanner.tfl.gov.uk as the requester and asking for an XML-friendly output. This gives XML access to any computer with an Internet browser.

However, even with this, I have not managed to use latitudes and longitudes as inputs for the Journey Planner. This is obviously very important if a user wants to plan a journey from his/her location to another stop. For some reason, TFL says that this should work, and have posted an example link in their documentation PDF. Unfortunately, that link also does not work. The API gets confused when it is given a set of latitudes and longitudes. As long as this issue remains, it will be difficult to fully utilize the API’s power to help mobile users plan a journey from their current location (understandably a useful future). I would probably receive a lot of criticism for putting out a half-finished journey planner (one that lacks this capability), so I will have to wait.

Also, some Londoners may have noticed new apps on the App Store that tell you when your next bus is coming (using a feature called “Countdown”). They all make use of the following website: http://m.countdown.tfl.gov.uk/. However, there is currently no API available to access the data from “Countdown.” Thus, I feel very wary about just accessing the website and downloading what it outputs, which is probably what most apps are doing. What if the output format changes all of a sudden? That could leave thousands of Londoners stranded at night. It is best to wait for an official API from my point of view.

To resolve my problems with the journey planner and Countdown, I have tried to contact TFL through many channels. Since it is currently the holiday season, their response delay is understandable. If issues persist, I will try to reach out to senior TFL employees. It may take months for me to receive a useful response, gain access to the APIs, and implement everything. Robotics starts soon at school as well, which will be both exciting and time-consuming! For my users, that means any future changes may have to wait even longer.

I’m really sorry about this as always, and have seen the impact of a few sour reviews on my downloads. This doesn’t financially affect me much since all the ad revenue goes to charity, but I am disappointed that I don’t have the time to help as much as I should.

Happy holidays and thanks for tuning in!

Laksh