Map Projections

I first learned about cartography and mapping technologies back in 2007 or so when I was working on Danger's bluetooth stack.  In order to test my work on the serial port protocol (SPP), I connected to a bluetooth GPS "puck".  This led to finding interesting things to do with that data, namely a mapping and location tracking application.

Back to the present... As part of trying to find a really interesting project to really drive my web development goal in my spare time, I've been doing a lot of reading on map projections again, looking into what's required (technically) to convert things like park maps or trail maps into something that could be overlaid on a map.

Here's the 30 second background summary: Since the earth is a sphere, but your screen is a rectangle, some magic is required in order to display maps of the earth on your screen.  This magic is actually mathematics, and is called projection.  You can liken the problem of projection to the problem of flattening out an orange peel on a table top.  You can do it, but there are trade-offs involved.

There are some really interesting projections out there, but popular maps on the web use the Mercator projection.  This projection is accurate around the equator, but more and more distorted the further north and south you travel.  This projection is popular because it flattens out longitude, so that they orthogonal to latitude, making for a nice x, y coordinate system that is intuitive and square, among other things.

It may blow your mind to realize that what you think you know about the earth is actually wrong... or distorted, but what does that mean for trying to display a park map as an overlay on one of these Mercator maps?  It depends on the projection of the map of the park, but generally it means that the park map will have to be distorted to fit the faux reality of Mercator.

Figuring out and applying that distortion is what I'm interested in, and has led me down all sorts of interesting paths over the past few days.

Scala + Play

Many moons ago, I was a web developer.  I created my own web framework, developed a bunch of libraries (charting, scraping, etc.), wrote articles and everything.  I even managed to created a website that had close to a million views in a month, and that was mentioned in a PopSci article.  I've tried to keep up with web technology over the years, but haven't really had the need to do much web development and am far behind the curve.

Every once in a while I get the urge to start a project involving the web.  I end up spending a bunch of time reading about some new language or technology, and don't really get much actual coding done before I lose interest or catch a glint of something shiny in another corner.  (This is a recurring story for me that is definitely not limited to web initiatives.)

Anyway, I'm back to thinking about the web this time because I really want a good tool that can automate (more of) the ice scheduling that I do for my daughter's hockey association.

I have written some code towards this purpose already, but it's just a C# command-line app that performs some web scraping to pull in all team schedules into an aggregated list for a given week (impossible using the current navigation on the site).  I've also made some false starts toward a more powerful desktop form-based app to automate things, but that approach never sat well with me.

1. I want to be able to access this from anywhere, including my phone or tablet.
2. I want to be able to let other associations use the tool, if it works well.
3. I don't really want to tie it to Windows, and would prefer to push my skill-set

So here I am, reading about Scala.  This is a hybrid object-oriented/functional language that is definitely cross-platform (runs on the JVM), and has web frameworks like Play, which is less like a Java servlet environment, and more like PHP.

Let's see how far I get this time.

Fitbit Zip Teardown

Just saw this (pretty light) teardown of the Zip:

http://www.designnews.com/author.asp?section_id=1386&doc_id=256756

Since they came out in the same year, I expect the Zip and the One (and maybe even the Flex?) have fairly similar hardware characteristics.

Fitbit Dongle Enumeration

I just checked in my notes on how the USB dongle responds to enumeration.  You can see them here:

https://github.com/hiptopjones/fitbit/blob/master/UsbDongle/usb_enumeration.txt

I learned a lot about the workings of USB devices, but can now see that it publishes two HID interfaces, each of which produces reports of a generic 32 bytes at a time.  While this is not a HID device in the traditional (mouse, keyboard, joystick) sense, and using a HID interface means quite low data rates, I expect this is done (at least partly) to avoid needing special drivers on most common operating systems.

I haven't yet figured out the format of these packets, but I can see normal sync data (matching the blocks in the sync log), as well as a number of strings like "StartDiscovery", etc.

I had been wondering if the strings represented some sort of command-based protocol with the dongle, instructing it to perform bluetooth operations, but I see that most of these strings are on IN endpoints, which means they're probably just there for logging, and not instructions.  This is supported by the fact that we see lines like this in the log:

    <Notice>: IN<CTRL> trace: StartDiscovery

I can't yet rule out that one of the HID interfaces could be being used for the transport layer of the Bluetooth stack, but I don't see bluetooth stack-like strings in the binaries on Mac or Windows.

Continuing to investigate.

Fitbit USB Dongle Sniffing

Sync packets are all well and good, and decoding them continues to be a priority, but I decided to give it another go with the USB filter driver today.

Using busdog, I'm now seeing chatter text in a USB trace!  (Does that mean the link is not encrypted?)

11 In  (USB URB Function: 9) 0.007991 32 48 45 59 20 48 45 59 21 20 20 48 49 47 48 46 49 56 45 20 20 00 00 00 00 00 00 00 00 00 00 00 14 HEY HEY!  HIGHFIVE  ............

I expected that the filter driver would just pick up bluetooth transport (HCI?) bytes, but I also see strings in the log that make me wonder if the dongle uses something like a serial protocol to communicate with the host PC.  That would be weird, though, or at least not like most other bluetooth dongles.

13  In  (USB URB Function: 9)   0.000002    32  20 01 43 61 6e 63 65 6c 44 69 73 63 6f 76 65 72 79 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .CancelDiscovery...............
. . .
13  In  (USB URB Function: 9)   0.000003    32  20 01 47 41 50 5f 4c 49 4e 4b 5f 45 53 54 41 42 4c 49 53 48 45 44 5f 45 56 45 4e 54 00 00 00 00  .GAP_LINK_ESTABLISHED_EVENT....

Anyway... we'll see.  I don't have protocol docs for Bluetooth LE, but perhaps by subtracting the bytes in the sync log from this output and doing some more reading, I'll be able to deduce something more about the actual device communication.

Fitbit Sync Decode - Part 3

Slowly but steadily piecing together parts of the puzzle from the sync logs / fiddler traces.

Discovered that there is a so-called "mega-dump" and "micro-dump".  The mega-dump is used for general syncs, but the micro-dump was used when I tried a firmware update.

I've setup a github repo for my parsing code at https://github.com/hiptopjones/fitbit.  I'll let that code stand as documentation for now.

Fitbit Sync Decode - Part 2

If we think about what information would need to be passed back and forth, that might help when looking at the sync data, and provide ideas about what knobs to turn.  A good source for inspiration here is the user manual.

This summary data would probably go in both directions:

• Calories
• Steps (walking + running)
• Elevation (floors, distance)
• Distance (native units)
• Active score (# plant leaves, score)

These would be passed from device to server:

• Timer information (start/stop)
• Steps data series (per minute)
• Elevation data series (per minute)
• Battery level (charging status?)
• Firmware version
• Beginner mode flag (?)

These would be passed from server to device:

• Enable/Disable flag for all of the displayed metrics
• Chatter text
• Current date/time
• Stride length (walking, running)
• Body stats / BMR (for calorie estimations)
• Timezone
• Units (distance)
• Time format (ie. 12/24 hour clock)
• Left-hand/Right-hand mode
• Silent alarm settings (including repetitions on specific days)
• Sleep tracking mode (normal vs. sensitive)

Some additional inputs:

• There are 10 leaves on a fully-grown plant
• The device can store 14 days of minute by minute data, and summary data for 30 days
• The device can have 8 separate alarms