Wednesday, January 14, 2015

Support CalTopo: GPS Maps

There are still some environments where a smartphone won't do, and you need the battery life and weather resistance of a dedicated GPS.  SAR is one of those environments, but it's also one that demands accurate, high-quality maps.  Once you've gotten used to tracking your location on a satellite image, or dodging cliffs on a steep descent using slope angle shading, it's hard to go back to a confusing mess of blocky contour lines rendered in 160x240 resolution.

Enter custom GPS maps from CalTopo, available with large-area coverage on 32GB MicroSD cards.  Preload the same high-quality maps that power onto your GPS and never worry about having to download imagery or deal with inadequate maps again.  Before I get too deep into the cheerleading, bad news first.

Garmin doesn't want to just sell you a GPS, they want to sell you a pure-gravy BirdsEye subscription as well.  The only way that works is if they're the only game in town when it comes to GPS imagery, which they accomplish by "locking" their units to only work with official Garmin maps.  In order to use CalTopo's custom GPS maps, you need to unlock your GPS as described here.  I'd be willing to pay a licensing fee to Garmin in order to get my maps officially sanctioned, since that would allow me to sell trouble-free cards through major retailers, but I've asked and they're not interested.

So on to the good news.  You live in California, Oregon or Washington and are in the market for some maps that will work with a newer Garmin GPS that you have unlocked.  As an added bonus, you'd like those maps to match your printed CalTopo maps 1:1.  You're in luck!  The following are real screenshots taken from my 62s.


Yosemite Valley, Rodeo Beach and Mt Shasta.

The three layers available are:

  • Scanned USGS Topo Maps, to an equivalent zoom of 15.
  • 2 meter per pixel aerial imagery, to an equivalent zoom of 16.  NAIP Aerial layer used for California, US Topo Imagery layer used elsewhere.
  • Slope angle shading with Forest Service maps replacing USGS topos where available. 
Some additional screenshots, of Mt Rainier and the San Juan Islands:

Check out the CalTopo Store today!

Monday, January 12, 2015

Elevation and Vegetation at a Glance

Long distance hiking leaves plenty of time for reflection, and an accelerated JMT trip I did this fall was no exception.  Despite being armed with maps and elevation profiles, I realized that I still wasn't aware whether a given day would be spent in shady forests or on barren, south-facing slopes.  I'd also been pondering a better way to size up search areas for SAR, and decided to solve both those problems with a new stats dialog.  It took a couple months to put the idea into production, but it's finally here.

Like with elevation profiles, there are two ways to bring up a stats dialog.  One is to right click on a line or polygon, and choose Stats.  The second is to right-click on the map background, choose Measure, and then either Line Stats or Area Stats.

Either option will probably involve a short delay as CalTopo pulls up the relevant elevation and vegetation data, followed by a dialog.  Unlike with elevation profiles, polygons will bring up data for the polygon's interior rather than just its perimeter.

The first two charts are elevation and slope histograms, along with min, max and average values; color-coding on the slope histogram matches CalTopo's slope shading layers.  The third chart is a circular histogram tracking aspect across 45 degree slices.  The pie slices are area-proportional rather than radius-proportional, i.e. if N aspects are 50% as frequent as W aspects, then the N pie slice will cover half the area of the W pie slice, with a radius that's 70% as large.

The tree cover histogram shows tree canopy coverage - 100% means that trees completely block all views of the sky.  Land cover shows land coverage as listed in the National Land Cover Dataset (NLCD).

In addition to other benefits, this provides another quick sanity check on planned routes for backcountry skiing.

Friday, December 12, 2014

Feature Week: Print to KMZ

I've received a few emails asking where the KMZ link in the left bar went.  It's been replaced by a print-to-KMZ option:

Why?  Because instead of just map layers, KMZ exports can now have markers and lines "printed" on them.  These are not traditional KML placemarks - for that, use the standard Export option.  Instead, they're rendered as graphics along with the map.  This can look a little fuzzy in Google Earth:

However, it really shines when used as a Garmin custom map.  You can draw far more lines on the map than most GPSs can handle as tracks or routes, and it allows you to create more colorfully marked up GPS maps.  Example screenshot below:

Thursday, December 11, 2014

Feature Week: Bulk Operations

If you've ever wanted to change the color on a number of lines at once, the new bulk operations feature will nicely complement folders.  The Markers and Shapes boxes, along with folders, now have a Bulk Ops link at the bottom.

Bulk operations can only be run on one object type at a time, so if you click Bulk Ops on a folder with both Markers and Shapes, you'll have to choose.

Once that's done, you'll see a full page dialog listing each object with some options at the bottom.  Click or shift-click to select a group of objects.

The Print Individual Maps option is only available for saved maps.  It will open a multipage PDF in a new browser window, with one page per object.  Each page is automatically centered on the object and scaled to fit, with a minimum scale as specified.  The default is 1:12k, which strikes a nice balance between capturing small map details and not losing context.

Change Attributes will bring up a second full page dialog, similar to the object's details dialog.  All fields start off blank, any changes made will be propagated to the selected objects.  Beyond setting marker and line styles, you can move objects between folders en masse.

Delete Objects will obviously delete the selected objects, although not without bringing up a confirmation window.

Wednesday, December 10, 2014

Feature Week: Folders and WYSIWYG Printing

Over the years I've had a number of users ask for a way to organize markers, lines and polygons within a map.  There have also been requests for a way to print only some objects, or to print markers with labels and shapes without, and so on.  Today's post covers a new way to do just that: folders.

Start by adding a folder to the map the same way you would any other object: right-click on the map background or click + Add New Object in the left bar.

The details dialog for markers, lines and polygons now has a folder dropdown.  Choose a folder at creation time, or move objects between folders the same way you would set any other property.

Individual folders will show up as boxes in the left bar alongside other object types.  Objects without folders will remain in their object type box, e.g. Markers.

As with the Markers and Shapes boxes, you can use the checkbox to show or hide all objects within a folder.  If you want to hide labels for a folder, perhaps because they're too noisy, the settings box now has a third, custom label option.

When set to "some", each folder gets a label toggle at the bottom.  Click on it and "Labels Shown" will change to "Labels Hidden".

Alongside these changes, printing has moved to a what-you-see-is-what-you-get (WYSIWYG) model: what you see on the screen is what you get when you print a map.   If an object is visible when you click print, it will show up on the print.  If it's hidden, it won't.  If it has a visible label, that label will show up on the print.  If the label is hidden, it won't.

Because this requires a form POST, I can now only offer the "share or save this configuration" link for a blank map, not one that has objects on it.  However I think the extra flexibility when printing is worth the cost.

Monday, December 8, 2014

Feature Week: Automatic Syncing and Collaboration

The past several weeks have seen significant behind-the-scenes performance upgrades to CalTopo, including mass migration of lines and polygons to a more efficient storage format.  Some of these changes were not without risk, so if anything looks off, please don't hesitate to let me know.

Only time will tell, but it looks like I'm out of the woods on performance work, and that has allowed me to roll out some long-deferred changes.  To celebrate, I'm going to run a series of posts highlighting new features.  The first one is automatic syncing and collaboration.

Without running afoul of trademark law, I've often tried to describe CalTopo as being like "Google Docs for maps".  While not as fully featured as a desktop program, from the beginning it enabled collaborative sharing and editing of maps in a way that simply wasn't possible with software like Nat Geo's now-discontinued Topo!.  However, for performance reasons, true Google Docs style "everyone sees all changes instantly" editing remained locked behind an optional checkbox in the sharing dialog.

No longer.  Starting now, saved CalTopo maps will automatically pick up on changes as they are made by other users.  Unlike Google Docs, there will still be a bit of a delay - from 5 seconds for actively edited maps up to 30 seconds for maps that have not been changed in the last 10 minutes.  After a couple weeks I may be able to dial this down depending on server load.

So how do you start this shared editing?  Time for  a quick refresher.

  1. The map needs to be saved (URL ends with /map?id=ABCD).  If you're working on an unsaved map, click the red Save link at the top of the left bar.
  2. Click Share.
  3. In the permissions dialog that comes up, either give write access to all users who know the map's URL, or set a password and give write access to users who know the password.
  4. Send the map's URL (e.g. to a friend.
  5. If you set a password, they will need to click on the Share link and enter it.

That's it.  Happy collaborative editing.

Friday, November 7, 2014

Mo Users, Mo Problems

Over the past several weeks, I've been getting progressively more frequent emails from the Binary Canary monitoring service letting me know that CalTopo was down.  It was sometimes unresponsive for extended periods of time, but since it generally came back on its own, I did what any good developer would do and ignored the problem until it reached red alert levels.
After spending 2 days a very long ways away from cell reception for SAR, I came back to a flurry of emails, including another Binary Canary outage warning - and CalTopo still appeared to be down, 5 hours later.  This was the tipping point; something had to be done.

Throwing together some changes and deploying them with a half-assed amount of testing was something, so that's what I did.  While it solved the performance issues, it broke imports and exports - something which CalTopo users let me know in short order.  Now that the fire drills are over, here's a quick recap of the issues.

One problem was in the way I sent line/polygon points from the server to the browser.  Instead of using a numerical array, each line point was a separate object.  This isn't a big deal in and of itself, but the JSON library I'm using to serialize everything created a heavyweight map object for each lightweight waypoint object, resulting in a huge amount of object churning relative to the memory available.  Points are now sent as an array, which doesn't have that overhead.

The second problem was - and possibly still is - with supplying cursor elevation.  Because I push elevation data to the client as a plain-text array of numbers, the 256px tiles are significantly larger than PNG or JPG map tiles.  As an added bonus, because elevation data went through the same server-side process as shapes and markers, the browser was instructed not to cache any of it.  Every page load generated a flurry of elevation requests.

One small checkbox, one giant bill.
Since turning the "Include Elevation" checkbox on by default, it turns out I was spending $150/month just to send elevation data to clients.  Oops!  I have a rather hefty Amazon bill, most of which is reimbursed by smartphone apps that license my maps.  Amazon makes it difficult to separate EC2 data charges (my server) from S3 data charges (map tiles), which is why a number several times larger than my monthly beer budget had slipped by unnoticed.

Now that elevation data is getting cached, I'll give it a week or two and decide whether I need to go back to having the checkbox disabled by default; the answer will probably be yes.

What this incident has made clear is that I need to find a way to make CalTopo commercially viable.  I've asked people to send donations to BAMRU because I thought that as a 501(c)3, it would encourage more liberal giving.  However, even if the donations BAMRU receives were funneled directly into operating the site, they would still be insufficient to cover my monthly operating costs.  Fortunately there are commercial map licensees for that, but while they pay me enough to cover my fixed costs, they're not really not a profit center.

When asked about my plans for commercialization, I've always maintained that while I don't want to cover the site with ads or turn it into useless baitware, I'd rather offer a better experience at a reasonable price than a mediocre one for free.  If it allows me to devote more time to the site and provide a better feature set, some kind of paid model would be an all around win.

After a lot of delay, I think it's time to go down that road.  More details will be provided as I figure them out, but I think the following is a safe high-level picture:
  • Most of the site will always be free.  From a purely self-interested perspective, I want to be able to hop on someone's laptop and sketch out a map without having to log in, verify my account, etc.
  • Any paid features would be part of a subscription rather than pay-per-use.  A number of people have suggested pay-per-print, but that seems rather petty; I'd prefer that people not have to ration their usage.
  • I'll try to align paid features with my cost structure, i.e. the features that cost me more time or server resources to support will be the ones I charge for.
  • Everything, including offline map data, will continue to be free for SAR and other first responders.
  • I'll be selling related items like GPS maps and 3D posters as an additional funding source.
So with that in mind, what are some things that might become part of a subscription?  Nothing is settled, but ideas that come to mind include large PDF sizes like 24x36, PDF links that last longer before expiring, WMS map access and having elevation data show up by default.  I'd also like to integrate non-free map sources (i.e. I need to pay the map provider), and a subscription seems like the logical way to tackle that.

The goal is to devote more energy to CalTopo, not to let it wither while I try to wring money from an ever-shrinking user base - though that seems to be a popular approach, too.  Paid features are an experiment, and if they turn out to be a bad idea then I'll drop them.  Changes will be neither drastic or immediate, and you don't need to worry that PDF generation or some other critical feature will suddenly wind up behind a paywall.

All of that said, I'm open to suggestions.