Friday, August 28, 2015

Up Again

At 8pm on Tuesday, I got paged out for a typically inconvenient mutual aid SAR: pack everything I need for a 72-hour deployment, sleep a couple hours, wake up at 3:45, drive 6 hours, fly into the middle of the Sierra backcountry, and hike out 40 miles.

Thanks for the lift, SEKI H-552

When I get back to civilization, my phone normally goes crazy with follow-on SAR pages.  This time, there was also no shortage of emails from worried CalTopo users.  Some people saw this:

some couldn't connect to the site at all:

some couldn't print maps, and a number sent me worried emails stating that they no longer had any maps saved to their account.

While I normally find someone to keep an eye on CalTopo for me when I leave on long personal trips, there was no time to do so before this search, and the site was down for at least a day while I was off the grid.

What Happened?

Pretty simple: the CalTopo server ran out of disk space.  This is mostly due to the tile cache - PDF generation, the view from here feature and various layers require me to fetch tiles from external sources, do some work on the server, and then send them off to the client.  Rather than fetching a map tile each time it's needed, I cache it locally on the disk.

In theory, the cache is supposed to have a maximum size and purge tiles to prevent itself from growing too large.  In reality, this isn't happening.  Since the cache gets reset every time I push a code change, this has generally not been an issue.  However with CalTopo growing, and the number of tile requests increasing proportionately, I've apparently hit the point where it will fill up in only a couple days.

Easy enough to fix, at least in the short term.

What About My Data?

The lack of space caused database connection issues, which meant that CalTopo couldn't pull back maps linked to individual accounts.  The maps were never actually gone, and I didn't have to do anything like restoring from a backup.

If something catastrophic were to happen, I back the database up to Amazon S3 nightly, so - again in theory - I should never lose more than a day's worth of changes.  I do occasionally download the nightly backups to a local drive in case my Amazon account credentials get compromised, but that would mean all kinds of problems.

If you're still concerned about data preservation, you can back up maps locally by choosing "Download CalTopo Backup" under the Export menu.  This is also a way to copy one map's contents into another, while preserving line styles, comments and so forth.

Tuesday, August 18, 2015

Please Share Your Maps

Since the beginning, CalTopo has had the ability to make maps publicly visible, i.e. someone can find the map without knowing its URL.  However all this did was add the map to an obscure page that few visited other than Google.

Today that changes with the introduction of the shared maps layer.  It's available as an overlay checkbox, along with a layer for your own maps that's visible when you sign in:

At wider zooms, each shared map is shown as a simple bounding box and center marker.  This can get a little confusing - some additional UI love may be necessary - but mousing over highlights the box and center mark.  Clicking brings up an info window with a link to that map:

Zoom in a little further and individual features get displayed:

Mouse over a feature and everything from the source map will get highlighted, giving you an idea of what that map covers.  Clicking gives you the feature's name and comments, if any, and another link back to the source map.

So how do you share your maps with other CalTopo users?  When saving a new map, just choose "publicly visible" under the "Share Your Map" section of the save dialog.  Go ahead and fill in some details in the "Tell Us About Your Map" section as well.

For maps you already have, start by clicking on your account name at the top of of the left bar.

Then go ahead and and set the sharing dropdown to "Publicly Visible" for each map you want to share.

This might be a good opportunity to visit the "Your Account" tab in the same dialog and change how your account is displayed to other users.  I'm also putting a messaging system in place so that other CalTopo users can ask questions about your maps.  If you don't want them to be able to contact you, uncheck that box.

There are only around 500 maps currently shared on CalTopo.  As that number grows I may need to change some of the implementation details to improve performance or reduce information density, but I think the basic approach is here to stay.  I'm pretty excited about it - I've already discovered a few maps that I wish I'd known about sooner.

Thursday, April 2, 2015

Improved and Printable Elevation Profiles

Elevation profiles are one of the few remaining gaps people have noticed when comparing CalTopo to desktop software.  No more.

There are now two profile modes - the existing interactive one that uses Google elevation data, and a full-page version that builds on the terrain statistics dialog with CalTopo sourced elevation data.  You can get to the new profiles using the terrain statistics menu option, or clicking the expand link at the top right of the interactive profile dialog:

The expanded version is a full-page dialog:

One of the most obvious differences is axes in place of rise/run data:

There's also a heatmap below the profile showing slope angle, land cover and tree cover along the route:

Below that is the previously existing terrain stats content (lines only - terrain stats along a polygon's perimeter, to match the profile, would be meaningless or misleading).  A more subtle addition is that nearby markers are automatically placed along the profile:

Using bulk ops, you can also plot multiple lines on the same axes:

Absolute plots the lines' actual elevations, while relative shows how each line changes from its starting elevation.  Below are 3 routes plotted against each other in absolute mode (top) and relative (bottom).

PDF versions of any profile are available using the print link at the top right of the dialog.

Note that because the interactive profile uses Google data and the expanded profile uses CalTopo data, gross gain and loss numbers may be slightly different.  Like terrain stats, expanded elevation profiles will only work within the continental US.

Wednesday, March 25, 2015

Weather Stations

CalTopo now has a source for NOAA weather stations - some are in standard airport and urban locations, but a number of others are in quasi-backcountry locations useful for trip planning.  Note that the data is screen-scraped off of a web page rather than coming through a documented web service, so it may be less reliable than the SNOTel and water gauge layers.

available as a data overlay checkbox

Similar to the other data layers, at broad zoom levels only station locations are displayed, not station data.  Zooming in, stations with winds greater than 10mph are rendered with arrows showing the wind direction, while stations with light wind or no wind data are shown as black circles.  Mousing over a station brings up a tooltip with the last observation time:

As with SNOTel and stream gauges, clicking on a station brings up additional details and a link to the station's official observation pages.  The official pages contain additional information I'm not sourcing, like solar radiation, fuel moisture and cloud ceiling.

The details dialog also has links to government-sourced diagrams showing typical monthly daytime and nighttime wind patterns.

Between weather stations, SNOTel and stream gauges, CalTopo now provides hourly data from over 12,000 remote sensors nationwide!  As usual, please report any issues.

Friday, March 6, 2015

Killer Visuals Through Enhanced Relief

One of CalTopo's core features has always been the ability to blend multiple map layers together.  Until now, that's been done by creating a layer stack and varying the opacity of each layer to suit.  This works, but shaded relief has always been a weak point.  In order to keep the relief layer from washing out the rest of the map, I had to crank up the exaggeration.  This meant you could use less relief and get less washout, but it reduced the layer's ability to capture small details.

Even so, it's never played well with aerial imagery.  Consider the image of Mt Shasta below; you can make out terrain features from the relief layer, but the imagery itself is starting to get washed out.

Professional mapmakers solve this problem by using more advanced techniques than opacity based layer stacking.  One of those is to do a multiply blend, where each pixel on the map is multiplied by the intensity of the hillshade, on a 0-1 scale.  This darkens the image a bit, but does a much better job of preserving it.  The photo below is the same image of Shasta with multiply blend relief shading . . .

There's no way to a multiply blend in Google Maps, so I have to fetch all the tiles on my server, combine them into a single image, and then pass them back to the browser.  This means that when using the enhanced relief layer, the map won't react to react to opacity adjustments in real time - it has to fetch a new batch of images instead of just changing a CSS parameter.  It also means that images not available to my server - like Google's - will default to the standard relief shading.

While the effect is more subtle with maps than with images, but I still think it's an improvement.  Below are maps of Mt Shasta with both standard and enhanced shading:

This is still experimental, so I may make some changes going forward, including switching to an approach that lightens some parts of the image and darkens others.  While enhanced relief works with printing, it reverts to standard shading for KMZ exports.  Still, I'd rather get it out there and get feedback than tinker away privately until it's perfect.

Wednesday, February 25, 2015

Dude, where's my UI?

CalTopo just gained a redesigned user interface.  Normally I hate invasive UI rewrites, and I'm sure this one won't please everyone.  However, over time it's become apparent that CalTopo's previous long-standing UI had too many obscure and confusing features.  It was painful watching people struggle with the same issues over and over, and while I may not have solved all of them, I felt that I had to at least try.

The most obvious change is a transition to pull-down menus across the top of the screen, grouping features from the left bar, top right and context menu into a single location:

Some of the context menu options proved particularly hard for people to find, like view from here and terrain analysis,.  While they still exist in the context menu, they've also been pushed out to a Measure pulldown menu:

Layer selection is largely unchanged, but has been moved into a more obvious standalone control that mimics the de facto standard look for a number of programs these days:

Mousing over the control, layer addition and removal should look familiar:

One big change is that clicking on a marker or shape will now bring up an info window with distance (lines), coordinates (points), and several edit and analysis options.  Clicking an object in the left bar still centers the map on that object, but will also bring up the info window:

Account and map management options in the left bar have also been consolidated and should hopefully be a little more obvious.

The "Share this map" option now only gives you the URL for sharing, while "Manage this map" lets you set both general map information and read/write access for other users.  The account link leads to an updated and hopefully more straightforward one-stop account dialog.

There are numerous small changes as well - as just one example that's hard to screenshot, the new marker dialog is now accompanied by a draggable on-screen marker rather than being in a fixed location.  As you drag the marker the coordinates dialog will update, and vice versa.

While I've done my best to test the new code, I'm not delusional enough to think it's error free.  If something looks off, let me know.

Wednesday, February 18, 2015

Printable Peak Views

The view from here feature has been around for a while, but you've never been able to print the results and take them into the field, except perhaps for making a screenshot.  I'm happy to announce that has now changed.

There's now a print link to the right of the share link:

Instead of launching a separate print page the way that the map viewer does, this simply brings up a couple print options at the top of the left bar.  You can choose page size, whether to show labels, and whether to show a custom selection or a 360 degree view.  A red draggable rectangle appears for the custom selection option.

The generated PDF has as many peak labels as possible, in diagonal orientation:

The 360 degree view has 3 rows, each covering 120 degrees.  Although they're not yet labeled, the first row covers NW-NE, the second row covers NE-S and the third row covers S-NW.

There are a few finishing touches that need to get added, like marking compass directions, but I don't want to wait for those to get finished up before announcing it.