Saturday, October 19, 2013

New Land Management and Fire History Layers

I've recently added two new layers to CalTopo, both built built on government datasets but with homegrown rendering.

Northern California burning
The fire history layer shows wildfire activity since 2000.  There are some regional datasets that go back much further, but this was the best I could find for national coverage.  Fires are color-coded by year along a yellow-red spectrum, with yellow fires being the oldest and red being the newest.  Exact dates are listed in the map info pane at the bottom right.

I've tried to show both the year and incident name for all fires.  Large fires are labeled as you begin to zoom in, and then all fires at the lower zoom levels.  Unfortunately, in areas with overlapping fire perimeters, it can be hard to figure out which label belongs to which fire - and some of the overlapping incidents even share the same color.

Large incidents labeled at moderate zoom

The second layer shows public land management (technically, the "surface management agency").  While data for this layer is sourced directly from the government and seems reliable if a touch dated, hunters should verify with a second source - don't blame me if something's mislabeled!

A wide mix of agencies in Southern California
As you zoom in, the polygons change from opaque-ish to translucent with borders, just like the fire layer.  I chose not to label them as national forests were the only areas with unit designations, e.g. Tahoe National Forest; most areas are simply attributed to BLM or state ownership with no further information.

National Forest boundaries - have they ever made sense?
I'm open to modifying the colors used for various agencies.  While I tried to follow existing map conventions as best I could, there are only so many options available and I can see how some might cause confusion: the Bureau of Land Management and Bureau of Reclamation are both orange, but I don't know how similar their rules on activities like hunting and camping are.

Public land management in the continental US

Tuesday, October 15, 2013

SARTopo - CalTopo for SAR

CalTopo started as a project to provide better maps for search and rescue.  As I focused more effort on caltopo.com, the SAR version lagged behind and it became harder to bring new features from CalTopo back to the SAR code.  This summer I put a lot of effort into reworking things, and while CalTopo hasn't seen many new features, the SAR version is finally ready for a re-launch.

So without further rambling, I give you SARTopo.com - CalTopo for SAR.  SARTopo takes the existing CalTopo UI that you've learned to at least live with if not love, and adds additional search and rescue object types like operational periods, assignments, clues and resources.

These extra objects make it easy to do things that are cumbersome at best with traditional mapping software: editing the same dataset from several computers, printing assignment maps on multiple backgrounds, dumping segments to GPS, filtering stray tracks and waypoints, color-coding data by operational period or resource type, and a lot more.

I encourage anyone involved in search and rescue to watch this 3.5 minute video and then give it a try at sartopo.com.

Both the software that makes this happen and the map layers I've created are available for free to search and rescue teams.  You can load them on a laptop and run a mobile version of SARTopo in the field, miles from the nearest internet connection, with statewide map data including USGS topos, USFS topos, aerial imagery, shaded relief, contour lines, slope shading, viewshed analysis and even the "view from here" feature.

For more information on offline use, please visit sarsoft.org.

Assignments, Clues, Resources: some of the added SAR features

Now that SARTopo is finally live, I look forward to returning more attention to CalTopo, and several new feature blog posts are coming shortly.

Monday, June 3, 2013

Summit Views and Aerial Maps

I can finally announce the CalTopo Viewfinder (working name), a new page that shows a simulated view for any spot in the lower 48.  Play with it today by right-clicking on the map background and choosing "View From Here", or simply go to caltopo.com/view.


The default view is a ground-level wireframe with peak names, which is great but hardly new.  Peakfinder.org has been doing that for a while, and packages the functionality in a nice mobile app to boot.

Looking towards the palisades from Big Pine, CA
Step 1 was to add aerial imagery to the view, which helps give a better sense of what you are (or will be) looking at:

The view from Mt Tamalpais, CA
You could do this in Google Earth by importing a KML file with all the local summits, but Earth's ground-level view can be frustrating to work with.  And you can't zoom in on distant summits, only move towards them.

I noticed that it was hard to drop the "eye" exactly where you want it, and sometimes the actual summit would obscure part of the view.  So I added an option to control the eye altitude.  And I kept making it taller and taller, eventually taking it up up and away into the stratosphere.  The rendering math I'm using assumes a small eye elevation relative to the radius of the earth, so this isn't a perfect representation of what you'd actually see from that height, but I do think it provides some new and interesting terrain visualizations.  Click the linked captions for large, interactive versions:

East Slope of the Sierra

Glacier National Park

West Slope of the Sierra

Front Range

North Cascades National Park

It may be personal bias, but for some uses I prefer the zoom in from a fixed point approach used by CalTopo over the zoom by panning approach used in Google Earth.  The perspective feels more like the relief models found in NPS visitor centers, and makes it easier for me to wrap my brain around large-scale terrain features.

This is still under active development.  Beyond performance improvements, the USGS summit data could do with some refining as peaks are sometimes off by several hundred meters.  And, I hope to be able to offer custom print-on-demand posters that capture these high-altitude views in a nice, wall-friendly format.

Friday, May 17, 2013

Viewshed Analysis

Viewshed analyses, typically the realm of expensive GIS software, show you all the areas that are visible from a given point.  They can be useful for estimating radio effectiveness, planning photographs or simply checking out the view from a high point.  CalTopo now supports them in easy, drag-and-drop form.

slowly growing list of data analysis layers

To start a viewshed, simply click Add New Layer and then Add Viewshed Analysis.


A bullseye target icon will appear on the map; drag it to the location you want to run the viewshed from.  You can also choose how high above ground the analysis should start from; the default is 20m, which helps when you don't drop the icon exactly on top of a high point.  Click save and presto, viewshed:

the view from Jigsaw Pass

Mt Diablo in California is (falsely, according to Wikipedia) rumored to have one of the largest viewsheds on the planet due to its prominence above flat terrain.  What does it look like?  Let's find out:

the Earth's curvature comes into play about 20 miles past the Farallon Islands

One of the things I learned while doing this is that it's actually pretty easy to write a viewshed analysis program.  Writing one that runs quickly over numerous zoom levels and a wide array of distances is a different story.  It's simply not feasible to check the elevations between Mt Diablo and Modesto at 10 meter intervals for every pixel on the screen; you need to do some kind of performance optimization, and that means it's possible for inaccuracies to creep in.

I spent a lot of time trying to find the right balance between performance and accuracy, and I think the end result is perfectly adequate for non-professional uses (obviously cell companies aren't going to be using CalTopo to research tower placement).  As QA tasks go, this was actually a pretty fun one.  As an example, photos of half dome from the central valley and bay area always elicit lots of discussion and occasional allegations of photoshopping.  So I decided to check the viewshed layer against known photo locations:

line-of-sight locations between the central valley and summit of Half Dome

This isn't perfect as it's line of sight only and doesn't account for atmospheric refraction, but the iconic photo location outside Turlock is well within the red zone.  What about the Lick Observatory on Mt Hamilton, site of another classic shot?

yes, you can see Half Dome from Mt Hamilton on a super clear day

The viewshed layer has the most precision at the origin point and the pixel being checked, and the least halfway in between.  While this normally works great, the place where it's most misleading is when a peak or narrow ridge is halfway between you and the location being checked.

Finally, without giving too much away about future projects, here's a texture-mapped visualization and viewshed from the same location.  Both use largely the same algorithm for determining visibility, so if the visualization looks good it helps confirm that the viewshed is working properly.  Castle peak is poking up to the left of the foreground and the frog lake cliffs are visible to the right:


And here's an actual photo from the same location, although from a slightly lower elevation (and through a very telephoto lens).  Because of trees, it's hard to tell whether the low ridge to the left of Castle is actually visible.  However, the photo seems to match well.


I'm pretty satisfied with the result of my viewshed QA efforts, but if you see anything that looks seriously off, please drop me a line.

Friday, April 26, 2013

Freehand Drawing

Update: Based on user feedback I've swapped click-hold and shift-click-hold to better mimic the standard behavior.  Freehand drawing is now enabled through shift-click-hold; click-hold will drag the map as usual.

CalTopo now supports freehand drawing, allowing you to place lines by moving your mouse rather than individually dropping each point.  This required booting Google's drawing manager for my own, which always runs the risk of browser compatibility issues - so part of the reason for this post is simply to solicit bug reports of something seems amiss.

When drawing lines or polygons, you have the following options available:

  • click to add new points
  • click and hold to draw freehand segments
  • shift-click and hold to drag the map, since click and hold is now overridden
  • escape to undo drawing actions.  This will undo individual points one at a time or freehand sections in 50 point blocks.
It's now easier to draw wandering routes like this one

Thursday, April 25, 2013

Offline Maps With CalTopo To Go

As hinted at with the introduction of browser maps, you can now take CalTopo on the road, offline, without an internet connection.  Download maps ahead of time, edit them in the field, and then re-upload them when you hit civilization - or simply take some layered map goodness with you for offline viewing.

Step One: Save Maps to Your Browser.  See this post for details.

Step Two: Cache Map Tiles.  Go to http://caltopo.com/togo.  You'll get a rectangle select just like on the Print and KMZ Export pages.  Choose the area and zoom level you need, and check the layers you want to bring with you.

layer and zoom selection
Note that while there is a 1600 tile limit, the HTML5 Application Cache is a fragile beast.  Your browser may run out of space and throw an error before reaching 1600.  When you're ready, click the "Take This To Go" button:



You'll see "Processing . . ." for a while as the server builds up a cache manifest, and then a browser specific progress message.  When all the files are cached, you'll get an alert and be redirected to the offline page:


Step Three: Go Offline.  When you go offline, or when you just want to take it for a test drive, point your browser at http://caltopo.com/offline.  You'll get a simplified version of the CalTopo UI, using the OpenLayers map engine instead of Google Maps.

The simplifed CT2GO UI
The layer dropdown is limited to your cached layers

Some random thoughts and caveats:

  • This will not work with IE (9 and below) as it doesn't support HTML5 Application Caches.
  • This feature is both more fragile and harder to test than a standard online webapp.  Please send me bug reports if something doesn't work right.
  • Tile caching is currently limited to one rectangle at a time.
  • While it worked fine with an iPad, phone support is tricky, mostly because of the screen real estate required for the caching page.  I'll work on allowing you to draw a rectangle with your desktop and then cache it on your phone.
Enjoy!

Tuesday, April 23, 2013

One-Off and Browser Maps

I'm happy to announce a couple of long-overdue additions to CalTopo.  The first, one-off maps, allows you to create and share a map without tying it to your account.  You can also save maps to your browser using HTML5 local storage, which helps lay the groundwork for an offline version of CalTopo.  And, maps are now easier to delete/manage.

New Save Dialog

As the name suggests, One Off Maps let you create quick, single use maps you can email to a friend or post in a forum and then largely forget about.  If you're signed in when you create one, it will still be linked to your account so that you can make changes, but it won't show up under your map list.  If you don't want to sign in using OpenID, you can enter a password to allow future edits.

Browser Maps store data on your browser using HTML5 local storage.  They're not backed up to CalTopo, and there's no way to share them as the data only exists on your computer.  However, you can take them with you when you don't have internet access, and then transfer the data back to CalTopo later - more on this when offline maps are made public.  The browser map UI looks almost exactly the same as the standard map UI, except that the Share link is missing:


With one off and browser maps done, it was finally time to revisit the "Your Maps" screen.  There's now a second table showing your browser maps, and tools to copy maps between CalTopo and your browser:


When you click on the red X, you get two options: delete the map entirely, or turn it into a one-off map.  If you still want a link you sent people to work, but don't care about managing it through your account anymore, you can detach it and forget about it:


Thursday, February 28, 2013

Printing PDF Topo Maps (and Map Packs)

CalTopo has supported PDF printing for a little while, but it was tucked away until I made sure that generating the PDFs wouldn't overload my server.  It's now front and center in the UI, along with higher resolution PDFs and an important new feature - multi-page map packs.

First off, clicking the print icon in the top right will no longer bounce you straight into browser print mode.  Instead it brings up a menu for choosing between browser printing and the PDF generator:

The new print menu
The PDF option was initially tucked away as a link on the browser print screen, built out to satisfy some grumbling users who wanted exact 1:24k prints.  Since then I've found myself using it almost exclusively - the prints are higher resolution and you aren't limited to a fixed set of zoom levels.  It seemed time for a promotion to first-tier status.

Now for the good part - when you choose "Create a PDF", you can mark up to 15 pages on the map and dump them into a single PDF file for printing.  Here I've marked several pages on a user's Big SEKI Loop map, all with a 1 inch : 1 mile scale:


When I click "Generate PDF" they come back in a single file, along with an overview page that shows how the maps are positioned relative to each other:

The first page of the PDF shows individual map placement
So try it out!  You can also check out a sample 1:24k PDF of the popular Bunny Flat routes up Shasta, with slope shading.

Please note that this may become a paid feature at some point, depending on how much extra work it creates for my server and the degree to which I need to make a living off CalTopo.

Wednesday, February 27, 2013

Improved Elevation Profiles

Elevation profiles have been a bit of a weak area on CalTopo.  You could draw a profile and get point elevation data along the path, but that was it.  The profile function has now been updated with a couple key features:
  • More sample points used
  • Distance measurements provided for the x-axis
  • Gross elevation gain and loss calculated
  • Horizontal sampling interval and vertical exaggeration shown
  • Climbs and descents marked with distance and elevation change

Here's an example profile:


 And here's what it used to look like:

Using the data rendered for custom DEM shading, I was hoping to update the elevation profile with slope and aspect data as well.  I've seen some cool circular plots of elevation v. aspect and slope angle v. aspect, and thought those would be a nice addition.  However after playing with an elevation v. aspect graph, I've concluded that they're only useful for a narrow subset of routes.  On ridgelines, summits and valleys the aspect can bounce around dramatically over very short distances.  This leads to a graph that exaggerates aspect swings at best, or turns into a useless and confusing mess at worst.

Instead, I've incorporated that data into a "point info" feature that will give you elevation, slope and aspect readings for any spot in the continental US.  I don't know how useful this will be, but it's more of a temporary bridge until I find a better way to provide this data in graph form.  Note that because the point info feature uses CalTopo's elevation data and the profile graphs rely on Google's elevation service, there may be small discrepancies between the two.

Selecting the Point Info tool.
Not the most exciting dialog you've ever seen:


Wednesday, February 20, 2013

New Visitor Maps

Visitor maps - the nice brochures they give you at park entrances or the giant Forest Service road maps with nary a topo line on them - aren't terribly useful for backcountry navigation.  However they can be great for visualizing roads and trails at a 10,000' view and then switching over to other layers as you zoom in.

My first pass at a visitor layer involved manually downloading PDF and TIFF files, erasing areas outside the park boundary, lining up 4-5 points on each map with real world coordinates, and then tiling them.  Where parks and forests met with curvy borders, erasing everything outside their respective borders was painstaking.  Goof one of the alignment points and I had to delete the existing tiles and then re-tile the map.  I got pretty good coverage for California and then gave up.  It was simply too much work and too little benefit.

Since then the park and forest services have done a better job releasing maps as GeoTIFFs and GeoPDFs, and I felt like it was time to take another pass at visitor maps.  In order to avoid the cropping issues that cost me so much time in the first iteration, I went with two layers, one for the NPS and one for the USFS.  There are almost no contiguous national parks, so I could render the park maps as-is without cropping.  Forest Service land is a different story, and after investing too many hours tweaking an edge-detection algorithm I said "good enough".

The original visitor layer.  Although close, it's not a perfect fit with the aerial imagery.  This is partly the map's fault, and partly my inability to perfectly georeference and warp it.

In the new NPS layer,  streams in Tuolumne Meadows line up dead-on with aerial imagery.

New USFS layer.  The automated cropping left a little bit of the left map's border in place.
As the NPS and USFS continue to release new versions of their maps, I'll be able to grow these layers to match.  The bad news is that I now have 3 visitor map layers.  I'll eventually be able to pull the original, but not until all forests in California get updated maps, which likely won't be for a while.  And there will probably never be a good way to get NPS and USFS maps back together in one layer - but if there's one feature that defines CalTopo, it's the ability to mix and match multiple map sources on the fly.

Wednesday, February 13, 2013

New Forest Service Maps

Without much fanfare, the Forest Service recently redid its 7.5 minute quads.

The previous version, on CalTopo until now, was called the Primary Base Series.  They were produced "back in the day", offset printed, and then scanned into TIFF files.  They had roads and trails that were lacking on USGS maps, and perhaps just as critically, numbered road designations.  However they only existed for some forests, and these manmade features were quite a bit out of date.

These have now been replaced by a new, computer generated iteration called FSTopo.  Because the maps are rendered straight into a graphics file, they look nicer and have a higher resolution than the PBS series.  Coverage includes almost all national forest lands, including Alaska.

Unfortunately, while the FSTopo maps have some updated information, they're still a little out of date compared to actual roads and trails on the ground.  Even without any updates, the increased coverage and higher resolution would still make them a huge step up over the previous maps.

FSTopo has replaced PBS as the default map for "US Forest Service" and any saved maps or URLs will be automatically upgraded.  If you need access to the old maps, you can still get to them under Available Data Sources, as "USFS PBS".


Tuesday, February 12, 2013

Custom DEM Shading

While rendering the new slope shading layers, I also rendered data tiles for elevation, slope and aspect.  CalTopo can now use these to create custom shading for the continental US.  Want to color all north facing aspects above 7000' and steeper than 32 degrees blue?  Create an elevation relief map?  No problem.  Unfortunately I haven't had time to create a slick UI, so the purpose of this blog post is to serve as documentation.

Click the checkbox for the "Custom Shading" overlay layer and a text area will appear.  Use this to enter condition-color combinations, one per line.  The conditions can be any range of slope, aspect or elevation; the color is an RGB hex code.  This is best explained with some examples:

s15-30 FF0000
s30-45a90-270 0000FF

Slopes between 15 and 30 degrees shaded red, as per the first line

The first line will cause all slopes between 15 and 30 degrees to be colored red (#FF0000).  The second shades all slopes between 30 and 45 degrees, with aspects between 90 and 270, blue (#0000FF).  If you're not familiar with this notation, search for "hex color codes" or play with the W3Schools color picker.  You can also use elevation in meters, or append the number with an f for feet

e1000-2000 matches all elevations between 1000 and 2000 meters.
e5000-7000f matches all elevations between 5000 and 7000 feet.

Each line can match one range each of elevation, slope and aspect, represented by the letter e, a or s and followed by start and end numbers separated by a hyphen.  So e7000f-30000fs32-90a225-45 would match all north facing slopes greater than 7000' and 32 degrees or steeper.

North facing slopes above 7k and 32 degrees or steeper

If you have only one condition, you can specify a pair of colors to create a gradient.  The following creates two gradients: elevations between 5k and 7k are shaded white->blue, and elevations between 7k and 9k are shaded blue->red.

e5000f-7000 FFFFFF-0000FF
e7000f-9000 0000FF-FF0000

Mountains around Squaw Valley, CA shaded using the above gradients
You can also use the service to create nationwide visualizations.  And custom layers will also print using CalTopo's PDF generator as well!

A high level view of terrain 25 degrees and steeper in the Pacific Northwest.

Tuesday, January 15, 2013

Slope Shading Revisited

The CalTopo slope shading layer, originally done as an experiment of sorts, is overdue for a makeover. From a usability perspective, there are not enough colors - a wide variety of terrain gets lumped together into a few categories.  From a technical perspective, tiles are rendered on the fly using Mapnik and prebuilt polygons.  Not only is coverage limited to certain areas, but rendering speed is slow and I can only display select zoom levels.

The slope shading was originally built with backcountry skiing in mind, but it's also seen a lot of interest for summertime route planning, and I'd like to find a single color scheme that can accomodate both users.  After a lot of fiddling, I have two prototypes that I'm seeking comment on.

Both use a similar color scheme, but one has a smooth gradient (v1) while the other has fixed steps (v2).


Version 1, Avalanche Gulch
Version 2, Avalanche Gulch
In version 1, the colors are (slope angle in degrees): 15 = green, 27 = yellow, 40 = red, 60 = blue.  All intermediate slopes are an average of the two nearest fixed colors.  In version 2, slopes less than 20 degrees are not shaded.  20-26 = green, 27-29 = yellow, 30-31 = light orange, 32-34 = dark orange, 35-45 = red, 46-50 = purple, 50+ = blue.  Here's another take:

Version 1, Shastarama
Version 2, Shastarama
Version 1 makes it easier to spot subtle changes in terrain, and makes for a nicer looking picture.  However, it's hard to tell a slope's exact angle - and for avalanche safety, there can potentially be big differences that result from small changes in slope angle.  For planning safe travel routes in avy terrain, I think v2 is 10 times more useful than v1.

Right now I'm leaning towards producing both of these versions as map layers, so the question isn't really which one you prefer.  Rather, it's how can I improve them?  Should v2 have more colors, especially in the 0-27 or 35-45 ranges?  Should I pick different colors?  Highlight different slope ranges?