Saturday, December 22, 2018

Android Open Beta

The CalTopo blog has been pretty quiet over the past year, but not due to a lack of innovation.  Rather, we've been focused on a top secret project - one that after 18 months of development is finally secret no longer: a phone app.  Today marks the release of an open beta test for Android; iOS will hopefully follow before summer.

First, let's set expectations: the app is still in development, and this is a beta test.  There are probably some major bugs related to specific platforms or android versions that we haven't caught yet, and some obvious functionality is missing, such as the ability to place a marker at your current location.  We're releasing this not because it's a finished product, but because we're at the point where we need testing and feedback from a broader audience.

With that said, what does the app offer?  I'm going to keep this blog post short and refer you to the Play Store listing, but the short version is that tight integration with caltopo.com provides some amazing benefits even if the app's functionality isn't fully fleshed out.

The offline map downloader uses a pre-packaged 15'x15' grid for fast downloads.

The app is free, and for now the only restricted feature is offline map downloads, which are tied to existing CalTopo plans.  Basic accounts get 5GB/yr, which is enough to cover the entire JMT at high resolution (2 meter / pixel or zoom 16) with the USGS, NAIP Imagery, MapBuilder Topo/Hybrid/Overlay, shaded relief and slope angle shading layers.  Pro accounts get 30GB/yr, which can cover the PCT with those layers twice over at a moderate resolution (4m/px or zoom 15).

To keep downloads small, some offline layers are computed from others: if you have elevation data, you automatically get slope angle shading, and if you have MapBuilder Topo and NAIP Imagery, you automatically get MapBuilder Hybrid.

Go check it out, we hope you're as excited as we are.

Tuesday, November 20, 2018

New MapBuilder Roads Option

One feature request I get fairly often is a better printable roads layer, primarily from SARTopo users for use in both urban searches and disaster response.  The USGS layer is out of date, MapBuilder doesn't show enough street names, and OpenStreetMap gets shrunk to the point of illegibility when printing.  I've always assumed that the proper answer was to bring the default OpenStreetMap style into the MapBuilder rendering stack, which would allow scaling it to match a PDF's resolution - but because that introduced a number of complications, I kept kicking the can down the road.

Necessity is the mother of invention, and a little over a week ago I was called to the Camp Fire in my role as a SAR team member and found myself needing to generate a lot of street maps.  One requirement was that they display as many street names as possible, so that responders from around the state could use them for navigation despite not being familiar with the area or having a cell signal.  A clean white background was also important to allow marking up maps in the field.  Fortunately I'd just wrapped up a MapBuilder import and had a second rendering server sitting idle with the new database, so I was able to hack together a new layer style quicker than expected.

There are a lot of lessons learned from trying to apply my wilderness SAR background to an urban disaster, but one of those is that the new roads layer is here to stay, and I'm sorry I didn't have one out sooner.  It will probably get tweaked further and eventually added as a primary layer, but for now it's only available as a custom MapBuilder layer (pro level account or higher required):

Check "compress text".  Recommend setting roads to "thin lines".
With the "compress text" box checked, a number of changes are made to support showing as many street names as possible:

  • Suffixes are dropped (drive, road, street, etc)
  • Font size for road labels is reduced
  • Labels are allowed to bend more
  • Labels are spaced closer together
The Oakland hills with most streets labeled at a1:12k scale
The result isn't exactly pretty compared to something like Google Maps, but it's fairly effective as a first response tool since it will show a lot of street names at usable map scales like 12-24k.

San Francisco at 1:12k

Even at 1:24k, a lot of Berkeley street names still get shown

The layer didn't exist 10 days ago, and hasn't seen a lot of careful development or evaluation, so feedback and suggestions are definitely welcome.

Tuesday, September 18, 2018

Major Offline Improvements

A new version of CalTopo Offline and SARTopo offline has just been released (4151), with several major improvements:

1.  Real-time syncing between offline and online versions.  All of your maps from the website will show up in the offline version, and changes will automatically sync between the two.  If you make a change while offline, it will be queued until you regain internet access.  This is helpful both when starting a project online and then taking it into the field, and also when working in an area with unreliable internet, so that you can keep changes synced to the outside wold without being at risk if the internet dies.

2.  New map downloader that downloads offline map data within Cal/SARTopo Offline.  This makes it much quicker and easier to download large areas than the old web-based downloader.  Because the new downloader offers multiple resolutions and is built on a 15x15 minute grid, it also gives you more control over how large your offline maps are.


3.  Revised layer selection.  Elevation, slope and aspect data are all rolled into a single layer.  Shaded relief and slope angle shading are automatically computed from elevation data, and no longer need to be downloaded separately.  Most excitingly, MapBuilder Topo, Hybrid and Overlay layers now have full coverage for the contiguous US.

4.  Layer caching.  Map areas viewed in the offline version when you do have internet access are cached for later use offline.  So even if you haven't pre-downloaded an area, it may still be available when offline.

Tuesday, September 11, 2018

Come Work For CalTopo

CalTopo is growing!  I'm looking for a contract graphic artist and a full-time UI developer.  Come work on a product you actually use, in a job with significant flexibility.  For more information, email mjacobs@caltopo.com.

This could be your office (deck not included in job offer, applicant to supply their own)


Graphic Artist

I'm looking for a graphic artist to take on a contract project revising and expanding the current icon symbol set.  If things go well there's potential followup work (UI icons, MapBuilder symbology, general UI design), but nowhere near enough to add up to a regular job.

I'm also looking for assistance creating a logo, whether that's the same designer described above or a specialized professional.

UI Developer

The baseline for this full-time remote position is a competent developer with a solid web UI skillset, able to be turned loose on projects with minimal supervision.  Preferably, you can also take on UI and interaction design, coordinating with a graphic designer on the visual pieces as necessary.  Ideally, you can tackle the entire UI development cycle from graphic design through development.

The following are not required but are considered pluses:

  • A CalTopo user or outdoor enthusiast.  This is a position where domain knowledge matters.
  • California or Washington residency.  As a full time remote employee, CalTopo LLC will have a business presence in your home state.  Some states are easier to work with than others, but CalTopo is already registered with CA and WA.
  • OK with occasional travel to Truckee, CA (potentially up to several days a month, generally less).  Although the position is remote, face time can be important for collaboration, particularly at the beginning of a design.
  • First response experience, particularly in the backcountry (SAR, fire, ranger, etc).  This is an important and growing customer base for CalTopo, and as with the recreational side, domain knowledge is useful.
  • Mobile development experience. 

Friday, October 13, 2017

Wind Forecast Layer

When I added the weather forecast layer this summer, I mentioned my frustrations with the way the NWS requires you to bring up endless point forecasts to get an accurate picture of local precipitation amounts.  With the Northern California fires, I'm running into the same issue with wind - I'll read about a forecasted red flag warning, but the Napa and Santa Rosa forecasts barely hit double digits.  I'll admit that I had no idea wind speeds were so locally variant - or at least, no idea that the NWS forecast grid captured such small-scale variations.

24hr wind gust plot.  From 15mph in St Helena to 50mph on Mount Hood just 3 miles away.

Fortunately I already had wind speed mapping on the back burner - it didn't get deployed along with the earlier temperature and precipitation work, but most of that code was reusable.  I can't show wind direction - even if I were to render directional arrows on the map, frequent direction changes make it impossible to show a single meaningful 24hr or 36hr wind direction.  However peak forecasted wind speeds and gusts are shown for 1hr, 6hr, 12hr, 24hr and 36hr intervals, using the same green-yellow-orange-red-purple-blue-black gradient scale as the temperature layer.

One option that does provide wind direction is the crowd favorite windy.com.  However, windy does not reflect the small-scale variations in the NWS forecast grid (see below).  As with temperature and precipitation, it's an open question as to how accurate the NWS grid variations are, but I like to provide as much raw data as possible and let users draw their own conclusions.

Windy.com plot of the same location
The wind layer is a checkbox option alongside temp and precip:


The forecast grid option immediately below will also show point speeds (remember, the point is simply the center of a grid square, and the forecast applies to the entire square), and clicking on a point will bring up the hourly weather chart.


I feel like I've pretty much run through the backlog of fire-related items I had sitting around, so I think this will be the last major layer change in response to the Northern California fires.

Update: there are now two wind layers, the "max wind speed" layer as described above, and a "wind plot" that provides forecasted directions and speeds for specific points in times, at 3 hour intervals to 12 hours, and then 6 hour intervals to 36 hours.  The same color chart is used for speed, with short lines tracing direction.  The length of the lines has no meaning.


High Resolution Post-Fire Imagery

Thanks to a generous offer from a contact on the Google Crisis Response team, CalTopo is temporarily displaying high-resolution imagery for portions of the fire-affected areas in Napa and Sonoma counties.  This imagery was taken by DigitalGlobe on Wed the 11th, and although the fires are still ongoing, it provides some degree of insight into what had burned by that point.  While there is a lot of smoke present, in many areas it's clear enough for a house-by-house assessment.

Because the original imagery is false-color infrared (which gives vegetation a strong red tint), I'm converting it to black-and-white to avoid any possible confusion between the red tint and currently burning areas.  I'm told that true-color imagery will be available at some point on Fri, and hopefully I'll be able to upgrade the layer to match.

Available imagery footprints, with fire outlines shown in red.

Step 1: Go to https://caltopo.com/l/16PD

Step 2: Enter your address in the search field at the top of the page and click GO:


Step 3: Use the zoom control at the top left of the map to zoom in:


Step 4: Mouse over the layer selector at the top right (it should read "Hybrid +1"), and in the window that comes up, adjust the "NorCal Fire" opacity slider to expose the pre-fire imagery:


It may also be helpful to temporarily turn on the "Current Fire Activity" checkbox in order to see the fire perimeters:


I've put some temporary infrastructure in place to help handle this project, but if the site is impacted too much, I may need to make some changes.  The shortlink at the top of this article (https://caltopo.com/l/16PD) will always point to the right place - if you share a URL, please share that.

Wednesday, October 11, 2017

NorCal Fire Response

The quick summary:

  • If you are seeking information on the Northern California wildfires, or trying to understand CalTopo's current fire activity layer, please see this blog post.
  • Edit: big thanks to the Google crisis response team for rapidly lifting the map quota.  The OpenLayers warning no longer applies.  CalTopo will be intermittently running on OpenLayers instead of Google Maps.  If the zoom controls at the top left of the map viewer are blue instead of white, Google layers (map, terrain, satellite) will be unavailable, and text based location search will not work.  You can use the search bar to locate lat/long or UTM coordinates, but not named places.
  • The current fire activity layer now includes VIIRS 375m detections as well as MODIS.
  • CalTopo now sports a new mobile UI.  If you have issues with it, please let me know.
The details: in case you've been living under a rock since Sunday night, California is on fire:

Sunday night's VIIRS capture: fires that look like cities

On Tue morning, CalTopo started seeing increased visits from people looking at the fire activity layer, and pageviews rapidly grew to well above normal.  This caused some scaling issues, but barring another order of magnitude in growth, those are temporarily solved with a combination of software fixes and throwing hardware at the problem.


The larger issue is that I ran into the 100k pageview limit for the Google Maps standard plan, and CalTopo stopped loading.  I've fixed this by switching to OpenLayers, which is the map viewer used in CalTopo Offline, but it's not a perfect substitute: in addition to missing Google's map layers, text-based location searches (like "santa rosa, ca") will no longer work.  The version of OpenLayers I'm using is also old and heavily customized, so there may be some compatibility issues with touch gestures on mobile devices.

I'm working with Google on fixing the issue, but in the meantime, I'll turn on Google Maps at some point in the morning, and leave it on until the quota runs out.  If the zoom +/- buttons at the top left of your screen are white, you're using the Google Maps viewer and everything should function normally.  If they're blue, it means you're using OpenLayers.

In order to better serve people looking for fire information, I've also held a marathon coding session to wrap up and deploy two back-burner upgrades.  The first is the addition of VIIRS 375m fire detections alongside MODIS.  At more than double MODIS's resolution, VIIRS should provide a more accurate picture of the fires' behavior.  Although VIIRS suffers from a limited number of passes (just like MODIS), adding a second source will increase the layer's all-important update frequency.  And certainly the most common question I've gotten today is "when will the layer update"?

Looks just like MODIS, but with more points

The current fire activity layer was already customizable (smoke vs no smoke); now you can choose all sources (a mix of MODIS and VIIRS on the same map) or view them one at a time for a cleaner image.


The second change is a revamped mobile UI.  Normally I'd hang on to a change like this for another month or two in order to test it thoroughly and smooth out the edges.  However, the vast majority - > 90% - of people seeking fire information are doing so on their phones.  And, the previous mobile UI was non-functional enough that I'm unlikely to make things worse.  I know it's 2017 already, but now the CalTopo mobile site supports 90% of the features the desktop site does.  More details in a later post once I get things cleaned up, but in the meantime let me know of any bugs.