What is TopoGrabber?

TopoGrabber is a package designed to simplify the task of obtaining high resolution USGS datasets in formats readable by modelling and analysis software packages. Files are downloaded in GeoTIFF format, then converted to raw tiles for the WRF preprocessing system (WPS), and NetCDF tiles for the Local Analysis and Prediction System (LAPS) and the Space and Time Multiscale Analysis System (STMAS).

The LAPS/STMAS interface with Topograbber is still experimental. The tiles produced may need some modifications to the 'gridgen_model.exe' program so they can be read in. A second way to use this with LAPS is to generate a WPS terrain file, and then read that in during the LAPS/STMAS localization process.

Why download high resolution datasets? For regional severe storm, fire weather, and accurate wind farming forecasts, high resolution topography is crucial. And since future modeling efforts will expect access to this improved topography and land use data, studying its affect on model performance and forecast accuracy is worthwhile.

Although TopoGrabber was created with high-res topography and land use data in mind, it can be used to obtain any available USGS data in any available format. The program has been tested primarily with the National Elevation Dataset (NED) and the National Land Cover Dataset (NLCD).

Download TopoGrabber

TopoGrabber 1.0 is the current beta release of TopoGrabber. Please send any questions, comments, or bug reports to topograbber@gmail.com.

  Latest Release

Version 1.0 for Linux, Mac, and Windows - released July 22, 2010:

Package Components

The contents of the TopoGrabber package are as follows:

Filename Purpose
TopoGrab.py Command line program that downloads domain specific geotiff data from multiple online sources
TopoGrabber.py GUI workflow wrapper for the entire package.
convert_geotiff.x Converts geotiff data into raw WPS tiles and NetCDF tiles
mechanize Browser-like web scaper
Beautiful Soup HTML parser for scraping the USGS Seamless Server

Installing TopoGrabber

System Requirements

Note that the LibTIFF and LibGeoTIFF libraries are included in TopoGrabber.zip. Only Python is required to download data from the USGS server. Everything else is required only for the conversion of GeoTIFF files to other formats.

Installation Instructions

To install TopoGrabber, simply run the INSTALL script. Doing so compiles convert_geotiff.x and creates a symbolic link to it for Topograb.py or TopoGrabber.py.

Additionally, environment variables pointing to the GeoTIFF library must be set before running convert_geotiff.x.

Setting environment variables in the Bash shell:

export LD_LIBRARY_PATH="/home/you/src/lib_geotiff/lib"
export LD_RUN_PATH="/home/you/src/lib_geotiff/lib"

Setting environment variables in the C shell:

setenv LD_LIBRARY_PATH /home/you/src/lib_geotiff/lib
setenv LD_RUN_PATH /home/you/src/lib_geotiff/lib

Using TopoGrabber With WPS

The steps involved in utilizing TopoGrabber to obtain elevation data for the Weather Reasearch and Forecasting Model (WRF) are listed below.

  1. Create a directory under your geogrid data root called topo_1s.
  2. Replace or link your GEOGRID.TBL (located under the /WPS/geogrid/ directory) with our GEOGRID.TBL.HRTOPO file. This will tell geogrid that there is now a directory under its geogrid data root called /topo_1s with your data in it. Modifications to GEOGRID.TBL are needed if you wish to use additional high-resolution datasets.
  3. Run TopoGrab.py or TopoGrabber.py
  4. Take the WPS raw files (eg xxxxx-XXXXX.yyyyy-YYYYY) that were created and put them in your topo_1s directory under the Geogrid data path (eg /bin/WPS/GEOG/topo_1s).
  5. Run geogrid.exe.

If your domain lies within the downloaded data set geogrid.exe should run through and output the high-resolution dataset into your geo_em files.

Example of Usage With The WRF Preprocessing System (WPS):

The following usage example downloads data for a Boulder domain (defined in domains.py) in GeoTIFF format from the National Elevation Dataset (ned3, defined in dataSources.py), then converts the data to a single geogrid raw tile.

python TopoGrab.py -o "output" -s ned3 -d boulder -x
cp ./output/* ../../WPS/GEOG/topo_1s
cd /path/to/WPS

The command line arguments that can be supplied to TopoGrab.py are documented in the README file. If TopoGrab.py is run without any arguments, the user will be prompted for input.


Interface and Workflow

High Resolution Elevation and Land Cover Examples


Future Work

Preliminary comparisons between model runs initialized with 1/3 arcsecond and 30 arcsecond elevation data have produced significant differences in wind currents. An investigation of the impact of high resolution topography on model performance is underway.

Other future plans include improving map geometry conversions, improving performance in very large dataset downloads, and adding support for more output formats.

Credits and Acknowledgments

TopoGrabber was created by Wesley S. Smith and Brad E. Beechler at the NOAA Earth Systems Research Laboratory. The following individuals also deserve credit for their contributions or support.

  • Jonathon D. Beezley, author of the autoWPS package for WRF Fire.
  • Haiyan Cheng, Willamette University Computer Science Department
  • Isidora Jankov, Yuanfu Xie and Huiling Yuan, NOAA ESRL Global Systems Division Forecast Applications Branch

Page Contents

Contact Us

Submit any questions, comments, or bug reports relating to TopoGrabber to topograbber@gmail.com, or use the form below.