Hill Shading support

[springmeyer]

I grabbed the patched from here and am attaching a diff against trunk:

https://lists.berlios.de/pipermail/mapnik-users/2009-February/001651.html

[artem]

Applied in r1027

(Marcin Rudowski)

After seeing hill shading in action, I did some research too in this matter. I based my work on: http://wiki.openstreetmap.org/wiki/HikingBikingMaps and again came across some issues with Mapnik.

Mapnik problems: - very basic support for rasters, no opacity, no advanced merging, only replacing already drawn background. - no raster scaling, only nearest neighbor. I need scaling up to 64x (12-18 zoom level) and mapnik generated only big squares - despite my last patch, 256 color palette was still bad for slow gradients like shading.

I corrected, in some degree, all of above limits, and want to share with You.

Here are some examples of my work in action: http://mapa.ump.waw.pl/ump-www/?zoom=11&lat=49.70878&lon=19.28666 http://mapa.ump.waw.pl/ump-www/?zoom=15&lat=49.30126&lon=19.94725 http://mapa.ump.waw.pl/ump-www/?zoom=13&lat=52.56388&lon=19.70792 http://mapa.ump.waw.pl/ump-www/?zoom=11&lat=51.29269&lon=19.3567 http://mapa.ump.waw.pl/ump-www/?zoom=11&lat=54.61966&lon=18.23952 http://mapa.ump.waw.pl/ump-www/?zoom=11&lat=53.2975&lon=18.36724 http://mapa.ump.waw.pl/ump-www/?zoom=14&lat=49.39029&lon=22.47346

I use SRTM-3 v2: ftp://e0srp01u.ecs.nasa.gov/srtm/version2/SRTM30/ converted to mercator projection (gdal) avoiding elipsoid->spheroid reprojection (only latlon->mercator) with pixel size: 76.437 x 76.437 m. Based on srtm-3 resolution, it should be 90x90, but alignment to output resolution was needed. 76.437 is closest number to 90 from (2*pi*6378137)/2^{n set. For Poland, this gives 16000x14000 GeoTIFF in resolution aligned to zoom 11. I made scaled copies 50% and 25% for lower zooms, to increase speed and avoid downscaling. I could do the same to avoid upscaling, but already it is 250MB file, and each zoom would make it 4x bigger then previous.}

Patches are in attachment (hopefully this time :) ), here I will describe in short, what is changed, and what can be done more.

1) Raster opacity

Added opacity handling using existing methods. It isn't needed for shading, but can be useful. To enable opacity, add css style opacity to RasterSymbolizer. It works both for cairo and internal agg renderer.

2) Raster merging

Added some basic merging methods. I use for hill shading which is gimp like "merge grain": http://docs.gimp.org/en/gimp-concepts-layer-modes.html#id2834930 Default is normal mode, to select differently, set css "mode" to desired value. Available modes based on given link: - grain_merge : bg + fg - 0.5 - grain_merge2 : bg + fg*2 - 1.0 - multiply : fg * bg - multiply2 : fg * bg * 2.0 - divide : bg / fg - divide2 : bg * 2.0 / fg - screen : 1 - (1-fg)(1-g) - hard_light : ..... (look ad link above)

Agg renderer have some similar code in agg/include/agg_pixfmt_rgba.h but I didn't use it. I wrote my code based on existing part for alpha blending from mapnik/include/mapnik/graphics.hpp. This fix works only with agg renderer, I couldn't find appropriate way to implement it using Cairo. I googled and found some references to filters for ie. hard light, but no documentation or examples, and I'm not sure if it exists in cairo. This fix and previous probably handle: http://trac.mapnik.org/ticket/157 if someone could implement it also in cairo, or at least show me right direction.

3) Raster scaling

Added bilinear scaling algorithm. Algorithm can be selected with css "scaling" parameter and available values are: - fast : nearest neighbor as it was already - bilinear : new bilinear interpolation for all 4 chanels (RGBA) - bilinear8 : as bilinear, but only one channel assumed. This probably needs some fix, as it doesn't handle alfa. I added it to speedup in my case, where raster is grayscale without transparency.

Scaling in cairo renderer could be reimplemented using matrix, but I use mapnik-0.5.1 with my patches, so basically I concentrated on agg renderer.

4) 256 color palette optimizations

My previous patch removed obvious bug in octree. Still, generated palette is not enough for hill shading. It looks more like 50 colors, then 256. The problem is with prunning only on lowest level, so all left leafs are almost on the same level. It is similar to dividing to even ranges, regardless of density. My patch prunes only branches with lowest pixel count from all levels, so subtrees with many leafs, remain deeper, then one with smaller sample count.

Some limiting was needed, to allow colors with small sample count to remain unharmed (ie, some distinctive feature on map with not enough pixels, to "fight" with gradients should be preserved). I didn't find any significant increase in processing time, and only 5-10% increase in size (due to more significant colors). Effects are almost as good, as optimal palette without error-diffusion in imagemagick or Gimp.

Still there are problem with color assignment, because quite often cube for classified color doesn't have the best color. Neighbor cubes should be compared, but traversing octree is fast, but not free. I made some fix, to check siblings and neighbour cubes (going up and down the three), but it increased processing time of quantization about 40% and effects aren't so much worse without it. It isn't included in attachment.

Example osm.xml fragment:

<Style name="raster18">
     <Rule>
         <MaxScaleDenominator>1000000</MaxScaleDenominator>
         <MinScaleDenominator>250000</MinScaleDenominator>
         <RasterSymbolizer>
             <CssParameter name="opacity">1.0</CssParameter>
             <CssParameter name="scaling">bilinear</CssParameter>
             <CssParameter name="mode">grain_merge</CssParameter>
         </RasterSymbolizer>
     </Rule>
</Style>
<Layer name="dem18" status="on" >
     <StyleName>raster18</StyleName>
     <Datasource>
         <Parameter name="type">gdal</Parameter>
         <Parameter name="file">&geotiffs;/hill_15m.tif</Parameter>
         <Parameter name="format">tiff</Parameter>
     </Datasource>
</Layer>

Attachments:

- mapnik.mr.palette2.patch.gz : palette fix 4) - mapnik.mr.scale.patch.gz : scaling methods - mapnik.mr.raster1.patch.gz : parsing RasterSymbolizer rule - mapnik.mr.process.patch.gz : using raster params to select scaling algorithm, opacity and merging method.

Additional notes:

If scaling and tiling, there can be some problems with alignment if raster resolution isn't integer multiply of output resolution and rendered regions have margin, like in mod_tile: A---------B---|---A-----------B Because size and position of source raster bbox is averaged to pixel steps, error can be different on each edge and '|' in A image could be displaced in B image. 0.5 pix in raster could mean 16px after 32x scaling (5 zoom steps) and this is noticeable even earlier.

I use raster in resolution of zoom 11, so scaling is power of 2 and any errors are same on each rendered edge. In my case, mod_tile renders in each zoom: 9*256 x 9*256 region. This is cut with 128 margin to 8x8 tiles, each 256x256 size. Everything is scaled by power of 2, and everything is aligned.

It shouldn't(?) matter if we consider no margin tile joining: A--------AB---------B.

Adding hill shading to my Mapnik using raster increased generation time only about 2x (35tiles/s -> 17 tiles/s).

In my case map is updated ~twice a week and only 300k are forced to redraw and send to cache server.

I use self-patched mapnik 0.5.1 with some 3 months old version of mod_tile for apache. Actually I have two mod_tile modules, to handle two separate osm.xml and two databases, which I switch on update.

My osm.xml is divided to separate files defining layers, styles, db settings etc. Styles are grouped by categories. In effect I have 22 xml files that sum up to 420kB. Thank You for libxml2 features, or I would go insane managing this in one osm.xml :)

[cmarqu]

There is a demo showing the principle working in http://mapnik-utils.googlecode.com/svn/sandbox/hillshading/

(Run get_srtm_data.sh first, then render_map.py, then look at hillshading.png)