Add OGR.input plugin to read all OGR supported vector formats

[springmeyer]

This has been requested a number of times on the mailing list.

[kunitoki]

Initial support (untested)

[kunitoki]

This is the ogr support for features. It is untested cause i have problems with manual compilation of mapnik here (but at least it compiles).

What i have done so far:

• ogr_dataset (instantiation of ogr, define extent)
• ogr_feature (conversion using wkb) with iteration

What could be improved:

• general testing
• add support fro more input parameters (for oracle data sources, files, etc.)
• make use of OGRDataSource->ExecuteSQL in mapnik query features
• implement query_at_point

[springmeyer]

I just rebuilt with this patch. Good stuff!

Compile went great.

Creating a layer worked!

>>> from mapnik import *
>>> CreateDatasource
>>> CreateDatasource({'type':'ogr','file':'/Users/spring/src/mapnik/demo/data/boundaries.shp','layer':'boundaries'})
<mapnik.Datasource object at 0x87670>

But via python I crashed with a bus error on mac 10.5 when I specified a layer that did not exist:

>>> lyr = CreateDatasource({'type':'ogr','file':'/Users/spring/src/mapnik/demo/data/boundaries.shp','layer':'foo'})
Bus Error

And got a segmentation fault trying to call dir on the layer object:

>>> dir(lyr)
Segmentation fault

[springmeyer]

OGR input plugin as patch

[kunitoki]

i've fixed the initial draft of the input plugin. the rundemo.cpp is working as expected and produces the png we want.

awaiting merge into trunk for better testing and improvement.

[kunitoki]

patch here (trac is diying when uploading an attachment here):

http://www.anticore.org/tmp/ogr_input_plugin_final.patch

[springmeyer]

Awesome, applied with minor modifications in r837

[springmeyer]

some basic tests(via python) available here: http://mapnik-utils.googlecode.com/svn/sandbox/ogr_plugin/

[springmeyer]

kunitoki, Okay I also applied the fixes for cairo, but not your changes to rundemo.cpp.

Can you please post a separate patch for that to #197 ?

Thanks!

[springmeyer]

I'm able to crash python when trying to read from a .gpx file:

Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Codes: KERN_PROTECTION_FAILURE at 0x0000000000000008
Crashed Thread:  0

Thread 0 Crashed:
0   libmapnik.dylib               	0x010db5c9 mapnik::wkb_reader::read_linestring(mapnik::feature<mapnik::geometry<mapnik::vertex<double, 2> >, boost::shared_ptr<mapnik::raster> >&) + 281
1   libmapnik.dylib               	0x010dbdad mapnik::wkb_reader::read(mapnik::feature<mapnik::geometry<mapnik::vertex<double, 2> >, boost::shared_ptr<mapnik::raster> >&) + 605
2   libmapnik.dylib               	0x010d9602 mapnik::geometry_utils::from_wkb(mapnik::feature<mapnik::geometry<mapnik::vertex<double, 2> >, boost::shared_ptr<mapnik::raster> >&, char const*, unsigned int, bool) + 258
3   ogr.input                     	0x02d36151 ogr_featureset::next() + 401
4   _mapnik.so                    	0x00473a7e mapnik::feature_style_processor<mapnik::agg_renderer<mapnik::Image32> >::apply_to_layer(mapnik::Layer const&, mapnik::agg_renderer<mapnik::Image32>&, mapnik::projection const&, double) + 2318
5   _mapnik.so                    	0x0046df9f render(mapnik::Map const&, mapnik::Image32&, unsigned int, unsigned int) + 335
6   _mapnik.so                    	0x0046e19c render_to_file2(mapnik::Map const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 76
7   _mapnik.so                    	0x0047241c boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void (*)(mapnik::Map const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), boost::python::default_call_policies, boost::mpl::vector3<void, mapnik::Map const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&> > >::operator()(_object*, _object*) + 220
8   libboost_python-mt-1_35.dylib 	0x006a9d64 boost::python::objects::function::call(_object*, _object*) const + 468
9   libboost_python-mt-1_35.dylib 	0x006ab365 boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) + 37
10  libboost_python-mt-1_35.dylib 	0x006b1b39 boost::function0<void, std::allocator<boost::function_base> >::operator()() const + 41
11  libboost_python-mt-1_35.dylib 	0x006b19f0 boost::python::detail::exception_handler::operator()(boost::function0<void, std::allocator<boost::function_base> > const&) const + 256
12  _mapnik.so                    	0x0046e734 boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<mapnik::config_error, void (*)(mapnik::config_error const&)>, boost::_bi::list3<boost::arg<1> (*)(), boost::arg<2> (*)(), boost::_bi::value<void (*)(mapnik::config_error const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void, std::allocator<boost::function_base> > const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void, std::allocator<boost::function_base> > const&) + 36
13  libboost_python-mt-1_35.dylib 	0x006b161f boost::python::handle_exception_impl(boost::function0<void, std::allocator<boost::function_base> >) + 63
14  libboost_python-mt-1_35.dylib 	0x006a77ea function_call + 90

[kunitoki]

Replying to springmeyer:

kunitoki, Okay I also applied the fixes for cairo, but not your changes to rundemo.cpp. Can you please post a separate patch for that to #197 ? Thanks!

yeah sure, i can package a more detailed rundemo with all the input plugins under testing

[kunitoki]

Replying to springmeyer:

I'm able to crash python when trying to read from a .gpx file.

The problem could be in the wkb transcode (ogr_featureset.cpp @ line 75):

char data[size];
geom->exportToWkb ((OGRwkbByteOrder) endian(), reinterpret_cast<unsigned char*>(&data[0]));
geometry_utils::from_wkb (*feature, &data[0], size, multiple_geometries_);

Probably not the best way to do this. Another way could be a big switch around the ogr geometry type to determine the derived class, then iterate over the points and initialize the mapnik feature from there (more longer to write but probably more faster, one less conversion). Is wkb standard in both implementations ?

Anyway would be cool if you can feed ogr2ogr with the problematic file and see if it is a problem of ogr in the first place...

[springmeyer]

Replying to kunitoki:

Yes, feeding ogr2ogr shows some problems with the file (but it converts okay):

File is this one: http://mapnik-utils.googlecode.com/svn/sandbox/ogr_plugin/data/test_point_line.gpx

ogr2ogr -f 'ESRI Shapefile' test test_point_line.gpx 
Warning 6: Field time create as date field, though DateTime requested.

Warning 1: GPX driver only reads 2 links per element. Others will be ignored. This can be changed with the GPX_N_MAX_LINKS environment variable
Warning 6: Field time create as date field, though DateTime requested.

Warning 6: Field time create as date field, though DateTime requested.

[springmeyer]

Another issue is that something is going wrong with the TextSymbolizer when reading from the ogr plugin.

See the new tests here: http://mapnik-utils.googlecode.com/svn/sandbox/ogr_plugin/

and the resulting map here: http://mapnik-utils.googlecode.com/svn/sandbox/ogr_plugin/maps/world_population_ogr.png

which should look like: http://trac.mapnik.org/wiki/XMLGettingStarted

[kunitoki]

patch on the current SVN revision

[kunitoki]

Ok here it is another version (4th):

• improved handling of geometries
• implementation of feature_at_point
• fixed problems with labels showing up as 1 (bad ogr string fields conversion)
• general code cleanup and reorder

[springmeyer]

Okay, this 4th patch was applied in r841 and with that all basic tests pass, labeling, filters, feature_at_point.

Closing this ticket. Thanks kunitoki!

[kunitoki]

rundemo with OGR instead of shape input

[kunitoki]

i hope this is the last !

[springmeyer]

applied 6th patch in r871 (to support the encoding parameter)