Filter Condition String Whitespace is Stripped

[icio]

Problem

There is an indifference in the way that the strings in Filters and the strings from the datasource are handled.

With a filter object, whitespace is trimmed from the matching value:

>>> from mapnik import *
>>> filter = Filter("[name]='value    '")
>>> print filter
([name]='value')

Whitespace is not trimmed from the data (the following examples were selected from a 'character' [not 'character varying'] column in a Postgres DB):

"Height Control             "
"Inland Water               "
"Political Or Administrative"

This results in filters not matching data it should.

Solution

There are two possible obvious solutions. Either the filter condition should not be trimmed or the data should be trimmed.

I propose trimming the data to conform with Quantum GIS which takes this approach and allow continued compatability with Quantumnik exports.

[artem]

Replying to icio:

== Problem == There is an indifference in the way that the strings in Filters and the strings from the datasource are handled. With a filter object, whitespace is trimmed from the matching value: {{{

from mapnik import * filter = Filter("[name]='value '") print filter

([name]='value') }}} Whitespace is not trimmed from the data (the following examples were selected from a 'character' [not 'character varying'] column in a Postgres DB): {{{ "Height Control " "Inland Water " "Political Or Administrative" }}} This results in filters not matching data it should. == Solution == There are two possible obvious solutions. Either the filter condition should not be trimmed or the data should be trimmed. I propose trimming the data to conform with Quantum GIS which takes this approach and allow continued compatability with Quantumnik exports.

It looks like trimming was removed : http://trac.mapnik.org/log/trunk/plugins/input/postgis/postgisfs.cpp?rev=650&stop_rev=617

I added trimming for 'bpchar' but not for 'text' and 'vchar'. If there are some reasons for this approach not to work?

[springmeyer]

Icio,

have you tested this yet?

[springmeyer]

noting that the 'bpchar' fix was added in r1184

[springmeyer]

Okay, I did some checking with postgres folks and based on their thoughts I support Artem's fix for trimming bpchar and nothing else.

RhodiumToad: trimming whitespace from anything other than a bpchar is probably unwise
RhodiumToad: the character() type is strictly for cobol programmers
[10:30am] RhodiumToad: (with some _very_ rare exceptions - and for those exceptions, you do NOT want to trim whitespace)

Best practices don't call for trimming fixed width field types, so it seems that both the data needs to be using varchar and QGIS needs to avoid trimming whitespace for fields other than bpchar.

[springmeyer]

Interesting note from Jef in #qgis:

jef: springmeyer: looks like it's PQgetvalue that strips. psql shows the same behaviour, see http://paste.debian.net/41363/

that paste is:

qgis_temp=> create table foobar(x varchar(80), y char(80));
CREATE TABLE
qgis_temp=> insert into foobar(x,y) values ('','');
INSERT 0 1
qgis_temp=> insert into foobar(x,y) values ('                         ','                      ');
INSERT 0 1
qgis_temp=> select '|'||x||'|', '|'||y||'|' from foobar;
          ?column?           | ?column? 
-----------------------------+----------
 ||                          | ||
 |                         | | ||
(2 rows)