GeoServer expects URL-encoded parameters as UTF-8 regardless of the host operating system character set. This is correct.
Some languages can encode characters in different ways in Unicode, for example using Unicode combining characters. This can affect comparisons. Could your data have different bytes to your filter literal? What is the exact byte sequence stored in your shapefile for the feature with the property with value 'سامی'? That is, does it have the same UTF-8 byte sequence? Do you see the same bytes in a WFS response? What is your servlet container? Tomcat or the embedded Jetty? There have been reports of CQL encoding issues with some Tomcat configurations. Quite old but may still be valid: http://osgeo-org.1560.x6.nabble.com/CQL-Filter-Syntax-td3806188.html https://sourceforge.net/p/geoserver/mailman/message/21111455/ One reported fix for Tomcat is to set URIEncoding="UTF-8" on your <Connector> in server.xml, but I have not tried this myself: https://wiki.apache.org/tomcat/FAQ/CharacterEncoding Kind regards, Ben. On 06/08/16 15:26, Saka Royban wrote: > > Thanks a lot Ben for your helpSo, if CQL-filter can handle non-ASCII > characters, it is misunderstanding of me. I've may done a step wrongly. > 1. Yes. My shapefile store is declared as UTF-8 charset in GeoServer. Plain > wfs requests returns correct data (=correct non-english characters). In fact, > there is no problem without CQL-filter. To add more, my shapefile is an > output of ArcGIS which in its recent versions only exports to UTF-8. (There > is .cpg file which has been set to UTF-8, after export.) > 2. My system encoding is Windows-1256 (the encoding that is used by default > in word editors). My webpage is declared in UTF-8. So, my javascript codes > must be in UTF-8 (I've also set encoding in notepad++ to UTF-8. If i change > that to Win-1256, i see some dummy characters which proves me that my filter > string is encoded in UTF-8 correctly) > 3. OpenLayers sends following string as part of URL to > server:CQL_FILTER=NAME%20%3D%20%27%D8%B3%D8%A7%D9%85%DB%8C%27Using online > decoders, i get back correct filter string. > 4. About log file, i tried different encoding in Notepad++ but unfortunately > i can't see the filter string correctly. My encodings are restricted to two: > UTF-8 and Win-1256. Filter string isn't displayed correctly in none of them. > I've also tried another shapefile in Win-1256 encoding. Changing meta tag of > webpage to chareset=windows-1256, writing againg filter string but in this > case, i can't get a good result as well.(Note: I noticed that Openlayers > encodes filter in win-1256 exactly as it was in UTF-8. Is it right?) > Thanks again for your help and patience > > From: Ben Caradoc-Davies <[email protected]> > To: Saka Royban <[email protected]>; GeoServer Mailing List List > <[email protected]> > Sent: Saturday, August 6, 2016 1:29 AM > Subject: Re: [Geoserver-users] GeoServer doesn't understand non-english > Characters in cql filter > > Yes, non-ASCII characters are supported in CQL and XML filters. > Successful filtering also requires correct configuration of your data > source. > > What charset is declared in your GeoServer shapefile data store > configuration? Is it UTF-8? > http://docs.geoserver.org/latest/en/user/data/vector/shapefile.html > > What is your platform encoding (the charset of your operating system)? > Log files are written in the encoding of your platform, so if your > platform cannot encode these characters, placeholders characters will be > written, even when filtering is working correctly. > > Kind regards, > Ben. > > On 06/08/16 05:09, Saka Royban wrote: >> Hi all have a shapefile loaded in GeoServer 2.8 and accessed by OpenLayers >> 2.13. This layer is in UTF-8 encoding and has some english and non-english >> records. >> When i use english fields in cql-filter - as follows, but no differnce >> between WMS and WFS request- GeoServer filters data very well and returns >> the result. layer1 = new ol.layer.Image({ >> title: 'Ostan', >> source: new ol.source.ImageWMS({ >> url: 'http://localhost:8080/geoserver/wms', >> params: { >> LAYERS: 'ws:layer1', >> VERSION: '1.1.1', >> CQL_FILTER: "NAME_ENG = 'sommy'" >> } >> }) >> });But if i try to use a field which contains non-english data in >> cql-filter, GeoServer returns no data. My webpage has following meta, as >> well: <meta content="text/html; charset=utf-8" http-equiv="content-type" >> />When i check geoserver log, i see some messy characters instead of my >> non-english string in cql-filter. >> | >> | | >> >> | Using Firebug, in Parameteres section, everything is correct. I see >> "CQL-Filter: Name = 'سامی' " which is right for my language. But in >> Geoserver log, i see some dummy characters instead of abovementioned >> characters >> Can anyone confirm that using characters other than ASCII is possible in >> CQL-Filter? >> >> Regards | >> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> >> _______________________________________________ >> Geoserver-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/geoserver-users >> > -- Ben Caradoc-Davies <[email protected]> Director Transient Software Limited <http://transient.nz/> New Zealand ------------------------------------------------------------------------------ _______________________________________________ Geoserver-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users
