Bug#695773: sgmlop.so breaks xmlrpclib numerical character entities

[Frank Gevaerts]

On Thu, Dec 13, 2012 at 09:29:00AM +0100, Matthias Klose wrote:
> severity 695773 important
> thanks
> 
> Am 12.12.2012 17:04, schrieb Frank Gevaerts:
> > Package: python-reportlab-accel
> > Version: 2.4-4
> > Severity: critical
> > Justification: breaks unrelated software
> > 
> > If python-reportlab-accel (or more precisely, the sgmlop.so file it
> > includes) is installed, software using the python xmlrpclib library
> > rejects valid xml data containing numerical character entities (such
> > as e.g. °) with "UnicodeDecodeError: 'utf8' codec can't decode byte
> > 0xb0 in position 4: invalid start byte".
> 
> why would that be? please could you give an example?
> 

Using the attached files, and with python-reportlab-accel installed:

Run the sample server:

$ ./server.py

Do a utf8-encoded xmlrpc request to demonstrate that "normal" cases work:

$ curl --data @working.xml http://localhost:8000/
<?xml version='1.0'?>
<methodResponse>
<params>
<param>
<value><string>°</string></value>
</param>
</params>
</methodResponse>

Do an xmlrpc request that uses numerical character entities:

$ curl --data @notworking.xml http://localhost:8000/
<?xml version='1.0'?>
<methodResponse>
<fault>
<value><struct>
<member>
<name>faultCode</name>
<value><int>1</int></value>
</member>
<member>
<name>faultString</name>
<value><string>&lt;type 'exceptions.UnicodeDecodeError'&gt;:'utf8' codec
can't decode byte 0xb0 in position 0: invalid start
byte</string></value>
</member>
</struct></value>
</fault>
</methodResponse>

Now stop server.py, and either move /usr/lib/python2.6/dist-packages/sgmlop.so
out of the way or uninstall python-reportlab-accel.

Run the server again:

$ ./server.py

Do the second (problematic) request again:

$ curl --data @notworking.xml http://localhost:8000/
<?xml version='1.0'?>
<methodResponse>
<params>
<param>
<value><string>°</string></value>
</param>
</params>
</methodResponse>

server.py was slightly adapted (because I needed a string argument)
from the sample code on http://docs.python.org/2/library/xmlrpclib.html
The code does not use anything related to python-reportlab.

Note that working.xml and notworking.xml are perfectly equivalent. Using
e.g. xmlling -encode ascii or xmlling -encode utf8 you can convert one
into the other.

I've now also tried on unstable (with version 2.5-1.1), and the problem
still occurs there.

Frank

-- 
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan

#!/usr/bin/python2.6

import xmlrpclib
from SimpleXMLRPCServer import SimpleXMLRPCServer

def do_print(n):
        print n
        return n

server = SimpleXMLRPCServer(("localhost", 8000))
print "Listening on port 8000..."
server.register_function(do_print, "print")
server.serve_forever()

working.xml
Description: XML document

notworking.xml
Description: XML document