When loading from json datas with simplejson, strings are unicode .
Le Thu, 18 Nov 2010 11:22:30 +0100, Romaric DEFAUX <[email protected]> a écrit : > Hi again, > > I try simplejson, but I have a problem. (I just replace pickle > everywhere by simplejson) > > Here's a test I did : > # python > Python 2.5.2 (r252:60911, Jan 20 2010, 21:48:48) > [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import simplejson > >>> disk_list = [] > >>> disk = {} > >>> disk['name'] = "disk1" > >>> disk['size'] = "52428800" > >>> disk_list.append(disk) > >>> print disk_list > [{'name': 'disk1', 'size': '52428800'}] > >>> disk = {} > >>> disk['name'] = "disk2" > >>> disk['size'] = "647648" > >>> disk_list.append(disk) > >>> print disk_list > [{'name': 'disk1', 'size': '52428800'}, {'name': 'disk2', 'size': '647648'}] > >>> disk_list_json = simplejson.dumps(disk_list) > >>> print disk_list_json > [{"name": "disk1", "size": "52428800"}, {"name": "disk2", "size": "647648"}] > >>> disk_list2 = simplejson.loads(disk_list_json) > >>> print disk_list2 > [{u'name': u'disk1', u'size': u'52428800'}, {u'name': u'disk2', u'size': > u'647648'}] > >>> print repr(disk_list_json) > '[{"name": "disk1", "size": "52428800"}, {"name": "disk2", "size": > "647648"}]' > > Explanation : > I "dumps" with json a list of dictionnaries into disk_list_json. > When I "loads" it, I don't get my list of dictionnaries like before (see > disk_list2). It adds "u" letter everywhere. > Why ? > Thanks for help ! > > Romaric Defaux > > Le 18/11/2010 09:43, Michael Ricordeau a écrit : > > Hi, > > > > you can use json for passing list and dict . > > Pickle is dangerous . > > > > Instead of pickle.loads/pickle.dumps use json.loads and json.dumps > > (using stdlib json in python>= 2.6 or simplejson in python< 2.6) > > > > Regards > > > > > > > > Le Thu, 18 Nov 2010 09:29:00 +0100, > > Romaric DEFAUX<[email protected]> a écrit : > > > > > >> Le 17/11/2010 18:52, geremy condra a écrit : > >> > >>> On Wed, Nov 17, 2010 at 6:44 AM, Romaric DEFAUX<[email protected]> wrote: > >>> > >>> > >>>> Le 16/11/2010 17:47, Romaric DEFAUX a écrit : > >>>> > >>>> > >>>>> Hi everybody ! > >>>>> > >>>>> First time I write to this mailing list :) > >>>>> I started writing in python last week, that's probably why I can't > >>>>> understand the following problem... > >>>>> > >>>>> > >>>>> I create a list called web_site_list. > >>>>> This list contain dictionaries called web_site. > >>>>> And some values in this dictionaries are list too. > >>>>> > >>>>> I do that in a function and I return this : > >>>>> return pickle.dumps(web_site_list) > >>>>> > >>>>> This is working fine :) > >>>>> > >>>>> If I do : > >>>>> print "%s" % pickle.loads(system.get_web_site_list()) > >>>>> > >>>>> I've got the right stuffs. For example it returns : > >>>>> [{'documentroot_size': '120', 'servername': '---default---', 'client': > >>>>> 'undefined', 'documentroot': '/var/www/', 'client_contact': 'undefined', > >>>>> 'serveralias': []}] > >>>>> > >>>>> I send this to a web service. I send it like that : > >>>>> #I put it in params > >>>>> def system_updateweb_site(server, login, password): > >>>>> params = {} > >>>>> params['login'] = login > >>>>> params['password'] = password > >>>>> params['action'] = 'updateweb_site' > >>>>> params['servername'] = get_servername() > >>>>> params['hosted_web_site'] = get_web_site_list() > >>>>> return call_system_ws(server, params) > >>>>> > >>>>> #Here's how I send it (I tried in GET and POST) > >>>>> def call_system_ws(host, params): > >>>>> query_string = urllib.urlencode(params) > >>>>> #GET > >>>>> # f = urllib.urlopen("http://%s/ws?%s"; % (host, query_string)) > >>>>> #POST > >>>>> f = urllib.urlopen("http://%s/ws"; % (host), query_string) > >>>>> result = f.readline().strip() > >>>>> if result == 'ERROR': > >>>>> msg = f.readline().strip() > >>>>> return (False, msg) > >>>>> return (True, result) > >>>>> > >>>>> > >>>>> On the server side : > >>>>> if action == 'updateweb_site': > >>>>> if not (fields.has_key('servername') > >>>>> and > >>>>> fields.has_key('hosted_web_site')): > >>>>> raise WSError('missing > >>>>> parameter : > >>>>> servername or hosted_web_site') > >>>>> log ('ERROR : missing > >>>>> parameter : > >>>>> servername or hosted_web_site') > >>>>> else: > >>>>> > >>>>> servername=g.db.escape_string(fields['servername']) > >>>>> > >>>>> hosted_web_site=g.db.escape_string(fields['hosted_web_site']) > >>>>> output = > >>>>> systemserver.updateweb_site(cursor, servername, hosted_web_site) > >>>>> > >>>>> In systemserver.py : > >>>>> def updateweb_site(cursor, host, hosted_web_site): > >>>>> web_site_list = pickle.loads(hosted_web_site) > >>>>> return "%s" % (web_site_list) > >>>>> > >>>>> I catch this error :* > >>>>> > >>>>> <type 'exceptions.EOFError'>*: > >>>>> > >>>>> args = () > >>>>> message = '' > >>>>> > >>>>> Why ? > >>>>> > >>>>> If I just print hosted_web_site, I get this on my web page : > >>>>> > >>>>> > >>>>> (lp0\n(dp1\nS\'documentroot_size\'\np2\nS\'120\'\np3\nsS\'servername\'\np4\nS\'default\'\np5\nsS\'client\'\np6\nS\'undefined\'\np7\nsS\'documentroot\'\np8\nS\'/var/www/\'\np9\nsS\'client_contact\'\np10\ng7\nsS\'serveralias\'\np11\n(lp12\nsa. > >>>>> > >>>>> It's the "pickled view" of > >>>>> [{'documentroot_size': '120', 'servername': '---default---', 'client': > >>>>> 'undefined', 'documentroot': '/var/www/', 'client_contact': 'undefined', > >>>>> 'serveralias': []}] > >>>>> > >>>>> Can someone help me please ? I spend my afternoon to google to try to > >>>>> find > >>>>> a solution... > >>>>> > >>>>> > >>>>> Thanks in advance !!! > >>>>> > >>>>> Romaric Defaux > >>>>> > >>>>> > >>>>> > >>>> After entirely rewrite my code to not use Web service but socket (a real > >>>> client/server program) I finally found the problem... And it's not > >>>> linked to > >>>> the POST or GET method... > >>>> It's because of that : > >>>> g.db.escape_string(fields['hosted_web_site']) > >>>> (escape_string is the function in MySQLdb library) > >>>> It escapes the simple quote of the pickled object, and break it... > >>>> > >>>> It's good to know, NEVER escape a pickled object :) > >>>> > >>>> Romaric Defaux > >>>> > >>>> > >>> I'm not sure I understand what you're doing here, but I trust you've > >>> read about and understand the security problems with pickle? > >>> > >>> Geremy Condra > >>> > >>> > >> I read quickly the security problems with pickle. But I don't feel > >> concern about that because I run my program in a private network, not > >> over internet. And now I use socket to communicate on a non-standard > >> port, not anymore web service on the 80 port. If I plan to run it > >> through wan, I will encrypt datas for sure with SSL or something like > >> that :) > >> > >> Romaric Defaux > >> > >> > > -- http://mail.python.org/mailman/listinfo/python-list
