[issue12134] json.dump much slower than dumps

Raymond Hettinger added the comment: As Antoine and Eric stated, the module is working as intended and we don't document implementation details and generally stay away from talking about performance in the docs. -- resolution: -> rejected status: open -> closed _

[issue12134] json.dump much slower than dumps

poq added the comment: dump() is not slower because it's incremental though. It's slower because it's pure Python. I don't think there is necessarily a memory/speed trade-off; it should be possible to write an incremental encoder in C as well. -- _

[issue12134] json.dump much slower than dumps

Terry J. Reedy added the comment: With 'will try to ' and the next 'will ' omitted, I agree that Antoine's version is better than mine. -- ___ Python tracker ___ __

[issue12134] json.dump much slower than dumps

Antoine Pitrou added the comment: > "In CPython, json.dumps(o), by itself, is faster than json.dump(o,f), > at the expense of using more space, because it creates the entire > string at once, instead of incrementally writing each piece of o to f. > However, f.write(json.dumps(o)) may not be fast

[issue12134] json.dump much slower than dumps

Ezio Melotti added the comment: Are you against the proposed wording or the note itself? Stating that in CPython json.dump doesn't use the C accelerations is a reasonable thing to do imho. -- ___ Python tracker

[issue12134] json.dump much slower than dumps

Raymond Hettinger added the comment: Please don't add notes of this sort to the docs. -- assignee: docs@python -> rhettinger ___ Python tracker ___ _

[issue12134] json.dump much slower than dumps

Ezio Melotti added the comment: The name dump and dumps exist to match the same API provided by pickle and marshal. I agree that a note marked as CPython implementation detail should be added. -- keywords: +easy -patch stage: -> needs patch versions: +Python 2.7, Python 3.2

[issue12134] json.dump much slower than dumps

Terry J. Reedy added the comment: >From just reading the docs, it appears that json.dump(obj,fp) == >fp.write(json.dumps(obj)) and it is easy to wonder why .dump even exists, as >it seems a trivial abbreviation (and why not .dump and .dumpf instead). Since, >'_one_shot' and 'c_make_encoder' a

[issue12134] json.dump much slower than dumps

poq added the comment: Alright. I wouldn't mind a little note in the docs; I certainly did not expect that these two functions would perform so differently. Would it be very difficult though to add buffering support the C encoder? -- ___ Python tr

[issue12134] json.dump much slower than dumps

Éric Araujo added the comment: I believe Antoine is saying “works as intended”, i.e. not a bug. I’m not sure it’s worth a doc change. -- nosy: +eric.araujo ___ Python tracker

[issue12134] json.dump much slower than dumps

Antoine Pitrou added the comment: This is indeed the case. And the solution is obvious: call dumps() and then write() the result yourself. If dump() used _one_shot=True, it would defeat the purpose of minimizing memory consumption by not buffering the whole result. -- assignee: -> do

[issue12134] json.dump much slower than dumps

Changes by Ezio Melotti : -- components: +Library (Lib) -Extension Modules nosy: +ezio.melotti versions: +Python 3.3 -Python 2.7, Python 3.2 ___ Python tracker ___ __

[issue12134] json.dump much slower than dumps

New submission from poq : import json, timeit obj = [[1,2,3]*10]*10 class writable(object): def write(self, buf): pass w = writable() print('dumps: %.3f' % timeit.timeit(lambda: json.dumps(obj), number=1)) print('dump: %.3f' % timeit.timeit(lambda: json.dump(obj,w), number=1)) O