25.04.2018, 19:08, "Thiago Macieira" <thiago.macie...@intel.com>: > On Wednesday, 25 April 2018 08:45:41 PDT Thiago Macieira wrote: >> On Wednesday, 25 April 2018 07:13:18 PDT Konstantin Tokarev wrote: >> > 25.04.2018, 09:43, "Thiago Macieira" <thiago.macie...@intel.com>: >> > > So Qt 6 QJsonDocument & family will have the 128 MB limit >> > > removed, at the expense of the binary JSON format requiring parsing. >> > >> > That's sad, as we lose the single solution inside Qt for serialization >> > without parsing >> >> That was going to happen anyway, because of the 128 MB size limit. If we did >> nothing else, we'd create a new format with a much expanded size limit, >> which means the current format would need to be parsed and converted. > > By the way, QJsonDocument::fromBinaryData does still perform a correctness > check, to make sure it won't crash later reading corrupt data. You can skip > this step and then loading your data is extremely fast.
For example, QtWebChannel implementations in QtWebKit and QtWebEngine use QJsonDocument as a wire format for exchanging data between processes, of course validation is skept because producer and consumer are reliable. > > Here are my numbers comparing loading that 60+ MB file in both binary JSON > format (with validation) and CBOR: > > Binary JSON: > 69,844846 task-clock:u (msec) > 196.906.259 cycles:u > 422.255.714 instructions:u > [There's no readAll(); 70.2% of the time is spent inside > QJsonPrivate::Object::isValid] > > JSON: > 255,809132 task-clock:u (msec) > 771.771.000 cycles:u > 2.690.966.058 instructions:u > [80.2% inside QJsonPrivate::Parser::parseValue, 58.7% inside > QJsonPrivate::Parser::parseString and 16.3% inside QUtf8Functions::fromUtf8] > > CBOR: > 239,059121 task-clock:u (msec) > 562.474.857 cycles:u > 1.431.590.428 instructions:u > [71.6% inside QCborValue::fromCbor, 65.0% inside > QCborContainerPrivate::decodeStringFromCbor, 25.5% inside > QCborStreamReader::readStringChunk plus 12.6% inside QUtf8::isValidUtf8] > > So it's just under 4x slower, but we're still talking about consuming over > 250 MB/s of data. > > PS: YMMV, especially if you don't use CPU-optimised UTF-8 methods like I do. > You need to compile your own Qt to get those. > > -- > Thiago Macieira - thiago.macieira (AT) intel.com > Software Architect - Intel Open Source Technology Center > > _______________________________________________ > Interest mailing list > Interest@qt-project.org > http://lists.qt-project.org/mailman/listinfo/interest -- Regards, Konstantin _______________________________________________ Interest mailing list Interest@qt-project.org http://lists.qt-project.org/mailman/listinfo/interest
