On Wednesday, 25 April 2018 08:45:41 PDT Thiago Macieira wrote:
> On Wednesday, 25 April 2018 07:13:18 PDT Konstantin Tokarev wrote:
> > 25.04.2018, 09:43, "Thiago Macieira" <thiago.macie...@intel.com>:
> > > So Qt 6 QJsonDocument & family will have the 128 MB limit
> > > removed, at the expense of the binary JSON format requiring parsing.
> >
> > That's sad, as we lose the single solution inside Qt for serialization
> > without parsing
>
> That was going to happen anyway, because of the 128 MB size limit. If we did
> nothing else, we'd create a new format with a much expanded size limit,
> which means the current format would need to be parsed and converted.
By the way, QJsonDocument::fromBinaryData does still perform a correctness
check, to make sure it won't crash later reading corrupt data. You can skip
this step and then loading your data is extremely fast.
Here are my numbers comparing loading that 60+ MB file in both binary JSON
format (with validation) and CBOR:
Binary JSON:
69,844846 task-clock:u (msec)
196.906.259 cycles:u
422.255.714 instructions:u
[There's no readAll(); 70.2% of the time is spent inside
QJsonPrivate::Object::isValid]
JSON:
255,809132 task-clock:u (msec)
771.771.000 cycles:u
2.690.966.058 instructions:u
[80.2% inside QJsonPrivate::Parser::parseValue, 58.7% inside
QJsonPrivate::Parser::parseString and 16.3% inside QUtf8Functions::fromUtf8]
CBOR:
239,059121 task-clock:u (msec)
562.474.857 cycles:u
1.431.590.428 instructions:u
[71.6% inside QCborValue::fromCbor, 65.0% inside
QCborContainerPrivate::decodeStringFromCbor, 25.5% inside
QCborStreamReader::readStringChunk plus 12.6% inside QUtf8::isValidUtf8]
So it's just under 4x slower, but we're still talking about consuming over
250 MB/s of data.
PS: YMMV, especially if you don't use CPU-optimised UTF-8 methods like I do.
You need to compile your own Qt to get those.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
_______________________________________________
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest
