Words and large files
Hi, opened the odf spec in words the other day. This is a document with 800+ pages and a TOC of 60+ pages. I did the same in LO to compare. Don't take the absolute times too seriously as my box is well into its teenage years. But, I think comparison with LO says a lot: Open: 2,5 mins, LO: 40 secs. Except that TOC page numers is not shown (just ###), navigation, scrolling etc works fine. Save: 4 mins, LO: 30 secs. And now to the bad part: When a try to type a character words freezes for about 50 secs. It seems the whole doc is re-layouted and also the TOC is updated (page numbers appear). Then when auto-save kicks in words freezes again without any feedback. I expected to see a status meassage and a progress bar, but no. Krita solved freeze by making a copy of the doc and save in the background. LO is better (but not good). Generally you can type new text wo problems but it freezes for shorter periods (maybe when updating page numbers), and it freezes when auto-saving. It never updates the TOC, this needs to be done manually. Suggestions (just my 2 cents): 1) TOC should be updated manually to avoid re-layout. 2) Auto-save in the background. 3) Minimize re-layout by e.g. only re-layout dirty pages before and including the displayed page(s). -- Mvh Dag
Re: Merge request in need of a review
On Thursday, 18 March 2021 10:39:24 GMT Pierre wrote: > Hi everybody > > While doing some cleanups (QRegExp => QRegularExpression conversion) I found > in Words KWStatisticsWidget a TODO that was looking easy to do and would be > a relaxation moment after spending days in cleanups : extract the > statistics logic into a non-UI object so it can be reused by both QWidget > and QML UIs. I've create a merge request for this change because I really > would like to be sure it matches the expectation for UIs like gemini, and > because of some trickery I used to compute statistics only when someone is > listening to them. > > https://invent.kde.org/office/calligra/-/merge_requests/23 Already replied on the thing, but this is a great effort, and definitely really useful - i'll have to look into adding this to the components, and consequently Calligra Gemini, at some point :) > I would of course understand if nobody had time to spend on this topic. If > there is no review nor objection, I will merge this request next week and > let changes be made by iteration rather than before merging. Everybody's busy, but it's worth throwing a bit of time at reviewing other people's work. In no small part because it just straightforwardly is a useful thing to be doing, but also because well, while mental context switches are expensive, we also all /need/ a break from hammering at that one annoying problem we've been working on for hours, and... doing a review or two is a great departure :) > BTW, for the french-reading people, I've written a small article about my > recent "contribution spree" on calligra. Don't get it wrong, I'm not doing > this to show off, I'm only trying my best to get new people to contribute to > the project, showing that it's still alive. > > https://linuxfr.org/users/pied/journaux/723-5736-5696-un-mois-de-travail-de-> > resurrection-d-un-projet-libre Oh please do show off! It's a solid effort that's worth showing off :) > Regards > > Pierre -- ..dan / leinir.. http://leinir.dk/
Re: Words and large files
On Monday March 22 2021 09:01:21 Dag wrote: >But, I think comparison with LO says a lot: Sadly this is also my experience. Generally speaking, LO has more features, and while it's monolithic behemoth it just works for the most part. I've really wanted Calligra to be an alternative because there are things in its interface that I prefer (including the use of Qt), but at the end of the day it's sadly a bit like so many webbrowsers nowadays. Trying to play catch-up with the big guys (and those browser projects do have it a lot simpler, I realise). Great as a learning project and platform for testing out ideas when you're a developer, not so much when you're an end-user trying to be productive. The availability of official installer packages is a big plus for LO, too. I do keep a special place for Karbon as I still sometimes need something with Adobe Illustrator capabilities (I'm not aware of other AI alternatives; Inkscape is more of a competitor). Idem for Krita. R. >
Re: Words and large files
On Monday, March 22, 2021 9:01:21 AM CET Dag wrote: > Hi, opened the odf spec in words the other day. > This is a document with 800+ pages and a TOC of 60+ pages. > I did the same in LO to compare. > Don't take the absolute times too seriously as my box is well into its > teenage years. > But, I think comparison with LO says a lot: > > Open: 2,5 mins, LO: 40 secs. > Except that TOC page numers is not shown (just ###), navigation, scrolling > etc works fine. > Save: 4 mins, LO: 30 secs. > > And now to the bad part: > When a try to type a character words freezes for about 50 secs. > It seems the whole doc is re-layouted and also the TOC is updated (page > numbers appear). > > Then when auto-save kicks in words freezes again without any feedback. I > expected to see a status meassage and a progress bar, but no. > > Krita solved freeze by making a copy of the doc and save in the background. > > LO is better (but not good). > Generally you can type new text wo problems but it freezes for shorter > periods (maybe when updating page numbers), and it freezes when > auto-saving. > > It never updates the TOC, this needs to be done manually. > > Suggestions (just my 2 cents): > 1) TOC should be updated manually to avoid re-layout. > 2) Auto-save in the background. > 3) Minimize re-layout by e.g. only re-layout dirty pages before and > including the displayed page(s). Hi Thanks for bringing this topic back on the table, long time I did not try to do that. On my computer, opening with LO takes 4s, 20s with words… so about the same ratio as you have. Since my last experiment in this topic, the performance analysis tooling has improved so much that this will be a fun thing to do. I'll see what this gives. And I completely agree with at least parts 1 and 3 of your conclusion. Can not tell for the second part, I must think a bit more about it. And I disagree with René's conclusions. Having performance issue today doesn't mean we can not fix them. And unlike the webbrowser comparison, we are not chasing a perpetually moving target with thousands of corporate developers adding features on their engines while being 0.1 unpaid developer on our engine… It's more like a few unpaid developers against less unpaid developers. Still not the best position for us, but far more manageable. Pierre signature.asc Description: This is a digitally signed message part.
Re: Words and large files
On Monday, 22 March 2021 11:24:28 GMT Pierre wrote: > On Monday, March 22, 2021 9:01:21 AM CET Dag wrote: > > Hi, opened the odf spec in words the other day. > > This is a document with 800+ pages and a TOC of 60+ pages. > > I did the same in LO to compare. > > Don't take the absolute times too seriously as my box is well into its > > teenage years. > > But, I think comparison with LO says a lot: > > > > Open: 2,5 mins, LO: 40 secs. > > Except that TOC page numers is not shown (just ###), navigation, scrolling > > etc works fine. > > Save: 4 mins, LO: 30 secs. > > > > And now to the bad part: > > When a try to type a character words freezes for about 50 secs. > > It seems the whole doc is re-layouted and also the TOC is updated (page > > numbers appear). > > > > Then when auto-save kicks in words freezes again without any feedback. I > > expected to see a status meassage and a progress bar, but no. > > > > Krita solved freeze by making a copy of the doc and save in the > > background. > > > > LO is better (but not good). > > Generally you can type new text wo problems but it freezes for shorter > > periods (maybe when updating page numbers), and it freezes when > > auto-saving. > > > > It never updates the TOC, this needs to be done manually. > > > > Suggestions (just my 2 cents): > > 1) TOC should be updated manually to avoid re-layout. > > 2) Auto-save in the background. > > 3) Minimize re-layout by e.g. only re-layout dirty pages before and > > including the displayed page(s). > > Hi > > Thanks for bringing this topic back on the table, long time I did not try to > do that. On my computer, opening with LO takes 4s, 20s with words… so about > the same ratio as you have. i've got slowdowns at weird times while writing as well, which is one of the things i spent a bit of time on in the past, but obviously not enough (it used to be much, much worse). > Since my last experiment in this topic, the performance analysis tooling has > improved so much that this will be a fun thing to do. I'll see what this > gives. They definitely have - attaching hotspot would be a highly interesting kind of exercise :) Also it turns out that intel vtune is now free (as in lunch, not recipe) on linux, and the time slicing analysis stuff in that thing is just outright amazing, and might also be worth a punt. > And I completely agree with at least parts 1 and 3 of your conclusion. Can > not tell for the second part, I must think a bit more about it. Auto-saving in the background isn't really causing me much trouble, but yes, it would very likely be worth the effort trying to backport Krita's work on the topic (it also, if memory serves, is a touch safer, in particular on Windows, but also just in general). All in all, though, as a general thing, there's not necessarily low hanging fruit here, but however far up the tree it is, it's definitely worth trying to pick it :) -- ..dan / leinir.. http://leinir.dk/
Re: Words and large files
On Monday, 22 March 2021 12:35:28 CET Dan Leinir Turthra Jensen wrote: > Auto-saving in the background isn't really causing me much trouble, but > yes, > it would very likely be worth the effort trying to backport Krita's work on > the topic (it also, if memory serves, is a touch safer, in particular on > Windows, but also just in general). Ah, that would be quite tricky, because it depends on making a shallow copy of the document, which at least for Krita, was a lot of work to get right. -- https://www.krita.org
Re: Words and large files
So… firing up perf record and hotspot, with OpenDocument spec 1.2 (that's what I found in my homedir)… It takes about 21s to load, of which 34% is spent parsing the document (7s, still too much, but let's accept it so far) and 59% of the time is spent layouting. 14% of time is spent in KoTextRange::document() from KoTextRangeManager::textRangesChangingWithin. I've optimized this to remove this call to document, but I gained only 1s. I've done another minor optimization and found on my way that having private classes embedded in std::unique_ptr kills performance, but this will deserve a message on its own. Since the fastest code is the one we don't call, I've written a completely different way that doesn't call textRangesChangingWithin anymore. Much faster since I'm now down to 15s. All this is in my work/ducroquet/perfs-words branch. Feel free to have a look, I'll keep digging into this in the afternoon and the next days. signature.asc Description: This is a digitally signed message part.
Re: Words and large files
On maandag 22 maart 2021 14:18:32 CET Pierre wrote: > So… firing up perf record and hotspot, with OpenDocument spec 1.2 (that's > what I found in my homedir)… I had it in Downloads/OpenDocument-v1.2-part1.odt ;-) For comparison: this one loads faster https://webodf.org/demo/ci/wodotexteditor-0.5.9/localeditor.html but is unmaintained and does no pagination. > It takes about 21s to load, of which 34% is spent parsing the document (7s, > still too much, but let's accept it so far) and 59% of the time is spent > layouting. > > 14% of time is spent in KoTextRange::document() from > KoTextRangeManager::textRangesChangingWithin. I've optimized this to remove > this call to document, but I gained only 1s. > I've done another minor optimization and found on my way that having private > classes embedded in std::unique_ptr kills performance, but this will > deserve a message on its own. > > Since the fastest code is the one we don't call, I've written a completely > different way that doesn't call textRangesChangingWithin anymore. Much > faster since I'm now down to 15s. It's faster than Abiword. After OpenDocument-v1.2-part1.odt was loaded doing ctrl-a is fast. Only Calligra is fast at that. Undo after deleting all contents takes quite some seconds. But still fairly good. Certainly compared to LO, WebODF and Abiword. ⤳Jos signature.asc Description: This is a digitally signed message part.
