On 9/24/22 05:00, Sean Murphy wrote:
I'm having an issue using the QtConcurrent module for processing large
images, specifically allowing the user to cancel processing an image
once it has begun.
Ahhh, welcome to the single-threadiness of Qt and one of the seemingly
limitless faces of QTBUG-12055
https://bugreports.qt.io/browse/QTBUG-12055
Please let me take you on a short journey so you understand how you got
screwed and perhaps you can find a way to unscrew yourself? I will steal
a name from Dr. Who to hopefully make this more accessible to those who
need it.
In the land of real computers with real operating systems we never had
threads and never needed them. We had proper weighted processes and an
OS accounting system that kept track of every byte of memory and
resource allocated by every process. It kept track of all parents and
children. Whenever you needed a "thread" to do something you created it
via one of the many library routines and received its pid.
Acolytes of the Papal Mainframe were taught from birth that Gluttony was
the deadliest of the Seven Deadly Sins. We never loaded big chunks or
entire files of anything having any size into RAM and we certainly
didn't create massive things in RAM. We always read/write reasonably
sized chunks so every process is always yielding the Papal Processor to
others while waiting for I/O. This yielding made the universe run
smoother, allowed others to achieve enlightenment, and provided for a
clean safe kosher kill via library routines or the command line
STOP/ID=pid
The OS would instantly kill the process, check the accounting system and
free all resources allocated/locked by said pid.
That's what you are believing happens in the world of QFutureWatcher and
it is not.
In the world of the x86-wanna-be-a-real-computer-one-day-when-I-grow-up
world, the OS creators never figured out how to create a proper weight
process. None of the operating systems created for this platform have a
robust OS accounting system keeping track of its universe. Instead, what
is a "process" on these platforms is too light to even be a thread on
the real computers. Since they needed the ability to create other
things, they invented threads which are lighter still and mostly outside
of the realm of accounting. Threads can be left dangling when a parent
is killed off due to the lack of accounting. Many times resources don't
get freed when a thread is killed off for the very same reason.
In the x86 based Qt world you have the single main event loop where all
GUI activity must occur. When you use the threading classes, especially
with things like QFutureWatcher some portion of your thread activity
will be at the mercy of the main event loop simply because some events
will be queued there.
The whole QFuture and QThreadPool concept is great . . . if and only if
you design your application from the mindset of a parent sending their
kid off to college knowing full well they will only call when they need
money. If you want them to live by your rules as they did under your
roof, then you are going to have a rough time of it.
Too many people developing software on the x86 platform, even those
writing its operating systems and libraries, never attended services at
the Church of the Papal Mainframe. They never studied to be an acolyte
and gluttony is the law of this land rather than being shunned for the
deadliest of the deadly sins as it truly is.
Control returns to an event loop "at idle" or when you call
processEvents(). Thiago hates people using that, but the design of Qt
mandates the use of it, repeatedly, in otherwise good designs.
Those processing images tend to be the worst of the sinners. I've seen
them read entire DVDs of content into RAM, then thread off a function to
chew on that data and create yet another entire DVD of content in memory.
What does this do?
It creates the tightest loop you've ever seen. There is no yielding of
the Papal Processor to others unless you get lucky the the dynamic
memory allocation does it for you. There is no "idle" time for the event
loop of your thread to process any events that got queued there. This
mLoadWatcher->disconnect();
never happened. It got queued on an event loop that never got processed.
See QTBUG-12055. Likewise, this
mLoadWatcher->cancel();
got queued behind it. Your finished() signal was a direct connection,
and it didn't know it wasn't supposed to happen so when the tightest
loop in the world finished the function pointer assigned to execute when
finished went ahead and executed.
Before you go thinking "Hey, I'll dig into the QFuture and QThreadPool
classes and find out how to get a pid for the thread so I can kill it"
don't. That is designed for things that run to completion and is a house
of cards when they don't. Lots of things won't be deallocated/freed. The
thread from the thread pool most likely will not be returned to the pool
so you will exhaust the pool after just "a few" of these hard cancels.
**Ultimately you have to get rid of your tight loop.**
If you are using a library that has to have the entire image loaded into
RAM or cannot have its tiling process pause for a processEvents() call
then it is an improper library for your use case. If you can force said
function/library to write the output to a file instead of physical RAM,
you will create I/O yields that should allow your event loop to execute.
Don't forget to nuke the file on cancel().
Make your thread into a stand-a-lone executable and get good with spawn.
https://linuxhint.com/posix-spawn-c-programming/
or exec
https://linuxhint.com/exec_linux_system_call_c/
Keep in mind that moving to a version of Qt that has the fix for 12055
won't fix your problem. As far as I know cancel() is still a queued
task. If it is changed from a queued task to a hard kill it is going to
leave all kinds of dangling allocations and resources because none of
the operating systems developed for the x86 have a proper OS accounting
system. They cannot properly clean up behind a hard kill of a thread.
They will do a better job (not perfect, but better) cleaning up behind a
hard kill of an actual stand-a-lone PID created via spawn/exec.
I know that's not what you want to hear, but that's where you are at. No
good way to kill a Glutton thread.
--
Roland Hughes, President
Logikal Solutions
(630)-205-1593
http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com
http://www.logikalblog.com
http://www.interestingauthors.com/blog
_______________________________________________
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
