Hi,
Thank you for your comprehensive reply!
Comments inline.
On 11/22/2016 02:11 PM, Cleber Rosa wrote:
On 11/22/2016 07:53 AM, Zubair Lutfullah Kakakhel wrote:
Hi,
Hi Zubair,
There are quite a few threads about this and a trello card
https://trello.com/c/xNeR2slj/255-support-running-tests-in-parallel
And the discussion leads to a complex multi-host RFC.
https://www.redhat.com/archives/avocado-devel/2016-March/msg00025.html
Our requirement is simpler.
All we wanted to do is run disjoint simple (c executables) tests in
parallel.
Sounds fair enough.
I was wondering if somebody has a WIP branch that has some level of
implementation for this?
I'm not familiar with a WiP or PoC on this (yet). If anyone has
experimented with it, I'd happy to hear about it.
Or If somebody is familiar with the code base, I'd appreciate some
direction on how to implement this.
Avocado already runs every single test in a fresh new process. This is,
at least theoretically, a good start. Also, the test process is
handled based on the standard Python multiprocessing module:
https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L363
The first experimentation I'd do would be to attempt using the also
Python standard multiprocessing.Pool:
https://docs.python.org/2.7/library/multiprocessing.html#using-a-pool-of-workers
In this case, there would be a separate python thread for each test being run
in parallel.
Each python thread would actually call the test executable using a sub-process?
That can be OK for Desktops but won't scale well for using avocado in memory
constrained Embedded devices.
Please correct me if I am reading this incorrectly.
Regards,
ZubairLK
This would most certainly lead to changes in how Avocado currently
serially waits for the test status:
https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L403
Which ultimately is added to the (Job wide) results:
https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L455
Since the results for many tests will now be acquired in unpredictable
order, this will require changes to the ResultEvent based plugins (such
as the UI).
Thanks
Regards,
ZubairLK
I hope this is a good initial set of pointers. If you feel adventurous
and wants to start hacking on this, you're more then welcome.
BTW: we've had quite a number of features that started as
experiments/ideas/not-really-perfect-pull-requests from the community
that Avocado "core team" members embraced and pushed all the way to
completeness.
