Quoting Diederik de Haas (2022-01-11 23:09:49) > On 11 January 2022 19:17:56 CET Jonas Smedegaard wrote: > > Quoting Diederik de Haas (2022-01-11 16:39:47) > > > One of the upstream maintainers ran a build and reported that 22 > > > tests ran and they all succeeded. On buildd 1 test out of *20* > > > failed. > > > > The Debian package currently disables 2 tests, due to them > > (succeeding locally using cowbuild, but) failing on Debian build > > daemons using sbuild: > > https://salsa.debian.org/debian/iwd/-/blob/debian/master/debian/rules#L5 > > Mentioning both as that explains the 20 versus 22 tests, but also > > Good to know where the difference in the number of tests comes from. > > But I consider disabling tests a very last resource method.
[ speculations about kernel team intents snipped ] > > Simplest is to just skip failing tests - as I did already. > > As described above, not a big fan of such 'solutions'. Correct, that is not a solution, only a workaround - the simplest. > > Next would be to share that upstream, and (as they deem sensible) > > allow them to implement gracefully skipping tests generally, for the > > benefit of other users as well - as you did already for one of the 3 > > tests. > > I'm all for degrading gracefully, but I did not suggest that; one of > the upstream maintainers came up with that. Sorry for being too terse above. My point was that after identifying which tests fail (and sloppily working around that by disabling them - or maybe instead cancel Christmas and fix life-threatening bugs revealed by the failing tests and rip IWD out of Debian until that work is safely completed, depending on the severity of the test failures) it makes sense to share with upstream which tests fail under which conditions. It is my understanding that what you did was share with upstream one test failing under certain conditions - did I understand that correctly? > The reason I said to do it in Debian and not upstream is that tests > whether certain kernel modules are available (builtin or as module) is > specific per distribution. In Debian you can check > /boot/config-$(uname -r), while on other systems you need to grep > /proc/config or (zgrep) /proc/config.gz (or (z)cat-ing and grep-ing), > optionally after modprobe-ing a kernel module which loads it there. > Other distributions may also store it on disk like Debian does, but > uses a different filename convention. Debian provides a default kernel, but supports running with custom-compiled kernel as well. I agree that it would be elegant if the package could handle the possible variability of kernels in the system the package is installed into, but I expect that to be a complex task. > > Maybe the kernel modules needed for (some of) these 3 failing tests > > would be generally beneficial for Debian users to have compiled > > built-in for the default Debian kernel. If that seems likely, it > > would be helpful to open a bug against the package "linux" > > suggesting that to the linux kernel maintainers. > > Generally beneficial is a reason to enable a module ... as loadable > module. > Only if it's absolutely required, will it be builtin. [ details on kernel modules snipped ] I mentioned that option only in case you might find that path relevant to explore (not because I expect it likely to be relevant myself). You seem confident it is an irrelevant option, so let's just not reflect further on that. > > I guess (but don't know for sure) the official build daemons cannot be > > told to load modules - i.e. that builds are executes inside some > > container not permitting that. As you also elaborate on, that might > > open a can of worm related to reproducibility. > > I think the DSA team can (and should be able to) answer that? That's a good idea. Feel free to ask them. > If it is indeed not permitted, then disabling that test is the only > solution. Uhm, no not "the only solution", just the simplest workaround. Another option is to have upstream improve their tests to distinguish between failures executing their code and preconditions for a test environment not satisfied - and in the latter case gracefully skip instead of failing. A third option is to patch tests if upstream choose not to do the work, and we can and want to do that work ourselves. There might be other options as well. I am elaborating here only because you seem to react strongly on my using and mentioning further above the option of sloppily skipping tests. I myself have no special interest in trying to enumerate and rank all posible options. > But I think it's worth asking/having that discussion as explicitly > loading kernel modules is (normally) considered perfectly fine. > And as iwd explicitly and deliberately uses kernel functionality, > which may be available (on disk), just not loaded into RAM (yet), it > seems like a very valid use case to me. I appreciate your enthusiam, and encourage you to drive such discussion with the relevant parties - e.g. kernel team and (as you suggested yourself) DSA team. [ details better discussed with others snipped ] > > > PS: Based on the discussion I had on IRC, the following patch was > > > proposed (and applied) upstream: > > > https://lists.01.org/hyperkitty/list/i...@lists.01.org/thread/USXHQD27OI7RJ > > > > > > EC52MPJLNGQCKEBBJKO/ > > Great. If I read that patch correctly, that means we can re-enable > > unit/test-dpp when that patch lands (i.e. from next upstream > > release). > > I actually think it won't make a difference as I get the *feeling* > that it is testing for availability for SHA256 kernel module, which is > the one which is actually loaded in all systems (see above). Good point. Time (and issuing a few packaging releases) will tell... > > Would be helpful if you could suggest upstream to try identify and > > similarly patch tests unit/test-wsc unit/test-eap-sim - assuming the > > cause of failure is similar (since the pattern - failure not locally > > but on build daemons - is same). > > As I think this should be resolved in Debian, even disabling tests if > there is no other way, and I'm entirely clueless about pretty much the > whole codebase, so I won't be able to give any (intelligent) answer to > questions they'd undoubtedly have, I have no plans to do so. (sorry) Fair enough. I am not more clever on you on the topic, and find it less important to chase, so likely won't invest much time in this myself. Thanks for your input, - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private
signature.asc
Description: signature