Hi all, *warning* long mail! Please take time reading it. Given how long I am now writing on it: schedule half an hour or so ;-)
I will try to formulate some thoughts on what I think might help to improve the Quality Management in Plasma. Some ideas are based on what works well in KWin. First of all I want to say that I find it awesome how many mentioned Quality in Aaron's 4.10 reflection thread. It means that we identified an area for improvement. I think that's the most important part to actually get something changed. And considering where we have been just a few years ago, I find it even more awesome that we consider Quality as the issue of the release cycle. We are complaining here on a very high scale. Yes we had regressions, but we also fixed them in time for the release. I think the problem with our QM process is, that we don't have a tool to support it. Our bugtracker is (in it's current state) totally useless. Let me just show a few stats for Plasma: * 858 bugs (including wishlist) created since 4.9.0 tagging ** 343 are marked as resolved *** 43 (!) are fixed - that are 5 % of all reported bugs *** 29 are invalid *** 222 are duplicates - a quote of 26 % ** 384 are still unconfirmed (45 %) *** 96 of those are crash reports ** 130 of the not resolved bugs are still in component general (25 % of the not resolved bugs) now for comparison I show the stats of KWin: * 432 bugs (including wishlist) created since 4.9.0 tagging ** 329 are marked as resolved *** 87 are fixed - that are 20 % of all reported bugs *** 28 are invalid *** 149 are duplicates - a quote of 34 % ** 42 are still unconfirmed (9.7 %) *** 4 of those are crash reports ** 17 of the not resolved (or needsinfo) bugs are still in component general Two people work on the KWin bugs. Plasma gets twice the amount of bug reports than KWin, so it would need 4 people to keep the bug tracker in a managable state. Looking at the stats it would need 4 people triaging one bug per day to keep it clean. That is doable! I have two requirements for bug reports: 1. There should not be any unconfirmed bugs 2. There should not be any bugs in component general What is an unconfirmed bug? It's a bug which has not been reproduced/verified that it is valid! A bug report should not stay in this status for a long time. Our users spend time to report it, so we should process them. If the user provided the correct input, it should be possible to get the bug to status "CONFIRMED". If the user cannot provide the necessary data to get the bug into status CONFIRMED, it has to be set to RESOLVED WAITINGFORINFO. A bug should never be longer than one week in status UNCONFIRMED without any action. If the user provided input which is not sufficient: ask for more. If he cannot provide it -> WAITINGFORINFO. Yes it sounds harsh, but you want to keep the tracker clean. What is a bug in component general? A bug that applies to everything of Plasma? Impossible! There is no general. A bug in general means that it is unknown where it belongs to. It should be moved into the best fitting component. If it doesn't exist create a new component! But don't keep it in general - nobody is responsible for those bugs. Now the action items I would suggest to do: *Re-evaluate all components for Plasma* Let's check whether the components we have are still valid and fitting. Let's rename what can be improved and create what makes sense to create. Maybe even rethink whether it makes sense to split the product "plasma" into multiple products. *Find a maintainer for each component* The current maintainer model of Aaron and Marco for everything doesn't scale. We have so many people who could take over maintainership for a small component. Whoever is maintainer of a component, should become the default assignee for bugs in that component. No longer a one address for all assignee. Let's make the people responsible for their components. *Assign some "super" maintainers* Having a maintainer for each component carries the risk of people dropping out without being noticed and that would render the component unmaintained (example: all the kde-workspace components currently maintained by Lubos). We need some people who regularly check whether there is activity and poke the right people when lots of bugs get unmanaged (hey Jimmy are you still working on those? No? Should we get a new maintainer?). In Bugzilla talk this is called a QA contact IIRC. Random thought: give those super maintainers the power to pull the plug on a component if the quality decreases. *Restrict the available version numbers* At KWin we only allow bugs to be reported for: * the latest beta/RC * the last two minor versions of the current stable * the last two minor version of the last stable That is currently: 4.8.4, 4.8.5, 4.9.4, 4.9.5, 4.10 RC2 We won't do a release for 4.8 or 4.9 any more, so we hardly care about these versions. There are just too many changes to properly support those versions. I would restrict even further, but there are users out there and it's not nice telling them that the distro they installed two months ago is already so outdated that we don't support it. The restriction on the latest two minor versions is to catch all the .1 bug reports we get through distro install media, but are fixed in .2 or .3. It helps the user to tell them: look there are bug fix releases, you might want to update. *Close all existing bug reports* We are currently in the process of developing Plasma 2 and porting everything to QML. Do we really care about a bug reported for the C++ version? Do we really care about a crash report which is for an old version and the backtrace doesn't match the code any more? Do we really want to spend person years of going through the bugs to confirm them? Now with the step towards Qt 5/KF 5/libplasma2/QML we have the chance to do the clear cut. We did it before (KDesktop/Kicker). Let's do it again. I'm pretty sure our users prefer the honesty that we won't work on it then having the bug open for so long. I just checked and there are 811 reports open which haven't changed over the last 365 days (and all of them are wishlist). And there are an additional 561 bugs which have not changed over the last half year (no comment, nothing). That's all for the bugzilla part, now let's do some more general ideas I have :-) *Every commit should be referenced to a bug* What is the motivation for a commit? It's either a bug fix or it is a new feature/improvement. If it's a bug it's clear that there has to be a bug report for it (out of experience: there is always a bug, if not: create it). If it's a feature, it should also have a feature request in the tracker. Create it yourself if you need to. Sounds beaurocratic, but it comes quite handy as it allows to generate changelogs from it, allows people to easily test new features. *Bug fixes should come with a unit test* Plasma so far does not have any unit tests. Why? Let's not fool ourself with "it's UI, that's difficult to test". No it's not, because Plasma has a great separation between UI and logic. It would not be a problem to create a unit test for the data engines. If there is a bug do yourself the favor of creating the test. It helps you understanding the problem and fixing it. Also remember: we have now a Jenkins installation running the tests after each commit, so we see when they break (currently 3 tests in kde-workspace are failing). *New features should come with unit tests* If you work on something new it's easy to restructure the code in a way that it gets testable. New logic should be covered by tests. GUI is difficult, though we could spend a GSoC on getting a test framework. So that's it. Looking forwards to your comments on it :-) Cheers Martin
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Plasma-devel mailing list Plasma-devel@kde.org https://mail.kde.org/mailman/listinfo/plasma-devel
