Jeroen van Meeuwen had some harsh criticism on the way KDE handles Q&A with Bugzilla. He has taken on bugzilla maintainership after someone told him there was no 'single throat to choke', and started to investigate the current state and how it is used by KDE. He presents his findings and suggestions on improvements.
There's no defined process - and that shows. Bugs linger around forever. There are bugs from 2002 which have not been asigned to anyone or confirmed in any way. Worse, the 'final' state we use in KDE, 'resolved', is not, and should not be, the final state of a bug! After fixing a bug, there is testing, confirming and releasing still to do before the bug should be allowed to fall off our radar.
Due to Bugzilla not working as it should, we work with other tools to keep a list of what to do. Reviewboard is an example and many developers have their own list. All that while Bugzilla is more than capable of doing this!
Jeroen talked about how we, as a community, should try to adapt a new process with bugzilla. For that, our setup needs some improvements and he's offering to work on those.
Specifics include having proper types, a 'CLOSED' state and cleaning up products. An example of the latter is that there are two products for 'KMail': 'kmail' and 'kmail2'. Both are open for bug submissions - and there are lots of bugs coming in for the 'old' kmail, adding to the almost 3000 open bugs. However, the developers don't work on kmail anymore: they have all moved to kmail2. How to solve this? Resolve the current unresolved tickets and tell users to move to kmail2. Then close kmail and then move kmail2 to kmail.
Jeroen explained how we can use tools like jared and jenkins to automatically run tests, check if things compile for a certain branch and things like that. How we need to differentiate between Q&A, Triaging, Development and release engineering.
Jeroen proposes to go through the following steps:
- do a bugzilla cleanup
- have to contact project developers and ask them what they do with the tickets, then adopt the tickets and setup to that reality.
- formulate and propose standardized use of tickets parameters
- help explain how you can use bugzilla as a todo queue
- align documented process and establish responsibilities
- align use of versions/milestones with source code management
- mass-updating tickets through sysadmin requests
- offer one throat to choke
Jeroen realizes he's new to KDE and that he's not the only one who (thinks he) is an expert. Also, KDE works on consensus - so it'll take effort and discussions to get a new way of working in.
There'll be a BOF about QA and triaging on Monday so if you're interested, look at the BoF schedule.
A big take-away from me is that this should actually make development easier, more focussed and more fun - it's not about creating a lot of process to bother developers.
Thanks for looking into this. A thing that I would like to point out, is that closing bugs should be done with some tact/done at least semi-manually.
ReplyDeleteThe standard response to many user issues is: "then why don't you file a bug?", and honestly I find it can be very disappointing if, after a while, you filed your bug, it has been ignored, and it is closed with an automated message.
I know that bugzilla is for the developers, not the users, but for instance I feel that with ubuntu's bugzilla, many bugs go ignored and are then automatically closed, so nowadays I hardly even bother to report things there.
I think an important part of making bugzilla work again, is establishing user confidence on bugzilla: when they report something, at least someone will give it an honest look, and try to at least categorize it/assign it/confirm it/etc. Otherwise, users just give up on reporting issues.
Well, these were exactly the things Jeroen mentioned and which he wants to (help) solve.. I think it's a good thing for sure.
DeleteJenkins is pretty cool. There should be more publicity around that.
ReplyDeleteWhat I wonder is what knowledge is drawn from bugs. I mean, is there a process to see if fixes could be generalised or transformed into generic tests?
Ha, that's another thing Jeroen mentioned - and the answer is NO.
Delete