LpSolit’s blog (original, isn’t it?)

A long time ago, I promised to write an article on how to display custom fields under certain circumstances… and never wrote it. But as late is better than never, here we go!

One new feature in Bugzilla 3.4 is the ability to display a custom field only when some other field has some given value. For example, let’s say I want to display a custom field named “Impact” only when the priority of bugs is P1. The first thing to do is to create (or edit) the custom field (Administration > Custom Fields > Add a new custom field):

The important part is on the right of the screenshot. Available fields to depend on are drop-down and multi-select fields and custom fields. Typically, this means the priority, severity, OS, platform, bug status, resolution and the product fields. In our example, “Priority” is selected, and available values for the priority field are automatically listed, from which we can select “P1″, as desired. Now, when a bug has priority P1, the Impact custom field will be displayed, else it won’t:

The change is dynamic, thanks to JS, so you don’t need to reload the page to see the custom field appearing/disappearing. Once you select another priority, the Impact field will immediately disappear from the page, and it will reappear if you re-select P1. (In the screenshot above, I assume you know how to populate values for the Impact field: Administration > Field Values > Impact > Add)

Another cool feature is the ability to decide which values to display in drop-down and multi-select custom fields based on another field value. in our example above, let’s say I want to display values in the Impact field depending on what the severity of the bug is. The first thing is to add this dependency as follows:

Compared to the first screenshot, I added the dependency to the severity field. Available fields to depend on are the same as above, i.e. priority, severity, etc…. You then have to either click the “Edit legal values for this field” link on the left of the screenshot, or go to Administration > Field Values > Impact. Then you have to edit each field value which is affected by the severity of bugs:

As we added a dependency, a new item has been added when you edit field values. By default, the “Only appears when Severity is set to” field is left empty, meaning that the field value will always be available, independently of the bug severity. Let’s say we want to display the “Global: affects all other components” value of the Impact field only when the bug severity is “blocker”. In this case, we edit the value as shown in the screenshot. Let’s also say that we want to display the “Global: affects many other components” value only when the severity is “critical”. We edit the value in a similar way to what we did above. Now when you view a bug, you will see something like this:

The “Global: affects all other components” value is not listed, because the severity of the bug is not “blocker”. The “Global: affects many other components” value is listed because the bug severity is “critical”. The other two values are listed because I didn’t set any restriction on them, and so they are always available when the Impact field is displayed.

You will probably ask “How do we display the Impact field when the priority is either P1 or P2?”. The short answer is that you cannot yet, unfortunately. This is a limitation which we are working on, see bug 479400. Another question could be “How do we display a value when the bug severity is either blocker or critical or major?”. The answer is the same as for the previous question: you cannot do this yet, see bug 522971. I hope to see them fixed for Bugzilla 3.6, but the freezing date is pretty close and I fear these features won’t be available on time. Maybe in Bugzilla 3.8!

Now that Bugzilla 3.4.1 has been released earlier this month, we can focus on development again. I’m going to give you a brief overview of new features and major changes we did in our code recently:

• Bug 214861: You can now use your saved searches to generate new series for New Charts. Till now, you had to create series from scratch.
• Bug 349336: When you create a new user account and choose your password, you are now automatically logged in.
• Bug 471620: Passwords are no longer limited to 16 characters. They can be as long as you want.
• Bug 480986: The Bitmap (BMP)-to-PNG conversion tool which was based on Image::Magick is no longer in the core code of Bugzilla. It has been moved into an extension which will be shipped with Bugzilla 3.6 (this extension is accessible using CVS, in the extensions/bmp_convert/ directory). To enable this extension, delete the extensions/bmp_convert/disabled file.
• Bug 507493: checksetup.pl’s output now uses colors to highlight missing or too old Perl modules. This should prevent a large number of questions we got on IRC these last few weeks about upgrading problems.
• Bug 508xxx: various improvements have been made to checksetup.pl, which should make upgrades significantly faster.
• Bug 509027: There is now a hook in Bugzilla::Attachment::_check_data() which lets extensions manipulate attachments before they are added to the DB. The BMP-to-PNG converter mentioned above uses this hook. You could also imagine an extension which looks at the “isurl” attribute and downloads the document pointed by this URL (do it at your own risk, of course, in case the URL points to a 4Gb DVD ISO).
• Bug 509497: GROUP_CONCAT(), natively implemented in MySQL, will soon be available for PostgreSQL and Oracle as a custom sql_group_concat() function. The function is ready for checkin for PostgreSQL, but we are still waiting for an updated patch for Oracle. It should land on time for Bugzilla 3.5.1.
• Bug 108243: Thanks to the new sql_group_concat() function mentioned above, bug flags can now be displayed in buglists! I will check in this patch as soon as the patch for sql_group_concat() is ready for Oracle DB. It should also be available on time for Bugzilla 3.5.1. (Note that I said “bug flags”, meaning that attachment flags won’t be displayed in buglists.)

More cool stuff should come in the coming weeks!

There is no planned date for Bugzilla 3.5.1 yet, and in all cases, keep in mind that it will be a development snapshot. It will not be suitable for production. Our next stable release will be Bugzilla 3.6.

We released Bugzilla 3.4.1 a few minutes ago to fix a security bug reported two days ago. Your installation is only vulnerable if at least one of your products has the “Entry” bit turned on for at least one group. Note that users cannot do any harm: security checks are working fine and so no user can file or move a bug into a product if the user is not allowed to access this product. We marked this bug as a security one because a user could see the name of some products despite he should not be aware of their existence (when these products have Entry + Mandatory/Mandatory set).

Here is what happened since we released Bugzilla 3.4 on Tuesday:

Tuesday, July 28

11:00 GMT: Bugzilla 3.4 is available for download.

Thursday, July 30

15:02 GMT: Sergej Pupykin files bug 507389 about too much product names being visible in the “Product” drop-down field in show_bug.cgi to users with no access to them.

17:05 GMT: I confirm that the bug is a regression in 3.4.

18:40 GMT: A first fix is proposed.

Friday , July 31:

10:23 GMT: A second fix is proposed. This one gets r+

Saturday, August 1:

10:59 GMT: Bug 507800 is filed. We are going to release Bugzilla 3.4.1 today.

12:38 GMT: The security fix is checked in and the bug marked as FIXED.

12:48 GMT: Automated QA tests (running Selenium) report several errors.

13:04 GMT: I confirm that the security fix (which I wrote; oops) is bogus and is responsible for the bustage.

13:22 GMT: New fix proposed.

13:51 GMT: All QA tests now pass successfully. We are ready to go.

14:01 GMT: The fix is checked in.

15:00 GMT: mkanat is done with the website update.

15:41 GMT: Bug 507800 is marked as FIXED. Bugzilla 3.4.1 is officially available for download.

If you already upgraded to 3.4, you can safely upgrade to 3.4.1 as the changes between both versions are really non invasive. I hope we won’t need to release 3.4.2 next week!

We released Bugzilla 3.4 today, which also means Bugzilla 2.x is EOL (including Bugzilla 2.22). Here is a quick (and incomplete) list of new features/improvements compared to Bugzilla 3.2:

• The page to file bugs is now simpler, with “advanced” fields being hidden by default.
• The home page has been redesigned to be easier to use by new users.
• Logged out users can no longer see email addresses of other users at all, to prevent spam. Only the name of users is displayed.
• All passwords are now encrypted using SHA-256 instead of the crypt() function. This means passwords can now be longer than 8 characters.
• It’s now possible to send emails asynchronously when updating bugs (default: off), so that your browser doesn’t have to wait for all emails to be sent before displaying the page.
• Dates and times in comment headers and in emails are now displayed using your timezone instead of using the server timezone. Set your timezone from your Preferences panel.
• Several improvements to custom fields (new species of custom fields, as well as the possibility to display them under some given conditions).
• You can now reorder columns to display in buglists. Till now, you could only choose which columns to display, but you couldn’t reorder them. This is now possible.
• By default, obsolete attachments are hidden when viewing a bug. You can click the “Show obsolete” link to display them.
• You can use custom drop-down fields in tabular and graphical reports.
• Many new webservice methods.

If you are still using Bugzilla 2.x, or a development snapshot (3.3.x), you are highly encouraged to upgrade to 3.4. Also, if you are already running Bugzilla 3.2.x, the upgrade to 3.4 should be straightforward as there is no charset conversion (remember when we moved to UTF8 in 3.2), and almost no DB changes (besides new foreign keys to ensure your DB integrity). Note that you should run sanitycheck.cgi and fix errors reported by this script before upgrading, to avoid problems later.

We had a Bugzilla meeting yesterday, and one question was about the number of downloads of recent releases. Bugzilla 3.4rc1 and 3.2.4 were released last week, on July 8, i.e. 6 days before the meeting. There were 1204 downloads of Bugzilla 3.4rc1, and 4426 downloads of Bugzilla 3.2.4 so far. These numbers look low to us, but maybe that’s due to summer vacations. But the good news is about Bugzilla 3.2.3, which was released on March 30, i.e. 3.5 months ago: 131235 downloads! This number is consistent and slightly higher than for Bugzilla 3.0.4 (released on May 4, 2008), which has been downloaded 120500 times after the same time.

One year ago, I blogged about major Bugzilla installations and the version they were running. Many of them were running the latest Bugzilla version available. I think it’s interesting to see what these installations are running today. The number in brackets is the version which was used in August 2008.

• ClamAV: 3.4rc1 (2.22.3)
• Mozilla: 3.2.4 (3.0.4+)
• KDE: 3.2.3+ (3.0.5)
• RedHat: 3.2.3+ (3.1.4+)
• WebKit: 3.2.3 (2.20.1)
• Apache: 3.2.3 (3.0.4)
• Wine: 3.2.3 (3.0.4)
• Mandriva: 3.2.3 (3.0.5)
• OpenSSH: 3.2.3 (3.2rc1+)
• kernel.org: 3.2.2 (2.22.2)
• Novell: 3.2.2 (3.0)
• WikiMedia: 3.0.8 (3.0)
• FreeDesktop: 3.0.8 (3.0.3)
• Songbird: 3.0.5 (3.0.4)
• W3C: 3.0.4 (unchanged)
• Facebook: 3.0.4 (unchanged)
• Eclipse: 3.0.4 (unchanged)
• ActiveState: 3.0.3 (unchanged)
• Itos (NASA): 3.0.2 (unchanged)
• Samba: 2.22.1 (2.20)
• Maemo: 2.22.1 (unchanged)
• Yahoo!: 2.22.x (unchanged?)
• Gentoo: 2.22 (unchanged)
• Gnome: 2.20.5 (unchanged)
• GCC: 2.20+ (unchanged)
• OpenOffice: 2.11 (unchanged)

We can see a few major upgrades (especially ClamAV, WebKit, and kernel.org), but many of them didn’t upgrade at all in the last 11 months. All those installations are unfortunately vulnerable to different security bugs. Keep in mind that Bugzilla 2.20.x is no longer supported, and that the support for 2.22.x will stop at the end of the month, when Bugzilla 3.4 will be released.

In my previous post, I talked about the new front page which will be used in Bugzilla 3.4 (it will very likely change again in Bugzilla 3.6 to look something like this). Now I’m going to talk about the new form to file bugs. Technically, it’s the same page as before, but we now hide some fields by default which we don’t consider to be critical when reporting a new bug.

The form to file bugs in Bugzilla 3.2 and older looks like this:

There are many fields which advanced users probably love to see when reporting a new bug, but they may look too complicated for newbies. With the idea that newbies know mostly nothing about the internals of Bugzilla, and that they probably won’t know what to set, we know display important fields only, by default:

The information requested here seems small enough to not frighten new reporters. We could even imagine hidding the OS + hardware fields, as well as the component field, but I suppose the form would then look a bit too much like Hendrix. Anyway, it’s still a big improvement over what Bugzilla 3.2 displays to the user. And before you ask: yes, you can display all fields if you want to. Simply click the “Show Advanced Fields” link at the top of the page. Bugzilla will store your choice in a cookie so that you don’t have to worry about it in the future. Cool, isn’t it?

In my next post, I will show you how administrators can configure Bugzilla 3.4 to display some fields under certain circumstances only (for instance, on a per-product basis, or only for a given severity/priority, etc…) Another cool feature.

This is the first post about new features/UI improvements available in what will become Bugzilla 3.4. I realized that many readers don’t waste their time following links in articles; if a screenshot is not included in the article itself, they ignore links pointing to them. So I decided to insert as many screenshots as possible so that you have them immediately in front of you. And also, you can no longer complain that you didn’t know about such or such (bad) improvements when your Bugzilla installation is upgraded.

Today, I will talk about what will be the most noticeable UI change: the front page.

Above can you see the new front page displayed to logged out users. There are two major changes: the 3 big buttons in the center of the page, and the login form in the header and footer of the page (the login form is now in the header and footer of all pages). The first button (the green one) will let you file a new bug. Of course, it will display the login form first as you cannot file anonymous bug reports. The second button (the orange one) will display the search page (the specific or advanced search based on your cookie; the specific one being the default). And the third button (the blue one) will let you create a new user account. Below buttons can you see a search field, well known as a “quick search” (which is more powerful than the specific search form, which is mostly useless, but less powerful than the advanced search form. My vote to kill the specific search form completely, but that’s another story).

Some users already complained that there are now 6 ways to run searches from this single page: the “Search” link in the header, the “QuickSearch” form in the header (the one with the “Find” button) the big orange button in the middle, the “QuickSeach” form below big buttons, the “Search” link in the footer and the “QuickSearch” form in the footer (again, the one with the “Find” button). I agree this may look confusing to new users. Also, there are now 3 links to create a new user account: the “New Account” link in the header and in the footer as well as the big blue button. And finally, some complained that the “File a Bug” button is useless as you have to log in first anyway. Not ideal? Your feedback is welcome!

When you are logged in, the front page will look like this:

The only major difference compared to the logged out page is that the blue button now points to your user preferences. Also, the login form disappeared and is replaced by your login name, as in previous Bugzilla versions. We all agree that this page is suboptimal for logged in users, as much more useful data could be displayed instead of these big buttons (advanced users already use links in the header/footer of the page anyway). But we plan to improve it for Bugzilla 3.6.

Do you like it? Do you hate it? You have no opinion? Please let us know!

We finally released Bugzilla 3.2.3 and Bugzilla 3.3.4 last night, March 30. All existing Bugzilla 3.x installations will get automatic notifications within 24 hours (if administrators enabled this feature).

Bugzilla 3.2.3 fixes some important bugs:

• One of the security fixes implemented in Bugzilla 3.2.1 broke the ability to edit several bugs at once if the installation was using a shadow DB with the –read-only option (Bugzilla was trying to write into the shadow DB instead of the master DB). Installations not using a shadow DB are not affected by this issue.
• Due to our new token protection implemented in Bugzilla 3.2.1 to prevent unwanted bug changes, some Bugzilla clients were unable to edit bugs anymore as they had no valid token in hands. The XML format of bugs now contains a valid token which can be used to edit them (keep in mind that the token is generated based on the user ID, so you cannot use your own token with someone else, which is the goal of using tokens!).
• Saved searches with UT8 characters in their name no longer crash Bugzilla (again a regression due to one of the security fixes implemented in Bugzilla 3.2.1).
• Due to a change in MySQL 5.1.31 and newer, “SET SESSION max_allowed_packet” is no longer allowed, making previous versions of Bugzilla 3.2.x to fail. This problem is now fixed.
• Attachments now have the same token protection as bugs themselves. This is our last security fix related to cross-site request forgery (read the previous security advisory if you want to see which other areas were fixed in our previous security release).

All known regressions introduced in Bugzilla 3.2.1/3.2.2 have been fixed in Bugzilla 3.2.3. This means you really should upgrade your installation to 3.2.3.

Bugzilla 3.3.4 is our last development release before Bugzilla 3.4rc1. This means it’s feature complete and we will now only focus on bug fixes (enhancements are no longer allowed on this branch). Newly introduced features are listed in the Status Update. The most significant change is the new front page (screenshot). Feedback is much welcome! Please keep in mind that Bugzilla 3.3.4 got no QA at all, meaning that it’s potentially unstable. If you need a stable release, use Bugzilla 3.2.3.

To developers, note that we have reopened the trunk and new enhancements are again allowed. The current trunk version is Bugzilla 3.5 (which will become Bugzilla 3.6 when it’s stable). The 3.4 branch is tagged BUGZILLA-3_4-BRANCH.

On February 2, around 17:40 PST (Feb 3, 1:40 GMT), we happily released Bugzilla 3.3.2, 3.2.1, 3.0.7 and 2.22.7, which fixed several security issues. Less than 3 hours later, bug 476594 was filed on b.m.o, reporting a problem with srand() on mod_perl (read details in the bug), which is called when you call rand() in your code. This bug was undetected while writing security patches and while reviewing them, because most developers don’t use mod_perl on their test installations. Also, this problem hasn’t been detected on our “secret” mod_perl test installation, probably because race conditions couldn’t occur due to the low number of testers playing with it at the same time. But as soon as b.m.o upgraded to Bugzilla 3.2.1, the problem became very clear, and we have to address a big THANK YOU to Philippe M. “gozer” Chiasson to help us debug and fix the problem very quickly. Only 9 hours after the releases mentioned above, i.e. on February 3 around 2:40 PST (Feb 3, 10:40 GMT), Max Kanat-Alexander, our release manager, uploaded new tarballs on the FTP server, and the website was updated to announce the immediate release of Bugzilla 3.3.3, 3.2.2 and 3.0.8, with this single change in them (it’s actually a single line change). We didn’t release Bugzilla 2.22.8 as 2.22 doesn’t support mod_perl, and so is not affected by the problem.

That was a pretty intense and busy evening/night/morning (depends where you live on the planet) for Bugzilla developers: first, we had to commit 20 security patches (backports included) to CVS, then fix some QA scripts which were affected by the security fixes above, create and upload tarballs, update the website, then track the problem with srand() on mod_perl, review the patch, write and review the new security advisory and new release notes (it was the first time I had to do reviews from work…. you know, I’m teaching ), recreate and upload tarballs again, and finally update the website once more. Wow! But things seem to be going well now; no regression has been reported to b.m.o (yet).

On planet.m.o, I see tons of tinyurl’s used to point to lists of bugs (a.k.a buglists). When you run a complex query, this is indeed the best solution for now (till bug 400063 is fixed), but in many cases, the query is simple enough to avoid tinyurl. How? Use QuickSearch!

For those who don’t know, QuickSearch is a built-in feature of Bugzilla. This is the small textbox on the left of the “Find” button you can see on all Bugzilla pages, in the header and in the footer. Maybe you think that all it can do is to get a bug ID and go to this bug when you click the “Find” button, but it can do much more than that!

Let’s take a few examples from recent posts on planet.m.o:

Example 1: Alex Faaborg’s post about polish bugs:

The “Hard Polish Bugs” link currently points to http://tinyurl.com/5e9dd7 but could be replaced by https://bugzilla.mozilla.org/buglist.cgi?quicksearch=prod%3ACore%2CFirefox%2CToolkit+sw%3A%22[polish-hard]%22+kw%3A%22polish%22 (click here for a try). How to get it? In the textbox mentioned aboved (or if you installed the Bugzilla Quick Search plugin, you can get the same result from here), you type:

prod:Core,Firefox,Toolkit sw:"[polish-hard]" kw:polish

“prod:” means “product”, “sw:” means “status whiteboard” and “kw:” means “keywords”. As you are querying in more than one product, you enumerate them separated by a comma. About the status whiteboard, I wrote [polish-hard] between quotes, but they are not mandatory in this case as there is no whitespace nor comma. But I’m used to add them by default for this field as most of the time I’m looking for something of the form [foo bar], i.e. with whitespaces. Without quotes, Bugzilla would read sw:[foo only and wouldn't know what to do with bar] (well, it knows what to do with it, but that’s not what you expect in this case).

You immediately see how to easily build the link for “Easy Polish Bugs“. All you have to do is to type:

prod:Core,Firefox,Toolkit sw:"[polish-easy]" kw:polish

and you get this URL.

Example 2: Mozilla Platform meeting minutes: 2008-10-28:

The “JS 1.9.1 Bugs” link about wanted bugs for JavaScript could easily be obtained by:

prod:Core comp:"JavaScript Engine" flag:wanted1.9.1+

which generates this URL. Here, “prod:” is again the product, “comp:” means “component”, and “flag” is… a flag. In this example, quotes are mandatory for the component because its name has a whitespace in it. Note that you don’t need to specify that you only want open bugs; that’s the default!

Example 3: Calendar Community Testday On Thursday, October 30:

The “some fixed tb-integration bugs that need to be verified” link could be obtained by:

RESO prod:Calendar resolution:FIXED flag:integration

which would generate this URL. Note that we start to type RESO, which means you want bugs with the RESOLVED bug status only. As I said above, the default is to only look for open bugs. If you don’t want to use this default behavior, you have to mention this as the first argument. Then, “resolution” is the bug resolution, here bugs marked as FIXED only. And finally, “flag:” contains a flag name with no +/-/?, meaning that you don’t care, you want all of them. Also, “integration” is not the complete flag name, it’s “tb-integration”. So Bugzilla would return all bugs matching the above criteria, and would also catch bugs having e.g. “fx-integration” if it exists.

Last trick of the day:

If you want to see all bugs waiting for your (super-)review, you can easily get them all at once typing e.g.:

flag:review?LpSolit

Just replace my nick on b.m.o by yours, of course. A shorter alternative is:

review?LpSolit

Bugzilla will automagically consider this last case as a flag due to “?” being detected in the string. That’s the only case where “flag:” is optional. If you suspect there may be requests for review in bugs which are already resolved (either RESOLVED or VERIFIED), you can ask for all of them thanks to:

ALL review?LpSolit

ALL is specified as the first argument to override the default and will make Bugzilla look for both open and closed bugs.

Note that you cannot do everything with QuickSearch. For instance, those testcases are not possible for now.

As I see more and more people asking when Bugzilla 3.2 final will be available for download, I decided to blog about it here: We will first release Bugzilla 3.2 RC2! This hard decision has been taken during our last Bugzilla meeting last week. The reason is that we did many checkins since RC1 and we may have potentially regressed something. But for all those who are impatient, you must know that Bugzilla 3.2 RC2 should be stable enough for production, and is currently in use by both Mozilla and RedHat, just to name a few. But be aware that QA is not complete yet, so there may be hidden (severe) regressions somewhere (even if I don’t think that’s the case; but who knows?).

As most of you know, Bugzilla 3.2rc1 is available for download since August 12 and bugzilla.mozilla.org upgraded to this version two weeks later, last Thursday. As I wrote in my last blog, b.m.o is not the first major installation to upgrade to 3.2rc1, so now seems a good time to get some feedback from people using this version of Bugzilla to know what’s their feeling with it, both about its UI and about its usability and new features. Mozilla folk reported a pretty large number of bugs, usability regressions, and complains, but as someone told me, they didn’t file bugs for things they like. So I thought this blog would be a better place to get your feedback, both positive and (highly?) negative, to get a better picture of what people think before releasing the final version of Bugzilla 3.2.

Note that this feedback can come from everybody using Bugzilla 3.2rc1 (or 3.1.4+), such as users of RedHat’s Bugzilla or OpenSSH’s Bugzilla, or even from smaller installations which already upgraded to this version. The wider the audience, the better.

After reading Kairo’s article about bugs triaged in Seamonkey these last few weeks, I wanted to do a quick comparison between main products released by Mozilla: Firefox, Thunderbird, Seamonkey, Camino, Bugzilla, Calendar, and I also included Core and Toolkit. The comparison is interesting, as you can see below (click the image to make it larger… and readable).

As you can see, the two popular products of Mozilla, Firefox and Thunderbird, have many unconfirmed bugs. There are as many unconfirmed bugs as fixed bugs. This surprised me a bit when you know that with such a wide userbase and the QA team, bugs should become confirmed pretty quickly… or quickly be triaged as invalid or wontfix. Maybe are these products so popular that bugs are being filed much faster than they are resolved? I don’t know (someone more involved in these two products would know more).

The other six “products” (products as defined in Bugzilla) have a pretty similar behavior: only a few unconfirmed bugs, and two to three times more fixed bugs than open bugs (bugs marked as ASSIGNED and REOPENED have not been included as they are negligible compared to bugs marked as NEW). Does it mean triaging is more efficient and bugs fixed more quickly in these products, or does it mean these products are less popular, giving more time to developer teams to triage and fix bugs?

Anyway, my goal is not to say who is good and who isn’t in triaging and fixing bugs quickly. I just thought this graph was interesting. You are free to interpret it as you want.