So, Bugzilla 4.4rc2 is finally here! It took us more time than expected, but the performance bugs are all fixed and we also fixed two security bugs which are described in the security advisory. Bugzilla 4.2.5, 4.0.10 and 3.6.13 also contain these two security fixes, so you should definitely upgrade.
Compared to 4.4rc1, the new release candidate contains the last feature which we wanted for 4.4: the ability to add several criteria in a query against the same field. In Bugzilla 4.2,
"Flags changed to approval+" AND "Flags changed by firstname.lastname@example.org"
were disconnected, which means that these criteria would match all bugs which had the approval+ flag set and which had a flag changed by email@example.com. In Bugzilla 4.4, you can now force these two criteria to match the same field, i.e. that you only want bugs where the approval+ flag has been set by firstname.lastname@example.org. Thanks to Byron Jones (glob) for this great feature. Of course, this feature is not limited to flags, but works with all fields. See the release notes for more details. Also, 4.4rc2 should in most cases be much faster than 4.2 to run some complex queries. In my testing, some queries which took several minutes to complete now run in a few (tens of) seconds only. This is especially noticable with many columns displayed in the buglist. Another good reason to test it!
The next release should be 4.4 final as I don’t think a 4.4rc3 is needed.
You may wonder what’s going on since we released Bugzilla 4.4rc1 on November 13. Well, very few bugs have been reported which seems to indicate that 4.4rc1 is pretty stable. But we also found that running some queries were very slow since Bugzilla 4.2. Bugzilla 4.2 got a major rewrite of its search code (Bugzilla::Search), where the focus was readability, maintainability and the removal of some hacks, but this new code had some severe performance impact for some queries, especially those involving subselects in MySQL. That’s why several bugs related to performance have been filed since we released 4.4rc1 and we decided that they were critical enough to fix them in the 4.4 branch rather than waiting another year to include them in Bugzilla 5.0 only. As it’s unclear if these recent changes can have negative impact on some queries, or break some other ones, we decided that a 2nd release candidate was necessary to give admins and developers some time to test the new code and give us feedback before 4.4 final. There is no ETA for 4.4rc2 yet, but it will probably be somewhere in January 2013.
If you have some queries which you consider as being very slow, I would be interested in hearing you about them: which criteria did you use, which columns are displayed in buglists, which Bugzilla installation did you use for your testing, are you logged in or logged out, etc…? Please don’t run your queries on Bugzilla 3.x or older. I’m really not interested in such old installations (and the code changed too much anyway with what we have in 4.4). In my own testing, I found that some queries which were previously timing out now run in a few seconds only. Of course, the time taken to execute these queries highly depends on the data you have in your DB and so only relative timing matters.
Last night, we released Bugzilla 4.2.4, 4.0.9 and 3.6.12 to fix several security issues. Bugzilla 4.2.4 also notably fixes some crashes with Oracle when viewing buglists. Oracle support in 4.2.4 is definitely much better than in previous 4.x or 3.x releases.
We also released the first release candidate of Bugzilla 4.4, which we expect to release near the end of the year, though it will depend on the feedback we get. If a second release candidate is needed, we will delay 4.4 final. Now is a good time to test its new features, including the new tagging system (which replaces the previous tagging system which you could see in the footer of pages), the ability to save tabular and graphical reports in the same way you can save your searches (no need to bookmark them in your browser anymore), customizable columns displayed in emails sent by the whining system, many new and updated WebService methods, real auto-detection of the MIME type of attachments uploaded to Bugzilla (currently only if the browser is unable to determine the MIME type itself, else we still trust what the browser says), etc… Do not forget to fix your Apache configuration file (httpd.conf) when upgrading as explained in the release notes, else Bugzilla 4.4 won’t work. Have fun!
I saw several requests about how to use GMail as server to send bugmails from Bugzilla. I also saw too many (bad) articles about how to hack Bugzilla to make it work. Most were suggesting to install 3rd-party applications and to badly hack the source code, especially when installed on Windows. Forget all that! What you must know is that Bugzilla supports GMail as SMTP server for more than a year (!), and Bugzilla 4.4rc1, which is going to be released next Tuesday, supports it natively!
When running checksetup.pl, make sure that Net::SMTP::SSL is installed:
Checking for Net-SMTP-SSL (v1.01) ok: found v1.01
Then edit the following parameters (under Administration > Parameters > Email):
mail_delivery_method = SMTP mailfrom = email@example.com smtpserver = smtp.gmail.com:465 smtp_username = firstname.lastname@example.org smtp_password = your_gmail_password smtp_ssl = on
Do not forget to save your changes, and that’s all, Bugzilla is now able to use GMail to send bugmails.
If you are running Bugzilla 4.2 or older, then you should apply this patch. That’s the patch which is now part of Bugzilla 4.4, but it also applies cleanly to all supported branches. And the good news is that the upgrade to 4.4 will run smoothly as that’s the exact same code as for 4.4. Once applied, you can follow the steps above.
I see many bugs filed in the Bugzilla product related to bmo such as "please add flag tracking-firefox89", "please merge my account email@example.com with my new firstname.lastname@example.org", "please delete bug XX", "please add component X to product Y", "I need to be added to the core-security group", etc… etc…. All these requests specific to bmo must be filed in the bugzilla.mozilla.org product. I think the name of this product is explicit enough to be understood by everybody. It would be great if all users getting a @mozilla.com email address would be informed about this distinction as they represent 95% of wrongly filed bugs (the Bugzilla product is only for bugs/requests related to the Bugzilla project itself, not the Mozilla instance). Fortunately, it’s very easy to move a bug into its correct product, but some developers are spammed uselessly.
Bugzilla installations running Oracle as their database server fail to display flags, tags and keywords in buglists. Oracle crashes with:
DBD::Oracle::db prepare failed: ORA-30482: DISTINCT option not allowed for this function (DBD ERROR: error possibly near <*> indicator at char 313 ..., group_concat(<*>T_CLOB_DELIM(DISTINCT map_tag.name, ', ')) tag
All the details are in bug 780053. The group_contact() function is defined in Bugzilla/DB/Oracle.pm. The maintainer of the Oracle DB module, who was an Oracle employee, disappeared, leaving us in the dust. The code in Oracle.pm is totally obscure to us and I have no idea how to fix it. I would like to have this bug fixed on time for Bugzilla 4.4, which should be released before the end of the year, and so I need your help as soon as possible to fix this problem. Someone already made a suggestion in the bug, but I need a second review or another sugggestion as I don’t understand anything about the cryptic internals of Oracle.
So if you are familiar with Oracle or use Oracle as your DB server, please give it a look. Many thanks!
All those of you who are administrators of a Bugzilla installation already had to configure products in the past. Most of you probably found it was pretty hard to configure security correctly on these products: Entry, MemberControl, OtherControl, Canedit, editbugs, canconfirm, editcomponents. What’s all this and how do they interact with each other?
I made a proposal to rewrite this page entirely, and you can see the result here (it’s a html page). If you are a Bugzilla administrator or have privileges to edit product settings on your Bugzilla installation, please give me your feedback, ideally as a comment in the bug itself, in the worse case here as a comment. If this new UI is accepted, it will be part of Bugzilla 5.0.
I saw several admins being confused about why their Bugzilla installation stopped working after upgrading to Bugzilla 4.3.3. First of all, remember that 4.1, 4.3, 4.5, etc.. are developement releases, not stable releases! Stable releases are of the form 4.0, 4.2, 4.4, etc… Now, the reason for this specific issue is because we added "Options -Indexes" to bugzilla/.htaccess to prevent directory browsing, but this requires that your httpd.conf configuration file allows the usage of Options in .htaccess. Till now, you probably had:
<Directory /var/www/html/bugzilla> AddHandler cgi-script .cgi Options +Indexes +ExecCGI DirectoryIndex index.cgi AllowOverride Limit FileInfo Indexes </Directory>
Now, you must have:
<Directory /var/www/html/bugzilla> AddHandler cgi-script .cgi Options +ExecCGI DirectoryIndex index.cgi index.html AllowOverride Limit FileInfo Indexes Options </Directory>
Note that +Indexes has been removed from the Options line, that index.html has been added to DirectoryIndex (for the doc) and more importantly that we added Options to AllowOverride. This last change is the one required to make Bugzilla work again.
If you are like me and really dislike the removal of the favicon and the blue/green background for SSL connections from the address bar in Firefox 14 and newer, I recommend you install these two Firefox addons: Favicon Restorer and Site Identity Button Colors.
And you probably remember that Firefox 6 and Firefox 7 removed the http:// protocol from the URL and highlighted the domain name of the URL? If you don’t like them, you can revert these changes by setting browser.urlbar.formatting.enabled = false and browser.urlbar.trimURLs = false.
Maybe I’m the only one to think this way, but I really think improvements made to the address bar are going the wrong way.
I think now is a good time to give you some news about the current development of Bugzilla. First of all, we are very close from releasing Bugzilla 4.2.3 and 4.3.3. There is one blocker left which needs its patch to be reviewed, and we can start the release process. Hopefully this will happen this week. Both releases will work with Perl 5.16 (uploading attachments was broken due to a change in Perl 5.16), databases created with PostgreSQL will now be encoded with UTF8 (the encoding wasn’t enforced, and was entirely depending on how initdb created template1), Oracle will be able to display buglists again instead of crashing (unless you try to display keywords, flags or tags. I have no idea how to fix this problem, help welcome), the obsolete -moz-border-radius CSS property which was no longer understood by Firefox 13 and newer has been replaced by -border-radius (meaning that other browsers will take advantage of it too), and a regression in 4.2.2 about the keyword auto-completion feature has been fixed. There are a few more fixes included in 4.2.3, but you will read them in the release notes.
About 4.3.3, we can also mention that the "Browse" link in the page header and footer now correctly sorts products by classification, you can now save your tabular and graphical reports (till now, you could only save your searches), the User.get WebService method now returns your saved searches too (and only yours!), PATH_INFO is now removed by default from all URLs, graphical reports are automatically resized based on the size of your window, if you type the alias of a bug you cannot see, Bugzilla no longer tells you which bug ID has this alias (it just tells you that this alias is already in use), flags which you cannot set/edit are now hidden instead of being disabled only, there is now a "(take)" link besides the QA contact (till now, only the assignee had such a link), and we now use HMAC-SHA256 to generate tokens instead of MD5. More WebServices methods are expected before 4.4rc1; they are currently being worked on (and some patches already in the review queue).
Once Bugzilla 4.2.3 and 4.3.3 are out (hopefully this week), we will create a branch for 4.4. On the 4.4 branch, we will limit changes to new WebServices methods, bug fixes and some not too invasive patches. The plan is to release 4.4rc1 next month or the month after, and 4.4 final before the end of the year.
On the trunk, we will immediately start the development cycle for Bugzilla 4.6/5.0 (we didn’t decide its version yet, but this is not the most important part of the story). One of the main goals will be to improve its User Interface (yeah, that’s not a joke). We are currently using YUI2, and I hope that using YUI3 or jQuery (the choice isn’t made yet, see bug 453268) will help to reach this goal. We will also need feedback and suggestions from the community to improve things (this is way better than waiting for the next version to be released and start complaining at that time that you don’t like the new UI. Try to be constructive, though). We will also bump the min version of Perl from 5.8.1 to 5.10.1 so that we can use some new features implemented in 5.10.1. New contributors are always welcome!
I got an email this morning from the maintainer of Testopia to inform me that Testopia 2.5 is finally released! This is the first version to work with Bugzilla 4.2.1 (if you are still running Bugzilla 4.0.x or 4.2, upgrade to 4.2.1 first). The announcement has not be made official yet, but this should be done in the coming hours, hopefully. Meanwhile, you can already download Testopia 2.5. Note that there is no need to apply any patch anymore to make it work (which is why you need Bugzilla 4.2.1 instead of e.g. 4.2, because 4.2.1 has some changes included in it to avoid to patch the core code to make Testopia work). And if you find any problem which hasn’t been reported yet, feel free to file a new bug (bugs only, no support questions). Have fun!
UPDATE: The announcement has finally been made here.
During the week-end, I received a pertinent email from a former Bugzilla developer who replied to an email I sent to all reviewers about the pretty low activity in the Bugzilla project during the current development cycle. He argued that one of the reasons which made him go away, and which probably took some other contributors away as well, is our high code quality standards we have in this project. The point is that we deny review if submitted patches do not follow our guidelines or are poorly written compared to what we expect in our codebase. He suggested that reviewers should accept and commit lower quality patches, and file follow-up bugs to clean up the new code. I then brought this discussion on IRC with other core developers, and we realized how hard it is to define the right level of code quality. The problems are:
- Some new features require some rewrite of some parts of the code to be easily extendable in the future and/or to reuse some existing code. These features could also be implemented without the code rewrite, which is easier to do for the contributor, but then would be harder to maintain, or would require to duplicate some existing code, which we try to avoid as much as possible. This is how Bugzilla 2.x was written, and it took us two full development cycles (Bugzilla 2.22 + 3.0) to clean up and rewrite the codebase to be able to implement new features in a reasonable way (the 2.x codebase became too ugly and complex to easily implement anything new on top of it). So the risk to accept such patches is that the codebase would become more messy again. But on the other hand, the risk is also to make contributors leave if we keep our high standards, because a rewrite is generally not something trivial nor exciting, and contributors generally do not understand why we reject a patch which does the job.
- Accepting patches which are of lower quality and filing follow-up bugs to fix the new code before the next release seems reasonable only if the community around the project is large enough to have the manpower to do the job in the expected timeframe. The current Bugzilla team is very small, maybe 4-5 core developers + some occasional contributors helping with some not too hard bugs. This isn’t large enough to expect these follow-up bugs to be fixed promptly. But we also cannot release a new version with poor code in it, some of the reasons being security implications, performance issues, or possible regressions in some use cases. As the community is small, fixing these bugs would delay the next major release a lot, which is not a desirable effect.
- Depending on how the submitted patch is written, it would take twice the time to review + rewrite the new code to match our standards compared to the time it would take for a core developer to write the patch himself. The lack of time being the main enemy, I personally wouldn’t want to spend more time on such or such feature because it’s not written as we would like it to be. The time spent to rewrite bad code means less time to work on other stuff.
So what’s the right threshold to not make new and occasional contributors go away without badly impacting our codebase? Which rules do other Mozilla and non-Mozilla projects use to solve this problem? Please share your experience with us.
We released Bugzilla 4.3.1, 4.2.1, 4.0.6 and 3.6.9 a few hours ago, and they all contain 2 security fixes. All installations are highly encouraged to upgrade to these new releases. Bugzilla 3.6.9 and 4.0.6 only contain the two security fixes mentioned in the security advisory. Bugzilla 4.2.1 also contains a pretty large list of bug fixes, which makes it a good candidate to upgrade to the 4.2.x series if you didn’t upgrade to 4.2 yet. Note that Bugzilla 4.2.1 is also the first release to work with Testopia without needing to be patched first. A new release of Testopia should be available soon, which will take advantage of the improvements and new hooks available in 4.2.1.
We also released Bugzilla 4.3.1 which is the first development snapshot of the 4.4 series. I’m going to list some of the new features and improvements available:
- A new user preference lets you decide if you should be CC’ed when someone requests something from you via a flag. For instance, requesting someone for review would automatically CC the reviewer to the bug.
- Unset flags are now hidden by default, in bug reports.
- Duplicates are now listed at the top of the bug report, close to the dependency lists.
- URLs pointing to GitHub (github.com) are now accepted in the See Also field in bug reports.
- Personal tags can now be displayed in buglists. In Bugzilla 4.3.2, you will also be able to tag bugs from the bug reports themselves. This means the ugly box at the very bottom of every page will go away.
- You can now hide the description displayed at the top of buglists containing all the arguments passed to the query.
- You can now use pronouns (%reporter%, %user%, etc.) with the flag setter and requestee fields in boolean charts.
- A new email preference has been added to get emails when the product or the component of a bug changes.
- Bugzilla now keeps track of the last time a user used his account to access Bugzilla. This will let admins know which accounts are inactive.
- Bugzilla now supports SSL connections to SMTP servers.
- Target milestones can now be 64 characters long (instead of 20) to match the same limit as versions.
- Whine emails now correctly take the sort order of buglists into account.
- Several new WebServices methods: User.update, Product.update, Group.update, Bug.update_tags, Bugzilla.parameters (read-only), Bugzilla.last_audit_time.
- Several existing WebServices methods have been improved to return more data.
- The documentation is now generated using xmlto instead of jade.
- Several performance improvements have been made when viewing large bug reports.
- The Bugzilla source code now uses the MPL2.0 license, instead of the MPL1.1 one.
We are one month away from the freezing date for new features in Bugzilla 4.4. So if some of you really want something for 4.4, you have exactly one month left to submit patches, or find a kind developer to write the patch for you. Also note that Bugzilla 4.4 will be the last release to support Perl 5.8.x. The next major release, Bugzilla 4.6, will require Perl 5.10.1 or better.
You probably noticed that there has been no activity related to the development of Testopia, a Bugzilla extension, for more than a year. The reason is that its maintainer, who was the single contributor to this project, had a new job and has no time to work on it anymore. Consequently, the latest version of this extension, Testopia 2.4, which was released in October 2010 (!), only works with Bugzilla 3.6, but not 4.x.
You will be happy to hear that with the help of some new contributors wanting to make Testopia work with Bugzilla 4.2, I committed several patches to the bzr repository which make it work pretty decently with Bugzilla 4.2.1 (probably also with Bugzilla 4.0.x, but I didn’t test). There is no official release yet (it should be named Testopia 2.5), but you can download updated files using bzr. Make sure to revert changes made to the core code of Bugzilla first before applying the new code.
To make things clear: I’m not the new maintainer of this extension, and I have no plan to take this role. I’m busy enough with my roles as assistant project lead + reviewer + QA lead + bug triager + developer for the Bugzilla project. I simply decided to help the new contributors who jumped in these last few days and used my commit access to the Testopia repository to commit some patches. The Testopia maintainer is still alive, and emailed me this morning. So he is still the one who will take the decision to release 2.5 when it’s ready.
Update: Starting with Bugzilla 4.2.1, you no longer need to patch the source code of Bugzilla to make it work with Testopia 2.5! If you are upgrading from Testopia 2.4 or older, make sure to revert the changes made to the source code first.
This is just a quick note to let you know that once we branch for Bugzilla 4.4 in May, I will commit a patch which will make Bugzilla 4.5 and newer to require Perl 5.10.1 as a minimum. This means Bugzilla won’t support Perl 5.8.x anymore. So Bugzilla 4.4 will be the last release to support the oldish Perl 5.8.x. This new requirement is tracked in bug 655477.
There has been some complains these last few days on IRC and in the support mailing-list/newsgroup that admins couldn’t upgrade their Bugzilla installation to 4.2 due to the lack of a CVS mirror for this branch. As announced 3 years ago in the developers mailing-list and on b.m.o., and 1.5 years ago on the bugzilla.org website, the Bugzilla team switched from the old CVS to the more modern Bazaar (or bzr for short) VCS. If you use our tarballs to download Bugzilla, then you don’t really care about this change, and the process to upgrade won’t change for you. If you use CVS and you wonder how to upgrade using Bzr, here is how you can do it:
- For now, there is no need to shut down Bugzilla. We will do this when we start the upgrade process itself. If you made changes to the Bugzilla code itself (instead of using its extension system), you will have to port these changes to the new version, else they will be lost. If you made no changes, or all changes are contained into extensions, you can jump to step 2 directly. To generate a patch with all the changes you made, go into your bugzilla/ directory and run this command:
cvs diff -puN > patch.diff
The file patch.diff will contain all the changes you made to your current installation.
- Let’s download the new version of Bugzilla into a separate directory, to not mess with the current installation. Now you will need bzr. All Linux installations have it; have a look at your package manager. For instance, Fedora 16 has bzr-2.4.2-1.fc16.i686.rpm. On Windows, you can download it from canonical.com. The standalone application is recommended; for instance: bzr-2.4.2-1-setup.exe. Once bzr is installed, run this command to download Bugzilla 4.2:
bzr co bzr://bzr.mozilla.org/bugzilla/4.2 bugzilla42
Here, bzr co means "checkout", i.e. it’s the first download of 4.2 with bzr. Then you have the URL to our Bugzilla 4.2 repository, and finally you have the name of the local directory into which to dowload the source code. If you don’t like the name bugzilla42, feel free to choose another one.
- Now go into the new bugzilla42/ directory and run the following command:
The –check-modules part is important for two reasons. First of all, we didn’t copy the configuration files from the old bugzilla/ directory into the new one. The advantage to do so is to prevent Bugzilla 4.2 from finding where your DB is located and start interacting with it. Remember that we did no backup of your DB yet! And the second reason is that we first want to make sure that we have all the required Perl modules installed. Between major releases of Bugzilla, the requirements may change, either by requiring new Perl modules, or by requiring a newer version of an existing Perl module. If you have missing or too old Perl modules, you will have to install or upgrade them, ideally using your package manager. On Windows, ActivePerl 5.12 and newer have all the required modules available, so that’s not a problem. On Linux, only old distros may have some missing or too old Perl modules. If this happens, you can use the install-module.pl script which is in the same directory as checksetup.pl to install them. The commands you need to execute are given by checksetup.pl –check-modules itself. So open your eyes. Note that you don’t need to install all the optional Perl modules. The reason is that they are…. optional! Only install those which are relevant to your installation.
- If you made changes to your current Bugzilla installation, then you have to apply patch.diff you created at step 1 to your new installation. If you made no changes, you can jump to step 5 directly. If you are on Windows and you don’t have patch.exe, you can download it from sourceforge.net. Once downloaded, you must copy patch.exe into the Windows directory. At this point, it’s very likely that your changes will conflict with the new code, unless you are lucky or your changes are very minor. So we first check if there are conflicts or not. To do that, copy patch.diff into your new bugzilla42/ directory, and run this command from here:
patch -p0 –dry-run < patch.diff
–dry-run means that we made no changes to files; it was only a test. If all you get are messages such as "Hunk #1 succeeded at 79 (offset -26 lines)" or "Hunk #1 succeeded at 31 with fuzz 1 (offset 2 lines)", then you are fine. But if you get messages such as "Hunk #1 FAILED at 27" and "1 out of 2 hunks FAILED", then you are in trouble. And don’t blame the Bugzilla team and Bzr, you would get the same errors with CVS! If the conflicts are minor, you can easily fix them manually, else you probably will have to rewrite your changes directly into the new code. If you have no conflicts, you can drop the –dry-run argument and apply your patch for real:
patch -p0 < patch.diff
- Now we are ready to start the upgrade process itself. The first thing to do is to shut down Bugzilla, because the DB must not be accessed by anyone but you during the upgrade process. To do that, go to Administration > Parameters > General > shutdownhtml, and add some explanation of what’s happening. For instance: "This installation is being upgraded to Bugzilla 4.2. The downtime should last approximatively 2 hours.", and click the "Save Changes" button at the bottom of the page. I recommend that you don’t leave this page during the upgrade process, because all other pages will be deactivated besides this one. At this point, when someone tries to access Bugzilla during the downtime, this message will be displayed to them instead, so that you can upgrade your installation without having some users still interacting with it.
- Then, and this rule applies all the time when upgrading to a newer major version: do a backup of your database! The reason is that once you start the upgrade process (i.e. when you run checksetup.pl), there is NO way to downgrade later. So if the upgrade process fails for some reason (most of the time because someone hacked the source code or the DB schema) and you made no backup, you are lost. To backup your DB with MySQL, run the following command, where you must replace $db_user and $db_name by their value set in your bugzilla/localconfig file (by default: $db_user = bugs; $db_name = bugs):
mysqldump -u $db_user -p $db_name > db_backup.sql
- Copy the whole data/ directory and the localconfig file from your old bugzilla/ directory into the new bugzilla42/ one. data/ contains your parameters in the data/params file, your local attachments in data/attachments/ as well as data for Old Charts in data/mining/. And localconfig contains all the parameters to access the DB, including its password, the name of the web server group, and some other useful configuration information. If you wrote some extensions, now is a good time to also copy them into bugzilla42/extensions/. Only copy the ones you wrote, not the existing ones such as BmpConvert, Example, OldBugMove or Voting, which are maintained by the Bugzilla team.
- It’s now time to upgrade your DB to work with Bugzilla 4.2. From the bugzilla42/ directory, run ./checksetup.pl with no arguments (so no –check-modules anymore) and let it run the upgrade process for you. As you copied the configuration and parameters files, it knows where the DB is, its password, and everything else it should know about. Depending on the size of your DB and from which version you are upgrading, this may take from a few minutes to several hours. But typically, it’s of the order of a few tens of minutes for a DB with several tens of thousands of bugs. If everything went well, the last message displayed by checksetup.pl must be "checksetup.pl complete" displayed in green. Else this means something went wrong at some point. If half of the messages are in red, then there is no doubt about this.
- The upgrade is now complete, and it’s time to let your users admire this new version. Replace your old bugzilla/ directory by the new one, and re-enable Bugzilla. To do that, you must remove the text you wrote for the shutdownhtml parameter. As we replaced the old directory by the new one, the URL pointing to Bugzilla remains unchanged. You now have a fully-working bzr-based Bugzilla installation.
Now that Bugzilla 4.2 is released, I could finally focus on enhancements instead of fixing blockers and doing QA. This week, I decided to spend some time to improve the performance of Bugzilla. My main focus was show_bug.cgi, i.e. bug reports. I plan to look at other CGI scripts soon, hopefully within 2 weeks.
You will be happy to hear that in the last 3 days, I managed to divide the time spent to load large bug reports such as bmo bug 38862 (235 comments, 32 attachments and 55 CC’ed users) or bmo bug 18574 (760 comments, 63 attachments and 170 CC’ed users!!) by a factor of 2 when using mod_cgi (-50%), and by 30% with mod_perl. The exact load time in seconds depends on the bugs being viewed and on your hardware, but these percentages seem pretty consistent with the different tests done this week. All my patches have been checked in upstream (see rev. 8133 – 8142), and Bugzilla 4.3.1 will be the first release to benefit from them all. Two of them have been backported to 4.2.1 as they are well contained and fix obvious problems, and the other ones are either too invasive for a stable branch or have unclear benefits (the perf win varies between a few 1/1oth of second to 2-3 seconds depending on the test installation).
Now talking about bmo specifically, I gave a look at the InlineHistory extension for the first time, and I proposed a patch which highly decreases the load time of bug reports. dkl did some testing for me with and without mod_perl, and he found these results:
for bug 38862: 5.18 s (unpatched) -> 3.01 s (patched)
for bug 18574 + mod_cgi: 8.01 s (unpatched) -> 4.57 s (patched)
for bug 18574 + mod_perl: 5.76 s (unpatched) -> 4.06 s (patched)
Now I hope the same gain will be visible once these patches are applied to bmo, but the hardware + software configurations of bmo are so different from our test environments that it’s hard to say for sure till the changes are committed and pushed to production. Fingers crossed!
We released Bugzilla 4.2 today, exactly one year after our previous major release, 4.0! Bugzilla 4.2 now supports SQLite, lets you create attachments simply by pasting text into a text field, can send bug changes notifications in HTML format, supports more complex queries, lets you disable old target milestones, versions and components (so that you don’t need to delete them, but also don’t let users report new bugs to them), has accessibility improvements, and much more…
This release also means that Bugzilla 3.4.x is no longer supported. Installations still running 3.4.14 or older are highly encouraged to upgrade to 4.2, especially to benefit from the security improvements made in newer versions. This also means that Bugzilla 4.0.x will now only get security fixes, and other bug fixes won’t be accepted on this branch anymore, unless they fix critical flaws, such as upgrade issues or dataloss.
The Bugzilla team will now focus on the next major release, Bugzilla 4.4, which we expect to release before the end of the year. We expect to release the first development snapshot (4.3.1) in a few weeks. New features will be accepted for the next two months, till the end of April. Then we will focus on stabilization to prepare Bugzilla 4.4rc1.
If you are interested in helping with the development of Bugzilla, now is a good time to join the team and contribute with new features and/or bug fixes. Due to other activities and because life can sometimes make you very busy, some core developers had to stop their contributions to the Bugzilla project in the last few months and so we would be very happy to see new faces. Bugzilla needs to be faster, nicer, more user friendly, and all this is only possible with your help, your ideas and your feedback. So even if you aren’t a Perl expert, there is a lot of place for everyone (you can do a lot with HTML + JS + CSS only, think about the User Interface!). If you are not sure about how to contribute or help, feel free to join us on IRC in the #bugzilla channel. There is always someone around to answer your questions.
We released Bugzilla 4.2rc2, 4.0.4, 3.6.8 and 3.4.14 a few minutes ago. They all contain various security fixes which are described in the Security Advisory. 4.2rc2 should be our last release candidate before 4.2 final, which we expect to release in the 2nd half of February. On the other end, 3.4.14 is very likely our last release for the 3.4 branch. Once 4.2 final is released, we won’t support 3.4 any longer. This means that admins still running 3.4.x or older are highly encouraged to upgrade. Users should pester their admins to upgrade if they don’t do it themselves.
Now is a good time to explain (again) why upgrading is not only about getting new features and bug fixes, but also to keep your installation secure. Below are some security fixes and/or enhancements made to various releases:
Since Bugzilla 4.4, the X-XSS-Protection header is used to block simple XSS attacks.
CSRF vulnerabilities in attachment.cgi and post_bug.cgi
Till recently, no token check was done before accepting new bug submissions or before uploading an attachment to an existing bug. The rationale behind this was that in older versions of Bugzilla there was no easy way to do it from the WebServices API, and we didn’t want to break existing 3rd-party tools which were legitimately interacting with Bugzilla. As the lack of token validation could be used by attackers to submit unwanted new bugs or attachments, it has been decided that a token was required in these cases too, and not only when updating a bug or an attachment. But to not break 3rd-party tools, these token checks have been implemented in Bugzilla 4.2 only, meaning that Bugzilla 4.0 and older are still vulnerable to these attacks. If you want your installation to be protected against this kind of vulnerabilities, upgrade to 4.2!
Configurable password complexity for user accounts
Bugzilla 4.2 has a new parameter which lets admins decide how complex a password must be to be accepted by Bugzilla. Up to 4.0, Bugzilla accepted all passwords which were long enough (min 6 characters by default). Now you can enforce the complexity: uppercase + lowercase characters, letters + numbers, etc… If you want this feature, upgrade to 4.2!
Clickjacking in attachments with the text/html MIME type
As Bugzilla accepts all attachments independently of their MIME type, it was possible to attach HTML files which could try to abuse users using a method known as clickjacking. To prevent this, the "Details" page of attachments now display the source code of these HTML files instead of rendering them. This security enhancement has been implemented in Bugzilla 4.0.4 and newer (including 4.2). If you want it, upgrade!
X-Frame-Options: sameorigin header
Since Bugzilla 4.0, the X-Frame-Options: SAMEORIGIN header is sent for all pages (besides attachments when delivered from their alternate host). This prevents to load a Bugzilla page from within a frame outside Bugzilla itself. This, combined with the clickjacking protection above, prevents an attacker to create an HTML page with malicious code in it to force a user to make undesired changes to Bugzilla. If you want this, upgrade!
Strict-Transport-Security (STS) header
Since Bugzilla 4.0, a new parameter lets admins enable the Strict-Transport-Security header to force the communication between Bugzilla and the user to be made over SSL only. This prevents data to be sent unencrypted.
X-Content-Type-Options: nosniff header
To prevent Internet Explorer 8 and newer from sniffing the content of attachments, Bugzilla 4.0 and newer now pass the X-Content-Type-Options: nosniff header to avoid some malicious attachments to be rendered as HTML files.
Lockout policy to prevent brute force
Since Bugzilla 3.6, if you try to guess someone else’s password and you fail 5 consecutive times, your IP is blocked for the next 30 minutes. If you still run Bugzilla 3.4 or older, Bugzilla will accept all your attempts to crack the victim’s password, severely increasing the risk that the attacker manages to do it.
And much more…
Now I think you know enough about security features implemented in Bugzilla to decide what you want to do. If security matters to you, you should upgrade to at least Bugzilla 4.0.4, and seriously plan to upgrade to 4.2 once it’s released later this month.
After a very long delay due to some nasty blockers, we finally released Bugzilla 4.2rc1 last night! Just to name a few new features or improvements:
- SQLite is now supported and becomes the 4th supported database besides MySQL, PostgreSQL and Oracle. Its support must still be considered as experimental, though.
- It is now possible to create an attachment by pasting text into a text field, without having to save your text as a file on your machine. Of course, you can still upload files as you always did.
- By default, bugmails are now sent in HTML format instead of the plain text format used till now. There is a user preference to select the format you want (text only, or text+html).
- The searching system has been improved, especially the Custom Search section in the Advanced Search page. It’s now easier to build more complex queries.
- Old components, versions and milestones can be disabled if you no longer want users to use them. Bugs which already use them are not affected, but users won’t be able to report new bugs into them (e.g. against an old version, or against a deprecated component).
- A custom field can now be displayed based on multiple values of another field. For instance, this lets you display a custom field in several products. Till now, you had to choose between a single product and all products.
- Most administrative changes made in Bugzilla are now logged and stored in the audit_log table. There is no UI to access this table yet, but developers can already start build their own tools for auditing.
- There have been several accessibility improvements to become more compliant with the W3C Web Accessibility Initiative. The project just started, and a lot of work is still needed.
- Users without editbugs privileges can no longer remove other users from the CC list of bugs.
- The encoding of text files can be automatically detected when uploading them as attachments.
- Tabular reports are now sortable based on any column.
- Buglists have a new default column list: product | component | assignee | bug status | resolution | bug summary | last change date
- Math::Random::Secure is no longer used to generate cryptographically secure pseudorandom numbers. We use Math::Random::ISAAC instead.
- X-Frame-Options = SAMEORIGIN is now passed to all page headers to protect users from framing and subsequent possible clickjacking problems.
- Two new WebServices methods have been added: Product.create and Group.create.
- Bugmails are now fully localizable and customizable (no hardcoded strings anymore).
Read the complete Release Notes to discover other new features or improvements. As it’s still a release candidate, do not forget to report any regression or broken feature to us, so that we can investigate and fix them before 4.2 final.
IMPORTANT NOTE FOR ORACLE USERS: There is still a bug when upgrading to 4.2rc1 using Oracle. Make sure to apply the patch from bug 715870 before upgrading! This patch will be in 4.2 final (or 4.2rc2, if there is one).
We also released Bugzilla 4.0.3, which is our current stable release, and Bugzilla 3.6.7 and 3.4.13. All these releases contain two security fixes. Bugzilla 4.2rc1 also has some security improvements which have not been backported to stable branches to not break 3rd-party applications.
With the coming release of Bugzilla 4.2, the 3.4.x series will reach End Of Life, meaning that no more security fixes will be released for this series. If you have Bugzilla 3.4.x or older, you are highly encouraged to upgrade to Bugzilla 4.2rc1 or 4.0.3.