Now, three weeks after Q1 of 2012 has been passed by I finally have the time to give some details about our work happened during the first three months.
As usual we set a couple of goals we wanted to see by the end of the quarter. All of them were kinda important so we did our best to get all of them done – and we succeeded even if it was in the last minute! So everyone of us was happy about the goal achievements.
Below you will find an overview of each of those projects. For more detailed information I will follow-up with additional blog posts by next week.
Mozmill CI system for release, daily, and l10n builds
It has been taken a while for us to get to a state when we absolutely had to work on a continuous integration solution. Seeing that more and more tests are getting automated with Mozmill and that we still require manual interaction, or crontab jobs and tasks on Windows to get tests executed, we had to find a better way in getting tests run. After a bit of investigation we started by using Jenkins (formerly known as Hudson). And since beginning of this month we now have a system which runs our Mozmill tests for all the supported daily and beta candidate builds, and for each tinderbox build based on l10n check-ins on whichever branch. As trigger we make use of Mozilla Pulse which informs us directly when builds have been made available. Further we also had to ensure that tests can be triggered by on-demand, because update tests cannot be covered by Pulse yet and sometimes we have to re-run tests against a specific build. All in all the system works great but still needs a lot of improvements on which we will work on in the next couple of months.
Automation system for the Add-ons compatibile by default feature
With the change in Firefox 11 to make all add-ons compatible by default, we have been asked to create a system to automatically test add-on compatibility for the top 100 add-ons. As result a new testrun for Mozmill has been created which runs our endurance, update, and endurance tests again. That way we want to make sure that there are no performance regressions involved and updates of Firefox will be applied without any failures.
MemChaser – An extension to visualize memory and GC/CC related activities
Given the amount of manual testing for GC/CC related issues in Firefox by end of 2011, I had the idea to create an extension which makes it easy for users to track memory related activities. That means that all the important information is displayed directly in the add-on bar and don’t have to be retrieved from ‘about:memory‘ or the Error Console (which constantly wipes out old entries). The idea got approved as goal so Dave Hunt, David Guo, and myself were heavily working on getting this extension released on addons.mozilla.org as soon as possible. Meanwhile even more features have been implemented and more are about to come in the next couple of months. If you want to follow the development of the extension check the Github repository.
Mozmill Dashboard – Validation method for uploaded test reports
During a security review of our dashboard by end of 2011 it has been turned out that arbitrary data can be uploaded to our dashboard. The reason was that we haven’t checked the data which gets uploaded before storing it in our database. So we have had to implement a validation method for the CouchDB database. Now the report data gets checked and if the method fails the document will not be stored.
As mentioned before I will follow-up with detailed information later. So please watch out for upcoming blog posts in the next days.