New daily Mozmill test-runs for Firefox

For a long time, exactly since January 29th in 2010, the Mozilla QA team is running daily Mozmill tests to prove the functionality of Firefox and the update system across all supported platforms and release branches. With those tests we were able to find a couple of regressions, which then were fixed before the next release of Firefox.

Given those results we have decided to extend our daily test-runs and not only run the functional and update tests, but also cover endurance tests, add-on tests, and the new remote tests.

Especially with the endurance tests, Dave Hunt has shown that we can retrieve a lot of useful memory related information and find regressions or bad interactions with add-ons very easily. So it makes a lot of sense to additionally perform those tests on Nightly and Aurora builds of Firefox. The test results can be found on our dashboard.

But also for add-on authors who have written Mozmill tests for their add-ons, we now provide test results. Regressions in the add-on or in Firefox itself can now be identified as close as possible. Currently only 2 add-ons have Mozmill tests and are being tested. If you are an add-on author get in contact with us, and we can work together to get your add-on tested.

Remote tests, which we haven’t talked about yet, are tests which connect the testing of the Firefox UI and remote websites. Those are fairly new, means they have been added recently, and currently only cover tests for the ‘Get Add-ons’ pane of the new Add-ons Manager. Those were necessary to create because Selenium can not automate the installation of add-ons. For the latest test results see our dashboard.

We are now looking forward to any regression we can detect with the newly added test-runs. And those we can not only identify by our daily test-runs, but also from reports sent by users of the Mozmill Crowd add-on. So please help us testing Firefox by sending your test reports. Thanks!