MozMill Crowd 0.1.2 released

The Mozilla QA Automation team proudly presents the next version of the MozMill Crowd extension which has been released today.

With Mozmill Crowd 0.1.2 we now support the Endurance Tests, which have been used a lot in the last couple of weeks to find memory leaks in Firefox. Now as part of the crowd extension, those tests are as easy to execute as all the other supported test-runs. Only if you would like to run those tests with a given set of add-ons installed, you will have to run the tests outside of the extension. Dave Hunt wrote an excellent article about this topic recently.

Further we were also able to remove the dependency for the Python headers on Linux. That means there is no pre-requisite anymore which has to be fulfilled before Mozmill Crowd will work. Only on OS X you will have to install the Developer Tools because VirtualEnv requires it to work. We hope to fix that by not using VirtualEnv in one of the upcoming releases.

A complete list of fixed bugs you can find on the tracking bug for the Mozmill Crowd 0.1.2 release.

Important Note: The extension is still not able to detect if an updated test environment has been made available. If you already have used MozMill Crowd before make sure that you manually delete the ‘mozmill-env.zip’ file and ‘mozmill-env’ folder in the selected storage folder. Triggering a new test-run will automatically download and install the newest version of the environment.

Also please check out our tomorrows testday which is completely devoted to the MozMill Crowd extension and the test execution in Firefox 4.

New Structure of the mozmill-tests Repository

Since I have been started to work on Mozmill tests for Firefox early 2009, our QA Automation Team has been grown and got a lot of work done. A fair amount of functional tests have been created, followed by a separate test-run of automated software update tests. Those two kinds of tests have been proven to reduce the time for release testing for QA drastically, and as result we have added even more test-runs like for add-ons, l10n, and lately for endurance tests.

As more and more different runs are getting added the structure of the whole repository, we have agreed on in 2009, doesn’t make sense anymore. Also looking ahead to our upcoming API refactoring which will land soon, we had to chose a new structure of the whole repository. A discussion was necessary to make sure the new hierarchy will fit our needs for the next couple of months or even years. And I can say for sure that we will add a lot more of test-runs and those need the right place to live.

To get this step accomplished, I worked on top of the specs and got the new structure implemented today. If you are interested in check bug 637306. So without reading through the bug you may ask, what has been changed? Lets compare in detail the structure before and after my work.

Before:

  • addons (Tests for add-ons)
  • firefox (Folder of mixed tests for Firefox)
    • crashTests (Crash tests)
    • enduranceTests (Endurance tests)
    • helperClasses (Tests for shared code)
    • l10nTests (l10n tests)
    • restartTests (Functional restart tests)
    • softwareUpdate (Update tests)
    • test-files (Test data served via httpd.js)
    • test* (Folders which contain functional tests)
  • shared-modules (Shared code divided into components)
  • templates (Simple templates for starters)

After:

  • data (Test data served via httpd.js)
  • lib (Shared code divided into components)
    • tests (Tests for shared code)
  • tests (Folder for the supported test-runs)
    • addons (Tests for add-ons)
    • crash (Crash tests)
    • endurance (Endurance tests)
    • functional (Functional tests for Firefox)
      • restartTests (Functional restart tests)
    • l10n (l10n tests)
    • update (Update tests)
  • templates (Simple templates for starters)

As you can see sections are clearly defined and it’s easy to find the code you are looking for. It’s the perfect base for our future work, you will hear about in the next couple of weeks.