Today Mozilla proudly presents the next bug fix release for Mozmill. A couple of improvements have been made into this release. The most important part is definitely the support of Firefox 3.7, which allows us to run Mozmill tests against Minefield builds in the near future. But we also support Open Solaris now and offer a much easier way to setup Mozmill on Windows. There is no dependency for pywin32 anymore. And with all the other fixes we really have a shiny new release.
A list of fixes, which have been made it into this release, can be found on Bugzilla. Lets give an explanation for some of those:
- Bug 542000: With former Mozmill releases there were needs to install the Python for Windows extensions (pywin32). If you have used the MozillaBuild environment, changes to the registry were needed. All that work hasn’t to be done from now on. Installing MozillaBuild is enough to prepare the system for Mozmill. A big thanks goes to Atul Varma who removed that dependency and made our life more compelling.
- Bug 543501: One of our upcoming projects is the execution of Mozmill tests against add-ons. Therefore we have to make sure that the usage of Mozmill will be as easy as possible. That’s why the misleading –plugin command line option has been changed to –addons. Make sure to use this option from now on.
- Bug 548446: Mozmill had problems with installing some of the existing extensions because it was not able to find the extension id. Thanks goes to Jonathan!
- Bug 544896: If you have long test-runs it was possible that the software update dialog came up and removed the focus from the window currently under test. That’s why the automatic update for the application has been disabled. If you want to run tests against that feature make sure to enable the preference app.update.enabled before.
- Bug 558404 and Bug 559152: Some controller functions have been updated which missed some details about failures. Now we always have a stacktrace with full information available.
The new version of the add-on can be found on addons.mozilla.org. It’s under review right now but can already be installed. For all the others who are using the command line client of Mozmill, you can simply run the “easy_install -U mozmill” command to update to the new 1.4.1 release.
If you have questions don’t hesitate to send your feedback to the mozmill-dev mailing list or directly contact us on IRC in the #qa channel.
(No Ratings Yet)
Loading ...Tags: automation, firefox, mozilla, mozmill, software, testing
During my stay the Silicon Valley I haven’t only had a great day in San Francisco but also spent a day in the famous whine region of California called Napa Valley. Together with my co-worker Marcia we were driven up those 100 miles to mainly take pictures of the blooming mustard. The day was absolutely sunny but sadly we haven’t had luck to see lot of mustard. We were a bit too late or too early. But beside that we got a lot of impressions and enjoyed taking pictures. I was a bit disappointed that you do not have a chance to get closer to some areas, you have to know that most of those properties are private and visitors aren’t allowed to enter. The only way you can follow is up to the nice little castles and chateaus to taste the local wine. Everyone recommended that tastings but there was no time for us. If you stay in that area, you shouldn’t miss to attend at least one of those tastings.
Back in San Francisco we had a bit of time to drive up the hill in the north of the Golden Gate Bridge. We got a bit higher with the car as Martijn and myself were able the day before with the bikes. You will have a wonderful view from there down to San Francisco. By taking some animal pictures we ended that Sunday with a couple of nice pictures. But see yourself:
(No Ratings Yet) Loading ...Tags: photography, travel
About 2 weeks ago I got my new Nokia N900 device for testing purposes. During the next couple of months I will have to run some dogfooding against the mobile Firefox aka. Fennec. To have it handy all the time I will have to use it as my mobile phone. Therefore it’s necessary to synchronize all the relevant information, e.g. Contacts, Calendars, and Tasks, between OS X and the N900.
There is no official plug-in yet which will allow the synchronization with iSync. But after a quick check I have found a plug-in which is still in a beta state but works quiet well for all of my data, and that in both directions. You can download the software from this website. The installation is fast and you will be able to start the first sync process nearly immediately.
(No Ratings Yet) Loading ...Yesterday I had a great day in San Francisco. My colleague Martijn joined me and we did a walk through the city and rented bikes to visit the Golden Gate Bridge from the other side of the bay. The weather was quiet perfect this time. But see yourself…
(No Ratings Yet) Loading ...Tags: photography, travel
Mozmill, which is a framework for running functional tests, can be used for any application which is built on top of the Mozilla platform. This includes Firefox, Thunderbird, SeaMonkey, and many others. But it’s not only possible to test the application itself. Instead it can also be used to run any type of functional tests for installed add-ons.
To stimulate add-on authors to create their own Mozmill tests, Mozilla QA is holding a testday on Friday, March 5th, which is fully devoted to Mozmill testscripting for your add-on. Learn how Mozmill tests will be written and how they can be run in Firefox. The Mozmill team will be around the whole day to assist you wherever possible.
If you are interested in the testday, you should read through the following documentation about the creation of testscripts for extensions.
You can also attend, when you have general questions about Mozmill or when you want to help in creating Mozmill tests for Firefox. Get ready and join us in #testday next Friday.
(No Ratings Yet) Loading ...Everyone who was not able to make it to my presentation during the FOSDEM 2010 you can dig into my slides on Slideshare. Live demonstrations are not contained.
(No Ratings Yet) Loading ...Over the next weekend the next FOSDEM will take place once more in Brussels (Belgium). It’s a two day event completely devoted to the free and open source software.
Mozilla has its own developer room again and offers a couple of interesting talks during both days. For the first time I will not only be a participant but also have my own talk. That means that everyone who is interested to hear about the basics of Mozmill and the current state of Mozmill testing is welcome to join my Mozmill talk on Sunday, Feb. 7th, at 10am. There will be a lot of live demonstrations about various areas QA is working on.
(No Ratings Yet) Loading ...Starting today the Mozilla QA lab has it’s own automated Mozmill test-run in place which gets executed once a day. That means I do not have to run those tests manually anymore and we also have public available results which will give us more exposure regarding failed tests. Really, it’s a great step forward for us.
But lets talk in-detail about the progress…
During the last All Hands week in December 2009 I had to hold a presentation in one of our QA group meetings about the current state of the Mozmill tests and about the software update tests. Everyone was excited to see the improvements especially for the update tests which do not need any cumbersome file edits anymore. Finally we had an interesting discussion and collected some ideas about how to run those Mozmill tests in the future. As result we decided to setup a machine in the QA lab which will run Mozmill tests once a day against nightly builds of Namoroka and Shiretoko on OS X, Linux, and Windows. Further we wanted to use it as our machine to run Mozmill tests against release candidates and to verify the update channels. No sooner said than done…
Directly after the Christmas holidays my colleague Al ordered a Mac Mini and did the initial setup routine. He handed me over a system with 3 partitions. One for OS X 10.5, another one for OS X 10.6, and a third partition which is used for all the virtual machines and other necessary data. While talking about virtual machines the following will be in use: Ubuntu 9.10, Windows 2000, Windows XP, Windows Vista, and Windows 7.
My first task was to setup the initial Mozmill environment for all those operating systems. There is a nice documentation up on MDC which helps to make the task as easy as possible. To be able to share builds, scripts, and other stuff between the vms each of those has the shared folders feature enabled and a virtual drive connected to the data partition.
For the daily test-runs against en-US nightly builds of Firefox 3.6.x (Namoroka) and Firefox 3.5.x (Shiretoko), we decided to have OS X 10.5, Ubuntu 9.10, and Windows XP always open. With the help of the crontab on Linux and OS X, and the scheduled tasks on Windows we are able to start the tests at the same time each day. Given by the availability of the update snippets we have chosen 8am PDT for now. I’m not sure if that will work every day due to some slightly time shifts but it’s still something we have to improve. To run all the available tests I have written a wrapper script in Python which runs each of the different tests sequentially. This script will be used from any of the above mentioned vms. The order how the tests are run is given by:
- Software Update tests: With the software update tests we check that nightly updates can be downloaded and applied via the software update dialog. Also we make sure to run the successive Mozmill tests against the most recent nightly build.
- Normal tests: Those tests cover most of our implemented tests for the Smoketests, BFT, and FFT test group which do not require a restart of the browser.
- Restart tests: The restart tests cover the remaining tests and are able to even test complex paths which require a restart of the browser.
Now someone may ask how do we track the results. There is no-one sitting on that machine. That’s correct. Therefor we use the rich feature set of Mozmill 1.4 which is able to send the results as a JSON object to any CouchDB server. For our usage inside QA such a server has been setup a while ago and is also used by our Mozmill tests now. It’s publicly accessible which means that everyone can check and analyze our daily results. If you are interested check-out the summary page on brasstacks. Click on any of the entries to see all the details of each test-run. For the moment the web interface is really simple and hackish but it gives us the chance to easily track failed tests.
Within the next weeks or months a couple of improvements have to be made on the client side but also for the web interface. In general it covers:
- Finalize the decision which application information has to be acquired by Mozmill and has to be send to brasstacks.
- Make Mozmill more robust against failures. The biggest problem we have are modal dialogs which can cause a hang of the complete test-run.
- The software update tests have to be fixed to send results for each test module. Actually only for the first module results are available.
- Complete new design views are needed to give an easy and structured access to all of the existent results. That is probably the biggest chunk. A separate blog post will follow once we start to work on that item.
Even with those tasks in the queue we are ready to run daily Mozmill tests continuously each day and use the same machine for our release testing work.
As usual we do not rest, so expect more to hear about Mozmill testing in the near future.
(No Ratings Yet) Loading ...Tags: automation, couchdb, firefox, mozilla, mozmill, software, testing
The new version of Mozmill brings some under the hood changes to the Python back-end.
The patch on Bug 533227 adds report support for restart tests which can push results to a couchdb instance now. With Mozmill 1.3 only normal Mozmill tests had that capability.
Further we have improved the handling of Python callbacks from within Mozmill tests. While the former version of Mozmill has only supported those callbacks for restart tests, you can use them in normal tests (Bug 538058) now. The only restriction is that the test-run has to be started from the command line client. Users with only the extension installed will not have access to this feature. In the future we will probably use it to implement handling of system level elements and events. Ongoing work can be checked on Bug 525060.
Another great enhancement we are proud to announce are the simplified steps to install Mozmill as command line client on Windows. The only requirement we have is the Mozilla Build environment. There is no need to install Python or the pywin32 extension in parallel anymore. Complete instructions can be found on MDC. A big thanks goes to Atul Varma for his contribution. You helped us a lot!
For more information check-out the complete list of fixes.
Mozmill 1.4 is still in the review queue on addons.mozilla.org. It will take a bit until it is officially available there. So for now grab a copy from Mikeal’s repository and give it a try.
(No Ratings Yet) Loading ...Yesterday, the Mozilla Corporation has been released the brand new Firefox 3.6 which is the successor of Firefox 3.5 and has a couple of interesting new features and speed enhancements aboard. Let’s have a quick look at the release notes and checkout what makes it a shiny and brilliant new version:
- With the Lightweight Themes support users will be able to skin their browser chrome without having to install Personas. Browse through the list of the most popular Personas and hover the different themes to see a preview directly pinned onto your browser. A click on “Wear It” will finally choose the skin.
- The new outdated plug-in protection checks the installed plug-ins on your system and will warn you when a newer and safer version is available.
- Enjoy full-screen playback for HTML5 video elements.
- Improved start-up times, better responsiveness and faster Javascript execution which were a main focus for 3.6.
- Support for new CSS attributes like gradients, background sizing, and pointer events.
- Support for new DOM and HTML5 specifications including the Drag & Drop API and the File API.
- Firefox 3.6 blocks unknown and vulnerable libraries to lower the risk of crashes. In the past a couple of 3rd party tools have been circumvent the official way in installing add-ons which lead to a significant number of crashes after an upgrade of Firefox.
- Full list of features.
With a release cycle of only 7 months Mozilla has been shown that it’s possible to release a new major version of the Firefox browser in such a short interval. And that’s absolutely necessary when comparing Firefox with products of other browser vendors. Bringing the newest technology as fast as possible to our users will full-fill their needs and finally help the continuously increment of market share.
But all that work wouldn’t have been possible without the help of the overwhelming community around Firefox. Thousands of contributors spent hours of their spare time in developing, localizing and testing Firefox. We can always repeat: Thank you soo much!!
For myself it was the second Firefox release I have worked on as an employee of the Mozilla Corporation and each week has been shown that it is a pleasure to work with all those amazing people around the world. I’m feeling great to be able to spend all my time fully devoted to Firefox. It was a great experience and a path of learning new technology, improving social interactions, and making the web better.
Given Firefox 3.6 I was able to work on projects like:
- DLL blocklisting / Components folder lockdown: 3rd party applications which install their own modules inside the components folder of Firefox or LSP’s (Layered Service Provider) which get bind to the Firefox process can crash Firefox. With DLL blocklisting those crashing or malicious modules will be blacklisted and not loaded anymore. On the other hand only known XPCOM modules will be loaded thanks to the components folder lockdown feature. Both features have been tested with various top100 3rd party applications.
- Tab Network Prioritization: Restoring a Firefox session with a dozen of open tabs in one or multiple windows slows down the restoration process more or less. Until now there was no specification in which order those tabs get loaded. Means tabs outside of the visible tabbar or in underlying windows were restored in parallel. As result you had to wait longer until the currently selected tab has been finished loading. From now on the current tab has the highest priority, followed by visible tabs in the same window and tabs in underlying windows. As tests have been shown this feature works very well in Firefox 3.6. More improvements will come with the next version.
- Crashkill: To lower the number of crashes for Firefox the Crashkill project has been initiated a while back. In the last quarter I was able to work on a couple of those bugs. Thanks goes to Chris Hofmann who showed me a couple of nifty tricks and tools regarding crash analysis.
- Mozmill: As the lead of the Mozmill test automation process I had to put focus on Firefox 3.6. So all manual tests from the Smoketest and BFT testgroup, which have been already automated, can be used to run automated functional tests against Firefox 3.6. The same applies to the automated software update tests which make sure that update paths are working as expected. As seen for the previous 3.6 beta and 3.5.x security releases all those tests are very helpful and give time for QA to focus testing on more important areas.
Finally I can say that I’m feeling great with Firefox 3.6 as my daily companion but also looking forward to any upcoming work which has to be done for the next major version of Firefox…
(No Ratings Yet) Loading ...
Recent Comments