Ich wusste jedoch bereits vorab, dass ich hier meine Bestzeit vom letzten Oktober nicht wiederholen oder verbessern kann. Die 40km Wandertour vom Donnerstag lag mir einfach noch zu schwer in den Beinen. Trotzdem wollte ich die bestmögliche Zeit herausholen. Ich startet mit einem wohl etwas zu schnellen Tempo. Das legte sich aber wieder schnell, denn gleich von Anfang an gab es auf einigen Streckenabschnitten einen ordentlichen Gegenwind, dem wir alle trotzen mussten. Mein Tempo reduzierte sich somit auf die üblichen 5:00 min/km und ich entschied dabei zu bleiben, um durchzukommen. Das lief die folgenden Kilometer ganz gut, bis mich ab dem 7. Kilometer heftige Seitenstechen, hervorgerufen durch eine dumme Bewegung, quälten. Wie nervig das ist, kennt sicherlich jeder von uns mehr oder minder. Leider hielten diese fast 3 Kilometer an und raubten mir gute Zeit. Dann waren sie auf einmal weg und ich konnte im letzten Kilometer noch einmal gut zulegen. Ich erreichte das Ziel auf dem 68. Platz meiner Altersklasse und dem 540. Platz der Männer. Ich bin zufrieden auch wenn es etwas oberhalb der 50 Minuten-Marke liegt.

Beim nächsten Lauf (Sportcheck Stadtlauf in Dresden) am 16. Juni 2013 habe ich mir nun den Halbmarathon als Ziel gesetzt. Dafür gilt es nun gut zu trainieren…

Während meines Urlaubs diese Woche wollte ich eigentlich mindestens eine Wandertour absolvieren. Erhofft hatte ich Sonnenschein und etwas wärmere Temperaturen. Jedoch Pustekuchen. Es gibt Schnee, Schnee und noch mal Schnee, und das Thermometer zeigt auch Minusgrade. Ist das ein Grund alles abzublasen? Nein, nicht für mich. Also Pläne geschmiedet und rausgekommen ist dabei eine Tour von Bischofswerda nach Dresden. Laut Erfahrung, da ich diese Strecke ab und an mit dem Fahrrad fahre, sind es circa 40 Kilometer. Das wäre ja fast Marathon-Distanz. Schaffe ich das? Die letzte Wanderung liegt eine Weile zurück und in all der letzten Zeit bin ich auch nicht mehr als 10km gerannt. Schwere Entscheidung, aber ich will es probieren. Wer nicht wagt, der nicht gewinnt…

Gestartet bin ich letztendlich vorgestern gegen 8:20 Uhr in Bischofswerda. Mit angepeilten 6 km/h und ein paar kleinen Pausen sollte ich somit die Strecke in ca. 7.5h schaffen. Wird der Schnee hinderlich sein? Bestimmt, denn gerade in der Dresdner Heide erwarte ich nicht durchweg geräumte Wege. Und wie es sich herausstellte ist auch die Glätte nicht zu verachten. Gerade auf den ersten Kilometern über das freie Feld wurde ich dadurch behindert. Aber ab Rammenau wurde es besser.

Ganz im Eifer des Gefechts vergaß ich natürlich meine Garmin-Uhr zu überprüfen, ob denn die GPS-Satelliten gefunden wurden und die Strecke getrackt wird. Wie ich 3 Kilometer später feststellen musste, war das leider nicht so. Das erklärt die horizontale Linie im unteren Garmin-Track. Rechnet also bitte 3 Kilometer hinzu.

Eine der wohl langweiligsten Abschnitte der Wanderung war definitiv das Stück von Hauswalde über Bretnig und Großröhrsdorf. Mit dem Fahrrad ist es schon lang, aber zu Fuß fühlen sich diese 8 Kilometer durch diese zusammenhängenden Dörfer unendlich an. Hier muss ich falls es ein nächstes Mal gibt definitiv eine andere Route nehmen. Vorteil dieses Mal war aber auf jeden Fall der geräumte Fußweg, sodass ich gut voran kam.

Nach einer kurzen ersten Pause hinter Großröhrsdorf lag nun eine weitere Strecke über freies Feld vor mir. Radeberg ist von hier aus nicht mehr so weit weg, jedoch pfiff mir der eisige Wind genau ins Gesicht. Stetig merkte ich, wie ich trotz des Laufens immer weiter auskühlte. Dickere Handschuhe wären jetzt mehr als angebracht gewesen. Da aber keine verfügbar waren, kämpfte ich mich bis zu einem Imbiss in Radeberg. Dort musste ich die schon etwas steifen Finger erst erwärmen, bevor ich essen konnte. Das tat sehr gut, aber auch dass die Beine endlich eine Pause bekamen. Es waren zwar erst 24 Kilometer, aber die Oberschenkel meldeten sich bereits etwas. Zirka 16 Kilometer lagen noch vor mir.

Frisch gestärkt und aufgewärmt ging es kurze Zeit später weiter. Der Rest der Strecke sollte jetzt großteils durch die Dresdner Heide gehen. Dazu musste ich nur noch den richtigen Einstieg finden. Die Beschilderung war mager oder besser garnicht vorhanden. Mein Android Smartphone leistete mir in dem Moment wirklich gute Dienste und der C-Flügel war schnell gefunden. Der Weg wurde auch gleich deutlich schlechter begehbar. Glücklicherweise gab es jedoch eine Loipe inmitten des Weges, die bereits von anderen Fußgängern begangen wurde. Ich nahm mich ihr also gleich an. Nach etwa 3 Kilometern kam ich dann doch zu meiner Überraschung auf geräumte Waldwege. Grund waren die anhaltenden Forstarbeiten in dem Gebiet. Meine Oberschenkel fingen jetzt an, bei jedem Schritt zu schmerzen. Ich wusste nicht, ob ich all die restlichen Kilometer noch laufen kann. Ich nahm es mir jedoch stark vor. Kilometer für Kilometer zog sich hin und bei den vielen geraden Wegen spürt man auch kein wirkliches Vorankommen. Zähne zusammenbeißen, Henrik!

Als endlich irgendwann Spaziergänger auftauchten atmete ich auf. Jetzt kann es nicht mehr weit sein! Zumindest ging es bergab. Ein Zeichen, dass man ins Elbtal kommt. Doch es zog und zog sich. Die Beine schmerzten mittlerweile höllisch. Vor Allem das Stauchen in den Knien und eins-zwei leichte Durchsacker machten mir klar, dass ich mein Tempo drosseln und besser auf den Weg achten musste. Werde ich es noch schaffen? Schließlich sind es noch weitere 4.5km durch die Stadt.

Verkehrsgeräusche ließen mich endlich aufhorchen und gaben mir zusätzlich Kraft. Ein paar Minuten später stand ich auf der Stauffenberger Allee. Dichter Verkehr und Abgase machten sich breit. Eine doch krasse Veränderung zu all den letzten Stunden. Ich wäre gern wieder umgekehrt.

Ein Bus kam. Hm, die Nummer, die genau bis zu meinem Haus fährt. Fahre ich mit? Ich kann wirklich nicht mehr. NEIN! Keineswegs, du läufst weiter. Zähneknirschend überwandt ich den inneren Schweinehund und lief weiter. Vorbei an all den im Stau steckenden Autos. Wenn die Insassen wüssten, was ich schon hinter mir habe… Auf den restlichen 3km kamen weitere 3 Busse. Jedes Mal musste ich mehr dagegen ankämpfen, nicht schwach zu werden. Der innere Wille blieb stärker. Zum Glück, denn als ich endlich angekommen war, standen fast genau 40km auf meiner Uhr. Wow, wenn das nicht gut geschätzt war. Nur die benötigte Zeit war mit 7h länger als vorher gedacht. Das ist für mich kein Problem wenn man die Witterung und Bodenbeschaffenheit betrachtet.

Ich bin überaus glücklich, alles gelaufen zu sein und ein Gefühl bekommen zu haben, was eine Marathonstrecke ist. Unglaublich! Ich weiß nicht, ob ich das jemals laufen werden kann. Aber ein Ziel für die Zukunft hab ich jetzt vor Augen! Irgendwann werde ich die Kondition haben, einen Marathon laufen zu können und dann muss diese Strecke wieder daran glauben!

Jetzt heißt es erstmal zwei Tage pausieren, und am Sonntag geht es dann weiter mit dem ersten offiziellen 10km City-Lauf hier in Dresden.

Sport frei!

I hope you will like that change.

My task together with Rob Wood – we both are from the Automation Development team – will be a hacking session about WebAPI test automation. WebAPI’s are a crucial part of the Firefox OS (B2G) initiative, Mozilla is putting a lot of work in to bring the open web even to mobile devices.

If you are interested in raising your skills in automation and want to help us to strengthen the quality of Firefox OS please join us on Saturday at 3:05pm in B2G room 1. We still have a couple of seats left. To be best prepared please read through our session details and try to setup your machine before you leave for MozCamp. We have setup links to get all necessary tools and the prepared Ubuntu VM. If you aren’t able to do that, don’t worry, we will have USB thumb drives available for you to grab all the data you will need.

As a sneak-peak we will also give you a link to our introduction slides so that you can change your mind if you are unsure about your attendance:

We are looking forward and can’t await to meet-up with you all. So see you @MozCamp on Saturday!

Update: We are planning to send out prizes for the best hand-crafted test(s) some days after MozCamp when we have reviewed all the written tests. So please be aware of code correctness, coverage, and style while implementing the tests.

So what is that amazing on such a session? Given that in the past we as team were mostly working on Bugzilla to get a plan how to solve a specific problem, a live discussion is indeed a lot more helpful. Every attendee can directly participate, ask if there are unclear steps or if there are better ways to accomplish the wanted goal. The discussion is ongoing until you really met your expectation, or the time is up. That’s the difference to threaded conversations via email, where you should expect delays in the turnaround. You will lose valuable time which is even harder when you already have a strong timeline to finish up a feature.

The Automation Development team strongly believes that we all can work together to improve the Test Automation area at Mozilla even more. To achieve that goal we will continue in having live discussions each week and to help others in increasing their skills in automation.

And as a hint for all MozCamp attendees: If you are around keep an eye out for our “Make Sure Your Code Works” hacking session on Saturday at 3:05pm. I will post details about it before MozCamp but just as a heads-up so you can mark it in your calendar.

At Mozilla we have dozen of test frameworks in use; like Mochitests, Selenium, Mozmill; which mostly dependents on the product under test. With this large matrix it’s sometimes not that easy to figure out when a test can be automated and which framework is best to use for. Even there can be overlaps because multiple frameworks would fit for a particular test. Confusion is pre-programmed.

So the Automation Development team wants to bring some light into the darkness by offering help for those questions. Therefore we will start to host office hour like sessions each week on Thursday at 12:00pm UTC. This will be done via our Vidyo infrastructure and you will only have to install the Vidyo desktop client as suggested when trying to enter the room the first time. For details and up-2-date information please visit our meeting page.

Given that we don’t exist that long yet and are 3 people only, we will start with the Mozmill, Selenium, and B2G testing frameworks. We will extend our coverage to other frameworks once we have the knowledge and man power, and know that sessions like those are helpful and accepted by others. Also please note that the current time has been chosen to start improving our relationship with community members in Europe first – here most of them are located for the above mentioned frameworks. In the future we are planning to have another session for US folks.

We are kinda excited about this opportunity and we would like to invite everyone who has interest to join and actively participate in the discussion.

As we have agreed on it is important to scale our visibility and make it easier for both developers and QA to determine if a feature has automated tests, whether in the tree, in the Mozmill tests repository, or any other framework we would have to create in the future. A whiteboard entry didn’t make much sense for us, and to better align with the existent ‘in-testsuite’ flag, we decided on the ‘in-qa-testsuite‘ flag.

So if you think that a feature needs coverage by Mozmill, make sure to set this flag and assign myself (use “:whimboo” as name). I will then ensure the request will be handled appropriately and the test gets implemented.

Further documentation on automation frameworks and this flag will be available soon. I will blog about once it is ready.

This work is part of our overall code rewrite of the Mozmill Automation scripts for Mozmill 2.0, whereby a lot of useful code will be merged with existing mozbase packages, or new packages like the mozdownload one will be created.

Keep in mind that this release changes the API of several functions. We are aware of that and hope that everyone of you will be able to adapt to it easily. It was necessary before more and more projects rely on this package. For any details please see the appropriate README file. Otherwise here some examples:

CLI:

$ cd /Volumes/mozilla/builds/
$ mozinstall firefox-13.0-en-US.dmg
/Volumes/mozilla/builds/FirefoxAurora.app/Contents/MacOS/firefox
$ mozuninstall firefox/

With those two commands the specified DMG image of Firefox 13 will be installed into a sub folder of the current working directory and removed right after. The installer will print the location of the actual binary to stdout.

API:

import mozinstall
path = mozinstall.install('firefox-13.0-en-US.exe)
print mozinstall.get_binary(path, 'firefox')
mozinstall.uninstall(path)

This code is the equivalent to the CLI example above. It installs Firefox into a sub folder of the current working directory, and returns the path. With the call to get_binary and by specifying the type of application, the actual binary will be returned. Replace ‘firefox’ with ‘thunderbird’ if you want to install a Thunderbird build. Finally the uninstaller expects the path Firefox has been installed into, and removes the application.

Given the EOL of Firefox 3.6 and the Gecko 3.6 platform we do not intent to push another hotfix release for Mozmill 1.5 at this time. If you have to test applications which are based on Gecko 1.9.2 you should use Mozmill 1.5.11.

If you feel that this doesn’t solve your problem and we should re-add support for Firefox 3.6, please get in contact with us and together we can figure out what will be the best strategy for you.

1. We have to do reviews faster. Right now I see a lot of time wasted because the reviewer of tests is in a different timezone and cannot react immediately to requests. That means that patches which have been put up for review will not get reviewed before the evening in Europe. Revising the patch has to happen the next day. Normally we have at least two iterations on a patch so it takes about 2 or 3 days until the patch is ready for check-in. I would like to reduce this time to 1 or at maximum 2 days by promoting at least us (Dave Hunt and me) as reviewers. This applies to new and broken tests. Please split up review requests between us.
2. We have to ensure that the test covers the appropriate manual test on Litmus. Not sure how close we are at this right now because I never did those reviews since they moved over to Anthony Hughes. If we were good in that area lets keep up that work, otherwise we have to improve. That said we will split up the process in a review and feedback section. While the reviewer takes care about the actual code implementation, the person for feedback checks the test
   for completeness. Only if both areas are fulfilled the test can be checked in. While Dave and myself will take care about reviews, Anthony is the one who knows the most of the current Firefox development and should act as the feedback person for now. Both can happen independently and should not be canceled if the other request gets denied.
3. We have to fix broken tests and do not only skip them. In the past few months a couple of tests were failing and have been marked as skipped. It has happened a couple of times so that we now end-up with a lot of skipped tests. That doesn’t really help us and we have to get better. So before we create and work on new tests we have to make sure to really fix the broken tests. There should be used a round robin method between all of us (Automation Development and Desktop Automation) so that we will not have any skipped tests due to failures for each of the branches. I think that should be doable.
4. We have to start the education. Learning from the experience of users while giving reviews will train Dave and me to see where the problems are located. That means that we can make sure to get more and better documentation up for those areas. If we wouldn’t work on reviews we would simply miss those parts. As result everyone who wants to contribute will benefit from and become a writer for automated tests in a shorter time.

I’m sure that with all those changes we can drastically improve the code quality and coverage of automated functional tests with Mozmill.

Automation is a key part when it comes to qualifying new and existing code. Over the last years it’s getting more and more important at Mozilla for developers and QA to proof the quality of Firefox on a fast pace without the hassle of manual testing. With the change in the Firefox release process to a rapid train model a ton of different builds have to be tested by QA on a regular basis. Doing all that manually is impossible. A test framework which has a high impact for QA is Mozmill and the tests and automation tools written around by the Automation Services team over the last couple of months help a lot. But at the current state it’s not enough. We have to be better and cover way more of the manual testing with automated tests.

To make that possible the Automation Services team is now part of the Automation and Tools Engineering team (aka A-team) and has been renamed to Automation Development. We, which means Dave Hunt and myself, are part of the crew who will be responsible for any kind of automation question. For now we can only cover the Mozmill and Selenium frameworks but with additional team members added hopefully soon, more and more harnesses can be covered. Another key aspect for our crew will be mentorship and education, which means that we are eager to bring you into the automation land and to know all the stuff to write automated tests on your own. Sounds interesting? It definitely is, and I like it a lot!

So what will we work on during the current quarter?

There is a lot of work which has to be done for Mozmill to finally get out version 2.0 and to update our automation scripts and Mozmill tests, including a new API to write the tests. So those will be our prioritized projects.

1. Mozmill 2.0: A new major version of Mozmill has to be released to fix a lot of our problems we still have on the 1.5 branch, and which can’t be easily solved there. The development is already active for a lot of months but we were never able to finish it up. Given that this new version is a blocker for us to create more reliable Mozmill tests, lets get it hopefully done by this quarter. If you have Python or Javascript skills stop by and let us know. We could need any help we can get.
2. Automation Scripts: Those scripts drive our Mozmill test-runs for different versions of Firefox. They have been written in the early stages around Mozmill 1.0 and need a complete refactor to work with Mozmill 2.0. Also a couple of features will have to be moved to MozBase so that even other harnesses can benefit from. This is all Python related work.
3. Test API refactor: In parallel to the above mentioned projects we will also have to update our API, used by Mozmill tests, to make it easier to write new and maintain existing tests. This is absolutely important especially for new members who start with Mozmill. If you have Javascript skills and want to improve them, let us know too.

Beside those projects we will also continue our work in the following areas:

• Mozmill CI: More and more tests are getting covered by our new Mozmill CI system, which I will talk about more in detail soon. So we have to move this instance to an ESX cluster to have a ton of machines for test execution, include handling of the DTC (default to compatible) test-run, and make sure that the upcoming Firefox Beta Nightly builds are handled too.
• Mozmill Tests: We take a stab in handling and coordinating the creation of new Mozmill tests and fixing the broken ones. The reason for this change I will also explain in a separate blog post shortly. But to say in general we need more test coverage and skipped/failing tests over a long time are bad practice and counter-productive. So lets get all of them re-enabled again and new tests written.
• Addons: There will not be that much time for us this quarter to continue developing the add-ons we own. But we will make sure to fix breakage for MemChaser and Nightly Tester Tools if those occur. If you want to help out, you are more then welcome!
• WebApps Testing: While this is not on our official goals for this quarter we have to assist the WebApps QA team to figure out the right test harnesses to create automated functional tests for open web apps. It will be a challenge and we are kinda interested to see that upcoming.

As you can see there are plenty of projects to work on. Any of those are public and are waiting for your contribution. If you are interested in working with us and to raise your personal skills, please contact us via our public mailing list or #automation IRC channel.

But things can change, and sometimes faster as expected. So by last Friday (it was already late in the evening) Mike Conley contacted me on IRC and mentioned that Mozmill will be broken starting with tomorrows build. The reason for it was the ‘Global-Per-Compartment’ patch which has been landed in Nightly builds of Firefox and Thunderbird at that time. Great! Why do things like those happen just before when I’m already in my weekend?

Anyway, given the importance of this bug (it broke the whole test infrastructure of Thunderbird which mostly relies on Mozmill) I was talking immediately to Clint Talbert. Given the good bug report from John Schoenick we were directly aware of what the problem was and agreed on a new API to get the issue fixed.

So I have spent a good portion of my Saturday and Sunday evening to put together a patch which removed the broken code in waitForPageLoad() and replaces it with the new API. So by now we do no longer add a custom ‘mozmillDocumentLoaded’ property to all windows, but have a nice windowMap object that handles all the loaded states of each window on its own. I think that’s great and modifying the DOM of each window was even a bad idea since the beginning of the project. Good to see that killed.

That means early today we got all patches landed and I was able to release Mozmill 1.5.12 to PyPI and even the uploaded extension on addons.mozilla.org has been approved in the meantime. If you haven’t updated yet and you make use of Nightly builds, you should do it now!

Lets cross the fingers that no other unexpected issues will arise and force another 1.5 branch release.

While the extension works fine most of the time, we unfortunately fail at times when a version bump for Firefox happens. Exactly this action repeats in a 6 week interval and is the release cycle of Firefox.

So what happens here? The extension registers binary components for the crashme feature in the manifest file and that’s why it is not handled as a ‘compatible by default’ extension by the add-ons manager. So right after a version bump the extension gets marked as incompatible until the compatibility information on addons.mozilla.org has been updated. This results in an unavailability of the extension in Nightly builds for a couple of hours or even a day, depending on when one of us has time to actually do the version bump. That had to be changed, and thankfully Szabolcs Hubai has spent some time on it to fix the problem and to make our extension ‘compatible by default’. From now on you will never see the Nightly Tester Tools disabled after a version bump of Firefox, which is great. I hope you all will enjoy it!

With the 3.3 release we also ship some other fixes which you can read about in the changelog.

You want to help in the development of the extension? Feel free to contact us as usual via MozIRC or our mailing list.

As usual we set a couple of goals we wanted to see by the end of the quarter. All of them were kinda important so we did our best to get all of them done – and we succeeded even if it was in the last minute! So everyone of us was happy about the goal achievements.

Below you will find an overview of each of those projects. For more detailed information I will follow-up with additional blog posts by next week.

Mozmill CI system for release, daily, and l10n builds

It has been taken a while for us to get to a state when we absolutely had to work on a continuous integration solution. Seeing that more and more tests are getting automated with Mozmill and that we still require manual interaction, or crontab jobs and tasks on Windows to get tests executed, we had to find a better way in getting tests run. After a bit of investigation we started by using Jenkins (formerly known as Hudson). And since beginning of this month we now have a system which runs our Mozmill tests for all the supported daily and beta candidate builds, and for each tinderbox build based on l10n check-ins on whichever branch. As trigger we make use of Mozilla Pulse which informs us directly when builds have been made available. Further we also had to ensure that tests can be triggered by on-demand, because update tests cannot be covered by Pulse yet and sometimes we have to re-run tests against a specific build. All in all the system works great but still needs a lot of improvements on which we will work on in the next couple of months.

Automation system for the Add-ons compatibile by default feature

With the change in Firefox 11 to make all add-ons compatible by default, we have been asked to create a system to automatically test add-on compatibility for the top 100 add-ons. As result a new testrun for Mozmill has been created which runs our endurance, update, and endurance tests again. That way we want to make sure that there are no performance regressions involved and updates of Firefox will be applied without any failures.

MemChaser – An extension to visualize memory and GC/CC related activities

Given the amount of manual testing for GC/CC related issues in Firefox by end of 2011, I had the idea to create an extension which makes it easy for users to track memory related activities. That means that all the important information is displayed directly in the add-on bar and don’t have to be retrieved from ‘about:memory‘ or the Error Console (which constantly wipes out old entries). The idea got approved as goal so Dave Hunt, David Guo, and myself were heavily working on getting this extension released on addons.mozilla.org as soon as possible. Meanwhile even more features have been implemented and more are about to come in the next couple of months. If you want to follow the development of the extension check the Github repository.

Mozmill Dashboard – Validation method for uploaded test reports

During a security review of our dashboard by end of 2011 it has been turned out that arbitrary data can be uploaded to our dashboard. The reason was that we haven’t checked the data which gets uploaded before storing it in our database. So we have had to implement a validation method for the CouchDB database. Now the report data gets checked and if the method fails the document will not be stored.

As mentioned before I will follow-up with detailed information later. So please watch out for upcoming blog posts in the next days.

As by late yesterday Mozmill 1.5.11 is now available for everyone on PyPI. This version will be the last one we expect to see for the 1.5 branch. Current work will only happen on Mozmill 2.0, so it can be released by end of this quarter. If you are interested in helping us, please let us know and we definitely will find something for you.

As usual I would like to point out new features and bug fixes which are part of version 1.5.11:

• Bug 686320: JSBridge is not limited to a single port (24242) anymore but probes for a free one now. This allows you to run multiple concurrent testruns with Mozmill at the same time. Keep in mind that the focus manager has to be put into testing mode for this.
• Bug 696834: Securable modules have not been imported as singletons so that sharing data between modules didn’t work.
• Bug 709773: The Mozmill API has been updated internally to let customers override the manifest (-m) and tests (-t) settings provided by the command line options.
• Bug 727660: HTTP proxies are now supported for sending reports to a remote database.
• Bug 740773: With the update of our Mozmill Dashboard invalid reports will be declined. Mozmill hasn’t shown any failure message yet which was kinda unhelpful in tracking down broken reports.

With the changes as mentioned above we can now go ahead and work on the transition of our Firefox tests to Mozmill 2.0.