Over the last weekend Mozilla Camp Europe 2009 has been taken place in Prague. About 150 people from l10n, qa, dev, and advocacy were invited to join this conference which Mozilla Europe is organizing each year.
Given my project to get manual Litmus tests automated with Mozmill I have prepared some slides with a special focus on l10n. But sadly I wasn’t able to join the conference because of sickness. I have to say a big thanks to my colleague Marcia Knous and also to one of our main contributors for Sunbird tests Merike Sell who both hold the session. As informed at the end of the session via IRC the talk was a great success and a lot of questions were ask.
Due to the amount of sessions not everyone was able to join the Mozmill session. Also given all the people who weren’t be able to come I have uploaded my slides for all of you now. Please check the embedded Slideshare content below:
Feedback
Because I haven’t got any feedback from localizers so far I’m anxious to hear what you think about the usefulness of Mozmill and testing with localized builds. Given by the current number we have over 70 official locales available which are not tested by automated tests and require manual testing from localizers and contributors on a regular basis. With all the 250 BFT and another 750 FFT tests enabled in Litmus manual testing is a time taking action. Running all the tests with Mozmill will take much lesser time, could be run more often, and could cover all platforms which will result in a higher quality of Firefox and helps us to minimize any new regressions for our huge user base.
Please check the following questions I’m interested in getting an answer:
- How often does your l1on team run Litmus tests against your locale whether those are BFT/FFT or the localizer test-run?
- Would you like to see much of those tests automated and are you interested in running those tests on your local machine for each major and stability release?
- Are you interested to help QA in writing Mozmill tests so we have most of them available as soon as possible?
- Do you have further ideas how Mozmill can be used in the l10n area additionally to the points I have pointed out in my slides?
Thanks in advance for your feedback!
About duplicate acces keys:
A bugzilla query shows that many locales detected duplicate access keys in a short time period, but at a very different time for each locale. (See bugs 425xxx and bugs 454xxx). Is this from manually running compare-locales?
Wladimir Palant (trev), Adblock Plus and Songbird developer, coded and wrote about access keys in add-on translations and Firefox built-in ones, maybe this script can be altered (maybe even by him? Songbird would also benefit).
http://adblockplus.org/blog/managing-locales
Too bad you were not able to attend, we missed you! I hope you’re felling better now.
–Tristan
Having been in both the litmus2 and the mozmill talk, I think the two are perpendicular to each other.
Litmus is really for human testing, and in particular, those tests should target translation quality. As in, the stuff that goes beyond typos. Is the dialog understandable etc.
Mozmill on the other hand is really good at catching technical errors. XML parsing errors for one, so making the mozmill tests do coverage tests would be good. accesskey conflicts would be good, too. We should see if we can port the tests I have in my test extension over to mozmill.
I think we could even go as far as to check for dialog sizing errors, by checking if all elements in a dialog have their boundaries inside the window. Or inside the containing element? Not sure what’s the right test there.
For the popularity of litmus, it’s “darn low” at this point, I guess. http://mxr.mozilla.org/mozilla/source/webtools/litmus/json.cgi#158 has “docs” on how to get assembled reports out of litmus, too.