During the whole last year I had a lot of situations when one of my virtual machines on the server died due to an OOM killer process. Those crashes were not predictable and happened randomly. Sometimes it didn’t happen for weeks but there were also situations when it crashed after 1 day again. Given that a good list of customers are hosting their websites on it, raises a lot of trouble for me. I did a lot of work in trying to fix particular running services on that host, but nothing helped to stop those crashes. Recently I have even doubled the memory for that machine but without success. It always ran into an out of memory crash.

Given all my former research and attempts to fix the problem, I wasn’t sure what else I could do. But thankfully I have found a website which has the explanation and even offered steps to solve the problem.

So what’s happened? The reason can be explained shortly: The Linux kernel likes to always allocate memory if applications asking for it. Per default it doesn’t really check if there is enough memory available. Given that behavior applications can allocate more memory as really is available. At some point it can definitely cause an out of memory situation. As result the OOM killer will be invoked and will kill that process:

Jun 11 11:35:21 vsrv03 kernel: [378878.356858] php-cgi invoked oom-killer: gfp_mask=0x1280d2, order=0, oomkilladj=0
Jun 11 11:36:11 vsrv03 kernel: [378878.356880] Pid: 8490, comm: php-cgi Not tainted 2.6.26-2-xen-amd64 #1

The downside of this action is that all other running processes are also affected. As result the complete VM didn’t work and needed a restart.

To fix this problem the behavior of the kernel has to be changed, so it will no longer overcommit the memory for application requests. Finally I have included those mentioned values into the /etc/sysctl.conf file, so they get automatically applied on start-up:

vm.overcommit_memory = 2
vm.overcommit_ratio = 80

The results look good so far and I hope it will stay that way. The lesson I have learned is to not trust any default setting of the Linux kernel. It really can result in a crappy and unstable behavior.

A list of fixes, which have been made it into this release, can be found on Bugzilla. Lets give an explanation for some of those:

• Bug 542000: With former Mozmill releases there were needs to install the Python for Windows extensions (pywin32). If you have used the MozillaBuild environment, changes to the registry were needed. All that work hasn’t to be done from now on. Installing MozillaBuild is enough to prepare the system for Mozmill. A big thanks goes to Atul Varma who removed that dependency and made our life more compelling.
• Bug 543501: One of our upcoming projects is the execution of Mozmill tests against add-ons. Therefore we have to make sure that the usage of Mozmill will be as easy as possible. That’s why the misleading –plugin command line option has been changed to –addons. Make sure to use this option from now on.
• Bug 548446: Mozmill had problems with installing some of the existing extensions because it was not able to find the extension id. Thanks goes to Jonathan!
• Bug 544896: If you have long test-runs it was possible that the software update dialog came up and removed the focus from the window currently under test. That’s why the automatic update for the application has been disabled. If you want to run tests against that feature make sure to enable the preference app.update.enabled before.
• Bug 558404 and Bug 559152: Some controller functions have been updated which missed some details about failures. Now we always have a stacktrace with full information available.

The new version of the add-on can be found on addons.mozilla.org. It’s under review right now but can already be installed. For all the others who are using the command line client of Mozmill, you can simply run the “easy_install -U mozmill” command to update to the new 1.4.1 release.

If you have questions don’t hesitate to send your feedback to the mozmill-dev mailing list or directly contact us on IRC in the #qa channel.

About 2 weeks ago I got my new Nokia N900 device for testing purposes. During the next couple of months I will have to run some dogfooding against the mobile Firefox aka. Fennec. To have it handy all the time I will have to use it as my mobile phone. Therefore it’s necessary to synchronize all the relevant information, e.g. Contacts, Calendars, and Tasks, between OS X and the N900.

There is no official plug-in yet which will allow the synchronization with iSync. But after a quick check I have found a plug-in which is still in a beta state but works quiet well for all of my data, and that in both directions. You can download the software from this website. The installation is fast and you will be able to start the first sync process nearly immediately.

Download and install Fritz!Fax on your Windows machine. A printer driver is installed automatically which is used to send b/w or colored faxes. Works like a charm.

Thanks AVM even it is an unsupported software.

Yesterday I was finally able to play around with this plugin and promptly run in a couple of problems due to wrong application dependencies. Most of my applications have been installed via MacPorts while for Mozmill and hg-git setuptools has been used. As result Mercurial wasn’t able to find the hg-git plugin. It has been taken nearly half an hour before I figured out whats going on. So for any future reference and as a tutorial for others I will give instructions below how to get this plugin working on your Mac.

To install other software on your Mac it’s mostly helpful to do this via MacPorts. The installation package and needed instructions can be found on their website. Just download the appropriate build for your version of OS X and install it.

Now use MacPorts to install Python and setuptools. The command below can take a while to execute depending on the power of your machine:
$ sudo port install python25 py-setuptools

With setuptools installed you can use easy_install to install all the remaining packages:
$ sudo easy_install mercurial hg-git

To activate the hg-git plugin the following lines have to be added to the .hgrc file in your home directory:
[extensions]
hgext.bookmarks =
hggit =

And that’s all! Now you can use hg-git to synchronize your local copy of the repository with the remote one via hg or git. As example I will use our Mozmill test repository which has two branches (default and mozilla1.9.1).

Create a local copy of the remote repository and change into its folder.
$ hg clone http://hg.mozilla.org/qa/mozmill-tests
destination directory: mozmill-tests
requesting all changes
adding changesets
adding manifests
adding file changes
added 251 changesets with 840 changes to 132 files (+1 heads)
updating to branch default
98 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd mozmill-tests/


Because we have two branches we will have to create two bookmarks. Both are used to map those branches to separate branches in the git repository. Then push all the data to the github repository:
mozmill-tests$ hg bookmark -r default master
mozmill-tests$ hg bookmark -r mozilla1.9.1 1.9.1
mozmill-tests$ hg push git+ssh://git@github.com/whimboo/mozmill-tests.git
pushing to git+ssh://git@github.com/whimboo/mozmill-tests.git
importing Hg objects into Git
creating and sending data
git::refs/heads/mozilla-1.9.1 => GIT:70300757
git::refs/heads/master => GIT:092e7cf3


If you want to pull updates from the Github repository to include those into your local version just run a normal pull command in hg with the appropriate URL:
mozmill-tests$ hg pull git+ssh://git@github.com/whimboo/mozmill-tests.git
pulling from git+ssh://git@github.com/whimboo/mozmill-tests.git
importing Hg objects into Git
Total 0 (delta 0), reused 0 (delta 0)
importing Git objects into Hg
(run 'hg update' to get a working copy)


If you do not want to add the path of the remote github repository each time you can simply add it to the .hg/hgrc file:
cat .hg/hgrc
mozmill-tests$ cat .hg/hgrc
[paths]
default = http://hg.mozilla.org/qa/mozmill-tests
git = git+ssh://git@github.com/whimboo/mozmill-tests.git


And run this command afterward:
mozmill-tests$ hg pull -u git

As you can see it’s really simple to keep your Mercurial and Github repository in sync. Personally I will move to Github now which makes handling patches on different branches much easier.

Everyone who is using Mozmill regularly is welcome to help us in testing the beta version. As long as no big issues will come up the release of Mozmill 1.3 will happen next week.

If you want to test the extension please download it from Github.

Users of the pyPI packages only have to run “easy_install -U mozmill” to get the latest packages for Mozmill, JSBridge, and Mozrunner.

Thanks in advance.

After a couple of hours using it I will definitely not miss it anymore!

Given my project to get manual Litmus tests automated with Mozmill I have prepared some slides with a special focus on l10n. But sadly I wasn’t able to join the conference because of sickness. I have to say a big thanks to my colleague Marcia Knous and also to one of our main contributors for Sunbird tests Merike Sell who both hold the session. As informed at the end of the session via IRC the talk was a great success and a lot of questions were ask.

Due to the amount of sessions not everyone was able to join the Mozmill session. Also given all the people who weren’t be able to come I have uploaded my slides for all of you now. Please check the embedded Slideshare content below:

Feedback
Because I haven’t got any feedback from localizers so far I’m anxious to hear what you think about the usefulness of Mozmill and testing with localized builds. Given by the current number we have over 70 official locales available which are not tested by automated tests and require manual testing from localizers and contributors on a regular basis. With all the 250 BFT and another 750 FFT tests enabled in Litmus manual testing is a time taking action. Running all the tests with Mozmill will take much lesser time, could be run more often, and could cover all platforms which will result in a higher quality of Firefox and helps us to minimize any new regressions for our huge user base.

Please check the following questions I’m interested in getting an answer:

1. How often does your l1on team run Litmus tests against your locale whether those are BFT/FFT or the localizer test-run?
2. Would you like to see much of those tests automated and are you interested in running those tests on your local machine for each major and stability release?
3. Are you interested to help QA in writing Mozmill tests so we have most of them available as soon as possible?
4. Do you have further ideas how Mozmill can be used in the l10n area additionally to the points I have pointed out in my slides?

Thanks in advance for your feedback!

If you want to use Mozmill to run tests against Shiretoko / Firefox 3.5.x or Namoroka then you can switch between the different branches with the commands seen below.

$ hg clone http://hg.mozilla.org/qa/mozmill-tests
$ cd mozmill-tests
$ hg branches
default 108:d8529dd87a77
mozilla1.9.1 106:221ecb1a8ea9
$ hg branch
default
$ hg up -C mozilla1.9.1
$ hg branch
mozilla1.9.1
$ hg up -C default
$ hg branch
default

Given the output default is our current trunk version of the Mozmill tests while the mozilla1.9.1 branch is for Firefox 3.5.x.

henrik@mbp:/mozilla/mozmill-tests[privatebrowsing.patch]$

The only thing you have to do is to enter the following entry into your .bashrc or .bash_profile:
PS1='\u@\h:\w[\[`hg qtop 2>/dev/null`\]]\$ '

That’s it!

The first thing I tried was to kill the Vmware application via the Activity Monitor of OS X. But I wasn’t successful. After I’ve started VmWare again the same screen as above was displayed for the XP machine. What a bummer! So how to process? After searching a while on Google I found the solution. You simply have to kill the process which handles this machine. Grep the output of ps to get a list of all Vmware related processes:

henrik$ ps -ef | grep vmware
0 31737 1 0 0:00.00 ?? 0:00.01 /Library/Application Support/VMware Fusion/vmnet-dhcpd -cf /Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf -lf /var/db/vmware/vmnet-dhcpd-vmnet8.leases -pf /var/run/vmnet-dhcpd-vmnet8.pid vmnet8
0 31745 1 0 0:00.00 ?? 0:00.00 /Library/Application Support/VMware Fusion/vmnet-dhcpd -cf /Library/Application Support/VMware Fusion/vmnet1/dhcpd.conf -lf /var/db/vmware/vmnet-dhcpd-vmnet1.leases -pf /var/run/vmnet-dhcpd-vmnet1.pid vmnet1
0 31776 1 0 0:08.81 ?? 0:11.93 /Library/Application Support/VMware Fusion/vmware-vmx -E de -D h5VWrEgtEdEDLwAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= -# product=1;name=VMware Fusion;version=2.0.4;buildnumber=159196;licensename=VMware Fusion for Mac OS;licenseversion=6.0 build-159196; -@ pipe=/var/folders/w2/w2myuXRLE1q1Ex7puQAPYk+++TI/-Tmp-//vmware-henrik/vmxebfd750b90971fa0;readyEvent=6 /Volumes/data/vmware/Windows XP Professional.vmwarevm/Windows XP Professional.vmx
501 31781 153 0 0:00.72 ?? 0:04.42 /Applications/VMware Fusion.app/Contents/MacOS/vmware -psn_0_14101874
501 31795 13404 0 0:00.00 ttys000 0:00.00 grep vmware

As you can see the process with the id 31776 is the one for the Windows XP guest. Run the following command to do a hard switch-off:

henrik$ sudo kill -9 31776

Now you can start the machine from scratch and it should be started as expected.

There are only two steps necessary to see the plugin appear in each Firefox installation. It’s not magic but a simple registry hack, which let the plugin be shared. I’ve no idea why Microsoft hasn’t already done that for this plugin. They are still aware in using this way, e.g. for “Windows Presentation Foundation plug-in“. I’ll get in contact with Microsoft and hopefully it can be solved in the near future.

But lets return to the instruction how to get it working. Just follow those steps:

1. The Window Media Player will be installed to “c:\Program Files\Windows Media Player”. So lets re-use this location. Create a sub-folder called “Plugin” and copy the library “np-mswmp.dll” over there.
2. Go to bug 421820 and download the registry hack.
3. If the installation folder of the Windows Media Player application differs from the one in step 1, you have to update the path inside the registry file before applying the file.
4. Double click the reg file to add the entries to your registry.

That’s all. Now each installed version of Firefox should be able to use the plugin to show embedded WMV content. At least you have to restart the browser, when it is already open. To make sure it has been recognized correctly open the Add-ons manager and click the plugins tab. The plugin has to be listed now.

Digging into searches gives me following interesting discussion on Apple.com. It reports about some additionally filters for free. You can download and unzip them into your own or the systems library folder. That’s all! From now on you have better control about needed file sizes and quality while saving different formats as PDF.

Enjoy!

I found a helpful blog post which helped me a lot to solve the issue. As you can see the timeout problem was introduced with the upgrade to OS X 10.5.2. As the solution you simply have to update the firmware of the router. That’s all! Thanks for the nice hint.