blog ! {tim, taubert}

Lately, Asa Dotzler posted to dev.apps.firefox regarding the download size of Firefox:

“This evening I noticed that my full win32 mar update for Firefox was 21MB. That caused me to look at what our full win32 installer size was. I was a bit surprised to see it’s up to 17MB. When we shipped Firefox 1, our Windows installer build was 4.7MB. [...]

Firefox 12 is a 16.1 MB download.
Firefox 4 was a 12.0 MB download.
Firefox 3.6 was a 7.7 MB download.

In less than three years we’ve more than doubled in size. (fuller chart here http://grab.by/cSHA)”

While there’s no doubt that adding new features and supporting new platforms are good reasons for increasing the build size it’s definitely a metric that impacts users. It hits them the hardest when downloading Firefox the first time, especially with slow internet connections. It still hits them every time we provide application updates.

To better illustrate the steady growth over the last few years I created http://arewesmallyet.com/. It’s updated daily, shows differences between nightly builds and links to the corresponding changelog if one would like to investigate the cause of increasing build sizes.

While this surely isn’t the most important battle we have right now I hope this will turn out useful to anyone willing to pick this up and tackle some build size optimizations.

Data: http://arewesmallyet.com/files/data.json
GitHub: https://github.com/ttaubert/are-we-small-yet

You may already know the story of how I became a Firefox contributor. Back in early April of 2011, having volunteered full-time for three months (a rather short time compared to other core contributors), I was given the opportunity to start as a paid contributor working for Mozilla.

Over the year I met a lot of great people and had the chance to visit our awesome offices in Mountain View, San Francisco and soon Toronto. I attended JSConf.eu, MozCamp and FOSDEM, with the JSDay yet to come.

But Mozilla isn’t about traveling or attending conferences. It’s about passion for open source, passion for the open web. Working for a non-profit, where decisions are driven by reason and mission, is something that not many software engineers will ever experience in their whole professional career. That’s only one of the reasons I’m really glad to be a part of the global Mozilla community.

Inspired by Lucas Rocha I’ll end this post with some neat statistics about my contributions to the Mozilla project (we Germans love statistics):

I fixed 223 bugs and reviewed patches for 116. I pushed 417 changesets, 110 of them being merges between trees. I changed roughly 1367 files (31862 insertions(+), 19081 deletions(-)).

In my post Leak hunting in browser-chrome mochitests I wrote about the measures we were considering to prevent regressing efforts to get rid of leaks in Firefox. Now that bug 683953 has landed we finally have a way to detect the leakage of whole DocShells and DOMWindows for the lifetime of the browser when running the browser-chrome mochitest suite.

How does it work?

While our browser-chrome mochitest suite runs we parse stdout to track starting and ending tests as well as the creation and removal of DocShells and DOMWindows. Just before the test suite shuts down we schedule a precise GC and wait until it’s completed. Any DOMWindows and DocShells still active are now counted as leaks and assigned to the tests that created them. Additionally we collect the URLs of DOMWindows to help debugging a bit.

How does this prevent new leaks?

We implemented a threshold of (currently) 130 leaks that must not be exceeded. If a test run leaks more than the limit we configured it goes orange and the patch should be backed out from the tree. These are the current numbers:

Linux (64): 116 (116)
OS X (64): 79 (89)
Windows (XP): 120 (118)

Additionally, I filed bug 730797 to integrate these leaks statistics into our Talos infrastructure. So the leak count for each push will be recorded and compared to previous runs to make sure the numbers don’t regress. As the leak numbers differ quite heavily between OSes it makes sense to apply a custom threshold per OS, this will be implemented in bug 730800.

Why is there even a threshold?

First, there are DocShells and DOMWindows that are intentionally kept alive until the browser closes. Second, it’s nearly impossible to bring all these leaks down to “zero” at once. It’s a list of bugs that have to be addressed and we will slowly decrease the threshold to approach “zaroo”.

Thanks to Dão who has been doing great work in bug 658738 discovering all those leaks manually, which in the first place gave me the idea of automating it.

Over the last weeks we worked hard on getting the New Tab Page into Firefox. It’s not quite ready yet but we need your help testing it. We enabled it by default on Nightly and decided to give it a week on Aurora to get feedback from those users as well.

Nightly: http://nightly.mozilla.org/
Aurora: https://www.mozilla.org/en-US/firefox/aurora/

We’ll disable it for Aurora again on February 16th (next Thursday). If you liked the feature and want it back then just set the preference ‘browser.newtab.url’ to ‘about:newtab’, ‘browser.newtabpage.enabled’ to ‘true’ and restart the browser. You can easily file bugs using the following link:

https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&component=General

Please make sure you don’t report a duplicate bug and check the dependencies of the original New Tab Page bug before filing – bug 455553.

December 2009. I’ve been a freelancer for quite some time now and decided to dedicate some weeks to something that always fascinated me: contributing to a big open source project. I started some smaller open source projects in the past (like Video4Linux.Net and ViGedit+) and contributed every so often to Gentoo and the Linux kernel. I’ve always been a great fan of the open source movement and I felt that it’s time to give back some love.

I made a list of all the things that interested me and that I could possibly contribute to. Besides having things like Linux, ReactOS and Wine on the list I picked “Firefox” because it simply has been a loyal companion for years. I think I’ve used it first in version 1.5 and it was one of the most valuable tools that helped me to earn money, get my everyday work done and the most trivial: just browse and experience the web.

Some weeks before that I switched to Firefox 4.0 beta3/4 because that included the first version of Panorama (Tab Groups / TabCandy) that worked on Linux. I loved this feature but noticed that it was in an early stage and needed some fixes. I set up a Firefox build environment, went through Bugzilla to find open bugs, nagged people on IRC and was totally overwhelmed by the warm welcome and the appreciation of my work. This was something I did not at all experience when trying to contribute to other open source projects. Finally, the Panorama team and me managed to get this feature into shape and land it in Firefox 4, yay!

Long story short, I’m now a full-time contributor and love what I’m doing.

If you don’t know IRCCloud, check it out. It’s become a very important tool for my every day work and I really don’t want to miss it. The one thing I never liked about it is that is currently uses a Flash fallback if it detects that the browser doesn’t support the WebSocket API.

The Firefox WebSocket API is currently prefixed (called MozWebSocket) and that’s why even with the newest Firefox you’re forced to use the Flash fallback. They even check for MozWebSocket and explicitly don’t use it if detected. As I didn’t quite understand the reasons behind that I decided to write an add-on that convinces IRCCloud to use native WebSockets in Firefox. Works good so far. I hope that’ll encourage the IRCCloud guys to think about using it again.

Add-on: https://addons.mozilla.org/en-US/firefox/addon/websockets-for-irccloud/
Source: https://github.com/ttaubert/irccloud-websockets

While talking to a friend of mine recently I got to know that he really dislikes that Facebook hides the Logout link in a sub-menu. He told me that he even uses a separate browser only for Facebook because he is very well aware of Facebook’s business model relying on tracking users wherever they are (this is not a big issue for me because I’m very happy with Ghostery).

A quick search revealed that there seem to be lot more users than I expected that would find an auto-logout of Facebook very useful. If not for privacy issues it’s also quite useful if someone else uses your computer and wants to post weird status updates.

So I wrote a Firefox add-on that logs the user out of Facebook when quitting Firefox or after a configurable amount of time has passed since he last closed a Facebook page (and there’s no active tab). It removes all cookies belonging to facebook.com so even tracking should not be an issue anymore (unless Facebook implements alternative tracking techniques).

Add-on: https://addons.mozilla.org/en-US/firefox/addon/facebook-auto-logout/
Source: https://github.com/ttaubert/facebook-auto-logout

Some weeks (even months) ago Dão Gottwald started the hunt for leaked DOMWindows and DocShells while running our browser-chrome mochitest suite (see bug 658738). That means that there are some expensive objects whose lifetimes are longer than they should be – they are kept alive until the test runner shuts down. Sometimes these are caused by only a little typo in the test and sometimes they unveil bigger problems in the core.

Dão has done some great work so far, fixed lots of those leaks and also pointed out patches that introduced new leaks. Inspired by his script that parses the mochitest build log and lists all leaked URIs I wrote a Python script that additionally assigns those URIs to the tests that created these DOMWindows and DocShells. I filed bug 683953 to automatically have those statistics at the end of each mochitest run. Here is an example:

TEST-INFO | leaked 15 DOMWindows and/or DocShells



[browser/components/sessionstore/test/browser/browser_589246.js]

  5x [about:blank]

  4x [chrome://browser/content/browser.xul]

  1x docShells



[browser/devtools/styleinspector/test/browser/styleinspector.js]

  2x [chrome://browser/content/csshtmltree.xhtml]

  1x [data:text/html,basic%20style%20inspector%20tests]

  1x [about:blank]

  1x docShells

This would definitely be very helpful as you don’t have to parse a build log manually after the test run finished. It would also allow us to fail (in a far future where all leaks are fixed) when we detect that the current patch would introduce a new leak.

Another approach would be to have an API that allows to check whether a given object should be regarded as “alive” or “dead”. This is what bug 633670 is about. Every test would need to check if the DOMWindows, DocShells and other objects created by it are still considered alive after it has finished. One problem with this is that we would have to run GC after each test to determine an object’s lifetime – which would negatively affect the overall mochitest suite runtime.

No matter which solution (or maybe a combination of both or something completely different) will make it – we definitely need some kind of better leak detection than we currently have. Many of us are not aware that they are accidentally introducing new leaks with new patches they write. Manually checking for new leaks after each push is a real waste of time and shouldn’t be necessary.

You probably have all heard of this weird new thing called Electrolysis (a.k.a. e10s). Basically it’s all about running the browser UI and its tabs in separated processes. I recently rewrote a part of Panorama to be e10s-future-proof and thought I should share what I’ve learned so far…

(If you don’t know why we’re all doing this, please read: http://blog.mozilla.com/products/2011/07/15/goals-for-multi-process-firefox/)

Update: There is a global message manager. You can send messages through the global or the per-window message manager. Corrected the different loadFrameScript() behaviors. Thanks to Mark Finkle for these corrections!

The message manager
(https://developer.mozilla.org/en/The_message_manager)

We’re using messages to establish communication between the chrome process and the content processes. The message manager sends messages and registers message listeners. It’s also capable of injecting so-called frame scripts (or content scripts) into a content process – these send or receive messages and interact with the DOM loaded into the <browser>. There are four types of message managers:

global messageManager

This message manager sends messages to and receives them from  every <browser> loaded in every window. You can access it by doing:

/* chrome script */

let globalMM = Cc["@mozilla.org/globalmessagemanager;1"].

               getService(Ci.nsIChromeFrameMessageManager);

window.messageManager

This message manager sends messages to and receives them from every <browser> loaded in the given window.

browser.messageManager

This message manager is specific to a single <browser>.

Available methods

messageManager.addMessageListener(messageName, listener)

messageManager.removeMessageListener(messageName, listener)

messageManager.sendAsyncMessage(messageName[, json])

messageManager.loadFrameScript(url[, allowDelayedLoad])

content/frame scripts

The methods of a content script’s message manager are available as global functions. Note that a content script can send synchronous messages, unlike the message managers accessible from chrome scripts. The chrome process is not allowed to block on content processes.

addMessageListener(messageName, listener)

removeMessageListener(messageName, listener)

sendAsyncMessage(messageName[, json])

sendSyncMessage(messageName[, json])

e10s components

Here is an overview of all Electrolysis components. That’s not exactly how e10s is implemented in Gecko but rather a logical view to clarify interactions between these parts.

A simple example

The chrome script

This is the part of the code that runs in the browser process and listens for messages sent by frame scripts. We’re processing a “click” message and respond with an “alert” message.

/* chrome script */

let mm = window.messageManager;



mm.addMessageListener("click", function (msg) {

  let browser = msg.target;

  let data = {text: "You clicked a <" + msg.json.tagName + ">!"};

  browser.messageManager.sendAsyncMessage("alert", data);

});



mm.loadFrameScript("chrome://project/content/content.js", true);

The frame script

The frame script (or content script) runs in the same process as the web page and has access to the contentWindow, document and their events. We listen for any click event and send a “click” message providing the tagName of the clicked element. Additionally we listen for “alert” messages and show an alert dialog when we receive them.

/* content script */

addEventListener("click", function (event) {

  let data = {tagName: event.target.tagName};

  sendAsyncMessage("click", data);

}, false);



addMessageListener("alert", function (msg) {

  alert(msg.json.text);

});

Process communication

This diagram illustrates what the communication between all processes involved looks like (regarding our simple example).

Which parts of your project will be affected by e10s?

DOM Objects

It’s no longer possible to work with or directly access DOM objects (window, document and normal DOM nodes) from the chrome process. As an example, you are not allowed to access a page’s content window through browser.contentWindow. You’ll have to send a message to a content script that does all the work for you. In content scripts, the “global variable” content is the DOM window of the page loaded in the browser.

DOM Events

DOMEvents are no longer propagated to the parent <browser> and you’re not able to call DOMElement.addEventListener(). Use addEventListener() in a content script and then send a message to a listener in the chrome process.

DocShell

The docShell is no longer accessible. It’s available as a “global variable” named docShell in content scripts.

nsIWebProgress(Listener)

If you still need to monitor a page’s web progress all you need is (surprise) a content script. This could look like the following:

/* content script */

let ifaceReq = docShell.QueryInterface(Ci.nsIInterfaceRequestor);

let webProgress = ifaceReq.getInterface(Ci.nsIWebProgress);



let WebProgressListener = { ... };

let mask = Ci.nsIWebProgress.NOTIFY_STATE_ALL;

webProgress.addProgressListener(WebProgressListener, mask);

nsIDOMWindowUtils

The DOM utility interface is also no longer accessible. You still can retrieve it in a content script like this:

/* content script */

let ifaceReq = content.QueryInterface(Ci.nsIInterfaceRequestor);

let utils = ifaceReq.getInterface(Ci.nsIDOMWindowUtils);

Tips and hints

sendSyncMessage or sendAsyncMessage?

Sending synchronous message is not allowed for chrome processes. Only content processes can block on the parent. In general you should always try to use sendAsyncMessage() to not block while waiting for the message to be processed. You should try to rewrite your code if it isn’t ready for asynchronous communication, yet. There are a few valid cases where a message needs to be sent synchronously – if you have one of those you should at least try to handle this message as quickly as possible to not block the content process longer than necessary.

sendSyncMessage()

If you use sendSyncMessage() then you should know that the response is an array of all values returned from each listener.

/* chrome script */

let mm = browser.messageManager;

mm.addEventListener("mymessage", function () "hello world");

mm.addEventListener("mymessage", function () "hello the 2nd");



/* content script */

let results = sendSyncMessage("mymessage", {foo: "bar"});

print(results[0]); // prints "hello world"

print(results[1]); // prints "hello the 2nd"

globalMessageManager.addFrameScript()

Use this if you want a frame script to be attached to every existing tab/browser of every existing window out there. Set the second parameter, allowDelayedLoad, to true, to automatically load the desired frame script in newly created browsers/tabs (of possibly newly created windows) as well.

window.loadFrameScript()

The window-specific message manager has the same frame script loading behavior as the global one, but it will add your frame script to every browser in the given window, only.

browser.loadFrameScript()

If the second parameter, allowDelayedLoad, is false this method will add the frame script only if the browser is ready. If you set it to true and the browser isn’t ready, yet, the script will be added when it becomes ready.

messageManager.addMessageListener()

When a message from a content script is received the listeners attached via browser.addMessageListener() are called first, and then the ones added via window.addMessageListener(), then the ones via globalMessageManager.addEventListener().

Message properties

The first argument passed to message listeners is the message they just received. This is an object with the following properties:

name   - the name of the message

json   - the custom message data

sync   - false if the message was sent asynchronously

         (always false for messages from chrome scripts)

target - the browser associated with the content that this

         message came from

Message name prefixes

At the beginning there won’t be many message users in the Mozilla code base and we should not start consolidating messages before the requirements of all those are fully fleshed out. So it’s better to be over-specific for now and name your messages like Project:click instead of just click to avoid conflicts.

Conventions for frame scripts

Frame scripts (or content scripts) should be stored in the same folder as the code that calls loadFrameScript() to load them. A good convention is to name them something like content-project.js or just content.js.

Next steps

Part two will address a more complex example. I’ll try to follow-up as soon as possible.

My name is Tim Taubert (ttaubert on IRC and Twitter). I work as a Firefox Engineer for Mozilla, based in Berlin, Germany.

Here are the rules for this particular meme:

Link to your original tagger(s) and list these rules in your post. Share seven facts about yourself in the post. Tag seven people at the end of your post by leaving their names and the links to their blogs. Let them know they’ve been tagged.

I wasn’t tagged by anyone but after reading Jared Wein’s blog post I felt inspired to do the same and keep the idea alive.

These are my seven things:

1) At the age of sixteen I started to play multiple instruments, including guitar, piano and drums. I may not be the best musician out there and have quite a hard time reading music but it’s a lot of fun anyway. My current band is called “Rabbit Wrong” (which is easier to understand with a German background) and I play a white and adorable Gibson Studio connected to an Engl Screamer.

2) I have always been a big fan of guinea pigs. Three of them live at my home and they are called Charlie, Whiskey and Sydney (left to right).

3) Back in 2008 I managed to contribute ~12 lines of code to the Linux kernel 2.6.25 to add support for my newly bought USB TV adapter.

Patch: http://linuxtv.org/pipermail/linux-dvb/2008-January/022742.html
Changelog: http://lwn.net/Articles/266704/

4) All electronic devices I own (including TV, mobile phones, a tablet and laptops) are set to English language (hint, I’m German). I always loved everything about English – I watch movies and sitcoms almost exclusively in English and guess which language all my books are written in.

5) My first car about 4-5 yours ago was a sixteen years old Renault Rapid. It was assembled out of three wrecks of the same type. I used to call it the “death mobile” because it was really old and rusty, also the brakes didn’t work very well.

6) I’ve been a vegetarian for almost two years now and I feel quite good about it. I’m not one of those who tries to talk you into that but I’m much more comfortable about my own way of life since then. And that’s what counts in the first place.

7) One of my first employers was an agency for digital communication in Berlin. They were apparently very pleased with my work and decided to introduce the “Tim Taubert award” that is bestowed on every Christmas staff party. There have been three award winners so far, me being the first one :)

These are the people I’d like to tag:

(Just because they’re all truly awesome and there’s only place for seven of them.)

Frank Yan – http://frankyan.com/
Ian Gilman – http://iangilman.com/
Rob Campbell – http://antennasoft.net/robcee/
Dietrich Ayala – http://autonome.wordpress.com/
Marco Bonardo – http://blog.bonardo.net/
Hernán Colmeiro – http://peregrinogris.com.ar/
Michael Yoshitaka Erlewine – http://mitcho.com/