I got to spend Wednesday through Friday in Edinburgh last week to attend Scotland.JS. Edinburgh is a lovely city and I will definitely return to get to know it better. It has great people, beers, food and even a castle - what could one want more?

I arrived on Wednesday, just in time for the TechMeetup. It took place in the Appleton Tower and had about 70-80 attendees with quite diverse technical backgrounds. I had a very interesting talk to a freelance patent attorney and also met a couple of people I knew from recent conferences and meetups in Berlin. After we were out of free pizza and beers we met most of the Scotland.JS attendees in a pub right next to the Appleton Tower. Cue more beers and tech conversations.

Thursday was the first day of Scotland.JS that started with a great keynote from Jan Lehnardt. The most memorable talk of the day was about Functional Reactive Programming using Bacon.js by Philip Roberts. There of course were a lot more great talks but the possibilities of creating and combining event streams kept my head spinning the most. At the end of the day we again went to a neat pub near the venue and I headed to bed rather early, still a little tired from travel and the day before.

Friday started with a short 20-minute walk through Edinburgh’s old town that I needed to pass on my way to the venue. My favorite talk of the second and last day was held by Dominic Tarr that tought us about the internals of leveldb. I also really enjoyed Mark Boas talking about making audio a first-class citizen of the web - a very entertaining and impressive mix of technologies. A GitHub drinkup in the venue’s own pub was a great way to end the conference - lots of good conversations with Scotland.JS attendees and speakers as well as people from the local tech community and visitors of the Scottish Ruby Conf.

I flew home the very next day and already miss Edinburgh with its neat little alleys and pubs everywhere. The JavaScript community has once again shown its value as a very interesting mix of nice welcoming people coming from so many different technical backgrounds. Like after most conferences I feel a little exhausted and my head is full of ideas for future or current (side-)projects. I will try my best to return next year, it has been a lot of fun!

Update: Please note that this post describes the current implementation in SpiderMonkey. The final ES6 specification will require generator functions to have a ‘*’ token as in function* nat() { … }. The flatten() function from the last example could use yield* to delegate instead of a nested loop.

JavaScript comes with most of the little functional tools you need to work on finite sequences that are usually implemented using Arrays. Array.prototype includes a number of methods like map() and filter() that apply a given function to all items of the Array and return the resulting new Array.

[1, 2, 3].map(x => x + 1); // result: [2, 3, 4];

These tools however are not a good fit for infinite sequences as they always consume the whole sequence at once to return a new one. Implementing infinite sequences by yourself means you would have to come up with your own API that clients need to adhere to. You often would keep state variables whose values need to be maintained for the duration of the computation process.

Generators to the rescue

Using ES6 generators implementing the infinite sequence of all natural numbers turns out to be a trivial task. We even have language support to iterate over them.

function nat() {
  var i = 1;
  while (true) {
    yield i++;
  }
}

var it = nat();
console.log(it.next(), it.next(), it.next()); // prints 1 2 3

Now that we have a first infinite set we need a couple of functions that help us working with, combining, and building new sequences.

Mapping

Let us start with map() - a function at the very heart of functional programming. It builds a new sequence by applying a function to all elements of a given sequence.

function map(it, f) {
  for (var x of it) {
    yield f(x);
  }
}

Using the generator implementation of map() we can now easily write a function called squares() that represents the set of squares of all natural numbers (1², 2², 3², …, n²).

function squares() {
  return map(nat(), x => x * x);
}

var it = squares();
console.log(it.next(), it.next(), it.next()); // prints 1 4 9

Using for…in instead of for…of works fine but using of has a neat advantage in that it properly iterates Arrays as well. If you want to map values of a finite sequence you can just do that.

var it = map([1, 2, 3], x => x * x);
console.log(it.next(), it.next(), it.next()); // prints 1 4 9

Filtering

Another common task is filtering specific values from a sequence. Our custom implementation of filter() takes an iterator and a predicate - the returned sequence will consist of all items of the original one for which the predicate holds.

function filter(it, f) {
  for (var x of it) {
    if (f(x)) {
      yield x;
    }
  }
}

We can now use filter() to create the set of all even natural numbers.

function even() {
  return filter(nat(), x => x % 2 === 0);
}

var it = even();
console.log(it.next(), it.next(), it.next()); // prints 2 4 6

A common derivation from filter() is filterNot() that simply negates the given predicate. We can use that to implement even() as well.

function filterNot(it, f) {
  return filter(it, x => !f(x));
}

function even() {
  return filterNot(nat(), x => x % 2);
}

Mersenne primes

Suppose we were to implement a sequence that represents all Mersenne prime numbers. Mersenne primes are defined as prime numbers of the form M_n = 2ⁿ - 1, that is the set of all numbers of the given form that have no positive divisors other than 1 and themselves. The set of Mersenne primes is assumed to be infinite though this remains unproven, yet.

Let us first define some helper functions. range(from, to) and forall() are common helpers in functional programming languages. range() returns the set of natural numbers in a given range. forall() returns whether the given predicate holds for all items in the sequence and should therefore only be used for finite sequences.

function range(lo, hi) {
  while (lo <= hi) {
    yield lo++;
  }
}

function forall(it, f) {
  for (var x of it) {
    if (!f(x)) {
      return false;
    }
  }

  return true;
}

mersenneNumbers() is the set of all numbers of the form M_n = 2ⁿ - 1. isPrime() is a very simple and naive (and slow) primality checker that returns whether the given candidate is divisible by any of the numbers in the range of [2, candidate - 1]. We will use isPrime() as a filter to remove all non-prime numbers from mersenneNumbers().

function mersenneNumbers() {
  return map(nat(), x => Math.pow(2, x + 1) - 1);
}

function mersennePrimes() {
  function isPrime(n) {
    return forall(range(2, n - 1), x => n % x);
  }

  return filter(mersenneNumbers(), isPrime);
}

var it = mersennePrimes();
console.log(it.next(), it.next(), it.next()); // prints 3 7 31

Flattening

As a last example we will implement a function that flattens nested sequences.

function flatten(it) {
  for (var x of it) {
    if (Object.prototype.toString.call(x) === "[object Generator]" ||
        Array.isArray(x)) {
      for (var y of flatten(x)) {
        yield y;
      }
    } else {
      yield x;
    }
  }
}

Note that using for…of comes in handy again as we can use it to iterate over Arrays and generators. Using flatten() we can now do:

flatten([1, [2, 3], [[4], [5]]]); // result: [1, 2, 3, 4, 5]

Combining flatten() and map() to flatMap() we can implement another very common function that flattens the result of applying a given function to all items of a sequence. Let us use it to re-build the set of all natural numbers from the set of all even natural numbers.

function flatMap(it, f) {
  return flatten(map(it, f));
}

var it = flatMap(even(), x => [x - 1, x]);
console.log(it.next(), it.next(), it.next()); // prints 1 2 3

Generators are powerful

It is quite obvious that studying ES6 generators really repays. They are a very powerful language construct and I personally cannot wait until they are available in V8/Node.js as well. They will be in the toolbox of every professional JavaScript developer soon and I am sure we can count on the community to come up with lots of great uses and libraries.

You have probably already heard of generators and iterators coming to a browser near you. They have been available in Firefox for a long time and are used extensively all over the Mozilla code base. The V8 team will implement iterators and generators once ES6 has been finalized.

This post describes the current implementation in SpiderMonkey and tries to include the current state of the ES6 draft and discussions.

A simple generator

Let us take a look at a simple example of a generator function that represents an infinite sequence containing all the numbers from 0 to Number.MAX_VALUE. Once it reaches MAX_VALUE it will not increase any further but always return the same number.

function myInfiniteGenerator() {
  var i = 0;
  while (true) {
    yield i++;
  }
}

var iter = myInfiniteGenerator();

while (true) {
  console.log(iter.next());
}

Any object of the following shape is an iterator:

{ next: function() -> any }

The next() method simply returns the item next in the sequence.

Finite sequences

As you surely noticed the generator of the first example produces iterators that will never run out of items. The next example shows an iterator representing a finite sequence:

function MyFiniteIterator(max) {
  this.cur = 0;
  this.max = max;
}

MyFiniteIterator.prototype.next = function () {
  if (this.cur < this.max) {
    return this.cur++;
  }

  throw StopIteration;
};

The given code implements a custom iterator without writing a generator function. Note that it throws StopIteration as soon as it reaches the maximum value to signal that the sequence is exhausted. It is a lot more elegant to implement the same sequence using a generator function:

function myFiniteGenerator(max) {
  var i = 0;
  while (i < max) {
    yield i++;
  }
}

Generator functions will automatically throw StopIteration when terminating. So how should one consume iterators with finite sequences?

Consuming sequences

In Java for example, you would check iter.hasNext() and stop when it returns false. In JavaScript however you need to use a try…catch statement to catch StopIteration when it is being thrown.

var iter = myFiniteGenerator(10);

while (true) {
  try {
    console.log(iter.next());
  } catch (e if e === StopIteration) {
    break;
  }
}

You might wonder if there is a better way to do this and indeed there is. Using for…in or for…of you do not have to catch StopIteration yourself, the JavaScript engine will do it for you. As soon as the sequence is exhausted the loop will terminate normally without the exception being propagated:

var iter = myFiniteGenerator(10);

for (var i in iter) {
  console.log(i);
}

StopIteration is special

StopIteration actually is a standard variable that is bound to an object of class StopIteration. It is an ordinary object with no properties of its own and it is not a constructor function.

try {
  throw StopIteration;
} catch (e if e instanceof StopIteration) {
  // This works because:
  StopIteration instanceof StopIteration === true;
}

As StopIteration is a singleton of type StopIteration you can also catch it by checking for equality:

try {
  throw StopIteration;
} catch (e if e === StopIteration) {
  // ... handle exception
}

StopIteration is mutable

You should be aware that StopIteration is a mutable global. Just like undefined it can be modified to hold any other value. If you write a library and want to shield against modifications from outside you can use this neat little trick I found on Dave Herman’s blog:

(function(){try{(function(){true||(yield)})().next()}catch(e){return e}})()

The inner function is a generator that terminates immediately and therefore will throw a StopIteration. The outer function simply catches and returns it.

StopIteration may become a constructor

The current iterator strawman states that StopIteration will become a constructor to maintain compatibility with generator functions returning values.

Iter.prototype.next = function () {
  if (this.cur < this.max) {
    return this.cur++;
  }

  var stop = new StopIteration();
  stop.value = "sequence exhausted";
  throw stop;
};

The equality check from above would not work anymore so it might be better to just use instanceof.

StopIteration may not be part of ES6

The Python way of throwing to denote the end of a sequence is backwards compatible with old ECMAScript versions but there seem to be people not happy with the current proposal. While I can’t tell whether StopIteration is really to be removed a couple of alternative suggestions have been made:

Introduce a keyword to end a frame

To not misuse exceptions for normal control flow ES6 could introduce a stopiteration or endframe keyword that would end the current frame with an optional return value. The obvious downside is that it is probably not backwards compatible.

Iter.prototype.next = function () {
  if (this.cur < this.max) {
    return this.cur++;
  }

  stopiteration [reason];
  // or endframe [reason];
};

Add an iterator.hasNext() method

Just like Java the iterator API could consist of the two methods next() and hasNext(). The client would then need to check hasNext() every time before calling next().

Iter.prototype = {
  hasNext: function () {
    return this.cur < this.max;
  },

  next: function () {
    if (this.hasNext()) {
      return this.cur++;
    }

    throw new Error("sequence exhausted");
  }
};

Let next() always return an object

Custom iterators would be required to implement a single method but would not need to throw. Instead they would return an object with the property done set to true to indicate that the sequence has ended. The value property would be used to store values passed to yield or return in a generator function.

{
  next() -> { done: false , value: any }
          | { done: true[, value: any] }
}

Food for thought

This is in no way a complete list of possibilities or proposals that were brought up on es-discuss so please make up your own mind about the current iterators implementation and the suggested improvements.

The future is bright

Whether or not StopIteration will end up in ES6, generators and iterators are great and you should make sure to be prepared when they become available in modern browsers as well as on Node.js.

I concentrated particularly on StopIteration but there are lots of great posts out there that go way more into depth about generators and their usage. Make sure to also take a look at libraries like Task.js that combines generators with promises to cooperative tasks.

Now that you should already know how to build a live green screen and an EyeToy-like mini-game using nothing but plain JavaScript and a modern browser supporting WebRTC, let us move on to another interesting example: simple motion detection in a live video.

The initialization code

To detect motion in a video we need to compare at least two frames. We will use typed arrays to store the lightness data of the previous frames:

function initialize() {
  // ... code to initialize the canvas and video elements ...

  // Prepare buffers to store lightness data.
  for (var i = 0; i < 2; i++) {
    buffers.push(new Uint8Array(width * height));
  }

  // Get the webcam's stream.
  nav.getUserMedia({video: true}, startStream, function () {});
}

We want two frame buffers - a single one results in a heavily flickering motion video but the more frames we store the more motion blur we will see. Two seems like a good value for demonstration purposes.

Illustrating lightness changes

The main draw() function from part 1 did not change except that we now call markLightnessChanges() for every frame. This is also the probably most interesting function of the whole demo:

function markLightnessChanges(data) {
  // Pick the next buffer (round-robin).
  var buffer = buffers[bufidx++ % buffers.length];

  for (var i = 0, j = 0; i < buffer.length; i++, j += 4) {
    // Determine lightness value.
    var current = lightnessValue(data[j], data[j + 1], data[j + 2]);

    // Set color to black.
    data[j] = data[j + 1] = data[j + 2] = 0;

    // Full opacity for changes.
    data[j + 3] = 255 * lightnessHasChanged(i, current);

    // Store current lightness value.
    buffer[i] = current;
  }
}

We determine the lightness value of every pixel in the canvas and compare it to its values in the previously captured frames. If the difference to one of those buffers exceeds a specific threshold the pixel will be black, if not it becomes transparent.

function lightnessHasChanged(index, value) {
  return buffers.some(function (buffer) {
    return Math.abs(value - buffer[index]) >= 15;
  });
}

Blend mode difference

The simple method we use to detect motion is called a blend mode difference. That is a quite fancy word to say: we compare two images (also called layers or frames) by putting them on top of each other and subtracting the bottom from the top layer. In this example we do it for every pixel’s L-value of the HSL color model.

function lightnessValue(r, g, b) {
  return (Math.min(r, g, b) + Math.max(r, g, b)) / 255 * 50;
}

If the current frame is identical to the previous one, the lightness difference will be exactly zero for all pixels. If the frames differ because something in that picture has moved then there is a good chance that lightness values change where motion occured. A small threshold ensures that we ignore noise in the signal.

Demo and screencast

That is all! Take a look at the live demo or watch the screencast below:

You can create some really great demos with this simple technique. Here is a neat one of a xylophone you can play by waving your hands (which unfortunately does not work in Firefox).

Whatever your ideas may be, I encourage you to fiddle around with the small demos I provided in my three getUserMedia() examples so far and let me know if you built something amazing!

Back in October 2012 I wrote two blog posts, getUserMedia part 1 and part 2, including demos which unfortunately would run in Firefox, only. I did not explicitly want to be exclusive but I think I just did not feel like looking up why my code did not work in Opera and why exactly webkitGetUserMedia() behaved differently than mozGetUserMedia(). I was being lazy.

I also intended to mix in a couple of nice JavaScript features, like block-scoped variable definitions with let, destructuring assignments or Sets (did I just do it again?). In hindsight this does not really make sense as I should not expect visitors to want to learn about cutting-edge JavaScript features when viewing a getUserMedia() post.

Before finishing my third piece on getUserMedia() I decided to update the demos of my older posts to run in any modern browser. I also seized the chance to overhaul code examples which did not adhere to my coding standards anymore.

If you should ever be in a similar situation - please take a couple of minutes to write code that runs in all modern browsers so people can enjoy your demos in their browser of choice. Please don’t be lazy.

This post is a follow-up to my previous one about building a live green screen with getUserMedia() and MediaStreams. If you have not read it yet, this might be a good time. We will extend the small example to build an EyeToy-like mini-game.

Some additions

var video, width, height, context;
var revealed = Object.create(null);

function initialize() {

First, we will add a variable called revealed that keeps track of all pixels that have already been revealed by holding a green object in front of the camera. Instead of replaceGreen() we will call our method revealGreen() from now on:

function revealGreen(data) {
  var len = width * height;

  for (var i = 0, j = 0; i < len; i++, j += 4) {
    // This pixel has already been revealed.
    if (i in revealed) {
      data[j + 3] = 0;
      continue;
    }

When iterating over all of the canvas’ pixels we check whether the current index is marked as revealed. If so we do not need to check its color but set its opacity to zero and continue with the next iteration.

    // Convert from RGB to HSL...
    var hsl = rgb2hsl(data[j], data[j + 1], data[j + 2]);
    var h = hsl[0], s = hsl[1], l = hsl[2];

    // ... and check if we have a somewhat green pixel.
    if (h >= 90 && h <= 160 && s >= 25 && s <= 90 && l >= 20 && l <= 75) {
      data[j + 3] = 0;
      revealed[i] = true;
    }
  }
}

If the pixel has not been revealed yet but is a green one, we make it transparent like before and mark it to stay that way.

Demo and screencast

That is all! Take a look at the live demo or watch the screencast below:

I know…

… this is not much of a game but rather a small demo one could turn into a mini-game with little effort. Play around with the code and see what you can come up with!

While recently watching a talk about the new WebRTC features I was reminded of Paul Rouget’s great green screen demo and thought that this would be a cool thing to have for live video as well. Let us build a live green screen!

The markup

<body>
  <video id="v" width="320" height="240"></video>
  <canvas id="c" width="320" height="240"></canvas>
</body>

Those are the parts we need. A <video> element that plays the media stream and a canvas we will use to read and transform image data.

The JavaScript

function initialize() {
  // Get the webcam's stream.
  navigator.getUserMedia({video: true}, startStream, function () {});
}

function startStream(stream) {
  video.src = URL.createObjectURL(stream);
  video.play();

  // Ready! Let's start drawing.
  requestAnimationFrame(draw);
}

We call navigator.getUserMedia() and pass {video: true} as the first argument which indicates that we want to receive a video stream. We assign the MediaStream to the video’s .src property to connect it to the <video> element.

The video starts playing (which means the camera will be activated and you will see your webcam’s live video) and we request an animation frame using the requestAnimationFrame() API. This is perfect for drawing to our canvas as the browser schedules the next repaint and we will be called immediately before that happens. Now for the last and most important part of our green screen:

function draw() {
  var frame = readFrame();

  if (frame) {
    replaceGreen(frame.data);
    context.putImageData(frame, 0, 0);
  }

  // Wait for the next frame.
  requestAnimationFrame(draw);
}

function replaceGreen(data) {
  var len = data.length;

  for (var i = 0, j = 0; j < len; i++, j += 4) {
    // Convert from RGB to HSL...
    var hsl = rgb2hsl(data[j], data[j + 1], data[j + 2]);
    var h = hsl[0], s = hsl[1], l = hsl[2];

    // ... and check if we have a somewhat green pixel.
    if (h >= 90 && h <= 160 && s >= 25 && s <= 90 && l >= 20 && l <= 75) {
      data[j + 3] = 0;
    }
  }
}

What happens here is actually quite simple: we read the current video frame and extract its image data. We then iterate over all pixels in the frame and check if we found a green one - if so its opacity byte is set to zero, which means fully transparent. The manipulated image data is put back into the canvas and we are done for now until the next animation frame is ready.

The demo

Take a look at the live demo, you will need a recent Firefox/Chrome/Opera build. Make sure that getUserMedia() support is enabled in your browser of choice. Hold a green object in front of the the camera and try it out yourself. Your camera and light setup is probably very different from mine so you might need to adjust the color check a little to make it work. Alternatively, here is a screencast of the demo:

The end

This is an admittedly very simple example of a green screen but you can use this little template to manipulate your webcam’s live video stream and build all kinds of fancy demos with it.

I could not be happier ever since I switched from Wordpress to Octopress. I usually write and publish blog posts from where I live, Berlin. The time zone here is CET (UTC+1). While recently visiting Mozilla’s HQ in Mountain View I wrote another blog post just as usual and typed “rake generate” to turn my Markdown files into static HTML files.

Looking at the output though, got me a little puzzled. All timestamps were changed to be calculated off the PDT time zone. While certainly that is not a big deal as they are still the same timestamps, I did not feel like changing all of those every now and then I am somewhere in a different time zone.

If you want to use a “static” time zone when generating your page, do it like this:

TZ=CET rake generate

TL;DR - put your time zone into the TZ variable if you want to force Jekyll to use a specific time zone when generating your HTML files.

CSS transitions are awesome. You can use them to easily animate the transition of one or multiple CSS properties from a given state to another. But how does that work if your element has just been created and inserted into the DOM dynamically?

Let’s take a look at this simple example:

div {
  /* ... */
  transition: opacity 500ms;
}

var elem = document.createElement("div");
document.body.appendChild(elem);

// Make the element fully transparent.
elem.style.opacity = 0;

// Fade it in.
elem.style.opacity = 1;

We dynamically insert a new <div> element into the DOM with its initial opacity set to zero. Subsequently we want it to fade to full opacity. This - as you might have guessed - does of course not work that way.

How about a timeout?

It is clear that we somehow need to make sure the initial state with zero opacity is “applied” before trying to fade in:

var elem = document.createElement("div");
document.body.appendChild(elem);

// Make the element fully transparent.
elem.style.opacity = 0;

// Make sure the initial opacity value is applied.
setTimeout(function () {
  // Fade it in.
  elem.style.opacity = 1;
}, 0);

This is only marginally better. It seems to work with Webkit and Opera (and maybe even IE) but not in Firefox (in 99% of the cases). Using setTimeout() is a little too much overhead and nobody guarantees you that the style has really been applied after some milliseconds. It may be unsupported and unreliable, we need something better.

getComputedStyle to the rescue

There is another way to apply the element’s current style that even works synchronously:

var elem = document.createElement("div");
document.body.appendChild(elem);

// Make the element fully transparent.
elem.style.opacity = 0;

// Make sure the initial state is applied.
window.getComputedStyle(elem).opacity;

// Fade it in.
elem.style.opacity = 1;

Although it looks like we only query the current opacity value, getComputedStyle() in combination with accessing a property value actually flushes all pending style changes and forces the layout engine to compute our <div>’s current state. This workaround works in all major browsers and does not yield different results like the setTimeout() approach.

As you probably already know, Firefox 13 introduced a neat new feature - the new tab page. We replaced the old blank page with a list of thumbnails of recently visited sites. While the feature itself works great for many people it has definitely made opening new tabs a little more noisy.

Do not show loading indicators

As we are now loading a real (although local) page, there are loading indicators when opening a new tab. The throbber starts to spin and the tab title changes to “Connecting…” until the page has loaded. That is a lot of unnecessary noise.

In bug 716108 (Firefox 17) we removed loading indicators for newly opened tabs. No spinning throbber, no flickering tab label. It only is a very subtle change but the whole action of opening a new tab feels a lot smoother again.

Preload new tab pages in the background

If you happen to have a slower machine you will notice that loading the new tab page takes a little while. It is a normal HTML (and partly XUL) page that we need to parse and render. As all tabs start out with a blank docShell you will first see a white canvas that then is replaced by “about:newtab”. As a last step all thumbnails will be loaded and drawn progressively.

Opening a new tab is a very frequent action so it should feel snappy and not get in your way at all. As optimizing the parsing and rendering stages any further is more than a non-trivial task I came up with a little trick in bug 753448. The idea is to preload the new tab page in the background so it has already loaded when users open a new tab. All we now have to do is switch docShells and the new tab page gets shown instantly.

You can give it a try as it landed in yesterday’s Nightly (2012-08-14). Just go to “about:config” and set “browser.newtab.preload” to “true”. This option is not yet enabled by default as we first have to figure out some minor talos regressions until it is ready for prime time.