Robert Price

As part of the OCR in my Draw Something Solver, I needed to turn animage into a two tone black and white image. This is really simple to do. Firstly you need to turn the image to grayscale, then you need to turn all pixels with a value over a certain threshold to 255 for the red, green and blue values, or to 0 if under the threshold.

I’ve already covered Grayscaling an HTML 5 Canvas so I will assume this has already taken place and you have the ImageData from an HTML 5 Canvas in an object called imageData.

To turn this to a black and white image we can use the following code…

for (var i = 0; i < imageData.data.length; i+=4) {
  imageData.data[i] = imageData.data[i+1] = imageData.data[i+2] = imageData.data[i] > threshold ? 255 : 0;
}

You just need to change the value of the threshold variable to adjust the sensitivity of the conversion.

Taking the previous red channel image conversion…

Setting the threshold to 200, the converted black and white image looks like this…

I’ve already mentioned about my first OCR attempt for my Draw Something Solver, and the disappointing results.

I said the letters did not appear to be consistant, this turned out to be on my iPhone 4S, the spacing of the letters is not consistant, there are a few pixels difference between the letters on the left and those on the right. Once I had discovered this, the letters matched.

However, taking this approach meant only images taken from a retina enabled iPhone would work, those from a lower resolution screen would not match. I would need to take screenshots of every letter on every device at every resolution to be consistant. Not a very good idea.

My next approach was to scale to image down to a common size using an HTML5 canvas, that didn’t produce consistant results. I tried to grayscale, and to turn to black and white, but although close, they were still too different.

My final approach, and one that seems to work for now is to still scale and turn the images to black and white, but instead of hashing this result, I am instead doing some pixel comparison and generating probabilities of matching a letter. This is a simple neural network. I have trained it with all the letters from my iPhone 4S so that runs consistantly each time. When I ran against images from an iPhone 3GS, not all images matched, letters like “I” and “J” caused confusion, so where a match was wrong I added some extra training data for those letters so they all now match. I have yet to try images from an iPad as I don’t have one, but I hope they will match first time without further training.

One of the great advantages of using a library like jQuery is that is provides functions like ready() so you can activate your JavaScript code when the DOM is ready, and not when everything has been loaded. This can be significantly earlier, especially if you an image heavy page.

The ready() function is quite chunky, and has to account for a variety of browsers and versions. However, if you know you are running on a recent browser, you can always look at hooking directly into the DOMContentLoaded event. This is part of the HTML5 spec.

To use it you need to add an event listener using the document.addEventListener method. This takes two parameters, the name of the event – in this case DOMContentLoaded, and a callback function to call when the event is triggered.

document.addEventListener("DOMContentLoaded", function domready() {
  document.removeEventListener("DOMContentLoaded", domready);
  console.log("DOM Ready");
});

Notice how I have given the function a name – domready(), this is so I can unhook the function from the event listener once the event has fired.

I’ve been looking at how to turn a colour image into a grayscale one using JavaScript on an HTML 5 Canvas. I had done something similar a few years back to an image using Perl.

Wikipedia has a good article on converting colour to grayscale, and it’s very simple formula based on the luminance of the image. The recommendation is to add together 30% of the red value, 59% of the green value and 11% of the blue value.

var luma = red * 0.3 + green * 0.59 + blue * 0.11;

We can easily apply this formula over the ImageData in an HTML 5 Canvas Context object to turn the image held in the Canvas to grayscale. The ImageData.data holds an array with four pieces of data per pixel, the red, the green, the blue and the alpha values. These are integer values between 0 and 255. The code to grayscale the ImageData would look like this.

for (var i = 0; i < imageData.data.length; i+=4) {
  var luma = Math.floor(imageData.data[i] * 0.3 +
  imageData.data[i+1] * 0.59 +
  imageData.data[i+2] * 0.11);
  imageData.data[i] = imageData.data[i+1] = imageData.data[i+2] = luma;
  imageData.data[i+3] = 255;
}

Notice the use of Math.floor(), this is just to round the luminance down into an integer value.

There is another common way to grayscale an image, and that is to get the red, green and blue pixels to all match. You can pick the colour you want to be the key, for example red, then just make sure the values for the green and blue for each pixel match that of the red one. This is making use of a single channel, and can give very different effects depending on your source image.

To use the red channel we could the following code over some ImageData.

for (var i = 0; i < imageData.data.length; i+=4) {
  imageData.data[i+1] = imageData.data[i+2] = imageData.data[i];
  imageData.data[i+3] = 255;
}

To use the green channel, use the following.

for (var i = 0; i < imageData.data.length; i+=4) {
  imageData.data[i] = imageData.data[i+2] = imageData.data[i+1];
  imageData.data[i+3] = 255;
}

To use the blue channel, use the following.

for (var i = 0; i < imageData.data.length; i+=4) {
  imageData.data[i] = imageData.data[i+1] = imageData.data[i+2];
  imageData.data[i+3] = 255;
}

As part of my Draw Something Solver, I’ve been working with HTML 5 Canvas objects quite a bit. As I want my application to work on all iPhones, I need to support both regular and retina displays. As I need to detect the letters on the image, I really need the images to be of the same size, however a regular image is 320×480, whereas a retina image is 640×960.

The eagle eyed amongst you will see that this is a factor of 2 difference between the two.

The Canvas Context object provides a scale() method, that you pass in x and y scaling factors into.

So if I have a Canvas object called screenshotCanvas, and an Image object called screenshotImage I want to scale down by half, I can use the following JavaScript code.

screenshotCanvasContext = screenshotCanvas.getContext('2d');
screenshotCanvasContext.scale(0.5, 0.5);
screenshotCanvasContext.drawImage(screenshotImage,
                                  0,
                                  0,
                                  screenshotImage.width,
                                  screenshotImage.height);

If I want to double the image in size, I just change the scale factor to 2.

screenshotCanvasContext = screenshotCanvas.getContext('2d');
screenshotCanvasContext.scale(2, 2);
screenshotCanvasContext.drawImage(screenshotImage,
                                  0,
                                  0,
                                  screenshotImage.width,
                                  screenshotImage.height);