28 May 2018

Teaching a Neural Network to write text

A simple neural network that writes letters using synaptic and jimp in pure javascript.

Checkout a working example below

I will be running the code in the browser but you are free to use Node.js. We need to include two libraries: Jimp and Synaptic. We will be using the Architect and the Trainer from synaptic.js to create our neural network. If you are using Node.js, you will also have to require Jimp. Next create the “catagorical” helper function. This will transform a letter to a valid input for the neural network. For example: 1(B) => [ 0, 1, 0, 0, …, 0 ].

To create the images from letters we will use Jimp. After loading a jimp font we will start constructing the training data. For every letter in the alphabet we will create an image version of the uppercase letter. “String.fromCharCode” takes in a ascii code and returns the character. In ascii, 65 is the character code for “A”. 66 is the character code for “B”. etc. If you are using Node.js you can write the image (using image.write) to your file system. The training data will be in the format “{ input: number[26], output: number[256] }”. Now use the raw pixel data from image.bitmap.data as output and the catagorical letter index as input. To calculate the greyscale value lets take the sum of the rgb channels and divide by 765. Lets’s console.log to check our training data.

With the training data is in place we can create our neural network by instantiating a new instance of Architect.Perceptron. Pass in the number of nodes per layer to the constructor. In this case we will use 26 nodes in our hidden layer. To train the network you can use the built-in synaptic.Trainer. It’s important to set the learning rate to something below 0.05 for this example as a higher learning rate will make the neural network forget it’s previous results. To improve the results you could use LSTM Networks. Check out a visualization of the network in the image below:

Green lines mean high weight values and red lines low weight values. Green nodes mean a high bias values and red nodes a low bias values.

If you are running this example in the browser you can add a loading message.

Remove the loading message when the neural network is done training.

Finnaly we add a “keypress” event listener to the document. To listen for valid letters. Every time the user presses a valid key we append the neural network’s interpetation of that letter to the document. This can done using Jimp’s image.getBase64 method. If you are using Node.js you could create your own implementation.