Image Captioning from Novels

captioningimagefromtags

Some progress, trying to caption images using sentences from novels. This first step is working ok, next will be to train a neural-network to do this automatically from my captioned set, then output new captions algorithmically.

Part of a short-term collaboration with the Social Dynamics group at Bell Labs, Cambridge.

“Preferred Stimuli” for DNNs

FeaturesAtLastLayerOfAlexNet_PreferredStimuliForEachNeuron-web

Not *entirely* sure what’s going on here (paper forthcoming) but I believe this is the equivalent of tickling a deep neural network: “understanding deep neural networks by synthetically generating the preferred stimuli for each of their neurons.” Via Anh Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, and Jeff Clune at EvolvingAI.org.

Torch-rnn: Mac Install

TorchRNN_TinyShakespeare-web

Update! For El Capitan and users of newer version of OS X, you may run into issues installing Torch or Lua packages. A fix is included now.

There have been many recent examples of neural networks making interesting content after the algorithm has been fed input data and “learned” about it. Many of these, Google’s Deep Dream being the most well-covered, use and generate images, but what about text? This tutorial will show you how to install Torch-rnn, a set of recurrent neural network tools for character-based (ie: single letter) learning and output – it’s written by Justin Johnson, who deserves a huge “thanks!” for this tool.

The details about how all this works are complex and quite technical, but in short we train our neural network character-by-character, instead of with words like a Markov chain might. It learns what letters are most likely to come after others, and the text is generated the same way. One might think this would output random character soup, but the results are startlingly coherent, even more so than more traditional Markov output.

Torch-rnn is built on Torch, a set of scientific computing tools for the programming language Lua, which lets us take advantage of the GPU, using CUDA or OpenCL to accelerate the training process. Training can take a very long time, especially with large data sets, so the GPU acceleration is a big plus.

Continue reading “Torch-rnn: Mac Install”