Update! For El Capitan and users of newer version of OS X, you may run into issues installing Torch or Lua packages. A fix is included now.
Update number two! Zach in the comments offers a really helpful fix if you’re on Sierra.
Update three! A lot has changed since 2016, so I’ll be posting a new version of this tutorial soon. In the meantime, please see the comments for common sticking points and troubleshooting.
There have been many recent examples of neural networks making interesting content after the algorithm has been fed input data and “learned” about it. Many of these, Google’s Deep Dream being the most well-covered, use and generate images, but what about text? This tutorial will show you how to install Torch-rnn, a set of recurrent neural network tools for character-based (ie: single letter) learning and output – it’s written by Justin Johnson, who deserves a huge “thanks!” for this tool.
The details about how all this works are complex and quite technical, but in short we train our neural network character-by-character, instead of with words like a Markov chain might. It learns what letters are most likely to come after others, and the text is generated the same way. One might think this would output random character soup, but the results are startlingly coherent, even more so than more traditional Markov output.
Torch-rnn is built on Torch, a set of scientific computing tools for the programming language Lua, which lets us take advantage of the GPU, using CUDA or OpenCL to accelerate the training process. Training can take a very long time, especially with large data sets, so the GPU acceleration is a big plus.
You can read way more info on how this all works here:
http://karpathy.github.io/2015/05/21/rnn-effectiveness
STEP 1: Install Torch
First, we have to install Torch for our system. (This section via this Torch install tutorial.)
A few notes before we start:
- Installing Torch will also install Lua and luarocks (the Lua package manager) so no need to do that separately.
- If Lua already installed, you may run into some problems (I’m not sure how to fix that, sorry!)
- We’ll be doing everything in Terminal – if you’ve never used the command-line, it would be good to learn a bit more about how that works before attempting this install.
- If you’re running a newer OS such as El Capitan, you may run into problems installing Torch, or installing packages afterwards. If that’s the case, you can follow these instructions.
In Terminal, go to your user’s home directory* and run the following commands one at a time:
|
1 2 3 4 |
git clone https://github.com/torch/distro.git ~/torch --recursive cd ~/torch bash install-deps ./install.sh |
This downloads the Torch repository and installs it with Lua and some core packages that are required. This may take a few minutes.
We need to add Torch to the PATH variable so it can be found by our system. Easily open your .bash_profile file (which is normally hidden) in a text editor using this command:
|
1 |
touch ~/.bash_profile; open ~/.bash_profile |
And add these two lines at very bottom:
|
1 2 |
# TORCH export PATH=$PATH:/Users/<your user name>/torch/install/bin |
…replacing your username in the path. Save and close, then restart Terminal. When done, test it with the command:
|
1 |
th |
Which should give you the Torch prompt. Use Control-c twice to exit, or type os.exit().
* You can install Torch anywhere you like, but you’ll have to update all the paths in this tutorial to your install location.
STEP 2: Install CUDA Support
Note: this step is only possible if your computer has an NVIDIA graphics card!
We can leverage the GPU of our computer to make the training process much faster. This step is optional, but suggested.
Download the CUDA tools with the network install – this is way faster, since it’s a 300kb download instead of 1GB: https://developer.nvidia.com/cuda-downloads.
Run installer; when done, we have to update PATH variable in the .bash_profile file like we did in the last step. Open the file and add these three lines (you may need to change CUDA-<version number> depending on which you install – Kevin points out that CUDA 8 may cause errors):
|
1 2 3 |
# CUDA export PATH=/Developer/NVIDIA/CUDA-7.5/bin:$PATH export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH |
You may also need to modify your System Preferences under Energy Saver:
- Uncheck Automatic Graphics Switch.
- Set Computer Sleep to “Never”.
Restart Terminal and test the install by running this command:
|
1 |
kextstat | grep -i cuda |
You should get something like:
|
1 |
286 0 0xffffff7f8356e000 0x2000 0x2000 com.nvidia.CUDA (1.1.0) 5AFE550D-6361-3897-912D-897C13FF6983 <4 1> |
There are further tests in the NVIDIA docs, if you want to try them, but they’re not necessary for our purposes. If you want to go deeper into this process, you can follow these instructions from NVIDIA.
STEP 3: Install HDF5 Library for Lua
Torch-rnn comes with a preprocessor script, written in Python, that prepares our text for training. It will save our sample into an
h5 and
json file, but requires the HDF5 library to be installed.
First, install HDF5 using Homebrew:
|
1 2 |
brew tap homebrew/science brew install hdf5 |
(If you have issues with the install or in the next step, Joshua suggests adding the the flag --with-mpi to the Homebrew command above, which may help. If that doesn’t work, Charles has a suggested fix. If you get an error that says Unsupported HDF5 version: 1.10.0 , you can try Tom’s suggestion.)
Move to the Torch folder inside your user home directory (ie: /Users/<your user name>/torch/). The following commands download the Torch-specific HDF5 implementation and installs them:
|
1 2 3 |
git clone git@github.com:deepmind/torch-hdf5.git cd torch-hdf5 luarocks make hdf5-0-0.rockspec |
If you haven’t used git or Github before, as Luke points out in the comments, you might get an SSH key error. You can get a key, or just download the repository manually from here.
STEP 4: Install HDF5 Library for Python
We also need to install HDF5 support for Python. You can do this using Pip:
|
1 |
sudo pip install h5py |
You may get a bunch of warnings, but that’s ok. Test that it works by importing the library:
|
1 2 |
python import h5py |
If it imports without error, you’re good!
STEP 5: Install Torch-rnn
Now that we’ve prepared our computer with all the required libraries, it’s time to finally install Torch-rnn!
- Download the ZIP file from the project’s GitHub repository.
- Unzip it and rename to torch-rnn.
- Move the Torch-rnn folder to your Torch install folder inside your user home directory (ie: /Users/<your user name>/torch/torch-rnn )
- (You can also do this by cloning the repo, but if you know how to do that, you probably don’t need the instructions in this step 😄)
STEP 6: Prepare Your Data
We’re ready to prepare some data! Torch-rnn comes with a sample input file (all the writings of Shakespeare) that you can use to test everything. Of course, you can also use your own data; just combine everything into a single text file.
In the Terminal, go to your Torch-rnn folder and run the preprocessor script:
|
1 |
python scripts/preprocess.py --input_txt data/tiny-shakespeare.txt --output_h5 data/tiny_shakespeare.h5 --output_json data/tiny_shakespeare.json |
You should get a response that looks something like this:
|
1 2 3 4 5 6 |
Total vocabulary size: 65 Total tokens in file: 1115394 Training size: 892316 Val size: 111539 Test size: 111539 Using dtype <type 'numpy.uint8'> |
This will save two files to the data directory (though you can save them anywhere): an h5 and json file that we’ll use to train our system.
STEP 7: Train
The next step will take at least an hour, perhaps considerably longer, depending on your computer and your data set. But if you’re ready, let’s train our network! In the Torch-rnn folder and run the training script (changing the arguments if you’ve used a different data source or saved them elsewhere):
|
1 |
th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json |
The train.lua script uses CUDA by default, so if you don’t have that installed or available, you’ll need to disable it and run CPU-only using the flag -gpu -1. Lots more training and output options are available here.
It should spit out something like:
|
1 2 3 4 5 |
Running with CUDA on GPU 0 Epoch 1.00 / 50, i = 1 / 17800, loss = 4.163219 Epoch 1.01 / 50, i = 2 / 17800, loss = 4.078401 Epoch 1.01 / 50, i = 3 / 17800, loss = 3.937344 ... |
Your computer will get really hot and it will take a long time – the default is 50 epochs. You can see how long it took by adding time in front of the training command:
|
1 |
time th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json |
If you have a really small corpus (under 2MB of text) you may want to try adding the following flags:
|
1 |
-batch_size 1 -seq_length 50 |
Setting -batch_size somewhere between 1-10 should give better results with the output.
STEP 8: Generate Some Output
Getting output from our neural network is considerably easier than the previous steps (whew!). Just run the following command:
|
1 |
th sample.lua -checkpoint cv/checkpoint_10000.t7 -length 2000 |
A few notes:
- The -checkpoint argument is to a t7 checkpoint file created during training. You should use the one with the largest number, since that will be the latest one created. Note: running training on another data set will overwrite this file!
- The -length argument is the number of characters to output.
- This command also runs with CUDA by default, and can be disabled the same way as the training command.
- Results are printed to the console, though it would be easy to pipe it to a file instead:
1th sample.lua -checkpoint cv/checkpoint_10000.t7 -length 2000 > my_new_shakespeare.txt - Lots of other options here.
STEP 8A: “Temperature”
Changing the temperature flag will make the most difference in your network’s output. It changes the novelty and noise is the system, creating dramatically different output. The
-temperature argument expects a number between 0 and 1.
Higher temperature
Gives a better chance of interesting/novel output, but more noise (ie: more likely to have nonsense, misspelled words, etc). For example,
-temperature 0.9 results in some weird (though still surprisingly Shakespeare-like) output:
“Now, to accursed on the illow me paory; And had weal be on anorembs on the galless under.”
Lower temperature
Less noise, but less novel results. Using
-temperature 0.2 gives clear English, but includes a lot of repeated words:
“So have my soul the sentence and the sentence/To be the stander the sentence to my death.”
In other words, everything is a trade-off and experimentation is likely called for with all the settings.
All Done!
That’s it! If you make something cool with this tutorial, please tweet it to me @jeffthompson_.
Never mind, i believe i got it to work, i was just being a bell end
Please help! I am struck at step 7:
here is the error:
iMacs-iMac:torch-rnn-master imac$ th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json
/Users/imac/torch/install/bin/luajit: /Users/imac/torch/install/share/lua/5.1/trepl/init.lua:389: /Users/imac/torch/install/share/lua/5.1/trepl/init.lua:389: module ‘hdf5’ not found:No LuaRocks module found for hdf5
no field package.preload[‘hdf5’]
no file ‘/Users/imac/.luarocks/share/lua/5.1/hdf5.lua’
no file ‘/Users/imac/.luarocks/share/lua/5.1/hdf5/init.lua’
no file ‘/Users/imac/torch/install/share/lua/5.1/hdf5.lua’
no file ‘/Users/imac/torch/install/share/lua/5.1/hdf5/init.lua’
no file ‘./hdf5.lua’
no file ‘/Users/imac/torch/install/share/luajit-2.1.0-beta1/hdf5.lua’
no file ‘/usr/local/share/lua/5.1/hdf5.lua’
no file ‘/usr/local/share/lua/5.1/hdf5/init.lua’
no file ‘/Users/imac/.luarocks/lib/lua/5.1/hdf5.so’
no file ‘/Users/imac/torch/install/lib/lua/5.1/hdf5.so’
no file ‘./hdf5.so’
no file ‘/usr/local/lib/lua/5.1/hdf5.so’
no file ‘/usr/local/lib/lua/5.1/loadall.so’
stack traceback:
[C]: in function ‘error’
/Users/imac/torch/install/share/lua/5.1/trepl/init.lua:389: in function ‘require’
train.lua:6: in main chunk
[C]: in function ‘dofile’
…imac/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x01039bda10
@Raja – did you complete Step 3? The error says that you don’t have the hdf5 library for Lua installed.
Fantastic work, Jeff! I trained a network using the tweets and transcripts of a very controversial politician from my country and I made a Twitter account that tweets his thoughts (clearly, I was inspired by @DeepDrumpf :) ). I have a question: I don’t quite understand how the -sample parameter affects the output. Could you please provide an insight?
@Alejandro – glad it helped! When you get it done, post a link here! Re
sampleI’m not entirely sure – it has something to do with the way the RNN is encoded and sampled from. There’s a hint about it here. It seems that setting-sample 1will give more novel results, but probably also risk being less intelligible, liketemperature. Afraid that’s all I can surmise, for more you’ll probably have to do some research.Cool! I’m still trying to improve its grammar but I’ve obtained pretty funny results (for example: “Good morning, postmoderns” (he is an ultra conservative politician so it’s funny, hehe)).
This is the link: https://twitter.com/A_OrdonezDeep
(the tweets are in spanish).
I’d be very happy if you shared this little project. I’m having a lot of fun with it :)
Hello Jeff,
Sorry, I missed the third step. Now, I made it right. However, I am getting this error:
MARSs-MacBook-Pro:torch-rnn mars$ th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json
/Users/mars/torch/install/bin/luajit: /Users/mars/torch/install/share/lua/5.1/trepl/init.lua:389: /Users/mars/torch/install/share/lua/5.1/trepl/init.lua:389: /Users/mars/torch/install/share/lua/5.1/hdf5/ffi.lua:42: Error: unable to locate HDF5 header file at /usr/local/Cellar/hdf5/1.10.1/include;/usr/include;/usr/local/opt/szip/include/hdf5.h
stack traceback:
[C]: in function ‘error’
/Users/mars/torch/install/share/lua/5.1/trepl/init.lua:389: in function ‘require’
train.lua:6: in main chunk
[C]: in function ‘dofile’
…mars/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x010e303a10
Please help!
@Raja – did you get any errors when you installed the HDF5 library for Lua? Can you look in the HDF5 path it lists (
/usr/local/Cellar/hdf5/1.10.1/include;/usr/include;/usr/local/opt/szip/include/hdf5.hand see if it’s there?Here is the installation:
MARSs-MacBook-Pro:torch-hdf5 mars$ luarocks make hdf5-0-0.rockspec
Missing dependencies for hdf5:
totem
Using https://raw.githubusercontent.com/torch/rocks/master/totem-0-0.rockspec… switching to ‘build’ mode
Cloning into ‘torch-totem’…
remote: Counting objects: 28, done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 28 (delta 0), reused 13 (delta 0), pack-reused 0
Receiving objects: 100% (28/28), 20.25 KiB | 0 bytes/s, done.
Updating manifest for /Users/mars/torch/install/lib/luarocks/rocks
totem 0-0 is now built and installed in /Users/mars/torch/install/ (license: BSD)
cmake -E make_directory build;
cd build;
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=”/Users/mars/torch/install/bin/..” -DCMAKE_INSTALL_PREFIX=”/Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0″;
make
— The C compiler identification is AppleClang 8.1.0.8020042
— The CXX compiler identification is AppleClang 8.1.0.8020042
— Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
— Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc — works
— Detecting C compiler ABI info
— Detecting C compiler ABI info – done
— Detecting C compile features
— Detecting C compile features – done
— Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
— Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ — works
— Detecting CXX compiler ABI info
— Detecting CXX compiler ABI info – done
— Detecting CXX compile features
— Detecting CXX compile features – done
— Found Torch7 in /Users/mars/torch/install
— HDF5: Using hdf5 compiler wrapper to determine C configuration
— Found HDF5: /usr/local/Cellar/hdf5/1.10.1/lib/libhdf5.dylib;/usr/local/opt/szip/lib/libsz.dylib;/usr/lib/libz.dylib;/usr/lib/libdl.dylib;/usr/lib/libm.dylib (found suitable version “1.10.1”, minimum required is “1.8”)
— Configuring done
— Generating done
— Build files have been written to: /Users/mars/torch/torch-hdf5/build
cd build && make install
Install the project…
— Install configuration: “Release”
— Generating /Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0/lua/hdf5/config.lua
— Installing: /Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0/lua/hdf5/dataset.lua
— Installing: /Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0/lua/hdf5/datasetOptions.lua
— Installing: /Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0/lua/hdf5/ffi.lua
— Installing: /Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0/lua/hdf5/file.lua
— Installing: /Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0/lua/hdf5/group.lua
— Installing: /Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0/lua/hdf5/init.lua
— Installing: /Users/mars/torch/install/lib/luarocks/rocks/hdf5/0-0/lua/hdf5/testUtils.lua
Updating manifest for /Users/mars/torch/install/lib/luarocks/rocks
hdf5 0-0 is now built and installed in /Users/mars/torch/install/ (license: BSD)
I was able to find the hdf5.h in the /usr/local/Cellar/hdf5/1.10.1/include folder, but was able to locate the file in /usr/local/opt/szip/include/ folder. So I went and copied the file from the other folder and tried to run the script. But I am getting the same error!
Hello Jeff,
Sorry for my messages.
Copying the file ratified the previous error!
I did a restart and now I am getting a different error:
MARSs-MacBook-Pro:~ mars$ th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json
/Users/mars/torch/install/bin/luajit: cannot open train.lua: No such file or directory
stack traceback:
[C]: in function ‘dofile’
…mars/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x01072d8a10
Sorry again, the previous error still exists. please help!
MARSs-MacBook-Pro:torch-rnn mars$ th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json
/Users/mars/torch/install/bin/luajit: /Users/mars/torch/install/share/lua/5.1/trepl/init.lua:389: /Users/mars/torch/install/share/lua/5.1/trepl/init.lua:389: /Users/mars/torch/install/share/lua/5.1/hdf5/ffi.lua:42: Error: unable to locate HDF5 header file at /usr/local/Cellar/hdf5/1.10.1/include;/usr/include;/usr/local/opt/szip/include/hdf5.h
stack traceback:
[C]: in function ‘error’
/Users/mars/torch/install/share/lua/5.1/trepl/init.lua:389: in function ‘require’
train.lua:6: in main chunk
[C]: in function ‘dofile’
…mars/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x010dcb4a10
@Raja – I’m not sure. These installs sometimes depend one thing on another, so it might be best to uninstall all the Lua stuff and start over. Otherwise, you’ll have to try a Lua forum, I don’t know much about how to use or troubleshoot it.
I am receiving the following error on running Step 8:
$ th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json -gpu -1
In file included from /usr/local/Cellar/hdf5/1.10.1/include/hdf5.h:22:
/usr/local/Cellar/hdf5/1.10.1/include/H5public.h:59:13: fatal error: ‘mpi.h’
file not found
# include
^
1 error generated.
Any idea on how to fix this?
When I run step 7, I am unable to locate the HDF5 header file even though I completed all the previous steps properly (except the CUDA one)
This is the error that I receive:
Error: unable to locate HDF5 header file at /usr/local/Cellar/hdf5/1.10.1/include;/usr/include;/usr/local/opt/szip/include/hdf5.h
@Nerdcortex – it appears a lot of people have this issue with the HDF5 library. You can see this solution, which should work: http://www.jeffreythompson.org/blog/2016/03/25/torch-rnn-mac-install/comment-page-1/#comment-275266. Your other issue, I think, will be solved when you get HDF5 working, no?
@charles can you help me out with the hdf5 installation on mac?
I’m running OS-X Sierra and I see the following note:
———-
If you’re running a newer OS such as El Capitan, you may run into problems installing Torch, or installing packages afterwards. If that’s the case, you can follow these instructions.
———-
When I click the link to “follow these instructions” it’s just a bunch of posts of people debating the problem, there are no instructions there?
Is there a clean set of instructions for newer OS-X?
Thanks for the excellent tutorial! I’m stuck on step 3, however, when attempting to install hdf5. Running MacOS Sierra.
In terminal, brew install hdf5 does fine for a while but then I get to
==> ../configure –build=x86_64-apple-darwin16.7.0 –prefix=/usr/local/Cellar/gc
==> make
And it just hangs. And hangs. And hangs. For over an hour until I finally get fed up and ctl-C out of the program. I’ve followed step 1 to the letter (step 2 doesn’t apply since I don’t have NVidia). Am I doing something wrong?
Hi Jeff, I’m new to all this but so far everything’s been really easy to follow. My problem isn’t really with error messages as much as just nothing happening, I think the step I’m stuck on is importing the h5py library for python. Once I type Python and then Import h5py, I get a >>> that just sits there for ever.
At first I didn’t realize I hadn’t successfully negotiated that step and had been trying to train, and a similar thing was happening, I think I was getting a >… that was just hanging there indefinitely but I never got any feedback telling me anything was happening except my terminal window didn’t want to close.
For reference I’m on a Mac OS Sierra
@Tori, ah I (think) that’s an easy one! That’s actually a good sign, showing that
h5pyimported correctly and it’s ready to go – the>>is a Python prompt where you’d type code. You can just typeexit()and continue to the next step!oh, cool! no wonder it kept saying things like, “yes we have this installed” and, “that already exists damnit”
anyway i still have the problem of training, i don’t have a GPU so i’ve been adding the -gpu -1 flags but no matter how long i let my computer sit i never get any response. it doesn’t tell me anything’s “running with not-cuda on cpu,” i get no “epoch” anything, ever.
i also haven’t gotten any error messages though, so it’s hard to actually search what’s wrong. is nothing wrong? do i just need to wait more than ~10h for the first output?
oh wait never mind now i have the same unable to locate HDF5 header file at /usr/local/Cellar/hdf5/1.10.1_2/include;/usr/local/opt/szip/include/hdf5.h error as everybody else! horay!
OK, i think i’ve managed to drag my way through the worst of it but now the issue is:
invalid argument: input_h5
when i try to train
what’s that about
OH MMMY GOD IT’S WORKING
Hello. I got this running in the past but had some trouble recently on Mac OS Sierra. I’m not sure what changed, but after following several suggestions in these comments and with much DuckDuckGo’ing, I’ve got it working again — at least the training part. (I’m still waiting for it to finish before I can test any output.)
So here’s what I did. I’ll try to format as clearly as I can. Mostly it has to do with editing the
.luafiles in the install directory, which for me was/Users//torch/install/share/lua/5.1/hdf5/.Now, I did a few other updates and things that I can’t quite remember, so no guarantees — but I at least did these three things.
When I got an error with something like
ffi.lua:56: ')' expected near '_close' at line 1472, the solution was to edit the fileffi.lua, and edit line 44 so that instead of this:it now looks like this
The next error said something like
HDF5File.__init() requires a fileID - perhaps you want HDF5File.create()?, and the solution for this one was to editfile.lua so that theopenFuncfunction around line 145 goes from this:to this:
And finally, when I got an error like
group.lua:88: attempt to concatenate 'int64_t' and 'string', the solution was to edit that line (88) ingroup.luafrom this:to this:
And then, FINALLY, it started training again with the
thcommand from Step 7.Hopefully this works for you too!
Yikes, thanks Zach for such detailed info! I’ve put off installing Sierra because I know it will break *everything*.
Great tutorial Jeffrey! I was able to successfully create the my_new_shakespeare.txt file! Now, I want to try to do this with an audio file. I have combined 500 mp3 files into one big mp3 file, but am having trouble putting it into the Machine Learning engine.
Please let me know of any sources I can use to load audio into the torch system.
@Ruchir – thanks! Torch-rnn is really just meant for text, so it won’t work easily with audio. MP3s would be particularly difficult since the data is compressed instead of linear, much like a JPG. You might be able to try it with raw WAV data and add the header at the end using Audacity.
@jeff Thompson
Thanks for the quick response! I tried converting it into raw WAV data, however, I get a UnicodeDecodeError since the ASCII representation of a WAV file when converted to text has some unknown characters. Here is a screenshot to better understand my situation:
https://i.stack.imgur.com/RZz3S.png
Do you know if there is a way to get a textual data representation of the audio file without a UnicodeDecodeError?
Hi!
I have a AMD Radeon R9 M290 2048 MB
Is this installation still possible by just skipping Step 2?
Thank you!
@Matt G – dunno, you’ll have to try it!
@Ruchir – I see “LAME” in there which suggests maybe you didn’t actually create a WAV file? There should be some info above about modifying the Python code to deal with Unicode characters.
I worked through the issue above, it had something to do with an outdated version of Xcode and Luajit.
I have made it to step 7, and now I’m running into this error. I have no idea what the problem is here. Any help is appreciated.
[My Macbook Pro:torch-rnn davidblacklow$ th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json -gpu -1
/Users/davidblacklow/torch/install/bin/luajit: …davidblacklow/torch/install/share/lua/5.1/trepl/init.lua:389: …davidblacklow/torch/install/share/lua/5.1/trepl/init.lua:389: …s/davidblacklow/torch/install/share/lua/5.1/hdf5/ffi.lua:56: ‘)’ expected near ‘_close’ at line 3400
stack traceback:
[C]: in function ‘error’
…davidblacklow/torch/install/share/lua/5.1/trepl/init.lua:389: in function ‘require’
train.lua:6: in main chunk
[C]: in function ‘dofile’
…klow/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x01048be340
Thanks!
@ZackB – did you add it to your PATH variable in step 1?
@ZackB – I have no idea, you might have to post to a Lua forum for that.
I fixed the above error by following instructions here: https://github.com/deepmind/torch-hdf5/issues/83#issuecomment-254427843
After doing this I needed to go back and follow these instructions:
http://www.jeffreythompson.org/blog/2016/03/25/torch-rnn-mac-install/comment-page-2/#comment-287591
I’m stuck starting Step 7 running in OSX El Capitan 10.11.6.
Anyone know how to fix these errors?
Roberts-Macbook-Retina:torch-rnn robert$ th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json -gpu -1
Running in CPU mode
/Users/robert/torch/install/bin/luajit: /Users/robert/torch/install/share/lua/5.1/hdf5/file.lua:10: HDF5File.__init() requires a fileID – perhaps you want HDF5File.create()?
stack traceback:
[C]: in function ‘assert’
/Users/robert/torch/install/share/lua/5.1/hdf5/file.lua:10: in function ‘__init’
/Users/robert/torch/install/share/lua/5.1/torch/init.lua:91: in function
[C]: in function ‘open’
./util/DataLoader.lua:17: in function ‘__init’
/Users/robert/torch/install/share/lua/5.1/torch/init.lua:91: in function
[C]: in function ‘DataLoader’
train.lua:76: in main chunk
[C]: in function ‘dofile’
…bert/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x010c8d2d10
I successfully installed everything and am able to get the RNN to run the train command, but get this error:
/Users/flatironschool/torch/install/bin/luajit: train.lua:77: Expected value but found invalid unicode escape code at character 957
stack traceback:
[C]: in function ‘read_json’
train.lua:77: in main chunk
[C]: in function ‘dofile’
…hool/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x010acd0e90
Can’t find much via searching about what’s wrong. I assume there’s an unsupported character in my data file?
@Jason – there’s a bit about that in the tutorial, probably a Unicode character that throws an error. You can strip non-ascii characters, or try my mod above.
@Robert, I had your problem also and solved it using this https://github.com/deepmind/torch-hdf5/issues/81 — specifically, “So I commented out the offending assert in file.lua and added an explicit conversion
fileID=tonumber(fileID)”
I also had to modify one other file, to wrap a “tostring()” around something.
This may be a redundant post and if so, my apologies, but I could only get torch-rnn working on macOS 10.13.3 by installing hdf5 1.8.20 and not 1.10.0. Taking that change into account, everything else worked fine.
Hi,
I am also running into a problem of ‘Expected value but found invalid unicode escape code’ while attempting to train rnn model.
It works fine training model on tine-shakespare example dataset so I suppose its some characters cannot be processed. I’ve tried taking out non Ascii characters in pre-processing, but it still throws same error.
Could you please advice me.
Thanks!
I don’t know if anyone has already mentioned it somewhere in the comments here and I overlooked it but I wanted to add it just in case:
concerning @Tom Schofield fix, for me it was additionally necessary to change the path of the hdf5 library in the config.lua file to the appropriate library in /Cellar/… after that it worked beautifully.