Pocket Book of economic indicators by Manuel Jesus-Backus

Show description

You can imagine that re-processing the sentences after each file open would be slow. Another option would be to convert each sentences into lists of word indexes before running your neural network, and to save your word to index mapping to a file as well. That way, your input data can be just a bunch of arrays of word indexes. This would still need to be in multiple files since it would take up too much RAM, but at least you won’t have to convert the data each time you open the file. Yet another option might be to use a simple database like SQLite to store the arrays of word indexes.

We’ll see in the coding lectures how this can already do some amazing things, like exponentially decrease the number of hidden units we would have needed in a feedforward neural network. Prediction and Relation to Markov Models In this section we are going to look more closely at what a recurrent neural network can predict and talk about how, under certain circumstances, we can relate it to what we know about Markov models. Adding a time component gives us a few more options in terms of the objective, or in other words, what we’re trying to predict.

Finding Word Analogies In this section we are going to talk about how you can actually do calculations like show king - man + woman = queen. It’s quite simple but worth going through anyway. I will describe it in 2 steps: 1) is to convert all 3 of the words on the left to their word embeddings, or word vectors. Once they’re in vector form, you can subtract and add very easily. Remember that we can just grab the word’s corresponding word vector by indexing the word embedding matrix with the index of the word.