September 22, 2019

427 words 3 mins read



Natural Language Processing Tutorial for Deep Learning Researchers

repo name graykode/nlp-tutorial
repo link
language Jupyter Notebook
size (curr.) 315 kB
stars (curr.) 5340
created 2019-01-09
license MIT License


nlp-tutorial is a tutorial for who is studying NLP(Natural Language Processing) using TensorFlow and Pytorch. Most of the models in NLP were implemented with less than 100 lines of code.(except comments or blank lines)

Curriculum - (Example Purpose)

1. Basic Embedding Model

2. CNN(Convolutional Neural Network)

3. RNN(Recurrent Neural Network)

4. Attention Mechanism

5. Model based on Transformer

Model Example Framework Lines(torch/tensor)
NNLM Predict Next Word Torch, Tensor 67/83
Word2Vec(Softmax) Embedding Words and Show Graph Torch, Tensor 77/94
TextCNN Sentence Classification Torch, Tensor 94/99
TextRNN Predict Next Step Torch, Tensor 70/88
TextLSTM Autocomplete Torch, Tensor 73/78
Bi-LSTM Predict Next Word in Long Sentence Torch, Tensor 73/78
Seq2Seq Change Word Torch, Tensor 93/111
Seq2Seq with Attention Translate Torch, Tensor 108/118
Bi-LSTM with Attention Binary Sentiment Classification Torch, Tensor 92/104
Transformer Translate Torch 222/0
Greedy Decoder Transformer Translate Torch 246/0
BERT how to train Torch 242/0


  • Python 3.5+
  • Tensorflow 1.12.0+
  • Pytorch 0.4.1+
  • Plan to add Keras Version


comments powered by Disqus