Zero To One For NLP

All the best NLP resources out there

---

Come join Maxpool - A Data Science community to discuss real ML problems

NLP has grown into a field of wide applications ranging from chatbots to news generation. I started my journey into NLP with standard courses but a majority of my practical learnings came from personal blogs of researchers.

This metablog is a collection of my favourite blogs and courses I go through every once in a while to refresh myself.

Basics

• Intro to NLP(Theory) — Stanford OR Jurafsky & Manning book

• Spacy 101

• Text Mining

• Coursera sequence models

• CS224n — Natural Language Processing with Deep Learning — 2019

• BigO for Data Scientists

fastai

1. Fastai ML

2. Fastai DL

3. Fastai NLP

Embeddings

https://github.com/graykode/nlp-roadmap

Word2Vec

• Jalammar

• Lilian

• McCormik — Part 1

• McCormik — Part 2

GLOVE

Attention

• Attention by Lilian

Language models

ELMO

• ELMO by mlexplained

• ELMO by Edward Ma

• ELMO by Eric

Transformer

• Transformer by Ashish Vaswani

• Transformer by Peter

• Transformer by Jalammar

• Transformer by MLexplained

• Attention mechanism

• Attention articles by school of AI

• Understanding transformers

BERT

• BERT by Jalammar

• BERT by mlexplained

• Łukasz Kaiser’s talk

• McCormick BERT series

GPT

• GPT-2 by Jalammar

• Annotated GPT-2

• BERT Vs GPT-2

XLNet by MLexplained

DistilBERT

ALBERT

Embeddings summary by Lilian Weng

Transfer learning

ULMFiT (Universal Language Model For Fine Tuning)

• RNN theory

• Seq2Seq theory

• Neural machine translation(NMT)

• AWD-LSTM

• ULMFiT

Ruder on Multi-task learning

Transfer learning

• NAACL

• INRIA

• Notebook with all transfer learning examples

PyTorch

Get Pro in PyTorch For NLP

Deep Learning (with PyTorch)

NN concepts

• Improving NNs

• Why momentum works?

• Learning-rate-tuning

• Cyclical Learning Rates

• Pros and cons of activation functions

Normalisation

• Why batch normalisation works

• Weight and layer normalisation

• Overview of normalisation techniques

4 Sequence Encoding Blocks You Must Know Besides RNN/LSTM

Conversational AI

Neural Approaches review paper

What makes a good conversation?

Denny’s

• DL for chatbots Part 1

• DL for chatbots Part 2

Han’s

• Question answering systems Part 1

• Question answering systems Part 2

• Neural Information Retrieval

Rasa

• Bot levels

• Level 3 bot

• 13 rules for chatbot design

• Rasa NLU in Depth: Part 1 — Intent Classification

• Rasa NLU in Depth: Part 2 — Entity Recognition

• Rasa NLU in Depth: Part 3 — Hyperparameter Tuning

Huggingface

• A Transfer Learning Approach for Neural Network Based Conversational Agents

Putting ML to production

• Full Stack Deep Learning

• Rules of ML by Google

• Continuous delivery for ML

• Deploying ML models

• Deploying NLP models

• ML system design

• ML Ops Day — Oscon 2019

• Deploying transformers

• Technical debt in ML

• Model management

Airbnb

• Zipline — Data management

• Bighead — ML infra

Uber

• Michelangelo

• Scaling Michelangelo

• Big Data Infrastructure

• Uber engineering

Databricks Mlflow

Extra tips

• Training NN on GPUs

• pytorch-lightning

• Distributed training in pytorch

• Faster training with large batches

Interview

• NLP Interview Questions

• Reverse interview

• How to build your project portfolio?

Libraries

• Custom training in Spacy

• Spacy transformers

• Spacy crash course

• Serving Google BERT in Production

Semantic search

On Semantic Search

• Generic Neural Elastic Search

• Sentence-transformers

• Approximate nearest neighbors

• Sentence-embedding

Random

• Tricks in NLP

• Tranformers time benchmarking

• Text data augmentation

• Topic modelling by Nanonets

• How to detect fake text?

• An Embarrassingly Simple Approach for Transfer Learning

• Machine Learning Fairness

Hackathons

• Reflecting back on one year of Kaggle

• Kaggle ensemble guide

AutoML

• Automl for predictive modeling

Papers and code

• Papers on language models

• Tasks in NLP

• NLPprogress

• State of the art NLP papers

This was originally published at Modern NLP.

---

Come join Maxpool - A Data Science community to discuss real ML problems!

I am also on Medium, Twitter & LinkedIn!!