Pratik’s Pakodas 🍿

Share this post

Zero To One For NLP

pakodas.substack.com

Zero To One For NLP

All the best NLP resources out there

Pratik Bhavsar
Jun 10, 2019
3
Share
Share this post

Zero To One For NLP

pakodas.substack.com

Come join Maxpool - A Data Science community to discuss real ML problems

NLP has grown into a field of wide applications ranging from chatbots to news generation. I started my journey into NLP with standard courses but a majority of my practical learnings came from personal blogs of researchers.

This metablog is a collection of my favourite blogs and courses I go through every once in a while to refresh myself.

Basics

  • Intro to NLP(Theory) — Stanford OR Jurafsky & Manning book

  • Spacy 101

  • Text Mining

  • Coursera sequence models

  • CS224n — Natural Language Processing with Deep Learning — 2019

  • BigO for Data Scientists

fastai

  1. Fastai ML

  2. Fastai DL

  3. Fastai NLP

Embeddings

https://github.com/graykode/nlp-roadmap

Word2Vec

  • Jalammar

  • Lilian

  • McCormik — Part 1

  • McCormik — Part 2

GLOVE

Attention

  • Attention by Lilian

Language models

ELMO

  • ELMO by mlexplained

  • ELMO by Edward Ma

  • ELMO by Eric

Transformer

  • Transformer by Ashish Vaswani

  • Transformer by Peter

  • Transformer by Jalammar

  • Transformer by MLexplained

  • Attention mechanism

  • Attention articles by school of AI

  • Understanding transformers

BERT

  • BERT by Jalammar

  • BERT by mlexplained

  • Łukasz Kaiser’s talk

  • McCormick BERT series

GPT

  • GPT-2 by Jalammar

  • Annotated GPT-2

  • BERT Vs GPT-2

XLNet by MLexplained

DistilBERT

ALBERT

Embeddings summary by Lilian Weng

Transfer learning

ULMFiT (Universal Language Model For Fine Tuning)

  • RNN theory

  • Seq2Seq theory

  • Neural machine translation(NMT)

  • AWD-LSTM

  • ULMFiT

Ruder on Multi-task learning

Transfer learning

  • NAACL

  • INRIA

  • Notebook with all transfer learning examples

PyTorch

Get Pro in PyTorch For NLP

Deep Learning (with PyTorch)

NN concepts

  • Improving NNs

  • Why momentum works?

  • Learning-rate-tuning

  • Cyclical Learning Rates

  • Pros and cons of activation functions

Normalisation

  • Why batch normalisation works

  • Weight and layer normalisation

  • Overview of normalisation techniques

4 Sequence Encoding Blocks You Must Know Besides RNN/LSTM

Conversational AI

Neural Approaches review paper

What makes a good conversation?

Denny’s

  • DL for chatbots Part 1

  • DL for chatbots Part 2

Han’s

  • Question answering systems Part 1

  • Question answering systems Part 2

  • Neural Information Retrieval

Rasa

  • Bot levels

  • Level 3 bot

  • 13 rules for chatbot design

  • Rasa NLU in Depth: Part 1 — Intent Classification

  • Rasa NLU in Depth: Part 2 — Entity Recognition

  • Rasa NLU in Depth: Part 3 — Hyperparameter Tuning

Huggingface

  • A Transfer Learning Approach for Neural Network Based Conversational Agents

Putting ML to production

  • Full Stack Deep Learning

  • Rules of ML by Google

  • Continuous delivery for ML

  • Deploying ML models

  • Deploying NLP models

  • ML system design

  • ML Ops Day — Oscon 2019

  • Deploying transformers

  • Technical debt in ML

  • Model management

Airbnb

  • Zipline — Data management

  • Bighead — ML infra

Uber

  • Michelangelo

  • Scaling Michelangelo

  • Big Data Infrastructure

  • Uber engineering

Databricks Mlflow

Extra tips

  • Training NN on GPUs

  • pytorch-lightning

  • Distributed training in pytorch

  • Faster training with large batches

Interview

  • NLP Interview Questions

  • Reverse interview

  • How to build your project portfolio?

Libraries

  • Custom training in Spacy

  • Spacy transformers

  • Spacy crash course

  • Serving Google BERT in Production

Semantic search

On Semantic Search

  • Generic Neural Elastic Search

  • Sentence-transformers

  • Approximate nearest neighbors

  • Sentence-embedding

Random

  • Tricks in NLP

  • Tranformers time benchmarking

  • Text data augmentation

  • Topic modelling by Nanonets

  • How to detect fake text?

  • An Embarrassingly Simple Approach for Transfer Learning

  • Machine Learning Fairness

Hackathons

  • Reflecting back on one year of Kaggle

  • Kaggle ensemble guide

AutoML

  • Automl for predictive modeling

Papers and code

  • Papers on language models

  • Tasks in NLP

  • NLPprogress

  • State of the art NLP papers

This was originally published at Modern NLP.


Come join Maxpool - A Data Science community to discuss real ML problems!

I am also on Medium, Twitter & LinkedIn!!

3
Share
Share this post

Zero To One For NLP

pakodas.substack.com
Previous
Next
Comments
Top
New
Community

No posts

Ready for more?

© 2023 Pratik
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing

Our use of cookies

We use necessary cookies to make our site work. We also set performance and functionality cookies that help us make improvements by measuring traffic on our site. For more detailed information about the cookies we use, please see our privacy policy. ✖