NLP 学习相关网站

BERT 

• 📽️ Overview of NLP Tasks

• 📽️ BERT and its family - Introduction and Fine-tune

• 📽️ BERT and its family - ELMo, BERT, GPT, XLNet, MASS, BART, UniLM, ELECTRA, and more

• 📄 BERT 时代与后时代的 NLP - 知乎 (zhihu.com)

GPT-3

• The GPT-3 Architecture, on a Napkin (dugas.ch)

• GPT3 Examples

Bert fine-tune 策略

bert fine-tune 策略

• The top layer of BERT is more useful for text classification;
• With an appropriate layer-wise decreasing learning rate, BERT can overcome the catastrophic for getting problem;
• Within-task and in-domain further pre-training can significantly boost its performance;
• A preceding multi-task fine-tuning is also helpful to the single-task fine-tuning, but its benefit is smaller than further pre-training;
• BERT can improve the task with small-size data.

CTR 预估模型

• 前深度学习时代的CTR预估模型
• 基于深度学习的CTR预估模型