GPT

GPT was first introduced in the Improving Language Understanding by Generative Pre-Training (2018) paper. GPT’s framework consists of two stages: GPT Pre-Training and GPT Fine-Tuning.
One year later a GPT-2 was introduced in the Language Models are Unsupervised Multitask Learners.

## Key Contributions of GPT - **Unsupervised Pre-training**: The model was trained on a large corpus of text using a **language modeling objective**. Essentially, the task was to predict the next word in a sequence, which allowed the model to learn useful language representations without requiring labeled data. - **Supervised Fine-tuning**: After the pre-training phase, the model was fine-tuned on specific downstream tasks (like question-answering, sentiment analysis) with labeled data, and this process showed significant improvements in task-specific performance.