ModernBERT

ModernBERT, a family of state-of-the-art encoder-only models representing improvements over older generation encoders across the board, with a 8192 sequence length, better downstream performance and much faster processing.
ModernBERT is available as a slot-in replacement for any BERT-like models, with both a base (149M params) and large (395M params) model size.

• Distiling DeepSeek reasoning to ModernBERT classifiers - https://danielvanstrien.xyz/posts/2025/deepseek/distil-deepseek-modernbert.html