New Paper - On the Power of Context-Enhanced Learning in LLMs

See twitter/X thread here.

Kids use open textbooks for homework. Can LLM training benefit from "helpful textbooks" in context with no gradients computed on these tokens?

We call this Context-Enhanced Learning – it can exponentially accelerate training while avoiding verbatim memorization of “textbooks”!… pic.twitter.com/NnxZIBeRNN

— Xingyu Zhu (@XingyuZhu_) March 5, 2025