Archive - The Counterfactual

Building inductive biases into LLMs

Language models are hungry for data—can we initialize their weights in ways that help reduce how much data they need?

Mar 4 •

February 2025

January 2025

December 2024

The Counterfactual’s 2024 in review

Wrapping up another year.

Dec 29, 2024 •

Deep Dive: Molmo

Exploring the new multimodal model from AI2.

Dec 12, 2024 •

Towards an unambiguous notation for LLMs

Names like "o1-preview" are pretty vague—can we do better?

Dec 2, 2024 •

and

November 2024

October 2024

What if we ran ChatGPT "by hand"?

A thought experiment to probe the boundaries of our intuitions.

Oct 15, 2024 •

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts