The Counterfactual
Subscribe
Sign in
Home
Archive
About
Latest
Top
Discussions
We should be cautious about LLM-generated benchmarks
Synthetic data has its place, but stimuli are at the heart of how we probe and assess LLMs (and humans)—and that calls for special care.
Jul 1
•
Sean Trott
2
Share this post
The Counterfactual
We should be cautious about LLM-generated benchmarks
Copy link
Facebook
Email
Notes
More
June 2025
How I use (and don't use) ChatGPT
An update on when and how I use LLM-equipped software tools, and when (and why) I don't.
Jun 24
•
Sean Trott
6
Share this post
The Counterfactual
How I use (and don't use) ChatGPT
Copy link
Facebook
Email
Notes
More
6
Why isn't language more iconic?
If non-arbitrariness helps with word leaning, why are languages still mostly arbitrary?
Jun 5
•
Sean Trott
3
Share this post
The Counterfactual
Why isn't language more iconic?
Copy link
Facebook
Email
Notes
More
May 2025
When words sound (or look) like what they mean
Not all form-meaning mappings are arbitrary.
May 14
•
Sean Trott
7
Share this post
The Counterfactual
When words sound (or look) like what they mean
Copy link
Facebook
Email
Notes
More
2
Informed consent is central to research ethics
On the unauthorized experiment conducted on a subreddit community.
May 8
•
Sean Trott
3
Share this post
The Counterfactual
Informed consent is central to research ethics
Copy link
Facebook
Email
Notes
More
April 2025
Identifying signatures of LLM-generated text
What metrics from open-source LLMs can tell us about the differences between human essays and those written by ChatGPT-3 and ChatGPT-4.
Apr 18
•
Sean Trott
6
Share this post
The Counterfactual
Identifying signatures of LLM-generated text
Copy link
Facebook
Email
Notes
More
March 2025
LLM-ology and the "moving target" problem
How can a science of LLMs keep up with technological development?
Mar 27
•
Sean Trott
11
Share this post
The Counterfactual
LLM-ology and the "moving target" problem
Copy link
Facebook
Email
Notes
More
An overlooked problem with LLM benchmarks
Proxies for a capability may not map linearly onto the capability itself.
Mar 12
•
Sean Trott
5
Share this post
The Counterfactual
An overlooked problem with LLM benchmarks
Copy link
Facebook
Email
Notes
More
2
Building inductive biases into LLMs
Language models are hungry for data—can we initialize their weights in ways that help reduce how much data they need?
Mar 4
•
Sean Trott
9
Share this post
The Counterfactual
Building inductive biases into LLMs
Copy link
Facebook
Email
Notes
More
2
February 2025
Learning across timescales
Knowledge distillation happens in many forms.
Feb 24
•
Sean Trott
3
Share this post
The Counterfactual
Learning across timescales
Copy link
Facebook
Email
Notes
More
2
On reading the "classics"
The value of academic archaeology
Feb 4
•
Sean Trott
12
Share this post
The Counterfactual
On reading the "classics"
Copy link
Facebook
Email
Notes
More
2
January 2025
Did cooking make us human?
A review of Richard Wrangham's "Catching Fire: How Cooking Made Us Human".
Jan 18
•
Sean Trott
4
Share this post
The Counterfactual
Did cooking make us human?
Copy link
Facebook
Email
Notes
More
4
Share
Copy link
Facebook
Email
Notes
More
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts