Thought provoking post. Somewhat relatedly, and given the recent drama at OpenAI, I've been trying to call people's attention to some of the AI safety papers authored by Helen Toner, who sits on the OpenAI board (at least as of this writing) and also leads CSET at Georgetown. In the paper I'll link to below, she hits on the challenges of specification with AI models, the potential for them to diverge from intended behavior increases with the complexity of the environment they are deployed within. There are no easy solutions but there are some techniques outlined in the paper you may find interesting.
Great pointer, thanks! I think this is especially relevant as so many applications built on GPT revolve around some form of prompt specification. Which is, as others have pointed out, not too unlike an incantation.
Thought provoking post. Somewhat relatedly, and given the recent drama at OpenAI, I've been trying to call people's attention to some of the AI safety papers authored by Helen Toner, who sits on the OpenAI board (at least as of this writing) and also leads CSET at Georgetown. In the paper I'll link to below, she hits on the challenges of specification with AI models, the potential for them to diverge from intended behavior increases with the complexity of the environment they are deployed within. There are no easy solutions but there are some techniques outlined in the paper you may find interesting.
https://cset.georgetown.edu/wp-content/uploads/Key-Concepts-in-AI-Safety-Specification-in-Machine-Learning.pdf
Great pointer, thanks! I think this is especially relevant as so many applications built on GPT revolve around some form of prompt specification. Which is, as others have pointed out, not too unlike an incantation.
Better than incanting "feel the AGI!" as apparently was being incanted by some at a certain AI org that is currently imploding.