Discussion about this post

User's avatar
Russ Abbott's avatar

Very nice article.

I recently had a discussion with Claude about its intelligence. You may find it interesting. (https://russabbott.substack.com/p/in-which-i-convince-an-llm-that-it)

Expand full comment
Nuclear's avatar

h = x + self.attention.forward(

self.attention_norm(x), start_pos, freqs_cis, mask

)

out = h + self.feed_forward.forward(self.ffn_norm(h))

This is the transformer code of llama. I really don’t understand which part of this code has anything to do with neurons.

If you can help me answer it, I would be very grateful.

Expand full comment
3 more comments...

No posts