6 Comments
User's avatar
Vova Zakharov's avatar

How many super-weights do you think the human brain has?

Expand full comment
Grigory Sapunov's avatar

Oh, that's the good question!

Expand full comment
Misha Belkin's avatar

Very interesting. I find it quite counter-intuitive.

Expand full comment
Grigory Sapunov's avatar

That was unexpected for me as well.

There are definitely a lot of interesting things hidden in sparsity, loss landscapes, lottery tickets, grokking, double descent, and other (dis?)similar phenomena.

Expand full comment
Kartik Singhal's avatar

Thanks Grigory. Does the paper discuss the reason behind why such super weight is formed at the first place ?

Expand full comment
Grigory Sapunov's avatar

No, waiting for a separate research :)

Expand full comment