Discussion about this post

User's avatar
Mark Daley's avatar

This result could change the power calculus. Roughly, they're "speeding up multiplication" by getting the order of magnitude (exponent) correct and then being... liberal... with the details (mantissa). That it works is only surprising if you're also surprised that neural nets survive quantization so well; the robustness of deep neural nets continues to amaze.

https://arxiv.org/html/2410.00907

Expand full comment
Faizan Abbasi's avatar

Tragedy of the commons or self fulfilling prophecies? 😅

Expand full comment
13 more comments...