byteCoderTurns out, LLMs represent numbers on a helix and use trigonometry to do addition. A new paper rever...
Shared by Lior Alexander @LiorOnAI on X
Turns out, LLMs represent numbers on a helix and use trigonometry to do addition. A new paper reverse engineers addition in models like GPT-J-6B and finds a “Clock” algorithm. Numbers are encoded using sine and cosine terms, then added like angles. https://t.co/Ru4jkYNddl
Turns out, LLMs represent numbers on a helix and use trigonometry to do addition. A new paper reverse engineers addition in models like GPT-J-6B and finds a “Clock” algorithm. Numbers are encoded using sine and cosine terms, then added like angles. https://t.co/Ru4jkYNddl