Skip to content
System One

Writing one character at a time (X)

A video demo building a small language model out of a classifier: Jev answers 29 yes/no questions per character, the highest-probability answer is appended, and the updated text is fed back in for the next character.

Still from Writing one character at a time (X)Play the video in the post below
Still from x.com

Post on X

Views
167,181
Likes
887
Reposts
18

Read from X on . Counts change daily.

Reported by the author

Questions/char
2929 yes/no questions per characterx.com

Vogel built what they call an LLM from first principles, made of nothing but yes/no questions to Jev. For each character, the system asks Jev 29 yes/no questions, one per candidate: should the next key be a letter from a to z, a space, a comma, or a period?

The candidate with the highest probability gets appended to the text. The updated text is then fed back in and the 29 questions run again for the next character. Vogel calls it an autoregressive loop made out of a classifier. Autoregressive means each new character depends on everything generated before it, the way a language model writes one token at a time.

Open on X

PrimitivesNoul

More like this

Keep browsing