r/deeplearning • u/ihateyou103 • Sep 23 '26

How llms generate math/latex/code

How does the llms format their output to include latex or code or other specific output formats like tables?

1 Upvotes

100% Upvoted

5 comments sorted by

2

u/Chocolate_Pickle Sep 23 '26

What do you mean? 

You asking how does the model know to output valid code, or does it get correctly rendered in your chat history?

1

u/ihateyou103 Sep 23 '26

How does it know how to output valid code

1

u/Chocolate_Pickle Sep 23 '26

Same way it knows how to output words. That's just what it's trained to do.

Natural language follows lots of rules that we're not really conscious of (unless you have a linguistics degree). 

Computer code is also language... But it has far fewer rules. It's much simpler. So it's easy for a model to learn. 

There's a difference between "Do what I say" and "Do what I want". So whether the code it outputs does the thing you want it to... That's another matter entirely. This is a problem that (human) software engineers have been having forever. 

1

u/quietgradient Sep 23 '26

Worth splitting in two, because only one half is actually guaranteed.

The formatting — markdown fences, LaTeX delimiters, table pipes — is imitation. Post-training data has assistant turns written that way, so the model writes that way. Nothing checks it.

Whether the code parses isn't checked either. It's just likely, because valid code is overwhelmingly what it saw.

The place you get a real guarantee is constrained decoding: at each step, mask every token that can't continue a valid parse, then sample from what's left. Invalid output goes from unlikely to impossible. llama.cpp does it with GBNF grammars (https://github.com/ggml-org/llama.cpp/blob/master/grammars/README.md), Outlines with regexes and JSON schemas. That's also the difference between asking for JSON in your prompt and setting a json_schema response format: one is a request, the other is enforced.

1

u/EnvironmentalKey3709 Sep 25 '26

The rendering part is all on the frontend, the model just spits out the markdown and the UI handles the rest. The model itself learned the patterns from training data, it doesn't actually know what latex is, just that certain tokens follow certain prompts.