Structured Outputs with Ollama

ollama.com

265 points · Patrick_Devine · 19 days ago

70 comments

rdescartes · 19 days ago

If anyone needs a more powerful constrain outputs, llama.cpp support gbnf:

https://github.com/ggerganov/llama.cpp/blob/master/grammars/...

Show replies

chirau · 19 days ago

This is wonderful news.

I was actually scratching my head on how to structure a regular prompt to produce csv data without extra nonsense like "Here is your data" and "Please note blah blah" at the beginning and end, so this is much welcome as I can define exactly what I want returned then just push structured output to csv.

Show replies

quaintdev · 19 days ago

Yay! It works. I used gemma2:2b and gave it below text

   You have spent 190 at Fresh Mart. Current balance: 5098

and it gave below output

   {\n\"amount\": 190,\n\"balance\": 5098 ,\"category\": \"Shopping\",\n\"place\":\"Fresh Mart\"\n}

Show replies

guerrilla · 19 days ago

No way. This is amazing and one of the things I actually wanted. I love ollama be because it makes using an LLM feel like using any other UNIX program. It makes LLMs feel like they belong on UNIX.

Question though. Has anyone had luck running it on AMD GPUs? I've heard it's harder but I really want to support the competition when I get cards next year.

Show replies

bluechair · 19 days ago

Has anyone seen how these constraints affect the quality of the output out of the LLM?

In some instances, I'd rather parse Markdown or plain text if it means the quality of the output is higher.

Show replies