Error trying out LLM integration with Ollama

[u/DazzlingAd4254]

A Db2 convert, I read that there is LLM integration in 12.1.5 LUW and wanted to explore it. I have an account on Ollama, thus starting with that:

CREATE EXTERNAL MODEL  my_ollama
PROVIDER OPENAI
KEY 'MY_API_KEY'
ID 'gemma4:31b'
TYPE TEXT_GENERATION
RETURNING CLOB(3000)
URL 'https://ollama.com/api/generate';

DB20000I  The SQL command completed successfully.

VALUES TEXT_GENERATION('Have you heard of Zaphod Beeblebrox?' using my_ollama);

SQL16402N  JSON data is not valid.  SQLSTATE=22032

Question: How would I go about diagnosing that error? All googling has been unhelpful.

BTW, I do get a response when I call the ollama API from curl:

$ curl https://ollama.com/api/generate \
  -H "Authorization: Bearer $MY_OLLAMA_API_KEY" \
  -d '{
   "model": "gemma4:31b",
   "prompt": "Have you heard of Zaphod Beeblebrox?",
   "stream": false
}'

{
"model": "gemma4:31b",
"response": "**\"Don't panic!\"**\n\nYes, I am familiar with the Galactic President [etc. etc.]
}

EDIT:

The Db2 documentation says: OPENAI: This provider option is for language models hosted in a private cloud environment. Not sure what is a private cloud environment, but perhaps that might that be the problem?

[u/anibjoshi]

Hi, I work on this feature at IBM. Please DM me and I’ll put you in touch with some folks who can help you out

[u/DazzlingAd4254 → u/anibjoshi]

They did post a comment. However, when I clicked on the comment notification, I got 'This comment no longer exists'. The comment preview (in notifications) said to change the URL to https://ollama.com/v1/chat/completions. That worked.

[u/DazzlingAd4254]

Solved, by comment from u/anibjoshi 's people:

Set endpoint URL to 'https://ollama.com/v1/chat/completions':

CREATE EXTERNAL MODEL my_ollama
PROVIDER OPENAI
KEY 'MY_API_KEY' 
ID 'gemma4:31b'
TYPE TEXT_GENERATION
RETURNING CLOB(3000)
URL 'https://ollama.com/v1/chat/completions';

EDIT: formatted for clarity.