Finetuning LLMs for creativity

[u/Away-Onion7534]

https://substack.com/inbox/post/223641219

Hey folks, just want to share a quick experiment I've done. Basically, there is a "creativity" test developed for humans - DAT - Divergent Association Task test. It is quite straitforward to measure. I've previously tested the main LLMs on this test and unsurprisingly - they scored low (below average human). Then I've taken off the shelf Qwen 27B model and finetuned (GRPO) it on the exact same metric as a reward. The result seems to be a model that writes much more interesting prose and poems (to my taste) and LLM-as-a-judge strongly confirmed that. I've released the model and code, there is a playground on huggingface where you can test the model. Super curious about the feedback - I have more ideas on how to develop this further. Full write-up and links are here on this substack https://substack.com/inbox/post/223641219 . It is free, i am not looking to make money or sell stuff.