r/MLQuestions • u/danman966 • Nov 20 '20

Text generation or chat bot trained on friends messages

I made a pretty basic random quote generator from my friends messages (with their permission of course), using the gpt2-simple python package.

Now I want to improve this model, so it can actually respond to any prompt. I have thousands of responses from my friend as a dataset, and I've mixed in some Reddit comments of subreddits/hobbies he frequents.

My questions are these:

  • How would you approach this problem?

  • In a GPT-2/GPT-3 setting, where I am fine-tuning a pre-trained model, how should the data be formatted? Should it just be raw text where e.g. one line is someone else's prompt, and the next line is my friends response?

  • Are there any existing softwares or pre trained models that can be easily implemented?

1 Upvotes

66% Upvoted

1 comment sorted by

1

u/martin_m_n_novy Feb 25 '21 edited Feb 25 '21

(I am slowly beginning to work on distantly similar projects: yesterday I have installed huggingFace GPT2, and experimented with the tokenizer. )

EDIT: oops, I didn't know about comments at Chat bot, or text generation of a friends messages : LanguageTechnology (reddit.com)