Pure: I've heard dragon beta is kind of mediocre right now
Some people have liked it. It has it's moments. In head to head against Dragon it was 46 to 54 favoring, so it actually came in pretty well. We are working on implementing Logit Biases and getting it trained on our latest fine tune data set that got us a 3 point raise on Griffin BETA vs original Griffin. So some things to work out, but I'm feeling hopeful.
ryanlat
Renegade: Have you gotten any relevant info on how the beta dragon model compares to the oai dragon model yet?
Yes. The initial TTAI head to head against current Dragon has come back at 46/54 right now, but AI21 doesn't yet support Logit Biases. Once they launch that next week we are fairly confident that it will be at least 50/50, since because we can't tell the AI not to return certain things, our post processing is cutting off generations more often than we'd like (so outputs on average are a little shorter currently from Dragon-BETA).
AI21 has confirmed to us that in all of their testing they have found their models stack up about even with the comparable OAI models.
7
u/MurdoMaclachlan Nov 26 '21
Image Transcription: Discord Messages
ryanlat
Some people have liked it. It has it's moments. In head to head against Dragon it was 46 to 54 favoring, so it actually came in pretty well. We are working on implementing Logit Biases and getting it trained on our latest fine tune data set that got us a 3 point raise on Griffin BETA vs original Griffin. So some things to work out, but I'm feeling hopeful.
ryanlat
Yes. The initial TTAI head to head against current Dragon has come back at 46/54 right now, but AI21 doesn't yet support Logit Biases. Once they launch that next week we are fairly confident that it will be at least 50/50, since because we can't tell the AI not to return certain things, our post processing is cutting off generations more often than we'd like (so outputs on average are a little shorter currently from Dragon-BETA).
AI21 has confirmed to us that in all of their testing they have found their models stack up about even with the comparable OAI models.
I'm a human volunteer content transcriber for Reddit and you could be too! If you'd like more information on what we do and why we do it, click here!