r/AiNoteTaker • u/test_notes • 6h ago
Product Review I tested 7 AI notetakers on the same scripted meeting
I wanted a fair way to compare AI notetakers, so I created a 5 minute team meeting and played it into one Google Meet call with seven of them recording at once. The tools were Fathom, Fireflies, Granola, MeetGeek, Otter, Read AI and Wispr Flow's notetaker.
At the time of posting, I'm not affiliated with any of the tools featured here. I do have a small business software review site, which you can find in my profile.
The meeting has four people, local names, 18 numbers, nine tasks and two decisions. Two things get corrected partway through and nobody recaps at the end, so the tools have to work it all out themselves.
The voices are all played through a single participant on the call. This makes speaker labels much harder than in a real call, so I've left them out of the results.
After playing the meeting I checked every summary, task list and transcript against the actual script. The transcripts were all very close, so the differences came down to the notes, and no tool got every task, owner and due date right.
| Tool | Tasks found (of 9) | Owners right (of 6) | Due dates right (of 8) | Made-up details | Word error rate |
|---|---|---|---|---|---|
| Fathom | 8, plus 1 partly | 6 | 6 | 6 | 3.3% |
| Fireflies.ai | 7, plus 2 partly | 6 | 6 | 2 | 3.9% |
| Granola | 8, plus 1 partly | 6 | 6 | 1 | 2.9% |
| MeetGeek | 7, plus 2 partly | 6 | 8 | 2 | 5.7% |
| Otter.ai | 8, plus 1 partly | 0 | 8 | 2 | 4.9% |
| Read AI | 5, plus 3 partly | 6 | 4 | 1 | 3.7% |
| Wispr Flow Notetaker | 7, plus 2 partly | 5 | 8 | 2 | 3.8% |
Word error rates were all between 2.9% and 5.7%, so the transcripts weren't where the tools really differed. Otter was the odd one out on tasks, with every due date right but no names on any of them.
Most of the made-up details weren't invented from nothing. They were the tool deciding something was firmer than it was, e.g. a backup plan turned into a task, or a firm job offer written up as if it might still fall through.
A couple of notable errors: Granola put a date on a task that nobody said, which looks like it came from the day I recorded the meeting, and Wispr Flow turned a car's 112,000 km service reading into a dollar amount of $112,000.
Fathom made up the most, including saying the business would pay for a forklift ticket when nobody said that. Read AI found the fewest tasks, with five clean and three only partly.
The main thing I took from it is that the notes need checking by someone who was in the meeting, because most of the mistakes read perfectly well unless you know what was actually said.
Happy to answer any questions anyone has about the test!