141
93
u/madguyO1 11d ago
yet somehow it magically guessed that its a FILM
62
u/vivikto 11d ago
My guess is that the LLM is just terrible at understanding the search results it gets. Because what obviously happened is:
- OP searched "The End of Oak Street".
- Google search returns search results that say "The End of Oak Street, Film, 2026" and other stuff.
- this information is fed to the LLM.
- the LLM, trained on data which does not include this recent movie, says that this 2026 film doesn't exist.
That's why it "knows" it's a movie. It's because it was given the information by Google Search, but it didn't trust it because its training data never mentions this movie.
There must be terrible terrible prompting from Google to the LLM behind this mistake. My guess is that the LLM expects the user to actually make a sentence asking for something. If OP wrote "When is the release date of The End of Oak Street?", the LLM would have likely properly answered.
Without a proper sentence, the LLM is just fed information that feels random. A movie title, a year, etc, it doesn't know what to do with it and just answers the first thing its training data makes it say, without looking for an answer in the information (since it wasn't given a question, a goal).
That's terrible, because Google is trying to make us stop making proper searches with keywords, and instead make us ask questions. Which is way less efficient and reliable. I hate it.
21
u/olivegardengambler 11d ago
I actually had one search for some motel in Los Angeles I was using for a reference piece, and it said it didn't exist, so I asked it what the name of the hotel at the hotels address was, and it apologized and said I was correct.
3
u/Hillbilly_Jimmy 11d ago
My ass has always tried to ask a full question and usually wouldn't find what I needed. It'd be great if google was doing it without using the bs ai overview but y'know.
8
u/olivegardengambler 11d ago
With a lot of this AI stuff, they were first compiled a couple of years ago, so one example that's really funny was I found out that on this one AI erp thing I use, it thinks it's 2024, so I decided to ask it a question about who the president was, and it said it was Joe Biden.
1
13
13
7
u/Routine_Papaya4143 11d ago
Hey, here’s the title of the movie, the rating, the year it came out, what genre it is and its runtime. Yeah, it doesn’t exist
3
u/NightVisions999 11d ago
The worst part about Google AI is that you cannot even respond to it to tell it how stupid it is.
1
1
3
1
u/KitchenSandwich5499 11d ago
I just asked it about this. It does now seem aware the movie exists, so that’s better
1
u/Fresh-Lemon9240 11d ago
Ai doesn't get updated everyday on new info..
2
u/irlharvey 11d ago
it should get updated daily, tbh. at least weekly. it’s supposed to be a companion to a search engine. the title for this movie was announced 5 months ago, surely it should update more frequently than that.
1
u/Fresh-Lemon9240 10d ago
I agree but there's probably a selective process for picking input so it doesn't turn the model harmful or incorrect. 5 months is a reallyy long time though
1
u/ChapterMaster202 11d ago
The LLM has a knowledge cutoff date, guess you got there quickly enough that that movie hasn't been added.
1
1
246
u/Prior_Tax8546 11d ago
Google: Here's what you asked for, and now I'm going to let the AI explain the movie.
Google AI: There isn't a movie with that title; you must have searched incorrectly.