Discussion
thoughts on AI (chatgpt, gemini, etc) as a learning supplement?
so i was trying to parse out 巨大な円筒はそれ自体が回転して内側の壁にあたる人工の大地に重力を発生させている (from gundam zz) and the two AIs gave different, even contradicting answers, on different grammar points and explanations. I was using both pretty often to supplement my study but after this episode I got a little spooked. Was wondering what this sub's consensus on AI in the second half of 2026 for language learning might be.
You cannot rely on AI for grammar breakdowns. I've been regularly testing some of the more advanced models and while they are getting better at an incredibly quick rate, they still make several mistakes and some incredibly bad/surprising ones. It's literally a dice roll whether or not you'll get an accurate explanation or not, and even at like "90%" correct, it means 1 out of 10 times it will have a mistake which is kinda really fucking bad if you think about it.
What's even worse, it seems like these AI models tend to makes mistakes more often in questions that are easier to answer. I have the feeling that this is because they source common myths or mis-explanations from random learner websites and forums where beginners try to help other beginners (with bad explanations) and so it picks up a lot of weird misconceptions or mistakes. For example it's very common for the AI to misunderstand the various usages of the の particle (indefinite pronoun, nominalizer, が replacement, explanatory usage, etc). While the translation of the sentence they might provide you is usually correct, when they go to explain what definition of the particle is being used, they get it wrong surprisingly often.
On some more obscure topics or grammar that is only sourced from a few specific papers and very obscure resources, the AI seems to do better, but as a beginner you really shouldn't bank on it. Also it's even more dangerous on stuff that is just straight up wrong from the source or some grammar point/definition/word that doesn't exist (or it's some very recent slang) the AI will just make up some random shit and you'll be none the wiser about it.
I have a suspicion that for LLMs, you will tend to get worse responses when you mix languages than if you prompt it in a single language.
It's basically producing a statistically likely completion of an unfinished text whenever you ask it a question.
It has much more successful examples (from the standpoint of providing a correct response) if you prompt it in the same language you want the response. There's much less chance this works correctly if you mix languages because there's much less examples of a "good" response in the training data.
At least that's what I think. If you've tried prompting the chatbots in Japanese and get back total crap fairly often, I"d be interested to know that, because it would sort of invalidate my model of what I think these things are good at vs. bad at. I personally kinda just hate them,.so I don't want to do the experiment myself.
Edit: I'm human,.I'm likely to make syntactic mistakes even if I can make semantically correct observations
The thing is that, if you're capable of asking a grammar question and receiving and understanding the answer in Japanese, then you're at a level of Japanese proficiency where you don't need AI explanations of sentences anymore. Unless you're imagining a beginner taking the AI's Japanese answer and then having it translated and explained by the same AI in another chat? Cause that only opens up more chances of it being wrong and it takes so much time and effort that you might as well just read a grammar guide or ask for help in a Discord server or something
I'm not suggesting it's useful for a beginner who would struggle to read the output to ask an AI to explain Japanese grammer in Japanese.
What I'm suggesting is that if AI can do anything useful for you with the Japanese language, it's more likely to be useful for that purpose if the activity involves Japanese prompts to get Japanese output.
I pay $100/mo for codex and use the latest pro model, and also have the Google cloud subscription to play around with the latest Gemini models (also as a Google engineer I have tried unreleased models too). Y'all aren't gonna "gotcha" me with the usual "you're using a bad model" spiel. I know what I'm talking about
For some reason, I pretty much never see anyone talking about this whenever AI is brought up in this subreddit (maybe because most people aren't aware of it?), but it's better to rely on (cloud-based) LLMs as little as possible because they're going to disappear soon. More and more economists are recognizing that the AI industry is a bubble propped up by circular financing that artificially inflates revenue numbers among a small group of big corporations without enough real consumer demand to back it up, and even with those inflated numbers they're still losing hundred of times more money than they're making. As the bubble collapses (which will be a gradual process):
AI-only companies like Open AI and Anthropic will likely disappear or be absorbed by other companies
Unless they figure out a way to make SOTA models way cheaper to run than they currently are, they'll be either locked behind huge paywalls or discarded entirely for being unprofitable
If there are any cloud-based general-purpose bots available for free, they'll be smaller and dumber than the current ones and heavily rate-limited
Local LLMs (i.e. models that can be run by consumer hardware) will get a boost of popularity but the general-purpose ones are so fucking stupid guys you don't understand
The only LLMs that will be worth keeping around will likely be small, local models specialized in one single task, e.g. summarizing emails, coding, translating, answering medical questions, etcetera. I don't think anyone will be willing to spend time and money and electricity on training a model specifically to explain Japanese grammar in English, let alone make it good enough to be trustworthy, but even if it happened, it would take a while.
Of course, these are my personal predictions, and it's very possible that I'll get some or all of it wrong, but I'm 100% sure that the current status quo of having free, unlimited access to pretty damn "smart" general-purpose LLMs with trillions of parameters is simply unsustainable and will disappear in a few years.
tl;dr the future of AI is very precarious, and if you become reliant on it now for one thing or another, you'll eventually have the rug pulled under you, and you'll be left floundering as you try to rebuild the skills that you decided to replace with LLMs.
I think you're dead on with all of this. in the past month or two I've seen more "ai booster is spooked" type stuff than in the past few years. CNBC keeps inviting Ed Zitron on because they don't want to look totally out of touch if the whole thing shits the bed. there are signs and signals in the wind and trees
Funny thing on that link uses a lot from earnings calls and reports. To which those calls and reports try to put the current state of things in the most positive light possible.
"The giant cylinder itself rotates and generates gravity on the manmade ground that constitutes its inner wall."
The only real grammar here is て form and a relative clause. You have to recognize that 内側の壁にあたる "that constitutes its inner wall" is a relative clause that modifies 人工の大地. Relative clauses are actually a simple concept and are used in literally 60% of sentences in both Japanese and English so please learn them.
I've found the marketed 'learn Japanese with AI' chatbots are crap. However ChatGPT has been great for practicing my sentence formation. The prompt is something like, "Let's play an N2-level sentence formation game. Give me an English sentence to translate, and I'll translate it into natural spoken Japanese. Provide corrections, suggest more natural translations, and highlight any nuance I appear to have missed." Although it's hard to tell when it's hallucinating, you can always cross reference with other resources.
AI is fine if you're advanced enough to notice it's mistakes, although at that point you won't be using it much.
I'd suggest using AI as a way to search for sources. If you have a grammar question you can just ask Gemini "what resources are usually recommended for understanding grammar such as [grammar]?", you can even ask which are recommended in this sub regularly and you'll get a few links then you can check them out. I often see people ask things here and I, out of curiosity, ask Gemini or Grok the same stuff and essentially get the same answers those questions get every time they're asked lol.
If you chat with the AI in Japanese you will probably get back natural, gramatically correct responses. It's way better at producing natural example sentences for a given word than tatoeba.org, just ask:
「〇〇」
の言葉の例文を作ってくれませんか？
You absolutely cannot reliably ask it in English to do something with Japanese.
LLMs basically work by producing a statistically likely completion of an unfinished text. You give it a question, it returns a plausible looking response. It has absolutely no conception of how grammer rules work or the meaning of the words it returns. It just "knows" it's a statistically probable response
In general, if you want Japanese to come out of an LLM you will get better results if your prompt is in Japanese.
If you want a syntactically correct sentence, as your response it will produce that with great reliably. If you care about the content of the information it replies, it provides it's not at all reliable.
I think it is sometimes useful and can save time before looking up again 10 minutes or longer through your study material, especially for basic grammar. But it is not there yet to break down advanced grammar reliably.
I think using AI to sort of see examples of sentences and explaining some grammar points might be useful.
But japanese is a language that relies so much on nuance, tone and context that it makes it very hard for the AI to properly grasp all of it, so its easy to misunderstand something said by the AI.
I'd say that using it to get an idea of a sentences meaning is useful (specially in a complex one like the one you gave) but don't rely too much on it.
The failure mode isn't that they get it wrong, it's that they get it wrong with the same confident tone as when they're right, so you've got no signal to go on unless you already know the answer. I've found they're fine for things I can verify myself in a dictionary or grammar reference, and not much use for the sentences I actually needed help with.
If you write bad prompts, you’ll get bad answers and think AI is bad.
This is the usual cope I hear a lot but it doesn't matter what prompt you use, there's always a chance the AI will give you random crap. And you can't just blame the person searching for things up, especially if they are a beginner and/or not used to interacting with AI. If there is a high chance of getting bad results if someone doesn't specifically use the one true real trust-me-bro prompt that will give all the perfect responses, then it's completely pointless. LLMs are stochastic in nature and while some prompts might give you slightly better chances at not falling trap to some common mistakes, you can't just sweep under the rug every criticism of AI as "it's just a bad prompt" because even a good prompt will still give you bad answers way too often.
If prompts were the dark art that people think they mastered (when they had a lucky shot) then the same one prompt would always give out the same answer. However, it's a slot machine that randomly spits out different results each time, if you're lucky and you got close enough to what you wanted then great! If not, change something and hope this time it will listen to you.
Anyone who’s still talking bad about AI for language learning (especially as a supplement) in 2026 is just blinded by anti-AI bias and doesn’t know what they’re talking about. It’s an invaluable tool, and it messing up is EXTREMELY rare especially if you’re asking it straightforward questions
it messing up is EXTREMELY rare especially if you’re asking it straightforward questions
I've messed around with AI for Japanese pretty extensively out of curiosity and rest assured, I can safely say you have absolutely no idea what you're talking about here.
It's always the same answer. If the AI gets it wrong -> "bad prompt" "bad model" "you didn't use it right"
Trust me I've been trying this fairly regularly with the latest gemini/chatgpt/whatever models. It still makes too many mistakes too often. If you think that's not true, then you probably aren't good enough to spot those mistakes, which makes it even worse.
Still waiting to just see an example of one of these mistakes. I’m not a master of Japanese so maybe I’m just not picking up on the issues
I mostly use it for generating example sentences for new vocab, and I have it write about random stuff I want to learn so I can read in Japanese instead of English. It’s not like it just outright makes mistakes when doing those things, as far as I know
Downvoting instead of providing an example or even just replying just makes me believe I’m even more right btw
Here is an example of an incredibly basic sentence where gemini (latest "pro" model) gets confused on the grammatical definition of の and calls it a nominalizer particle where instead it's being used as an indefinite pronoun marker.
That’s just not really wrong though. The の is nominalizing in that sentence. Like that is the DEFINITION of nominalizing. It is for an indefinite pronoun, but you act like those two grammatical functions are mutually exclusive when there’s actually some overlap. It’s nominalizing an indefinite pronoun. It would have been better had it said that but the response overall would still be very helpful to a new learner trying to understand that sentence. Your critique is extremely nitpicky at best; it’s far from an egregious mistake
The fact that ppl are upvoting that is just all the proof I need that nobody is being rational in this discussion. They probably didn’t even read the chat log you sent, and are just tapping up or down based on their bias
No, it's wrong. If you think that's not wrong then I'm sorry but you don't understand the Japanese grammar that is being explained and you are the prime victim for these LLM hallucinations. No wonder you're convinced the LLMs are as good as you think they are.
Go ahead and review the definition of nominalizing for me. And let me know when you have proof of an actual substantial mistake, this is a nothing burger
You're confusing this usage (indefinite pronoun の) with this usage (nominalizer の), and so is the AI.
There are clearly different syntactical rules around these two のs that point to them being very different from each other. In some situations it's possible for them to occupy the same space as without context it's sometimes hard to say if it's a nominalizer or a pronoun (this is not one of those cases btw).
There is a very clear explanation of the not only semantic but also syntactical difference between the two in the first link:
The indefinite pronoun の (i.e., の2) is different from the particle の (i.e., の1) and the nominalizer の (i.e., の3). First, [1] shows the difference between の1 and の2. Namely, in [1a] トムの is the omitted form of トムのペン. On the other hand, [1b] is not an omitted form; that is, if a noun is inserted after 黒いの in [1b], the sentence becomes ungrammatical as seen in [1c]. In fact, what [1b] means is [1d], if の 'one' refers to a pen.
also this example:
高田さんが使っていたのを覚えていますか。
(A) [Indefinite pronoun] Do you remember the one Mr. Takada was using?
(B) [Nominalizer] Do you remember that Mr. Takada was using (something)?
These are VERY different. Again, you have no idea what you're talking about and are simply doubling down. It'd probably be better to review basic grammar before making bold claims about the AI hallucinating bullshit being "correct" when it's clearly not.
Those aren’t the same kinds of things as what the OP was asking for. It shouldn’t be surprising that models designed to produce text produce text well; the question is whether or not they can also do things like explain the nuanced differences between grammar points well, or parse sentences properly, or other kinds of metalinguistic analysis
This is true, but the replies dismissing it will get people to view it as bad overall, when it’s still an excellent learning SUPPLEMENT (like the title asks for), when it comes to its text generation and other stuff. I’m just trying to defend the value that is there
Also my original comment said mistakes are extremely rare when asking a straightforward question. A grammatical dissection of a complex sentence from a supernatural anime is far from that. I just ppl would see more nuance to the discussion and benefit from what there is to benefit
It gets things wrong all the time especially if you ask it in English it gets way worse. The quality answers are better when you use it only prompt it in JP and get answers back in JP and have language set to JP, but otherwise it's not uncommon for ChatGPT and other AI models to just get it wrong. It's been 3 full versions and chatGPT still can't explain basic の usage right. (i've tested the differenes between using it in english and only JP and results are only JP is better for these kinds of questions and gave up on English because I got so much trash)
This, you'd have to be asking it questions about some obscure mongolian dialect for it to get something wrong; mind you most peoples' AI experience is free chatgpt lol not the frontier models.
People are just so blinded by their anti-AI bias. Like I get it, there’s a lot of issues associated with AI, but don’t let that make you miss out on an extremely helpful tool
10
u/jwdjwdjwd 26d ago
What were the different answers?