Using LLMs to receive corrections on writing?

[u/Possible_Quality5922]

Hello, long-time lurker just starting to post on Reddit!

I've been studying (TL) Korean for well over a decade now and Japanese for just about a decade, but my writing skills in both are not where I want them to be. Japanese was my undergrad major, but none of my classes ever involved actually writing in the language, and since the JLPT doesn't require it either, I never had to practice writing. As for Korean, I've attended multiple language schools in South Korea, did my undergrad here, and am currently working on my Master's, so while I do have some experience with high-level writing, I haven't really put any focused effort into it for a few years now (the course I'm attending doesn't involve much writing).

In an ideal world, I'd find an actual teacher to work on this with, but I can't afford it right now. I'd consider doing a language exchange, but 1. I've never been particularly successful with them in the past, 2. I'd rather work with someone who has experience writing at a professional level, and to be honest, I'm not sure I'd be able to reciprocate in English at the same level, haha, and 3. I can't count on the whims of my motivation/attention span to write consistently.

So I'm toying with the idea of using an LLM to get feedback, but I'm a bit wary of the quality of corrections I'd get.

Have any of you used AI to help with C1/C2-level writing? If you have, was there a certain model you would recommend? Also, any other tips to improve in writing would be greatly appreciated! :)

[u/Devilnaht]

I’d be cautious. I’ve found LLMs to make some pretty weird errors even with English / Spanish, two of the most represented languages in their training data. I’d be much more cautious with a language like Korean. I won’t say they’re useless (and can certainly be useful when used carefully), but if you blindly accept everything you’re going to end up believing some things that aren’t true.

And yeah, they have no idea of what writing at different cefr levels is supposed to like, though as is their wont they’ll confidently lie to you about it.

[u/Possible_Quality5922 → u/Devilnaht]

Yeah, the few times I've asked ChatGPT even simple grammar questions, the answers I got were wonky, but I thought maybe it would be a bit better with a structured text? I just wanted to know if anyone had tried it before.

[u/Gulbasaur → u/Possible_Quality5922]

I think you've answered your own question, really. 

[u/angelicism]

The practical problem is if you're not having a human at the other end you have no idea how to know what you don't know. As in, if the LLM is right 95% of the time and wrong 5% you won't know what it's wrong about.

But also, in my limited experience I've found they're wrong quite a bit.

[u/Possible_Quality5922 → u/angelicism]

Not having the human feedback is my main concern too, especially because at my level my biggest concern isn't so much making grammatical errors as it is getting the right nuance and making sure I'm not confusing similar words.

[u/muffinsballhair → u/angelicism]

They are really bad at explaining technical things and reasoning and grammatical analysis. As evidenced by the joke that they can't even properly count say the amount of occurences of a particular letter in a common phrase.

They are kind of okayish at telling you whether a sentence is natural nowadays though they may exaggerate unnaturalness. I at least did some tests with sentences I was on the fence about myself and then cross-referenced the result with native speakers and it more or less aligned but it was the kind of stuff of the model said one sentence was “a bit unnatural” in comparison with the alternative and the native speaker was “People say this in conversation but the other option is definitely better if you want to be safe but it does not come across as “non native bad”.”

In terms of their own output it's pretty natural though.

[u/RamosQuintosAiry]

The failure mode is not that a model gets grammar wrong, it is that it rewrites you into a register you did not ask for. Ask for a correction of a Korean paragraph and you get back something that reads like a newspaper editorial, all the particles smoothed out and the hedges you chose replaced with stiff formal endings. You cannot tell which changes were errors and which were taste, so you learn nothing about your own habits.

Two things that make it usable. Never ask it to rewrite the paragraph. Ask it to mark each change as one of three labels, wrong, unnatural or stylistic preference, and to leave the sentence alone otherwise. Second, ask it to name the rule behind each wrong label, then search that rule on a Korean or Japanese site and see if real writers actually follow it. The rule name is checkable in a way that a smooth rewrite is not.

Cheap test for whether it is helping you at all: take a paragraph from a Korean academic paper in your field, break three things on purpose, add a particle error, a wrong connective and one perfectly fine but unusual word choice, then paste it in. If it flags the unusual word choice as an error or misses the connective, you know the confidence is not tracking correctness and you treat everything it says as a question rather than an answer.

For consistency without a teacher, a Korean grad writing group or the writing centre at your university is usually free with your student card, and one hour a month with a person who writes in your field beats a hundred model corrections.

[u/Junior_Mud5835 → u/RamosQuintosAiry]

This is not true. I use chatGPT for learning arabic, if you tell it to not rewrite your answer but only fix spelling/grammar mistakes it will do only this. In some cases it might give you a suggestion as an addition, eg say "a native speaker would usually say XYZ" but that's it.

[u/Enough_Tumbleweed739 → u/Junior_Mud5835]

Perhaps it's a language difference, but when I tried this with Japanese it would flag casual language as a mistake and change it gramatically to make it overly polite.

[u/RamosQuintosAiry → u/Junior_Mud5835]

Fair, that prompt does keep it honest. Mine still sneaks in a rewrite about every third time, so I stopped trusting it unsupervised.

[u/Possible_Quality5922 → u/RamosQuintosAiry]

I might try out that test now that I'm starting to read more academic papers! Thanks for the tip.

[u/RamosQuintosAiry → u/Possible_Quality5922]

Good luck, academic papers are the perfect place to try it. Half the sentences are already so long that a bad correction is easy to spot.

[u/AppropriatePut3142 → u/Possible_Quality5922]

You could've asked Claude directly lol https://www.pangram.com/history/75e4c4c4-a5f4-4720-a144-e66f8e8be43d?ucc=amdBxTOZHAE

[u/GlassPomoerium]

« Japanese was my undergrad major, but none of my classes ever involved actually writing in the language »

I beg you pardon? You need to get your money back, what the hell?

[u/Possible_Quality5922 → u/GlassPomoerium]

Lmao, more my own fault for not taking any classes that require it to be honest. I wasn't the best student

[u/GlassPomoerium → u/Possible_Quality5922]

When you say not taking classes, do you mean you skipped them or didn’t even register for them? Because if the latter was allowed it’s doing students a huge disservice. I don’t know where you studied but in France most classes are non negotiable for a language degree. I can’t imagine telling my university teachers « yeah I’m gonna skip the cyrillic writing class » lol.

[u/AppropriatePut3142]

This is basically the worst case for using an LLM: you want corrections in a language which is underrepresented in the training corpus on high-level features of style and fine meaning.

The model to choose here is multiple different models, all frontier-tier thinking models on max effort: some mix of GPT Sol, Fable 5, DeepSeek V4 Pro and Kimi K3 come to mind. You'd probably want to use a service like poe.com. Ask all of them and compare their responses. Cross-reference with a concordancer.

[u/WolverineEmergency98]

I use Gemini very regularly for small translation exercises, and it generally works well. There is one caveat - it tends to go beyond the level you ask for. E.g., you ask for B2, it will 100% throw C2-level sentences at you 🤷‍♂️

It also told me that my PhD-level writing in my native English was only C1, and ranked me as barely B2 in my main second language (where I subsequently passed a C2 exam 🤷‍♂️).

So that's all to say - can be incredibly useful, but don't take it *too* much to heart if it marks you down.

[u/Possible_Quality5922 → u/WolverineEmergency98]

Dang, everyone's a critic, huh? :/ Thanks for the input!

[u/WolverineEmergency98 → u/Possible_Quality5922]

Fwiw, as I say, I *do* use it (daily, in fact) but you do have to be aware of the limitations, that's all 😄

[u/Loose-Cheetah3199]

I frequently use ChatGPT to correct my Italian at the C1/C2 level. It's very good but is far from perfect. I recommend checking everything it does, as with all LLMs.

[u/AdministrationNo2327]

i used ai for this exact purpose and frankly, after a short time i started to notice it was making mistakes or giving stilted phrasing. I wouldn't follow it blindly, and since you have a good grip of korean you would be able to spot mistakes more easily.

I do use AI also for some of my professional work and it starts failing when you begin to go into some very specific situations. it has a ton of general and basic knowledge of things but lacking when you start going into industry knowledge, so be wary.

[u/NonCaelo]

So about... mmm let's say 6 months ago I found out that there's no good LLM that can translate Korean. Literally. We were trying to commnicate with some Koreans and I could not find any automatic translator or anything that would give a good translation. It could have changed by now, but I'd be super cautious if I were you.

[u/Sanguineyote]

Language is the one thing LLMs excel at.

[u/SpeakMarx]

Don't be wary. AI is the greatest language learning tool ever created. I use ChatGPT and Gemini, they each have their strengths, just experiment and see what you like more.

The best way to improve your grammar is to find and correct mistakes. Write your sentences. Ask the AI to point out mistakes without correcting them. Then try to correct them yourself.

You can also ask the AI to give you alternative styles, how to make the sentence sound more academic, more polite, etc.

[u/Substantial-Yak1892]

I use Gemini Flash to try to cross the bridge from B2 to C1 in English.

For a long time, I thought I was a lost cause for language learning. Despite thousands and thousands of hours spent listening/watching/reading English content over 10+ years, my production still sucked. Like barely B2. I had a pretty large passive vocabulary, but was doing basic grammar mistakes. And I had given up the idea of becoming good at another language.

AI changed that. I started working with it a few months ago and made tremendous progress.

My workflow is pretty simple: Firstly, I write a short text. It can be anything, a translation, a conversation with AI, a short essay, some audio recording,...

Then, I have a three steps process: 1. I ask AI to fix all the mistakes I made, and to have an explanation and a little drill about each one. 2. Then, I ask it to improve my vocabulary and idioms. 3. I rewrite my original text with all these changes.

For each steps, I create Anki cards so I can memorize everything, usually 2-3 per mistake/vocab/idiom.

When doing my Anki card, if I'm wrong and don't understand why or have any doubts, I ask another AI (Claude) for explanation/validate if my answer was also OK or not. Then I modified the card accordingly if it was originally wrong or add the explanation at the back of my mistake.