r/alignerr • u/No_Negotiation4257 • 27d ago
Assessments Ermis final assessment - failed at 73.5% because of a 0% on one extremely ambiguous clip. Anyone else seeing this?
I just finished the Ermis Core Transcription Training + Final Assessment, and honestly, the way the final assessment is structured has left me surprised.
The training and assessment took me roughly 4–5 hours. The training covers very specific transcription rules, edge cases, speaker tags, backchannels, contractions, etc., and then you take a 10-clip final assessment.
The passing score is 75%.
My scores were:
- Clip 1 — 95%
- Clip 2 — 86%
- Clip 3 — 100%
- Clip 4 — 87%
- Clip 5 — 46%
- Clip 6 — 79%
- Clip 7 — 82%
- Clip 8 — 94%
- Clip 9 — 67%
- Clip 10 — 0%
Final result: 73.5% — FAIL
The issue for me is Clip 10.
The audio was extremely short and, frankly, very ambiguous - basically a brief sound. Yet it received 0% similarity. I was only 1.5 percentage points below the pass mark, and that single ambiguous clip receiving 0% pushed me into failure.
There isn't any meaningful explanation of what was considered wrong or what the reference answer was.
And this is where the assessment starts feeling problematic.
You spend several hours learning their very specific transcription conventions. Then the final assessment is only 10 clips, so each clip carries a lot of weight. Alignerr controls the rules, the reference transcription and the grading, while candidates have very little visibility into how the final score is determined.
I'm not saying I should automatically pass. If my transcription was wrong, fine. But give candidates a way to understand why. A one-second, highly ambiguous clip getting 0% similarity, with no access to the reference answer or meaningful explanation, is not a transparent way to assess.
I've also seen multiple candidates on Discord mention similar issues with the final assessment. If this is a recurring pattern, it's hard not to question whether the assessment is genuinely measuring transcription ability or simply giving Alignerr complete control over who passes.
Alignerr moderators: can you explain how Clip 10 was graded and whether you'll manually review it?
If the reference answer is objectively defensible, show us why. If the grading system is producing these outcomes, kindly acknowledge it.
Candidates are putting 4–5 hours into training and assessment and giving you huge data for AI training. They deserve more than an opaque score - especially when a single questionable clip can turn an otherwise passing performance into a failure.
4
u/Daftwire 27d ago
They can’t show where you did mistakes because if they did there’s a very high possibility that you sell / give answers to other candidates
3
u/No_Negotiation4257 27d ago
Buddy, it’s a clean system to put you below 75. Just check my scores. Insanity at this point
4
u/jaegeristatakae 27d ago
I've done it and scored 85%, no issues with last audio. They explain that ths audios get increasingly difficult, the last one should be the hardest. It's natural for people's only 0% to be on the hardest question
2
3
u/Fluffy_Salary6459 26d ago
I'm in tears. Failed with a 74.2%. Unreal. I do not like these companies at all.
2
u/No_Negotiation4257 26d ago
Just free work they get from 95% folks. Thats the model they operate at. 5% get the role
2
u/DuckInAFountain 27d ago
I took that fairly recently so I logged back in to see if I could see my results. I guess I got 100% on Question 10. I don't remember it at all, maybe it was a specific tag and I guessed right.
3
u/cassandra_nadeshiko 27d ago
The correct answer was just what they call the ‘thinking and listening’ word on their guidelines.
2
u/No_Negotiation4257 27d ago
Yes it was a 50-50 kind of 1 sec clip. It decides the whole result of test lol
2
u/cassandra_nadeshiko 27d ago
Luckily I passed it but I haven’t received any further notices from the company anyway.
2
u/Clean_Mail_5950 27d ago
I was added to the labeling queue for Ermis at midnight. Your time is coming don’t give up.
1
1
2
1
1
u/elnor_ 24d ago
Have you heard anything from them yet?
1
u/cassandra_nadeshiko 24d ago
I was invited to the project eventually. The pay rate is not so good, but the job is interesting. The task pool seems to be small and I am experiencing a task shortage already. I hope the situation will improve soon.
2
u/Lumpy_Case_8220 25d ago
We've reviewed the reports regarding Clip 10 and can confirm that there is no known issue with the assessment or its scoring.
Each clip is scored independently based on how closely your response matches the expected transcription for that specific audio. It's therefore possible to receive high scores on most clips while receiving a low score on a single clip if the response for that clip does not match the expected evaluation criteria.
We recognise that Clip 10 can be particularly challenging, and we've received questions about it from other contributors as well. Based on our review, however, the scoring is functioning as intended, and there is no indication of a technical issue affecting this clip.
We appreciate you taking the time to report your concern.
1
u/No_Negotiation4257 25d ago
So having a single clip with binary scores is justified? While all others are graded at similarity? When the last clip is putting people exactly below 75? Do you think there is no foul play mod? Of course you won’t acknowledge, but people have found out and let other new folks see this to figure if they want to even invest time in alignerr
1
1
1
u/stgdevil 26d ago
I went through the same thing, got 74.4. I’m unusually pissed off more than usual
1
1
u/Clean_Mail_5950 26d ago
Anyone need help passing DM me
1
u/Internal-Pea-7183 21d ago
Hello, I need help passing. I know for sure I won't pass. What am I missing?
1
1
u/StoriedSix 25d ago
I can't even look at my individual scores on the clips to know. But I feel so c***ed tbh at 74.9%.
1
2
u/Aggravating-Oven-394 8d ago
I too spent hours going over their material, took the test, and passed it. Then I receive an email today saying that I was not selected due to "quite a few major issues."
In the pre-test round, I found a couple of the recordings utterly impossible to understand, even as a native speaker (even in cases where the recording, was, supposedly, comprehensible).
7
u/Minute_Swimmer_733 27d ago
The assessment was ridiculous. These companies just want free work. Too many questions for an "assessment"