How scoring works
How AUN results are decided:
six skills, scored 0 to 5
AUN (阿吽) is a Japanese speaking test that splits speaking into six skills and scores each from 0 to 5. The scores are decided by a Japanese listener who listens to your recording. From those scores come a ○△× mark for 13 situations and the one thing to fix next.
What the six skills look at
AUN looks at content, and at the five things that support it: organisation, fluency, grammar, vocabulary and pronunciation. The six are scored separately so that one weakness does not pull the other scores down. If your sentences are broken but your point comes across in detail, grammar is scored low and content high.
| Skill | What is looked at | What is left out |
|---|---|---|
| Content | Detail, whether reasons and feelings are clear, whether it came across without misunderstanding | The order of the talk (that is organisation); how impressive the opinion is |
| Organisation | The order of events, linking words such as “so” and “but”, the line from reason to example to conclusion | Whether each sentence is correct (that is grammar) |
| Fluency | The length and number of pauses, restarts, speed | Time stopped because a word would not come (that is vocabulary); how clear the sounds are (that is pronunciation) |
| Grammar | How sentences are built, and whether errors mislead | Word choice (that is vocabulary). Natural omissions do not lower the score. |
| Vocabulary | Range of words, whether the right word is chosen, whether you can rephrase | How the sentence is built (that is grammar) |
| Pronunciation | Whether it sounds like a different word, the effort for the listener, phrasing and intonation | Pauses and speed (that is fluency); how strong an accent is in itself |
How the scores are decided
The scores are decided by a Japanese listener (Namba, the founder of AUN). The AI prepares a transcript and a draft, but it does not decide the scores or the situation marks you receive.
To stay close to a real conversation, the listener plays the recording once without stopping and replays it at most twice. For each skill he uses the description and examples for each level, and picks the level that most of your answer is closest to. One slip, or one place where you got stuck, does not lower the level.
Your score is the highest level you showed in at least two tasks. That way it reflects what you can do repeatedly. Answers given after an AI follow-up count one level lower for content and organisation.
How the ○△× mark for each situation is decided
For each of the 13 situations, we have set the skills and levels needed to talk without trouble. Booking by phone, for example, asks more of pronunciation, because the other person cannot see your face.
| ○ You can handle it | Every skill the situation needs is at or above the required level |
|---|---|
| △ Almost | Just one of the needed skills is one level short (and that skill is at 2 or above) |
| × Not yet | Anything else |
| Cannot judge | Your answers did not show enough of a skill the situation needs |
In the sample above, grammar is 2 where customer service and the job interview need 3, so they are marked △ even though the other skills are high enough. The full list of 13 situations is in What AUN is.
How “what to fix next” is chosen
It is the skill that most often falls short across your △ and × situations. △ situations, being closer to ○, count for more than × ones. So it is not always your lowest score. The result sets out how you speak now beside how the next level up sounds.
What these scores do not tell you
Scores and marks are a guide, not an official certificate. Back-and-forth, such as asking the other person to repeat or checking what they meant, and keigo (polite language) are not measured.
Scores can move with what you talk about and how you are on the day. There is one listener at present, and we have not yet published any study of differences between listeners or of how scores move on a retake. The standards for each situation will be revised as we see more results.
What each level from 0 to 5 sounds like
A 0 is given when there is almost nothing to judge: silence, speech only in your first language, or speech that cannot be made out. Levels 1 to 5 are set out below. Each level includes everything described at the levels below it.
Content
Whether you answered the question, and how specifically and accurately your point came across.
| 1 | The topic can be guessed from single words. The listener has to fill in most of the meaning. |
|---|---|
| 2 | The core of what you mean (what, where and so on) is clear, but details do not come across. |
| 3 | Beyond the core, reasons, feelings and some details come across. The listener fills in here and there. |
| 4 | Examples and details come across without misunderstanding. Almost nothing needs filling in. |
| 5 | Fine shades of meaning, conditions and caveats come across accurately. |
Organisation
Whether the listener can follow the order of what you say and how the parts connect.
| 1 | Words and short phrases are scattered. The connection cannot be followed. |
|---|---|
| 2 | Short sentences are lined up with words like “and” and “but”. |
| 3 | Events and the like are told in order, in a way that can be followed. |
| 4 | Reasons, examples and conclusions connect clearly; the listener does not have to rearrange them. |
| 5 | Even a long answer is clearly built, with natural shifts of topic and summing up. |
Fluency
Whether you can keep talking without stopping.
| 1 | One word at a time. Long stops; the talk breaks off. |
|---|---|
| 2 | Short answers are possible. Many stops and restarts, but understandable if the listener waits. |
| 3 | There are pauses and restarts, but the talk keeps going. |
| 4 | Close to conversational pace. Even when unsure, long stops are rare. |
| 5 | Effortless and natural. The flow does not stop even on difficult topics. |
Grammar
Whether you build sentences correctly, and how complex a sentence you can say.
| 1 | Mostly single words and fixed phrases put side by side. |
|---|---|
| 2 | Short sentences. Many errors, but the meaning is clear. |
| 3 | Sentences are joined with common forms (〜て, 〜から, 〜とき). Errors are occasional and do not cause misunderstanding. |
| 4 | Somewhat complex sentences are used, such as reasons, conditions and comparisons. Errors that could mislead are rare. |
| 5 | Complex sentences are used accurately. Errors are rare and quickly self-corrected. |
Vocabulary
Whether you find words that fit what you want to say.
| 1 | Only fixed words come out; you stop because words do not come. |
|---|---|
| 2 | Familiar things can be said in common, simple words. You sometimes stop for lack of a word. |
| 3 | On familiar topics, the words you need mostly come. When they do not, you rephrase and carry on. |
| 4 | Words that fit opinions and explanations come out, with little repetition. Wrong word choices that could mislead are rare. |
| 5 | You are not short of words even as the topic widens. You choose your wording and express fine differences. |
Pronunciation
Whether your sounds can be understood without the listener asking again.
| 1 | Many parts sound like other words; the listener has to fill in a lot. |
|---|---|
| 2 | The listener sometimes wants to ask again, but can understand by listening carefully. |
| 3 | Mostly easy to understand. The listener rarely wants to ask again. |
| 4 | Almost no effort for the listener. Phrasing and intonation are easy to follow. |
| 5 | No effort for the listener. Intonation carries feeling and emphasis. |
AUN scores do not correspond to JLPT N1 to N5. The level descriptions were written with reference to the descriptions of spoken ability in the CEFR, the European framework for languages, but an AUN score is not a statement of a CEFR level.
Frequently asked questions
Do the scores map to JLPT N1 to N5?
No. The JLPT tests reading and listening; AUN looks at speaking.
Does AI give the scores?
No. The AI prepares a transcript and a draft. The scores and situation marks you receive are decided by the Japanese listener.
Will my scores change if I take the test again?
They can move with what you talk about and how you are on the day. We have not yet published any study of how much.