The Truth for RTO Owners
Assessment is not the final administrative layer of a course. It is the load-bearing structure that supports every claim an RTO makes about competence, progression, and the credibility of an AQF qualification.
The definition is straightforward:
Compliant assessment is a planned process that produces valid, sufficient, authentic and current evidence of a learner’s achievement against the requirements of the training product.
That means assessment design must do more than generate completed paperwork. It must demonstrate that the learner can perform the required task, under the required conditions, to the required standard.
ASQA’s current assessment expectations focus on fit-for-purpose assessment, accurate judgements, the principles of assessment, the rules of evidence, and regular validation. For higher education providers, TEQSA similarly expects assessment systems to assure learning outcomes and protect academic integrity in an environment shaped by generative AI.
The issue is not usually a lack of effort. It is structural weakness.
Here are seven assessment design mistakes I regularly see placing accreditation, audit readiness, and the integrity of an award at risk.
1. The Assessment Is Not Mapped to the Unit of Competency
Definition: Assessment mapping is the documented relationship between each assessment task and the elements, performance criteria, performance evidence, knowledge evidence, and assessment conditions of the unit.
A task can look polished and still be non-compliant if it does not generate evidence against every relevant requirement.
This is one of the most consequential mistakes in vocational education course design. A quiz may cover terminology but fail to assess application. A written assignment may address one performance criterion while ignoring the required frequency or volume of performance. A practical task may appear authentic but fail to meet mandatory assessment conditions.
What an auditor sees
An auditor samples the tool and mapping matrix, then finds:
- performance criteria with no direct evidence;
- knowledge evidence assessed only through superficial recall;
- performance evidence not demonstrated at the required frequency;
- assessment conditions omitted or treated as optional;
- mapping that claims coverage without showing where the evidence is produced.
The consequence is not merely an untidy matrix. The RTO may be unable to defend its competency judgements.
The practical fix
Build the assessment architecture from the unit requirements outward.
For every element, performance criterion, evidence requirement, and condition, identify:
- the assessment task that produces the evidence;
- the specific learner action or artefact required;
- the assessor observation or benchmark used;
- the location of the evidence in the tool;
- the reason the evidence is valid and sufficient.
A mapping document should explain the structure. It should not disguise gaps with ticks.
For further context, read Assessment Integrity by Design: Valid Evidence, AQF Compliance, and AI Literacy.
2. The Assessment Relies Too Heavily on Recall
Definition: Recall assessment tests whether a learner can reproduce information; competency assessment tests whether the learner can apply knowledge and skills in an appropriate context.
These are not the same thing.
Multiple-choice questions and short-answer tasks have a legitimate role. They can efficiently test foundational knowledge. The problem arises when they carry the full burden of proving competence, particularly in units requiring judgement, problem-solving, communication, technical execution, or workplace performance.
What an auditor sees
The assessment tool contains pages of definitions, matching activities, and written responses, but little evidence that the learner can:
- make decisions in context;
- perform a task;
- respond to variables or unexpected conditions;
- explain their reasoning;
- produce an outcome to an industry-relevant standard.
The auditor may conclude that the tool measures familiarity with content rather than competency.
The practical fix
Use a deliberate evidence mix:
- a knowledge task for essential concepts;
- a practical or simulated task for application;
- observation or demonstration for performance;
- targeted questioning for reasoning and transfer;
- a reflective or explanatory component where judgement matters.
The principle is not “more assessment”. It is better triangulation.
Assessment should reveal what the learner can do, not merely what the learner can repeat.
3. There Are No Clear Rules for AI Use
Definition: An AI assessment policy specifies when artificial intelligence may be used, how its use must be disclosed, what remains the learner’s responsibility, and which tasks require unaided performance.
An undisclosed assumption that “students must not use AI” is not an assessment strategy. Nor is an unqualified statement that “AI is allowed”.
Generative AI has changed the conditions under which evidence is produced. ASQA expects authentic evidence of the learner’s own work. TEQSA’s assessment reform guidance similarly emphasises transparent, ethical AI use, process evidence, and secure points of assessment.
What an auditor sees
The provider cannot answer basic questions:
- Was AI permitted for this task?
- If so, what uses were authorised?
- How did the assessor verify the learner’s understanding?
- What evidence shows the submitted work is authentic?
- Which outcomes were demonstrated independently of AI?
A generic AI detector is not a substitute for an evidence strategy. Detection tools can produce uncertain or contestable results.
The practical fix
Classify assessment tasks by AI conditions:
- AI prohibited: complete under controlled or supervised conditions.
- AI permitted with disclosure: require prompts, outputs, verification notes, and a rationale for use.
- AI integrated: assess the learner’s ability to evaluate, refine, govern, and ethically apply AI output.
Add a short oral defence, live demonstration, annotated draft, or process log where appropriate. The purpose is not surveillance. It is to make learning and judgement visible.
For a broader perspective on this shift, see From “Author” to “Agent Governor”: The Evolution of Instructional Design.
4. Assessor Judgement Is Inconsistent
Definition: Reliable assessment produces consistent competency decisions when appropriately skilled assessors apply the same requirements, benchmarks, and evidence standards.
Two assessors should not reach materially different outcomes because the tool leaves the standard to interpretation.
What an auditor sees
Completed student work reveals that:
- one assessor accepts partial evidence while another requires full performance;
- feedback is generic or absent;
- marking guides use vague terms such as “good understanding”;
- observation checklists do not define acceptable performance;
- reassessment decisions are inconsistent;
- assessor comments do not explain the judgement.
This is a reliability failure, and it exposes the RTO’s quality assurance system.
The practical fix
Strengthen the assessor-facing architecture:
- define observable performance standards;
- include model answers and acceptable variations;
- distinguish competent from not-yet-competent evidence;
- provide question prompts for oral clarification;
- identify critical errors and mandatory requirements;
- include instructions for reasonable adjustment;
- moderate a sample of completed assessments across assessors.
A robust assessor guide is not an administrative attachment. It is the instrument that converts a policy intention into consistent professional judgement.
5. Authenticity Is Assumed Rather Than Demonstrated
Definition: Authentic evidence is demonstrably the learner’s own work and reflects their actual capability.
A signed declaration can support authenticity, but it cannot carry the entire burden of proof: particularly for online, remote, or AI-enabled assessment.
What an auditor sees
The provider submits identical or near-identical responses, unusually polished work from learners with limited demonstrated capability, or evidence that cannot be traced to the individual student.
There is no record of identity verification, follow-up questioning, supervised performance, third-party confirmation, or progressive drafts.
The practical fix
Design authenticity into the assessment pathway:
- use learner-specific scenarios or workplace contexts;
- require drafts, planning notes, or version history;
- include short verbal questioning;
- observe practical performance live or by secure recording;
- require learners to explain key decisions;
- use third-party reports where appropriate;
- retain evidence of how authenticity was checked.
The shift is from asking, “Does this submission look plausible?” to asking, “What evidence shows this learner can explain and perform the work?”
6. Conditions of Assessment Are Unclear or Unrealistic
Definition: Assessment conditions specify the environment, resources, equipment, supervision, time, location, and other circumstances under which competence must be demonstrated.
A task cannot be valid if the conditions under which it is completed do not reflect the requirements of the unit.
What an auditor sees
The assessment tool uses phrases such as “complete the task in a realistic environment” without defining what that means. It does not specify:
- required equipment or software;
- access to workplace documents;
- supervision requirements;
- whether simulation is permitted;
- acceptable resources;
- time or attempt conditions;
- how reasonable adjustment may be applied without changing the competency standard.
The result is an assessment process that cannot be replicated or defended.
The practical fix
Write conditions as operational instructions. State what the learner must have access to, what the assessor must observe, what may be simulated, and what cannot be omitted.
If simulation is used, justify its authenticity. Describe the scenario, constraints, resources, roles, and performance expectations in enough detail that another assessor could reproduce the assessment.
Clarity is not bureaucracy. It is the foundation of fairness and reliability.
7. There Is No Real Validation or Review Cycle
Definition: Validation is a quality-assurance process that examines assessment tools, practices, evidence, and judgements to confirm that they remain valid, reliable, fair, flexible, and aligned to the training product.
Validation is not the same as asking a colleague to proofread a document.
What an auditor sees
The RTO has a validation calendar but no meaningful findings. Records show attendance without analysis. Completed student evidence was not reviewed. Identified issues were not assigned, corrected, or followed up.
Under the 2025 Standards, validation must occur at least once every five years for each training product on scope, with more frequent review where risk, industry change, complaints, poor outcomes, or training product updates justify it.
The practical fix
Make validation evidence-based. Review:
- the assessment tool and its mapping;
- a representative sample of completed student work;
- assessor judgements and feedback;
- authenticity processes;
- accessibility and reasonable adjustment;
- changes to the training product or industry;
- the effectiveness of previous improvement actions.
Document what was found, what changed, who owns the action, and when the improvement will be checked.
A review cycle without implemented improvement is not quality assurance. It is a calendar event.
Build the Assessment Structure Before the Audit Finds the Weakness
The common thread across these mistakes is misalignment. The course outcome, unit requirement, learning resource, assessment task, assessor judgement, and validation process must form one coherent structure.
Audit readiness is not document preparation. It is the visible result of disciplined assessment design.
If your RTO is preparing for an ASQA audit, developing an accredited course, or revising assessment for an AI-enabled learning environment, I can help you rebuild the framework before the regulator exposes the gaps.
My work covers:
- assessment strategy and mapping;
- accredited course development;
- learning resource design services;
- assessor guides and marking tools;
- authenticity and AI-use procedures;
- validation planning and evidence review;
- ASQA and TEQSA-facing documentation;
- vocational education course design for complex technical fields.
I bring together instructional design, AQF alignment, assessment integrity, and regulator liaison so that the course is not merely complete, but defensible.
Read What Does an Instructional Designer Actually Do? for a broader explanation of how this work joins pedagogy, compliance, and practical capability.
If your assessment system would struggle to explain where evidence comes from, how authenticity is assured, or why an assessor’s judgement should be trusted, the time to address the weakness is before the next audit.
Explore my course development services or instructional design services to begin the assessment strategy and mapping conversation.