In Vocational Education and Training (VET), the term validation has a number of meanings and applications which can make it confusing. Here is some clarity on some of the differences. 


Independent validation

This is a requirement linked to Clause 1.25 of the Standards for RTOs 2015 which deals with the provision of training and assessment of an AQF qualification or assessor skill set from the Training and Education (TAE) Training Package.

 

Pre-use validation

Also known as names including ‘assessment review’, ‘assessment health check’ , 'pre-assessment validation' and 'pre-implementation validation', pre-use validation has long been a best-practice activity to verify assessment tools are fit-for-purpose.

We consider pre-use validation as a quality assurance process as it is an analysis of materials before they are put into use to confirm requirements will be fulfilled.

Pre-use validation might include a validation of mapping to confirm coverage against the unit and a check that the instruments have been written to collect the right type of evidence as required by the training package/unit of competency. Validation of assessment also confirms that the assessment tool is fit-for-purpose and has been designed to allow for the provision of sufficient evidence that is valid, reliable, authentic, and current.   

Pre-use validation is not mandated by the Standards for RTOs 2015. However, it is being considered as an inclusion in the revised Standards due for implementation 1.1.25. It is also now a requirement in the new TAE unit of competency TAEASS413 - Participate in assessment validation (R1).

And, for RTOs based in Western Australia, the state-based regulator, TAC, has expected for some time that all assessment resources, whether purchased or designed internally, must be validated prior to use in accordance with the Principles of Assessment and Rules of Evidence.

 

In-use validation

As a quality check, in-use validation is a process to confirm with industry and other stakeholders that the assessment tools and processes are fulfilling requirements. i.e. doing what they said they would do

 

Post-use validation - validation of assessment judgements

As a quality review, this validation activity is the formal process mandated by the Standards for RTOs 2015 and that every RTO must undertake. This validation can be known as 'formal validation', post-use validation', 'post-implementation validation' and 'validation of assessment judgements'. The aim of post-use validation is to confirm that your RTO's assessment system did consistently produce valid assessment judgments. This validation is conducted after assessment is complete and students have been notified of their results - hence post-use validation. 

Within a post-use validation process, validators are to consider the validity of both assessment practices and assessment judgements (how the assessment has been conducted/the assessment system used, and how the assessments have been marked). It is the latter which can be confused with moderation - more on that below.

In a post-use validation, checks are done on the assessment tools to confirm they have produced valid, reliable, sufficient, current and authentic evidence - evidence that has allowed an RTO to make reasonable judgements about whether training product requirements have been met. This is why it is vital (and makes sense) to have assessment tools validated at pre-use stage and/or in an assessment review activity to catch any issues before getting to the formal validation stage.

RTOs must comply will all requirements associated with post-use validation - including who can be involved, when it should occur, assessments for which training products should be validated, when, and the creation of a 5-year plan

In terms of the plan and a formal validation process, the Standards also require the following:

  • A systematic process in place (a validation plan)
  • A plan which covers specific quantities over set time-frames (five-year cycle to validate all training products on an RTO's scope; 50% to be completed within the first three years of any five-year cycle)
  • Details within the plan as to what will be validated when, and by who, and how any actions will be recorded and acted upon
  • Sampling method and statistically valid sample sizes to be used
  • Improvements to be documented and acted upon - including any identified rectification requirements (again, here is a good point for why pre-use validation is so valuable)


Moderation

Essentially a quality control process, moderation is a process of comparing judgments and confirming that across the board, and over time, assessors are giving similar results for similar evidence. Moderation is completed before students are advised of their results. It aims to ensure all results decisions for a unit are aligned.

Pre-use Validation - Validation of Mapping

Purpose of a Mapping Document

Before we look at validating the mapping document, let's have a very quick refresh on the purpose of the mapping document.

A mapping document is developed to go with the assessment. It is used to:

  • Plot and record coverage of an assessment against the unit of competency
  • Record exactly which parts of an assessment can be attributed as gathering evidence for which unit requirements
  • Set the foundation for good validation outcomes


Importance of Mapping

Done correctly, mapping ensures proper coverage of unit requirements and that assessment tools meet the Training Package requirements

  • Clauses 1.4 – 1.8 – 2.1 


The Role of Mapping in Compliance

Make it easy for someone looking for evidence that the assessment tool will do as it should

Use mapping to demonstrate compliance

  • It's more than just having a mapping document
  • The mapping must be valid too! It should be accurate
  • Accurate mapping is a skill


Purpose of Validating a Mapping Document

Before we even get into analysing whether an assessment tool has instructions to users, contains elements to address the Principles of Assessment etc, we need to ensure it will collect the evidence required for competency.

Given the mapping document is a way of saying "here's how this assessment tool is covering the unit of competency", it makes sense that this should be confirmed. 

A validation of mapping is to verify:

  • The unit of competency is covered (no gaps)
  • Assessment questions/tasks are collecting the right type of evidence
  • Accuracy of existing mapping


If you're an Education Matters annual subscriber, click through to see a video example of a validation of mapping activity.

(Exclusive content for subscribers - you must be signed in and part of Education Matters with a current annual subscription to view).

Pre-use Validation - Validation of Assessment Tool

A validation of assessment tool is undertaken to verify that:

  • Assessment instruments have been designed to collect the right type of evidence as required by the training package/unit of competency
  • The assessment has been designed to allow for the provision of sufficient evidence that is valid, reliable, authentic, and current


In this activity, the validator should look to confirm that the resources are logical, meet the unit and Training Package requirements, and will provide information consistent with these and industry requirements. In other words, this is the time to confirm that the assessment tool has been designed to meet the Principles of Assessment and when used, will allow a process of evidence provision that will satisfy the Rules of Evidence.

If not already completed as a discrete activity, a validation of the mapping document should be completed in pre-use validation of the assessment tool to verify unit of competency coverage.

Often, this validation activity will uncover systemic issues in tools, that once fixed, can offer efficiencies in future validations.

For example, if assessments for all units within a qualification are built on the same assessment tool template and that template has fundamental errors, it is probably safe to assume those errors are replicated throughout the qualification. 

If the template and repeatable components of the tool are fixed to meet requirements, it's only a validation of mapping that will need to be repeated each time - to ensure the assessment for each unit is covering the requirements specific to that unit.

As an assessment developer / VET instructional designer, knowing what outcomes need to be achieved for both the learning and/or assessment outcomes as well as usability of the tools is critical for a quality assessment system.

As an internal auditor and/or validator, knowing what to look for in a tool to gauge its suitability must be coupled with the skill to complete a thorough, critical analysis.

This session was developed by an experienced validator and auditor and goes through 5 things they look for in assessment tools.

The information below gives a few tips of what a valid assessment tool should look like.

If you're an Education Matters annual subscriber, click through to see a short video explanation of what inclusions to look for when validating an assessment tool

(Exclusive content for subscribers - you must be signed in and part of Education Matters with a current annual subscription to view).

Validations: VET Assessment Validation Services is part of Specialised VET Services. 

The above is based on information from the Big Book of kNOw which was originally developed in 2018 as an internal training tool for our team.

If you're looking for a template to use in pre-use validation, see this option.  

Prefer to get VET experts to complete pre-assessment validation for you?

See our validation service page