Guide to online training Best practice integrity measures
30 August 2017 V1 Guidelines for online training best practice integrity measures
Contents
GUIDE TO ONLINE TRAINING BEST PRACTICE INTEGRITY MEASURES. 1
The Future of Online Learning. 2
Price of online training is a reflection of quality. 2
Learning Management System Requirements. 3
Confirming Student Identity. 3
Unique Student Identification. 3
Website with HTTPS Security. 4
Identification during phone support 4
Assessing Student Competency. 4
Signing Off of Student Competency. 5
Minimising the Potential for Fraudulent Activity. 5
Issuing Statements of Attainment 7
Disclosure statement – Enrolo is a custom online training platform that complies with all of the best practice integrity measures detailed in this document. At Enrolo we believe that for the betterment of the online training industry as a whole all online training providers of nationally recognised training must implement these integrity measures as a minimum.
Executive Summary
To ensure that learning outcomes are achieved in an online environment certain minimum integrity measures must be in place. Integrity measures also need to be in place to reduce fraudulent activity. Whilst examples of industry best practice guidelines for online training do exist, many training providers are either not aware of them or, in some cases, are deliberately not implementing them to cut costs. Whilst there are examples of regulators stipulating mandatory online integrity measures, these requirements are often not enforced due to lack of resources or technical understanding. Training providers that are not complying with best practice online integrity measures are producing what is known as ’page turning’ or ‘tick and flick’ learning with assessments such as true/ false or multiple choice that can be passed by guessing. Providers that are cutting corners with online training are leading a race to the bottom in terms of the quality of training, assessment and ultimately the student’s value for money. The result of lack of integrity is not only poor learning outcomes and inadequate training, leading to inefficiencies in the workplace but also damage to the reputation of the online or the ‘e-learning’ industry as a whole. This best practice guide is intended to provide regulators and online providers with integrity measures that must be in place to ensure the integrity of online learning outcomes.
The Future of Online Learning
Automation in the training profession is continuing to advance at an ever increasing speed. Examples include interactive videos, free text auto marking, interactive video assessment and virtual reality scenarios for the demonstration of skills. The experience API is also allowing online training to move beyond the boundaries of formally structured content development.
Many qualifications that are taught using sophisticated online training methods can already deliver the same, if not better learning outcomes than in-class training. Since online training innovation is growing at an exponential rate it is advantageous if training providers are allowed to author their own learning materials to keep up with these changes. In the future regulators that attempt to provide learning material to training providers may face increased costs and struggle to keep up with the rapidly evolving innovation. An alternate approach that regulators could consider would be to provide elements such as videos or interactive content that must be included as part of a course.
From a training provider perspective, it is essential to realise that to achieve efficient and effective online learning, content and assessment are inseparable. This is because assessments are directly related to the learning material and in order to achieve accurate mapping to a unit of competency assessment questions must be written in such a way that may require regular edits to the content. Furthermore, if an assessment question is causing confusion or returning a high rate of incorrect answers it may be the case that the content needs to be edited rather than the assessment question. Another issue that can occur with content provided by third parties is if the third parties are renamed or restructured often they lose funding to maintain content and this content is then not able to be updated for name or content changes by training providers as there are often IP issues with authored content.
Price of online training is a reflection of quality
There is a perception that online training is a way for training providers to cut costs and increase revenues. However, if a training organisation is meeting the requirements for vocational education standards, despite the reduced cost of hiring classrooms, the cost of delivering effective online training is equal to if not greater than in-class training. This is because not only do all of the existing requirements such as qualified trainers and assessors, assessment material, mapping, internal audits, continuous improvement registers and so on all need to be in place but in addition to these costs, online providers need to purchase and run an online learning environment and build online learning material and robust assessments. Effective and engaging online training includes videos, games, interactive scenarios, and animations with voice overs and challenging exercises as well as additional reading materials. If the online content is achieving real learning outcomes then it can cost up to AUD$30,000 per hour of e-learning output. Many “online providers” are cutting corners by developing simple e-books or a PDF with a voice over and a next button between pages. Whist it is possible to map such material to the elements and criteria of a unit of training package there needs to be more emphasis on being able to demonstrate that a student can achieve competency from such material through rigorous assessment. If high integrity online training does not reduce costs for the training providers then what are the benefits?
- Improved accessibility particularly for remote or regional areas
- Students can learn in their own time and do not have to take time off work
- Reduced travel costs such as transport and parking during training and also reduced carbon emissions
- Greater consistency in terms of content delivered and the assessment method
- Quicker response times to updating learning content
- More empirical data can be collected for demonstrating competency
- Reduced administrative and non-value-adding work such as marking or issuing certificates
- Less monotonous tasks for trainers and more time spent with students that need their assistance
- Reduced human error related to monotonous and repetitive tasks
- Learners can work at their own pace and concentrate on areas they do not understand
- Students can spend as much time with qualified trainers as they need
- Students can access learning material ongoing and can also be provided with updates to the learning material
- Reduced room hire costs which are however, offset by online tools, online delivery, technical support and content development.
Learning Management System Requirements
In order to achieve integrity, a Learning Management System (LMS) must be used by training providers that has the sophistication to be able to collect data for integrity and track student activity and progress through the learning material and assessments. Whilst some LMS’s can track the amount of time that a student has interacted with learning content, how many attempts they have made for each assessment, and capture the result of each attempt, other LMS’s may only record the course sections that have or have not been accessed and that assessments have or have not been successfully completed, i.e. without capturing the granularity of the data. It is important that regulators understand these differences and to improve integrity by specifying the requirement for high granularity tracking of student interactions and assessment. Without the high granularity of data related to the e-learning environment it is more difficult to demonstrate that integrity has been maintained.
Integrity Measures
Confirming Student Identity
Confirming student identify in an online setting is of paramount importance in reducing fraudulent activity, i.e. by ensuring that the person doing the training is the person obtaining the Statement of Attainment/Certificate.
Fingerprint and retina scans are becoming more prevalent however, until these are more widely accepted the following procedures and features are considered best practice to verify that a student who is enrolled in an online course is the person who also completes the course and receives the certification:
Valid Photo Identification
Students must provide via a secure online website a copy of current government-issued photo ID which is reviewed and accepted or rejected by trained staff. Details captured and verified must include the full name, date of birth, and contact details (including residential, valid email address, and contact phone number) of the student.
The residential address of the student must be validated. This can be improved using an online address validation service or address look up tool.
Students must not be allowed to change personal details themselves – they need to lodge a request and provide evidence, such as government-issued photo ID, marriage certificate, change of name certificate, etc. where required.
For administrators the valid photo ID should be displayed against a student account when videos are being assessed, or during video conferencing so trainers can ascertain that they are speaking with the student who has made the recording.
Unique Student Identification
In Australia it is a government requirement for nationally recognised training that before completing their enrolment students must obtain a Unique Student Identifier (USI) and the details must match their valid photo identification provided. In Australia it is a government requirement that students supply their Unique Student Identifier (USI), the details of which must match their valid photo ID provided, before they can be issued with their Statement of Attainment/Certificate (qualification).
Website with HTTPS Security
Students must only access online courses through a https secure website that requires a login and password.
Unique username
Students must create their own unique username such as an email address
Secure Password
If an overly complex password is provided to the student then they will need to write it down and this reduces the security of the password so students must create a secure password themselves.
Students must be provided with best practice information regarding setting up and maintaining the security of their passwords.
Student password recovery must be via clicking a link to recover their password that sends them details via their registered email. Passwords must not be able to be changed without the student logging into the account.
Identification during phone support
When a student contacts a training provider by phone or email requesting help, the provider must have procedures in place to check their identity before providing any account details or assistance.
Terms and Conditions
Students must agree to terms and conditions at the start of an online course, confirming that they are the person doing the training. This can be done via a statutory declaration or equivalent.
Once complete and before generating and accessing the final PDF Statement of Attainment/Certificate the student must declare that they were the person that did the training and that they did so without assistance. This must be recorded in the electronic management system and be able to be provided as evidence for legal breaches.
Assessing Student Competency
Assessments must be mapped to a unit of competency to demonstrate that all of the required learning elements have been assessed.
In an online environment assessments that are designed to provide a true reflection of the knowledge and skills that have been achieved are one method of proving competency has been gained. Offline activities can also be undertaken and witnessed by a suitably qualified supervisor.
In an online environment in order for assessments to demonstrate that a student has achieved understanding and competency they must have the following integrity measures:
- Require at least 25% of questions to be answered with free text sentences allowing a proper critical response to questions, e. not single word answers or numbers
- In the case of demonstration of skills, require video footage to be submitted to capture a student’s ability to demonstrate a task or skill
- If a student gets the same question wrong 3 times they are “locked out” and requiredto talk to the training organisation and if they need assistance they are referred to a qualified trainer rather than keep guessing until they get the question correct through a process of elimination
- Multiple choice questions must have more wrong combinations than available attempts and designed so that students will be locked out before guessing through a process of elimination.
- Students must not be provided with the correct answer before they get the answer correct themselves.
- The order of assessments must be randomised to reduce the likelihood of cheat sheets.
- Because of the complexity of meeting mapping requirements if questions are randomised, banks of similar questions must be created and questions randomly selected from those banks to ensure that the particular questions achieve the required mapping.
In contrast to assessments that comply with the above integrity measures, assessments that are 100% marked by a computer that are predominantly multiple choice by elimination or true/false by elimination, provide no requirement or opportunity for a student to demonstrate their comprehension of the learning material. Whilst it can be argued that by eventually getting a question right is a form of learning, allowing students to keep attempting questions until they get the correct answer does not deliver meaningful learning outcomes. Demonstrated competency through rigorous and auditable assessment in combination with the above integrity measures is considered an integral requirement for the demonstration of online learning outcomes.
Signing Off of Student Competency
Online training providers may try to cut corners and costs by using administration staff with no training qualifications to assist with training. To cut costs training providers may also reduce access to qualified trainers to very short periods. It is essential that qualified trainers are available in an online environment. These trainers may communicate with the students via:
- Telephone
- Live chat
- Webinars
- Video conferencing
- Skype
Integrity measures include:
- The use of qualified trainers for assisting with training support. All trainers employed by a training organisation must be on a training register.
- Total trainer hours must reflect availability during normal office hours.
- Trainers must be available during normal business hours and must assist learners requesting training within 30 minutes of a request.
- An online statement of attainment must not be issued until the enrolment details confirmed and assessment and learning activities of the student have been reviewed by a qualified trainer.
Minimising the Potential for Fraudulent Activity
The following integrity measures must be in place to minimise the potential for fraudulent activity
- Capture the student’s IP address and monitor if bulk users are coming from the one IP.
- A system for monitoring cheating must be in place that flags when different students give the exact same free text answers to free text assessment questions..
- Student detail fields must be locked so that these can only be changed by administration staff upon request and subsequent verification of the student’s identification.
- Students are ‘locked-out’ of being able to complete an assessment after failing a set number of attempts of each question and must speak with a qualified trainer to be unlocked if they do not know the answer.
- Must be able to intervene and make direct contact with the student if required.
- Notify students that certification may be voided if fraudulent information is provided or fraudulent activity is detected.
- Have the ability to report immediately any suspicious activity by students undertaking the course to the appropriate authority.
- Deliver training via SSL to reduce third party fraudulent activity.
- Capture electronic evidence that the student has agreed to terms and conditions relating to not being assisted in any way during assessment and that they are the one that undertook and completed the training.
- If adequate integrity measures are not in place, or a student has breached an integrity measure, students must sign and upload a witnessed statutory declaration that they are undertaking the training without assistance.
Course Content
Volume of Learning
The volume of learning includes guided learning, individual study, research, learning activities in the workplace and assessment activities. It could be argued that rather than applying this at a qualification level, the volume of learning should be applied at a unit of competency level and rather than minimum duration the relationship between the quality of the content being able to map to a rigorous assessment should be the deciding factor in assessing competency as the AQF goes on to say:
“The duration of the delivery of the qualification may vary from the volume of learning specified for the qualification. Providers may offer the qualification in more or less time than the specified volume of learning, provided that delivery arrangements give students sufficient opportunity to achieve the learning outcomes for the qualification type, level and discipline.”
This could easily be applied to a competency rather than the whole qualification.
When looking at VET training, we need to remember that students come from hugely various backgrounds and experience, from Year 11 students undertaking a VETiS program to someone with a PhD who is looking for a career change to an older person with a wide-ranging life experience. This all affects how quickly they can become competent, not only based on their level of experience and in the skills they already have or need to learn but their ability to undertake and comprehend the training in the first place.
While setting minimum durations based on the time a learner (who is new to the industry area) would be required to undertake supervised learning and assessment activities (ASQA report A review of issues relating to unduly short training, page 15 https://goo.gl/z9v6jr) has merit, the dilemma remains that some people undertaking formal study will already be to some degree competent and not require the same minimum duration.
In terms of auditing the appropriate volume of learning on a student by student basis, training providers using data from the AVETMISS enrolment should be able to demonstrate whether a student is new to the field and requires what could be considered as minimal learning hours as opposed to someone who is requalifying and could reasonably be expected to complete a competency in a much shorter time frame. I.e. based on evaluating the needs of the student from the language, literacy and numeracy (LL&N) together with the AVETMISS enrolment data a training provider should be able to specify the expected or minimum volume of learning required. In a class room this could be part of the learning plan. In an online environment this tailored learning plan must be established and managed in a dynamic way, i.e. by having certain locks or minimum interactive learning hours completed before assessment is made available.
Integrity measures
- Access to assessment should not be permitted until a minimum amount of learning that is determined on a student by student basis has been achieved
- The LMS must be able to manage the minimum time limits and must also be able to record the amount of time spent in learning activities.
Issuing Statements of Attainment
Copying or editing statements of attainment can be done no matter how they have been issued. However If statements of attainment are issued online the following integrity measures must be in place to minimise fraudulent activity:
- The student must not be able change their name online and an administrator should only be permitted to do this after identifying the student and then sighting an updated form of photographic identification meeting the identification integrity requirements with the new name.
- The certificate must contain a unique number that is linked to the unique student username.
- The certificate must have a background water mark with the unique details of the certificate, e.g. student name, date of birth, issue date, certificate number, signature of issuing authority, expiration date over the top of the watermark(s).
- The certificate must be in a PDF format and the PDF must have security enabled that locks the PDF and prevents edits.
