Auditing Learning Quality in Online Programmes

Many online programmes are formally compliant and educationally weak.

Many online programmes are formally compliant and educationally weak.

Their policies are current. Learning outcomes appear in the correct templates. Accessibility statements are published. Assessments have been approved. Internal verification records are complete. Learner satisfaction is reported and completion figures are available.

These arrangements may demonstrate administrative control. They do not demonstrate that learners are being taught effectively, assessed validly or supported consistently.

The central failure of checklist based quality assurance is that it rewards the presence of processes without adequately testing their educational consequences. It confirms that something exists, but not that it works. A programme may therefore pass an audit while learners experience incoherent delivery, delayed feedback, inaccessible resources, weak assessment and fragmented support.

A rigorous learning systems audit should not ask only whether the organisation followed its procedures. It must ask whether those procedures protect learning, expose failure and lead to measurable improvement.

Compliance can conceal rather than reveal poor quality

Standards are necessary, but compliance with a standard is not equivalent to educational quality.

The UK Quality Code provides 12 sector agreed principles for securing academic standards and assuring and enhancing quality. Quality Matters uses eight general standards and 44 specific review standards to examine the design of online and blended courses. ISO 21001:2025 provides management system requirements for educational organisations, including universities, training providers and corporate learning functions.

These frameworks can identify missing objectives, unclear navigation, weak alignment and inadequate institutional controls. Their value should not be dismissed. Their limitations should not be ignored either.

A framework can confirm that learning outcomes, activities and assessments are present. It cannot, without deeper investigation, prove that the teaching develops the intended capability. It may verify that feedback arrangements are documented without establishing whether feedback is accurate, timely or useful. It may confirm that a learner support service exists without testing whether learners can obtain a resolution from it.

The danger is performative compliance. Organisations learn how to present the expected evidence, use the approved terminology and complete the required templates. The audit then validates the quality system partly through records produced by that same system.

A course shell is not a learning experience

Many audits inspect online programmes as collections of documents and digital pages.

Auditors review programme specifications, module templates, assessment briefs, recorded materials and policy statements. They may not enrol as learners, attempt the activities, follow the support pathway, submit work or experience the accumulated demands of the programme.

This creates a serious distortion. A course can look coherent when viewed by an auditor who already knows its structure but remain confusing to a learner encountering it for the first time. A credible audit must test the programme as it is experienced, not merely as it is documented.

The Office for Students review of blended learning warned that decisions made at individual module level still need to produce a coherent experience across the complete course. Yet many internal reviews continue to examine modules separately. Each module may appear acceptable while the programme as a whole contains duplicated content, conflicting instructions, inconsistent navigation and badly distributed workloads.

Five compliant modules do not necessarily create one coherent programme. The audit should therefore follow the learner journey horizontally across recruitment, induction, teaching, assessment, feedback, support, progression and completion. Without this systems view, fragmentation is repeatedly mistaken for local variation.

Alignment must be demonstrated through learner work

Programme documents almost always claim that learning outcomes, activities and assessments are aligned. Repetition of the same terminology across several templates is not evidence of genuine alignment.

The audit must trace each significant capability through the learning system. If learners are expected to evaluate complex evidence, where are they taught how to do so? Where do they practise? What formative feedback do they receive? Does the final assessment require evaluation, or can it be passed through description and factual reproduction?

Weak alignment is often hidden behind correct language. An outcome may use analyse, while the learning materials provide information and the assessment rewards summary. The topic appears throughout the module, but the required intellectual demand disappears.

The audit must therefore examine actual learner work and assessor decisions. It should establish whether successful submissions genuinely demonstrate the stated outcomes and whether unsuccessful learners were given sufficient teaching and practice before being judged. Without this evidence, alignment remains an organisational claim made about itself.

Quality assurance often scrutinises learners more than the programme

Online systems generate detailed evidence about learners. They record logins, clicks, attendance, submissions, completion and assessment performance.

Far less evidence may be collected about delayed tutor responses, broken links, misleading instructions, inaccessible materials, excessive workloads or repeated assessment problems. This imbalance matters because it shapes the diagnosis of failure.

A learner who stops accessing the platform may be classified as disengaged. The audit may not ask whether the module became confusing at that point, whether tutor contact had failed or whether the workload was unrealistic. A learner who repeatedly resubmits an assessment may be treated as academically weak, while the assessment brief and teaching remain unchallenged.

Analytics can therefore individualise what is actually a systems problem. A rigorous audit should reverse the direction of scrutiny. When learners repeatedly fail at the same point, the programme should become the primary object of investigation. It should ask what the system is repeatedly doing before learners fail. Until that question is answered, terms such as non engagement, low ability and poor motivation may simply transfer organisational responsibility to the learner.

Delivery quality cannot be inferred from design quality

Quality Matters explicitly focuses on course design. Its rubric can provide valuable evidence about whether the intended components of an online course have been designed appropriately. It cannot establish that the course was delivered well.

A carefully designed programme can be undermined by excessive tutor workloads, slow responses, inconsistent facilitation, weak subject knowledge or unreliable assessment. Conversely, skilled educators may temporarily compensate for a weak design through intensive support that the organisation cannot sustain.

An audit that examines only the course shell may therefore certify an experience that learners never actually receive. Actual delivery evidence should include:

• Learner communications.
• Tutor response times.
• Marked assessments.
• Feedback quality.
• Discussion facilitation.
• Support escalation.
• Consistency across different tutors and cohorts.

This evidence may expose uncomfortable organisational decisions. Delayed feedback may result from excessive caseloads rather than tutor negligence. Assessment inconsistency may arise from ambiguous criteria rather than assessor incompetence. Poor learner support may reflect fragmented ownership rather than individual staff failure. The audit must identify the decision that created the conditions for failure, not merely the person closest to the visible problem.

Assessment approval does not establish assessment validity

Assessment is where superficial quality assurance becomes particularly dangerous. An assessment may have been written, approved, internally verified and securely stored while still failing to measure the intended capability.

A quiz may test factual recall when the outcome requires professional judgement. A written assignment may reward polished presentation without confirming authorship or independent understanding. A simulation may reproduce a simplified process that bears little resemblance to real practice.

Moderation cannot repair a fundamentally invalid task. It can only make decisions more consistent within the limits of that task. A rigorous audit must therefore investigate what the assessment permits the organisation to claim about the learner. It should examine whether successful learners can explain, apply and defend their work, not merely whether assessors completed the correct documentation. This is particularly important where generative artificial intelligence, extensive external support or automated assessment may separate the submitted product from the learner’s actual capability.

Accessibility conformance is not proof of inclusion

Accessibility audits often become another form of checklist assurance. An automated scan may confirm colour contrast, alternative text and document structure while failing to identify whether learners can understand diagrams, navigate complex activities or participate within restrictive time limits.

WCAG 2.2 provides essential testable requirements, but W3C explicitly acknowledges that the guidelines do not meet every user need. An organisation should therefore not use technical conformance as a complete defence of accessibility.

Captions can exist and still misrepresent technical language. A document can be readable by assistive technology and remain cognitively exhausting. A platform can meet web criteria while its authentication, assessment or mobile experience creates avoidable barriers.

A rigorous audit must include human testing by learners with different needs. The relevant evidence is not that an accessibility statement exists, but that learners can access the content, participate in the activities and demonstrate achievement without avoidable disadvantage.

A support directory is not a support system

Online providers frequently list numerous support services. The existence of several contact points can create the appearance of comprehensive provision while leaving responsibility dangerously fragmented.

A learner may be passed between a tutor, technical support, student services, an assessment team and an external platform supplier. Each team may complete its own process while no one resolves the learner’s problem. An audit that verifies contact details will miss this failure.

The support pathway must be tested through realistic cases. Who owns the problem? How quickly is it acknowledged? Can it be transferred without the learner repeatedly explaining it? Is the resolution recorded? Does the organisation analyse recurring causes? Quality should be judged at the point of resolution, not at the point of referral.

Corporate L&D audits frequently certify activity rather than capability

Corporate online learning has its own version of performative compliance. A programme is judged successful because employees completed it, recorded enough learning hours or rated it positively. These measures confirm exposure to training. They do not establish learning transfer or improved workplace performance.

The CIPD’s 2023 Learning at Work survey found that only 7 per cent of respondents strongly agreed that their organisation had a process for supporting learning transfer. This suggests a significant disconnect between delivering learning and ensuring that it changes practice.

A corporate learning systems audit should therefore challenge the legitimacy of completion as the dominant measure of success. An employee can complete a cybersecurity course and continue unsafe behaviour. A manager can pass a leadership programme without changing how they lead. A sales employee can perform well in a digital simulation but struggle in an unpredictable client interaction.

The audit must trace the chain from organisational need to learning, practice, application and performance evidence. Where that chain cannot be demonstrated, the organisation has evidence that training occurred, not that capability improved.

Audits are shaped by power and institutional interest

A critical audit must also examine who controls the review. Internal auditors may be expected to evaluate systems designed or commissioned by senior colleagues. Course teams may select the evidence shown to reviewers. Commercial providers may be assessed against contractual measures that prioritise completion, satisfaction or cost efficiency rather than educational depth.

None of this automatically invalidates an audit. It creates structural risks that should be made explicit. The most important question is whether the auditor can challenge the organisation’s operating model or only confirm that the model is being followed.

Can the audit conclude that tutor allocations make quality impossible? Can it challenge an assessment strategy approved by senior leadership? Can it question a platform contract that creates accessibility or data problems? Can learner evidence outweigh an institutionally convenient performance indicator? An audit without sufficient independence may identify local nonconformities while protecting the strategic decisions responsible for them.

Closing actions is not the same as improving quality

The weakest audit ends when recommendations are entered into an action plan. A policy is revised. A template is updated. Staff attend training. The organisation records the action as complete. None of these activities proves that the original problem has been corrected.

If learners misunderstood an assessment, rewriting the brief is not sufficient. The organisation should establish whether misunderstandings and referrals declined. If feedback was delayed, introducing a response target is not sufficient. Subsequent delivery data should show whether timeliness and usefulness improved.

ISO/PAS 25171:2026 now provides specific guidance for auditing ISO 21001:2025, including the use of open questions, objective evidence and measures showing whether conformity is monitored over time. This reinforces an important distinction: quality assurance cannot end with a one time demonstration that a requirement appears to have been met. A credible audit operates through a continuous loop, and an action should close only when there is evidence that the learner experience or educational outcome changed.

The real test of a learning systems audit

A rigorous audit should be able to answer four uncompromising questions:

Frameworks, rubrics and standards remain useful. They offer structure, establish expectations and reduce arbitrary judgement. They become dangerous when they are treated as substitutes for critical inquiry.

The failure of many learning systems audits is not that they examine too little paperwork. It is that they accept organisational claims, convenient metrics and procedural conformity without adequately testing the educational reality beneath them.

A rigorous audit is not a search for reassurance. It is an organised attempt to discover where the learning system is failing, whose interests its definition of quality protects and whether the organisation is willing to change the conditions producing poor outcomes.

The final measure of quality is not that the programme passed an audit. It is that learners were taught well, assessed fairly, supported effectively and left with capabilities that the organisation can credibly defend.