Online proctoring is essentially overseeing tests or exams, using a computer or a combination of a computer and a remote proctor (an approved invigilator supervising remotely).
By Arpinder Singh and Vineet Mehta
With the conventional form of classroom teaching being suspended due to COVID-19, educational institutes have ramped up digital learning alternatives to respond to the new normal. The impact of the pandemic is likely to extend for several months as the world continues its struggle to contain it. In these challenging times and amidst an already delayed academic year, educational institutions are faced with several dilemmas, including administering exams online without compromising on the integrity of the process. The use of online proctoring platforms can be a practical solution to the predicament.
Online proctoring is essentially overseeing tests or exams, using a computer or a combination of a computer and a remote proctor (an approved invigilator supervising remotely). Its key proposition is that exams can be taken from any location without compromising on the security and integrity of the process. The technology involves using advanced decision-making algorithms (machine learning and artificial intelligence) to detect possible instances of cheating by continuously monitoring the candidate’s live feed and screen recording for any anomalies. There are mainly three ways to conduct it: live online proctoring (monitored by a combination of a remote proctor and computer), recorded proctoring (monitored by a remote proctor after the exam is finished), and advanced automated proctoring (monitored by a computer only).
Educational institutions in India have several platforms available today and can choose one that resonates with them from an operational, fairness, accountability and privacy perspective.
Operational: Algorithmic decision-making in online proctoring to detect cheating is not absolute or conclusive. It is typically advisory in nature as algorithms factor in only some of the key input data points and not all. This means that a high dependence on the output of such algorithms may lead to decisions based on inconclusive evidence. An alternative here can be using a closed-circuit camera (CCTV) feed that can corroborate the authenticity of the exam being undertaken. To establish confidence on the online proctoring platform’s ability to maintain exam integrity, accuracy measurements of the algorithms are often published. However, accuracy may not be the right metric to evaluate such algorithms, as even a high accuracy algorithm may not fare well at identifying instances of cheating. This is likely because the training data at the initial design stage may not be exhaustive. Reliability of such algorithms is a function of the entire development lifecycle of the algorithm encompassing design, training data, validation, precision and recall. Non – exhaustive Decision-making algorithms are prone to evasion attacks and can lead to misclassification of the output. A minor change to an input pattern (e.g. an image from the webcam) by the student who wants to cheat can make decision-making algorithm to misclassify the incidence as ‘no cheating’. For educational institutes, it is imperative to assess the possible extent of anomalies detected by the online proctoring platform. Many platforms rely on one webcam facing the student; however, it is important to have a 360degree check during the exam. Similarly, sound, objects and text can also become important to detect any instance of cheating.
Fairness: As per media reports, research studies have been conducted to assess if a decision-making algorithm is biased against certain groups. Given India’s diversity, the algorithm should be fair toward all demographic groups (based on race, gender, age, religion and colour). Such algorithms require the representation of diverse demographic data during the initial development stage. Continuous improvements should also be made to improve and minimize any errors found so that all students are treated equally and without any discrimination.
Privacy: Online proctoring platforms capture student data, which is an important from a privacy standpoint. Credentials may include government issued photo identity cards, photographs and background, among others. It is important to understand aspects of data consent, collection, storage, anonymity and possible information sharing with third parties.
Accountability: Cases of cheating often get challenged in courts and probed on how the decision-making algorithm has reported red flags. However, the technicalities of the algorithms, detailing complex code and statistics may be difficult to explain during litigation proceedings. Therefore, the process and model should be auditable to identify responsibilities if there is a failure. This will foster trust, accountability and transparency in the process and platform.
Media reports highlight that the Bureau of Indian Standards is working to build a framework for the standardization of big data analytics, including decision-making algorithms. This move can set the basic standards and quality requirements, acting as a benchmark for online proctoring solution providers. The standard can also become a powerful tool in reinforcing trust and confidence around these models and propel digital learning to new heights. For educational institutes, evaluating the technology platform and conducting regular audits will be paramount. While human intervention will be a critical part of the decision-making process, online proctoring can become a viable option to conduct exams with integrity while maintaining social distancing given the current pandemic.
Arpinder Singh is Partner and Head – India and Emerging Markets, Forensic & Integrity Services, EY and Vineet Mehta is Associate Partner, Forensic & Integrity Services, EY