Proctor Quality and Learning Manager, Scaling Operations

• Our mission at Duolingo is to develop the best education in the world and make it universally available. It's a big mission, and that's where you come in!
• At Duolingo, you'll join a team that cares about educating our users, experimenting with big ideas, making fact-based decisions, and finding innovative solutions to complex problems.
• You'll have limitless learning opportunities and daily collaborations with world-class minds — while doing work that's both meaningful and fun.
• Join our life-changing mission to develop education for our half a billion (and growing!) learners around the world.
• Read our blog to learn more.

About the role

• The Duolingo English Test (DET) is a high-stakes assessment used by institutions around the world.
• Its integrity depends on proctors making accurate, consistent, and defensible decisions — even when sessions present ambiguous evidence or new forms of cheating.
• As the Proctor Quality and Learning Manager (L3) in Scaling Operations, you will own the end-to-end quality standard and learning system that keep our global proctor network aligned and effective as security threats, technology, and workflows evolve.
• You'll define what "correct" looks like, run our calibration and adjudication program, and build ongoing training that measurably improves decision quality.
• You will partner with Psychometrics, Security, Operations, Product, and Engineering to ensure our human and AI-assisted workflows meet rigorous accuracy standards before and after deployment.
• This is an individual-contributor role within DET Proctoring Operations, with functional leadership over Training Specialists and adjudicators executing the program.

🧠 You will...

• Own the DET proctoring quality standard — maintain rubrics, decision rules, examples, and guidance that define accurate decisions across routine and difficult edge cases.
• Lead calibration and adjudication — operate recurring Proctor Certification Test cycles, resolve disputed decisions, identify patterns of disagreement, and ensure outcomes are consistent and auditable.
• Build and run an ongoing learning program — publish a forward-looking training calendar, deliver monthly interventions, and launch targeted training in response to quality signals, threats, or workflow changes.
• Measure effectiveness — use calibration results, audits, and performance trends to identify learning needs, evaluate training impact, and decide when to clarify standards, retrain, or update guidance.
• Guide specialist execution — set expectations for Training Specialists and adjudicators, assign deliverables, review outputs, and operate onboarding and certification that ramp new specialists quickly.
• Maintain durable sources of truth — keep calibration, adjudication, and training runbooks current so critical processes are documented and auditable.
• Evolve quality alongside proctoring — translate new threats and workflow changes into clear standards, training, and quality controls in partnership with Psychometrics, Security, Operations, Product, and Engineering.
• Represent the proctoring craft — bring frontline proctor evidence into product experiments and workflow decisions; assess whether launched changes preserve decision quality.
• Connect human and technology-assisted quality — establish benchmarks for AI-assisted workflows and validate automated capabilities against real sessions before deployment.

✅ You have...

• 5+ years in assessment development, scoring/rater quality, certification, quality assurance, or a field involving complex, consequential judgments; experience aligning reviewers to a shared standard through calibration programs.
• Adult-learning program experience spanning onboarding, certification, ongoing training, remediation, and effectiveness measurement (beyond one-off materials).
• Exceptional attention to detail and strong judgment in ambiguous cases; ability to produce clear, defensible decisions and guidance.
• Analytical fluency to interpret quality data, disagreement patterns, and performance trends and translate them into practical interventions.
• Clear written and verbal communication; ability to turn complex standards into precise, usable guidance and explain difficult calls.
• Cross-functional leadership without formal authority; experience directing contractors or partners toward clear accountability and outcomes.
• Demonstrated skill in quality standards and rubric design, building living systems with decision logs, change histories, and controls for auditability.
• Experience defining quality benchmarks and validation methods for human–technology workflows, including AI-assisted processes.
• Adaptability — comfort revising standards and programs as risks, products, and technology change.

⭐ Exceptional candidates will have...

• Experience in language assessment, test security, remote proctoring, trust and safety, or fraud prevention in high-integrity environments.
• Experience maintaining quality across large, distributed examiner/rater/reviewer/contractor populations.
• Familiarity with assessment concepts such as inter-rater agreement, reliability, standardization, and performance monitoring.
• Experience designing quality systems that span human judgment and technology-assisted workflows.
• Familiarity with learning-management systems, assessment platforms, quality-audit tools, or operational analytics.

• The offered salary is dependent upon several factors, including work experience, skills, and internal peer comparisons.
• The posted range is subject to change in the future.
• For this role, base salary is supplemented by equity compensation.
• We encourage you to talk with your recruiter for more information related to compensation for this role!

Salary Range:

£90,630 — £122,617 GBP

Benefits

Take a peek at how we care for our employees' holistic well-being with our benefits here .

We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.

• Duolingo and our employees will never ask for your Social Security number, bank details, or passport info, and we'll never ask you to deposit a check, purchase equipment, or exchange money during the interview process.
• Real Duolingo employees always use an email that ends in @duolingo.com or @recruiting.duolingo.com.
• Stay alert and double-check these details before sharing any information.
• By applying for this position your data will be processed as per the Duolingo Applicant Privacy Notice .