Module 1
Dublin — NovaTech Financial HQ — Friday, 4:30pm
HR Business Partner at NovaTech Financial — a mid-size fintech with 1,200 employees across Dublin, London, and Frankfurt.
Most of the office has left for the weekend. You’re about to close your laptop when a new email arrives — a polite enquiry from a rejected candidate.
This is a decision-driven scenario. You’ll face real decisions that AI compliance professionals encounter — and your choices shape how the story unfolds.
Tip: Look for highlighted text throughout the scenario:
§ Article references — click to read the relevant AI Act article
Key terms — hover for a quick definition
Dear Ms Chen,
I applied for the Senior Risk Analyst role (Ref: NVT-2026-0847) and was notified my application wasn't progressed to interview.
I have 30 years in risk management, 8 in fintech compliance. MSc in Financial Risk Management from LSE, FRM-certified.
I'm not suggesting I'm the best candidate — but given my background, I'd genuinely appreciate understanding what fell short.
Thank you for your time and consideration.
Kind regards,
David Okonkwo
Something about the email nags at you. His CV is strong — 30 years in risk, 8 in fintech compliance, LSE, FRM-certified. For Senior Risk Analyst, he's arguably overqualified.
You open TalentScreen and pull up his application. The tool scored him 72 out of 100 — below the 80-point interview threshold. But it doesn't explain why. Just a number.
You export the rejection data for the last 3 months and sort by age. Your stomach drops.
Of the 11 candidates rejected this round, 9 were over 50 and scored below the threshold. The flags: 'adaptability potential' and 'cultural alignment' — metrics nobody can define. Meanwhile, zero candidates under 35 were rejected. Not one.
| Candidate | Age | Score | Primary rejection flag |
|---|---|---|---|
| D. Okonkwo | 58 | 72 | Adaptability potential (low) |
| M. Petrov | 54 | 69 | Cultural alignment (insufficient) |
| H. Nakamura | 61 | 71 | Adaptability potential (low) |
| A. Balogun | 56 | 68 | Cultural alignment (insufficient) |
| C. Fitzgerald | 52 | 74 | Adaptability potential (low) |
| R. Kowalski | 59 | 70 | Cultural alignment (insufficient) |
| T. Mensah | 63 | 66 | Adaptability potential (low) |
| S. Hoffmann | 51 | 73 | Cultural alignment (insufficient) |
| P. Chandra | 55 | 71 | Adaptability potential (low) |
| L. Müller | 29 | 68 | Progressed (threshold waived — volume hire) |
| J. Park | 31 | 79 | — |
The export confirms what you suspected. Every rejected candidate over 50 lost points on the same two undefined metrics. Under the AI Act, this is not just a fairness question — it's an Article 13 transparency failure and a potential Annex III high-risk classification issue. The system is making employment decisions it cannot explain.
It's just past 5pm. The office is nearly empty. Interviews for 12 shortlisted candidates are scheduled for Monday morning. You have a spreadsheet showing a pattern that could be coincidence or could be systematic age discrimination.
Email James Hartley to pause Monday's interviews until you investigate
The pattern is concerning enough to warrant a pause. If the tool is discriminating, every interview based on its shortlist is tainted. James coordinated 12 candidate schedules — he'll be furious. And you might be wrong.
Add David to the shortlist manually and let interviews proceed
David clearly deserves an interview. You can fix this one case now, investigate the broader pattern next week. The system will do this again on the next hire, but at least David gets a fair shot on Monday.
Go home — you need more data before making accusations
9 out of 11 is a pattern, but it's a small sample. If you raise the alarm and you're wrong, you've undermined a tool the VP championed and damaged your credibility for nothing. David has already been rejected — one weekend won't change that.
James,
I've spotted an anomaly in TalentScreen AI's rejection data. I'd like to walk you through it before Monday's interviews. In person, not over email.
I know it's late notice. I wouldn't raise this if I didn't think it mattered — for the candidates and for our compliance position.
Meet at 8am Monday, before the first interview?
Sarah
Sarah, I've spent two weeks on these interviews. The hiring panel has blocked all of Monday. Three candidates are travelling in. You want to blow it up over a 'data anomaly'? This had better be serious.
It is. I wouldn't ask otherwise. 8am Monday — I'll have the data ready.
Fine. But the interviews aren't getting cancelled. 8 sharp, and they go ahead at 9.
James didn't refuse. You've got the weekend to make your case.
Article 6: recruitment AI is high-risk. Article 26: deployers must monitor and act.
You add David to the shortlist manually. The system flags it: 'Candidate score (72) below threshold (80). Manual override logged.'
Sarah, you added a candidate manually? David Okonkwo — score 72? That's below threshold. What happened?
His CV is exceptionally strong. The score didn't reflect that.
If we override the AI, what's the point of having it? James won't like this.
It's one candidate, Priya. Let's just make sure he gets a fair interview.
Fine. But if anyone asks why we're cherry-picking outside the AI's recommendations, that's on you.
David gets an interview. The other 9 rejected over-50 candidates don't. Article 14 wants systematic oversight, not one-off rescues.
Article 14 requires effective human oversight, including overriding decisions when needed. A single override isn't oversight — it's an exception. Real oversight is a repeatable process.
David Okonkwo
Risk Management Professional
Rejected again. 30 years in risk management. 8 in fintech compliance. MSc from LSE. FRM certified. Didn't even get an interview.
I'm not naming the company. But I'm starting to wonder whether the 'AI-powered recruitment' tools companies are adopting are filtering out experience rather than filtering for it.
Anyone else over 50 seeing this?
1,247 likes
Same experience here. Three rejections in a row from companies using automated screening. 28 years in financial services. Not one interview.
893 likes
I work in HR tech. Some of these tools use 'cultural fit' and 'adaptability' proxies that effectively penalise career stability and age. It's a known problem.
2,341 likes
I'm a journalist at the Financial Times and I'm working on a piece about AI recruitment bias. David, would you be willing to speak with me? DM open.
You close your laptop and go home. David doesn't. By Monday morning the post has 4,200 reactions and 380 comments. Someone has named NovaTech.
James sees it before you do. He's at your desk at 7:30am.
Article 26 requires action on identified risks. You spotted the pattern and chose not to act. Article 4 (AI literacy, enforceable since Feb 2025) requires staff to spot and respond to risks like this.
Regardless of what you did on Friday, the situation has converged. James Hartley is at your desk. He's heard — through Priya, through LinkedIn, or through your email — that you've been 'questioning the AI tool.'
His expression is hard to read. He's not hostile, exactly, but he's guarded. He closes your office door and sits down.
James was the executive sponsor who brought TalentScreen AI to NovaTech. He presented the business case to the board. He personally reported the 40% reduction in time-to-hire. The tool is, in many ways, his project.
Sarah, I'll be direct. The tool works. Time-to-hire is down 40%. The board cited it last quarter. Are you really going to blow this up over one candidate?
It's not one candidate. Nine of eleven rejected candidates over fifty scored below threshold. Zero under thirty-five.
We rejected people under 35 too, just on other rounds. And David's been at one company for 12 years. Maybe the tool read that as low adaptability, not age.
Are you sure you're not seeing a pattern that isn't there? If you raise this and you're wrong, you've told the board their flagship initiative is discriminatory. That's not a bell you can un-ring.
Present the data directly — this is age discrimination, intended or not
9 of 11 over-50 candidates rejected on undefined metrics. Article 26 requires deployers to monitor for discriminatory output. NovaTech, not the vendor, carries the liability.
Agree with James publicly but flag the issue to Legal quietly
You might be wrong, but the risk is too high to ignore. Let Legal investigate while interviews proceed.
Accept James's explanation — career trajectory, not age
'Adaptability' could legitimately track career trajectory, not age. You don't have enough to be certain. Get David an interview and move on.
James, look at this. I've highlighted age, score, and the two metrics that drove the low scores. 'Adaptability potential' and 'cultural alignment.' Neither is defined anywhere in the platform docs. I've checked.
So?
So we're running a high-risk system — recruitment AI is high-risk under Article 6 — and we can't explain its decisions. If David files a complaint, we have nothing to show him.
The vendor assured us the tool was compliant.
Their compliance is their problem. Ours, as deployers, is ours. Article 26 requires us to monitor for risks to fundamental rights. The pattern exists. The question is what we do about it.
What are you proposing?
I want Legal in today. And the vendor's transparency documentation on those metrics. If they can explain it, great. If they can't, we have a bigger conversation.
Fine. But I want to be in the room when Legal reviews this. On the record that I'm cooperating, not being investigated.
Of course. This isn't about blame. It's about getting ahead of a problem before it gets ahead of us.
James shifts from 'you're wrong' to 'what do we do.' You framed it as compliance, not accusation. That's the outcome you needed.
The AI Act splits the world into providers (who build) and deployers (who use). Article 26 requires deployers to monitor output and report incidents. Article 13 requires the system to be transparent enough that you can understand it.
Helen,
I've spotted a pattern in our AI recruitment tool that may indicate age discrimination. 9 of 11 rejected candidates over 50 scored below threshold on metrics with no documentation: 'adaptability potential' and 'cultural alignment.'
James thinks there's a non-discriminatory explanation. He may be right. But recruitment AI is high-risk under Article 6 — Legal should review the data independently.
Sarah
Fair point, James. Career trajectory could explain some of it. I'll dig into the data before raising anything formally.
Good. Let's not turn a coincidence into a crisis. Interviews at 10 — are we good?
We're good.
Helen replies within the hour. You've got your paper trail. But interviews go ahead with a tainted shortlist, and James thinks it's settled.
Article 26 requires action when you spot a risk. Reporting it while the system keeps running isn't acting — it's knowingly letting the risk continue.
You're probably right. Career trajectory is a legitimate signal. I'll get David an interview and we'll watch the metrics.
Sensible. I appreciate that you're thorough — that's why you're good at this. But sometimes a pattern is just a coincidence.
Interviews proceed. David is not among the candidates. Three weeks later, Helen forwards an FT article: 'AI recruitment tools under scrutiny as EU AI Act enforcement begins.' Her note: 'Sarah — are we exposed?'
Now you have to explain that you spotted the pattern three weeks ago and took James at his word. Intent doesn't matter. EU equality law cares about outcomes.
AI discriminates through proxies. 'Career stability' tracks with age. Article 9 says providers AND deployers must spot and mitigate it. The model not using 'age' as an input doesn't matter if the output is discriminatory.
Legal is in. Helen Park has set up a call with Marcus Webb, TalentScreen AI's Head of Product.
Marcus, how are 'adaptability potential' and 'cultural alignment' calculated? What inputs drive them?
Those sit inside our proprietary Talent Compatibility Engine. The weightings are commercially sensitive.
Article 13 says high-risk systems must give deployers enough transparency to understand the output. We're the deployers.
We provide a compliance summary document. I can send that over.
We've read it. 'Multi-factor model incorporating role-relevant competency indicators.' That doesn't explain how 'cultural alignment' is calculated.
What I can offer is our AI Compliance Audit Package — review by our internal team. 6–8 weeks, EUR 30,000.
Six to eight weeks?
They can't or won't explain their own tool. That's an Article 13 problem — theirs and ours.
It's a black box. 'Proprietary' isn't a defence — Article 13 demands transparency. And they want EUR 30,000 to audit themselves.
Sarah, EUR 200,000 in the platform. The vendor wants EUR 30,000 more. Three open roles, and the CFO is going to ask why time-to-hire is climbing. What are you recommending?
Suspend the tool until the vendor produces Article 13 documentation
If you can't explain its decisions, you can't defend them. Accept the political cost. The discrimination stops today, not in 6–8 weeks.
Keep the tool, but human-review every AI rejection
Every below-threshold candidate gets manual review. Anyone over 50 who fails on 'adaptability' or 'cultural alignment' escalates to senior HR.
Buy the EUR 30,000 vendor audit and keep using the tool
The audit will confirm whether there's a real problem. Six to eight weeks isn't ideal — but better than suspending a tool that saves 200 hours a quarter on an unverified pattern.
James, I'm recommending we suspend TalentScreen AI. Effective immediately. I'll draft the formal recommendation for Helen and the CFO today.
Immediately? We run 200 applications a month through it. We'll be back to manual screening — that's the 200 hours a quarter I saved us.
I know. But Article 99 allows fines up to EUR 15 million or 3% of global turnover. NovaTech's turnover was EUR 340 million. Three percent is EUR 10.2 million.
That's the maximum. No regulator is going to fine us EUR 10 million for a recruitment tool.
Even 1% is EUR 3.4 million. Before reputational damage. If the FT runs a story on NovaTech's AI discriminating against older candidates, what happens to Frankfurt's regulator relationships?
How long?
Until we have transparency documentation we can review. If the vendor can explain the algorithm, we turn it back on. If they can't, we find a vendor who can.
The board is going to want to know why.
Better they hear it from us than from a regulator.
The hardest call and the most defensible. You gave James a clear path back: not banned, just suspended until transparency. Proportionate. Professional.
Article 26 requires suspension when there's reason to believe the system threatens fundamental rights. Article 99 sets penalties up to 3% of turnover. Suspending the moment you spotted the risk is strong evidence of good faith.
Keep the tool, but human-review every rejection. Anyone over 50 below threshold on 'adaptability' or 'cultural alignment' escalates to senior HR.
That's more work for your team.
Less work than a regulatory investigation. And screening keeps running while we push the vendor for transparency.
Reasonable interim measure. Doesn't fully satisfy Article 14 — oversight must be effective, not performative. If reviewers rubber-stamp AI scores, we're exposed.
Agreed. Reviewers won't see the AI score until after their own assessment. Blind review first, then comparison.
Better. But we still need vendor transparency documentation. This is temporary, not permanent.
Pragmatic compromise. But you're adding human oversight to compensate for a system you can't explain. Article 14 wants oversight that lets you understand the system — which you still can't.
Article 14 wants overseers who understand the output and catch anomalies. Without scoring logic, you're stuck detecting patterns — you can't do root cause. Article 13 transparency? Still unresolved.
The audit is the right path. The vendor knows their system best. Six to eight weeks is manageable.
Sarah, I have concerns. We're asking the vendor to audit themselves. That's a conflict.
They have internal compliance people. It's standard practice.
Not one regulators accept. If we end up in front of a national authority, 'we paid the vendor to audit themselves' won't inspire confidence.
If the audit comes back clean — which it almost certainly will — and a regulator later finds the same pattern you found, where does that leave us?
Let's just do the vendor audit and move on.
A vendor auditing themselves rarely finds problems. Meanwhile the tool screens for another 6–8 weeks. Article 9 requires independent bias testing.
Article 9 requires risk management that identifies AND mitigates discrimination. Self-audits are conflicted by design. An independent third-party audit is far more defensible.
Monday, 8:15 AM
Your Friday email worked. But overnight, things escalated.
Before we start — the board approved Frankfurt expansion on Friday. TalentScreen will handle recruitment across all three offices. Contract signed at 4pm.
That changes the compliance picture significantly. Cross-border deployment of a high-risk AI system triggers additional obligations.
The vendor assured us it's compliant in all EU jurisdictions. Helen signed off. Are you saying the CEO made a mistake?
TalentScreen is now processing candidates across three EU jurisdictions. James has board backing. Helen signed the contract. What do you recommend?
Commission an independent conformity assessment under Article 43 and suspend cross-border deployment until complete
Cross-border expansion is a substantial modification. The vendor's self-assessment doesn't transfer. Article 26(5) requires suspension if you have reason to believe it presents a risk.
Implement a human review panel for all AI-rejected candidates while keeping the tool operational
Address the immediate bias risk with Article 14 human oversight. Maintains operational continuity. Review the conformity question in parallel.
Proceed with Frankfurt deployment with enhanced bias monitoring dashboards
The data shows a potential issue, not a proven one. Put monitoring in place to detect problems early rather than disrupting a board-approved expansion.
James, the Frankfurt expansion isn't just a new office — it's a new jurisdiction. The vendor's self-certification was for UK deployment. Cross-border use is a substantial modification under Article 43. We need an independent conformity assessment before TalentScreen processes a single Frankfurt candidate.
You're telling me to suspend a tool the board approved 72 hours ago. Do you have any idea what that conversation looks like?
I know exactly what it looks like. It looks like the compliance team doing their job before the German regulator does it for us. The BfDI doesn't accept "the vendor said it was fine" as a defence. And the potential fine is up to €15 million or 3% of global turnover — whichever is higher.
(long pause) How long does this assessment take?
Eight to twelve weeks if we engage an accredited body this week. I'll have a shortlist of assessors on your desk by end of day.
James just messaged me. Sarah, is this as serious as you're suggesting?
Helen, I'd rather explain a ten-week delay to the board than a regulatory investigation to shareholders. I'm recommending an emergency board briefing this Thursday.
...Book it. And get me a one-page brief by Wednesday evening.
You spotted the distinction most professionals miss. The vendor's self-certification doesn't follow you when the deployment context changes. Cross-border expansion is a substantial modification — that triggers new conformity obligations. James is frustrated, but Helen's reaction tells you the board listens when you quantify the risk.
When a high-risk AI system gets substantially modified — including being deployed in a new country — you may need a fresh conformity assessment. The original assessment covers the original deployment context only. Cross-border expansion changes the regulatory landscape, the data protection framework, and the risk profile.
I'm proposing a human review panel. Every candidate TalentScreen rejects gets reviewed by a trained assessor before the decision is finalised. We keep the tool running, but no one falls through the cracks.
Now that's more reasonable. How many extra hours are we talking?
About 30 hours per quarter across three reviewers. Far less than going back to fully manual screening.
I can live with that. Set it up. And this resolves the compliance issue?
Sarah, I've reviewed your proposal. Human oversight addresses Article 14, and I support the panel. However — I need to flag that cross-border deployment to Frankfurt may require a fresh conformity assessment under Article 43. Human review doesn't resolve that question. We should discuss.
...Understood. I'll set up time with you this afternoon.
James is relieved — you found a solution that doesn't blow up the expansion. But Priya spotted the gap you missed. Human review treats symptoms of the bias problem. It doesn't tell you whether the system is lawfully deployed in a new country. If a German regulator asks for conformity docs, "we added human review" doesn't cut it.
Human oversight is a core requirement for high-risk systems. But it's one obligation among many. A review panel can catch discriminatory outputs, but it can't explain discriminatory logic. If the conformity status is uncertain, oversight alone doesn't get you off the hook under Articles 26 and 43.
Good news — the Frankfurt pipeline is already filling. TalentScreen processed 45 candidates in the first batch. The dashboards look clean.
That's... good to hear. What's the rejection profile looking like?
Haven't dug into the details. The dashboard says bias indicators are within normal range. Why?
That afternoon, Priya forwards you an email. A 54-year-old candidate in Frankfurt, rejected by TalentScreen with a score of 68, has filed a complaint with the Hessian data protection authority. His lawyer references the EU AI Act directly. He wants to know how "adaptability potential" was calculated and why his 22 years of banking experience scored lower than graduates with two years.
Sarah, I need the transparency documentation for TalentScreen's scoring methodology. The candidate's lawyer has given us 14 days to respond. Do we have it?
...No. We don't.
Commercial pressure won. You expanded a potentially discriminatory system into a new country while hoping dashboards would catch what you'd already spotted. The dashboards measured what TalentScreen chose to show — not the metrics driving the discrimination. Article 26(5) says you have to suspend systems you've got reason to believe are risky. You had that reason two weeks ago.
If you've got reason to believe a high-risk system is dangerous, you have to suspend it and tell the provider. "Enhanced monitoring" doesn't count. The age discrimination pattern in the UK data was the reason — and it followed TalentScreen straight to Frankfurt.
Wednesday, 2:00 PM — The Vendor Pushes Back
Our legal team reviewed Article 6. TalentScreen recommends — it doesn't decide. Under Article 6(3), we're not high-risk.
Article 6(2) references Annex III directly — which lists "AI systems intended to be used for recruitment" without qualifying the automation level.
We have clients in 14 EU countries. None have raised this. Your own legal team signed off on our compliance pack.
The board meets Thursday. If we suspend, I explain why we're back to manual screening at 200 hours per quarter.
The vendor claims they're not high-risk. Your legal counsel agreed six months ago. The board meets tomorrow. What do you recommend?
Present a formal risk assessment to the board — recommend a 90-day compliance programme with independent audit
Accept the commercial cost. The vendor's Article 6(3) argument has merit but creates unacceptable risk if a regulator disagrees. A fundamental rights impact assessment under Article 27 is required regardless.
Keep TalentScreen but require human review of ALL decisions, plus quarterly bias audits
Pragmatic middle ground. Human review satisfies Article 14, bias audits demonstrate diligence. The board keeps their tool, candidates get oversight. Not perfect, but defensible.
Accept legal counsel's position that TalentScreen is "decision-support" and not high-risk — document your concerns formally
Your legal team cleared it. The vendor has 14 EU clients. Maybe the Article 6(3) interpretation is correct. Document concerns to protect yourself, but don't blow up a board-approved strategy.
Let me make sure I understand. You're asking the board to approve a 90-day pause on a tool I personally signed off on, less than a week ago.
I'm asking the board to approve a compliance programme that protects a €340 million company from a regulatory action that could cost €15 million. The tool works. The question is whether it works lawfully across three jurisdictions. Right now, we can't prove it does.
And the vendor's position that they're not high-risk?
The vendor's interpretation has arguable merit under Article 6(3). But if a regulator disagrees — and the European AI Office has signalled that recruitment tools will be scrutinised early — we bear the risk as deployers. Not them. The independent audit settles the question before a regulator asks it.
For the record, I think this is overcautious. But I understand the logic.
(to the board) I'm approving the 90-day programme. Sarah, I want weekly progress reports. And I want the independent assessor's name on my desk by Friday.
You picked compliance integrity over commercial convenience — even when your own lawyer disagreed and the CEO was hostile. Helen's anger turned to respect once you quantified the risk. A 90-day programme with independent audit is the gold standard. It costs political capital today. But it's the position you want when the regulator calls.
Deploying high-risk AI in employment? You have to do a fundamental rights impact assessment before you switch it on. That obligation is on you, not the vendor. An independent audit programme satisfies it and creates the compliance record you'll want later.
I'm proposing we keep TalentScreen operational with two conditions: mandatory human review of every decision, and quarterly bias audits conducted by an external firm. The tool stays. The candidates get protected.
That I can sell to the board. We keep the efficiency gains and show we're taking oversight seriously.
Approved. Sarah, make sure the first audit is completed before the Frankfurt office opens in Q3.
The board accepts. James is visibly relieved. Six months later, an email arrives from the European AI Office — a sector-wide regulatory inquiry into AI recruitment tools. They want conformity documentation, fundamental rights impact assessments, and evidence of Article 13 transparency compliance.
Sarah, the human review logs and bias audits help. They show good faith. But they're asking for a conformity assessment we never did and a fundamental rights impact assessment we never conducted. We have 30 days to respond.
Pragmatic, but not bulletproof. Human review plus audits is defensible — it shows diligence and catches individual cases. But it dodges the real question: is the system lawfully deployed? You've bought time, not compliance. The compromise helps. It isn't airtight when the regulator comes knocking.
Human oversight (Article 14) is one obligation among many. It addresses output risk, not systemic compliance. A regulator won't accept "we reviewed every decision" instead of conformity documentation (Article 43) or a fundamental rights impact assessment (Article 27). The compromise reduces harm but doesn't kill the legal exposure.
You documented your concerns in a formal memo to Priya three months ago. Filed it. Moved on. TalentScreen kept processing candidates across all three offices. The age pattern continued — 73% of rejected candidates over 50 scored below threshold on "adaptability potential." You saw the quarterly report. You said nothing.
The European AI Office announces an investigation into recruitment AI across the financial services sector. NovaTech is on the list. Priya calls an emergency meeting.
They want everything. Conformity assessment, fundamental rights impact assessment, transparency documentation, deployment logs. We have 60 days. Sarah — what do we actually have?
We have the vendor's original compliance pack and my memo from three months ago flagging the concerns.
So you identified a risk, documented it, and the system kept running for three more months. That memo doesn't protect the company, Sarah. It incriminates us. It proves we knew.
How did we get here?
CYA isn't compliance. Documenting concerns protects you personally in a narrow way — and actively harms the organisation by creating a paper trail of known, unaddressed risk. "My legal team said it was fine" isn't a defence when deployers carry their own obligations. And every candidate processed after your memo is one NovaTech knowingly exposed to a potentially discriminatory system.
Deployers have their own obligations under the AI Act. Deferring to the vendor's lawyer doesn't get you off the hook. Document a risk without acting on it, and you've built the worst possible regulatory position: proven knowledge plus continued deployment. The AI Act doesn't treat "I wrote a memo" as risk mitigation.
In the previous situation, the key issue was recognising that TalentScreen AI processes job applications.
Article 6 classifies AI in employment as high-risk. This triggers: human oversight (Art. 14), transparency (Art. 13), data governance (Art. 10), and risk management (Art. 9).
As a deployerDeployerAn organisation that uses an AI system under its authority — as opposed to the provider who built it. Under the AI Act, deployers carry their own compliance obligations., NovaTech has independent obligations under Article 26 — even if the vendor claims compliance.
Remember: TalentScreen is high-risk under Article 6. You have obligations under Article 26.
Monday morning. James pushes back — the tool saved 200 hours/quarter. What do you do about the bias pattern?
Present the bias data and recommend pausing the tool until the vendor provides transparency documentation
Article 26(5) says deployers must suspend if they believe there's a risk. The data suggests age discrimination. Article 13 documentation should explain how the tool decides.
Add a human reviewer to check all AI rejections before they're finalised
Human oversight (Article 14) is required for high-risk systems. This catches discriminatory rejections before they affect candidates.
Wait for more data — one pattern doesn't prove discrimination
Maybe the pattern is coincidental. Acting too quickly could damage your relationship with the board.
James, I need you to look at this. Nine of eleven rejected candidates over 50 scored below threshold. Zero candidates under 35 were rejected. The two metrics driving the scores — "adaptability potential" and "cultural alignment" — aren't defined anywhere in the vendor's documentation.
It saved us 200 hours last quarter, Sarah. You want me to go back to the board and say we're pausing it because of a spreadsheet?
I want you to go back to the board and say we caught a potential age discrimination pattern before a candidate's lawyer did. Under Article 26, we're required to suspend a system when we have reason to believe it presents a risk. This data IS that reason.
(long silence) ...How long?
Until the vendor provides proper transparency documentation under Article 13. If they can explain how those metrics work, and the explanation is non-discriminatory, we turn it back on. If they can't — we've dodged a bullet.
Fine. But you're presenting this to Helen. And you'd better have the maths ready.
James reluctantly agrees. He's not happy, but you've framed the risk in terms he can't ignore — a candidate's lawyer finding the pattern first. You have the weekend to prepare a formal brief for Helen. The tool is paused. No more candidates will be processed until you have answers.
When a deployer has reason to believe a high-risk AI system presents a risk to health, safety, or fundamental rights, they must suspend its use and inform the provider. The bias data — 9 of 11 over-50 candidates rejected on unexplained metrics — constitutes that reason. Requesting Article 13 transparency documentation is the correct next step.
I'm recommending we add a human reviewer to check every AI rejection before it's finalised. No candidate gets screened out without a person confirming the decision.
That's reasonable. We keep the tool, candidates get a second look. How quickly can you set it up?
By end of week. Three trained reviewers, rotating on a schedule.
Good. Problem solved.
The review panel catches three more questionable rejections in the first two weeks — all candidates over 45, all scoring low on "adaptability potential." The reviewers override the AI and advance them. But something nags at you: the reviewers can see that the tool rejects certain candidates. They can't see why. They're catching symptoms. The system still can't explain its logic.
We're putting a safety net under a bridge we're not sure is structurally sound. If someone asks how "adaptability potential" is calculated, we still can't answer.
Human review addresses Article 14 and catches individual cases of bias. But the underlying system remains a black box. Without transparency documentation under Article 13, you can't explain why the tool makes the decisions it does — only that you sometimes disagree with the output.
Human oversight is a core requirement for high-risk systems — so this instinct is right. But Article 14 works alongside Article 13 (transparency), not instead of it. A human reviewer who can override decisions but can't understand the system's logic has limited ability to identify systemic discrimination versus individual errors.
Another round processed. Twelve new candidates. Three rejections — all over 50. All scored below threshold on "adaptability potential." The pattern isn't one pattern anymore. It's two.
You pull the full data. In two months of TalentScreen operation, 14 of 16 rejected candidates over 50 scored below threshold on the same opaque metric. Zero candidates under 35 have been rejected. You open LinkedIn to distract yourself and freeze.
Have you seen David Okonkwo's LinkedIn post? It's got 400 comments. He's naming us. Well, not us specifically — but "a Dublin fintech using AI to screen out experienced candidates." His former colleague at Barclays just shared it.
I saw it.
Helen wants a meeting. Today. She's asking what we knew and when we knew it.
...I flagged the pattern two weeks ago. I was waiting for more data.
You flagged it to ME. And I told you to wait. Helen's going to want to know why neither of us escalated.
Waiting was not compliance. Nine of eleven was already a pattern — 14 of 16 is a crisis. Every day you waited, more candidates were potentially discriminated against. Under Article 26(5), you had reason to believe the system presented a risk to fundamental rights. The obligation was to act, not to gather a statistically perfect dataset.
The threshold is "reason to believe" — not "proof beyond reasonable doubt." When 9 of 11 candidates in a protected category score below threshold on an unexplained metric, that IS reason to believe. The AI Act doesn't require you to complete a peer-reviewed study before acting. It requires you to protect fundamental rights when you have credible evidence of risk.
Deployers and providers have separate obligations. Even if TalentScreen claims compliance, NovaTech has its own duties:
Article 26: Deployers must use the system per instructions, ensure human oversight, monitor risks, keep logs, and suspend if there's a risk.
The vendor saying "we're compliant" doesn't discharge YOUR obligations.
Remember: As deployer, NovaTech has independent obligations under Article 26.
The vendor wants €30,000 for a transparency audit. James says the CFO won't approve it. What do you recommend?
Suspend the tool until the vendor provides proper documentation
The potential fine (up to €15M or 3% of turnover) far exceeds manual screening costs. Article 26(5) requires suspension if you believe there's a risk.
Keep the tool but add human review of every rejection and document everything
Addresses immediate risk. Human review satisfies Article 14. Documentation shows good faith.
Do nothing — the vendor has 14 EU clients and none have had issues
Maybe you're overreacting. The vendor seems confident and your legal team cleared it.
The CFO won't approve €30,000 for an audit we might not need. And now you want to suspend the tool entirely? We'll be back to 200 hours of manual screening per quarter.
Let me give you different numbers. Potential fine under the AI Act: up to €15 million or 3% of global turnover. NovaTech's turnover last year was €340 million. Three percent is €10.2 million. Manual screening costs £48,000 a year. Which number do you want to present to the CFO?
(pause) ...You've done the maths.
I have. And I've drafted a one-page brief for Helen. The recommendation is a temporary suspension while we require the vendor to provide proper Article 13 documentation. If they comply, we could be back online in four to six weeks. If they can't explain their own system, we find a vendor who can.
Four to six weeks I can live with. A €15 million fine I cannot. Send me the brief — I'll co-sign it.
Suspension is the right call. James is unhappy but the maths is irrefutable — £48,000 in manual screening versus €10.2 million in potential fines isn't a close decision. The tool is paused. The candidates are protected. And you've positioned this as temporary, not permanent — giving the vendor a clear path to reactivation through compliance.
Suspension isn't punishment — it's risk management. The AI Act requires deployers to suspend high-risk systems when they have reason to believe they present a risk to fundamental rights. The cost of temporary manual processes is always lower than the cost of regulatory enforcement. Framing suspension as a business decision, not a compliance lecture, is what gets buy-in.
The review panel has overridden 7 of 43 rejections in three weeks. All seven were candidates over 45. All scored low on "adaptability potential." The human reviewers are catching the worst outcomes.
James is satisfied. The board received your documentation showing proactive oversight. The candidates who would have been unfairly rejected are getting interviews. On paper, the system looks responsible.
The review panel is working. We caught seven bad decisions. I'd call that a success.
We caught seven outputs we disagreed with. We still don't know why the system generates them. If a regulator asks how "adaptability potential" is calculated, we can show them our override logs. We can't show them how the AI works.
Isn't that the vendor's problem?
...That's the part I'm not sure about.
Human review plus documentation is better than nothing — significantly better. You're catching discriminatory outputs and creating an audit trail that shows good faith. But the underlying system is unchanged. The AI still processes candidates the same way. You're filtering its decisions, not fixing its logic. If a regulator determines the system itself is non-compliant, your workarounds don't satisfy the full conformity requirements.
High-risk AI systems must be designed with sufficient transparency to enable deployers to interpret and use outputs appropriately. If you can't explain how "adaptability potential" is calculated, you can't satisfy this requirement — regardless of how many human reviewers you add. Oversight without understanding is damage limitation, not compliance.
Two months of silence. TalentScreen keeps processing. You stopped checking the rejection data. The vendor has 14 EU clients. Your legal team cleared it. Maybe you were overreacting.
Have you seen LinkedIn this morning? David Okonkwo just posted a 1,200-word essay about age discrimination in fintech hiring. He names AI screening tools. He doesn't name us specifically but the details are unmistakable. It's already got 2,000 reactions.
I'm reading it now.
The post is devastating. Okonkwo writes about 22 years of banking experience, a stellar track record, and being rejected by an AI tool that scored his "adaptability potential" at 31 out of 100. A journalist from the Financial Times has already commented asking to DM. By lunchtime, the post has 8,000 reactions and three former NovaTech candidates have replied with similar stories.
I need everyone on this call to tell me: did we know about this? Was there any indication our AI tool was discriminating against older candidates?
(silence)
...I identified a statistical pattern two months ago. I decided to wait for more data before escalating.
You knew. For two months. And the system kept running.
"Everyone else is doing it" was never a defence — and now it's a crisis. The vendor's other clients haven't been publicly named by a rejected candidate with 15,000 LinkedIn followers. Your legal team's clearance was based on incomplete information. Under the AI Act, ignorance you could have corrected is not a defence. And you had the data to correct it two months ago.
The AI Act imposes independent obligations on deployers. "The vendor has 14 EU clients" and "our legal team cleared it" are not defences when you have evidence of risk. Inaction in the face of known risk is itself a compliance failure. The reputational damage — a viral LinkedIn post, FT interest, candidates comparing notes publicly — compounds the regulatory exposure.
The decisions you made as Sarah Chen rippled outward — to David Okonkwo, to NovaTech's board, to the next 200 candidates. Here's what happened.
Article 4
AI Literacy
Article 6 + Annex III
High-Risk Classification
Article 9
Risk Management
Article 13
Transparency
Article 14
Human Oversight
Article 26
Deployer Obligations
Article 50
Transparency for Users
Article 99
Penalties
Want to see how your team stacks up? Ask L&D for the leaderboard.
Next up, you're Elena Vasquez at NovaTech's marketing team. A journalist's got 48 hours. The chatbot's been promising things it shouldn't. And nobody's written the AI disclosure policy.
Supplemental Resource
A printable summary of every article covered across all five modules. Save as PDF for offline reference.
Module 1 Complete
You navigated the compliance dilemma. Try a different path to see how the story changes.