Deepfakes & Generative AI Risks

Deepfakes and Generative AI Risks

 
Risk & Defence Reference

Deepfakes & Generative
AI Risks

A complete, deeply researched reference guide — from what deepfakes are and how they work, to the real-world damage they cause, who is at risk, and what organisations and individuals can do to protect themselves.

20 SectionsDeep Coverage4 SVG FiguresVisual Explainers20 SourcesCited ReferencesJune 2026Current Edition
01
Foundations

What Is a Deepfake?

A deepfake is a piece of synthetic media — a video, audio recording, image, or written text — artificially created or altered using AI to look, sound, or read as though a real person said or did something they never actually said or did.

A deepfake is a digitally fabricated media artefact so convincingly realistic that it can deceive viewers into believing a real person said, appeared in, or created something that never happened — manufactured with generative AI tools now accessible to virtually anyone with a smartphone.
— Synthesised from KPMG, UNESCO, and DHS research, 2024–2025

The Simple Explanation (For a 10-Year-Old!) 🧒

Imagine you have a very talented actor friend who can copy anyone’s voice and face perfectly. Now imagine that friend makes a video of your teacher saying something embarrassing — something the teacher never actually said. If it looks and sounds convincing enough, other children might believe it is real. That is exactly what a deepfake is, except the actor is a computer program trained on thousands of hours of real footage and audio. The alarming part is that the computer can now do this in minutes, for almost no cost at all.

🔤 Breaking Down the Word

Deep — comes from “deep learning,” the AI technique used to generate these fakes. Fake — exactly what it says: the content is fabricated even though it appears authentic. Together, the word captures how a sophisticated AI method is weaponised to create convincing falsehoods about real people.

3s
Audio needed to clone a voice
70%
People who cannot spot fake voice
$1T
Projected deepfake fraud cost
$1.33
Average cost to make a deepfake
⚠️
Why This Matters Right Now

Deepfake creation tools that once required a specialist research team and months of work now run on a consumer laptop in minutes. Open-source models, low-cost APIs, and “deepfake-as-a-service” platforms on the dark web have lowered the barrier so far that malicious actors with no technical background can launch sophisticated impersonation campaigns. The threat is no longer theoretical — it is actively causing financial fraud, reputational destruction, and democratic harm today.

02
Foundations

How Deepfakes Are Made

Understanding the technical mechanism behind deepfakes is the first step toward defending against them. The process is powered by deep learning techniques that, when combined, enable astonishingly convincing synthetic media.

The Core Technology: GANs and Diffusion Models

Most deepfakes are generated using one of two AI architectures. The older, still widely used technique is the Generative Adversarial Network (GAN), which pits two neural networks against each other: one generates fake content while the other tries to detect it as fake. Through this adversarial competition, both networks improve until the generated content becomes indistinguishable from genuine material.

Newer deepfakes increasingly use Diffusion Models — the same technology behind image generators like Stable Diffusion and DALL-E — which work by gradually removing random noise from a data sample until a coherent image or audio file emerges. Diffusion models produce higher-quality, more photorealistic results and are now the preferred technique for state-of-the-art deepfake creation.

FIG 01 — THE DEEPFAKE PRODUCTION PIPELINE🎥SOURCE DATAReal footage& audio→🧠TRAIN AIGAN or Diffusionmodel learns face→🎭FACE SWAPTarget’s likenessplaced on media→✅OUTPUTConvincing fakevideo/audio→⚠️DEPLOYEDas weaponTOTAL CREATION TIME IN 2026: AS LITTLE AS 5–10 MINUTES ON CONSUMER HARDWARE
Fig 01 — From a few seconds of real footage to a deployable weapon of deception in minutes. The entire pipeline now runs on an ordinary laptop.
🔊 Voice Cloning: The Fastest Threat

Voice cloning deserves special attention because it requires the least raw material. Credible synthetic speech can be produced from as little as three seconds of someone talking. This means that any phone call you have ever had recorded, any podcast you appeared in, any video you posted publicly, provides enough raw material for a criminal to clone your voice and impersonate you convincingly to your bank, your employer, or your family.

03
Foundations

A Short History of Deepfakes

Deepfakes did not appear overnight. They are the product of decades of incremental advances in computer vision, audio synthesis, and machine learning — advances that have accelerated dramatically in the past five years.

1990s
 

Early Digital Image Manipulation

Photoshop and digital editing tools made image manipulation possible but still required significant technical skill and time. This era established the precedent of treating photographic evidence as potentially unreliable — the earliest crack in the assumption that “seeing is believing.”

2014
 

GANs Invented — The Deepfake Engine Is Born

Ian Goodfellow at Google Brain introduced Generative Adversarial Networks. Though the first outputs were crude pixel-level experiments, the architecture was the breakthrough that would make photorealistic synthetic media possible within a few short years. A single research paper changed the trajectory of the threat.

2017
 

The Word “Deepfake” Appears

A Reddit user going by the pseudonym “deepfakes” began posting AI-generated face-swap videos, and the term entered the public lexicon. This marked the moment deepfake technology moved from academic research into public accessibility — with deeply troubling initial applications targeting celebrities without consent.

2019
 

First Major Corporate Voice Fraud

A UK energy company’s CEO was manipulated into transferring €220,000 after receiving a call using AI-cloned audio that convincingly imitated the voice of his German parent company’s chief executive. The entire transaction was authorised on the basis of a synthesised voice alone — the first widely documented deepfake financial fraud.

2022–23
 

Consumer GenAI Democratises Creation

The launch of ChatGPT, Stable Diffusion, Midjourney, and ElevenLabs voice synthesis brought deepfake-quality tools to ordinary users. Creating a convincing fake video or audio no longer required technical expertise — just an internet connection and a few minutes. The barrier to entry collapsed almost overnight.

2024
 

$25 Million Corporate Fraud and Biden Robocall

A Hong Kong finance worker was manipulated into transferring USD 25 million after criminals staged a deepfake video conference call impersonating multiple executives simultaneously. That same year, an AI-cloned voice of President Biden was used in automated calls targeting over 40,000 voters in New Hampshire. The World Economic Forum ranked AI-powered disinformation as the world’s number one risk.

2025+
 

Real-Time Deepfakes and Dark Web Services

Real-time video deepfaking — replacing a person’s face live during an active video call — became commercially available. “Deepfake-as-a-service” platforms emerged on the dark web, allowing anyone to purchase high-quality synthetic media on demand with no technical knowledge required whatsoever.

04
Foundations

Types of Deepfakes

Not all deepfakes come in the same format. Each media type carries different risks, creation methods, and detection challenges. Understanding each category is essential for building a comprehensive defence.

Type 01
Video Deepfakes

The most visually dramatic form. A neural network learns a target’s facial movements, expressions, and skin texture, then grafts them onto a different person. Used in CEO fraud, election interference, and non-consensual intimate imagery. Detection relies on spotting unnatural blinking, lighting inconsistencies, and edge artefacts.

Type 02
Audio Deepfakes

Synthetic speech generated from minimal voice samples using text-to-speech models trained on the target’s recordings. The most widely deployed deepfake type in financial crime — used to impersonate executives in phone fraud, bypass voice biometric authentication, and automate social engineering at industrial scale.

Type 03
Image Deepfakes

AI-generated still photographs that fabricate persons, places, or events. Used to forge identity documents, bypass Know-Your-Customer screening, create false news imagery, and generate non-consensual synthetic imagery of real people. Increasingly used in insurance fraud by submitting fabricated photographic claim evidence.

Type 04
Text Deepfakes

LLM-generated written content that mimics the exact writing style, tone, and knowledge of a specific person. Used to forge executive emails, fabricate quotes attributed to public figures, and power phishing campaigns at a scale no human writing team could match. The least visceral but arguably the most scalable threat vector.

Type 05
Real-Time Deepfakes

Live manipulation of video during an active call, replacing the caller’s face and voice simultaneously. The target cannot spot the fake by pausing a recording — the deception happens live. Already used in remote job interview fraud, corporate impersonation, and KYC bypass in financial services onboarding.

Type 06
Document Deepfakes

AI-altered or AI-generated official documents — passports, bank statements, certificates, contracts. Quality has improved so dramatically that trained fraud teams now struggle to distinguish genuine from synthetic documents without specialist analytical tools and forensic infrastructure.

05
Threat Landscape

Generative AI Security Risks — The Bigger Picture

Deepfakes are the most publicised danger of generative AI, but they sit within a much wider landscape of security risks that emerge when powerful AI tools fall into the wrong hands — or behave unexpectedly even in the right ones.

FIG 02 — THE GENERATIVE AI RISK WHEELGEN AIRISKSDeepfakesAI-PowerPhishingDataLeakageDis-informationPrivacyAlgorithmicBiasIP &CopyrightSocialEngineering
Fig 02 — Deepfakes are the most visible risk, but GenAI enables a full constellation of security threats that organisations must address as a unified challenge.
Critical
🎣
AI-Powered Phishing

GenAI enables attackers to draft perfectly grammatical, contextually aware phishing emails at industrial scale — personalised with details scraped from social media. The tell-tale spelling errors that once identified phishing attempts have largely disappeared.

High
📤
Sensitive Data Leakage

When employees input confidential business data into public AI tools, that data may be retained by the provider or exposed through breaches. Research found only 24% of generative AI deployments in enterprises are adequately secured against data leakage.

High
🦠
AI-Assisted Malware

GenAI can generate functional malicious code, help attackers discover software vulnerabilities, and write polymorphic malware that changes its signature to evade detection — dramatically accelerating the capability of less technically skilled cybercriminals.

🎯
Prompt Injection

Attackers embed hidden instructions inside documents, emails, or websites that a business’s AI agent will process — hijacking the agent to exfiltrate data, take unauthorised actions, or spread disinformation through trusted channels without any human intervention.

💡 The Unsecured AI Crisis

Research found that only 24% of generative AI initiatives are adequately secured, while the global average cost of a data breach reached USD 4.88 million in 2024. As AI becomes more deeply embedded in business operations, the attack surface for AI-specific exploitation expands simultaneously — yet security investment is not keeping pace with exposure.

06
Threat Landscape

Attack Vectors & Criminal Methods

Cybercriminals have rapidly developed a sophisticated playbook for deploying deepfakes. Each method exploits a different human vulnerability — our tendency to trust authoritative voices, recognisable faces, and official-looking documents.

📞

Vishing — Voice Phishing

An attacker clones the voice of a known executive, family member, or authority figure and calls a target requesting urgent action — transferring funds, sharing login credentials, or approving a transaction. Bank call centres globally are already overwhelmed by the volume of AI-voice attacks attempting to access customer accounts through cloned voices of their own clients.

📹

Video Conference Impersonation

Using real-time deepfake technology, attackers join video calls as fake executives, suppliers, or colleagues. The Hong Kong USD 25 million fraud case involved an entirely fake video conference where multiple participants — including the CFO — were simultaneously deepfaked, convincing the target every authorisation request was entirely legitimate.

🪪

Identity Document Forgery

AI generates synthetic passports, driving licences, utility bills, and bank statements convincing enough to defeat automated KYC verification systems at financial institutions — enabling fraudsters to open accounts, obtain credit, or launder money under completely fabricated identities at industrial scale.

📱

Social Media Disinformation

Deepfake videos of public figures are distributed at almost zero marginal cost. Malicious actors have reached 100,000 social media users for as little as seven cents per view. These campaigns manipulate stock prices, elections, and public health behaviour in ways that spread far faster than any official correction can travel.

🔐

Biometric Authentication Bypass

Systems that use facial recognition or voice verification for authentication are directly vulnerable. Synthetic voice and face data can be fed into authentication systems — by phone, video call, or physical access control cameras — to gain unauthorised entry to accounts, buildings, or computer systems that rely on biometric gates.

07
Threat Landscape

Industry-Specific Impacts

No industry is immune, but the risks manifest differently depending on the sector. Financial services faces direct monetary theft; healthcare confronts diagnostic manipulation; media faces the collapse of evidentiary credibility. Each sector needs a tailored response.

Industry Primary Deepfake Threat Specific Risk Severity
Financial Services Executive voice cloning, document forgery Fraudulent wire transfers, fake KYC documents, account takeover 🔴 Critical
Healthcare Synthetic medical imagery, record falsification Fraudulent insurance claims, false diagnostic records, drug diversion 🔴 Critical
Government & Elections Politician impersonation, fabricated statements Voter manipulation, policy fabrication, diplomatic incidents 🔴 Critical
Media & Journalism Fake news footage, synthetic evidence Credibility collapse, reputational damage, libel liability 🟠 High
Insurance Fabricated claim evidence (photos, video) Fraudulent payouts for non-existent accidents or damage 🟠 High
Human Resources Deepfake job candidates, credential forgery Remote hiring fraud, credential inflation, insider placement 🟠 High
Legal Fabricated evidence, forged contract signatures Manipulated court evidence, forged legal documents 🟠 High
Retail & E-Commerce Fake product reviews, return fraud imagery Brand damage, fraudulent refunds, counterfeit promotion 🟡 Moderate
08
Threat Landscape

Real-World Incidents & Case Studies

The most powerful argument for taking deepfakes seriously is the growing evidence base of real attacks with real consequences. These are not theoretical scenarios — they are documented incidents that caused millions in losses and lasting harm.

💰 The Hong Kong CFO Fraud — $25 Million

In early 2024, a financial employee at a multinational firm in Hong Kong was convinced to transfer the equivalent of USD 25 million after criminals staged a deepfake video conference call. Every other participant — including someone impersonating the CFO — was a deepfake. The employee attended multiple video meetings before executing the transfers, with no reason to doubt the familiar faces and voices he observed. This remains one of the largest single deepfake fraud events on record.

🏦 Retool — $15 Million in Crypto Assets Lost

Software developer company Retool fell victim to a sophisticated attack in which criminals used AI-cloned voices to impersonate IT staff and convince an employee to hand over multi-factor authentication codes. The attack enabled account access, and one client alone lost USD 15 million in cryptocurrency. The case demonstrated how voice cloning transforms conventional phishing into an almost irresistible form of manipulation that human verification processes were not designed to withstand.

🎙️ Biden Robocall — 40,000+ Voters Targeted

Ahead of the New Hampshire Democratic primary in 2024, voters received automated calls featuring an AI-cloned voice of President Biden instructing them not to vote in the primary. Over 40,000 voters were targeted. The incident demonstrated the specific and existential threat deepfake audio poses to democratic processes — election interference is now achievable at trivial cost with publicly available voice cloning tools.

📈 Pentagon Explosion Deepfake — Stock Market Panic

A fabricated image of a supposed explosion near the Pentagon spread widely on social media in 2023. Although officials debunked it within minutes, the image caused a measurable panic in financial markets before any correction arrived. This episode provided a clear preview: a more sophisticated and sustained campaign of synthetic financial disinformation could cause permanent damage to investor confidence and market stability.

09
Threat Landscape

Threat to Democracy & the Future of Truth

The deepest danger of deepfakes is not any single fraudulent act — it is the gradual erosion of the shared reality on which democratic societies depend. When people can no longer trust their eyes and ears, institutions, journalism, and democratic participation itself become fragile.

The World Economic Forum’s Global Risks Report ranked AI-fuelled disinformation as the single greatest threat facing the world in the near term — above war, pandemics, and climate change.

— World Economic Forum Global Risks Report, 2024

The Liar’s Dividend — The Secondary Harm

Beyond the direct harm of specific deepfakes, researchers have identified a secondary paradoxical effect: the mere knowledge that convincing deepfakes exist now allows people — particularly powerful ones — to deny genuine incriminating evidence as fabricated. A politician caught on authentic video can now simply claim “that was a deepfake.” This Liar’s Dividend means that genuine footage becomes plausibly deniable, corroding the evidentiary foundation of accountability journalism, legal proceedings, and public trust in institutions.

🧒 Why This Is Scary (Simply Explained)

Imagine if everyone in your class started making fake photos of each other doing things they never did. Even when something real happened, nobody would believe the photos anymore. That is exactly the problem — the more convincing fake videos that exist, the harder it becomes to trust any video, even completely real ones. Truth starts to feel impossible to find, and that is dangerous for everyone.

🗳️

Electoral Manipulation

Deepfake videos of candidates making extremist statements, audio of officials announcing false voting rule changes, or synthetic “leaks” of private conversations can swing elections — particularly in the final hours when there is insufficient time to effectively debunk them before voting closes.

📰

Media Credibility

Journalists already face public trust deficits. Deepfake technology accelerates this erosion by manufacturing convincing footage of events that never happened — undermining the ability of verified reporting to shape public understanding even when the reporting is entirely accurate.

🌐

Geopolitical Instability

State-sponsored deepfake campaigns can fabricate inflammatory statements by foreign leaders, manufacture evidence of military aggression, or create synthetic atrocity footage to justify military action. These capabilities transform the information environment of international diplomacy into a minefield of uncertainty.

10
Deeper Dimensions

Privacy & Data Risks from Generative AI

Generative AI creates profound privacy risks both through the deepfakes it enables and through the training data it requires — risks operating at the level of individuals, organisations, and entire populations simultaneously.

🔍 The Three-Second Rule of Voice Privacy

Modern voice cloning systems need as little as three seconds of speech to generate convincing synthetic audio. This means every voicemail you leave, every YouTube video you appear in, every recorded meeting, and every social media video represents raw material that could clone your voice without your knowledge or consent. There is no simple way to retract audio already published online.

  • Training Data Scraping: Web crawlers collect vast amounts of personal data — including faces, voices, writing styles, and biographical information — to train AI models without informed consent, potentially violating data protection regulations across multiple jurisdictions simultaneously.
  • Data Retention by AI Services: When employees input confidential documents, financial data, or customer records into third-party AI platforms, that data may be retained by the service provider. Corporate secrets, legal strategies, and personal health records have all been inadvertently shared with AI platforms this way.
  • Biometric Data Exposure: Facial recognition and voice print data collected through AI-enabled systems create unprecedented biometric profiles that, unlike passwords, cannot be changed once compromised.
  • Re-Identification Risk: Supposedly anonymised datasets can often be de-anonymised by combining them with other public data — recreating personally identifiable profiles that individuals believed had been safely redacted from training datasets.
  • Shadow Profiles: AI systems can infer sensitive personal attributes — health conditions, financial difficulties, political views, relationship status — from seemingly innocuous data, building comprehensive shadow profiles the individual never knowingly created.
11
Deeper Dimensions

Ethics, Consent & Identity Rights

The deepfake problem is fundamentally an ethical one about who controls representations of their own identity, and what obligations creators of synthetic media bear toward the real people they depict.

In 2023, a programmer combined OpenAI and ElevenLabs tools to create a real-time deepfake of Sir David Attenborough narrating his daily movements. While shared as a playful experiment, Attenborough himself expressed deep concern — he worried that AI would eventually be used to put words in his mouth that contradicted his life’s beliefs. The case illustrated a central tension: what some view as harmless creative play, the subject experiences as a fundamental violation of identity autonomy. As philosopher Derek Leben of Carnegie Mellon University analysed, people have a fundamental right to control how their identity is represented — regardless of whether a specific use seems harmful to an outside observer.

FIG 03 — THE DEEPFAKE ETHICS SPECTRUMClearly AcceptableGrey AreaClearly HarmfulExplicit consent given,creative use, clearlylabelled satirePublic figure parody,historical re-creation,unlabelled fan fictionFraud, NCSI, politicalmanipulation, identitytheft, harassment
Fig 03 — Deepfake ethics are not binary. Context, consent, labelling, and intent determine where any particular use falls on the harm spectrum.

Consent Is Non-Negotiable

No individual’s likeness, voice, or writing style should be used to create synthetic media without explicit, informed consent — regardless of whether the content appears harmful. The violation of identity autonomy is itself a harm, independent of any downstream consequences.

🏷️

Transparency and Labelling

All synthetic media depicting real people should carry clear, persistent, and machine-readable labels identifying it as AI-generated. Unlabelled synthetic content that could reasonably deceive a viewer should be treated as an act of deception regardless of the creator’s stated intent.

⚖️

Platform Responsibility

Social media platforms and communication tools that amplify deepfake content bear a proportional share of the resulting harm. The scale of distribution they enable makes passive indifference to synthetic content a form of complicity in the harms that distribution causes.

12
Deeper Dimensions

Risks to Children & Young People

Children and teenagers face unique and severe risks from deepfake technology — as targets of synthetic intimate imagery, as subjects of school-based harassment, and as a generation growing up where digital evidence is fundamentally untrustworthy.

🛡️
A Documented and Growing Crisis

In 2024, students at a New Jersey high school discovered that AI-generated fake intimate images had been created of their classmates using real photos without consent. In a separate Maryland case, a school administrator used AI voice cloning to impersonate a principal and fabricate inflammatory racist audio — causing serious community harm. These are not isolated incidents. As creation tools become cheaper and more accessible, similar cases are multiplying in schools worldwide.

  • Non-Consensual Synthetic Imagery (NCSI): Peer-created AI-generated intimate images of classmates. The psychological harm to victims mirrors that of genuine intimate image abuse and now carries criminal penalties in a growing number of jurisdictions worldwide.
  • Grooming Amplification: Synthetic audio and video of trusted adults — parents, teachers, coaches — can be fabricated to manipulate children into unsafe situations, significantly amplifying the toolkit available to predators who previously relied on real communications.
  • Cyberbullying at Scale: Realistic fake videos placing victims in humiliating scenarios can be created and shared rapidly. The permanence of digital content means victims face long-term reputational harm from fabrications created in seconds by a malicious peer.
  • Epistemic Harm: Growing up in an environment where any digital content is potentially fake impairs the development of healthy epistemic practices — the ability to evaluate evidence, trust credible sources, and form grounded beliefs about the world around them.
13
Deeper Dimensions

AI Bias, Discrimination & Broader Systemic Harms

Beyond deepfakes, generative AI systems carry systemic risks rooted in biased training data — producing outputs that can perpetuate and amplify discrimination at a scale and speed no human institution could match.

AI systems are trained on historical data, and historical data reflects historical inequities. A hiring algorithm trained on past records will learn that candidates with traditionally male names are more frequently hired for engineering roles — not because of capability differences, but because of past discrimination. Deployed at scale, such a system does not merely reflect bias; it industrialises it, making thousands of discriminatory decisions per second behind a veneer of algorithmic objectivity that makes those decisions far harder to challenge.

Risk Area 01
Hiring and Employment

AI-powered applicant screening tools have been found to systematically disadvantage candidates based on gender, ethnicity, age, and disability markers — outcomes that would be illegal if imposed by a human recruiter but are difficult to challenge when embedded in a supposedly neutral algorithmic system.

Risk Area 02
Healthcare Diagnostics

Medical AI trained predominantly on data from certain demographic groups performs worse — sometimes dangerously — on others. Skin condition classifiers trained on lighter skin tones show reduced accuracy on darker tones. Diagnostic tools give different risk scores for identical symptoms depending on the patient’s recorded race.

Risk Area 03
Criminal Justice

Predictive policing tools and recidivism scoring algorithms have been demonstrated to disproportionately flag and penalise individuals from historically marginalised communities — creating a feedback loop where over-policing produces data that justifies continued over-policing.

14
Defence

Detecting Deepfakes — Techniques & Tools

The arms race between deepfake creation and detection is asymmetric: creation tools improve faster than detection tools, driven by greater commercial investment. A layered approach to detection significantly reduces — but never eliminates — risk.

FIG 04 — DEEPFAKE DETECTION SIGNALS (VIDEO + AUDIO) Irregular blinking Too rare or uniform Blurry edge boundaries Face blends unnaturally Neck and jaw artefacts Lighting mismatch Shadows inconsistent Hair rendering glitches AUDIO DETECTION SIGNALS 🎵 Unnatural prosody — rhythm and stress feel off 🔇 Missing ambient background noise 🎙️ Slight metallic or synthetic timbre ⏸️ Computed pauses, not organic hesitation 🔄 Pitch too perfectly consistent throughout 💨 Missing breath sounds between phrases
Fig 04 — Visual and audio artefacts that can betray a deepfake. High-quality fakes suppress most of these — making technical AI-based detection tools essential rather than optional.
🔬
AI-vs-AI Detection

The most promising approach is using AI classifiers trained on vast libraries of known deepfakes to identify synthetic content at scale. These systems analyse metadata, frequency domain signatures, and micro-artefacts invisible to humans. The core insight: only AI working at the same speed as AI can catch it reliably.

🔏
Cryptographic Provenance

Digital signatures and watermarks embedded at content creation provide a verifiable chain of custody. If the signature is missing or broken, the content may have been altered. The C2PA standard (Coalition for Content Provenance and Authenticity) is building this infrastructure into mainstream media creation workflows.

📡
Metadata Analysis

Genuine camera footage carries rich metadata about device, GPS location, timestamp, and compression history. AI-generated content lacks authentic metadata or contains inconsistencies that forensic tools can detect — particularly useful for image and document deepfake identification.

🩺
Physiological Signal Analysis

Remote photoplethysmography (rPPG) detects subtle colour changes in skin caused by blood flow — a biological signal that deepfake videos cannot authentically replicate. This technique flags synthetic video even when visual artefacts are completely undetectable to human observers.

15
Defence

Protecting Organisations — A Practical Framework

Organisational defences against deepfake threats require a multi-layered strategy combining technical controls, human training, process redesign, and governance frameworks. No single measure is sufficient on its own.

  1. Establish Zero-Trust Verification Architecture: Never authorise high-value transactions or sensitive actions based on a single communication channel alone — regardless of how familiar the requester sounds or looks. All authorisation requests should require independent multi-channel confirmation through a pre-verified secure channel established before any suspicious contact occurs.
  2. Deploy AI-Powered Real-Time Detection: Integrate deepfake detection tools into video conferencing platforms, phone systems, and email infrastructure. Real-time AI monitoring can flag synthetic content before human judgment is applied, providing a critical first filter against even sophisticated attacks.
  3. Train Staff with Specific Scenarios: Employees must understand the deepfake threat concretely, not abstractly. Run tabletop exercises that simulate vishing calls, video conference impersonations, and fabricated executive emails. Human intuition becomes significantly more reliable when trained on specific real-world attack patterns.
  4. Secure the AI Pipeline: Audit every generative AI tool employees use for data retention practices, implement data loss prevention controls that detect confidential information being submitted to AI platforms, and restrict which AI services may be used for different categories of corporate data.
  5. Build Governance and Incident Response Plans: Establish clear policies for suspected deepfake attacks — who halts transactions, how forensic evidence is preserved, who is notified. A deepfake attack met without a response plan costs far more than one met with a prepared, practised team.
🔑 The Code Word Protocol

One of the simplest immediately effective defences is a confidential code word shared between trusted parties — a word not used anywhere in public or recorded communications. When receiving an unexpected authorisation request, ask for the code word before acting. No AI-generated impersonation can supply it. Particularly effective for family scam prevention and small business executive communication authentication.

16
Defence

Protecting Individuals — Practical Guidance

Individuals are not helpless against deepfake threats, but protection requires awareness, deliberate habit changes, and a healthy scepticism toward content that triggers strong emotions or urgent requests.

🤔
Verify
Before acting, independently confirm
⏸️
Pause
Urgency is a manipulation tool
🔍
Investigate
Look for artefacts and inconsistencies
📢
Report
Alert the platform and authorities
  • Minimise your public audio and video footprint: The less raw material available online, the harder it is to clone your voice or face. Restrict social media privacy settings and think carefully before posting videos where your voice and face are both clearly captured.
  • Create a family verification word: Agree on a private code word with close family members that only your inner circle would know. If you receive a distress call or urgent money request purportedly from a family member, ask for the code word before responding.
  • Treat urgency as a red flag: Legitimate requests from banks, employers, or family members rarely demand instant irreversible action. Artificial urgency — “act now or lose everything” — is a signature of social engineering whether deepfake-powered or conventional.
  • Use multi-factor authentication everywhere: Voice and face are no longer reliable authenticators. Use hardware tokens, authenticator apps, or physical keys for important accounts — methods that deepfakes cannot defeat.
  • Verify through a different channel: If a video call request, voice message, or email seems suspicious — even if it appears genuine — hang up and call the person back on a number you already have saved. Never use contact details provided within the suspicious communication itself.
  • Develop media literacy habits: Before sharing emotionally charged content, ask: when was this recorded? Does this match what I know about this person? Is there a credible independent source confirming this? Emotional intensity is a signal to slow down, not speed up your response.
17
Defence

Laws, Regulation & Global Policy

The regulatory landscape for deepfakes and generative AI is developing rapidly but remains fragmented. Different jurisdictions have taken different approaches, creating a patchwork of protections with significant gaps that bad actors are already exploiting.

Jurisdiction Key Legislation What It Covers Status
European Union EU AI Act (2024) Mandatory AI content labelling; bans on biometric surveillance; risk tiering for AI applications 🟢 In force
United States DEFIANCE Act (2024); state laws Federal civil remedies for non-consensual AI intimate imagery; California bans election deepfakes 🟡 Partial
United Kingdom Online Safety Act; AI Safety Institute Platform duties to remove harmful synthetic content; deepfake intimate imagery criminalised 2024 🟡 Evolving
China Deep Synthesis Regulations (2023) Mandatory disclosure of AI-generated content; real-name registration; provider liability 🟢 In force
Australia eSafety Commissioner guidance Non-consensual intimate deepfakes criminalised; platforms must remove flagged content within 24 hours 🟢 In force
Singapore MAS Deepfake Circulars; Elections Integrity Act Financial institutions required to implement deepfake risk controls; election deepfakes criminalised 🟢 In force
India IT Amendment Rules; Digital India Act Platforms must remove deepfakes within 24–36 hours; intermediary liability for synthetic content 🟡 Developing
🌐 What the Ideal Regulatory Framework Looks Like

Experts from MAS Singapore, the US DHS, and UNESCO converge on several key principles: (1) mandatory machine-readable watermarking of all AI-generated media at point of creation, (2) creator liability for unlabelled synthetic content that causes harm, (3) platform duty of care requiring proactive detection and removal rather than passive response to complaints, and (4) international cooperation standards to prevent jurisdiction-shopping by bad actors operating across borders.

18
Balanced View

Pros, Cons & the Nuanced Reality

The technology behind deepfakes is not inherently malicious. The same capabilities that enable fraud and disinformation also enable genuine creative, educational, and accessibility benefits. A balanced perspective is essential for proportionate policy.

✅ Legitimate Beneficial Uses

  • Film and entertainment: digital de-ageing, posthumous performances with explicit consent, stunt replacement, dubbing with accurate lip sync
  • Education: historical figures re-created to teach with full disclosure and factual accuracy; interactive language learning with native speaker synthesis
  • Accessibility: personalised text-to-speech for people with conditions like ALS who bank voice samples for future use
  • Healthcare simulation: medical training scenarios with synthetic patient cases not requiring real patient data or consent
  • Journalism: re-creating historical events for which no footage exists, with explicit editorial disclosure
  • Creative expression: new collaborative AI art and music involving consenting participants who retain creative control

✗ Harmful Applications

  • Financial fraud via executive impersonation causing billions in corporate losses annually worldwide
  • Non-consensual synthetic intimate imagery targeting individuals — primarily women and minors — with severe psychological consequences
  • Election interference through fabricated statements by candidates, officials, or electoral authorities
  • Corporate espionage and competitive disinformation campaigns targeting brand reputation
  • Identity theft and biometric authentication bypass in financial and government systems
  • State-sponsored propaganda and geopolitical destabilisation operations at national scale
  • The Liar’s Dividend — undermining the evidentiary value of all authentic video and audio evidence
🎯 The Core Principle: Consent + Transparency

The ethical line between legitimate and harmful use runs through two gates: consent (did the person depicted agree to be represented in this way?) and transparency (is the synthetic nature clearly disclosed to anyone who might be affected?). Technology that passes both tests can be genuinely valuable. Technology that fails either test demands regulatory and social intervention.

19
Reference

Glossary of Key Terms

Biometric Authentication
Using unique physical characteristics — voice print, facial geometry, fingerprints, iris patterns — to verify identity. Deepfakes directly target voice and facial biometric systems.
Deepfake-as-a-Service
Dark web or commercial platforms providing deepfake creation on demand, with no technical skill required — dramatically lowering the barrier to entry for malicious actors of all capability levels.
Diffusion Model
An AI architecture generating images or audio by reversing a noise-addition process, producing high-quality synthetic media. Used in Stable Diffusion, DALL-E, and most modern voice cloners.
GAN (Generative Adversarial Network)
An AI architecture using two competing neural networks — a generator and a discriminator — to produce increasingly realistic synthetic content. The foundational technology behind early deepfakes.
KYC (Know Your Customer)
Regulatory identity verification processes used by financial institutions to confirm customer identity. Document and biometric deepfakes are specifically engineered to defeat KYC controls.
Liar’s Dividend
The paradoxical secondary harm of deepfakes: the ability of real wrongdoers to claim authentic evidence against them is fabricated, because the public knows convincing deepfakes exist.
NCSI
Non-Consensual Synthetic Imagery — AI-generated intimate images of real people created without their consent. Criminalised in a growing number of jurisdictions globally.
Prompt Injection
An attack where hidden instructions embedded in data manipulate an AI agent into taking unauthorised actions when it processes that data — bypassing intended safety constraints.
rPPG
Remote Photoplethysmography — physiological signal detection using skin colour changes in video caused by blood flow, a biological marker deepfake videos cannot authentically replicate.
Social Engineering
Psychological manipulation exploiting human trust to extract information or authorise actions. Deepfakes make social engineering dramatically more credible by providing convincing impersonation.
Synthetic Media
Any content — audio, video, image, text, or documents — fully or partially generated or altered by AI rather than created entirely from authentic sources by human effort.
Vishing
Voice phishing — fraud conducted via voice calls, increasingly powered by AI voice cloning to impersonate trusted individuals and extract money, credentials, or sensitive information from targets.
20
Bibliography

Sources & References

This document synthesises, analyses, and significantly expands upon content from the following authoritative institutional sources. All prose has been independently researched and rewritten. No text has been reproduced verbatim; all original analysis, SVG diagrams, analogies, and structural frameworks are original works created for this document.

[01]
10 AI Dangers and Risks — IBM Think

IBM’s comprehensive risk catalogue covering bias, cybersecurity threats, data privacy, and environmental harms. Author: Rina Diane Caballar.

[02]
A New Wave of Deepfake Cybercrime — IBM Think

Detailed analysis of deepfake modalities, industry impacts, financial sector case studies, and deepfake fraud economics. Author: Srini Tummalapenta.

[03]
Deepfakes and the Crisis of Knowing — UNESCO

UNESCO’s analysis of synthetic media’s epistemological implications and media literacy frameworks. Author: Dr. Nadia Naffi, Université Laval, October 2025.

[04]
What Is a Deepfake? — Fortinet

Cybersecurity-focused technical overview of deepfake technology, creation methods, and detection approaches for organisational defence.

[05]
Deepfake Threats to Companies — KPMG

Board-level risk analysis of deepfake threats, the democratisation of creation tools, and five practical organisational defence steps.

[06]
Deepfakes and the Ethics of Generative AI — CMU Tepper

Carnegie Mellon ethics analysis of consent, identity rights, school deepfake incidents, and the Attenborough case. Author: Derek Leben, 2024.

[07]
AI, Deepfakes and the Future of Truth — Brookings

Brookings Institution research on deepfakes’ impact on democratic institutions and the liar’s dividend concept.

[08]
Generative AI Security Risks — SentinelOne

Technical analysis of ten generative AI security risks including prompt injection, data leakage, and AI-powered malware generation.

[09]
GenAI Security Risks: What Enterprises Need to Know — Proofpoint

Enterprise security perspective on AI data leakage, shadow AI usage, and DLP implications. Updated May 2026.

[10]
Cyber Risks Associated with Deepfakes — MAS Singapore

Monetary Authority of Singapore’s regulatory circular on deepfake risks for financial institutions and required risk controls.

[11]
Increasing Threats of Deepfake Identities — US DHS

US Department of Homeland Security assessment of deepfake identity threats to national security and law enforcement.

[12]
Deepfakes — Australian eSafety Commissioner

Australia’s national online safety authority on deepfake harms to individuals — particularly women and children — and regulatory response.

[13]
AI Deepfake Security Concerns — Cloud Security Alliance

CSA’s technical framework for AI deepfake risks in cloud environments, detection tooling, and organisational governance approaches.

[14]
Deepfakes Pose Business Risks — Booz Allen Hamilton

Strategic consulting perspective on organisational deepfake risk, threat modelling, and defensive investment priorities.

[15]
The Cybersecurity Threat No One Is Ready For — GSDC

Assessment of organisational unpreparedness for AI-powered attacks and the gap between threat sophistication and current defensive posture.

[16]
Risks of GenAI Deepfakes — King & Wood Mallesons

Legal analysis of generative AI deepfake risks spanning disinformation, eSafety, authenticity, and emerging corporate liability frameworks.

[17]
Dangers of Generative AI: Privacy and Security — BreachRx

Data breach response perspective on GenAI privacy risks, re-identification vulnerabilities, and regulatory notification obligations.

[18]
Generative AI Security Risks — Tigera

Technical guide to LLM-specific security vulnerabilities including prompt injection, model inversion attacks, and AI supply chain risks.

[19]
Cyber Risks Associated with GenAI — MAS Singapore

Singapore’s Monetary Authority circular on broader generative AI cyber risks for the financial sector, supplementing deepfake-specific guidance.

[20]
Additional Research: WEF Global Risks Report 2024; C2PA Standards; DEFIANCE Act

Supplementary sources including World Economic Forum risk rankings, Coalition for Content Provenance and Authenticity standards, and US deepfake legislation.

Leave a Reply

Your email address will not be published. Required fields are marked *