Deepfakes & Generative
AI Risks
A complete, deeply researched reference guide — from what deepfakes are and how they work, to the real-world damage they cause, who is at risk, and what organisations and individuals can do to protect themselves.
What Is a Deepfake?
A deepfake is a piece of synthetic media — a video, audio recording, image, or written text — artificially created or altered using AI to look, sound, or read as though a real person said or did something they never actually said or did.
A deepfake is a digitally fabricated media artefact so convincingly realistic that it can deceive viewers into believing a real person said, appeared in, or created something that never happened — manufactured with generative AI tools now accessible to virtually anyone with a smartphone.— Synthesised from KPMG, UNESCO, and DHS research, 2024–2025
The Simple Explanation (For a 10-Year-Old!) 🧒
Imagine you have a very talented actor friend who can copy anyone’s voice and face perfectly. Now imagine that friend makes a video of your teacher saying something embarrassing — something the teacher never actually said. If it looks and sounds convincing enough, other children might believe it is real. That is exactly what a deepfake is, except the actor is a computer program trained on thousands of hours of real footage and audio. The alarming part is that the computer can now do this in minutes, for almost no cost at all.
Deep — comes from “deep learning,” the AI technique used to generate these fakes. Fake — exactly what it says: the content is fabricated even though it appears authentic. Together, the word captures how a sophisticated AI method is weaponised to create convincing falsehoods about real people.
Deepfake creation tools that once required a specialist research team and months of work now run on a consumer laptop in minutes. Open-source models, low-cost APIs, and “deepfake-as-a-service” platforms on the dark web have lowered the barrier so far that malicious actors with no technical background can launch sophisticated impersonation campaigns. The threat is no longer theoretical — it is actively causing financial fraud, reputational destruction, and democratic harm today.
How Deepfakes Are Made
Understanding the technical mechanism behind deepfakes is the first step toward defending against them. The process is powered by deep learning techniques that, when combined, enable astonishingly convincing synthetic media.
The Core Technology: GANs and Diffusion Models
Most deepfakes are generated using one of two AI architectures. The older, still widely used technique is the Generative Adversarial Network (GAN), which pits two neural networks against each other: one generates fake content while the other tries to detect it as fake. Through this adversarial competition, both networks improve until the generated content becomes indistinguishable from genuine material.
Newer deepfakes increasingly use Diffusion Models — the same technology behind image generators like Stable Diffusion and DALL-E — which work by gradually removing random noise from a data sample until a coherent image or audio file emerges. Diffusion models produce higher-quality, more photorealistic results and are now the preferred technique for state-of-the-art deepfake creation.
Voice cloning deserves special attention because it requires the least raw material. Credible synthetic speech can be produced from as little as three seconds of someone talking. This means that any phone call you have ever had recorded, any podcast you appeared in, any video you posted publicly, provides enough raw material for a criminal to clone your voice and impersonate you convincingly to your bank, your employer, or your family.
A Short History of Deepfakes
Deepfakes did not appear overnight. They are the product of decades of incremental advances in computer vision, audio synthesis, and machine learning — advances that have accelerated dramatically in the past five years.
Early Digital Image Manipulation
Photoshop and digital editing tools made image manipulation possible but still required significant technical skill and time. This era established the precedent of treating photographic evidence as potentially unreliable — the earliest crack in the assumption that “seeing is believing.”
GANs Invented — The Deepfake Engine Is Born
Ian Goodfellow at Google Brain introduced Generative Adversarial Networks. Though the first outputs were crude pixel-level experiments, the architecture was the breakthrough that would make photorealistic synthetic media possible within a few short years. A single research paper changed the trajectory of the threat.
The Word “Deepfake” Appears
A Reddit user going by the pseudonym “deepfakes” began posting AI-generated face-swap videos, and the term entered the public lexicon. This marked the moment deepfake technology moved from academic research into public accessibility — with deeply troubling initial applications targeting celebrities without consent.
First Major Corporate Voice Fraud
A UK energy company’s CEO was manipulated into transferring €220,000 after receiving a call using AI-cloned audio that convincingly imitated the voice of his German parent company’s chief executive. The entire transaction was authorised on the basis of a synthesised voice alone — the first widely documented deepfake financial fraud.
Consumer GenAI Democratises Creation
The launch of ChatGPT, Stable Diffusion, Midjourney, and ElevenLabs voice synthesis brought deepfake-quality tools to ordinary users. Creating a convincing fake video or audio no longer required technical expertise — just an internet connection and a few minutes. The barrier to entry collapsed almost overnight.
$25 Million Corporate Fraud and Biden Robocall
A Hong Kong finance worker was manipulated into transferring USD 25 million after criminals staged a deepfake video conference call impersonating multiple executives simultaneously. That same year, an AI-cloned voice of President Biden was used in automated calls targeting over 40,000 voters in New Hampshire. The World Economic Forum ranked AI-powered disinformation as the world’s number one risk.
Real-Time Deepfakes and Dark Web Services
Real-time video deepfaking — replacing a person’s face live during an active video call — became commercially available. “Deepfake-as-a-service” platforms emerged on the dark web, allowing anyone to purchase high-quality synthetic media on demand with no technical knowledge required whatsoever.
Types of Deepfakes
Not all deepfakes come in the same format. Each media type carries different risks, creation methods, and detection challenges. Understanding each category is essential for building a comprehensive defence.
The most visually dramatic form. A neural network learns a target’s facial movements, expressions, and skin texture, then grafts them onto a different person. Used in CEO fraud, election interference, and non-consensual intimate imagery. Detection relies on spotting unnatural blinking, lighting inconsistencies, and edge artefacts.
Synthetic speech generated from minimal voice samples using text-to-speech models trained on the target’s recordings. The most widely deployed deepfake type in financial crime — used to impersonate executives in phone fraud, bypass voice biometric authentication, and automate social engineering at industrial scale.
AI-generated still photographs that fabricate persons, places, or events. Used to forge identity documents, bypass Know-Your-Customer screening, create false news imagery, and generate non-consensual synthetic imagery of real people. Increasingly used in insurance fraud by submitting fabricated photographic claim evidence.
LLM-generated written content that mimics the exact writing style, tone, and knowledge of a specific person. Used to forge executive emails, fabricate quotes attributed to public figures, and power phishing campaigns at a scale no human writing team could match. The least visceral but arguably the most scalable threat vector.
Live manipulation of video during an active call, replacing the caller’s face and voice simultaneously. The target cannot spot the fake by pausing a recording — the deception happens live. Already used in remote job interview fraud, corporate impersonation, and KYC bypass in financial services onboarding.
AI-altered or AI-generated official documents — passports, bank statements, certificates, contracts. Quality has improved so dramatically that trained fraud teams now struggle to distinguish genuine from synthetic documents without specialist analytical tools and forensic infrastructure.
Generative AI Security Risks — The Bigger Picture
Deepfakes are the most publicised danger of generative AI, but they sit within a much wider landscape of security risks that emerge when powerful AI tools fall into the wrong hands — or behave unexpectedly even in the right ones.
GenAI enables attackers to draft perfectly grammatical, contextually aware phishing emails at industrial scale — personalised with details scraped from social media. The tell-tale spelling errors that once identified phishing attempts have largely disappeared.
When employees input confidential business data into public AI tools, that data may be retained by the provider or exposed through breaches. Research found only 24% of generative AI deployments in enterprises are adequately secured against data leakage.
GenAI can generate functional malicious code, help attackers discover software vulnerabilities, and write polymorphic malware that changes its signature to evade detection — dramatically accelerating the capability of less technically skilled cybercriminals.
Attackers embed hidden instructions inside documents, emails, or websites that a business’s AI agent will process — hijacking the agent to exfiltrate data, take unauthorised actions, or spread disinformation through trusted channels without any human intervention.
Research found that only 24% of generative AI initiatives are adequately secured, while the global average cost of a data breach reached USD 4.88 million in 2024. As AI becomes more deeply embedded in business operations, the attack surface for AI-specific exploitation expands simultaneously — yet security investment is not keeping pace with exposure.
Attack Vectors & Criminal Methods
Cybercriminals have rapidly developed a sophisticated playbook for deploying deepfakes. Each method exploits a different human vulnerability — our tendency to trust authoritative voices, recognisable faces, and official-looking documents.
Vishing — Voice Phishing
An attacker clones the voice of a known executive, family member, or authority figure and calls a target requesting urgent action — transferring funds, sharing login credentials, or approving a transaction. Bank call centres globally are already overwhelmed by the volume of AI-voice attacks attempting to access customer accounts through cloned voices of their own clients.
Video Conference Impersonation
Using real-time deepfake technology, attackers join video calls as fake executives, suppliers, or colleagues. The Hong Kong USD 25 million fraud case involved an entirely fake video conference where multiple participants — including the CFO — were simultaneously deepfaked, convincing the target every authorisation request was entirely legitimate.
Identity Document Forgery
AI generates synthetic passports, driving licences, utility bills, and bank statements convincing enough to defeat automated KYC verification systems at financial institutions — enabling fraudsters to open accounts, obtain credit, or launder money under completely fabricated identities at industrial scale.
Business Email Compromise
AI generates emails in the exact writing style of a known executive or supplier, complete with appropriate terminology, formatting, and contextual knowledge scraped from public sources. These synthetic impersonations request fraudulent payments, credential resets, or sensitive document transfers that bypass normal human suspicion.
Social Media Disinformation
Deepfake videos of public figures are distributed at almost zero marginal cost. Malicious actors have reached 100,000 social media users for as little as seven cents per view. These campaigns manipulate stock prices, elections, and public health behaviour in ways that spread far faster than any official correction can travel.
Biometric Authentication Bypass
Systems that use facial recognition or voice verification for authentication are directly vulnerable. Synthetic voice and face data can be fed into authentication systems — by phone, video call, or physical access control cameras — to gain unauthorised entry to accounts, buildings, or computer systems that rely on biometric gates.
Industry-Specific Impacts
No industry is immune, but the risks manifest differently depending on the sector. Financial services faces direct monetary theft; healthcare confronts diagnostic manipulation; media faces the collapse of evidentiary credibility. Each sector needs a tailored response.
| Industry | Primary Deepfake Threat | Specific Risk | Severity |
|---|---|---|---|
| Financial Services | Executive voice cloning, document forgery | Fraudulent wire transfers, fake KYC documents, account takeover | 🔴 Critical |
| Healthcare | Synthetic medical imagery, record falsification | Fraudulent insurance claims, false diagnostic records, drug diversion | 🔴 Critical |
| Government & Elections | Politician impersonation, fabricated statements | Voter manipulation, policy fabrication, diplomatic incidents | 🔴 Critical |
| Media & Journalism | Fake news footage, synthetic evidence | Credibility collapse, reputational damage, libel liability | 🟠 High |
| Insurance | Fabricated claim evidence (photos, video) | Fraudulent payouts for non-existent accidents or damage | 🟠 High |
| Human Resources | Deepfake job candidates, credential forgery | Remote hiring fraud, credential inflation, insider placement | 🟠 High |
| Legal | Fabricated evidence, forged contract signatures | Manipulated court evidence, forged legal documents | 🟠 High |
| Retail & E-Commerce | Fake product reviews, return fraud imagery | Brand damage, fraudulent refunds, counterfeit promotion | 🟡 Moderate |
Real-World Incidents & Case Studies
The most powerful argument for taking deepfakes seriously is the growing evidence base of real attacks with real consequences. These are not theoretical scenarios — they are documented incidents that caused millions in losses and lasting harm.
In early 2024, a financial employee at a multinational firm in Hong Kong was convinced to transfer the equivalent of USD 25 million after criminals staged a deepfake video conference call. Every other participant — including someone impersonating the CFO — was a deepfake. The employee attended multiple video meetings before executing the transfers, with no reason to doubt the familiar faces and voices he observed. This remains one of the largest single deepfake fraud events on record.
Software developer company Retool fell victim to a sophisticated attack in which criminals used AI-cloned voices to impersonate IT staff and convince an employee to hand over multi-factor authentication codes. The attack enabled account access, and one client alone lost USD 15 million in cryptocurrency. The case demonstrated how voice cloning transforms conventional phishing into an almost irresistible form of manipulation that human verification processes were not designed to withstand.
Ahead of the New Hampshire Democratic primary in 2024, voters received automated calls featuring an AI-cloned voice of President Biden instructing them not to vote in the primary. Over 40,000 voters were targeted. The incident demonstrated the specific and existential threat deepfake audio poses to democratic processes — election interference is now achievable at trivial cost with publicly available voice cloning tools.
A fabricated image of a supposed explosion near the Pentagon spread widely on social media in 2023. Although officials debunked it within minutes, the image caused a measurable panic in financial markets before any correction arrived. This episode provided a clear preview: a more sophisticated and sustained campaign of synthetic financial disinformation could cause permanent damage to investor confidence and market stability.
Threat to Democracy & the Future of Truth
The deepest danger of deepfakes is not any single fraudulent act — it is the gradual erosion of the shared reality on which democratic societies depend. When people can no longer trust their eyes and ears, institutions, journalism, and democratic participation itself become fragile.
The World Economic Forum’s Global Risks Report ranked AI-fuelled disinformation as the single greatest threat facing the world in the near term — above war, pandemics, and climate change.
— World Economic Forum Global Risks Report, 2024The Liar’s Dividend — The Secondary Harm
Beyond the direct harm of specific deepfakes, researchers have identified a secondary paradoxical effect: the mere knowledge that convincing deepfakes exist now allows people — particularly powerful ones — to deny genuine incriminating evidence as fabricated. A politician caught on authentic video can now simply claim “that was a deepfake.” This Liar’s Dividend means that genuine footage becomes plausibly deniable, corroding the evidentiary foundation of accountability journalism, legal proceedings, and public trust in institutions.
Imagine if everyone in your class started making fake photos of each other doing things they never did. Even when something real happened, nobody would believe the photos anymore. That is exactly the problem — the more convincing fake videos that exist, the harder it becomes to trust any video, even completely real ones. Truth starts to feel impossible to find, and that is dangerous for everyone.
Electoral Manipulation
Deepfake videos of candidates making extremist statements, audio of officials announcing false voting rule changes, or synthetic “leaks” of private conversations can swing elections — particularly in the final hours when there is insufficient time to effectively debunk them before voting closes.
Media Credibility
Journalists already face public trust deficits. Deepfake technology accelerates this erosion by manufacturing convincing footage of events that never happened — undermining the ability of verified reporting to shape public understanding even when the reporting is entirely accurate.
Geopolitical Instability
State-sponsored deepfake campaigns can fabricate inflammatory statements by foreign leaders, manufacture evidence of military aggression, or create synthetic atrocity footage to justify military action. These capabilities transform the information environment of international diplomacy into a minefield of uncertainty.
Privacy & Data Risks from Generative AI
Generative AI creates profound privacy risks both through the deepfakes it enables and through the training data it requires — risks operating at the level of individuals, organisations, and entire populations simultaneously.
Modern voice cloning systems need as little as three seconds of speech to generate convincing synthetic audio. This means every voicemail you leave, every YouTube video you appear in, every recorded meeting, and every social media video represents raw material that could clone your voice without your knowledge or consent. There is no simple way to retract audio already published online.
- Training Data Scraping: Web crawlers collect vast amounts of personal data — including faces, voices, writing styles, and biographical information — to train AI models without informed consent, potentially violating data protection regulations across multiple jurisdictions simultaneously.
- Data Retention by AI Services: When employees input confidential documents, financial data, or customer records into third-party AI platforms, that data may be retained by the service provider. Corporate secrets, legal strategies, and personal health records have all been inadvertently shared with AI platforms this way.
- Biometric Data Exposure: Facial recognition and voice print data collected through AI-enabled systems create unprecedented biometric profiles that, unlike passwords, cannot be changed once compromised.
- Re-Identification Risk: Supposedly anonymised datasets can often be de-anonymised by combining them with other public data — recreating personally identifiable profiles that individuals believed had been safely redacted from training datasets.
- Shadow Profiles: AI systems can infer sensitive personal attributes — health conditions, financial difficulties, political views, relationship status — from seemingly innocuous data, building comprehensive shadow profiles the individual never knowingly created.
Ethics, Consent & Identity Rights
The deepfake problem is fundamentally an ethical one about who controls representations of their own identity, and what obligations creators of synthetic media bear toward the real people they depict.
In 2023, a programmer combined OpenAI and ElevenLabs tools to create a real-time deepfake of Sir David Attenborough narrating his daily movements. While shared as a playful experiment, Attenborough himself expressed deep concern — he worried that AI would eventually be used to put words in his mouth that contradicted his life’s beliefs. The case illustrated a central tension: what some view as harmless creative play, the subject experiences as a fundamental violation of identity autonomy. As philosopher Derek Leben of Carnegie Mellon University analysed, people have a fundamental right to control how their identity is represented — regardless of whether a specific use seems harmful to an outside observer.
Consent Is Non-Negotiable
No individual’s likeness, voice, or writing style should be used to create synthetic media without explicit, informed consent — regardless of whether the content appears harmful. The violation of identity autonomy is itself a harm, independent of any downstream consequences.
Transparency and Labelling
All synthetic media depicting real people should carry clear, persistent, and machine-readable labels identifying it as AI-generated. Unlabelled synthetic content that could reasonably deceive a viewer should be treated as an act of deception regardless of the creator’s stated intent.
Platform Responsibility
Social media platforms and communication tools that amplify deepfake content bear a proportional share of the resulting harm. The scale of distribution they enable makes passive indifference to synthetic content a form of complicity in the harms that distribution causes.
Risks to Children & Young People
Children and teenagers face unique and severe risks from deepfake technology — as targets of synthetic intimate imagery, as subjects of school-based harassment, and as a generation growing up where digital evidence is fundamentally untrustworthy.
In 2024, students at a New Jersey high school discovered that AI-generated fake intimate images had been created of their classmates using real photos without consent. In a separate Maryland case, a school administrator used AI voice cloning to impersonate a principal and fabricate inflammatory racist audio — causing serious community harm. These are not isolated incidents. As creation tools become cheaper and more accessible, similar cases are multiplying in schools worldwide.
- Non-Consensual Synthetic Imagery (NCSI): Peer-created AI-generated intimate images of classmates. The psychological harm to victims mirrors that of genuine intimate image abuse and now carries criminal penalties in a growing number of jurisdictions worldwide.
- Grooming Amplification: Synthetic audio and video of trusted adults — parents, teachers, coaches — can be fabricated to manipulate children into unsafe situations, significantly amplifying the toolkit available to predators who previously relied on real communications.
- Cyberbullying at Scale: Realistic fake videos placing victims in humiliating scenarios can be created and shared rapidly. The permanence of digital content means victims face long-term reputational harm from fabrications created in seconds by a malicious peer.
- Epistemic Harm: Growing up in an environment where any digital content is potentially fake impairs the development of healthy epistemic practices — the ability to evaluate evidence, trust credible sources, and form grounded beliefs about the world around them.
AI Bias, Discrimination & Broader Systemic Harms
Beyond deepfakes, generative AI systems carry systemic risks rooted in biased training data — producing outputs that can perpetuate and amplify discrimination at a scale and speed no human institution could match.
AI systems are trained on historical data, and historical data reflects historical inequities. A hiring algorithm trained on past records will learn that candidates with traditionally male names are more frequently hired for engineering roles — not because of capability differences, but because of past discrimination. Deployed at scale, such a system does not merely reflect bias; it industrialises it, making thousands of discriminatory decisions per second behind a veneer of algorithmic objectivity that makes those decisions far harder to challenge.
AI-powered applicant screening tools have been found to systematically disadvantage candidates based on gender, ethnicity, age, and disability markers — outcomes that would be illegal if imposed by a human recruiter but are difficult to challenge when embedded in a supposedly neutral algorithmic system.
Medical AI trained predominantly on data from certain demographic groups performs worse — sometimes dangerously — on others. Skin condition classifiers trained on lighter skin tones show reduced accuracy on darker tones. Diagnostic tools give different risk scores for identical symptoms depending on the patient’s recorded race.
Predictive policing tools and recidivism scoring algorithms have been demonstrated to disproportionately flag and penalise individuals from historically marginalised communities — creating a feedback loop where over-policing produces data that justifies continued over-policing.
Detecting Deepfakes — Techniques & Tools
The arms race between deepfake creation and detection is asymmetric: creation tools improve faster than detection tools, driven by greater commercial investment. A layered approach to detection significantly reduces — but never eliminates — risk.
The most promising approach is using AI classifiers trained on vast libraries of known deepfakes to identify synthetic content at scale. These systems analyse metadata, frequency domain signatures, and micro-artefacts invisible to humans. The core insight: only AI working at the same speed as AI can catch it reliably.
Digital signatures and watermarks embedded at content creation provide a verifiable chain of custody. If the signature is missing or broken, the content may have been altered. The C2PA standard (Coalition for Content Provenance and Authenticity) is building this infrastructure into mainstream media creation workflows.
Genuine camera footage carries rich metadata about device, GPS location, timestamp, and compression history. AI-generated content lacks authentic metadata or contains inconsistencies that forensic tools can detect — particularly useful for image and document deepfake identification.
Remote photoplethysmography (rPPG) detects subtle colour changes in skin caused by blood flow — a biological signal that deepfake videos cannot authentically replicate. This technique flags synthetic video even when visual artefacts are completely undetectable to human observers.
Protecting Organisations — A Practical Framework
Organisational defences against deepfake threats require a multi-layered strategy combining technical controls, human training, process redesign, and governance frameworks. No single measure is sufficient on its own.
- Establish Zero-Trust Verification Architecture: Never authorise high-value transactions or sensitive actions based on a single communication channel alone — regardless of how familiar the requester sounds or looks. All authorisation requests should require independent multi-channel confirmation through a pre-verified secure channel established before any suspicious contact occurs.
- Deploy AI-Powered Real-Time Detection: Integrate deepfake detection tools into video conferencing platforms, phone systems, and email infrastructure. Real-time AI monitoring can flag synthetic content before human judgment is applied, providing a critical first filter against even sophisticated attacks.
- Train Staff with Specific Scenarios: Employees must understand the deepfake threat concretely, not abstractly. Run tabletop exercises that simulate vishing calls, video conference impersonations, and fabricated executive emails. Human intuition becomes significantly more reliable when trained on specific real-world attack patterns.
- Secure the AI Pipeline: Audit every generative AI tool employees use for data retention practices, implement data loss prevention controls that detect confidential information being submitted to AI platforms, and restrict which AI services may be used for different categories of corporate data.
- Build Governance and Incident Response Plans: Establish clear policies for suspected deepfake attacks — who halts transactions, how forensic evidence is preserved, who is notified. A deepfake attack met without a response plan costs far more than one met with a prepared, practised team.
One of the simplest immediately effective defences is a confidential code word shared between trusted parties — a word not used anywhere in public or recorded communications. When receiving an unexpected authorisation request, ask for the code word before acting. No AI-generated impersonation can supply it. Particularly effective for family scam prevention and small business executive communication authentication.
Protecting Individuals — Practical Guidance
Individuals are not helpless against deepfake threats, but protection requires awareness, deliberate habit changes, and a healthy scepticism toward content that triggers strong emotions or urgent requests.
- Minimise your public audio and video footprint: The less raw material available online, the harder it is to clone your voice or face. Restrict social media privacy settings and think carefully before posting videos where your voice and face are both clearly captured.
- Create a family verification word: Agree on a private code word with close family members that only your inner circle would know. If you receive a distress call or urgent money request purportedly from a family member, ask for the code word before responding.
- Treat urgency as a red flag: Legitimate requests from banks, employers, or family members rarely demand instant irreversible action. Artificial urgency — “act now or lose everything” — is a signature of social engineering whether deepfake-powered or conventional.
- Use multi-factor authentication everywhere: Voice and face are no longer reliable authenticators. Use hardware tokens, authenticator apps, or physical keys for important accounts — methods that deepfakes cannot defeat.
- Verify through a different channel: If a video call request, voice message, or email seems suspicious — even if it appears genuine — hang up and call the person back on a number you already have saved. Never use contact details provided within the suspicious communication itself.
- Develop media literacy habits: Before sharing emotionally charged content, ask: when was this recorded? Does this match what I know about this person? Is there a credible independent source confirming this? Emotional intensity is a signal to slow down, not speed up your response.
Laws, Regulation & Global Policy
The regulatory landscape for deepfakes and generative AI is developing rapidly but remains fragmented. Different jurisdictions have taken different approaches, creating a patchwork of protections with significant gaps that bad actors are already exploiting.
| Jurisdiction | Key Legislation | What It Covers | Status |
|---|---|---|---|
| European Union | EU AI Act (2024) | Mandatory AI content labelling; bans on biometric surveillance; risk tiering for AI applications | 🟢 In force |
| United States | DEFIANCE Act (2024); state laws | Federal civil remedies for non-consensual AI intimate imagery; California bans election deepfakes | 🟡 Partial |
| United Kingdom | Online Safety Act; AI Safety Institute | Platform duties to remove harmful synthetic content; deepfake intimate imagery criminalised 2024 | 🟡 Evolving |
| China | Deep Synthesis Regulations (2023) | Mandatory disclosure of AI-generated content; real-name registration; provider liability | 🟢 In force |
| Australia | eSafety Commissioner guidance | Non-consensual intimate deepfakes criminalised; platforms must remove flagged content within 24 hours | 🟢 In force |
| Singapore | MAS Deepfake Circulars; Elections Integrity Act | Financial institutions required to implement deepfake risk controls; election deepfakes criminalised | 🟢 In force |
| India | IT Amendment Rules; Digital India Act | Platforms must remove deepfakes within 24–36 hours; intermediary liability for synthetic content | 🟡 Developing |
Experts from MAS Singapore, the US DHS, and UNESCO converge on several key principles: (1) mandatory machine-readable watermarking of all AI-generated media at point of creation, (2) creator liability for unlabelled synthetic content that causes harm, (3) platform duty of care requiring proactive detection and removal rather than passive response to complaints, and (4) international cooperation standards to prevent jurisdiction-shopping by bad actors operating across borders.
Pros, Cons & the Nuanced Reality
The technology behind deepfakes is not inherently malicious. The same capabilities that enable fraud and disinformation also enable genuine creative, educational, and accessibility benefits. A balanced perspective is essential for proportionate policy.
✅ Legitimate Beneficial Uses
- Film and entertainment: digital de-ageing, posthumous performances with explicit consent, stunt replacement, dubbing with accurate lip sync
- Education: historical figures re-created to teach with full disclosure and factual accuracy; interactive language learning with native speaker synthesis
- Accessibility: personalised text-to-speech for people with conditions like ALS who bank voice samples for future use
- Healthcare simulation: medical training scenarios with synthetic patient cases not requiring real patient data or consent
- Journalism: re-creating historical events for which no footage exists, with explicit editorial disclosure
- Creative expression: new collaborative AI art and music involving consenting participants who retain creative control
✗ Harmful Applications
- Financial fraud via executive impersonation causing billions in corporate losses annually worldwide
- Non-consensual synthetic intimate imagery targeting individuals — primarily women and minors — with severe psychological consequences
- Election interference through fabricated statements by candidates, officials, or electoral authorities
- Corporate espionage and competitive disinformation campaigns targeting brand reputation
- Identity theft and biometric authentication bypass in financial and government systems
- State-sponsored propaganda and geopolitical destabilisation operations at national scale
- The Liar’s Dividend — undermining the evidentiary value of all authentic video and audio evidence
The ethical line between legitimate and harmful use runs through two gates: consent (did the person depicted agree to be represented in this way?) and transparency (is the synthetic nature clearly disclosed to anyone who might be affected?). Technology that passes both tests can be genuinely valuable. Technology that fails either test demands regulatory and social intervention.
Glossary of Key Terms
Sources & References
This document synthesises, analyses, and significantly expands upon content from the following authoritative institutional sources. All prose has been independently researched and rewritten. No text has been reproduced verbatim; all original analysis, SVG diagrams, analogies, and structural frameworks are original works created for this document.
IBM’s comprehensive risk catalogue covering bias, cybersecurity threats, data privacy, and environmental harms. Author: Rina Diane Caballar.
Detailed analysis of deepfake modalities, industry impacts, financial sector case studies, and deepfake fraud economics. Author: Srini Tummalapenta.
UNESCO’s analysis of synthetic media’s epistemological implications and media literacy frameworks. Author: Dr. Nadia Naffi, Université Laval, October 2025.
Cybersecurity-focused technical overview of deepfake technology, creation methods, and detection approaches for organisational defence.
Board-level risk analysis of deepfake threats, the democratisation of creation tools, and five practical organisational defence steps.
Carnegie Mellon ethics analysis of consent, identity rights, school deepfake incidents, and the Attenborough case. Author: Derek Leben, 2024.
Brookings Institution research on deepfakes’ impact on democratic institutions and the liar’s dividend concept.
Technical analysis of ten generative AI security risks including prompt injection, data leakage, and AI-powered malware generation.
Enterprise security perspective on AI data leakage, shadow AI usage, and DLP implications. Updated May 2026.
Monetary Authority of Singapore’s regulatory circular on deepfake risks for financial institutions and required risk controls.
US Department of Homeland Security assessment of deepfake identity threats to national security and law enforcement.
Australia’s national online safety authority on deepfake harms to individuals — particularly women and children — and regulatory response.
CSA’s technical framework for AI deepfake risks in cloud environments, detection tooling, and organisational governance approaches.
Strategic consulting perspective on organisational deepfake risk, threat modelling, and defensive investment priorities.
Assessment of organisational unpreparedness for AI-powered attacks and the gap between threat sophistication and current defensive posture.
Legal analysis of generative AI deepfake risks spanning disinformation, eSafety, authenticity, and emerging corporate liability frameworks.
Data breach response perspective on GenAI privacy risks, re-identification vulnerabilities, and regulatory notification obligations.
Technical guide to LLM-specific security vulnerabilities including prompt injection, model inversion attacks, and AI supply chain risks.
Singapore’s Monetary Authority circular on broader generative AI cyber risks for the financial sector, supplementing deepfake-specific guidance.
Supplementary sources including World Economic Forum risk rankings, Coalition for Content Provenance and Authenticity standards, and US deepfake legislation.