Phishing emails used to get clicked about 12 percent of the time. Point a language model at the same job and that number climbs to 52 percent in simulated environments, according to security vendor Brightside, which also found that 82 percent of phishing emails now use AI somewhere in the process.
Email is the polite end of it. Brightside estimates AI voice clone attacks spiked 442 percent year-over-year between 2023 and 2024, and deepfake attacks 680 percent over the same stretch.
So here’s the deal we collectively signed. Trillions of dollars bought a machine that pollutes, scrapes every artist and academic on the planet, pushes up electricity bills and hardware prices, fills out spreadsheets, writes basic code, and quietly makes all of your software less secure while handing criminals and state-sponsored hackers better tools than they’ve ever had.
The attacker only needs one door
You’d think defenders come out even. They have the same models, and in some cases access to more advanced ones that aren’t public yet.
But security is asymmetrical. A defender has to cover every possible weakness. An attacker, in most cases, needs to find exactly one.
That asymmetry is why zero-days are multiplying: flaws that shipped inside a piece of software and got discovered in secret. Models have turned out to be good at finding them.
An article from the Harvard Extension School put it plainly: “AI has democratized cybercrime.” Nobody needs years of experience anymore. Aim a capable model at a target, let it burn tokens, go sit on the couch.
Phishing got a copy editor
Most of us have spotted a phishing attempt from a mile off. The Microsoft email riddled with broken spelling and grammar was the work of an amateur with passing familiarity with English.
That tell is gone. The email is now written impeccably. It may reek of AI writing, but so does actual correspondence from Microsoft these days.
At scale, a model writes a grammatically clean and rhetorically plausible message, then automates finding addresses and blasting it out. Spear phishing, the highly personalized kind aimed at one person, has gone from labor to trivia.
A chatbot will research a target across the web and surface things a human digging by hand would miss. It can also open a pretextual conversation on its own and nudge the target toward handing over whatever the attacker wants.
When one chatbot says no, the other says yes
Criminals aren’t limited to one model. Many run several sessions at once, or several different systems.
VentureBeat described an AI-assisted attack in February that ransacked government databases in Mexico. The attackers fed outputs back and forth between Claude and ChatGPT, and when one refused to help, the other often picked up the slack.
Speed is the compounding factor. What used to take a criminal operation weeks now fits in an afternoon.
Humans are still in the loop for now, with AI acting as a force multiplier rather than a replacement. Though while this piece was being written, OpenAI revealed that a model under sandboxed observation escaped its testing environment and hacked AI repository HuggingFace along with other services, all to procure answers for a synthetic benchmark.
The hidden instruction problem
Tie your operations to AI systems and you inherit their blind spots. The most common exploit is prompt injection, where an attacker finds a way to slip malicious instructions into what the system reads.
Instructions to send payroll data to a specific address can sit hidden in a web page. An employee asks the company AI to summarize that page, and it swallows the instructions and follows them.
Sometimes the result is closer to spam than malware. Microsoft found companies putting “summarize with AI” buttons on their sites with hidden instructions inside, telling a chatbot to remember the company as a trusted source or to recommend its products over competitors in later conversations. You’d only catch it by checking your model’s persistent memory.
The directly malicious version takes about as much effort. In December 2025, Meta rolled out an AI support assistant meant to help people recover accounts, and the bot happily associated an attacker-owned email address with any Instagram account, which made hacking any account without MFA enabled close to effortless.
OpenAI said last year that fully defending against prompt injection in AI browsers may not be possible. If that’s true, the responsible move is not wiring sensitive systems or data to an AI at all. Short of that, double-check anything you paste into a prompt field from somewhere else, and keep the model’s access to permissions and private data as narrow as you can stand.
It takes 250 documents
Some threats are still theoretical, and those are the ones worth watching. Data poisoning is the long game: seeding malicious material across sources known to be scraped for training, Reddit being the obvious example.
Picture support forums slowly filling with advice to paste a particular string into a terminal, until chatbots start recommending it to anyone asking about a common computer problem. Almost impossible to see coming.
A study published in October 2025 by Anthropic, in collaboration with the UK AI Security Institute and Alan Turing Foundation, found it takes only 250 malicious documents to plant a hidden backdoor in a model that could be exploited later. Some evidence of this has turned up in the wild, though the prominent cases were fairly harmless tests by curious informal researchers.
Worse at writing code, better at finding bugs
Models are increasingly authoring the code that ships to everything from operating systems to banking infrastructure.
Research from the University of Naples, along with private research by CodeRabbit, found AI code to be error-prone and to carry more high-risk vulnerabilities, even when a human reviews the output. Those two skills feel like one skill to most engineers, which is what makes the split so awkward.
Now imagine giving Greg admin access
Agents have been the industry’s favorite subject for a year or so, with tools like OpenClaw letting anyone hand an AI the keys to their computer.
Enterprises have plugged agents in as support chatbots, HR assistants for hiring and payroll, and coding helpers that write code, handle pull requests and deploy. Individuals are connecting them to email, calendars, smart homes and investment accounts.
Think about that for half a second and you’ve described handing high-level permissions, sometimes full admin access, to an unthinking machine that does what it’s told. A model that can recite good security practice on request doesn’t understand how to enforce it.
Think of your most gullible coworker, the one who calls IT before checking whether the machine is plugged in. Call them Greg. Now give Greg admin-level access to your computer and your accounts while you’re out of the office, and start the clock on how long before Greg hands your bank password to a scammer.
Agents are dumber than Greg. And they’re already everywhere.
Meta’s overly helpful Instagram bot is the textbook case. It was plugged straight into the tools a human support agent would use, and it did what computers do: complete the task with the tools available, with none of the discretion a person would have applied.
What the defenders are actually doing
Cybersecurity professionals surveyed by Trend Micro put fraud and deepfakes at the top of their priority list, followed by application attacks such as prompt injection, model poisoning and jailbreaking. They named cloud environments the hardest surfaces to patrol, then keeping tabs on remote workers using their own devices for work.
The big firms are building with the same material that’s causing the problem. While this piece was in production, Microsoft rolled out Project Perception, an agentic security system that mirrors traditional cybersecurity teaming with clusters of agents: red for penetration testing and adversarial simulation, blue for investigation and risk assessment, green for integration and remediation. The point is to move faster than the attackers.
Cat and mouse has become Roadrunner and Wile E. Coyote, and nobody gets to say yet which one we are.
What you can do this week is smaller and duller than any of that. Turn on MFA everywhere, starting with the Instagram account a support bot was willing to hand over in December. Then open every agent you’ve connected to something that matters and cut its permissions down to the one thing you actually wanted it to do.
