AI Is Writing Code with Backdoors — Without Even Realizing It

A recent study has exposed a hidden danger in code written by generative AI: models frequently "hallucinate" software components that don’t actually exist, creating potential gateways for cyberattacks. Researchers from the University of Texas at San Antonio analyzed 576,000 code snippets generated by 16 popular AI models, including GPT-4 and Claude. In 19.7% of cases, the code included references to fake libraries — totaling around 440,000 invalid dependencies.

Hackers can exploit this by registering packages with names that AIs commonly invent by mistake and filling them with malicious code. If developers install these components without verifying their authenticity, the malware gets in. During the experiment, test packages with made-up names were downloaded tens of thousands of times.

Key findings:

This technique, known as Dependency Confusion, works by replacing a legitimate package with a malicious one that has the same name. For example, a hacker publishes a Trojan-laced package to a repository. If the AI suggests it instead of the official version and a developer installs it without double-checking the source, the system gets compromised.

With Microsoft predicting that AI will write 95% of all code by 2030, these findings are a wake-up call. Until neural networks can reliably distinguish fiction from fact, the burden of security remains firmly on human shoulders.