Hugging Face, a prominent online repository for generative AI, is facing significant security challenges as hackers have uploaded thousands of malicious models to its platform, the Forbes reports. This situation has raised alarms among security researchers who warn that these models can contain hidden code capable of poisoning data and stealing sensitive information.
According to researchers from security firms ProtectAI, Hiddenlayer, and Wiz, Hugging Face has become a prime target for cybercriminals. The platform currently hosts over a million models available for download, and many of these have been found to harbor malicious code. Ian Swanson, CEO and founder of ProtectAI, noted that the traditional computer viruses have evolved into more sophisticated threats in the AI era. “The old Trojan horse computer viruses that tried to sneak malicious code onto your system have evolved for the AI era,” he stated.
The researchers discovered tens of thousands of these harmful models during their scans of Hugging Face. Some hackers have even created fake profiles on the platform, impersonating well-known technology companies such as Meta, Facebook, and Visa to deceive users into downloading their malicious models. Swanson highlighted a particular model that falsely claimed to be from the genomics testing company 23AndMe, which had been downloaded thousands of times before it was identified as a threat. This model was designed to silently search for Amazon Web Services (AWS) passwords, which could be exploited to steal cloud computing resources.
In response to these security threats, Hugging Face has integrated ProtectAI’s scanning tool into its platform. This tool allows users to see the results of scans for malicious code before they download any models. Hugging Face has also taken steps to verify the profiles of major companies like OpenAI and Nvidia since 2022. Julien Chaumond, the Chief Technology Officer of Hugging Face, expressed hope that their collaboration with ProtectAI would enhance trust in machine learning artifacts, making sharing and adoption safer.
The risks associated with these malicious models have prompted a joint warning from the United States Cybersecurity and Infrastructure Security Agency (CISA) and security agencies from Canada and Britain. In April, the National Security Agency (NSA) and its counterparts cautioned businesses to thoroughly scan any pre-trained models for harmful code and to run them only in isolated environments away from critical systems.
The hackers targeting Hugging Face typically inject rogue instructions into the code that developers download. This allows them to hijack the model when it is executed by an unsuspecting user. Swanson remarked, “These are classic attacks but they’re just hidden within models. Nobody would know that the model is doing these nefarious things and it would be incredibly hard for them to be able to trace that back.”
Hugging Face, which was last valued at $4.5 billion following a $235 million funding round in August 2023, has evolved significantly since its founding in 2018. Originally started as a teenage-focused chatbot app, the company has transitioned into a platform for machine learning and has raised a total of $400 million to date. It has been referred to as the “GitHub for AI researchers,” reflecting its central role in the AI community.
With the continuing growth of Hugging Face’s popularity, the potential for malicious actors to exploit its platform also increases. Chaumond acknowledged this reality, stating, “For a long time, AI was a researcher’s field and the security practices were quite basic. As our popularity grows, so does the number of potentially bad actors who may want to target the AI community.”
