You can query the HuggingFace API for details of the account and proceed with an attack on someone’s MLOps infrastructure:

Okay, let’s say that access to the chatbot’s history is quite a serious issue. But can misconfigured Flowise impact other systems in our organization? Yes, it can!

Flowise falls into the fancy-named LLMOps category of software. It’s a visual tool that enables users to build customized LLM flows using LangchainJS.

All of the issues described above aren’t really security vulnerabilities. The main mitigation for the threats that I’ve described is configuring applications in your MLOps stack correctly, so these aren’t exposed to the public network without proper authentication. You should also process your secrets and credentials carefully. Of course that would be nice to have MLOps tools “secure by default”, but it will take a while I guess…

Nevertheless we talk about some old-school industrial control system for sewage tanks, self-hosted NoSQL databases or modern MLOps software, one thing never changes – misconfigurations happen. No matter how comprehensive the documentation is, if the given software is not secure by default, there will always be at least a few people who would deploy their instance so heavily misconfigured, that you start to wonder whether what you’ve encountered is a honeypot.

I am not sure how it works, but some of the credentials in Flowise are encrypted, meanwhile some are just stored in plaintext format, waiting for cybercriminals to make use of them.

Omniboard is a web dashboard for the Sacred machine learning experiment management tool. It connects to the MongoDB database used by Sacred and helps in visualizing the experiments and metrics / logs collected in each experiment.

Prefect is a modern workflow orchestration [tool] for data and ML engineers. And very often, it’s available without any authentication on the Internet. That applies to the self-hosted usage of Prefect.

I’ve spent a cozy evening  with Shodan and in this post I will give you a few examples of funny misconfigurations in various MLOps-related systems. Maybe I will provide some recommendations as well. Last but not least, I want to highlight the issue with the worryingly low level of security in the MLOps/LLMOps (call it as you wish, DevOps for AI or whatever) area.

That’s right, a fresh and fragrant OpenAI API key. Why was it returned to my browser? I don’t know. What I know is the fact that dozens of OpenAI API keys can be stolen this way. But it’s not just OpenAI keys that are at risk, I’ve seen plenty of other keys stored this way.

Of course, by default it does not support the authentication, so there’s plenty of the Omniboard instances exposed on the Internet.

In Prefect Server, you can create flows (a flow is a container for workflow logic as-code), each of the flows composed of a set of tasks (task is a function that represents a discrete unit of work in a Prefect workflow). Then, once you have your flows created, you want to run them as a deployment. You can store configuration for your flows and deployments in blocks. According to the documentation of Prefect:With blocks, you can securely store credentials for authenticating with services like AWS, GitHub, Slack, and any other system you’d like to orchestrate with Prefect.

That contains: chatbots’ history, initial prompts of your LLM apps, all of the documents stored and processed by the LLM chains and even the API key (I guess the API key is useful only if the authentication is enabled, otherwise it is not needed).

In the example above you can see a combo – not only the source code is exposed, but also hardcoded credentials to access the MongoDB instance.

Of course, Flowise doesn’t offer authentication by default (it’s super-easy to set up, as far as documentation says, but it’s not default though). Access to the “control center” of LLM-based app it’s dangerous by itself, as by manipulation of the LLM parameters an attacker may spread misinformation, fake news or hate speech. But let’s check what else can we achieve through the access to the unauthenticated instances of Flowise.

At the very beginning I just assumed that this form named “OpenAI API Key” is just a placeholder or something like that. Nobody would store API keys like that, right?… Well, here’s what I saw after I clicked “Inspect” at this element:

This post is not a disclosure of any specific vulnerabilities, it mainly focuses on the misconfigurations. Companies or individuals that were impacted by the described misconfigurations have been informed and – at least in most of the cases – I got a quick response and misconfigurations were fixed.

Through Omniboard you can take a look into the source code of the experiment. That’s pretty risky, as if you are developing your model for commercial purpose, I assume you’d rather want your code to remain confidential and inaccessible to the competitors.

The thing is you can, but you don’t have to, if you really don’t want to. Some blocks enable user to store secrets in plaintext, for example JSON block:

Another block, that discloses the secrets is the SQLAlchemy Connector* (*but only in some cases). Below you can see an example of Postgres database credentials – available in plaintext, without authentication:

If you want to receive the latest news from the world of AI security, subscribe to my newsletter: https://hackstery.com/newsletter/

I have informed owners of the exposed credentials of the issue. But in the first place, there wouldn’t be any issue, if they took care and deployed Prefect properly. 

I watched this inspiring talk today. On the one hand, my interest in MLOps tooling security and vulnerabilities had been growing for some time, yet on the other hand, I was somewhat uncertain about how to approach it. Finally, after watching Dan’s talk, I decided to start with so-called low hanging fruits – vulnerabilities that are easy to find and often have a critical impact.

So, while OpenAI key theft may only lead to consumption of the funds on the card, which is connected to the OpenAI payment system, leak of GH or HuggingFace keys may lead to theft of your code, theft or deletion of your trained ML models etc.

In this case, leaked keys belong to a few individuals (I’ve contacted them and they have hidden their Flowise deployments and re-generated the keys).

There is a magic endpoint in the Flowise – /api/v1/database/export. If you query this endpoint, you may download all of the data available in the given instance of Flowise.