# (WIP) AI/ML/LLM Application Security Testing

Any AI/ML/LLM-related application should undergo pen testing and red teaming activities according to size and needs. We should pay the same attention to traditional applications here. There is no difference in the risk perspective here. Due to the model's capabilities and improvision nature, high-profile risks are more likely.

In the application security testing, I prefer to examine under two headlines:\
\- Predictive AI System Security Testing\
\- Generative AI System Security Testing

In general, both have a model and data the models are trained on. However, the first-category systems involve using machine learning models to forecast future events or behaviours based on historical data. The second-category systems are working on generating entirely new results using the experience they got from the provided data.\
\
Both sides share some architectural parts and joint attack surfaces. For example, both have training data, models, and tokenization as their central parts. For the attack surface parts, both have similar tokenization issues, privacy attacks, and model bias risks on their attack surfaces.

All experienced and skilled security testers have a common approach: Know your target! So, whether you're Predictive or Generative, you should know your target. First questions to know your target:

* What kind of model does it use?
* How it's trained, and how the fine-tuning cycles will go, if there are any?
* What is the data source? Could it be poisoned?
* Where is the tokenization? What kind of tokens can you use for manipulating the model via inputs? Are there any special tokens that can trigger some behaviours like prediction? For example, MASK token.
* What kind of data storage is used? <br>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://lookbook.cyberjungles.com/ai-ml-llm-security/wip-ai-ml-llm-application-security-testing.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.