JustForFun
  • Hello World
  • AI/ML/LLM Security
    • 🐣What is this AREA?
    • 📚Self-Study AI/ML/LLM Roadmap for Security Folks
    • 🌟AI/ML Dictionary
    • 🌰Generative AI in a Nutshell
    • 👹(WIP) AI/ML/LLM Application Security Testing
      • 💉(WIP) Offensive Approach for Prompt Injection Attacks
      • 👾Standard Input: Prompt Injection
      • ⚠️(WIP) Training Issues
      • 🎑(WIP) Multi-Modal LLM Application Security Testing
      • ✨(WIP) Resources
  • Random Research Area
    • What is this AREA?
    • Phishing with MS Office Docs
      • VSTO and Malicious Office Docs
    • Malware Analysis & Development
      • Malware Development
  • AppSecNotes
    • 3rd Party Recon with Javascript - Part1
    • DAV Methods and Old Features
    • API Security Notes
  • OSEP Preperation Notes
    • OSEP Journey Begin!
    • Basics and More
    • Payload Types (Staged vs. Non-Staged)
    • File Smuggling with HTML & JS
    • VBA Basics
    • Basic Phishing Macro Creation Tricks
  • Somethings and Past
    • HackTheBox Lab – Invite Code Write-Up
    • OSCP Yolculuğum
    • VulnHub – SkyTower CTF Walkthrough
    • Markdown Syntax
    • Web Uygulama Güvenliği Ve Güvenli Kod Geliştirme LYK-2014 Notlarım
    • Yalnızca Eğlenmek İçin
Powered by GitBook
On this page

Was this helpful?

  1. AI/ML/LLM Security
  2. (WIP) AI/ML/LLM Application Security Testing

(WIP) Offensive Approach for Prompt Injection Attacks

As we discussed earlier, Prompt Injection can be a way to exploit many different vulnerabilities in your Generative AI Application.

All the security engineers and folks may not be scientists, but security can be practical in fascinating ways. The best way to secure complex stuff is to test them in practical and simplified ways.

Let's jump on the approach.

  1. Analyze the architecture - Recon

Offensive testing starts with good reconnaissance techniques. Security is still security, so you should know your target. I assume you're going through approved white-box testing for your role.

  • Read all the architectural references, even for little plugins that may impact the system. Then, re-draw the architecture in your way. In my way, I track down the user input's journey and re-draw that journey in the security language.

  1. Track down the Input

In every testing discipline, input manipulation-response detection is the most helpful tactic for discovering weaknesses. It's the same in LLM-integrated apps. You need to identify if your input changes after you enter it. This change can increase the context quality with extra reasoning, safety filter masking/shaping before feeding it to the LLM, etc.

  1. Code Review is your BFF

In a good code review, you should identify prompt filters, reasoning points, RAG search queries/field names, agency relationships like executions, data searches, prompt template details like template-supported filters/guardrails, etc.

Identifying all those critical points in the code review may give you better visibility for your tracked input. If you track down your input like in the 3rd step, now you can plan which point you can manipulate.

Previous(WIP) AI/ML/LLM Application Security TestingNextStandard Input: Prompt Injection

Last updated 10 months ago

Was this helpful?

👹
💉