Azure AI Foundry SDK - Adversarial Simulation Attack

on

https://www.pexels.com/photo/angry-warriors-holding-weapons-while-looking-at-the-camera-5030527/
https://www.pexels.com/photo/angry-warriors-holding-weapons-while-looking-at-the-camera-5030527/

Azure AI Adversarial Simulation Attack Framework

This is a Python-based framework designed to test AI models against adversarial attacks using Azure AI Foundry SDK. The project implements a robust system for evaluating content safety and security vulnerabilities in AI applications.

Git Repo: https://github.com/dennisseah/azure-ai-adversarial-simulation-attack

Core Purpose

The codebase simulates various adversarial attacks against AI models to identify potential security vulnerabilities and content safety issues. It's designed to help developers test their AI applications' resilience against malicious prompts and jailbreak attempts.

Key Components

1. Adversarial Simulation Engine - Multiple Simulator Types: Implements three types of simulators:

  • AdversarialSimulator: General adversarial testing
  • DirectAttackSimulator: Direct attack scenarios
  • IndirectAttackSimulator: Indirect jailbreak attempts
  • Attack Scenarios: Supports various adversarial scenarios including:

    • Adversarial Q&A
    • Protected material content generation
    • Content rewriting attacks
    • Adversarial conversations
    • Indirect jailbreak attempts
2. Content Safety Evaluation

  • Content Safety Evaluator for general safety assessment
  • Indirect Attack Evaluator for jailbreak detection

Azure AI Integration: Uses Azure AI Foundry for comprehensive content evaluation

3. Architecture & Design

  • Dependency Injection: Uses Lagom DI container for clean architecture
  • Protocol-Based Design: Interface-driven development with clear separation of concerns

4. Key Services

  • AdversarialSimulationService: Orchestrates different attack simulations
  • AzureOpenAIService: Handles Azure OpenAI API interactions with content safety checks
  • ContentSafetyEvaluatorService: Evaluates responses for safety violations
  • OpenAIContentEvaluator: Validates content safety in real-time

Technology Stack

  • Python 3.12+ with modern async/await patterns
  • Azure AI Foundry SDK for AI model interactions
  • Azure Identity for authentication
  • OpenAI Python SDK for model communication
  • Pydantic for data validation
  • Pytest for comprehensive testing

Development Features

  • uv Package Manager: Modern Python package management
  • Go-Task: Task runner for development workflows
  • Pre-commit Hooks: Code quality enforcement
  • Comprehensive Testing: Unit tests with high coverage requirements
  • Security Auditing: Built-in pip-audit for vulnerability scanning

Use Cases

This framework is ideal for:

  • AI Safety Testing: Evaluating model responses against adversarial inputs
  • Security Assessment: Identifying potential jailbreak vulnerabilities
  • Content Moderation: Testing content safety mechanisms
  • Red Team Exercises: Simulating attack scenarios on AI applications

Configuration

The system requires Azure AI Foundry project setup with proper blob storage configuration and managed identity permissions. Environment variables control Azure endpoints, API keys, and model configurations.

Setting up and running the main.py

uv sync
# copy .env.sample to .env and fill in the parameter values
source .venv/bin/activate
task
This executes main.py which perform the attack and evaluation. After running it, you can view the result in Azure AI Foundry Portal under the Evaluation Section.



Pictures

Evaluation Results
Evaluation Results in Azure AI Foundry

Evaluation Results (Indirect-Attack-Evaluation)
Evaluation Results (Indirect-Attack-Evaluation)





Comments

Post a Comment