Azure AI Foundry — Deep Dive
Goal: Understand Azure AI Foundry end-to-end and build real-world AI applications on top of it.
📚 Research & Learning Resources
| # | Resource | Type |
|---|---|---|
| 1 | Azure AI Foundry Overview | 🎬 Video |
| 2 | AI Foundry Agent Intro | 🎬 Video |
| 3 | Official Learning Path | 📖 Docs |
🏗️ Platform Architecture
graph TD
A["Azure AI Foundry"] --> B["Hub"]
B --> C["Project A"]
B --> D["Project B"]
C --> E["Model Deployments"]
C --> F["Prompt Flows"]
C --> G["Evaluations"]
E --> H["GPT-4o / Llama / Mistral"]
F --> I["LLM Nodes"]
F --> J["Python Nodes"]
F --> K["AI Search Nodes"]
G --> L["Groundedness"]
G --> M["Relevance"]
G --> N["Coherence"]
style A fill:#0078d4,color:#fff
style B fill:#50e6ff,color:#000
style C fill:#50e6ff,color:#000
style D fill:#50e6ff,color:#000
🔧 Background & Prerequisites
1. Azure AI Services Ecosystem
AI Foundry acts as a unified surface over these Azure AI services:
| Service | What It Does | Key Use Case |
|---|---|---|
| Azure OpenAI | GPT-4, GPT-4o, DALL-E, Whisper, Embeddings | Chat, code gen, summarization |
| Azure AI Search | Vector + keyword hybrid search | RAG knowledge retrieval |
| Azure AI Speech | Speech-to-text, text-to-speech | Voice interfaces |
| Azure AI Vision | Image analysis, OCR, captioning | Visual understanding |
| Document Intelligence | Extract data from PDFs, invoices, forms | Structured extraction |
| Content Safety | Detect hate, violence, sexual, self-harm | Guardrails & moderation |
💡 Key insight: AI Foundry doesn't replace these services — it orchestrates them into cohesive applications.
2. Core Concepts
graph LR
Hub["🏢 Hub
(Shared resources)"] --> Project1["📁 Project
(Your AI app)"] Project1 --> Deploy["🚀 Model Deployment
(API endpoint)"] Project1 --> PF["⚡ Prompt Flow
(LLM pipeline)"] Project1 --> Eval["📊 Evaluation
(Quality metrics)"] Hub --> Connections["🔗 Connections
(OpenAI, Search, Storage)"] Hub --> Compute["💻 Compute
(GPU/CPU)"]
(Shared resources)"] --> Project1["📁 Project
(Your AI app)"] Project1 --> Deploy["🚀 Model Deployment
(API endpoint)"] Project1 --> PF["⚡ Prompt Flow
(LLM pipeline)"] Project1 --> Eval["📊 Evaluation
(Quality metrics)"] Hub --> Connections["🔗 Connections
(OpenAI, Search, Storage)"] Hub --> Compute["💻 Compute
(GPU/CPU)"]
| Concept | Description |
|---|---|
| Hub | Shared container — connections, compute, config. Created once per team. |
| Project | Workspace for a specific AI app. Has its own deployments, flows, evaluations. |
| Model Catalog | 1600+ models (Azure OpenAI, Meta Llama, Mistral, Cohere, HuggingFace). Deploy as MaaS or MaaP. |
| Playground | Interactive UI to test chat, completions, and image generation before coding. |
3. Prompt Flow
The primary tool for building LLM-powered applications inside AI Foundry.
flowchart LR
Input["📝 User Query"] --> Embed["🔢 Embedding Node"]
Embed --> Search["🔍 AI Search Node"]
Search --> LLM["🤖 LLM Node
(GPT-4o)"] LLM --> Safety["🛡️ Content Safety"] Safety --> Output["💬 Response"]
(GPT-4o)"] LLM --> Safety["🛡️ Content Safety"] Safety --> Output["💬 Response"]
Flow Types:
- 🔹 Standard Flow — General purpose LLM pipelines
- 🔹 Chat Flow — Conversational with history management
- 🔹 Evaluation Flow — Measure quality (groundedness, relevance, coherence)
Key Features:
- Nodes — LLM, Python, Prompt, Embedding — chain them together
- Connections — Stored credentials to Azure OpenAI, AI Search, custom APIs
- Variants — A/B test different prompts or parameters in the same flow
4. AI Agents in Foundry
graph TD
User["👤 User"] --> Agent["🤖 AI Agent"]
Agent --> Tools["🔧 Tools"]
Agent --> Knowledge["📚 Knowledge
(AI Search / RAG)"] Agent --> Safety["🛡️ Safety Rules"] Tools --> API["🌐 External APIs"] Tools --> Code["💻 Code Execution"] Tools --> Search["🔍 Document Search"]
(AI Search / RAG)"] Agent --> Safety["🛡️ Safety Rules"] Tools --> API["🌐 External APIs"] Tools --> Code["💻 Code Execution"] Tools --> Search["🔍 Document Search"]
| Building Block | Purpose |
|---|---|
| System Prompt | Instructions that define agent behavior |
| Tools | Functions the agent can call (defined as JSON schemas) |
| Knowledge Sources | Grounding data via Azure AI Search (RAG pattern) |
| Safety Rules | Content filters, prompt shields, PII detection |
🔮 Emerging pattern: Multi-agent orchestration — specialized agents coordinating to solve complex tasks.
5. Responsible AI & Safety
| Layer | What It Does |
|---|---|
| Content Filters | Block hate / sexual / violence / self-harm at 4 severity levels |
| Prompt Shields | Detect jailbreak attempts & prompt injection |
| Groundedness Detection | Verify responses are based on provided context |
| PII Detection | Identify & redact personal information |
✅ TODO — Remaining Work
| # | Task | Priority |
|---|---|---|
| 1 | Create an AI Foundry Hub + Project (with screenshots) | 🔴 High |
| 2 | Deploy GPT-4o from the model catalog | 🔴 High |
| 3 | Build a simple chat app in the Playground | 🔴 High |
| 4 | Create a Prompt Flow: query → AI Search → LLM response | 🔴 High |
| 5 | Build an AI Agent with tool calling (weather API + doc search) | 🟡 Medium |
| 6 | Run evaluation flow (groundedness + relevance metrics) | 🟡 Medium |
| 7 | Configure content filters & test adversarial prompts | 🟡 Medium |
| 8 | Compare AI Foundry vs LangChain/LlamaIndex for RAG | 🟢 Low |
| 9 | Document pricing model & cost optimization | 🟢 Low |
| 10 | Final architecture diagram of full Hub→Project→Deploy→Agent flow | 🟢 Low |