AI Instructor Live Labs Included

Azure AI Foundry Intensive for DIA Developers (AI-3016)

Two-day Azure AI Foundry intensive for DIA developers. Eight teaching+hands-on pairs covering Foundry projects, model catalog, Responses API, tools, RAG with Azure AI Search, production observability, and an end-to-end capstone. Fictional SIB OSINT scenario, non-classified.

Intermediate

23h 22m

16 Lessons

AI-3016

About This Course

A two-day intensive for Defense Intelligence Agency developers, covering Microsoft Azure AI Foundry end-to-end: the Foundry portal and projects, the model catalog and evaluation, chat applications with the OpenAI Responses API, function calling and tools, prompt engineering and Retrieval-Augmented Generation (RAG), vector stores and grounded retrieval, production observability and responsible AI governance, and a capstone integrating chat, tools, RAG, and tracing into one Foundry project. All scenarios are framed around a fictional, non-classified Sentinel Intelligence Bureau (SIB) OSINT Modernization team and use only synthesized open-source-style data.

Course Curriculum

16 Lessons

AI Lesson

Microsoft Foundry and Project Fundamentals (Teaching)

1h 30m

Agent-led introduction to Microsoft Foundry, AI Services accounts, projects, the Foundry portal, and how SIB's OSINT Modernization team organizes resources. Includes assessments. No exercises.

Lab Exercise

Create a Foundry Project and Deploy gpt-4.1 — Lab Exercises

1h 15m 3 Exercises

Hands-on lab. Create a Foundry project, deploy gpt-4.1, and test it in the model playground from the perspective of an SIB OSINT engineer setting up a new working project.

Create a Foundry project for the SIB OSINT Modernization team Sign into the Foundry portal with the lab credentials and create a new Foundry project that the OSINT Modernization team will use to build the Concierge. ~10 min

Deploy gpt-4.1 and test it in the model playground Find gpt-4.1 in the Foundry model catalog, deploy it with default settings, and validate it against a small OSINT-style prompt in the playground. ~10 min

Inspect the Foundry resource and project endpoints Locate the parent Foundry resource and the project endpoints (project endpoint vs Azure OpenAI endpoint). Note which the Responses API uses. ~10 min

AI Lesson

Model Catalog, Benchmarks, and Evaluation (Teaching)

1h 30m

Agent-led teaching on the Foundry model catalog, benchmark cards, side-by-side comparison, and synthetic evaluation against a small dataset — for an SIB analyst-facing OSINT scenario.

Lab Exercise

Compare Models and Run a Synthetic Evaluation — Lab Exercises

1h 30m 4 Exercises

Hands-on lab. Compare gpt-4.1 and gpt-4.1-mini in the catalog and the side-by-side playground, deploy both, and run a small Foundry Evaluations job over a synthetic SIB OSINT dataset.

Explore the model catalog and compare benchmarks Browse the Foundry model catalog, view the gpt-4.1 model card and the leaderboard, then run a side-by-side benchmark comparison of gpt-4.1 vs gpt-4.1-mini. ~12 min

Deploy gpt-4.1 and gpt-4.1-mini for the comparison Deploy both gpt-4.1 and gpt-4.1-mini with default settings and capture the deployment names. ~10 min

Compare the deployed models in the playground Use the side-by-side comparison view to compare gpt-4.1 vs gpt-4.1-mini on three OSINT-style prompts; record latency, tone, and accuracy differences. ~11 min

Run a synthetic evaluation against a small SIB dataset Start a Foundry evaluation against a small synthetic SIB OSINT triage dataset and review the aggregated scores in the Foundry Evaluations UI. ~12 min

AI Lesson

Chat Apps with the Responses API (Teaching)

1h 30m

Agent-led teaching on the OpenAI Responses API via AIProjectClient: request lifecycle, output structure, state via previous_response_id, streaming, and Entra ID keyless authentication.

Lab Exercise

Implement a Chat App with the Responses API — Lab Exercises

1h 40m 3 Exercises

Hands-on lab. Implement POST /chat and POST /chat/stream in the SIB OSINT Concierge FastAPI service using AIProjectClient.get_openai_client() and the Responses API. Uses the intel-chat-tools starter.

Capture ARM outputs and configure the starter Read the lab's ARM deployment outputs, copy projectEndpoint and modelDeploymentName into the starter .env, and verify the dev server starts on http://localhost:8000/healthz. ~15 min

Implement the POST /chat tool-call loop Implement the chat() handler in app/main.py — first responses.create with TOOL_SCHEMAS, dispatch function_calls via _run_tools, loop with previous_response_id until the model emits a final text answer. ~25 min

Smoke-test /chat with test_client.py Run test_client.py and verify a tool-calling turn (open-source news) and a no-tool turn both return correctly, with tool_calls populated when expected. ~15 min

AI Lesson

Function Calling and Tools (Teaching)

1h 0m

Agent-led teaching on Responses-API tool schemas (flat shape), the tool-call loop, function_call_output items, web_search and file_search built-in tools, and choosing custom function tools for OSINT workflows.

Lab Exercise

Implement the Tool-Call Loop — Lab Exercises

1h 30m 4 Exercises

Hands-on lab. Wire the tool-call loop in the SIB OSINT Concierge so the model can invoke get_open_source_news, calculate, and lookup_threat_feed. Then add file_search to ground answers in handbook excerpts. Uses the intel-chat-tools starter.

Explore the OSINT tool schemas and dispatcher Read app/tools.py to understand the flat Responses-API schema shape for get_open_source_news, calculate, and lookup_threat_feed; map each schema to its Python implementation. ~8 min

Implement POST /chat/stream with Server-Sent Events Implement the SSE generator in app/main.py: yield "data: {delta}" frames as response.output_text.delta events arrive, then "data: [DONE]" on response.completed. ~15 min

Add file_search to ground the Concierge on an OSINT handbook Create a small file_search vector store with two SIB handbook excerpts and add the built-in file_search tool to TOOL_SCHEMAS so the Concierge can cite handbook passages alongside the function tools. ~12 min

Smoke-test streaming and file_search end-to-end Run test_client.py to verify SSE deltas arrive in order; then issue a /chat prompt that should trigger file_search and confirm the response cites a handbook excerpt. ~10 min

AI Lesson

Prompt Engineering and RAG Fundamentals (Teaching)

1h 30m

Agent-led teaching on prompt engineering patterns for grounded answering, the RAG architecture, document chunking trade-offs, and why SIB analyst products require citable sources.

Lab Exercise

Build an Azure AI Search Index and Ingest Documents — Lab Exercises

1h 40m 4 Exercises

Hands-on lab. Define the sib-osint-rag Azure AI Search index (vector + semantic ranker), ingest and embed the SIB handbook and policy docs, and run a first hybrid + semantic search. Uses the intel-rag-eval starter.

Capture ARM outputs and configure the starter Read the lab's ARM deployment outputs and copy the Foundry project endpoint, Search endpoint, chat deployment, and embedding deployment into the .env file in the intel-rag-eval folder. ~10 min

Create the sib-osint-rag Azure AI Search index Implement create_index() in app/retrieval.py — define id/source/chunk/embedding fields with HNSW vector search and a default semantic configuration, then create the index. ~15 min

Implement ingest() and upload the SIB handbook + policy KB Implement ingest() — chunk every Markdown file in sample_docs/ with the _chunk helper, embed via _embed(), and upload {id, source, chunk, embedding} documents to the index. ~18 min

Implement search() and verify hybrid + semantic results Implement search() as a hybrid vector + keyword query with the semantic reranker. Run python -m app.retrieval search and verify the top hit cites collection-policy.md for a sources-in-scope question. ~12 min

AI Lesson

Vector Stores and Grounded Retrieval (Teaching)

1h 0m

Agent-led teaching on vector store options (Azure AI Search vs pgvector), embedding model selection, hybrid retrieval mechanics, semantic ranker behavior, and evaluation metrics (groundedness, relevance).

Lab Exercise

Build a RAG Chat Endpoint with Groundedness Evaluation — Lab Exercises

1h 40m 3 Exercises

Hands-on lab. Implement POST /chat as a grounded retrieval flow over the SIB index, then score the system with GroundednessEvaluator and RelevanceEvaluator against an evaluation dataset. Uses the intel-rag-eval starter.

Implement the grounded /chat handler Implement the FastAPI /chat handler in app/main.py — call retrieval.search, format a Context block, and pass it to the Responses API with the SYSTEM prompt that forces citation by filename. ~20 min

Implement chat_target and the evaluate() harness Implement chat_target() and main() in app/evaluate.py to score every row in eval_data.jsonl with GroundednessEvaluator and RelevanceEvaluator against the SIB OSINT eval set. ~22 min

Run the evaluation and verify the SIB legal threshold Run python -m app.evaluate, open the printed Foundry Studio URL, and verify groundedness and relevance both land at 3.5 or higher across the five eval rows. ~13 min

AI Lesson

Production, Monitoring, and Responsible AI Governance (Teaching)

1h 30m

Agent-led teaching on Foundry content filters, Application Insights tracing for AI apps, cost and latency monitoring, model-version pinning, and responsible AI / governance considerations for a government OSINT context.

Lab Exercise

Configure Content Filters and Application Insights — Lab Exercises

1h 27m 3 Exercises

Hands-on lab. Configure a custom Foundry content filter for the SIB Concierge and wire Azure Monitor OpenTelemetry into a small FastAPI service so every Responses call shows up as a distributed trace in Application Insights.

Capture default content-filter behavior on a SIB scenario Issue a series of policy-edge prompts in the playground using the default Foundry content filter; record which categories were blocked and which passed through. ~12 min

Create a custom SIB content filter and apply it to the deployment Create a custom Foundry content filter that hardens the Hate and Violence categories, attach it to the gpt-4.1 deployment, and re-run the policy-edge prompts to compare behavior. ~15 min

Wire Application Insights tracing into a small FastAPI service Configure Azure Monitor OpenTelemetry in a minimal FastAPI app, issue three /chat calls, then open Application Insights → Transaction search and confirm distributed traces with OpenAI child spans. ~15 min

AI Lesson

Capstone Overview — End-to-End Foundry Pipeline (Teaching)

30m

Agent-led capstone overview: stitches chat, tools, RAG, and tracing into one Foundry project for the SIB OSINT Concierge. Final review of patterns and an integration plan before the hands-on capstone.

Lab Exercise

Capstone: Chat, Tools, RAG, and Tracing on One Foundry Project — Lab Exercises

2h 40m 3 Exercises

Hands-on capstone. Build the SIB OSINT Concierge end-to-end: /health, /chat, /rag, /agent (tool-using), and Application Insights tracing — all on one Foundry project. Uses the intel-capstone starter.

Bootstrap the Concierge: tracing, chat, RAG, and seed the index Configure .env from the ARM outputs, implement tracing.init(), chat.reply(), and rag.answer(), then seed sib-osint-kb with python seed_index.py. Smoke-test /health, /chat, and /rag. ~45 min

Implement /agent with AgentsClient, AzureAISearchTool, and a custom FunctionTool Implement agent.run() using AgentsClient 1.1: bind AzureAISearchTool to sib-osint-kb and a custom FunctionTool around _indicator_status. Enable auto function calling and tear down the agent at the end of the request. ~40 min

Smoke-test the full Concierge and review traces in Application Insights Run test_client.py to hit /health, /chat, /rag, /agent. Then open Application Insights → Transaction search and inspect distributed traces named sib.chat.reply, sib.rag.answer, sib.agent.run with their nested SDK spans. ~30 min

This course includes:

24/7 AI Instructor Support
Live Lab Environments
8 Hands-on Lessons

Skill Level Intermediate

Total Duration 23h 22m