AI-901 FDP · Day 5 · All decks
DAY 5 · 09-Jul

Generative AI, Agents &
the Foundry SDK

The heart of AI-901 — Domain 2, 55–60% of the exam

Prompts — the three roles

RoleJob
systempersona · rules · grounding · format · guardrails
userthe actual request (+ image/audio)
assistantprior replies (context)

Techniques: clear instructions · few-shot · grounding · output format · chain-of-thought.

Rules/grounding go in the system message — never the user message.

Anatomy of a chat request

flowchart LR
  SY["system: rules + grounding"] --> M
  US["user: the request"] --> M
  AS["assistant: prior turns"] --> M
  M["Deployed model (deployment name)"] --> OUT["response.choices[0].message.content"]

Chat models are stateless per call — your app resends history.

Lightweight chat client (Foundry SDK)

from openai import AzureOpenAI
client = AzureOpenAI(azure_endpoint=EP, api_key=KEY,
                     api_version="2024-10-21")

r = client.chat.completions.create(
  model="gpt-4o-mini",             # deployment name
  messages=[
    {"role":"system","content":"You are a concise study coach."},
    {"role":"user","content":"Three tips for exam nerves?"}],
  temperature=0.7, max_tokens=300)
print(r.choices[0].message.content)

3 things to call a model: endpoint · credential · deployment name.

Check yourself

Where do you read the model's reply?
The generated text is in choices[0].message.content.

Pick the right model

Reasoning (o-series)

complex multi-step logic/math

Small (4o-mini/Phi)

cheap, fast, high-volume simple

Multimodal

image/audio input

Image-gen / Video-gen

create visuals

Embeddings

search / similarity / RAG

Choose by

capability · cost · latency · context window

Deployment types & config

Deployment types

  • Standard / Global Standard (pay-go)
  • PTU — reserved, predictable latency
  • Batch — async bulk
  • Serverless / Managed compute

Inference params

  • temperature/top_p randomness
  • max_tokens length
  • frequency/presence penalty repetition
  • seed reproducible · stop halt

Check yourself

Steady, high-volume production with predictable latency SLAs. Deployment type?
PTU reserves dedicated capacity → predictable latency. Batch = async bulk; Standard = pay-go, no reservation.

Agents — the equation

Agent = model + instructions + tools (+ knowledge)

…that can plan and take actions, not just reply once.

Tools: code interpreter · file search / knowledge · function calling · grounding/search

Agent vs chat: agent acts with tools; chat = text-in/text-out

Agent lifecycle

sequenceDiagram
  participant App
  participant Agent
  participant Thread
  participant Tool
  App->>Agent: 1 create_agent(model, instructions, tools)
  App->>Thread: 2 create_thread()
  App->>Thread: 3 add user message
  App->>Agent: 4 create_and_process_run(thread, agent)
  Agent->>Tool: (may call a tool)
  Tool-->>Agent: result
  Agent-->>Thread: assistant message
  App->>Thread: 5 read messages

Build & run a single agent

agent = project.agents.create_agent(
  model="gpt-4o-mini", name="study-buddy",
  instructions="You are an AI-901 tutor. Use the notes.")
thread = project.agents.create_thread()
project.agents.create_message(thread_id=thread.id,
  role="user", content="Explain grounding in one line.")
project.agents.create_and_process_run(
  thread_id=thread.id, agent_id=agent.id)
for m in project.agents.list_messages(thread_id=thread.id):
  print(m.role, ":", m.content)

Check yourself

Which needs an agent rather than a single completion?
Tool use + multi-step action = agent. The others are one-shot text-in/text-out.

Grounding & safety

Grounding / RAG

retrieve your data (embeddings + search) → add to prompt → fewer hallucinations

Content Safety

content filters (hate/violence/…) · prompt shields vs injection · evaluations

Day 5 labs

Lab 2 — SDK chat client

Run the Python client; change the system message + temperature.

Lab 3 — Single agent

Portal: agent + file-search over a PDF; then run it from the SDK.

Then: Practice Test 2 (live workshop).

Recap

  • system=rules, user=request, read choices[0].message.content
  • Model by capability/cost/latency; PTU for predictable latency
  • Agent = model + instructions + tools; lifecycle agent→thread→message→run→read
  • Grounding fixes hallucination; content filters + prompt shields

Next → Day 6: Revision + 50-Q mock