DAY 4 · 08-Jul
Text Analysis, Speech &
Information Extraction
NLP concepts + Azure Speech in Foundry Tools + Azure Content Understanding
How NLP works — 3 layers
Tokenization
split text into tokens for processing
Statistical
word counts / TF-IDF / bag-of-words — no real meaning
Semantic
embeddings + transformers = meaning & context
"counts word frequency" → statistical. "synonyms match / context-aware" → semantic.
Four text-analysis techniques
| Technique | Returns |
|---|
| Keyword extraction | main talking points/terms |
| Entity detection (NER) | Person / Location / Org / Date… |
| Sentiment analysis | positive / negative / neutral / mixed |
| Summarization | a shorter version |
Also: language detection · PII · entity linking · translation · CLU (intent) · question answering.
NLP traps
"positive or negative?" → sentiment
"pull the main topics" → keyword extraction
"remove emails/phones" → PII detection
"what does the user want to do?" → CLU (intent)
CLU (intent/action) ≠ Question answering (FAQ from a knowledge base). LUIS→CLU, QnA Maker→question answering.
Implement text analysis in Foundry
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role":"system","content":
"Return JSON: sentiment, key_phrases[], entities[], summary."},
{"role":"user","content": review}
],
temperature=0) # deterministic for analysis
Model + prompt
flexible, multilingual, less deterministic
Azure Language tool
consistent, structured, prebuilt
Check yourself
Best temperature for a data-extraction task?
≈0 = most deterministic/consistent output — what you want for extraction.
Speech — two capabilities
Speech recognition
audio → text (STT / transcription)
Speech synthesis
text → audio (TTS, neural voices, SSML)
Also: speech translation · speaker recognition.
Speech in AI-901 implementation
- Azure Speech in Foundry Tools — build STT/TTS apps
- Speech-capable agent — attach speech so users can talk to it
- Multimodal audio model — hear + reply with one model (simplest voice loop)
"transcribe a recording" → STT. "read the answer aloud" → TTS. "translate spoken Spanish to English audio" → speech translation.
Check yourself
A voice bot must hear a question and reply with the fewest moving parts. Best fit?
Multimodal audio model = one model hears & responds, no separate STT+LLM+TTS plumbing.
Information extraction basics
OCR
detect & read text inside images/scans — the first step
Field extraction & mapping
map values → named fields (this number → total)
That mapping is exactly a Content Understanding field schema.
Azure Content Understanding
flowchart LR
F["File: doc / image / audio / video"] --> AN[Analyzer]
SC["Field schema
(vendor, date, total…)"] --> AN
AN --> JS["Structured JSON
field = value (+confidence)"]
JS --> APP[Lightweight extraction app]
One multimodal service across documents, images, audio, video.
Content Understanding vs multimodal prompt
Content Understanding
consistent structured fields · many files · batch pipeline
Multimodal prompt
flexible one-off answer about a single item
"Extract vendor/date/total from 50,000 invoices" → Content Understanding. "Answer one question about this one photo" → multimodal prompt.
Check yourself
Extract structured fields from documents, images AND video with one approach?
Content Understanding is the single multimodal extraction service (docs/images/audio/video) via analyzers + field schemas.
Day 4 labs & homework
Text lab: prompt a model to return sentiment+entities+summary as JSON.
Speech lab: STT a clip, TTS a sentence, tweak a neural voice.
Lab 4: Content Understanding — build an invoice analyzer, run it, read the JSON.
Then: Practice Test 1, Q21–40.
Recap
- Tokenization → statistical → semantic
- Keyword · entity · sentiment · summarization (temp≈0)
- STT=recognition, TTS=synthesis; multimodal audio = one-model voice
- OCR + field mapping → Content Understanding (analyzer + field schema)
Next → Day 5: Generative AI, Agents & the Foundry SDK