Streaming, batching, and cost optimization at scale
View badge details
About This Course
Ship Claude at production volume. Server-sent event streaming for perceived latency, Message Batches API for 50% cost savings on non-interactive workloads, prompt caching at scale, rate-limit and retry patterns, and end-to-end cost optimization playbooks.By the end of this course you will be able to cut Orion Analytics's daily Claude spend by 40-60% without changing feature scope through routing, caching, batching, and streaming techniques you'll apply directly.
Course Curriculum
10 Lessons
Server-sent event streaming — what and why
SSE stream format, event types (message_start, content_block_delta, message_stop), and when streaming improves perceived latency.
Compare streaming vs blocking latency - Lab Exercises
Run the same prompt through blocking + streaming APIs. Measure first-token-time and total-time. See the perceived-latency win.
Message Batches API — 50% off for non-realtime jobs
Submit up to 10K requests in one batch, get 50% off input/output pricing, results ready within 24h.
Batch-classify 20 support tickets - Lab Exercises
Submit a Message Batch of 20 ticket classifications, poll to completion, print per-ticket labels — at 50% of sync cost.
Prompt caching at scale — TTL, breakpoints, tiers
Cache-aware system prompts, 1-hour vs 5-min TTL, breakpoint placement, and cost accounting (cache write 25% premium, cache read 90% discount).
Multi-turn conversation with cached system prompt - Lab Exercises
Three-turn conversation with a large cached system prompt. See cache_creation on turn 1, cache_read on turns 2 and 3.
Cost optimization playbook
Order of moves to cut cost: model routing → prompt caching → batching → prompt shrink → structured outputs. Case studies + a decision tree.
Optimize a hot path — before and after - Lab Exercises
Take a 25-call classification workload from Sonnet+no-cache to Haiku+cache. Measure the % savings.
Rate limits, retries, and backpressure
HTTP 429 handling, exponential backoff, request-per-minute vs token-per-minute limits, and shedding load gracefully.
Cut an Orion workload's cost by 80% capstone - Lab Exercises
Take a 100-ticket-per-day workload; apply routing + caching + retries. Cut cost by 80%+ while maintaining accuracy.
This course includes:
- 24/7 AI Instructor Support
- Live Lab Environments
- 5 Hands-on Lessons
- Completion Badge
Earn Your Badge
Complete all lessons to unlock the Streaming, Batching, and Cost Optimization at Scale achievement badge.
Streaming, Batching, and Cost Optimization at Scale
Awarded on completion of CLD-AI-111. The holder can ship Claude at production volume — SSE streaming, Message Batches API, prompt caching at scale, rate-limit handling, and end-to-end cost optimization playbooks.
Skills You'll Earn
Complete all lessons in this course to earn this badge