Build AI product sense
one run at a time

Interactive lessons where you run real models, step inside them in 3D, and push them until they break. Then take on the challenges and climb the leaderboards.

Step inside the model

Immersive, full-screen journeys through what happens inside an AI model

Challenges

Beat the goal on real model runs, or call what a model did, then share your best

All challenges

Foundations

How models behave: tokens, prompts, cost, and when they make things up

Start here3DBeginner
Follow a prompt through tokenization, the context window, and sampling, and see why temperature and limits matter.
TokensContext WindowTemperatureMax Tokens
Step inside
Beginner
Build a prompt from role, categories, format, examples, and rules, score it on 10 labeled items, and see what moves the score.
Prompt DesignFew-Shot ExamplesOutput FormatAblation
Beginner
Ask 12 questions, some nobody can answer, and count how often the model makes something up. Then see which fixes work.
HallucinationGroundingAbstainingTemperature
3DBeginner
Search 25 support tickets by keyword and by meaning, and watch your query land on a map where similar tickets sit together.
EmbeddingsSemantic SearchKeyword SearchSimilarity
Step inside
Beginner
Run 12 support jobs with a better prompt, with retrieval, and with a fine-tuning stand-in. Each fixes a different kind of problem.
PromptingRAGFine-tuningTrade-offs
Intermediate
Chat about a long contract and watch where every token and dollar goes, and how caching makes repeat requests cheaper.
TokensPricingPrompt CachingUnit Economics

Building

Putting models to work in a product: tools, retrieval, agents

Start hereBeginner
Turn 12 messy sales emails into orders. Ask for JSON four ways and see which replies your code can read, and which are right.
Structured OutputJSON SchemaParsingReliability
3DIntermediate
Watch the agent loop: the model decides to call tools, reads their results, and keeps going until it can answer.
Tool UseAgent LoopGuardrailsMemory
Step inside
3DIntermediate
Ask questions about a fictional product the model has never seen. Turn retrieval on and off to see grounding vs. guessing.
EmbeddingsVector SearchTop-KReranking
Step inside
Intermediate
Give a model 12 requests and a set of tools. Blur the names or add look-alikes from other MCP servers, and watch what it picks.
Tool UseMCPTool DescriptionsToken Cost
Advanced
A lead agent splits a product brief across specialists with their own data. Compare it to one agent on quality, time, and cost.
OrchestrationSpecialistsHandoffsTradeoffs
Advanced
Run 10 support jobs through fixed steps, an agent, and both. See which handles surprises, which stays reliable, and what each costs.
WorkflowsAgentsReliabilityCost

Shipping

Making it measurable, safe and affordable

Start here3DIntermediate
Run a prompt against a labeled test set and let graders score it. Change the prompt or model and see if the score goes up.
Test SetsLLM-as-JudgePass RateRegressions
Step inside
Intermediate
Run 12 questions at four thinking levels. See where a little thinking fixes multi-step problems, and where more just costs.
ReasoningThinking EffortLatencyCost
Intermediate
A ticket router aces its test set, then meets a real week of messages. Find what the tests missed, add it, and fix it.
MonitoringLive TrafficRegressionsTest Sets
3DAdvanced
An email assistant reads a poisoned newsletter and leaks internal numbers. Turn on defenses one at a time and see which ones stop it.
Prompt InjectionGuardrailsUntrusted DataLeast Privilege
Step inside
Advanced
Run 12 support questions through a small model, a large one, a router, and a cascade, and find the sweet spot on accuracy vs. cost.
Model ChoiceRoutingCascadesUnit Economics
Advanced
A support agent takes real actions on 12 requests. Choose which wait for a person, and trade bad actions against review time.
ApprovalsAutonomyRisk PolicyHuman Review
MCP

Bring AI PM Lab into your assistant

Connect it to Claude, ChatGPT, Cursor or VS Code, and your assistant can review AI specs, price features and design eval suites with what the lessons teach.

Ask things like
  • “Review this AI feature spec and tell me what's missing.”
  • “What would 50,000 requests a day cost on gpt-4.1-mini?”
  • “Design an eval suite for our support bot.”