Initech's support chats run long, and sending the whole history with every reply is getting expensive. Your call: what should the assistant keep from earlier in the conversation?
Key idea: A model remembers nothing between calls; memory is what you choose to send back. Every strategy short of the full history forgets something different, and the full history costs the most as a conversation grows.
Finish a run to see what a PM takes away from this lesson.
Facts remembered vs. tokens per question
Run a memory strategy to plot it here
Things to try
- Run Full history first. Every fact is right there, so this is the score to beat.
- Switch to Sliding window. Which questions can it still answer?
- Try Rolling summary. Look at which facts it rounds off or drops.
- Try Extracted facts, then compare its tokens per question with Full history on the chart.
- Try Compaction. It keeps the latest messages word for word. What does the summary of the older ones lose?