#4821 Add streaming inference cache to model server
vertex-ml · feat/inference-cache · Vertex Labs · opened Sep 8, 2026 by David Osei
Possible follow-up correction — #4835
A hotfix changed nearby eviction code and referenced #4821 two days later. This is a reason to look closer—not automatically counted as rework cost.
Inferred relationship · Medium confidence
AI cost
$214
Labor cost*
$940
Delivery cost
$1,154
AI sessions
7
Cycle time
3d 4h
Review rounds
3
Checks
18/18
Changed files
23
Delivery Trace
The path from first AI session to merge, reconstructed from provider and Git metadata.
AI session started
David opened Claude Code against feat/inference-cache.
Context added — repository discovery
Model read 18 files to understand the model server layout.
Implementation and test cycle
Three implementation attempts; tests began after the second attempt.
Commit and pull request opened
Commit 7d2a41 linked directly to pull request #4821.
Review feedback and revision
Two reviewers requested concurrency protection and new load tests.
Approved and merged
All 18 checks passed and the pull request was merged.
Possible follow-up correction
Hotfix #4835 changed nearby eviction code and referenced #4821.
Efficiency analysis
Observations supported by available metadata.
18 files were loaded after similar discovery in a prior session.
Est. $21 avoidable context cost
Tests began after the second implementation attempt.
Medium confidence
Routine test repair used a frontier model for 38 minutes.
Est. $17 opportunity
Three focused rounds; all checks passed before merge.
Direct Git evidence
Evidence coverage
How relationships in this trace were established.
Prompts and source-code contents are not collected. Relationships are labeled direct, inferred, or manually assigned.
AI association is contextual evidence, not proof of causation. Total delivery cost includes only explicit labor inputs.