Source evidence versus model inference

Notice the extra step in an explanation.

“The cache reduced repeat lookups” and “a longer cache lifetime will make this service faster” can appear together in a confident engineering summary. The second sentence adds a prediction. Even if the first came from a real design note, it does not establish the workload, freshness requirements or production effect of the proposed change.

That extra step is not inherently bad. Engineering depends on inference. The problem begins when a hypothesis loses its label and the next task receives it as an observed fact. A useful context record keeps both the support and the step beyond it visible.

The classification below is a proposed reasoning aid, not a scoring system or a claim that source-backed sentences are automatically true. A source may itself contain an assumption, an obsolete account, or someone else’s unverified report.

Separate the statement from its evidence type.

Ask what was actually inspected. Reading a note establishes that the note makes a statement. Reading code establishes something about the inspected code path. An executed check with its output supports the scenario it exercised. None is a shortcut to every stronger claim about current production behavior.

Keep the original speaker and the subject separate. “The incident note reports recovery” differs from “we observed recovery.” Copying a report into a second summary does not create a second independent observation.

Work through a fictional cache investigation.

This entire example is fictional. An old design note says a search service added caching to reduce repeated lookups. The inspected configuration enables caching. A unit-test run reuses an entry for one fixture. A trace from one sampled request records a cache miss. No representative production hit rate or latency comparison is available.

Those accounts can all be true. An enabled setting does not establish that every deployed process loaded it. A test of reuse does not show how often real requests repeat. A miss for one request does not show that the cache is universally broken.

An assistant suggests that the lookup and write paths construct different keys. That is a hypothesis worth checking, not a finding established by the miss. Other explanations include an empty cache, expiry, invalidation, or a request routed through a different path. Keep alternatives plausible and bounded to the system being inspected.

Give the inference a test that could change your mind.

Google’s SRE troubleshooting guidance describes moving from observations to hypotheses and testing them against evidence or controlled changes. It also warns against treating correlation as causation. The practical question is what observation would distinguish the proposed explanation from the alternatives.

For the fictional key-mismatch hypothesis, trace the lookup and write paths for the same permitted input and compare the resulting keys. If the keys agree, this check weakens that explanation for that input. If they differ, determine whether the difference is intended before proposing normalization. A correction must preserve distinctions the service relies on; merging keys indiscriminately can change behavior.

A replay with controlled inputs can then test the proposed correction’s expected reuse and result freshness. Its outcome remains unknown until the replay runs. Even a passing replay would cover those inputs, not establish a production latency improvement or prove every access boundary.

Carry a small claim ledger into the next task.

A brief needs enough provenance to prevent a claim from getting stronger during handoff. Keep the consequential sentences, their evidence type, the inspected boundary and the next check. Do not attach a pile of sources without saying which statement each supports.

This fictional ledger entry shows how a useful hypothesis can stay actionable without becoming a fabricated result.

Question: why did the sampled search request miss the cache?
Reported rationale: old note says caching reduces repeated lookups.
Observed: inspected config enables caching; fixture run reused an entry; sampled trace missed.
Inference: lookup and write keys may differ.
Alternative explanations: empty entry, expiry, invalidation or different path.
Next check: compare lookup/write keys for the same permitted fixture and inspected revision.
Before an edit: confirm which input distinctions the key must preserve.
Proposed validation: controlled reuse and freshness cases; execute and retain actual results.
Unknown: prevalence of the miss and production latency effect.
Refresh trigger: new trace, code change, executed result or clarified contract.

Useful context carries the boundary as well as the idea.

If the next engineer receives only “key mismatch caused poor performance,” several unsupported steps have become premises. If they receive the observations, hypothesis and next check, they can investigate without rediscovering the entire discussion or trusting its conclusion blindly.

A context engine can help locate candidate evidence. It cannot make an inference an observation by retrieving it. When the relevant notes, code and test output are already easy to find, ordinary inspection and a short ledger can do this job.

A team lead can try the classification on three consequential sentences in one engineering brief with its maintainer. Check whether another engineer can locate the support, identify the unknown and propose a discriminating check. Record missing evidence and maintenance effort; stop or simplify if the labels add ceremony without changing a decision. Passing this exercise is an editorial or engineering review observation, not measured ROI.

Choose one sentence whose conclusion goes beyond its evidence. Keep the useful idea, label the inference, and write the observation needed before it can justify the next change.