Core Idea
Definition
Replication checks whether a result can be observed again, while triangulation checks whether different methods or sources converge on the same underlying conclusion.
In Plain English
Do not trust a finding only because it appeared once. Ask whether it holds up when tested again or approached from another angle.
Framework Structure
Components
Flow
Get initial result -> Repeat the test -> Check with independent methods or evidence sources -> Judge whether confidence should rise or fall
How to Apply
- 1.Treat an initial finding as promising rather than final
- 2.Repeat the test where possible under similar conditions
- 3.Look for independent methods, samples, or perspectives that bear on the same claim
- 4.Compare whether results converge, weaken, or conflict
- 5.Increase confidence most when the claim survives multiple credible routes of challenge
When to Use
- •Research and experimentation
- •Product analysis and operational learning
- •High-stakes decisions where one result should not dominate
- •Evaluating surprising or high-impact claims
- •Any context where robustness matters more than novelty
When NOT to Use
- •When the underlying phenomenon changes too rapidly for meaningful replication
- •When perfect replication is impossible and used as an excuse to ignore useful evidence entirely
- •When triangulation becomes sloppy accumulation of low-quality sources
- •When a quick decision is needed and the cost of delay dominates
Example
Problem
A team sees a promising lift in one experiment and must decide whether the effect is real enough to roll out widely.
Application
- 1.Repeat the experiment or rerun it on another cohort
- 2.Check whether related metrics and qualitative signals move in the same direction
- 3.Compare the results across methods and segments
- 4.Raise confidence only if the effect survives multiple credible checks
Conclusion
The team avoids overreacting to a single pleasing datapoint and instead looks for durability.
Takeaway
Replication and triangulation turn isolated results into more trustworthy knowledge.
Common Mistakes
- •Treating one positive result as enough
- •Confusing repeated noise with real robustness because the same flawed method was reused
- •Counting many weak sources as equivalent to one strong independent line of evidence
- •Ignoring conflicting replications because the original result was appealing
- •Failing to ask whether methods share the same hidden bias
How to Practice
one more check
Before acting on a strong finding, ask what one additional independent check would most increase or decrease confidence.
method diversity
Try to validate important claims through different kinds of evidence rather than many copies of the same method.
conflict embrace
When one line of evidence disagrees with another, treat that friction as a cue for deeper learning rather than something to hide.
Related Cognitive Biases
novelty bias
People often overvalue striking first results before robustness is established.
confirmation bias
Teams may stop testing once they get the answer they hoped for.
survivorship bias
Visible positive results can crowd out the unseen failures to replicate.
Related Frameworks
Related Skills
Variants & Extensions
Typical Failure Modes
- •Single-result overtrust
- •Method redundancy disguised as triangulation
- •Ignored conflicting evidence
Further Reading
- The Art of Statistics by David Spiegelhalter
- Calling Bullshit by Carl T. Bergstrom and Jevin D. West
- Theory and Reality by Peter Godfrey-Smith