The distinction is crucial when studies seem alike but differ in methodology.

When is a meta-analysis appropriate?

There is not a simple checklist to decide if studies should be combined. This choice needs careful judgment.

Researchers must check if the studies tackle compatible versions of the same question. This involves looking at populations, interventions, comparators, outcomes, measurement tools, follow-up times, and study designs.

The statistical data must also support the planned analysis.

For instance, several studies may report “quality of life” but use different tools, time points, or definitions. Another set may look at the same intervention but use different doses, methods, or populations.

These differences do not automatically prevent meta-analysis. However, they must be understood before calculating an overall estimate.

Cochrane's framework suggests identifying which studies can be grouped and then assessing the available data for synthesis.

When should studies not be pooled?

A key decision in evidence synthesis is knowing when a pooled estimate could misrepresent the data.

Caution is needed when there are significant differences in the clinical questions, incompatible outcome definitions, major methodological differences, too few comparable studies, incomplete effect estimates, or statistical information. Averaging different effects could confuse rather than clarify.

Cochrane highlights situations where meta-analysis may not be suitable, such as limited evidence, incomplete reporting, varying effect measures, methodological or clinical diversity, and considerable statistical heterogeneity.

This is why simply having “several studies” is not enough to justify combining them.

A forest plot and pooled estimate can look convincing, even when the studies answer different questions.

Good evidence synthesis requires researchers to explain why studies were included and why particular ones were grouped in each analysis.

Heterogeneity does not just mean a high I²

Heterogeneity is often oversimplified to one statistic, typically I².

This view is too narrow.

You should consider various forms of heterogeneity:

• Clinical heterogeneity involves differences in patient traits, interventions, settings, or outcome definitions.
• Methodological heterogeneity covers variations in study design, execution, measurement, or bias risk.
• Statistical heterogeneity indicates differences in observed effect estimates beyond what chance alone can explain.

These dimensions are related but not interchangeable.

A high I² may show significant variation between studies, but the pooling decision shouldn't rely only on an arbitrary I² threshold. Similarly, a low heterogeneity statistic does not mean clinically different studies suddenly address the same question.

You must return to the studies for interpretation.

What happens when meta-analysis is not appropriate?

Not doing a meta-analysis does not mean giving up on synthesis.

The evidence still needs to be organized, compared, and interpreted systematically.

Depending on the review question and available data, this might involve narrative synthesis, effect tabulation, visual presentations, grouping studies by meaningful characteristics, or other suitable methods.

There is even a guideline Synthesis Without Meta-analysis for systematic reviews where conventional meta-analysis is not suitable. This guideline was created partly because poorly described “narrative synthesis” can confuse readers about how evidence led to conclusions.

So, the alternative to an unsuitable meta-analysis should not be an unstructured paragraph listing studies. It should still be a clear synthesis.

Sometimes only part of a review should be meta-analyzed

This point is often overlooked.

A systematic review does not require a strict choice between “meta-analysis” and “no meta-analysis.”

Different outcomes may need different synthesis methods.

One outcome may be consistently reported across comparable studies, making it suitable for quantitative pooling. Another outcome might involve incompatible definitions or different measurement scales, requiring a different synthesis approach.

The specific synthesis often serves as a better judgment unit than the review as a whole.

This is why planning the synthesis around the research question and study characteristics is vital.

An example: when selective pooling produces a stronger review

In a clinical systematic review and meta-analysis supported by Wells Research Consultancy, the evidence included randomized studies reporting various clinical outcomes.

This situation posed a real synthesis challenge. Studies varied in interventions, outcome measurement, follow-up, and statistical reporting, and only part of the evidence was comparable for quantitative synthesis.

Instead of forcing all outcomes into a meta-analysis, quantitative pooling was limited to areas where meaningful comparison was justified. Other findings were included in a narrative synthesis.

Risk of bias, certainty of evidence, and heterogeneity were also considered in the interpretation, rather than focusing solely on pooled estimates.

The result was a more transparent and methodologically sound synthesis that later supported publication in a Q1 peer-reviewed journal.

Read the Systematic Review & Meta-Analysis Case Study →

This highlights an important point: sometimes, the best choice in a meta-analysis is to not calculate another pooled estimate.

Where do PRISMA, risk of bias, and certainty of evidence fit?

These elements serve different roles and are not interchangeable.

PRISMA helps ensure transparent reporting. It explains why a systematic review was done, the methods used, and what was found. PRISMA 2020 offers guidance on study identification, selection, appraisal, and synthesis.

Risk of bias assessment looks at the methodological features of included studies. It identifies aspects that could skew their results.

Certainty of evidence approaches, like GRADE, evaluate how much confidence we can have in the evidence backing a conclusion.

Together, these processes shift the review focus from:

“Is the pooled result statistically significant?”

to the more useful question:

“How much confidence should we place in this finding, and what limitations influence its interpretation?”

Do you always need a meta-analysis consultant?

Not every systematic review requires outside statistical support.

For straightforward evidence with established methods and similar outcomes, an experienced review team might handle the entire process internally.

Consultant input is useful when the synthesis needs judgment. This includes cases where studies report outcomes differently, effect measures need derivation or transformation, heterogeneity is high, several analytical methods are possible, or when subgroup or sensitivity analyses are considered. It is also helpful if reviewers question the statistical methods or if the team is unsure about quantitative pooling.

Importantly, it is better to address these questions before starting the statistical analysis.

Decisions made after viewing results are harder to defend than a strategy set in advance and adjusted transparently based on the evidence.

A meta-analysis should answer the research question not just provide a pooled estimate

Meta-analysis is a powerful tool for evidence synthesis, but its value lies in how it is applied.

The goal should not be to maximize the number of studies in a forest plot.

Instead, it should focus on producing a synthesis that respects the evidence's structure and answers the research question meaningfully.

Sometimes this means combining studies statistically.

Other times, it means separating distinct clinical evidence into different analyses.

And sometimes, the best choice is not to pool an outcome at all.

This judgment is what distinguishes statistical aggregation from rigorous evidence synthesis.