Development programmes often invest considerable time and resources in field data collection. Surveys are programmed, enumerators are recruited, stakeholders are interviewed and datasets begin to accumulate.

But by the time the data reach the analysis stage, a more fundamental problem can emerge:

the information collected does not fully answer the questions the programme originally needed to address.

Important variables may be missing. Definitions may differ between locations. Sampling may not reflect the populations that matter. Survey questions may generate responses that are difficult to analyse. Quantitative and qualitative components may have been designed separately. Or data may have been collected without a sufficiently clear understanding of how the resulting evidence will inform programme, policy or investment decisions.

These are not primarily data-analysis problems.

They are research design problems.

For complex development programmes, good data starts long before fieldwork. It begins by establishing a clear methodological pathway between the decisions the programme needs to make and the evidence required to support them.

A useful way to think about that pathway is:

Programme objective → Research question → Population → Variables → Data collection → Analysis → Decision

When those components are designed together, field data become much more useful. When they are treated as separate tasks, even large and expensive datasets can have limited decision-making value.

1. Start With the Decision, Not the Questionnaire

One of the easiest mistakes in research planning is to begin by asking:

What questions should we include in the survey?

That question comes too early.

A better starting point is:

What does the programme need to understand or decide?

Consider a programme seeking to improve livelihoods within a rural value chain.

A broad objective might be:

Improve market participation and economic opportunities for producers and small enterprises.

That is a programme objective, but it is not yet a research question.

Before designing a survey, the research team needs to establish what evidence would actually support decisions around that objective.

For example:

• Which constraints limit market participation?
• Do those constraints differ between locations or stakeholder groups?
• Where are the greatest losses occurring within the value chain?
• Which infrastructure or service gaps appear most important?
• Are women and young people experiencing different barriers?
• Which constraints are potentially addressable through programme investment?

Those questions begin to determine what information needs to be collected.

Only then should the questionnaire follow.

Research design therefore works backwards from the decision to the evidence, rather than forwards from a list of survey questions.

2. Translate Broad Objectives Into Answerable Research Questions

Development programmes frequently begin with broad objectives involving concepts such as resilience, productivity, participation, access, inclusion, livelihoods or market development.

These concepts need to be operationalised before they can be measured.

For example:

Broad objective:
Understand barriers to market access.

That could become a set of more specific research questions:

• How far do producers travel to reach their primary market?
• What transport options are available?
• What proportion of production reaches formal markets?
• Which costs are associated with transportation?
• Do market-access constraints differ geographically?
• Are particular groups disproportionately affected?

Each question then has implications for:

• variables,
• indicators,
• respondent groups,
• measurement approaches,
• sampling,
• and analysis.

This process matters because a research question should not exist independently from the data required to answer it.

If the question cannot be linked to measurable information, it usually needs further refinement.

3. Decide Who and What You Need to Study

Once the research questions are clear, the next issue is the unit of analysis.

This is particularly important in complex programmes because there may be several units simultaneously.

A value-chain study, for example, might need information from:

• producers,
• processors,
• traders,
• market operators,
• service providers,
• small and medium-sized enterprises,
• government agencies,
• community organisations,
• women,
• young people,
• and other stakeholder groups.

The methodology then needs to determine which groups answer which research questions.

The same principle applies geographically.

Should data be collected from:

• all programme locations,
• selected representative sites,
• high- and low-performing areas,
• urban and rural locations,
• intervention and comparison areas,
• or locations selected according to specific characteristics?

These decisions should be made deliberately rather than being determined mainly by convenience.

4. Sampling Should Be Designed Before Fieldwork

Sampling is not simply deciding how many people to survey.

It requires consideration of who needs to be represented in the evidence and why.

Depending on the project, methodological questions may include:

• What is the target population?
• Is there a reliable sampling frame?
• Does the study require probability sampling?
• Should the sample be stratified?
• Are there important geographic groups that need representation?
• Are some stakeholder groups relatively small but analytically important?
• Should women or young people be intentionally represented?
• Are respondents clustered within communities, organisations or facilities?
• Does seasonality affect who is available or what can be observed?

Sample size also needs to be considered in relation to what the analysis is expected to do.

A sample that appears large overall may become much smaller once researchers attempt subgroup comparisons, multivariable modelling or analysis across locations.

That is why sampling and analytical planning should be developed together.

5. Design the Questionnaire Around the Analysis

A questionnaire should not simply contain questions that seem interesting.

Each question should have a reason for being there.

Before including a variable, it is useful to ask:

What research question does this variable help answer?

How will it be represented in the dataset?

How will it ultimately be analysed or interpreted?

For example:

Thinking this way creates a direct link between research question → variable → analysis.

It also helps control questionnaire length.

If a question does not contribute meaningfully to the research objectives, it may not need to be collected.

That matters in field research, where overly long instruments increase respondent burden, enumerator fatigue and the potential for poor-quality responses.

6. Define Variables Before the Data Exist

Variable definition is one of the least visible but most important parts of research design.

Terms that appear straightforward in a proposal can become surprisingly ambiguous during fieldwork.

For example:

What exactly counts as:

• an active enterprise?
• a market participant?
• household income?
• production volume?
• post-harvest loss?
• access to finance?
• youth participation?
• infrastructure availability?

If different teams interpret these concepts differently, the resulting dataset may contain inconsistencies that are difficult to repair later.

A strong methodology therefore defines key variables and indicators before collection begins.

Where appropriate, that may include:

• operational definitions,
• units of measurement,
• response categories,
• reference periods,
• coding conventions,
• permitted ranges,
• missing-data categories,
• and calculation rules.

This is especially important when several teams or locations are involved.

7. Mixed Methods Should Be Designed as One Research Strategy

Many international development studies combine quantitative and qualitative methods.

But simply having both a survey and interviews does not automatically create a strong mixed-method design.

The two components should answer complementary parts of the research question.

A useful division is often:

Quantitative evidence tells us what is happening.

Qualitative evidence helps explain why it may be happening.

For example, survey data might show that enterprises in one region experience substantially greater transport costs.

Qualitative interviews may then help explain whether those differences reflect road conditions, security, fuel availability, market structure, seasonal access or other factors.

Stakeholder consultation can provide another layer by testing whether the interpretation of the findings is credible within the local context.

The methods therefore strengthen one another when they are designed around the same analytical framework.

 

8. Build Field Protocols Before Data Collection Starts

Even a well-designed survey can produce inconsistent data if field procedures are unclear.

Research protocols should therefore specify what field teams should do in common situations.

This can include:

• respondent eligibility,
• respondent selection,
• replacement procedures,
• repeat visits,
• incomplete interviews,
• missing responses,
• consent,
• interviewer instructions,
• unusual responses,
• measurement procedures,
• field supervision,
• and escalation of data-quality concerns.

Standardisation becomes increasingly important when research is conducted across multiple sites or by several teams.

Without clear protocols, differences in how teams interpret the study can become differences in the data.

By the time those inconsistencies appear during analysis, the opportunity to correct them may have passed.

 

9. Electronic Data Collection Is More Than Digitising a Questionnaire

Platforms such as KoBoToolbox can make field data collection substantially more efficient, particularly for multi-site studies.

But digital implementation should be treated as part of the methodology rather than as a purely technical step.

A strong electronic instrument can incorporate:

• skip logic,
• validation constraints,
• mandatory fields,
• range checks,
• consistent response coding,
• structured variable names,
• conditional questions,
• metadata,
• enumerator instructions,
• and quality-control checks.

The objective is not simply to reproduce a paper questionnaire on a tablet.

The electronic instrument should operationalise the research design.

That means the digital structure should reflect how the variables are intended to work in the eventual dataset and analysis.

Testing before deployment is therefore essential.

A technically functional form is not necessarily a methodologically sound one.

10. Plan the Analysis Before the Data Arrive

The analytical strategy should not begin when someone opens the final spreadsheet.

Planning analysis in advance helps determine whether the proposed data collection is sufficient.

For each major research question, researchers should ideally understand:

• the primary outcome or indicator,
• the main explanatory variables,
• potential confounders or covariates,
• subgroup comparisons,
• the structure of the data,
• and the type of analysis likely to be required.

For example:

Research question: Does access to infrastructure differ between enterprise types?

Outcome: Defined infrastructure-access indicator

Explanatory variable:  Enterprise type

Potential additional variables: Location, enterprise size, gender of ownership or other relevant characteristics

Analysis: Descriptive comparison and, where appropriate, adjusted analysis

Working through that logic before fieldwork can expose methodological gaps.

Perhaps a key variable has not been included.

Perhaps the proposed response categories are not suitable for the intended analysis.

Perhaps the planned sample will not support the comparisons being proposed.

It is far better to discover those issues during research design than after the data have been collected.

 

11. Design for Comparability Across Locations and Time

Many development programmes need to compare evidence across:

• districts,
• countries,
• communities,
• stakeholder groups,
• seasons,
• or time points.

Comparability cannot simply be assumed.

It needs to be built into the methodology.

That may require consistent:

• definitions,
• questions,
• response categories,
• units,
• field procedures,
• coding,
• variable names,
• and analytical rules.

At the same time, completely rigid standardisation may not always be appropriate.

Different locations may have genuinely different contexts.

A practical solution can be to define a core set of standard variables while allowing carefully controlled context-specific additions.

The objective is to preserve enough consistency for comparison without stripping the research of local relevance.

 

12. Account for Seasonality When It Matters

A single round of fieldwork may provide an incomplete picture of systems that change substantially during the year.

Agriculture, fisheries, livelihoods, food security, transport and market activity can all vary seasonally.

If research is conducted only during one period, findings may reflect that period rather than the system more broadly.

Research design should therefore ask:

• Does the phenomenon vary seasonally?
• Could respondent behaviour differ during different periods?
• Are prices or production volumes seasonal?
• Does infrastructure accessibility change?
• Would repeated measurement be more informative?
• Does the programme eventually need to measure change over time?

These questions affect both fieldwork scheduling and interpretation.

 

13. Design Data Quality Into the Study

Data quality should not be something that is checked only after collection finishes.

It should be built into the research design.

Preventative approaches might include:

• clear definitions,
• tested survey instruments,
• automated validation,
• sensible allowable ranges,
• enumerator guidance,
• monitoring completeness,
• identifying unusual response patterns,
• field supervision,
• and regular review during collection.

There is an important distinction here:

data cleaning attempts to identify problems after they occur.

good research design attempts to prevent unnecessary problems from occurring in the first place.

Both matter, but prevention is usually more efficient.

14. Keep the Evidence Connected to the Decision

Complex research projects can produce very large amounts of information.

More data do not automatically produce better decisions.

The methodological framework should maintain a clear line between:

what was collected → what was analysed → what it means → what decision it informs

That is particularly important in programmes where research findings are expected to contribute to:

• investment prioritisation,
• programme design,
• resource allocation,
• policy development,
• intervention targeting,
• monitoring,
• or evaluation.

If the connection between the research objective and the final recommendation is unclear, the evidence becomes harder to use.

 

What Happens When Research Design Comes Too Late?

Many research problems cannot be fully repaired during analysis.

Examples include:

A key variable was never collected.
No statistical method can recreate information that does not exist.

Different field teams used different definitions.
Comparisons may become unreliable.

The sample excludes an important stakeholder group.
Their experience cannot be adequately represented afterwards.

The questionnaire used response categories that do not match the research question.
Analysis becomes constrained by the structure of the collected data.

The survey is too long.
Response quality may deteriorate and missingness can increase.

Quantitative and qualitative work were designed independently.
The findings may be difficult to integrate meaningfully.

No analysis plan existed before collection.
Important covariates, outcomes or subgroup information may be absent.

This is why methodological input is often most valuable before the first observation is collected.

 

Research Design in Practice: A Multi-Site Development Study

Wells Research Consultancy recently supported the methodological development of a complex multi-site fisheries value-chain assessment for an international development programme.

The project involved multiple units of analysis and respondent populations, including actors operating across markets, enterprises and different stages of the value chain. It also needed to combine quantitative and qualitative evidence, incorporate gender and youth considerations, account for seasonal variation and generate evidence capable of supporting investment decisions.

The methodological work included translating broad programme objectives into a structured research framework; developing survey and assessment approaches; defining variables and indicators; developing sampling protocols; integrating quantitative and qualitative data requirements; implementing electronic survey instruments using KoBoToolbox; considering seasonal data collection; incorporating gender and youth disaggregation; and developing quality-control and analytical requirements.

The resulting framework created a clearer pathway from research objectives → variables → data collection → analysis → decision-making. It was designed to improve consistency across field sites and respondent groups, reduce the risk of unusable data and support value-chain analysis, identification of bottlenecks and future investment prioritisation.

Read the Research Design & Methodology case study →

Good Research Design Is Ultimately About Usable Evidence

Strong methodology does not make research unnecessarily complicated.

Its purpose is the opposite.

It creates clarity about:

• what needs to be known,
• what evidence is required,
• where that evidence should come from,
• how it should be collected,
• how it will be analysed,
• and what conclusions it can reasonably support.

For complex development programmes, this becomes especially important because research often involves multiple locations, stakeholder populations, evidence sources and organisational priorities.

The earlier those components are aligned, the greater the likelihood that fieldwork produces evidence that is not only technically credible, but genuinely useful.

Planning Data Collection for a Complex Programme?

If your project involves multiple research questions, stakeholder groups, locations, surveys or methodological decisions, Wells Research Consultancy can help establish the research structure before data collection begins.

Support can include research question development, study design, sampling strategy, survey methodology, KoBoToolbox implementation, outcome and variable definition, mixed-method design, analytical planning and methodological review.

Explore Research Design & Methodology →