New Three-Layer Framework Aims to Evaluate Community-Designed Farm Innovations

When development projects try to transform agriculture, they rarely succeed through a single clever technology. Instead, the most ambitious interventions now arrive as bundles: packages of mutually reinforcing technological, social, organisational and policy innovations, co-designed with the communities expected to live with them. These socio-technical innovation bundles, or STIBs, are supposed to attack multiple barriers at once, from climate shocks to gendered gaps in information access. But their combined, synergistic effects are notoriously hard to measure, and a new paper in npj Sustainable Agriculture argues that the evaluation toolbox most funders rely on is simply not built for the job.

The study, led by Rupak Goswami of Ramakrishna Mission Vivekananda Educational and Research Institute in Kolkata, together with colleagues at the International Rice Research Institute, does not offer a shiny new metric. Its central claim is architectural. The authors propose a three-layer framework that wires together three well-established methods—fuzzy cognitive mapping, small-wins tracking and social network analysis—into a single evaluative system in which each layer’s outputs systematically inform the others through periodic stakeholder learning cycles. The integration, they argue, is what is novel, not the parts.

The problem the framework addresses is rooted in the nature of participatory change itself. Interventions premised on co-production do not follow linear cause-and-effect chains. Outcomes emerge through interactions among diverse actors, evolving relationships, local interpretations and iterative learning. Under those conditions, no single form of evidence can capture every dimension of change. How stakeholders understand causal pathways, how they experience change in their daily lives, and the social relationships through which innovation actually spreads all reveal different facets of the same process. Conventional monitoring, evaluation and learning approaches, the authors contend, rarely move beyond token stakeholder participation, which makes the transformative potential of participatory work difficult to define, measure and communicate to the donors and communities who underwrite it.

The framework’s first, cognitive layer connects a stakeholder-owned Theory of Change with participatory fuzzy cognitive mapping, or FCM. A Theory of Change here is not a static diagram but a deliberative, evolving explanation of how and why change is expected to occur, complete with its assumptions and competing interpretations. FCM formalises selected causal propositions from that explanation: nodes represent components of the Theory of Change, while weighted edges capture stakeholders’ elicited judgements about the direction and strength of causal influence. Crucially, the authors treat those edge weights as stakeholder judgements that can generate propositions for further examination, not as settled causal mechanisms. The maps can be simulated under changing assumptions and updated as feedback evolves, allowing stakeholders to add, remove or re-link concepts as the project unfolds.

The second, reflective layer draws on Small Wins Theory, the idea, famously articulated by organisational psychologist Karl Weick, that large social changes often arise from incremental progress rather than single breakthroughs. In this framework, small wins are tracked alongside formal monitoring indicators, pairing subjective reflection with measurable benchmarks. But the authors are careful to avoid a seductive trap: small wins are not transformations in miniature, and their accumulation is not assumed to produce transformation automatically. A small win becomes relevant to a transformative trajectory only when it alters the conditions for subsequent action—for example by strengthening collective capacities, changing decision rights, generating institutional commitments or redistributing access to resources. Evaluation therefore asks whether small wins remain isolated or become reinforced, extended and institutionalised over time.

The third, relational layer deploys social network analysis to trace how connections among actors evolve. SNA identifies key actors and their positions, exposes structural gaps, and enables targeted capacity-building and brokering. It also reveals how information and innovations spread through evolving networks, how influential actors emerge, and how social learning fosters behavioural change that can sustain outcomes after a project ends. Together, the three layers connect how stakeholders think about change, how they assess and measure it, and how they connect and learn, through iterative feedback cycles in which evaluation evolves alongside the intervention rather than judging it from outside.

What distinguishes the proposal philosophically is its embrace of what the authors call methodological bricolage. Causal maps, narratives, indicators and network data are never converted into a common metric. They remain distinct evidentiary forms whose convergences, complementarities and contradictions are examined through periodic reflection. The framework draws on realist evaluation’s interest in context-dependent causal explanation, developmental evaluation’s use of evidence within ongoing adaptation, Outcome Mapping’s attention to behavioural change, and the Most Significant Change technique’s demonstration that stakeholder narratives can reveal valued, unanticipated changes that predetermined indicators miss. Quantification itself is treated as situated and value-laden: decisions about which concepts enter a map, whose causal judgements are elicited and how network boundaries are drawn all shape what becomes visible, so those decisions must be documented.

The framework’s handling of disagreement is perhaps its most distinctive technical feature. Where stakeholder groups produce maps that share a broadly similar causal structure but differ mainly in edge weights, a documented aggregate may be constructed for exploratory simulation. But where groups disagree about the existence, direction or meaning of a causal relationship, their maps are retained as alternative causal accounts rather than averaged away. The authors report exactly such a case from the IRRI CGIAR Gender Equality West Bengal Learning Labs, where the framework was operationalised over its first two years. Women-only and mixed-group elicitations diverged on a pathway linking information to agency. Rather than reconciling the maps statistically, stakeholders surfaced the divergence in a reflection meeting as diagnostic of a gender-differentiated access barrier invisible in formal indicators, then endorsed a design response: digital literacy training and localised climate advisory services. The contested pathway was carried into the next simulation as a scenario to be tested, not a settled parameter.

That field application produced other adaptations not envisaged in the original design. Reflexive discussions revealed gaps in women’s access to climate and market information; cognitive modelling highlighted the potential of fisheries-based diversification; and network analysis identified key women farmers occupying brokerage positions. Together these insights informed iterative revisions to both the innovation bundle and its Theory of Change. The authors also stress what the framework cannot do. A changed cognitive map demonstrates a change in understanding, not necessarily a change in institutions. Network reorganisation may indicate shifting relationships but requires additional evidence to establish whether authority and benefits have actually been redistributed. Claims about adaptation, transformative potential or genuine transformation depend on cross-layer evidence concerning structural depth, breadth, institutionalisation and durability.

The authors are candid about limitations. Fuzzy cognitive mapping elicitation, repeated network panels and platform integration demand facilitator and analyst skills that are unevenly distributed, particularly among national agricultural research organisations in the Global South, and stakeholder fatigue and uneven digital infrastructure further constrain uptake. A staged or modular implementation may be more realistic than full deployment in resource-constrained settings, and for short or linear projects the marginal value may not justify the investment. The framework supplements rather than replaces donor results frameworks, and it remains conceptual, informed by ongoing Learning Lab experience but awaiting broader empirical testing. Still, its ambition is clear: to reframe evaluation as an adaptive system that evolves with the intervention, holding disagreement visible and treating transformation as something to be detected rather than presumed.

Subject of Research: An integrated three-layer evaluation framework for co-produced socio-technical innovation bundles in participatory agricultural research

Article Title: An integrated evaluation framework for co-produced socio-technical innovation bundles

Article References: Goswami, R., Mukhopadhyay, P., Chadha, D., & Puskur, R. (2026). An integrated evaluation framework for co-produced socio-technical innovation bundles. npj Sustainable Agriculture, 4(1), Article 84. https://doi.org/10.1038/s44264-026-00195-0

Image Credits: AI Generated

DOI: 10.1038/s44264-026-00195-0

Keywords: participatory action research, socio-technical innovation bundles, evaluation framework, fuzzy cognitive mapping, social network analysis, Theory of Change, small wins, agri-food systems, monitoring evaluation and learning, complexity-informed evaluation, adaptive management, gender equity