The Wales Centre for Public Policy is a Knowledge Brokering Organisation (KBO) that provides Welsh Government ministers and public services with robust independent evidence to inform decision-making. It is committed to maximising impact by understanding how and why knowledge brokerage can enable decision makers to better access and apply evidence.  

There has been a proliferation of KBOs in recent years which has been accompanied by numerous studies on their role and varying activities, but we know relatively little about their impact. Our research seeks to understand the relationship between our Centre’s knowledge brokering practices and the use of evidence in policy and practice decisions.  

The Embedded Impact Researcher 

To do this, we have developed an Embedded Impact Researcher (EIR) model. This is a researcher who works across live projects to support a range of impact planning and capture activities. We drew on existing examples of ‘insider’ research conducted by KBOs alongside evaluations conducted by ‘embedded’ researchers, to design and refine several impact capture tools and approaches and develop a method for applying them systematically across our work. 

The EIR takes a realist-informed approach to evaluating the impact of our knowledge brokering practices. This focuses on articulating and identifying the underlying mechanisms that lead to impact and the contextual factors that enable this. In practical terms, this involves working alongside Centre colleagues and others (e.g. academics, Welsh Government, or public sector officials) and employing a series of ‘tools’ throughout projects ─ including structured observation, stakeholder analysis, impact mapping, reflective logs, realist-informed interviews and reflection sessions ─ to capture the practice choices that are made to achieve impact.  

How is the Embedded Impact Researcher working? 

The EIR has been in place for more than a year. We captured our experiences in a paper which highlights three main reflections. 

The need for better models of the knowledge brokering process 

First, it is important to understand the utility of existing models of the knowledge brokering process. We designed the EIR approach to operate within a ‘linear’, demand-led model of the knowledge brokering process, where policy makers, or other evidence users, commission and/or collaborate with the Centre to produce or synthesise evidence to address a specific policy evidence need. This model offers a simple portrayal of how knowledge brokerage works in theory which all actors can understand, but it does not adequately account for the messy way in which knowledge brokering occurs in practice. For instance, policy makers often do not have clearly articulated evidence needs, so research questions are often ‘co-determined’ between the knowledge broker and the evidence user involving multiple back-and-forth steps in the process. 

There is a need for better theories and frameworks that reflect the complex, iterative nature of knowledge brokering in public policy. Crucially, in terms of impact assessment, current models tend to focus on short-term effects, such as participant satisfaction, rather than broader real-world impacts on policy processes, evidence users, and outcomes. The EIR is contributing to better theories and frameworks of knowledge brokering that strike a balance between avoiding oversimplification that ignores real-world complexity, while also steering clear of overly detailed or rigid models that are difficult to apply across different contexts.  

Think ‘mechanistically’  

Our second point of reflection concerns the challenges of planning for and assessing the impact of knowledge brokering within projects. Impact is often hard-to-measure, and it is difficult to attribute long-term policy or societal outcomes directly to these activities. Instead, impact often appears through ‘intermediary outcomes, such as improved capacity to use evidence, the extent of networking between evidence producers and users, and shifts in understanding. 

Traditional impact-planning approaches proved difficult to apply, as they tend to focus on clear, service-driven outcomes. In response, a more flexible approach was developed, focusing on why specific practices ─ e.g., traditional evidence synthesis, alternative forms of evidence compilation and communication, facilitated interactions with evidence through roundtables, workshops and other events, etc. ─ are chosen, how they are implemented, and the context in which they operate. This allowed for more realistic impact reflection as projects evolve. 

A key insight from the EIR role is the value of viewing knowledge brokering practices as mechanisms that help generate impact. This perspective encourages a shift away from rigid outcome-based planning towards a deeper understanding of how change happens in practice which supports more meaningful evaluation and better-informed decision-making. 

‘Embeddedness’ as a method of evaluation  

The third reflection is that evaluating the impact of a KBO from an embedded position brings some challenges. Sitting between an “insider” and “embedded” role, the EIR must balance being close enough to understand and support projects, while remaining sufficiently independent to provide critical insight. We describe the EIR as a supportive evaluator. Rather than providing practical tips regarding how to best achieve impact, they provide capacity for brokers to articulate and reflect on their tacit knowledge and assumptions about how their practice will lead to impact. 

There are also some methodological challenges. Project teams often act as gatekeepers to participants, which can bias evaluation towards more positive perspectives. Efforts were made to broaden participation and reduce this effect, though some voices remained harder to capture. Encouraging open reflection on challenges and failures also took time, but this improved as trust developed. 

A final tension lies between having a dedicated researcher focused on evaluating impact and fostering a shared organisational responsibility for impact. To address this, tools and approaches were designed to be accessible and used collaboratively, helping embed impact thinking into everyday practices. 

Where next? 

The work of the EIR is ongoing, and the researcher is now working on new projects commissioned by the Welsh Government and Welsh public services. We have taken the learning from the first phase of research into these new projects and refined our approach. Ultimately, we intend to ‘mainstream’ the use of impact assessment tools within the wide range of knowledge brokering projects that the Centre delivers.  

The paper on which this blog is based is available to read open access here: 

Smith, F., Durrant, H., and Downe, J. (2026) Evaluating the impact of knowledge brokering practices: Learning from embedded impact research at the Wales Centre for Public Policy, Research Evaluation, 35 https://doi.org/10.1093/reseval/rvag022