|
|
Build Reach into Your Logic
Model February
1998 | ||||||||||||||||||||||||||||||
|
Analysts have
frequently noted the importance of constructing logic models (a.k.a. logic
charts, causal models, logical frameworks, and most recently performance
frameworks - among other names) to explain the causal theory of a program
or initiative before attempting to monitor, measure, or assess
performance. While logic models have long been a fundamental part of
program evaluation, the use of a logic model has also recently been found
to be very useful in performance measurement initiatives at the project,
program (see for example Focusing on Results: A Guide to Performance
Measurement, Robert McDonald, Industry
Canada) and even government-wide level. (See for example, Joseph S. Wholey, "Clarifying Goals, Reporting Results,"
Progress and Future Directions in Evaluation: Perspectives on Theory,
Practice, and Methods, Jossey-Bass
Publishers, San Francisco, Winter 1997, Number 76, p 100. Also see John
Mayne, mimeo, 1998. See 1997 Report of the Auditor General,
Chapter 5, Exhibit 5.1 for a simplified logic model example.)
A key limitation to
the logic models of the 1980s, as well as many of those in current use,
has been their tendency to focus predominantly on causal chains without
reference to who and where the action was taking place. This has caused
three key problems: 1. Lack of
sensitivity to the impacts on different participant groups. Logic
models which do not include participants or 'reach' tend to
narrowly define the impacts chain. For example, in a community economic
development program we recently examined, their preliminary (traditional)
logic model did not explicitly include reach and therefore only noted
results for small business in the causal chain. Once the small working
group included a reach category in their logic model, they came up with a
myriad of other key results relating to community capacity building,
collaboration, and benefits to specific stakeholder groups like youth.
2. Potential
to confuse outputs and outcomes. The inclusion of reach in logic
models allows people to clearly distinguish events which happen as part of
program processes - normally called outputs (e.g., # of
publications, events, interventions, and other tangible things under the
control of a program) from outcomes or impacts which relate to the
reaction, satisfaction, knowledge gain, behaviour changes, and benefits
occurring in target groups. Without the distinct reach of an initiative
being defined, we have often found confusion in terms of what people mean
by 'improved access' (e.g., do we mean available? or do we mean
usage by target groups?), 'service quality' (e.g., do we
mean conformity to a process standard? or do we mean the satisfaction of
user needs?), or similar performance concepts. 'Reach' helps
to sort outputs from outcomes. 3. No reach
versus results trade-off recognition. Without an explicit reach
consideration, analysts and managers (particularly senior managers) may
get a simplified notion of the ease with which results will occur.
Similarly, they will often develop a false notion of accountability - not
recognizing the multiple co-dependencies in a given policy, program, or
initiative. For example, in
most areas of social, economic, safety, and environmental policy, there is
a multitude of jurisdictions and institutional actors involved for any
given objective. Generally, the more the co-dependence, the greater the
time involved and the greater the 'causal complexity' of the
results chain. (For example, early results may simply involve the
improvement of collaboration among co-delivery partners for many programs;
this needs to be recognized in the causal chain.) Furthermore, the
explicit inclusion of reach allows for strategic insight on the trade-offs
between reach and results. (See The Three Rs of
Performance: Core concepts for planning, measurement, and management, Part
2, Section 2 for a further discussion.) On several occasions, we have
found that work groups have come to realize that their results
expectations were unrealistic given their targeted reach and their given
resources. A performance
framework such as that contained in the exhibit below can help to
explicitly address the problems noted above.
This model can
serve planners as well as evaluators. (See Refocus Your Questions for
Better Business Planning.) A more traditional
logic modelling approach which included reach was noted by Michael Quinn
Patton in his most recent version of Utilization-Focused
Evaluation, 1997. This model dates back to the 1970s in the analysis
of educational initiatives. The approach is described below:
©1998 Performance
Management Network Appendix B.)
©1998 Performance Management
Network | |||||||||||||||||||||||||||||||
