How do we transform evaluation to be more strategically relevant and timely? Adopt a cumulative learning approach.

Recent evaluation leadership meetings in federal circles have suggested that the function needs to become more timely while also becoming more strategically useful to decision-makers.  In my evaluation practice and teaching I have come to the conclusion that a greater emphasis on cumulative learning vis avis program archetypes is a good part of the answer to this challenge. For a recent brief discussion on cumulative learning see featuring realist evaluation co-founder Ray Pawson. (The answer to his question btw re: who coined the term carrots, sticks and sermons can be debated, but to me was made prominent by Carleton’s Bruce Doern in his writings in public administration several decades ago. The term was directly referenced in the evaluation context by  M. L. Bemelmans-Videc, Ray C. Rist, and Evert Vedung in their book of same name published in 1998.)

Presentations at the 2010 CES Victoria conference (Building on our Strengths –  2011 AEA presentation in California (, the  2012 Halifax CES conference ( and a workshop at the 2013 CES Toronto conference – outlined some of  the basic thinking about how such an approach might be applied for Canada. In the last instance – I have even gone as far as to suggest a sort of streamlined approach to spur such cumulative learning  (

Over the past few years we have developed the approach for specific carrots (e.g. Grants + Contributions) sticks (e.g. Administrative Monetary Penalties)  and sermons (e.g. voluntary and compulsory standards and codes development, other types of information/education and policy programs) archetypes. This has already provided us with significant cumulative learning regarding what works for whom in what conditions and why. This knowledge in turn represents an asset and ‘leg-up’ to conducting essentially any future evaluation of these types of programs. Readers who would like to learn more and/or join in this movement are invited to contact us at and/or to stay tuned for presentations at future events!

Steve Montague, Partner PMN