Summative Evaluation

Subject: Re: Formative/Summative & Process/Outcome 2*2 Matrix? Date: Tue, 28 Dec 2004 12:24:10 -0500 From: Eileen Stryker Reply-To: American Evaluation Association Discussion List To: EVALTALK@BAMA.UA.EDU References:

I don't remember whether or where Dan Stufflebeam might have written this,

Jan Hense  29 Oct 2004 - 10:04  Formative Evaluation  Summative Evaluation  Textsammlung   

"Evaluation may be done to provide feedback to people who are trying to improve something (formative evaluation); or to provide information for decision-makers who are wondering whether to fund, terminate, or purchase something (summative evaluation)." (Scriven, 1980, S. 6-7)

Problematik des Begriffpaars

Der Begriff formative Evaluation (nicht das Konzept) geht auf Scriven (1972) zurück und bildet mit seinem Gegenstück summative Evaluation wohl das prominenteste Begriffspaar in der Evaluationsliteratur. Dennoch handelt es sich um einen problematischen Begriff, da er ungenau definiert, theoretisch unstimmig und in seiner praktischen Verwendung oft entsprechend beliebig ist (vgl. dazu etwa die Beiträge von Patton, Chen und Wholey in Evaluation Practice, 1996, Vol. 17, No. 2).

Empfehlungen zum Gebrauch von formativ und summativ

Da sich das Begriffspaar wegen seiner hohen Anmutungsqualität trotz dieser Probleme mit Sicherheit halten wird, scheint mir folgende Begriffsverwendung sinnvoll:

Die Begrifflichkeiten formativ/summativ werden ausschließlich zur Bezeichnung intendierter Evaluationszwecke verwendet, so wie es das obige Zitat Scrivens andeutet. Auf alle anderen von Scriven und Apologeten vertretenen Addenda wird verzichtet. Darunter fallen:

fgs dhjkfgösdljkfg ösldfkjg ösdflgkj södflkgj sdöfglk sjödf

Date:    Wed, 21 Jul 2004 08:29:49 +0100
From: bill fear Subject: Agendas, decisions and using evaluation

The debate about the role of the evaluator in relation to getting the evaluation used has had a long and perennial history re-emerging, as ever, about once every five years. There are a couple, or more, important points that are consistent (IMESHO):

1) No evaluator has the right to assume that their findings will, or should, be used as a number of people have just recently noted. This right is the preserve of auditors. 2) There are two ways to maximise the value of an evaluation: a) involve stakeholders from the off (Patton); b) link evaluation to budgets (Australia; the Netherlands). 3) Most interestingly, a piece of work by by the NAO (probably by Chelimsky and published around 5-8 years ago; sorry, I have a real problem remembering references) showed that high quality evaluations tend to be rejected initially. However, these same evaluations usually have an impact around five years - that's 5 years - later, usually at the conceptual level. Ergo, an acid test of a good evaluation that has been carried out independently of stakeholders may well be the degree of initial resistance and rejection. Indeed, it may be that an evaluation has more impact if the evaluator does not try to get it taken account of. Just think through what we know about decision making.

On that point, any good evaluator surely must, surely absolutely must, have an understanding of decision making from the individual level to the organisational level.

Helpful references are:

at an individual level

www.bps.org.uk then click on 'publications' then 'the psychologist' then 'search the psychologist online' then 'volume 15 (2002)' then 'volume 15 part 2(February 2002)' then look at articles 4, 5, 6, 7. Easy reading to a high standard (mostly).


Gilbert, D. and Wilson, T. 'Miswanting.' www.wjh.harvard.edu/~dtg/Gilbert%20&%20Wilson%20(Miswanting).pdf (or put 'miswanting' into google)

At an organisational level it is still, for me, the stock in trade publication of 'Organsiations: Structures, processes and outcomes' by Hall.

We might also want to consider that US Senators apparently spends just 7 minutes a day reading on average and that for a GP to keep up to date with current relevant medicine they need to read for 17 hours a week (mostly non-fiction, or at least not knowingly fiction).

And then of course there is the values of the evaluator. Our values tend to drive our behaviour - although they don't have to. Not judging others on the basis of their values, which may conflict heavily with our own, is immensely difficult. So, we may assume that our evaluation should be taken account of according to our values, but the values of the person on the other side may be different. And somehow we have to find a way not to let that influence our behaviour and to respect the values of the other/s. After all, there is no moral 'right' or 'wrong', and ethics are consensus of agreed rules depicting right and wrong, and not a universal absolute, and there is no known set of universal values.




Creative Commons License