Was ist Evaluation? / What is evaluation?

"Once upon a time there was a word. And the word was evaluation. And the word was good. Teachers used the word in a particular way. Later on, other people used the word in a different way. After a while, nobody knew for sure what the word meant. But they all knew it was a good word. Evaluation was a thing to be cherished. But what kind of a good thing was it? More important, what kind of a good thing is it?" (Popham, 1993, p. 1)

"Evaluation - more than any science - is what people say it is; and people are saying it is many different things." (Glass & Ellet, 1980, p. 211)

"Research is aimed at truth. Evaluation is aimed at action." (wird M.Q.Patton zugeschrieben, Quelle mir unbekannt) Richtig muss es heißen: "Research aims to produce knowledge and truth. Useful evaluation supports action." (Patton, 1997, p. 24)

"Irgend etwas wird von irgend jemandem nach irgendwelchen Kriterien in irgendeiner Weise bewertet." (Kromrey, 2001, S. 21)

"[...] evaluation has two arms, only one of which is engaged in data-gathering. The other arm collects, clarifies, and verifies relevant values and standards." (Scriven, 1991, p. 5)

"The evaluation responsibility is a responsibility to make judgements." (Stake, 1979, p. 55)

Evaluatoren / Evaluators

"In God we trust. All others must bring data." (Robert Hayden, Plymouth State College, zit. n. http://www.keypress.com/fathom/jokes.html; Berk, 2007 zitiert abweichend W. Edwards Deming als Urheber)

"Der Umfang der Gefahren bei der konkreten Forschung wirkt sich auf viele Praktiker mit Sicherheit nicht gerade ermutigend aus. Scheint doch das Einzige mit Gewißheit Vorhersagbare zu sein, daß immer etwas falsch gemacht werden wird." (Wittmann, 1985, S. 187)

"The notion of the evaluator as a superman who will make all social choices easy and all programs efficient, turning public management into a technology, is a pipe dream." (Cronbach et al., 1980, p. 4)

"Once upon a time, the evaluation researcher needed only the 'Bible' ('Old Testament', Campbell and Stanley, 1963; 'New Testament', Cook and Campbell, 1979) to look up an appropriate research design and, hey presto, be out into the field." (Pawson & Tilley, 1997, p. 1)

"To make research work when it is coping with the complexities of real people in real programs run by real organizations takes skill – and some guts." (Weiss, 1972, p. 9)

"[...] what the professional independent evaluator brings to the party is a fresh eye and some technical skills." (Scriven, 1997, p. 499)

Die Praxis / The practice

"One requires: a good sense of humour; and a thick skin. Above all else, don't take yourself too seriously (and try not to be paranoid when having inappropriate discussions in a public space.)" (Fear, Bill. Career in Evaluation - Opinions wanted. EVALTALK , 7 Dec 2004.)

"The world of evaluation is a frighteningly real world. [...] The actors in the educational drama are strikingly human, with all the attendant frailties of real people." (Popham, 1993, p. 217)

"Evaluators who steel themselves against the probable perils of reality will be less shocked when they try out their shiny new evaluation skills." (Popham, 1993, p. 217)

"Recently, I opened an evaluation process with a staff workshop in which I invited participants to share perceptions of and metaphors for evaluation. The program director went to a nearby closet, took out a vacuum cleaner, turned it on, and pronounced: 'Evaluation sucks!'" (Patton, 1997, p. 267)

"Doing a good evaluation is not a stroll on the beach." (Weiss, 1997, p. 325)

"I find that I have to begin every evaluation exercise by finding out what people’s previous experiences have been with evaluation, and I find many of those experiences have been negative." (Patton, 2002, p. 131)

Die Rolle von Evaluation / The role of evaluation

"The wolfdog of evaluation is acceptable as a method of controlling the peasants, but it must not be allowed into the castle – that is the message which each of these ideologies represents, in its own way." (Scriven, 2000, p. 252)

"Evaluation avanciert zum neuen Kampfbegriff in der Qualitätsdebatte" (Schratz, 1999, S. 64)

"The more evaluation, the less program development; the more demonstration projects, the less follow-through" ("Wilensky's Law", Wilensky, 1985, S. 9)

"In many educational systems everybody seems to hate external evaluation while nobody trusts internal evaluation." (Nevo, 2001, p. 104)

"We live in a knowledge-centred, value-adding, information-processing, management-fixated world which has an obsession with decision-making." (Pawson & Tilley, 1997, pp. xi-xii)

"[...] 'evaluation' has become a mantra of modernity." (Pawson & Tilley, 1997, p. 2)

"I've often referred to the difference between Evaluation and evaluation. Oddly enough evaluation is a much bigger endeavour. Everyone does it often with great rigour, sometimes with a rigour we don't comprehend or agree with. On the other hand Evaluation is our patch of earth and a small one in the grand scheme of things." (Williams, Bob . Re: A Sunday meditation (definitely about evaluation), EVALTALK , 15 Nov 2006)

"In the end it's politics!" (Capela, Stan . Re: A Wednesday clarification (longish and occasionally peevish), EVALTALK , 20 Nov 2006)

Evaluationsforschung / Evaluation research

"Evaluation research 1963-1997
Must do better. Too easily distractred by silly ideas. Ought to have a clearer sense of priorities and to work more systematically to see them through. Will yet go on to do great things." (Pawson & Tilley, 1997, p. 28)

"Evaluation no longer has the luxury of a-empirical theoretical development." (Smith, 1993. p. 241)

Evaluationsgegenstände / Evaluation objects

"What is evaluated? Everything. One can begin at the beginning of a dictionary and go through to the end, and every noun, common or proper, calls to mind a context in which evaluation would be appropriate" (Scriven, 1980, p. 4)

"Social programs are complex undertakings. They are an amalgam of dreams and personalities, rooms and theories, paper clips and organizational structure, clients and activities, budgets and photocopies, and great intentions." (Weiss, 1998, p. 48)

"Unfortunately, except in a few areas, planning of social programs proceeds more by the seat of the pants and the example of 'what everybody else is doing,' than it does by thoughtful and critical review of evidence and experience." (Weiss, 2002, p. 204)

Ziele von Evaluation / Goals of evaluation

"The purpose of evaluation is not to prove, but to improve." (Egon Guba, zit. n. Stufflebeam, 2004)

"Evaluation's most important purpose is not to prove, but to improve." (Stufflebeam, 2004, p. 247)

"Ergebnisse einer Evaluation sind nicht Daten, sondern Entscheidungen über Konsequenzen für die weitere Arbeitsplanung." (Burkard & Eikenbusch, 2000, S. 29)

"We are impressed by the creativity in the field of evaluation, yet at the same time concerned because evaluators often forget or fail to emphasize the basic purpose of their work." (Glass & Ellet, 1980, p. 212)

Terminologisches / Terminology

"While I do think that people who invent terms have some obligation to argue against careless shifts from their original meanings, they also have an obligation to be open-minded about serious arguments for modification or clarification of the original definitions." (Scriven, 2004, p. 17, in JMDE No. 1)

"For a time it appeared that an educational evaluation model was being generated by anyone who (1) could spell educational evaluation and (2) had access to an appropriate number of boxes and arrows." (Popham, 1993, p. 23)

"One gets the impression that what passes for evaluative research is indeed a mixed bag at best and chaos at worst." (Suchman, 1967, p. vii)

Ursprünge der Evaluation / Origins of evaluation

"From the ambitions of the academic disciplines, from the convulsive reforms of the educational system, from the battle-ground of the War on Poverty, from the ashes of the Great Society, from the reprisals of an indignant taxpaying public, there has emerged evaluation." (Glass, 1976, p. 9)

"There was a general concern over the poor academic performance of our nation's youth. ...? The quest for accountability had begun." (Baron & Baron, 1980, p. 85-86)

"Our search as lay historians reveals that the the first recorded instance of evaluation occurred when man, woman, and serpent were punished for having engaged in acts which apparently had not been among the objectives defined by the Program circumscribing their existence." (Perloff, Perloff & Sussna, 1976, p. 264)

"In the beginning, God created the heaven and the earth. And God saw everything that he made. "Behold," God said, "it is very good." And the evening and the morning were the sixth day. And on the seventh day God rested from all His work. His archangel came then unto Him asking, "God, how do you know that what you have created is 'very good'? What are your criteria? On what data do you base your judgment? Just exactly what results were you expecting to attain? And aren't you a little close to the situation to make a fair and unbiased evaluation?" God thought about these questions all that day and His rest was greatly disturbed. On the eighth day God said, "Lucifer, go to hell." Thus was evaluation born in a blaze of glory." Halcolm's The Real Story of Paradise Lost (Patton, 1997, p. 1)

Methodenstreit / Dispute on methods

"The difference [between quantitative and qualitative researchers] is that, while a quantitative reporter would say 'Only ten persons were present ...,' a truly qualitative reporter would say, 'Attendance at the session was depressing.'" (Sechrest & Figueredo, 1993, p. 655)

"We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute." (Smith & Pell, 2003, p. 1459)

Nutzung von Evaluationsergebnissen

"You mean you guys actually look at the evaluations? I taught two sections of the same class last semester, and I stopped reading the evaluations after about the sixth section I taught. Most are positive, some wish I would die, and none provide useful feedback." (tuxthepenguin auf http://chronicle.com/forums/index.php?topic=69226.0)

"There is nothing a Government hates more than to be well informed; for it makes the process of arriving at decisions much more complicated and difficult." (John Maynard Keynes, The Times, March 11, 1937, p. 18)

Programmtheorien und logische Modelle

"The program theory approach has exposed the impoverished nature of the theories that underlie many of the interventions we study." (Bickman, 2000, p. 107)

"A program is a theory and an evaluation is its test." (Rein, 1981, S. 141)