Alles, was die praktische Planung, Durchführung und Nutzung von Evaluationen betrifft. Potentiell können hier also Entscheider und Nutzer, (angehende) Evaluatoren, aber auch sonstige Akteure und Interessierte Informationen dazu finden, wie man Evaluationen plant, durchführt und nutzt.
Subject: Evaluation budget as a %% of program expenditure
Date: Sat, 27 Nov 2004 14:04:15 +1000
From: Sonia Whiteley
I realise this is a topic that comes up on almost an annual basis (Sept 2003 at last count) but I was wondering whether there were any new thoughts on the issue.
I'm more interested in large-scale programs - at least $5 million plus Aussie $$s - where evaluation is built into the program from day 1 (ie as the program is being carefully crafted from policy).
What percentage of the program (not the organisational) budget should be allocated to evaluation?
What actually happens in the real world of program evalution budgets?
Does this differ across across departments/areas of responsibility? Or more specifically, are health department budgets, for example, generally larger percentagewise than those from education or the environment?
Any pointers to recent references or case studies would be greatly appreciated.
Many thanks
Sonia Whiteley
EVALTALK - American Evaluation Association (AEA) Discussion List. See also
the website: http://www.eval.org
To unsubscribe from EVALTALK, send e-mail to listserv@bama.ua.edu
with only the following in the body: UNSUBSCRIBE EVALTALK
To get a summary of commands, send e-mail to listserv@bama.ua.edu
with only the following in the body: INFO REFCARD
To use the archives, go to this web site: http://bama.ua.edu/archives/evaltalk.html For other problems, contact a list owner at kbolland@sw.ua.edu or carolyn.sullins@wmich.edu
Welche Anforderungen?
Wer macht Evaluation?
Date: Wed, 21 Jul 2004 08:29:49 +0100
From: bill fearSubject: Agendas, decisions and using evaluation The debate about the role of the evaluator in relation to getting the evaluation used has had a long and perennial history re-emerging, as ever, about once every five years. There are a couple, or more, important points that are consistent (IMESHO):
1) No evaluator has the right to assume that their findings will, or should, be used as a number of people have just recently noted. This right is the preserve of auditors. 2) There are two ways to maximise the value of an evaluation: a) involve stakeholders from the off (Patton); b) link evaluation to budgets (Australia; the Netherlands). 3) Most interestingly, a piece of work by by the NAO (probably by Chelimsky and published around 5-8 years ago; sorry, I have a real problem remembering references) showed that high quality evaluations tend to be rejected initially. However, these same evaluations usually have an impact around five years - that's 5 years - later, usually at the conceptual level. Ergo, an acid test of a good evaluation that has been carried out independently of stakeholders may well be the degree of initial resistance and rejection. Indeed, it may be that an evaluation has more impact if the evaluator does not try to get it taken account of. Just think through what we know about decision making.
On that point, any good evaluator surely must, surely absolutely must, have an understanding of decision making from the individual level to the organisational level.
Helpful references are:
at an individual level
www.bps.org.uk then click on 'publications' then 'the psychologist' then 'search the psychologist online' then 'volume 15 (2002)' then 'volume 15 part 2(February 2002)' then look at articles 4, 5, 6, 7. Easy reading to a high standard (mostly).
and
Gilbert, D. and Wilson, T. 'Miswanting.' www.wjh.harvard.edu/~dtg/Gilbert%20&%20Wilson%20(Miswanting).pdf (or put 'miswanting' into google)
At an organisational level it is still, for me, the stock in trade publication of 'Organsiations: Structures, processes and outcomes' by Hall.
We might also want to consider that US Senators apparently spends just 7 minutes a day reading on average and that for a GP to keep up to date with current relevant medicine they need to read for 17 hours a week (mostly non-fiction, or at least not knowingly fiction).
And then of course there is the values of the evaluator. Our values tend to drive our behaviour - although they don't have to. Not judging others on the basis of their values, which may conflict heavily with our own, is immensely difficult. So, we may assume that our evaluation should be taken account of according to our values, but the values of the person on the other side may be different. And somehow we have to find a way not to let that influence our behaviour and to respect the values of the other/s. After all, there is no moral 'right' or 'wrong', and ethics are consensus of agreed rules depicting right and wrong, and not a universal absolute, and there is no known set of universal values.
At 1:36 PM -0400 13/10/04, Jill Ibell wrote:
>>Please let me know specific interview questions that you have found >>helpful in prior program evaluations. The use is for an internal >>program evaluation process, which has recently been started on a >>more formal basis than prior years trouble shooting operations.
Here's my generic, use anywhere, run out of ideas evaluation questions. They are based on Vygotskyian learning theory and action research practice.
these are my standby, run out of bright ideas, interview questions that have never failed to get some interesting and valuable responses. I've tried to turn them into something that relates to what you are interested in, but you get the general drift :-
As I get older I half begin to think that these may be the only questions you need to ask. In my experience, the responses are incredibly rich and insightful about people's judgement of worth, and it forces them to base their responses on observable or justifiable data.
Cheers
Bob
BOB WILLIAMS bobwill@actrix.co.nz Check out the free resources on my WEB site http://users.actrix.co.nz/bobwill
Mobile (64) 21 254 8983
... there are always exceptions. Reality is too complex to be captured by theory. I'm reminded of the general semantics principle that "the map is not the territory"-that a theory is distinct from the reality it purports to represent. Bob Dick
EVALTALK - American Evaluation Association (AEA) Discussion List. See also
the website: http://www.eval.org
To unsubscribe from EVALTALK, send e-mail to listserv@bama.ua.edu
with only the following in the body: UNSUBSCRIBE EVALTALK
To get a summary of commands, send e-mail to listserv@bama.ua.edu
with only the following in the body: INFO REFCARD
To use the archives, go to this web site: http://bama.ua.edu/archives/evaltalk.html For other problems, contact a list owner at kbolland@sw.ua.edu or carolyn.sullins@wmich.edu
Original Message --------
Subject: Re: Looking for methodologies to identify/choose stake holders
Date: Sun, 14 Nov 2004 12:38:50 -0800
From: Avichal Jha
Hi Jonny,
Michael Patton's "snowball" sampling technique comes to mind. You can find a discussion of different techniques in "Utilization Focused Evaluation," published by sage. I believe the 3rd is the most recent edition. Carol Weiss also has a great discussion on involving stakeholders in "Evaluation: Methods for Studying Programs and Policies."
What the discussion boils down to is context: What are you evaluating? The evaluand itself should suggest at least a limited group of stakeholders; i.e., those who asked for the evaluation. In the case where we're evaluating policy, this may not be the case. In that situation, the context becomes that of the policy. As long as you have a single stakeholder in mind, ask that stakeholder for who other stakeholders might be. This process, repeated with each new stakeholder, will "snowball" into a much larger sample.
This is just one of the ways that Patton and others have discussed. I hope it helps (although my gut feeling is that this is more useful for program evaluation than policy analysis). As I suggested, if you haven't already looked at Patton and Weiss, I think you'll find their work very helpful.
Best of luck, Avi
Avichal Jha, M.A. Doctoral Student Evaluation and Applied Methods Claremont Graduate University avichal.jha@cgu.edu
Original Message----- From: American Evaluation Association Discussion List To: EVALTALK@BAMA.UA.EDU Sent: 11/14/2004 10:20 AM Subject: Looking for methodologies to identify/choose stake holders
We all agree that it is important to involve stake holders in various phases of the evaluation life cycle. But how to identify the population of relevant stake holders and choose among them? My sense is that we tend to use the "I will know them when I see them" method. (It's what I do.) But are there more deliberate and systematic ways to go about it? Has anyone tried to develop a methodology? If anyone has relevant references, please send them my way. Thanks.
Jonny Jonathan A. Morell, Ph.D. Senior Policy Analyst
Street address: 3520 Green Court Suite 300, Ann Arbor Michigan 48105 Mail address: PO Box 134001 Ann Arbor, Michigan 48113-4001
Desk 734 302-4668 Reception 734 302-4600 Fax:734 302-4991 Email jonny.morell@altarum.org
Was ist Evaluation? / What is evaluation?
"Once upon a time there was a word. And the word was evaluation. And the word was good. Teachers used the word in a particular way. Later on, other people used the word in a different way. After a while, nobody knew for sure what the word meant. But they all knew it was a good word. Evaluation was a thing to be cherished. But what kind of a good thing was it? More important, what kind of a good thing is it?" (Popham, 1993, p. 1)
"Evaluation - more than any science - is what people say it is; and people are saying it is many different things." (Glass & Ellet, 1980, p. 211)
"A <acronym title="Gezielt konzipiertes Maßnahmenpaket, das eine erwünschte Wirkung auf individueller oder sozialer Ebene herbeiführen soll. ">program is a theory and an evaluation is its test." (Rein, 1981, S. 141)
"Research is aimed at truth. Evaluation is aimed at action." (wird M.Q.<acronym title="Seit den 1970ern bekannter Evaluationsexperte und Urheber der utilization focused evaluation. ">Patton zugeschrieben, Quelle mir unbekannt) Richtig muss es heißen: "Research aims to produce knowledge and truth. Useful evaluation supports action." (<acronym title="Seit den 1970ern bekannter Evaluationsexperte und Urheber der utilization focused evaluation. ">Patton, 1997, p. 24)
"Irgend etwas wird von irgend jemandem nach irgendwelchen Kriterien in irgendeiner Weise bewertet." (Kromrey, 2001, S. 21)
"...? evaluation has two arms, only one of which is engaged in data-gathering. The other arm collects, clarifies, and verifies relevant values and <acronym title="Vergleichs- oder Soll-Größe, die als Richtwert beim Fällen eines Werturteils über einen Evaluationsgegenstand angelegt wird. ">standards." (<acronym title="Seit den 1960er Jahren einer der einflussreichsten Denker der Evaluationsliteratur und Urheber bekannter Fachbegriffe wie formative Evaluation/summative Evaluation, Metaevaluation, Evaluand und Logik der Evaluation. ">Scriven, 1991, p. 5)
"The evaluation responsibility is a responsibility to make judgements." (Stake, 1979, p. 55)
<acronym title="Person(en), welche eine Evaluation durchführen, also v.a. für die Prüfung der Evaluierbarkeit, das Evaluationsdesign, die Datensammlung und -analyse sowie die Berichterstattung verantwortlich sind.
"In God we trust. All others must bring data." (Robert Hayden, Plymouth State College, zit. n. http://www.keypress.com/fathom/jokes.html;³³1³ Berk, 2007 zitiert abweichend W. Edwards Deming als Urheber)
"Der Umfang der Gefahren bei der konkreten Forschung wirkt sich auf viele <acronym title="Personen, die im engeren Sinne für die Entwicklung und/oder Umsetzung eines Evaluationsgegenstands verantwortlich sind. Im Kontext Programmevaluation sind sie diejenigen, die sicherstellen, dass die Leistungen des Programms so bei der Zielgruppe ankommen, wie das Programmkonzept es vorsieht. ">Praktiker mit Sicherheit nicht gerade ermutigend aus. Scheint doch das Einzige mit Gewißheit Vorhersagbare zu sein, daß immer etwas falsch gemacht werden wird." (Wittmann, 1985, S. 187)
"The notion of the <acronym title="Person(en), welche eine Evaluation durchführen, also v.a. für die Prüfung der Evaluierbarkeit, das Evaluationsdesign, die Datensammlung und -analyse sowie die Berichterstattung verantwortlich sind. ">evaluator as a superman who will make all social choices easy and all <acronym title="Gezielt konzipiertes Maßnahmenpaket, das eine erwünschte Wirkung auf individueller oder sozialer Ebene herbeiführen soll. ">programs efficient, turning public management into a technology, is a pipe dream." (Cronbach et al., 1980, p. 4)
"Once upon a time, the <acronym title="Person(en), welche eine Evaluation durchführen, also v.a. für die Prüfung der Evaluierbarkeit, das Evaluationsdesign, die Datensammlung und -analyse sowie die Berichterstattung verantwortlich sind. ">evaluation researcher <acronym title="Identifizierbares Defizit von Individuen, Organisationen, Systemen oder sonstigen Sachverhalten, dem durch ein Programm, ein Produkt, eine politische Strategie oder anderen Evaluationsgegenständen entgegengewirkt werden soll. ">needed only the 'Bible' ('Old Testament', Campbell and Stanley, 1963; 'New Testament', Cook and Campbell, 1979) to look up an appropriate <acronym title="Wann und wie von wem welche Daten erhoben und ausgewertet werden. Wird üblicherweise in der Planungsphase einer Evaluation vom Evaluationsteam als Teil des Evaluationskonzepts erstellt. ">research design and, hey presto, be out into the field." (Pawson & Tilley, 1997, p. 1)
"To make research work when it is coping with the complexities of real people in real <acronym title="Gezielt konzipiertes Maßnahmenpaket, das eine erwünschte Wirkung auf individueller oder sozialer Ebene herbeiführen soll. ">programs run by real organizations takes skill – and some guts." (<acronym title="Die grand dame der Evaluation, Autorin eines der ersten Lehrbücher (Evaluation Research, 1972), ist vor allem bekannt geworden durch ihre einflussreichen empirischen Arbeiten zur Evaluationsnutzung seit den 1970er Jahren. ">Weiss, 1972, p. 9)
"...? what the professional independent <acronym title="Person(en), welche eine Evaluation durchführen, also v.a. für die Prüfung der Evaluierbarkeit, das Evaluationsdesign, die Datensammlung und -analyse sowie die Berichterstattung verantwortlich sind. ">evaluator brings to the party is a fresh eye and some technical skills." (<acronym title="Seit den 1960er Jahren einer der einflussreichsten Denker der Evaluationsliteratur und Urheber bekannter Fachbegriffe wie formative Evaluation/summative Evaluation, Metaevaluation, Evaluand und Logik der Evaluation. ">Scriven, 1997, p. 499)
Die Praxis / The practice
"One requires:
a good sense of humour;
and a thick skin.
Above all else, don't take yourself too seriously (and try not to be paranoid when having inappropriate discussions in a public space.)" (Fear, Bill. Career in Evaluation - Opinions wanted. EVALTALK <listserv@bama.ua.edu>, 7 Dec 2004.)
"The world of evaluation is a frighteningly real world. ...? The actors in the educational drama are strikingly human, with all the attendant frailties of real people." (Popham, 1993, p. 217)
"<acronym title="Person(en), welche eine Evaluation durchführen, also v.a. für die Prüfung der Evaluierbarkeit, das Evaluationsdesign, die Datensammlung und -analyse sowie die Berichterstattung verantwortlich sind. ">Evaluators who steel themselves against the probable perils of reality will be less shocked when they try out their shiny new evaluation skills." (Popham, 1993, p. 217)
"Recently, I opened an evaluation process with a <acronym title="Personen, die im engeren Sinne für die Entwicklung und/oder Umsetzung eines Evaluationsgegenstands verantwortlich sind. Im Kontext Programmevaluation sind sie diejenigen, die sicherstellen, dass die Leistungen des Programms so bei der Zielgruppe ankommen, wie das Programmkonzept es vorsieht. ">staff workshop in which I invited participants to share perceptions of and metaphors for evaluation. The <acronym title="Gezielt konzipiertes Maßnahmenpaket, das eine erwünschte Wirkung auf individueller oder sozialer Ebene herbeiführen soll. ">program director went to a nearby closet, took out a vacuum cleaner, turned it on, and pronounced: 'Evaluation sucks!'" (<acronym title="Seit den 1970ern bekannter Evaluationsexperte und Urheber der utilization focused evaluation. ">Patton, 1997, p. 267)
"Doing evaluation is not a stroll on the beach." (<acronym title="Die grand dame der Evaluation, Autorin eines der ersten Lehrbücher (Evaluation Research, 1972), ist vor allem bekannt geworden durch ihre einflussreichen empirischen Arbeiten zur Evaluationsnutzung seit den 1970er Jahren. ">Weiss, 1997, p. 325)
"I find that I have to begin every evaluation exercise by finding out what people’s previous experiences have been with evaluation, and I find many of those experiences have been negative." (<acronym title="Seit den 1970ern bekannter Evaluationsexperte und Urheber der utilization focused evaluation. ">Patton, 2002, p. 131)
Die Rolle von Evaluation / The role of evaluation
"The wolfdog of evaluation is acceptable as a method of <acronym title="Die Sammlung und Interpretation von relevanten Daten für die Steuerung von Organisationen mit dem Ziel, die Planungs- und Steuerungsprozesse von Verfahren und Prozessen möglichst effektiv und effizient zu gestalten. ">controlling the peasants, but it must not be allowed into the castle – that is the message which each of these ideologies represents, in its own way." (<acronym title="Seit den 1960er Jahren einer der einflussreichsten Denker der Evaluationsliteratur und Urheber bekannter Fachbegriffe wie formative Evaluation/summative Evaluation, Metaevaluation, Evaluand und Logik der Evaluation. ">Scriven, 2000, p. 252)
"Evaluation avanciert zum neuen Kampfbegriff in der <acronym title="Innerer oder intrinsischer Wert eines Evaluationsgegenstands, unabhängig von jedem Anwendungskontext. Die Bestimmung von Qualität (und/oder Nutzen) eines Gegenstands ist das Kerngeschäft einer Evaluation. ">Qualitätsdebatte" (Schratz, 1999, S. 64)
"The more evaluation, the less <acronym title="Gezielt konzipiertes Maßnahmenpaket, das eine erwünschte Wirkung auf individueller oder sozialer Ebene herbeiführen soll. ">program development; the more demonstration <acronym title="Eine Vorhaben, das eine konkrete Zielsetzung verfolgt, zeitlich begrenzt ist, neuartig ist, organisatorisch komplex ist, meist mehrere Personen bzw. Personengruppen umfasst und eine gezielte Veränderung herbeiführen will. ">projects, the less follow-through" ("Wilensky's Law", Wilensky, 1985, S. 9)
"In many educational systems everybody seems to hate <acronym title="Evaluation, bei der das Evaluationsteam nicht zu jener Organisation gehört, die den Evaluationsgegenstand durchführt oder zu verantworten hat. Eine externe Evaluation ist immer eine Fremdevaluation. ">external evaluation while nobody trusts <acronym title="Evaluation, bei der das Evaluationsteam oder der Evaluator in der selben Institution angesiedelt ist, wie der Evaluationsgegenstand. ">internal evaluation." (Nevo, 2001, p. 104)
"We live in a knowledge-centred, value-adding, information-processing, management-fixated world which has an obsession with decision-making." (Pawson & Tilley, 1997, pp. xi-xii)
"...? 'evaluation' has become a mantra of modernity." (Pawson & Tilley, 1997, p. 2)
"I've often referred to the difference between Evaluation and evaluation. Oddly enough evaluation is a much bigger endeavour. Everyone does it often with great rigour, sometimes with a rigour we don't comprehend or agree with. On the other hand Evaluation is our patch of earth and a small one in the grand scheme of things." (Williams, Bob
"In the end it's politics!" (Capela, Stan
<acronym title="(a) Die theoretische und empirische Forschung über Bedingungen, Praxis und Wirkungen von Evaluation. (b) Explizit wissenschaftlich orientierte Evaluation.
">Evaluationsforschung / Evaluation research
"Evaluation research 1963-1997
Must do better. Too easily distractred by silly ideas. Ought to have a clearer sense of priorities and to work more systematically to see them through. Will yet go on to do great things." (Pawson & Tilley, 1997, p. 28)
"Evaluation no longer has the luxury of a-empirical theoretical development." (Smith, 1993. p. 241)
<acronym title="Das, was in einer Evaluation untersucht und bewertet wird, bzw. dessen Qualität oder Nutzen bestimmt wird.
">Evaluationsgegenstände / Evaluation objects
"What is evaluated? Everything. One can begin at the beginning of a dictionary and go through to the end, and every noun, common or proper, calls to mind a <acronym title="Faktoren in der Programmumwelt, die einen Einfluss auf den Evaluationsgegenstand und seine Wirkmechanismen haben. ">context in which evaluation would be appropriate" (<acronym title="Seit den 1960er Jahren einer der einflussreichsten Denker der Evaluationsliteratur und Urheber bekannter Fachbegriffe wie formative Evaluation/summative Evaluation, Metaevaluation, Evaluand und Logik der Evaluation. ">Scriven, 1980, p. 4)
"Social <acronym title="Gezielt konzipiertes Maßnahmenpaket, das eine erwünschte Wirkung auf individueller oder sozialer Ebene herbeiführen soll. ">programs are complex undertakings. They are an amalgam of dreams and <acronym title="Personen, die für die Produktion bestimmter Leistungen verantwortlich sind. Im Kontext von Personalevaluationen ist Personal ein möglicher Evaluationsgegenstand, beispielsweise im Assessment-Center oder beim Vorgesetzten-Feedback. ">personalities, rooms and theories, paper clips and organizational structure, <acronym title="Person oder Institution, die eine Evaluation bei einem Evaluationsteam (Auftragnehmer) in Auftag gibt. Ist in vielen Fällen auch Entscheider bzw. Abnehmer der Berichterstattung der Evaluation. ">clients and activities, budgets and photocopies, and great intentions." (<acronym title="Die grand dame der Evaluation, Autorin eines der ersten Lehrbücher (Evaluation Research, 1972), ist vor allem bekannt geworden durch ihre einflussreichen empirischen Arbeiten zur Evaluationsnutzung seit den 1970er Jahren. ">Weiss, 1998, p. 48)
Ziele von Evaluation / Goals of evaluation
"The <acronym title="Funktionen und Ziele, die mit einer Evaluation erfüllt werden (sollen). ">purpose of evaluation is not to prove, but to improve." (Egon Guba, zit. n. <acronym title="Seit den 1960er Jahren eine der einflussreichsten Evaluationskoryphäen, bekannt vor allem als Urheber des CIPP-Modells und für seine frühe Mitarbeit an den Evaluationsstandards des Joint Committee für Programme und Personal. ">Stufflebeam, 2004)
"Evaluation's most important purpose is not to prove, but to improve." (<acronym title="Seit den 1960er Jahren eine der einflussreichsten Evaluationskoryphäen, bekannt vor allem als Urheber des CIPP-Modells und für seine frühe Mitarbeit an den Evaluationsstandards des Joint Committee für Programme und Personal. ">Stufflebeam, 2004, p. 247)
"<acronym title="Ergebnisse einer Evaluationsstudie, also die methodisch begründbare Beantwortung der Fragestellungen der Evaluation und weitere Erkenntnisse, niedergelegt in der Berichterstattung. ">Ergebnisse einer Evaluation sind nicht Daten, sondern Entscheidungen über Konsequenzen für die weitere Arbeitsplanung." (Burkard & Eikenbusch, 2000, S. 29)
"We are impressed by the creativity in the field of evaluation, yet at the same time concerned because <acronym title="Person(en), welche eine Evaluation durchführen, also v.a. für die Prüfung der Evaluierbarkeit, das Evaluationsdesign, die Datensammlung und -analyse sowie die Berichterstattung verantwortlich sind. ">evaluators often forget or fail to emphasize the basic purpose of their work." (Glass & Ellet, 1980, p. 212)
Terminologisches / Terminology
"While I do think that people who invent terms have some obligation to argue against careless shifts from their original meanings, they also have an obligation to be open-minded about serious arguments for modification or clarification of the original definitions." (<acronym title="Seit den 1960er Jahren einer der einflussreichsten Denker der Evaluationsliteratur und Urheber bekannter Fachbegriffe wie formative Evaluation/summative Evaluation, Metaevaluation, Evaluand und Logik der Evaluation. ">Scriven, 2004, p. 17, in JMDE No. 1)
"For a time it appeared that an educational <acronym title="Bündel deskriptiver, präskriptiver und/oder normativer Aussagen zu Bedingungen, Wirkungen und/oder Umsetzungsvarianten von Evaluation, die optimalerweise auf theoretischen und empirischen Ergebnissen der Evaluationsforschung basieren. ">evaluation model was being generated by anyone who (1) could spell educational evaluation and (2) had access to an appropriate number of boxes and arrows." (Popham, 1993, p. 23)
"One gets the impression that what passes for evaluative research is indeed a mixed bag at best and chaos at worst." (Suchman, 1967, p. vii)
Ursprünge der Evaluation / Origins of evaluation
"From the ambitions of the academic disciplines, from the convulsive reforms of the educational system, from the battle-ground of the War on Poverty, from the ashes of the Great Society, from the reprisals of an indignant taxpaying public, there has emerged evaluation." (Glass, 1976, p. 9)
"There was a general concern over the poor academic <acronym title="Umfasst die in ein Programm investierten Ressourcen, die Aktivitäten des Programms sowie seine Outcomes, Impacts und Nebeneffekte. ">performance of our nation's youth. ...? The quest for accountability had begun." (Baron & Baron, 1980, p. 85-86)
"Our search as lay historians reveals that the the first recorded instance of evaluation occurred when man, woman, and serpent were punished for having engaged in acts which apparently had not been among the objectives defined by the <acronym title="Gezielt konzipiertes Maßnahmenpaket, das eine erwünschte Wirkung auf individueller oder sozialer Ebene herbeiführen soll. ">Program circumscribing their existence." (Perloff, Perloff & Sussna, 1976, p. 264)
"In the beginning, God created the heaven and the earth. And God saw everything that he made. "Behold," God said, "it is very good." And the evening and the morning were the sixth day. And on the seventh day God rested from all His work. His archangel came then unto Him asking, "God, how do you know that what you have created is 'very good'? What are your criteria? On what data do you base your judgment? Just exactly what results were you expecting to attain? And aren't you a little close to the situation to make a fair and unbiased evaluation?" God thought about these questions all that day and His rest was greatly disturbed. On the eighth day God said, "Lucifer, go to hell." Thus was evaluation born in a blaze of glory." <acronym title="Fiktiver Evaluationsguru und Alter Ego von Michael Quinn Patton. ">Halcolm's The Real Story of Paradise Lost (<acronym title="Seit den 1970ern bekannter Evaluationsexperte und Urheber der utilization focused evaluation. ">Patton, 1997, p. 1)
Methodenstreit / Dispute on methods
"The difference between quantitative and qualitative researchers? is that, while a quantitative <acronym title="Schriftliche Berichterstattung über die Ergebnisse einer Evaluationsstudie. ">reporter would say 'Only ten persons were present ...,' a truly qualitative <acronym title="Schriftliche Berichterstattung über die Ergebnisse einer Evaluationsstudie. ">reporter would say, 'Attendance at the session was depressing.'" (Sechrest & Figueredo, 1993, p. 655)
"We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute." (Smith & Pell, 2003, p. 1459)
<acronym title="Verwendung von Evaluationsergebnissen durch Entscheider, Nutzer oder Abnehmer von Evaluationsberichten.
"You mean you guys actually look at the evaluations? I taught two sections of the same class last semester, and I stopped reading the evaluations after about the sixth section I taught. Most are positive, some wish I would die, and none provide useful feedback." (tuxthepenguin auf http://chronicle.com/forums/index.php?topic=69226.0³³1³)
<acronym title="Konzept eines Programms in Form einer Zusammenfassung von Variablen und deren Kausalzusammenhängen, die für die Wirkweise des Programms relevant sind. Logische Modelle sind zentral für die theoriebasierte Evaluation.
">Programmtheorien und logische Modelle
"The <acronym title="Konzept eines Programms in Form einer Zusammenfassung von Variablen und deren Kausalzusammenhängen, die für die Wirkweise des Programms relevant sind. Logische Modelle sind zentral für die theoriebasierte Evaluation. ">program theory approach has exposed the impoverished nature of the theories that underlie many of the <acronym title="Gezielt konzipiertes Maßnahmenpaket, das eine erwünschte Wirkung auf individueller oder sozialer Ebene herbeiführen soll. ">interventions we study." (Bickman, 2000, p. 107)