evoluation.de - Assessment

Re: Evaluation, Assessment, and Rubrics

Wed, 15 Dec 2004 22:55:42 +0100

Original Message -------- Subject: Re: Evaluation, Assessment, and Rubrics Date: Wed, 15 Sep 2004 16:31:10 -0700 From: Richard Hake Reply-To: American Evaluation Association Discussion List To: EVALTALK@BAMA.UA.EDU

In her POD post of 14 Sep 2004 10:00:14-0700 titled "Evaluation, Assessment, and Rubrics," Leora Baron wrote:

I am looking for two items that my fellow POD'ers may be able to provide: First, a definition distinguishing between evaluation and assessment; and second, an online location that has a good description and illustration of rubrics.

I. ASSESSMENT vs EVALUATION If one:

(1) goes to the powerful but little used POD search engine ,

(2) types into the "Since" slot "2003" (without the quotes), and into the "Subject" slot,

(a) "assessment" (without the quotes), s(he) will obtain 90 hits,

(b) "evaluation" (without the quotes), s(he) will obtain 168 hits,

My own take on "assessment vs evaluation" can be found in Hake (2004). From the perspective of the physics education reform effort [Hake (2002a,b), I find it useful to make NO distinction between "assessment" and "evaluation," but to make a 4-quadrant discrimination cf., Stokes (2000) of types of assessment/evaluation on the basis formative vs summative on one axis and public vs private on an orthogonal axis.

The non distinction between "assessment" and "evaluation," is contrary to the preferences of: (a) Steve Ehrmann (2004), (b) most of those contributing to the POD thread "Assessment vs Evaluation," (c) Mark Davenport (2004), and (d) the "Glossary of Program Evaluation Terms" at Western Michigan University (Michael Scriven's new location).

II. RUBRICS If you mean by "rubric": "a technique, custom, form, or thing established or settled (as by authority)" (definition #4 in Webster's Third New International Dictionary Unabridged), then it all depends on what one is attempting to assess/evaluate.

IF it's student learning, and not *affective" impact as might be assessed by student evaluations of teaching (SET's)

(a) Peggy Maki's (2004) recent book might be useful, but I have not seen it. In a POD post of 22 Jul 2004 15:09:54-0400, Barbara Cambridge, Director of the Carnegie Academy Campus Program wrote: "Peggy Maki's new book on assessment is excellent. It is jointly published by Stylus and AAHE."

(b) You might consider pre/post testing using valid and consistently reliable tests developed by disciplinary experts in education research Hake (2004b,c). As indicated in Hake (2004b), this is becoming more and more popular in astronomy, economics, biology, chemistry, computer science, and engineering. In many cases it has been stimulated by the pre/post testing effort in physics education research, initiated by the landmark work of Halloun & Hestenes (1998a,b).

Richard Hake, Emeritus Professor of Physics, Indiana University 24245 Hatteras Street, Woodland Hills, CA 91367

REFERENCES Davenport, M.A. 2004. "Re: Assessment vs Evaluation," ASSESS post of 13 Aug 2004 12:08:46-0400; online at .

Ehrmann, S. 2004. "Re: Evaluation, Assessment, and Rubrics." POD post of 14 Sep 2004 14:31:48-0700; online at .

Hake, R.R. 2002a. "Lessons from the physics education reform effort," Ecology and Society 5(2): 28; online at . Ecology and Society (formerly Conservation Ecology) is a free "peer-reviewed journal of integrative science and fundamental policy research" with about 11,000 subscribers in about 108 countries.

Hake, R.R. 2002b. "Assessment of Physics Teaching Methods, Proceedings of the UNESCO-ASPEN Workshop on Active Learning in Physics, Univ. of Peradeniya, Sri Lanka, 2-4 Dec. 2002; also online as ref. 29 at .

Hake, R.R. 2004a. "Re: Assessment vs Evaluation," online at . In this post I misinterpreted Mark Davenport's interpretation - he DOES distinguish between assessment and evaluation Davenport (2004).

Hake, R.R. 2004b. "Re: Measuring Content Knowledge," online at Post of 14 Mar 2004 16:29:47-0800 to ASSESS, Chemed-L, EvalTalk, Physhare, Phys-L, PhysLrnR, POD, and STLHE-L.

Hake, R.R. 2004c. "Re: Measuring Content Knowledge," online at . Post of 15 Mar 2004 14:29:59-0800 to ASSESS, EvalTalk, Phys-L, PhysLrnR, and POD.

Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of college physics students." Am. J. Phys. 53:1043-1055; online at . Contains the landmark "Mechanics Diagnostic" test, precursor to the much used "Force Concept Inventory" Hestenes et al. (1992).

Halloun, I. & D. Hestenes. 1985b. "Common sense concepts about motion." Am. J. Phys. 53:1056-1065; online at .

Halloun, I., R.R. Hake, E.P Mosca, D. Hestenes. 1995. Force Concept Inventory (Revised, 1995); online (password protected) at . (Available in English, Spanish, German, Malaysian, Chinese, Finnish, French, Turkish, and Swedish.)

Hestenes, D., M. Wells, & G. Swackhamer, 1992. "Force Concept Inventory." Phys. Teach. 30: 141-158; online (except for the test itself) at . For the 1995 versions see Halloun et al. (1995).

Maki, P. 2004. "Assessing for Learning: Building a Sustainable Commitment Across the Institution." Stylus. Maki is the former Director of Assessment of the AAHE.

Stokes, D. E. 1997. "Pasteur's quadrant: Basic science and technological innovation." Brookings Institution Press.

EVALTALK - American Evaluation Association (AEA) Discussion List. See also

   the website:  http://www.eval.org

To unsubscribe from EVALTALK, send e-mail to listserv@bama.ua.edu

   with only the following in the body: UNSUBSCRIBE EVALTALK

To get a summary of commands, send e-mail to listserv@bama.ua.edu

   with only the following in the body: INFO REFCARD

To use the archives, go to this web site: http://bama.ua.edu/archives/evaltalk.html For other problems, contact a list owner at kbolland@sw.ua.edu or carolyn.sullins@wmich.edu

Abgrenzung von Evaluation zu Assessment

Sat, 14 Aug 2004 12:54:11 +0200

Message-ID:

 Date: Thu, 12 Aug 2004 21:25:36 -0700
 Sender: American Evaluation Association Discussion List 
 From: Richard Hake 
 Subject: Re: Assessment vs Evaluation
 To: EVALTALK@BAMA.UA.EDU

In his ASSESS post of 10 Aug 2004 titled "Assessment vs Evaluation" Mark Davenport wrote:

"I often read in the literature and hear on the conference circuit people using the terms 'assessment' and 'evaluation' interchangeably, as if they were synonyms. Even more confusing, I have found the word assessment is used to define evaluation, and vice versa . . . . Personally, I don't think we need two terms to explain identical concepts (unless they occur in two completely unrelated fields wherein the risk of confusion is minimal). Certainly academic and student affairs assessment are related enough that we can share terms. . . . I have documented my thoughts in an internal white paper to my constitutents and would be happy to share it if you will drop me a note privately."

I hope Mark will place his white paper on the web so as to increase the readership and decrease mailing expenses. His post stimulated a 12-post (as of 12 Aug 2004 16:20:00-0700) ASSESS thread accessible at .

A similar thread (4 posts) titled "distinction between evaluation and assessment was initiated by Jeanne Hubelbank (2003) on EvalTalk and is accessible at the EvalTalk archives . One post in this thread led me to a "Glossary of Program Evaluation Terms" at Western Michigan University (Michael Scriven's new location), where these definitions are given:

Assessment: "The act of determining the standing of an object on some variable of interest, for example, testing students, and reporting scores."

Evaluation: Systematic investigation of the worth or merit of an object; e.g., a program, project, or instructional material.

Nevertheless, I'm with Mark Davenport in preferring to make no distinction between "assessment" and "evaluation." In a post titled "Re: A taxonomy" Hake (2003a), I proposed an assesment taxonomy for consideration and comment that is best presented in quadrant form cf., Stokes (1999):

                      plus Y
                      PUBLIC
                        |
                        |
               Scientific Research
                        |

<--FORMATIVE ASSESSMENT | SUMMATIVE ASSESSMENT --> _ plus X

                       0|
                        |
    Action Research     |  Institutional Research
                        |
                        |
                        |
                      PRIVATE

Fig. 1. Quadrant representation of various types of assessment/evaluation. (Figure may be distorted by email transmission.)

For educational research, the X-axis represents a continuum from pure FORMATIVE to pure SUMMATIVE assessment of either teaching or learning. NO DISTINCTION IS MADE BETWEEN "ASSESSMENT' AND EVALUATION." The Y axis represents a continuum from complete privacy to complete public disclosure of results.

The locations of various types of research in terms of the type of assessment they offer are shown as:

"Scientific Research" see e.g. Shavelson & Towne (2002): upper two quadrants - always public and anywhere in the continuum between formative and summative.

"Action Research" [see e.g. Feldman & Minstrell (2000) and Bransford et al.]: lower left quadrant - usually private to some degree, and usually formative to some degree.

"Institutional Research": lower right quadrant - usually private to some degree, and usually summative to some degree, although it could approach the formative for those who study and attempt to improve institutional practice.

Leamnson's (2003):

(a) "classroom research" can be either "scientific" or "action" research.

(b) "institutional research" is generally NOT formative from the standpoint of classroom teachers.

In my opinion, the science education use of pre/post testing [for reviews see Hake (2002; 2004a,b,c)] is usually formative for both action and scientific research, since the object is to improve classroom teaching and learning, NOT to rate instructors or courses.

Richard Hake, Emeritus Professor of Physics, Indiana University 24245 Hatteras Street, Woodland Hills, CA 91367

REFERENCES Bransford, J.D., A.L. Brown, R.R. Cocking, eds. 2000. How People Learn: Mind, Brain, Experience, and School: Expanded Edition. Nat. Acad. Press; online at , pages 199-200. This is an update of the earlier 1999 edition.

Feldman, A. & J. Minstrell. 2000. "Action research as a research methodology for the study of the teaching and learning of science," in E. Kelly & R. Leash, eds., "Handbook of Research Design in Mathematics and Science Education." Lawrence Erlbaum; online at (72kB).

Hake, R.R. 2002. "Lessons from the physics education reform effort," Ecology and Society 5(2): 28; online at . Ecology and Society (formerly Conservation Ecology) is a free "peer-reviewed journal of integrative science and fundamental policy research" with about 11,000 subscribers in about 108 countries.

Hake, R.R. 2003a. "Re: A taxonomy"; online at . Post of 9 Jul 2003 12:47:42-0700 to STLHE-L, PhysLnrR, EvalTalk, and POD. See also Hake (2003b).

Hake, R.R. 2003b. "Re: A taxonomy"; online at . Post of 12 Jul 2003 13:07:54-0700 to ASSESS, EvalTalk, PhysLrnR, STLHE-L, and POD.

Hake, R.R. 2004a. " Re: Measuring Content Knowledge," online at . Post of 14 Mar 2004 16:29:47 -0800 to ASSESS, Biopi-L, Chemed-L, EvalTalk, Phys-L, PhysLrnR, Physhare, STLHE-L, and POD. See also Hake (2004b).

Hake, R.R. 2004b. "Re: Measuring Content Knowledge," online at . Post of 15 Mar 2004 14:29:59 -0800 to ASSESS, EvalTalk, Phys-L, PhysLrnR, and POD; online at .

Hake, R.R. 2004c. "Design-Based Research: A Primer for Physics Education Researchers," submitted to the "American Journal of Physics" on 10 June 2004; online as reference 34 at , or download directly as a 310kB pdf by clicking on .

Hubelbank, J. 2003. "distinction between evaluation and assessment." EvalTalk post of 13 Nov 2003 10:52:00-0500; online at . The encyclopedic URL indicates that one must subscribe to EvalTalk to access its archives, but it takes only a few minutes to subscribe by following the simple directions at / "Join or leave the list (or change settings)" where "/" means "click on." If you're busy, then subscribe using the "NOMAIL" option under "Miscellaneous." Then, as a subscriber, you may access the archives and/or post messages at any time, while receiving NO MAIL from the list!

Leamnson, R. 2003. "A Taxonomy," STLHE-L/POD post of 9 Jul 2003 10:32:02-0400; online at .

Shavelson, R.J. & L. Towne. 2002. "Scientific Research in Education," National Academy Press; online at .

Stokes, D. E. (1997). "Pasteur's quadrant: Basic science and technological innovation." Brookings Institution Press.

EVALTALK - American Evaluation Association (AEA) Discussion List. See also

  the website:  http://www.eval.org

To unsubscribe from EVALTALK, send e-mail to listserv@bama.ua.edu

  with only the following in the body: UNSUBSCRIBE EVALTALK

To get a summary of commands, send e-mail to listserv@bama.ua.edu

  with only the following in the body: INFO REFCARD

To use the archives, go to this web site: http://bama.ua.edu/archives/evaltalk.html For other problems, contact a list owner at kbolland@sw.ua.edu or carolyn.sullins@wmich.edu