CILT99 Demonstrations

Assessments for Learning

Measuring Students' Environmental Awareness and Skill in Data Analysis and Interpretation: A Web-Based Assessment for the GLOBE Program

Bill Penuel, Elaine Coleman
SRI International
http://www.wise.sri.com/servlet/globe.gat/GetFile/index1. html

In this session, we will describe and demonstrate a web-based assessment task designed to help evaluate the Global Learning and Observations to Benefit the Environment (GLOBE) program, an international environmental science research and environmental education program. The GLOBE involves students, teachers, and scientists from across Earth in collecting, sharing, and analyzing data about Earth's dynamic land, air, water, and biology systems.

The assessment is designed to measure middle- and high-school students' qualitative understanding of ecology or environmental awareness (i.e., the recognition of the environment as a coherent set of interdependent and interconnected adaptive elements), and to measure students' skill in data analysis and interpretation using data of the kind typically collected by students in GLOBE. Because GLOBE students interact with the broader GLOBE community, which includes scientists, other GLOBE teachers and students, and they submit their data via the Internet, we designed a web-based task to measure student performance. The web-based task requires students to (a) identify patterns and sets of relationships between ecological variables within images of a natural scene, and (b) to analyze climate data to decide where to host the next Winter Olympics, based on guidelines given to students for the best conditions under which the games might be hosted. In part b, students must develop an argument in favor of choosing a particular city and use the data to present evidence that supports their choice of site. A pilot conducted at a Bay Area high school implementing GLOBE indicates the assessment's viability for assessing student skill in data analysis and interpretation and for measuring student environmental awareness.

Performance Assessment Links in Science(PALS)

Edys Quellmalz
SRI International
http://pals.sri.com

SRI International has received funding from the National Science Foundation to develop a greatly needed, specialized type of electronic library-an on-line, standards-based, interactive resource bank of science performance assessments, Performance Assessment Links in Science (PALS). States, districts, and teachers are seeking alternative assessments appropriate for measuring achievement of their science reform goals. Experience indicates, however, that the level of effort and costs of developing performance assessment tasks, scoring rubrics, and rate training materials, and then ensuring their technical quality are very high. The PALS Web site provides an innovative approach for sharing exemplary assessment resources, collaborating on the development.of new ones, and understanding how the use of standards-based performance assessment can advance educational reform at all levels of the educational system.

PALS on-line assessment resources are designed to serve two purposes and user groups: (1) the professional development needs of classroom teachers, and (2) the accountability requirements of state education agencies, districts, and specially-funded programs. Our partners include the Council of Chief State School Officers (CCSSO), three states (Connecticut, Illinois, and Kentucky) and two assessment consortia (the CCSSO State Collaborative on Assessment of Students and Standards (SCASS) for science and the California Systemic Initiative Assessment Consortium (CSIAC). The PALS project has two primary goals. (1) To develop a two-tiered on-line performance assessment resource library composed of performance assessment tasks for elementary, middle and secondary levels from multiple sources such as state assessment programs and consortia and national reference exams (NAEP, TIMSS, New Standards). One tier will be for use by teachers and professional development organizations. The second tier will be a password-protected, secure Accountability Pool of science performance assessments for use by state assessment programs and systemic reform programs (e.g., Systemic Initiatives). The Professional Development Pool will provide performance-based science assessments that have been used successfully in large scale (state or national) assessment programs and that have been released. The Accountability Pool will be composed of science assessments that have undergone a test development process and with data documenting their technical quality. (2) To identify, study, and evaluate the effectiveness of policies, implementation models, and technical quality requirements for the use of the two tiers of PALS. The resources included on the PALS Web site consist of science performance tasks, student directions and response forms, administration procedures, scoring rubrics, examples of student work, and technical quality data calculated from field testing, and training packets for conducting on-line rater training. Performance tasks are compared for their alignment with the National Science Education Standards (NSES). In the second and third years of the project, SRI will embed on-line collaborations on science assessment within its virtual professional development center, TAPPED IN. We encourage organizations to use any of the PALS resources in a variety of ways and share their experiences with us.

Technology Transfer To the Community

Ron Stevens
UCLA
http://www.immex.ucla.edu

Over the last decade, the IMMEX Project at the UCLA School of Medicine has moved from software designed exclusively for future doctors to become a flexible instructional and assessment system for both secondary and college level courses. To date, the system has been used in over 70 schools in the Los Angeles area and involved over 300 teachers in training opportunities. Over 100 problem sets have been created by teams of high school, middle school, or elementary school teachers along with academic partners; many of these have been validated by teachers, university faculty, and now are used extensively in the classroom..IMMEX can best be described as software that allows teachers to create "snapshots" of complex real-life situations that require students to make important investigative decisions. IMMEX also gathers information on the processes students use in making those decisions, giving teachers insight into the more subtle aspects of student thinking and learning. These insights help individualize education and provide valuable information on how the curriculum can be reformed. Three important principles have guided the evolution of the IMMEX Project over the past decade. First, systemic educational reform - and the true realization of the potential of technology in the classroom - will not occur until teachers are confident and committed to the use of these technologies to enhance classroom practices. Second, educational technologies and software cannot be crafted for a single class, a single grade level, or a single school but must be part of a broader design that articulates curricular content and processes across disciplines and levels of ability. Third, the need for in-depth assessment of student performance, curriculum effectiveness, and teaching practices require that evaluation is a central component of all educational software development and implementation.

In this demonstration, Dr. Ron Stevens will present selected problems, provide details of the professional development activities and outcomes, and discuss how teachers are using the evaluation components of IMMEX in the classroom.