Title: Establishing a shared set of dimensions and criteria for analyzing online discussions
PI: Alex Cuthbert, University of California, Berkeley
Other collaborating institutions: The Odyssey; WestEd; University of Washington; Vanderbilt University; Indiana University
This seed grant seeks to develop methods for helping teachers and students effectively use and assess learning in an asynchronous discussion system. Students need to be guided as they develop an understanding of how to participate in a discussion, provide constructive comments, develop shared criteria, and select different representations and comment types. Similarly, teachers need to be able to generate engaging and productive topics as well as assess student contributions to those discussions. However, for these discussions to be used for assessment purposes, students and teachers need to develop shared criteria or at least compatible orientations for what qualifies as a productive discussion.
The outcome of this research will be to establish a shared set of dimensions and criteria for analyzing online discussions. To achieve this goal, we will evaluate online discussions from a range of environments from 8th grade science classrooms to undergraduate seminars. Expert critics will be recruited to share their experiences and evaluate the progress of the project.
In the process of developing criteria for analyzing online discussions, we will assemble a library of resources consisting of annotated Internet sites and discussions between the project collaborators. These resources will be made available to the CILT community and should inform the design of the next generation of innovative online discussion systems.
Project website: http://wise-discuss.berkeley.edu/coolsystem/devhtml/ciltdsc
Title: VisuAlliance - Integration and innovative assessment of dynamic visualization components in the classroom
PI: Eric Hilfer, TERC
Other collaborating institutions: Logal Software, Inc.; SRI International
(co-sponsored by the Visualization and Modeling Theme Team)
The Visualization and Modeling Theme Team at the CILT99 conference identified two vital directions of inquiry that needed attention and research: The integration of existing visualization and simulation tools into a flexible delivery environment, and Updating assessment and evaluation methods to capture the ways these innovative environments influence and contribute to learning. We propose to form a partnership to investigate the integration of two powerful data visualization systems, TERC's VideoScape™ linked video and measurement tools and Logal® Software's SimPlayer™ component-based Web delivery visualization environment. We further propose to work with SRI International to build upon their current work in developing new methods of assessment and evaluation of learning with innovative visualization and simulation. We would use the Seed Grant to fund the development of a preliminary standard specification for a visualization integration platform, and use VideoScape and SimPlayer in particular as a test case for demonstrating integration. Evaluation and assessment models will be concurrently developed to address all of the features supported by the integration standard. These assessment models will be as media-rich and flexible as the learning environment itself. The results of these studies will then be used as major components of a larger federal grant application.
Project website: http://projects.terc.edu/VisuAlliance
Title: Technology Supports for Science Instruction and Assessing Student Work
PI: Edys Quellmalz, SRI International
Other collaborating institutions: Northwestern University; Edmark
This project addresses two goals: (1) to identify and synthesize some promising examples of technology supports from science learning that could be generalized and repurposed to improve the assessment of student learning as well, and (2) to extract, tune, and pilot test one set of such tools to assess student science achievements in classrooms that are currently participating in the Learning Technologies in Urban Schools (LeTUS) project.
The project grows out of the perceived needs by participants in the CILT Assessment Theme Team discussions for a review of existing technology tools that are being used in technology-supported science curricula. One goal is to identify several of these tools be used to extend the targets and forms of student assessments. Stimulated by the presentation on 21st Century Assessment, the study team members found common cause in the potential applicability of such re-purposed technology tools for assessments in science programs in general and for projects on which the members are working specifically.
Title: Assessing Teacher Readiness for Constructivist Technology-based Projects
PI: Jason Ravitz, University of California, Irvine
Other collaborating institutions: Center for Children and Technology
This project will collect and syntheize survey questions and findings from current research efforts in order to provide a tool for assessig school and teacher "readiness" for constructivist technology-based projects. Our work will benefit from access to current findings from several large-scale studies. We seek opportunities to interact with other investigators, to disseminate and test our ideas, and to gain access to immediately useful materials. The tool to be produced will be offered for use in the selection and analysis of study sites and as a diagnostic tool for use by teachers and schools.
Each participant will have the opportunity to share their work and to entertain discussion of their ideas and efforts. An initial list of "key issues" will be developed based on a review of existing efforts and synthesis of agreed-upon principles. We plan to finalize a list of constructs for measurement by the end of September and to develop an instrument that builds on existing surveys by Thanksgiving.
This project will increase knowledge concerning key issues in technology-based reform, and provide insights into the success of constructivist technology-based projects, improving the effectiveness of research and development efforts aimed at supporting student learning through the use of technology.
The survey instrument produced on this project is available at http://www.gse.uci.edu/Ravitz/cilt_project/overview.html
Title: Establishing "Facet" Knowledge of River Ecosystems Concepts
PI: Jim Minstrell, Talaria Inc. and University of Washington
Other collaborating institutions: Vanderbilt University
Assessment aligned with national, state, and/or local goals is becoming increasingly important. To optimize learning, teachers need frequent opportunities to interact with data about students' learning, and students need opportunities for feedback about their understanding. Formative assessmenst embedded within instruction can provide teachers and students with that feedback.
The concept of "knowledge facets" is a way of characterizing students' thinking in subject matter contexts. It allows us to identify and rank the sorts of thinking and actions exhibited by students on their way to the learning goals. This facet approach to categorizing domain knowledge is the foundation of a program called Diagnoser. Diagnoser has been used to help assess and instruct students in learning physics and geometry. In conjunction with CILT's Synergy project, we (Vanderbilt and Talaria Inc.) plan to create a set of facets for life science, specifically related to river ecosystems. We will also collaborate with the University of Washington and the state of Washington on making formative assessment items and associated facet diagnosis and feedback available to science teachers over the web.
Title: Exploring an Automated Method for Assessment of Middle School Problem Solving
PI: Ron Stevens, UCLA
Other collaborating institutions: Vanderbilt University
The purpose of this project is to explore the capabilities of using the IMMEX problem solving program for assessing middle schools students' strategies when solving Jasper Adventures. The IMMEX system provides an automated process for tracking and analyzing the learner's problem solving process. The system was originally designed to assess immunology students' ability to diagnose patients' pathology, and uses a neural network to analyze their performance compared to experts. The Adventures of Jasper Woodbury problem solving series contains a collection of challenges designed to help students in grades 5 and up improve their mathematical thinking. The series consists of 12 challenges targeting concepts in trip planning, statistics, geometry and algebra.
We intend to implement one of the Jasper projects using the IMMEX framework as a design experiment to evaluate the potential of IMMEX as an automated formative assessment device. Ron Stevens of IMMEX at UCLA has made the authoring environment available to Vanderbilt for this evaluation. The IMMEX team will provide consultation to help with the implementation and evaluation of the design. We intend to create a functional prototype that could be used in schools for evaluation in Middle school mathematics and science. The desired outcome of this project is to create a computer-supported assessment environment that helps students monitor and correct their own problem solving process. Both the IMMEX and Jasper Series demonstrate great potential, and their combination can lead to advances in how we approach classroom based assessment in middle school.