2000 Seed Grants

PI: Daniel Hickey, University of Georgia, Atlanta

Other collaborating institutions: Georgia Tech; Georgia State University; Ogden County Schools; Virginia Tech; King's College; NASA Classroom of the Future; WestEd; SRI International; University of Washington

This project is using diverse research and insights to define and validate ways to assess and promote collaboration. The value of this effort is illustrated by the many educational standards documents that highlight the value of the ability to work with others (e.g., Kendall & Marzano, 1997). Unfortunately, collaboration is an elusive construct that means different things to different people. New methods are needed for assessing collaboration because the current accountability oriented climate minimizes the value of educational outcomes that cannot be readily measured and communicated to diverse stakeholders.

This effort builds on the PI's prior attempts to evaluate the impact of GA Tech's Learning by Design (LBD) middle school science curriculum. In one part of this evaluation, over 150 teams in LBD and comparison classrooms were videotaped while collaboratively completing a performance assessment obtained from the PALS (2001) Web site. These videotapes were scored using nine dimensions of collaboration advanced in prior research by Pomplum (1996). These scores, along with the scores of the students' performance assessments, were analyzed and reported as evidence of the effectiveness of the LBD curriculum (Hickey, 1999; 2000).

We extended and refined this assessment practice with the support of the CILT seed grant and the other sources listed above. Specifically, we are defined and validated a new set of dimensions, new anchoring descriptions, and a new rubric format for assessing participation in collaboration, and attempting to use these to help learners evaluate and improve their own collaboration.

Title: Tracking the Fate of Evidence and Ideas Among Collaborative Groups in Technology-Supported Contexts

PI: Richard Duschl, King's College, London

Other collaborating institutions: University of Oulu, Finland; Virginia Tech; University of Georgia; SRI International; Kingsbury High School, England

This project is building on the broader research program of Project SEPIA (Science Education through Portfolio Instruction and Assessment) (Duschl & Gitomer, 1997), which is concerned with understanding ways to improve discourse processes and formative assessment practices in science classrooms. The present research focus is on argumentation discourse in science classrooms and how argumentation discourse itself and formative assessment strategies around such discourse can be carried out in science classrooms that use a combination of face-to-face small group activities with computer-supported small group activities. The goal of the present research is to obtain a better understanding of the small group dynamics in these two conversational contexts. More exactly, we hope to better understand how to leverage students' argumentation skills.

Data collection is proceeding on two fronts. First, under the direction of Dr. Richard Dusch,l we are audiotaping the face-to-face small group conversations of all 5 student groups in Dr. Bateman's class. We are also electronically storing the Knowledge Forum files of both individual and group discourse. We have begun a first level of analysis that is identifying students' perceptions of core concepts for EHH and the arguments students are posing for good and accurate data. This initial analysis will form the baseline for tracking the fate of ideas during the unit.

The second data collection process is being coordinated by Dr. Sanna Jarvela. She is videotaping group interactions and presently has three sets of videotapes. Together, we have identified two groups of students to be the target groups for her study of the motivational issues operating in small group dynamics. Independently, Duschl and Jarvela are rating students' level and quality of activity for each lesson. The compiled data will be used to select students for follow-up structured interviews.

Title: Preliminary Planning for Holding a Special Conference on Teaching with Classroom Communication Systems

PI: Louis Abrahamson, Better Education Inc.

Other collaborating institutions: SRI International; Vanderbilt University; University of Massachusetts

The objective of this proposal is to do preliminary planning for holding a special conference devoted exclusively to teaching with Classroom Communication System (CCS) technology and associated issues of pedagogy, teacher-training, tools, curriculum, and cognitive science. When used with appropriate pedagogy and curriculum, CCSs have shown significant potential to increase the learner-centeredness, knowledge-centeredness, assessment-centeredness, and community centeredness in widely disparate educational environments. This proposed conference would be extremely timely. For many years CCS research and practical implementation was limited by hardwired, relatively costly systems with restricted functionality. Now with several manufacturers, wireless systems, increasing ranges of functionality, and competitive pricing, the opportunities for exciting growth are wide open. The Draft Project Description has been reviewed by NSF. It received favorable comments and we were requested to go ahead with its submission under the NSF SGER program.

Title: QIICC Analysis: Developing New Dimensions of Student Feedback for Online Testing: Increasing Student Confidence and Meaningfulness of Items

PI: Dwight Allen, Old Dominion University

Other collaborating institutions: Brunswick County Schools; University of Missouri, St. Louis

The QIICC Analysis feature of the Telequiz Assessment Protocol is a revolutionary method for analyzing student learning. The Telequiz Protocol has been used in Dr. Dwight W. Allen's Educational Curriculum and Instruction courses at Old Dominion University (ODU) since 1997. The protocol allows students to take quizzes over the Internet at their convenience. These quizzes are comprised of randomly assigned multiple-choice questions based on the content of the course. The QIICC Analysis feature allows for students to rate individual questions on several variables: Quality, Interest, Importance, Confidence, and Challenge.

The inclusion of the QIICC Protocol into Telequiz has been a complicated process. It involved taking a proven product that was already being upgraded, and engineer it even further on the fly. Many of the larger technical issues were worked out through testing in Dr. Allen's ECI classes this past spring, and now that the protocol is functioning properly, it is being implemented full time in Dr. Allen's courses at ODU. The CILT Grant allowed us to test QIICC Analysis in 3rd grade and 11th grade classrooms in Brunswick County, Virginia, and in 5th and 6th grade classrooms in St. Louis, Missouri. Brunswick County will continue to work with us in the further development of QIICC Analysis even without additional funding. St. Louis will not be able to continue to work with us unless we receive additional funding. Analysis of the QIICC protocol in all three settings shows that there are promising benefits from QIICC Analysis.

Now that the quizzing protocol has been tested at each of the sites successfully, and the results have demonstrated the strength of the QIICC scoring protocol, the next step is to redesign the Telequiz protocol in order to make it more user-friendly. The rigid structure, and inability to design quizzes over the Internet, have made it necessary for one person to design the quizzes centrally, while they are administered at a distance. Improvements that have been suggested include a more fluid question structure ' including improved randomization, incorporating the ability to design a quiz through the Internet, and incorporating the ability to analyze student quiz data over the Internet. These improvements would make the protocol much more useful for individual teachers by giving them greater control over the quiz design and administration process.