Educational Engineering

Content Insensitive Tools

small logo

Student Behavior Modeling

One of the most frustrating things about being a physics instructor is that the student's performance on tests is often unintelligible from the assignments and material presented. Instructors often end up feeling there is no way a student should have missed that. Students are very complicated. Our research shows only 26% of the performance of a student in a class is predictable knowing their pre-preparation entering the class and the amount of time spent on the class. The graph to the left shows the test score plotted against our best model of the set score using 15 behavioral parameters. It is still amazingly noisy.

Content Insensitive Measurements

There are three types of class evaluation tools in broad use in science classes today; (1) Instructor written exams, (2) Standardized tests and concept inventories, (3) Course evaluations and other attitude surveys. All of these have drawbacks that prevent a unbiased picture of the class emerging. Instructor written exams test to some extent the instructors skill in writing exams that match the class coverage. Standardized exams may have little relation to class coverage. Course evaluations and attitude surveys are sensitive to how personable the instructor is and what grade the student expects to receive in the class. We propose the use of a fourth class of evaluation that does not suffer from these drawbacks to supplement the evaluations already in use, content insensitive quantitative measurements. Measurements that do not depend on course content and that can be measured quantitatively without the intrusion of instructor opinion. The papers below investigate how a student's use of time affects performance, how classes and textbooks are integrated through references, how time use may be measured accurately, and what the language structure of a science course can tell us.

Student Behavior Measurement An extremely detailed measurement of the kinds of things students do to address a science class, how much time they spend on each behavior and how these decisions impact test scores and concept mastery. This measurement was performed by Jennifer McGee for her honors thesis and is being re-written for submission as a paper. A short version of the research is available from the talk given at the Denver APS meeting.

Measuring Student Time Use The behavioral modeling paper uses a method of time collection established by Steven Sandh is this paper.

Measuring Class Integration Course materials are investigate for the student of references within a material and between segments of the class and the material. This gives a picture of how often material is re-used in a class.

Measuring Class Language Content The complexity of word use has been used to measure reading level in elementary education for years. Science writing has additional features, math and graphics. Measuring the distribution of words, symbols, and drawing gives a content insensitive picture of class function. This is a preliminary work that is being taken to publication by a honor student in the fall.

Educational Effect of Language Content Measures the effect of student language use on student preformance.

 

About Us | Contact Us | ©2003 APEG