Unified web-based system for the delivery, scoring, and reporting of on-line and paper-based assessments

Abstract

A hardware and software implemented, standardize testing, scoring and reporting system has a network interface, being either LAN or the Internet. A test is selected from a library, assigned to a group of examinees, and a record is made of the assignment. The test is available on-line, or downloaded, or both. Downloaded tests are saved as PDF files and printed into booklets. Examinees record answers on bubble sheets, which are scanned at the customer site or shipped to a processing plant. Individual scanned answer sheets provided data for individual scanned records. Scanning software performs an initial auto-correct and then performs an initial check for errors in scanned records. Tests are batched for later inspection. Scanned records with errors are classified as invalid and the test papers are separated. Prompts are generated for a human test paper inspection and scanned record editing. Accepted batches of scanned records are sent to a scanned records server where a second scanned records error check is performed. Prompts are sent to the appropriate scanning station for further human editing of invalid scanned records. Once all records are valid they are scored and the test results stored in a database, which is accessed by a reporting subsystem associated with an on-line server.

Description

BACKGROUND OF THE INVENTION [0001] The present invention is directed to a system and a method for selecting, delivering, conducting, editing, scoring, and reporting on standardized achievement tests. [0002] The automation of test scoring is a complex problem that has been of interest for many years. There is considerable pressure to optimize the efficiency, accuracy, speed and the repetitiveness and therefore the reliability of such test scoring, data accumulation and reporting. Of primary interest has been the scoring of multiple choice answer sheets. Of further interest has been the recording and reporting of test results. [0003] Beginning in about the late 1960's, technology has been developed to machine score optically-scanned answer documents. Hardware has improved throughout the years. However, the basic testing approach has remained reasonably constant. Students / examinees respond to multiple choice questions by completely filling in “bubbles” on a machine scannable answer she...

AI Technical Summary

Benefits of technology

[0032] A third objective of the present invention is to digitize the test responses into test records regardless of source, and thereafter subject each test response to a validation process and a scoring process wherein the processing time of these processes is significantly reduced.

[0033] A fourth objective of the present invention is to detect errors in test records and to significantly reduce the processing time for editing test records containing errors or containing issues to be resolved.

[0041] During the same initial scan operation the software also checks for a first phase of errors using a format algorithm for the bubble answer sheet with the validation information. This algorithm is specific to the test, group, control level etc. First phase errors can include a wrong timing mark, missing identification, miss-entered examinee information and “double” bubbled answers (multiple bubbled). A bubble sheet with phase one scanning errors is selected out into a rejection pile after it is scanned, thereby providing for ease of locating a rejected physical bubble sheet which resulted in an invalid record, and for the visual reading of the rejected bubble sheet by a human editor. Bubble sheets which were cocked or misaligned when first scanned are re-scanned. Other errors are edited at the editor's workstation.

Problems solved by technology

The automation of test scoring is a complex problem that has been of interest for many years.

There is considerable pressure to optimize the efficiency, accuracy, speed and the repetitiveness and therefore the reliability of such test scoring, data accumulation and reporting.

Among these are a failure of a student / examinee to enter identification data properly or to leave out identification data, such a full name, school identification, class identification, teacher name, date, and other such data and / or to miss spell any of these.

Moreover, with large numbers of rows and columns of “bubbles” for answer selection, a student / examinee may miss-apply an answer, or fill-in more than one choice, or even incompletely fill-in a bubble to the point where the OMR / NIR equipment misses an answer where the student made an insufficient mark where he intended it to be an answer.

In the mechanized scoring of standardize tests in the past, non-academic errors (i.e., those other than answering questions) would have rendered the test sheet unreadable and would have either voided the student's performance or required the scoring organization to hand search for the student's test paper and then hand grade the test, or to force the student to retake the exam.

A latent problem with machine testing is the unavailability of sufficient numbers of workstations so that each student / examinee has a workstation available at the same time.

In the educational environment, where a school district administers standardized tests to large numbers of students at the same time, on-line testing becomes reasonably impractical.

This ETS system cannot be interfaced with the internet to operate in another mode of testing.

While these prior developments have advanced the art of automated test construction, automated test administering, automated test question development, computerized bubble answer scoring, and computer-aided human essay scorer performance, these prior developments have not, in whole or in combination, addressed a unified web-based system for the delivery, scoring, and the reporting on-line, of on-line and paper based assessments.

With the exception of the previously used on-line, real-time testing and scoring, the prior developments have not addressed significantly increasing the speed of scoring and reporting test results, whereby very large numbers of tests can be scored and reported on in very short periods of time with minimal human intervention.

Images