The Devil is in the Details
Evidence from the GED on the Role of Examination System Details in Determining Who Passes
John H. Tyler, Brown University
Richard J. Murnane, Harvard University Graduate School of Education and NBER
John B. Willett, Harvard University Graduate School of Education
NCSALL REPORT #16
As part of standards-based educational reform efforts, more than 40 states will soon require students to achieve passing scores on standardized exams in order to obtain a high school diploma. Currently, many states struggle with the design of their examination systems, debating such questions as the number of subjects to be tested and the rules regarding opportunities for students to re-take the examinations. There is, however, little systematic information on the impact alternative options would have on the number of students who will meet the standards for a high school diploma. Nor is much known about whether particular design options will have greater impacts on some students, such as students of color, than on other students.
In this paper we use data from a long-standing examination system, the General Educational Development (GED) certificate, to illustrate that the details of examination systems have marked impacts not only on the number of test takers who obtain the desired credential, but also on the racial/ethnic composition of passers. While the examination systems currently in place or being studied in K-12 systems across the nation are relatively new, the GED testing system has been in place since the 1940s. The stability and acceptance of this national examination system make it a unique and important vehicle for studying how the details of an examination system matter in who and how many ultimately pass.
In this study we use detailed data provided by the Florida Department of Education that contain basic demographic and test score information on 189,124 GED candidates who took the GED exams between 1988 and 1998. In these data we have information on the test scores of the first and the most recent tests in all five GED subject areas, allowing us a unique view of the testing history of these GED candidates.
Using these data, several key findings emerge. First we find that initial pass rates on the GED exams vary greatly by race/ethnicity and by age within race/ethnicity. About 77 percent of white GED candidates (of all ages) pass on the first attempt, while only 66 and 46 percent of Hispanic and black (respectively) candidates pass on their first attempt. Among white and Hispanic candidates, the lowest initial pass rates occur among candidates who are age 19-21 years when they take the tests, and the highest pass rates are for candidates who are thirty and older. However, among black GED candidates, the highest pass rates are among the youngest candidates, those who were 16-18 when they took the tests, and the lowest initial pass rates were among those who were age thirty and over. We found no substantial gender differences in initial pass rates.
In the GED system, candidates who fail the GED exams may retake the tests. (Different states have different guidelines regarding the timing of test retakes.) We find that about 65 percent of whites who fail on their initial attempt, retake the exams within three years, while about 60 percent of blacks and Hispanics retake the test. For all groups, younger candidates tend to retake the exams at higher rates than older candidates. Particularly notable is that while black GED candidates had initial pass rates that were 30 percentage points lower than white initial pass rates, there is only about a ten percentage point difference in the pass rates among retakers (71 percent for whites versus 60 percent for blacks). Also, Hispanics who retake the exams have pass rates that equal those of the white retakers. The ultimate pass rates in our data for all GED candidates is 88 percent for whites, 66 percent for blacks, and 80 percent for Hispanics. Again, we find no substantial gender differences in ultimate pass rates.
Our examinations of the time between test attempts show that the modal time between the first and the last attempt to be about two months. (About 20 percent of those who retake do so within two months.) It has been suggested to us by GED officials in Florida that it takes about two months for GED exams to be scored, recorded, and the results to be sent out. This suggests that many GED candidates in Florida who fail on their first attempt may be retaking the test with little additional preparation. Our results are only suggestive, however, and more work with more appropriate data is needed to determine how initial failure on the exams may affect further human capital formation among GED candidates.
We also find that among GED candidates who failed to obtain the credential, the writing test was the lowest score for the highest percentage of males, and the math test was the lowest score for the highest percentage of failing females. Slightly higher percentages of Hispanics had lowest scores on the reading portion of the test, relative to white and black candidates.
It is clear from this work that the ability to retake the GED exams is an important feature for a non-trivial portion of GED candidates. It is also clear that the ability to retake is relatively more important for black and Hispanic candidates than it is for white candidates. Without the ability to retake, and assuming no changes in preparation or the pool of testers, the black-white pass rate gap would be over 30 percentage points instead of 22 points. This suggests that providing individuals the opportunity to retry upon failing is an important provision if one goal of an examination system is diversity in the pool of individuals who “pass.”
The relatively high percentage of candidates who fail the exams and then retake shortly thereafter is a bit troubling. It is at least suggestive that for some portion of GED candidates, little human capital is being built between attempts. We do note, however, that our findings show that about 80 percent of those who fail on first attempt take longer than two months before they retake the tests, and there may be substantial human capital development for these individuals. The policy recommendations from these particular results are not clear. During the period covered by our study, there was no time constraint on time-to-retest for the second attempt in Florida. Requiring a six-month waiting period between first and second attempt, as some states do, might induce more human capital formation among individuals whose skills prevented them from passing on first attempt. However, such a policy might also be a hardship for individuals who need the GED for timely admission to training, jobs, or post-secondary education.
The age differences on initial pass rates suggest that GED preparation programs are most important for young individuals who have been out of school for two to four years and for black candidates who are age thirty or over. What is not clear from our research are the reasons why individuals who were 19-21 when they initially took the tests consistently passed at lower rates than individuals who were 16-18 when they took the tests. It could be that the older candidate group is composed of individuals who leave school with lower levels of skills than individuals who tend to test sooner after leaving school. Or, it could be that the two groups left school with the same types of skills, but that the skills of the 19-21 year-old group have eroded relative to the skills of the younger group. Understanding these mechanisms is important, because, given the answer, different types of GED preparation programs may be more or less effective in helping students of different ages pass the exams.
Finally, our finding that the math test posed the most difficult barrier for female candidates, while the writing test was the primary barrier to a GED for males, replicates gender differences that have been found in other settings. This finding should reinforce to GED teachers, however, the importance of targeted instruction as they attempt to help individuals obtain the GED credential. It also points out that careful consideration should be given as to how different subject areas might be weighted in any given examination system.
Overall, our research points to the importance that should be given to the details that are associated with high stakes testing systems currently being planned or implemented by many state K-12 systems. Specifically, our examination of the GED examination system in Florida indicates that decisions regarding retesting opportunities and guidelines, as well as the weights that will be given to different subject areas, will have implications for both aggregate pass rates and the composition of those who successfully meet the standard.
Full Report Available
To receive a copy of the full report, send a request indicating the report number or title
along with a check or money order in the amount of $10, payable to World Education, to:
44 Farnsworth Street
Boston, MA 02210
or call: (617) 482-9485