The Outcomes and Impacts of Adult Literacy Education in the United States
Rutgers University, Graduate School of Education
NCSALL REPORT #6
This study investigated the outcomes and impacts of adult literacy education through a qualitative assessment of the outcomes and impacts research conducted since the late 1960s. Outcomes are the changes in learners that occur as a result of their participation in adult literacy education. Impacts are the changes that occur in the family, community, and larger society as a consequence of participation.
The goals of the study were to make reasoned inferences about the effectiveness of adult literacy education in the United States; to identify common conceptual, design, and methodological problems inherent in the outcome studies conducted; to raise and discuss issues for policy; and to make recommendations for the design and conduct of future outcomes studies.
Through a comprehensive literature search, approximately 115 outcomes and impacts studies were identified. All were obtained in either hard copy or microfiche and the 68 of those that were found to have an outcomes component were abstracted. Each study was then evaluated according to the following criteria: the study included an outcome/impact component; the report was adequately documented with respect to design and methods; there was an adequate number of cases; the sampling plan was adequate; data collection procedures were adequate (i.e., were not tainted by substantial attrition or biased by other factors); objective measures, rather than self-report, were used to measure outcomes; measures, especially tests, were valid and reliable; the research design included a control or comparison group; and inferences logically followed from the design and data.
Based on this evaluation, 23 studies were selected as being the most credible, and case studies were prepared for each. Studies are presented in five categories: national, state-level, welfare, family literacy, and workplace literacy. From the 23 studies, inferences about program effectiveness are made.
The 23 case studies represented evidence rather than proof of impact, and, like evidence in a trial, their findings were weighed in order to reach reasonable conclusions. Weighing had two dimensions. The first was the extent to which the various studies converged or diverged in respect to their findings on specific outcome/impact variables. Consensus across studies pointed toward effectiveness/ineffectiveness, while lack of consensus suggested an inconclusive resolution. The second dimension was the credibility of the individual studies. When arriving at conclusions, more credible studies were weighed more heavily than less credible studies.
The conclusions set forth are deemed to be reasonable inferences from the findings reported in the case studies. They do not represent proof. Indeed, it is unlikely that any conceivable study or studies could arrive at certainty.
In interpreting our conclusions, three caveats are in order. First, the variables included are those studied by a sufficiently large number of studies to enable reasonable conclusions. However, variable definitions and their units of measure vary among studies. In some studies, for example, learning gain is measured by the CASAS, while in others the TALS or TABE are used. Second, if a given study reported a gain, the gain is considered as positive irrespective of the size of the gain or the quality of the study is methodology. In some cases the gains reported as positive are quite small, and in some cases the limitations of the study render claims of gains suspect. Third, the totals are aggregates of studies conducted at different times and on different populations of adult literacy learners, welfare clients and employees being examples. Drawing conclusions from such aggregates presumes that doing so is both valid and meaningful.
From the case studies, the following conclusions were made about the effectiveness of the adult literacy education program in the United States:
- In general, it is likely that participants in adult literacy education receive gains in employment.
- In general, participants in adult literacy education believe their jobs improve over time. However, there is insufficient evidence to conclude that participation in adult literacy education causes job improvement.
- In general, it is likely that participation in adult literacy education results in earnings gain.
- In general, adult literacy education has a positive influence on participants' continued education.
- Although the evidence suggests that participants in welfare-sponsored (e.g. JOBS Program) adult literacy education do experience a reduction in welfare dependence, the evidence is inconclusive as to whether adult literacy education in general reduces welfare dependence for participants.
- Learners perceive that participation in adult literacy education improves their skills in reading, writing, and mathematics.
- As measured by tests, the evidence is insufficient to determine whether or not participants in adult literacy education gain in basic skills.
- In general, adult literacy education provides gains in GED acquisition for participants entering at the adult secondary (ASE) level.
- Participation in adult literacy has a positive impact on learners' self-image.
- According to learners' self-reports, participation in adult literacy education has a positive impact on parents' involvement in their caldrons' education.
- Learners perceive that their personal goals are achieved through participation in adult literacy education.
In the final chapter, conceptual, design, and methodological problems inherent in the studies are discussed and implications for policy are presented, including recommendations for:
- relevant and measurable outcome standards and a feasible impact reporting system;
- a comprehensive national longitudinal evaluation measuring long-term impact; and
- systematic funding and improvement of state and local outcome studies.
Full Report Available
To receive a copy of the full report, send a request indicating the report number or title
along with a check or money order in the amount of $10, payable to World Education, to:
44 Farnsworth Street
Boston, MA 02210-122
or call: (617) 482-9485