NCSALL

This page is located at: http://www.ncsall.net/?id=352

A User's Guide to Standards-Based Educational Reform: From Theory to Practice

A User's Guide to Standards-Based Educational Reform:  From Theory to Practice


by Regie Stites
Standards have been one of the hottest topics in education reform for more than a decade. The drumbeat has been fed by fears that American kids —the future American workforce —are not keeping up with their peers in Western Europe and Japan. Such worries were given wide currency with the publication of the book "A Nation At Risk" in 1983. "A Nation At Risk" struck a responsive chord with the public. In the late 1980s, then-Governor Bill Clinton and his colleagues in the National Governor's Association began to see national goals and standards as the mechanism they needed to speed educational reform, a priority with voters. At a meeting with President Bush in 1989, the governors announced National Education Goals as the centerpiece of the America 2000 educational reform agenda. The US Department of Education adopted the America 2000 goals as policy when Clinton took office as President. Adult literacy and lifelong learning are addressed in Goal 6: "By the year 2000, every adult American will be literate and will possess the knowledge and skills necessary to compete in a global economy and exercise the rights and responsibilities of citizenship." The National Education Goals Panel (NEGP) in its 1994 Goals Report appended an "indicator" for the adult literacy goal as follows: "Increase the percentage of adults age 16 and over who score at or above or above Level 3 in prose literacy on the National Adult Literacy Survey (NALS)."

The ideal model of standards-based reform described in this article will provide some context for understanding why an indicator such as this one is needed — in theory — to make Goal 6 and the standards connected to it work. It will also provide some perspective on why — in practice — this particular indicator is problematic.

In Theory
In American educational policy discussions, three general types of educational standards are usually defined: content standards, performance standards, and opportunity-to-learn (OTL) standards (NCEST, 1992; National Academy of Education, 1993; Husen & Tuijnman, 1994). Each type has an indispensable part to play in the ideal model of standards-based reform.

According to the 1992 report by the National Council on Educational Standards and Testing (NCEST, 1992, p. 9), content standards define "everything a student should know and be able to do." In other words, content standards describe the range of desirable knowledge and skills within a subject area. Content standards for history, for example, specify the people, events, and ideas that should be included in the history curriculum (and texts) at each grade level. The content standards for adult literacy being developed through the Equipped for the Future (EFF) Initiative (see page 11) define a set of knowledge and skills needed for competent adult performance in the roles of worker, community member, and parent or family member.

The NCEST report (1992) defined performance standards as specifications of "how much" students should know and be able to do. Thus, while content standards shape what goes into a curriculum, performance standards set benchmarks — specified levels of achievement — that shape expectations for educational outcomes, provide a basis for measuring learning outcomes, and provide the criteria for imposing rewards and sanctions. Performance standards for mathematics, for example, specify the mathematical operations and concepts that should be mastered at each grade level as well as the types of assessments that should be used to measure that mastery. The National Council of Teachers of Mathematics (NCTM) took an interesting detour from the ideal model for standards development to develop a set of Assessment Standards for School Mathematics (NCTM, 1995). These standards provide guidance for teachers in the selection or design of tests to measure student progress in the knowledge and skills defined in NCTM's curriculum (content) standards.

Examples of performance standards for adult literacy are hard to find. The skills and competencies defined by the US Department of Labor Secretary's Commission on Achieving Necessary Skills (US DOL, 1991) might fit the bill for some, but SCANS definitions of skills and competencies are not detailed and specific enough to perform the function of performance standards in an accountability system. The descriptions of student performance levels being developed by the US Department of Education's National Reporting System (NRS - see box on page 4) as well as by individual states to meet the requirements of 1998 Workforce Investment Act (WIA - see box) might eventually function as adult literacy performance standards. In a way, this is "setting performance standards by the back door." In other words, provisions of the NRS and of the WIA may serve the purpose of performance standards without being explicitly labeled as such.

Opportunity-to-learn (OTL) standards specify the nature of educational inputs and resources that are needed to realize expectations for student — and school — performance (NCEST, 1992). The NCEST report also suggested that OTL standards are needed to respond to concerns over the potential inequity of raising expectations for all students without ensuring that all had an equal opportunity to meet higher expectations. For example, OTL standards might specify the number of hours and quality of instruction that students should receive before they are tested on desired levels of skills and knowledge specified in content and performance standards.

Although explicit mention of OTL standards was dropped from the American standards-based educational reform movement early on, they remain important in the theoretical model. This is clearly stated in the following description of the connections among the three types of standards from the 1993 report by the National Academy of Education: ". . . for meaningful and fair performance standards to be set, it is necessary to define the exact content areas to which these standards shall apply. Before performance can be fairly assessed, it is moreover necessary to determine whether all students have had adequate opportunities to learn the prescribed content" (quoted in Husen & Tuijnman, 1994, p. 2).

It should be clear by this point that performance standards carry most of the load in the theoretical model. They specify "how much" students should know and be able to do. They may also specify the tests used to measure whether students have learned enough. For these reasons, performance standards supply the leverage needed to hold learners and educational programs "accountable" for learning. While content standards may be the result of broadly inclusive efforts to achieve consensus on "what" students should know, in the end, they simply describe what "ought to be," not what "must be." Content standards alone do not drive a system of accountability for educational outcomes. Performance standards, with accompanying indicators to specify where to look to see how much is there and benchmarks to determine the level of performance that is enough, do.

In the ideal model of standards-based educational reform, content, performance, and OTL standards each have clear and distinct roles to play. In practice, however, the line between content and performance standards often becomes blurred and OTL standards have been mostly neglected. Still, it is helpful to consider the three types of standards — in practice — separately.

In Practice
According to the ideal model, defining national content standards would seem the logical next step to take after adopting national educational goals. In some K-12 subject areas, such work was underway when the national goals were developed. This work on national content standards was seen as a way to define the broad outlines of subject matter that should be studied at various levels in schools across the country (see NCEST, 1992; Ravitch, 1992). One of the most commonly heard arguments against the development of national K-12 content standards is that such standards might create a "standardized" national curriculum that lacks the diversity and flexibility that many see as among the main strengths of the decentralized American educational system (Apple, 1993; Eisner, 1993). Proponents counter by pointing out that content standards are meant to serve as general guides for curriculum and should ideally be "general, visionary, and not at all prescriptive" (Porter, 1993, p. 25). Pie in the sky or procrustean bed — you choose.

In theory, with content standards you should have a choice. In practice, choice may be lacking. Content standards may influence commercial textbook publishers to such a degree that the only available — or allowable — texts and materials are aligned with them. For example, content standards and curriculum frameworks developed by large states such as California and Texas have a direct impact on the content of textbooks for K-12 subject areas. Choice in content may also be lacking when federal, state, or locally mandated testing and reporting requirements encourage schools to "teach to the test." Based on the recommendations of the National Reporting System and some states' plans to use specified levels of performance on the CASAS or TABE tests as core indicators of learner outcomes, this may soon be the situation for many adult literacy programs. In such a scenario, results on these tests may become de facto performance standards and adult literacy programs may feel compelled to choose content and materials that are aligned with the skills and knowledge measured by these tests.

Whither OTL?
Opportunity to learn (OTL) standards — also known as delivery standards — were among the first victims of the encounter between the ideal model of standards and the realities of American educational politics. Delivery standards were included in the original House version of the Goals 2000 bill, but were dropped before the bill became law (Lewis, 1992). The original rationale for OTL standards went something like this: for performance standards to be fair, students and others who will be held accountable for outcomes must have the opportunity to meet those standards, therefore there should be standards for the quality of schools and schooling. Arguments about the dimensions of school quality and who would pay for it led OTL standards to an early demise.

Within the K-12 arena, the politics of educational standards has created some strange bedfellows. Opposition to standards in general — and to OTL standards in particular — has come from both the liberal and conservative ends of the political spectrum. Religious conservatives and radical leftists have sometimes found common ground in their shared support of "local" control of education. On the other side, supporters of OTL standards see them as guarantors of equity in educational opportunities and outcomes.

Political alignments on issues of adult literacy can also unite otherwise habitual adversaries. The provisions of the 1998 Workforce Investment Act (WIA) relating to adult literacy program accountability are a good example of this intersection of political interests. Title II, the Adult Education and Family Literacy Act, ties receipt of federal funding by states to the development and implementation of five-year plans for improving instructional and professional development outcomes. This requirement appeals to fiscal conservatives in that it can be seen as an accountability mechanism that compels each state to explain what it will do and how it will show results from federal funding. It also appeals to liberals in that it encourages states to engage in strategic planning to expand access and increase equity in adult literacy learning opportunities, especially in poor and minority communities where the need is especially acute. Will this two-pronged support last? The ultimate irony and very real danger is that in satisfying the program accountability requirements of the WIA, states may be seen to be defining something akin to OTL standards for adult literacy. They may well encounter the same sorts of bipartisan opposition to "unfunded mandates" and loss of local control that sank OTL standard setting for the K-12 system.

Driven by Assessment
If content standards leave us a choice and OTL standards foster resistance, where does that leave performance standards? They are right where they always were: at the center of standards-based reform. Both critics and advocates characterize the educational standards movement as an "assessment-driven" reform effort. The basic idea here is that since teachers often teach to the test, one way to improve teaching and learning is to create a better test.

The ultimate success or failure of standards-based reform rests heavily on the creation of new forms of assessment, specifically, new performance-based assessments. Performance-based assessments may take a variety of forms including complex tasks, investigations, portfolios of student work, or any other assessments that require learners to make use of prior knowledge, recent learning, and relevant skills in actively solving significant and realistic problems (Herman et al., 1992). The emphasis on performance standards and performance-based assessment has directed the attention of psychometricians away from issues of reliability, which is how consistently a test measures skills or knowledge, towards issues of validity: how well a test measures the skills or knowledge it was designed to measure (Messick, 1994).

Tasks in performance-based assessments are typically longer, fewer in number, and scored in a more subjective manner than tasks in more traditional standardized tests. This has raised concerns about the potential for bias and inequity in the use of such assessments (see Darling-Hammond, 1994; Linn et al., 1991). These concerns are heightened in situations where the outcomes of a test have significant consequences either for the learner or for the educational program. In such high-stakes environments, the tendency is to fall back on tests that have a track record of previous use and that produce consistent results. Standardized tests with multiple-choice, fill-in-the-bubble formats have the advantage of producing scores which only experts can effectively challenge. Performance-based tests have the advantage of measuring skills and knowledge in ways that make the content and levels of expected performance clear to everyone, including learners and teachers.

Salvation?
Will standards solve our educational problems and make American students world-class academic performers? This question has generated fierce debate among K-12 educators. Opposition to particular standard-setting efforts has been intense and often effective. In 1994, the US Senate in a nearly unanimous vote (991) rejected a draft version of standards for US history. More recently, the Clinton administration has faced an uphill battle in trying to institute a system of national tests in key subject areas. Nonetheless, despite setbacks at the national level, the standards movement marches on and seems to be gaining ground at the state and local levels.

Adult educators joined the standards fray rather late. In some ways, this is an advantage. As late-adopters we can benefit from the successes and failures of the K-12 efforts. In other ways, a late start is a significant disadvantage. One of the most important lessons of the K-12 efforts is that standards-setting works best when everyone is part of the process. This takes time. NCTM, for example, has been at work on standards for more than a decade.

Accountability is why the adult literacy field can't take its time with standards. While it is possible to have accountability without explicit content and performance standards, defining standards through a broad-based consensual process provides an opportunity for many voices to inform key decisions about who needs to be held accountable, how they should held accountable, and for what. At a minimum, we need to have performance standards and test results to show how many learners are making enough progress to be counted as success stories. Of course, the usual success stories that adult literacy programs tell about their learners include more than test scores. That's fine as far as it goes, but in policy and funding circles these days, it doesn't go far enough.

"Have states increased the percentage of adults who score at or above level 3 in prose literacy [in the National Adult Literacy Survey]?" (NEGP, 1998, p. 43) is the indicator for adult literacy defined by the National Education Goals Panel. Consider how this indicator might work within the accountability model of standards-based educational reform. First, we might ask what connection this indicator has to the content of adult literacy education. Are the skills measured by the prose scale of the National Adult Literacy Survey (NALS) (Kirsch et al., 1993) a reflection of broad consensus on what adults "should know and be able to do?" In the absence of content standards — or at least the absence of content standards aligned with skills measured in the prose scale of the NALS — our national indicator for measuring performance is not directly connected with the content (or goals) of the adult literacy education system. Should adult learners measure up to level 3 on the NALS scale? Level 3 seems to match pretty well to the literacy proficiencies of successful GED examinees, which gives some legitimacy to the benchmark (Baldwin et al., 1995). Without explicit performance standards, however, this benchmark is not directly linked to expectations held by the adult literacy field and communicated to students in adult literacy programs. And, since no one is talking much about opportunity-to-learn standards these days, no real discussion occurs of what resources and learning opportunities adult learners should have access to before we hold them — and the programs that assist them — accountable for reaching level 3 on the NALS scale.

Clarify Expectations
Adult literacy programs have always been accountable to their funding sources in one way or another. As competition for public money has increased, however, pressure to show results from investments in adult basic education has also increased. A coherent system of content, performance, and, I would argue, opportunity-to-learn standards for adult basic education could help to ease the pressure and clarify expectations on all sides.

At the national level, the EFF initiative has made much progress in defining content standards that would serve to guide development of curricular content for adult literacy education. At the same time, with encouragement from accountability provisions of the WIA, the US Department of Education and states have moved closer to consensus on a national reporting system. In addition, planning is now underway for a second National Assessment of Adult Literacy (NAAL), scheduled for 2002. These three initiatives seem to be moving in rather different directions. Each is setting its own content and performance standards for the field of adult literacy. EFF is building broad consensus around standards for what adults should know and be able to do to fulfill adult roles as "workers, citizens/community members, and parents/family members." EFF is expanding the range of "desirable" skills and knowledge in ways that seem to be a closer fit to the goals and outcomes to which adult literacy programs and adult learners aspire. The National Reporting System is attempting to make use of the best available measures to gather information on program outcomes. The NAAL is aiming to profile the range and distribution of literacy (document, prose, and numeracy) skills in the adult population of the United States.

Alongside these national developments, these days the real action in standards setting seems to have shifted to the state and local levels. California has already published and distributed Model Program Standards for Adult Basic Education (1996) and Model Standards for Adult English as a Second Language Programs (1991).

A number of other states, including Illinois, Iowa, Massachusetts, Minnesota, New York, Pennsylvania, and Texas, among others, have been actively developing program and learner outcome standards. It is too soon to tell how state-level standards for adult literacy will turn out or the extent to which they will adhere to the ideal model or to national standards emerging from EFF, NRS, and NAAL.

Closer to the Ideal
This is the current state of practice in adult literacy standards. According to the ideal model of standards-based reform, all forms of standards — content, performance, and opportunity-to-learn — should be aligned. To bring the practice closer to the ideal, we must somehow connect EFF, NRS, and NAAL as well as state-level standards. This will not be easy, but will offer many benefits. First, coherent content standards can provide a clear vision of what every adult should know and be able to do. Performance standards and related assessment matched to this vision will provide the tools for individual learners, literacy programs, and everyone else to monitor progress toward goals.

Opportunity-to-learn (OTL) standards may be especially critical for a system of education (adult literacy) that is chronically under-funded. As I have argued elsewhere (Stites, Foley, & Wagner, 1996), the stakes in standards setting are high. Adult educators and adult learners have a special stake in standards and need to be actively involved in all areas of standards setting for adult literacy. Finally, the particular characteristics of the field of adult literacy may call for the development of standards to meet such needs as assuring equity, improving coordination of services, and meeting the learning needs of an increasingly diverse population. A serious consideration of OTL standards for adult literacy would be a good place to start to address these needs.

References
Apple, M. W. (1993). "The politics of official knowledge: Does a national curriculum make sense?" Teachers College Record 95(2), 222-241.

Baldwin, J., Kirsch, I. S., Rock, D., & Yamamoto, K. (1995). The Literacy Proficiencies of GED Examinees: Results from the GED-NALS Comparison Study. Washington, D.C.: GED Testing Service.

Darling-Hammond, L. (1994). "Performance-based assessment and educational equity." Harvard Educational Review, 64(1), 5- 30.

Eisner, E. W. (1993). "Why standards may not improve schools." Educational Leadership, 50(5), 22- 23.

Herman, J.L., Aschbacher, P.R., & Winters, L. (1992). A Practical Guide to Alternative Assessment. Alexandria, VA: ASCD.

Husen, T., & Tuijnman, A. (1994). "Monitoring standards in education: Why and how it came about." In A. Tuijnman & T. N. Postlethwaite (eds.), Monitoring the Standards of Education (pp. 1- 22). New York: Pergamon.

Kirsch, I. S., Jungeblut, A., Jenkins, L., & Kolstad, A. (1993). Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey. Washington, DC: US Government Printing Office.

Lewis, A. C. (1992). "House bill includes school delivery standards." Phi Delta Kappan, 100-101.

Linn, R.L., Baker, E.L., & Dunbar, S.B. (1991). "Complex performance- based assessments: Expectations and validation criteria." Educational Researcher, 20 (8), 15- 21.

Messick, S. (1994)."The Interplay of evidence and consequences in the validation of performance assessments." Educational Researcher, 23 (2), 13- 23.

National Academy of Education. (1993). Setting Performance Standards for Student Achievement. Stanford, CA: The National Academy of Education, Stanford University.

National Commission on Excellence in Education. (1983). A Nation at Risk: The Imperative for Educational Reform. Washington, DC: US Department of Education.

National Council of Teachers of Mathematics (NCTM). (1995). Assessment Standards for School Mathematics. Reston, VA: NCTM.

National Council on Educational Standards and Testing (NCEST). (1992). Raising standards for American Education: A Report to Congress, the Secretary of  Education, the National Education Goals Panel, and the American People. Washington, DC: US Government Printing Office.

National Education Goals Panel (NEGP). (1994). The 1994 National Education Goals Report: Building a Nation of Learners. Washington, DC: National Education Goals Panel.

National Education Goals Panel (NEGP). (1998). The 1998 National Education Goals report: Building a Nation of Learners. Washington, DC: National Education Goals Panel. (www.negp.gov).

Pearson, P. D. (1993). "Standards for the English language arts: A policy perspective." Journal of Reading Behavior, 25(4), 457- 475.

Porter, A. (1993). "School delivery standards." Educational Researcher, 22(5), 24- 29.

Ravitch, D. (1992). "National standards and curriculum reform: A view from the Department of Education." NASSP Bulletin, 75(548),
24- 29.

Stites, R., Foley, E., & Wagner, D. A. (1996). Standards for Adult Literacy: Focal Points for Debate. Philadelphia, PA: University of Pennsylvania, National Center on Adult Literacy.

US Department of Labor, The Secretary's Commission on Achieving Necessary Skills (SCANS) (1991). What Work Requires of Schools. Washington, DC: US Government Printing Office.

About the Author
Regie Stites is an educational researcher in the Center for Education and Human Services at SRI International in Menlo Park, California. He was previously a research associate at the National Center on Adult Literacy, University of Pennsylvania Graduate School of Education.