Legislative Analyst's Office Analysis of the 2003-04 Budget Bill

Assessments

The Governor's budget proposes $131 million to develop, administer, and analyze state assessments, an increase of approximately $1 million over estimated current-year expenditures. Figure 1 shows the change in funding from 2002-03 to 2003-04. Appendix 1 provides background on major state assessments.

The Governor's budget includes five major assessment changes as shown in Figure 1 and described below:

• Eliminate Assessments in Career Education. The Governor proposes eliminating Assessments in Career Education, for savings of $0.9 million. These assessments are voluntary end-of course exams for students in certain vocational education programs. The State Department of Education (SDE) would use the Standardized Testing and Reporting (STAR) results to fulfill federal requirements for reporting achievement in vocational education.
• California English Language Development Test (CELDT). The budget proposes an augmentation of $5 million for increased contract costs for the CELDT and $1.5 million to pay district apportionments for 300,000 test takers at a reimbursement rate of $5 per pupil. These new funds would cover costs incurred in 2003-04, as well as costs incurred in 2002-03 that were deferred to 2003-04.
• California High School Exit Exam (CAHSEE). The budget augments this program by $2.9 million to pay district apportionments and CAHSEE workbooks for an additional 367,000 test takers.
• Golden State Exams. The Governor proposes to reduce current-year funding for the Golden State Exams by $8 million, leaving $7.4 million, as part of the mid-year budget adjustments. For 2003-04, the budget recommends an additional $1.5 million reduction due to savings in contract costs. The Department of Finance (DOF) indicates that the SDE will likely need to eliminate several of the 13 Golden State Exams, but should have sufficient funds for the three Golden State Exams that have been integrated with STAR—reading/literature, written composition, and high school mathematics.
• Test Development. The budget reduces funding for test development by $2.1 million.

The budget bill also includes a provision requiring the State Board of Education (SBE) to annually establish the amount of funding to be apportioned to school districts for the CELDT and CAHSEE. The amount of funding to be apportioned per test would be subject to DOF approval.

At the time of this writing, the Legislature was considering alternatives to the Governor's recommendations for the current year.

Overview and History of STAR Program

While the state conducts numerous tests, the "backbone" of its assessment system is the STAR program. Established by Chapter 828, Statutes of 1997 (SB 376, Alpert), this program currently consists of three elements: (1) the California Achievement Test-6 (CAT/6) nationally norm-referenced test (NRT); (2) the California Standards Tests; and (3) the Spanish Assessment of Basic Education (SABE/2) primary language test.

California's assessment system has evolved over the last 12 years from testing a sample of students to testing all students in grades 2 through 11 as part of the STAR program. During this time-period, two types of assessments have dominated the landscape:

• An NRT is a standardized test designed primarily to compare the performance of students with a nationally representative sample of students that have taken the same exam. An NRT does not directly measure how students perform in relation to a state's academic content standards.
• A criterion-referenced test is a standardized test aligned with a state's academic content standards that measures students' performance with respect to these standards rather than to the performance of their peers nationally.

Two key concepts that have guided the development of California's assessment system are the terms "valid" and "reliable." Valid means that a test measures the skills and content matter that it purports to measure. Reliable means that a test score can be trusted to reflect actual student achievement at the applicable level of analysis—student, school, or district. If the test were to be repeated with the same group of students, a reliable test would produce similar results.

Prior to the creation of STAR, the state had a variety of assessment systems (see Appendix 2 for a brief history of these systems). In 1995, the state began an emphasis on academic performance standards through the California Assessment Academic Achievement Act. This statute aimed to develop academic content and performance standards that would eventually be used for state-approved local tests and a statewide basic and applied skills test for selected grade levels.

STAR Program Begins With NRT. Because of the length of time necessary to develop standards and standards-based tests, the Legislature used an NRT as an interim strategy when it created the STAR program in 1997. Under the STAR program, the state requires schools to test all students in grades 2 through 11 using an NRT. Tests originally required for students in grades 2 through 8 included reading, spelling, writing, and mathematics, while tests originally required for students in grades 9 through 11 included reading, writing, mathematics, history/social science, and science.

The legislation creating STAR required SBE to adopt an NRT for the program by November 1997 and expressed the Legislature's intent to augment the NRT with items that assess the standards being adopted by SBE. Until SBE adopted these standards, the test had to be aligned with the curriculum frameworks and program advisories jointly adopted in 1996 by SBE, the Superintendent of Public Instruction, and the Commission on Teacher Credentialing.

In November 1997, the SBE designated the Stanford Achievement Test, Ninth Edition, Form T (Stanford 9) as the test for the STAR program. The state used the same version of the Stanford 9, including the same test questions, from 1998 through 2002. For 2003, the CAT/6 will replace the Stanford 9 as the state's NRT.

Primary Language Test Added to STAR. Chapter 828 also created a new requirement that English language learner (ELL) pupils in grades 2 through 11 take a test in their primary language if they have been enrolled in the state for less than 12 months. The legislation made the test optional for students who had attended public school in the state for more than 12 months. The SBE adopted the SABE/2 to meet the primary language test requirement. The test, which the state first administered in spring 1999, is an NRT that measures basic skills in mathematics, language arts, reading, and spelling.

The STAR Program Transitions to Standards-Based Tests

In 2000, Chapter 576 (AB 2812, Mazzoni) began the transition of the STAR program to standards-based tests. The legislation created a standards-based test for students in grades 2 through 11 as part of the STAR program. It also eliminated a previous requirement to develop a standards-based test of basic and applied skills for selected grades. In 2001, the Legislature named the new standards-based test the California Standards Test. Chapter 576 requires the test to measure the degree to which pupils achieve the academically rigorous content and performance standards adopted by SBE. The legislation requires tests in reading, spelling, writing, and mathematics for grades 2 through 8. For grades 9 through 11, the legislation requires tests in reading, writing, mathematics, history/social science, and science.

The California Standards tests have been phased into the STAR program over time. The state administered the English language arts and mathematics standards tests on a pilot basis in 1999. In 2001, SDE first reported English language arts scores using the following performance levels: advanced, proficient, basic, below basic, and far below basic. In 2002, mathematics scores also began to be reported as performance levels. In addition, the state piloted social science and science tests in 2001 and first reported scores as performance levels in 2002.

In 2001, Chapter 722 (SB 233, Alpert), reauthorized STAR and changed the emphasis from NRT to standards-aligned tests by making the following key changes:

• Made standards-based tests independent of the NRT.
• Expressed intent to shorten the length of the NRT.
• Deleted the social science NRT for grades 9 through 11.
• Required continuation of the science NRT for grades 9 through 11 until the state develops a general standards-based test in science for these grades.

Figure 2 shows how SDE has implemented the provisions of Chapter 722 to de-emphasize the norm-referenced portion of STAR. It compares the elements in the 2002 NRT to the 2003 NRT and illustrates how there will be fewer and shorter tests in 2003.

Accountability System Transitions to Standards-Aligned Tests. The SDE has also reduced the NRT's impact on the school accountability system. As shown in Figure 3 and Figure 4, the NRT was 100 percent of a school's Academic Performance Index (API) in 2001. As the California Standards tests have been phased into STAR and established as valid and reliable, they have been making up a greater portion of the API. As shown in Figure 3, the weight of California Standards tests will increase from 36 percent in 2002 to 80 percent in 2003 for elementary and middle schools. Figure 4 shows that the weight of California Standards tests will increase from 24 percent in 2002 to 73 percent in 2003 for high schools. High school API will also include the CAHSEE in 2003.

LAO Assessment-Related Recommendations

We have identified several areas where the Legislature could reduce the assessment burden on school districts while maintaining the state's emphasis on academic content and performance standards. The four recommendations below address eliminating or reducing the following tests: the NRT portion of the STAR program, the primary language test, the Golden State Exams, and the physical fitness test.

Reduce Emphasis on NRT  

We recommend that the Legislature continue to reduce emphasis on the norm-referenced test by requiring school districts to administer the California Achievement Test-6 (CAT/6) only in grades 4 and 8 instead of grades 2 through 11. We further recommend reducing the allocation for the Standardized Testing and Reporting program by $10 million to redirect funds for the CAT/6 to other education priorities. (Reduce Item 6110-113-0001 by $10 million.)

Students in grades 2 through 11 take the CAT/6 nationally normed test and the California Standards Test. We recommend that the Legislature decrease emphasis on the NRT by only requiring school districts to administer the CAT/6 in grades 4 and 8 for the following key reasons:

• Standards-Based and Norm-Referenced Assessments Test Duplicative Subjects, Decreasing Instructional Time. Under current law, students in grades 2 through 11 take both norm-referenced and standards-based tests as part of the STAR program. These assessments cover duplicative subjects. Students take tests in English language arts, mathematics, and science under both assessments. While the length of the NRT has decreased in the past year, duplicative testing still results in higher costs and decreased instructional time. Decreasing the number of grade levels taking the NRT would significantly reduce the testing burden on schools.
• Standards-Based Tests Have Developed Sufficiently to Be the Focus of the School Assessment and Accountability System. The history of California's assessment system shows a long-standing commitment to developing academic content standards and assessments that measure those standards. The Legislature implemented the NRT as a transition strategy until standards and associated tests could be developed and shown to be valid and reliable. As the state has developed valid and reliable standards-based tests, the Legislature has shifted the focus of California's assessment system from an NRT to standards-based tests by shortening tests and testing fewer subjects. Reducing the number of grade levels assessed with the NRT would continue this trend. The state's school accountability system has also evolved from being composed solely of the NRT to standards tests making up more than three-fourths of the API. Now that standards-based tests have been shown valid and reliable, we believe it is appropriate to decrease the emphasis of an NRT in the API even further.
• The Federal Government Now Requires an Assessment and Accountability System Focused on Standards-Based Tests. New federal requirements for assessments and accountability systems also move towards standards-based assessments. No Child Left Behind (NCLB) requires states to develop accountability systems that use assessments aligned to state academic content standards.
• The State Should Send a Uniform Message to Teachers and Students Emphasizing the Academic Content and Performance Standards. Another key reason to decrease emphasis on NRT is to send a consistent message to teachers and students on the importance of the academic content and performance standards. Schools across the state have been integrating the standards into their curriculum in the past few years and some school districts are moving towards standards-based report cards. Making the NRT a significant portion of the STAR program undermines focus on the academic standards. Decreasing emphasis on the NRT would solidify the state's commitment to the academic content standards.

Retaining the NRT in Grades 4 and 8 Would Still Allow Comparisons With the Federal National Assessment of Educational Progress (NAEP). The state also participates in NAEP, which the federal government administers to a sample of students in grades 4 and 8 in reading, math, writing, and science. While the NAEP measures how California is doing compared to other states, it does not provide district, school, or student level information. Using an NRT in grades 4 and 8 would maximize the state's ability to compare student performance with national samples of students because the federal government administers NAEP in reading and math in the same grade levels every two years.

Reducing the Grades Tested Will Reduce State Costs. Reducing the number of grades tested would reduce costs by $10 million in 2003-04. It would also reduce future costs under a pending mandate claim for the NRT portion of the STAR program. Based on parameters and guidelines adopted by the Commission on State Mandates, the claims total $184.1 million for costs from 1997-98 through 2003-04. The commission estimates that ongoing costs will be approximately $32 million per year. Reducing the number of grade levels taking the NRT would substantially reduce the ongoing costs of the mandate.

Because the NRT was an interim assessment strategy and standards-based tests have developed sufficiently to become the core of the state's assessment system, we recommend that the NRT be given only in grades 4 and 8. This would save the state $10 million in 2003-04.

Eliminate Primary Language Test Requirement

We recommend that legislation be enacted to eliminate the requirement that districts test English language learner pupils in their primary language. This would result in Standardized Testing and Reporting program savings of $1.6 million, which could be used for other education priorities. (Reduce Item 6110-113-0001 by $1.6 million.)

Under the STAR program, school districts are required to test ELL pupils in grades 2 through 11 in their primary language if such a test is available and the pupils have been enrolled in a California public school for less than 12 months. In addition, districts may administer a primary language test to pupils enrolled for longer than 12 months. In order to address this requirement, the SBE has designated the SABE/2 for native Spanish speakers but has not approved any other primary language tests. This test, which is in Spanish, measures student achievement in reading, language, mathematics, and spelling.

The Governor's budget includes $1.6 million in Proposition 98 funds for the SABE/2 in order to (1) pay the test publisher for materials and exam scoring and (2) reimburse school districts, county offices of education, and charter schools for administering the exam. We recommend eliminating the primary language test requirement and reducing funding by $1.6 million for the following reasons:

• School Districts Use Other Tests for Placement and Monitoring Purposes. Local districts often administer separate primary language tests for initial placement and to monitor the academic progress of their ELL pupils. The SABE/2 is not useful for this purpose because it is given in the spring, almost at the end of the school year. Eliminating the requirement to administer the SABE/2 would help reduce testing time at the local level. Local entities could continue to use the SABE/2 for local testing, if school officials deemed the test useful for local purposes.
• SABE/2 Is Not Consistent With the State's Emphasis on English Language Development. Proposition 227, passed by the voters in June 1998, in essence requires that children in California's public schools learn English by being taught all subjects predominantly in English, and to limit bilingual education classes to specified exceptional circumstances. The proposition states that: (1) ELL pupils should move from special classes to regular classes when they have acquired a good working knowledge of English, and (2) normally these classes should not last longer than one year. Given the changes made by the proposition and its objective to help ELL pupils acquire English as quickly as possible, we believe there is no longer a reason for the state to require certain ELL pupils to be tested in Spanish, especially since districts generally use other primary language tests for placement and monitoring purposes. Under our recommendation, the state would continue to administer the CELDT to each ELL annually, providing a comprehensive assessment over time of English proficiency based on state standards.
• Only a Small Proportion of the State's ELL Pupils Are Tested. In 2001-02, the state administered SABE/2 to about 7 percent of the state's 1.6 million ELL pupils. This small number is because most schools do not administer the test to pupils beyond the required first year. Because only a small subset of ELL pupils take the test, the results cannot be extrapolated to all ELL pupils. This limits the test's usefulness for directing state policy and local district decision-making.
• SABE/2 Does Not Fit Into the State's Accountability System. Current law requires districts to administer the primary language test only once to any particular student. Because the state assesses a new set of students each year, it is not possible to use the test to assess academic growth or to make comparisons of achievement across schools. For these reasons, SABE/2 cannot be easily incorporated into the state's accountability system, which aims to show growth in student achievement over time. This limits is usefulness from the state perspective.
• Test Is Not Aligned to Standards as Required by NCLB. The SABE/2 does not comply with federal requirements that assessments be aligned to state academic content standards. To comply with the federal law, the state would have to design a new test aligned to state standards. Because a primary language assessment is only required by the federal government "to the extent practicable," we do not recommend that the state invest its resources in development of a primary language test aligned to standards.

Because SABE/2 is not useful for state or local purposes, we recommend that the Legislature eliminate the primary language test requirement, saving $1.6 million in 2003-04. School districts would continue to administer primary language tests for placement and monitoring purposes.

Eliminate Golden State Exam

We recommend that the Legislature eliminate the Golden State Exam. This would result in savings of $5.9 million in 2003-04, which could be used for other education priorities. (Reduce Item 6110-113-0001 by $5.9 million.)

The Legislature created the Golden State Exam in 1983 through Chapter 498 (SB 813, Hart). These tests are voluntary, rigorous end-of-course exams given in grades 7 through 12. The tests are aligned to the state academic performance standards and contain open-ended analysis and writing questions in each test. There are 13 specific course tests: three math, two English/language arts, three social science, four science, and one Spanish. The state administers most of the tests at the end of the spring semester, but the government and economics tests are also administered in the winter. In 2001, students took a total of 1.2 million Golden State Exams.

In 2001, Chapter 722 required that SDE integrate the Golden State Exams into the California Standards Tests to reduce testing time and duplicative testing. The SDE has integrated three Golden State Exams into the California Standards Tests to date—reading/literature, written composition, and high school mathematics. Students wishing to take the Golden State Exam in these subjects take the test as an augmentation to the California Standards Test. The California State University (CSU) system also plans to start using the results from these three tests for placement purposes in 2003-04.

Students who perform well on the Golden State Exams can qualify for the Golden State Seal Merit Diploma. To qualify for this diploma, a student must achieve high honors, honors, or recognition on at least six Golden State Exams. The state has awarded more than 22,000 Golden State Seal Merit Diplomas since the program began in 1997.

The Governor recommends greatly reducing funding for the Golden State Exams. The 2002-03 Budget Act provided $15.4 million for this purpose. As part of the 2002-03 mid-year revision, the Governor recommended reducing funding for the exam by $8 million to $7.4 million. In 2003-04, the Governor recommends funding of $5.9 million, which is an additional $1.5 million reduction. The DOF indicates that SDE would have to eliminate some Golden State Exams, but that the administration's priority would be to continue funding the three exams that have been integrated into the California Standards Tests.

We recommend that the Legislature discontinue the Golden State Exams for the following key reasons:

• Tests Are Voluntary and Not Part of the Accountability System. The Golden State Exams are not required by either state or federal law. In addition, they are not included in the state's API.
• Other Tests Are Available to Show High Achievement. The Golden State Exams are duplicative of other tests that recognize high achievement. The California Standards Tests include two performance levels that indicate above-grade level performance—proficient and advanced. In addition, students can take Advanced Placement Exams in many of the same subjects offered on the Golden State Exams. Advanced placement tests are often used for placement and college credit at various higher education institutions statewide.

Although CSU plans to use the Golden State Exams for placement purposes, the test will only be taken by a small proportion of students. Other students will still need to take a separate placement test. Therefore, it is not imperative that the Golden State Exam be provided for this purpose.

Eliminate Physical Fitness Test Mandate

We recommend that the Legislature make the physical fitness test optional for grades 5, 7, and 9, and reduce funding for the test by $1.1 million. (Reduce Item 6110-295-0001 by $1.1 million.)

In 1995, the Legislature adopted Chapter 975 (AB 265, Alpert), which mandated a physical fitness test in grades 5, 7, and 9. The state uses the Fitnessgram, which measures six major health-related areas of physical fitness, including aerobic capacity, body composition, abdominal strength, upper body strength, trunk strength, and flexibility.

The state also requires physical education for all students in grades 1 through 9, plus one additional year in high school. Students in grades 1 through 7 are required to have 200 minutes of physical education every 10 days, and students in grades 7 through 12 are required to have 400 minutes every 10 days.

While physical fitness is important, we believe that the state should make the physical fitness test optional to reduce the number of assessments mandated, reduce the testing burden, and increase instructional time. Our recommendation would not affect the existing requirement for physical education. Under our proposal, school districts interested in continuing to administer the physical fitness test could continue to do so. Adopting this recommendation would save $1.1 million that would be available for other educational priorities.

Appendix 1  

Appendix 2

History of Assessments Through 1996

California Assessment Program (CAP) Provided Only Schoolwide Data. Until 1990, the state had the CAP. The CAP tests calculated average student scores that indicated how well students fared in mathematics, reading, writing, social science, and science in grades 3, 6, 8, and 12. The state did not intend CAP results to be reliable for individual students. Instead, the state designed the results to provide a reliable gauge of average achievement at the school or district level. To minimize costs and test taking time, students took only part of each test and the state aggregated results at the school or district level.

California Learning Assessment System Focused on Performance Testing. In 1991, Chapter 760 (SB 662, Hart), authorized the California Learning Assessment System (CLAS) testing program, which tested students in mathematics, reading, and writing in grades 4, 8, and 10. The CLAS differed from CAP in two ways. First, CLAS focused on performance testing, which asks students to solve open-ended questions rather than answer multiple-choice questions. Second, CLAS tests provided individual student scores, rather than school-level scores. The Governor vetoed CLAS funding from the 1995-96 Budget Bill after questions were raised over some test questions and the reliability of scores for individual students.

California Assessment Academic Achievement Act Launched Emphasis on Standards. In 1995, Chapter 975 (AB 265, Alpert), created the California Assessment Academic Achievement Act, which included both state-approved local tests and a statewide basic and applied skills test. It also set the stage for development and adoption of academic content and performance standards to be used for assessment purposes.

Chapter 975 set up a local testing incentive program that paid districts $5 per pupil to administer state-approved basic academic skills tests for students in grades 2 through 10. The legislation required the State Board of Education (SBE) to adopt tests aligned with the academic content standards. Until standards were in place, the tests had to be reasonably aligned with the state curriculum frameworks.

The legislation also authorized development of a test of basic and applied skills administered to students in grades 4, 5, 8, and 10 in reading, writing, mathematics, history/social science, and science. While the legislation prohibited SBE from adopting this assessment until it had adopted academic content and performance standards, it allowed SBE to contract for development of the assessment. In 1996, Chapter 69 (SB 430, Greene), postponed development of the basic and applied skills test until after SBE adopted content and performance standards. Ultimately, other standards-based tests were developed in lieu of this test.

The legislation also created a 21 member Commission for the Establishment of Academic Content and Performance Standards. This commission was required to develop "academically rigorous" content and performance standards for the K-12 curriculum and submit them to the SBE for approval by July 1, 1997. Content standards describe the knowledge and skills students should acquire in a given grade. Performance standards describe what students need to do to demonstrate they are "proficient" in the knowledge and skills outlined in the content standards. The statutes originally required SBE to adopt academic content and performance standards by January 1, 1998, in reading, writing, mathematics, history/social science, and science. Over time, the deadlines for some subjects were changed. The SBE adopted standards for: (1) English language arts and mathematics in December 1997, (2) English Language Development standards in July 1999, (3) history/social science and science standards in October 1998, and (4) visual and performing arts standards in January 2001.