Legislative Analyst's Office Analysis of the 2001-02 Budget Bill

Child Welfare Services

California's state-supervised, county-administered Child Welfare Services (CWS) program provides services to abused and neglected children, children in foster care, and their families. The CWS program provides (1) immediate social worker response to allegations of child abuse and neglect; (2) ongoing services to children and their families who have been identified as victims, or potential victims of abuse and neglect; and (3) services to children in foster care who have been temporarily or permanently removed from their family because of abuse or neglect. The 2001-02 Governor's Budget proposes $1.9 billion ($633 million General Fund) for CWS and $1.6 billion ($413 million General Fund) for Foster Care. These represent increases of 3 percent (1 percent General Fund) and 6 percent (7 percent General Fund), respectively, from the current year.

Improving CWS Through Structured Decision Making

Structured Decision Making (SDM) is a series of "tools" designed to aid child welfare workers in making critical child safety decisions. Research indicates that SDM improves child welfare outcomes, as compared to alternative approaches. Currently 14 California counties are using SDM and 10 additional counties are on the SDM waiting list. We recommend expansion of the program in the budget year and make several other recommendations to improve SDM implementation. (Increase Item 5180-151-0001 by $650,000.)

Background

Child abuse and neglect continues to be a serious problem in California. In 1999, over 600,000 allegations of child abuse and/or neglect were reported to county child protective services agencies. Approximately 400,000 of these reports were investigated; over 120,000 (30 percent) of those cases investigated were substantiated; and over 33,000 (28 percent) children who were victims of substantiated abuse or neglect were placed in foster care. In addition, a significant proportion of the families who were the subject of reports and substantiation of abuse or neglect had prior contact with child protective services agencies.

What is SDM? California, like many other states, has used risk assessment to increase consistency and accuracy of CWS decisions. Structured Decision Making is a series of research-based risk assessment tools designed to aid child welfare workers in making critical child safety decisions. This approach has been shown to be more accurate and consistent in classifying children and families according to risk than alternative approaches. Key components of SDM are tools for determining (1) when to investigate abuse/maltreatment allegations, (2) the degree of child safety at the time of investigation, (3) the risk of future child maltreatment, (4) the targeted services to be provided to families at the highest risk of reabuse, and (5) whether to remove a child to foster care.

For example, the questionnaire used at the time of an in-person investigation aids social workers in determining whether a child is in danger of future abuse or neglect, whether a case should be opened, and how frequently services should be provided. As compared to some non-SDM assessments which may rely heavily on subjective criteria, most of these items tend to be objective, although some require the clinical judgement of the worker (see Figure 1).

 The resulting total score assigns families to risk categories according to the likelihood of future child abuse or neglect. A low score suggests a relatively low risk of reabuse, while a very high score implies a very high risk of further abuse. These classifications ("low," "medium," "high," and "very high"), assist workers in determining whether a case will be opened and what level of services will be provided to the family. For example, a case opened for a low-risk family may require only one monthly visit from a social worker, whereas a case in which a family is assessed to be at very high risk of future abuse or neglect may require four social worker visits in a month. Because no assessment tool correctly predicts outcomes all the time, each tool allows child welfare workers discretion to reassign risk to a higher classification than the tool may otherwise indicate.

Structured Decision Making in California. Since the mid-1980s, the Children's Research Center (CRC), a division of the National Council on Crime and Delinquency, has developed and implemented SDM in a number of states, including New York, Michigan, Indiana, Georgia, New Mexico, Oklahoma, Wisconsin, Rhode Island, and Alaska. Structured Decision Making was implemented in California in 1999. Prior to implementation, CRC and several California counties analyzed over 2,000 local child abuse and neglect cases. Based on this analysis, the CRC designed California's assessment tools and then aided counties in implementing the program.

In 2000-01, a total of 14 counties are using SDM on a voluntary basis: Alameda, Fresno, Humboldt, Kern, Los Angeles, Merced, Monterey, Orange, Sacramento, San Bernardino, San Luis Obispo, Santa Clara, Sutter, and Trinity. These counties have been using the SDM tools for an average of approximately one year. In Los Angeles and San Bernardino Counties, only one regional office each is using SDM. After adjusting for these two counties not using the SDM tools countywide, approximately 30 percent of California's abuse and neglect reports are currently being investigated using the SDM approach. At this time, ten more counties have expressed an interest in using SDM: Del Norte, Marin, Placer, Riverside, Santa Barbara, Santa Cruz, Solano, Tulare, Yolo, and Yuba. However, due to a lack of funds for SDM in the current year, these counties have been unable to participate.

Research Indicates SDM Improves Outcomes

Research From Other States. Evaluations have concluded that SDM has significant value in predicting the likelihood of future abuse or neglect and that it improves child welfare outcomes. The most comprehensive evaluation of SDM was conducted by CRC in Michigan in 1995. In that study, 11 counties that were voluntarily using SDM were matched with 11 other counties in the state that were using other methods for managing CWS reports and caseloads.

After two years, all cases handled in these counties were compared. Statistically significant differences were found in both administrative process outcomes and child/family safety outcomes between the SDM counties and the comparison counties. The process findings indicated that services in SDM counties were being appropriately redirected from lower-risk cases to higher-risk cases, effectively shifting resources to the families where the likelihood of future maltreatment was highest. The study also concluded that SDM counties had significantly improved child/family outcomes in contrast to the comparison counties. For example, for families who had prior contact with child protective services, the SDM counties had lower rates of (1) reported repeat abuse and neglect, (2) substantiations of abuse and neglect, (3) removal from the home, and (4) injuries (See Figure 2).

Although both SDM and non-SDM counties had relatively few negative outcomes, SDM counties had even lower rates of reported repeat abuse and neglect, substantiations, removals to foster care, and child injuries. Because California has more than four times the number of children as Michigan, achieving these outcomes could improve the lives of thousands of California children and families.

Another evaluation, by CRC in Wisconsin and published in 1998, affirmed the findings of the Michigan study. In the Wisconsin study, child protective cases in three SDM counties were compared over a two-year period to determine (1) SDM's effectiveness in classifying families according to risk and (2) the impact of providing intensive services to high- and very-high risk families. Results showed SDM classifications were effective in helping set agency priorities and that more intensive interventions for high- and very-high risk cases improved outcomes significantly, reducing subsequent reporting of abuse.

A third study, conducted in Texas and published in 1997, was initiated to address SDM's (1) value in predicting reabuse or neglect and (2) ease of transfer to a different ethnic/cultural and geographic setting. This study concluded that many of SDM's risk assessment items were valuable in predicting future child maltreatment, could be transferred to a new geographic setting, and effectively applied to different ethnic groups.

Structured Decision Making May Reduce Bias in CWS Decisions. Although national researchers have concluded that the "true" rate of child abuse and neglect is equal across racial and ethnic groups, certain groups are significantly over represented in California's CWS system. For example, although African American children are only 7 percent of California's child population, these children are 35 percent of the children in foster care. In addition, African American infants under one year of age are four to five times more likely to be removed to foster care than infants of other racial groups.

While various factors may explain some of these differences, research indicates that some of these disparities may be due to bias at key decision points in child welfare cases. Although SDM and other research-based risk assessment tools were initially criticized as potentially further increasing the representation of children of color in the CWS system, process evaluations indicate that SDM reduces or eliminates this bias. In other words, children and families, regardless of race or ethnicity, are classified according to risk very similarly. Reducing the perception of bias is important because it is likely to (1) improve public confidence in the system and (2) improve confidence among the populations affected by the CWS system.

California SDM Implementation Challenges

While expansion of SDM could improve California's child welfare outcomes, there are barriers to further expansion as well as implementation issues. We discuss these problems below. The first two issues concern barriers to expansion, while the last issue concerns implementation.

Budget Does Not Propose Funds For Expansion. Fourteen counties are currently using SDM and another ten counties have expressed interest in utilizing the SDM system. However, the 2001-02 Governor's Budget does not propose expansion of SDM. The budget proposes the same level of funding in 2001-02 as in the current year, which is $324,000 ($81,000 General Fund). This amount reflects the costs for continuing the current contract with CRC for support and technical assistance to counties who have been using SDM. According to the Department of Social Services (DSS), the cost to expand the SDM contract in the budget year to the ten counties on the waiting list would be $1.3 million ($317,000 General Fund; $1 million federal funds).

Current Technology Insufficient for Expansion. The Child Welfare Services/Case Management System (CWS/CMS) provides a statewide database, case management tools, and reporting system for the state's CWS program. While the system is in operation in all 58 counties, changes and additions to the system are both costly and time-consuming. According to DSS, the vendor for CWS/CMS estimated that it would cost $2 million (all funds) to integrate the SDM tools into that system. Instead of pursuing this option, CRC wrote its own software program, within the cost of the current contract, to provide the SDM assessment tools on workers' computers. While this solution has been effective for many of the counties currently using SDM, this software program has not been sufficient for large counties such as Los Angeles, and creates inefficient and redundant processes in some of the smaller counties. In order to solve these problems, CRC has proposed a technology solution that would allow for statewide expansion of SDM. The CRC estimates that this software program would cost approximately $500,000 ($125,000 General Fund).

Structured Decision Making Tool Completion Rates Not Maximized. Because SDM assessments aid in case management and resource allocation, it is important that the assessments are completed and the recommended service plans are followed. California is conducting a process evaluation of SDM to determine worker utilization of the SDM tools and family classification patterns. Preliminary results of this process evaluation indicate that while the "Response Priority" tool (used to determine timing of investigations) is completed almost 90 percent of the time by caseworkers, the remaining tools are being completed approximately 70 percent of the time. Although 70 percent shows a solid completion rate, there is room for improvement. More information is needed to determine what barriers may be hindering worker completion of assessment tools. Once barriers have been identified, solutions such as additional training or technical assistance to counties could be used to maximize completion rates in SDM counties.

Analyst's Recommendations for Expanding SDM in California

Research from other states indicates that SDM may improve outcomes in child welfare by decreasing repeated reports of abuse or neglect and admissions to foster care. Research also suggests that SDM may reduce bias at key decision points in CWS. Improving the CWS system in these ways could result in both fiscal savings to government and broader benefits to families. Below we make several recommendations to expand and improve SDM in California.

Expand SDM to Include Counties on Waiting List. As discussed earlier, the budget for the SDM project in 2001-02 is $324,000 ($81,000 General Fund) to provide support services in the 14 counties that have been using SDM. Also, ten additional counties have expressed an interest in using SDM: Del Norte, Marin, Placer, Riverside, Santa Barbara, Santa Cruz, Solano, Tulare, Yolo, and Yuba. However, due to a lack of General Fund support, these counties have been unable to implement SDM. According to DSS, expansion of SDM to these counties would cost $1.3 million ($317,000 General Fund) in 2001-02. (These costs include one-time start-up activities that are not incurred by the 14 current counties.) This would pay for technical assistance to the additional counties by a contractor. The DSS also indicates that SDM expansion would require the addition of two state staff positions to support the implementation phase. These positions would cost approximately $165,000 ($83,000 General Fund). We recommend SDM expansion to counties on the waiting list, at a total cost of $1.4 million ($400,000 General Fund).

Fund a Technology Solution. As we indicated earlier, one of the barriers to the expansion of SDM is the limitations of the current software. New technology must be implemented that (1) addresses Los Angeles County's expansion to the remaining 85 percent of its caseload (approximately 100,000 investigations annually), (2) reduces process inefficiencies in smaller counties, and (3) does not require integration into CWS/CMS. We therefore recommend funding a technology solution that addresses these needs. We estimate such a solution would cost approximately $500,000 ($125,000 General Fund).

Fund an Independent Outcome Evaluation. The only planned evaluation of SDM in California is a process evaluation. While this type of evaluation will provide important information about family risk classification and worker utilization, it will not provide California-specific information on SDM's impact on child welfare outcomes over time. An independent outcome evaluation is needed because (1) it will show whether California is attaining the results shown in research from other states and (2) it may suggest improvements and modifications for SDM in California. For these reasons, we recommend an independent outcome evaluation of California's SDM project at a cost of approximately $500,000 ($125,000 General Fund) in 2001-02.

Conclusion

Above, we present recommendations for the Legislature that would expand and improve SDM in California. We believe existing research on SDM justifies the expansion to the ten counties on the waiting list. At this time, we recommend deferring a decision on further expansion of SDM until a California-specific evaluation has been completed.

The CWS/CMS Needs Strategic Plan

We recommend that the Child Welfare Services (CWS) Stakeholders' Group develop a strategic plan for the Child Welfare Services/Case Management System (CWS/CMS) as a part of its review of the CWS system. We further recommend that, after 2001-02, the Legislature deny funding for any CWS/CMS modifications until the strategic plan is completed.

Background. Pursuant to the 2000-01 Budget Act, the CWS Stakeholders' Group was established and funded for up to three years. Coordinated by DSS, the group was established to (1) review existing CWS programs, components, and systems; and (2) provide recommendations for improvements. The group is composed of approximately 60 members, including county, state, and federal government professionals; advocates; researchers; legislators; and former recipients of CWS. The CWS Stakeholders' Group plans to submit the following: (1) initial recommendations regarding immediate CWS improvements to the Director of DSS by June 2001, (2) progress reports on the implementation of action items beginning June 2001, and (3) an evaluation plan to measure progress toward objectives by October 2001.

Automation System. The CWS/CMS provides a statewide database, case management tools, and reporting system for the state's CWS program. The system is in operation in all 58 counties. The system has the potential to provide (1) more accurate, comprehensive, and timely information on which to base child welfare decisions; (2) key workload data and statutorily required information to managers; and (3) improved worker access to intercounty information.

While the system has now been implemented statewide for several years, the federal government and independent consultants have noted that CWS/CMS continues to be used inconsistently across the state and that barriers to more effective implementation exist. Because there is no program-level strategic plan for the CWS/CMS, changes and enhancements to the system have been authorized and funded in a fragmented fashion, sometimes without regard for statewide benefit.

Recommendation. Given its broad mandate to overhaul the CWS system, we believe the CWS Stakeholders' Group is well-positioned to provide direction on the program's future automation needs. Therefore, we recommend that the CWS Stakeholders' Group develop a five-year strategic plan for CWS/CMS. A long-range CWS/CMS strategic plan would connect the ongoing efforts of the CWS Stakeholders' Group to improve the delivery of child welfare services with the potential benefits of CWS/CMS. In addition, a strategic plan designed with CWS programmatic expertise would provide a framework in which to evaluate the costs and potential benefits of additional changes to CWS/CMS. Accordingly, we further recommend that the Legislature, after 2001-02, not approve any funding for CWS/CMS modifications until the strategic plan is completed.