The 2017-18 Budget

The New IT Project Approval and Funding Process

---

• Introduction
• Background
• Prior Project Approval Process Created Challenges
• What Is the New Project Approval Process?
• How Does the New Project Approval Process Address Shortcomings of Prior Process?
• What are The Potential Trade‑offs and Implications of the New Approval Process?
• What are the Legislative Decision Points Under the New Project Approval Process?
• Issues For Legislative Consideration

---

Introduction

Technology Can Improve State Operations and Delivery of Services. Technology has the potential to improve how Californians interact with government—making this interaction more efficient, reliable, and convenient. Strides in private sector technology allow the public to access tools that improve their daily lives—from online portals that facilitate communicating with doctors to mobile applications that schedule restaurant reservations. The public has come to expect a similar level of service from government. The state is currently undertaking numerous information technology (IT) projects that are intended to increase the quality of services provided to the public and improve the efficiency of state programs. (When an IT project is completed and fully operational, it is then referred to as an IT system.) Specifically, the state currently has about 30 IT projects in various phases of development that are approved by and under the oversight of the California Department of Technology (CDT), the state’s central IT organization. The total cost, should the state complete all of these IT projects as currently envisioned, is estimated to be about $3.2 billion.

State Has Had Challenges Implementing IT Projects. The state has experienced considerable challenges successfully implementing technology projects. While there have been some project successes, there have also been various high‑profile state IT project failures. These failures have resulted in either project suspension or termination and have received considerable legislative and media attention. In other cases, projects have been ultimately completed, but only after significant cost overruns and multiyear delays.

State Has Initiated a New IT Project Approval Process. With so much at stake, the Legislature has looked to CDT to determine what changes are necessary to ensure that IT projects are successfully completed. While there are various factors that can contribute to project challenges, one such factor is poor project planning. (Please refer to our May 2015 report, The 2015‑16 Budget: Centralizing State IT Project Management, to lean about CDT’s efforts to address another major contributor to project challenges—poor project management.) In recent years, CDT has begun implementing a new IT project approval process—known as the Project Approval Lifecycle—with the goal of helping to bolster project planning and reduce the likelihood of project challenges or failure.

In this report, we discuss CDT’s responsibilities, describe the challenges with the prior project approval process; describe the new project approval process; discuss the potential benefits, trade‑offs, and implications of the new process; identify the key legislative decision points within the new approval process; and identify issues for legislative consideration.

Back to the Top

Background

What Are the Responsibilities of CDT?

CDT, a department within the California Government Operations Agency, is the state’s central IT organization and has broad authority over all aspects of technology in state government. Below, we describe CDT’s role in the approval and oversight of state IT projects.

Approves and Oversees IT Projects. One of CDT’s responsibilities is to review and approve IT project proposals developed by a “sponsoring department”—that is, the state department undertaking the IT project. (We discuss below the concept of a “reportable” IT project, meaning projects that are subject to CDT’s approval and oversight authority.) CDT evaluates department proposals to ensure that proposed projects: (1) are based on well‑defined programmatic needs, (2) consider feasible alternatives to address the identified needs, (3) identify a sound technical solution, (4) implement project management best practices, and (5) comply with state policies and procedures, among other CDT considerations. Once CDT approves a department’s project proposal, its role changes to one of providing project oversight. Specifically, CDT provides an independent review and analysis of the project to determine if it is on track to be completed on schedule and within budget, and whether it will provide the benefits identified by the sponsoring department. As part of this review, CDT routinely reports to sponsoring departments on issues of concern that it has identified, shares lessons learned from other projects, and recommends risk mitigation and issue resolution strategies.

Reviews Revised Project Plans, as Necessary. Over time, a project may change in scope or deviate from the schedule and/or cost that was established during the approval process. Any significant changes to the project plan are documented and justified in Special Project Reports (SPRs). The revised project plans are developed by the sponsoring department and submitted to CDT for its review and approval. Approval of an SPR constitutes a new agreement between the sponsoring department and CDT. This process resets the scope, schedule, and cost from which the project’s progress and performance are assessed. Once CDT approves the SPR and the Legislature approves the associated funding, the department can move forward with the project based on the revised plan and CDT continues its oversight role. In some cases, projects change considerably and several SPRs are required over the life of the project.

Suspends, Terminates, and Reinstates IT Projects. As part of its project approval and oversight responsibilities, CDT has the authority to suspend, terminate, or reinstate an IT project based on its performance. CDT also has the authority to hold departments accountable for poor performance, including by restricting future project approvals pending demonstration of successful correction of the identified performance failure.

What Projects Are Subject to CDT’s Approval and Oversight Authority?

Only “Reportable” IT Projects. The most significant state IT projects are considered to be reportable and therefore subject to the approval and oversight of CDT. In contrast, non‑reportable projects are completely within the authority of sponsoring departments to manage. Projects that meet one or more of the following characteristics are reportable:

• Estimated project cost exceeds the sponsoring department’s cost threshold assigned by CDT. The assigned cost thresholds generally range from $200,000 to $2 million, and usually reflect CDT’s assessment of the department’s performance in previous projects.
• Projects with costs under the department’s assigned cost threshold but for which the costs are not absorbable by the sponsoring department and therefore require an appropriation by the Legislature.
• Projects that are specifically mandated by the Legislature.

For What Period Does CDT Oversee Projects?

CDT continues to oversee projects until the sponsoring department has submitted a Post‑Implementation Evaluation Report (PIER)—a report that details whether and how the project objectives were accomplished, documents lessons learned, and provides a final summary of actual versus expected costs—to CDT. Once the PIER is submitted, the project becomes an IT system maintained and operated by the sponsoring department and is no longer subject to CDT’s oversight.

Back to the Top

Prior Project Approval Process Created Challenges

Historically, when departments proposed IT projects, CDT required them to prepare Feasibility Study Reports (FSRs). The FSR identified the problem, evaluated alternatives, and identified a technical solution. The FSR was developed by the sponsoring department and submitted to CDT for review and approval. Various shortcomings with the FSR approval process meant that projects often experienced challenges once they were underway. These challenges were often associated with significant cost increases and schedule extensions. Below, we detail the primary challenges associated with the FSR process.

Cursory Business Analysis Lead to Unmet Needs. While the FSR required some analysis regarding the programmatic needs that motivated the proposed IT project, this “business analysis” was generally done rather superficially. As a result, some departments ultimately discovered a mismatch between the functionality offered in its newly implemented IT system and the needs of the system users—including department staff and/or members of the public. Departments then faced a difficult decision—they either had to make do with a new system that did not fully meet their needs and expectations or invest in potentially costly and time‑consuming system enhancements.

Limited Collaboration Between Sponsor Department and CDT Resulted in Poor Planning. Under the FSR process, departments undertook planning activities and developed FSRs largely independently of CDT and then submitted the completed analysis to CDT. Because departments largely planned projects without guidance from CDT, it was possible for a department to make an early erroneous assumption that jeopardized the quality of the entire planning effort. Even if CDT ultimately approved a sponsoring department’s FSR, it was difficult for CDT to identify potential planning deficiencies because it generally had not been involved in the planning process. Deficiencies in the planning generally manifested during the design, development, and implementation (the system deployment) of the project, when issues are significantly more challenging to resolve.

Approval Process Did Not Incorporate Knowledge From Project Vendors. Once a department completed an FSR, it would submit it to CDT for approval. Once CDT approved the project, the FSR and budget‑related requests would be submitted to the Legislature for approval and funding. Under this former project approval process, procurement occurred after CDT approved a department’s FSR and the Legislature approved initial project funding. (Procurement is the process through which departments work with potential project vendors to plan for the purchase of technology for IT projects and select vendors to install the technology.) Generally, because departments learn more about the actual cost of achieving their technology objectives through interactions with potential project vendors through the procurement process, the timing of procurement typically resulted in unanticipated cost increases, and schedule extensions beyond those established in the FSR. Departments would have to submit a revised project plan—an SPR—soon after a vendor was selected to update the project baseline, scope, schedule, and cost.

The Legislature Would Approve Projects Based on Incomplete Planning. Historically, the Legislature authorized funding for an IT project based on the baselines established in the FSR. However, given the shortcomings with the FSR approval process, it was common for departments to discover the project would be costlier and take longer to implement than previously anticipated. Projects frequently required several SPRs to revise the project scope, schedule, and/or cost once it was underway. The SPRs were sometimes problematic because the Legislature had approved an IT project based on expectations that were laid out in the FSR, but the final project may have ended up with significant scope, schedule, and/or cost differences. (We note that SPRs can reflect reasonable changes that can improve the quality and ultimate success of a project. SPRs do not always reflect poor project planning and/or management.) Refer to the nearby box for an example of a large IT project that has evolved considerably since its FSR was approved in 2005.

Limited Ability to Track Overall Project Success. SPRs reset the baselines from which CDT tracked a project’s progress and performance. This meant that, at its conclusion, a project’s performance was largely evaluated relative to the last approved SPR, which generally aligned closely with the actual project scope, schedule, and cost, but potentially deviated significantly from the approved FSR. Additionally, “project success” was not a formally defined concept. Project performance was tracked based on easily quantifiable measures, such as the number of months behind schedule or percent over budget the project was compared to its most recent SPR. However, those measures excluded more difficult to assess but more important measures of success, such as how well the IT system worked for its intended users. The absence of a clear and commonly accepted definition for project success, and the inability to track project performance because it was constantly evaluated based on reset baselines, limited CDT’s and the Legislature’s ability to gauge overall project success.

Back to the Top

What Is the New Project Approval Process?

In response to the issues with the FSR approval process described above, in 2016 CDT fully implemented a new IT project approval process—Project Approval Lifecycle (PAL). It divides CDT’s approval process into four stages—business analysis, alternatives analysis, procurement analysis, and bid analysis and finalization of project details. Each stage (1) requires sponsoring departments to conduct specific planning‑related analyses and submit an associated planning document to CDT and (2) provides CDT with a discrete decision point in its approval process. Upon review of the planning document associated with each stage, CDT can offer sponsoring departments one of three decisions:

• Approve. Approval of the planning document associated with the stage means CDT finds it to be of adequate quality and the sponsoring department is authorized to proceed to the next PAL stage.
• Reject. Rejection of the planning document associated with the stage means CDT finds it to be of inadequate quality and the sponsoring department is not authorized to proceed.
• Rethink and Resubmit. This means the planning document associated with the stage is of inadequate quality but the project has merit. CDT recommends the sponsoring department rethink, revise, and resubmit the analysis for future consideration.

Sponsoring departments must secure approval from CDT for each of the four stages before the department can begin the IT project. In the following section, we describe the specific analysis associated with each stage.

Stage 1—Business Analysis. In the first stage of the PAL process, a department that is considering an IT project must first layout the issue that could potentially be solved by an IT project. This business case is centered on (1) the programmatic problems that substantially and adversely affect the operation and delivery of a service, (2) the programmatic opportunities that may substantially improve operation and delivery, (3) the expected revenue generation or cost savings, or (4) compliance with legislative mandate. In this first stage, sponsoring departments also document the project objectives and assess their readiness to take on an IT project. (All departments are required to submit Stage 1 planning documents regardless of whether the proposed projects are anticipated to be reportable or non‑reportable. However, non‑reportable projects with an approved Stage 1 planning document are not required to proceed with the subsequent PAL stages. Instead, departments undertaking non‑reportable projects are internally responsible for the success of their own projects.)

Stage 2—Alternatives Analysis. The second stage requires departments to evaluate various alternatives for accomplishing the project objectives identified in Stage 1. Based on this analysis, departments identify the recommended alternative and develop a procurement strategy. Departments often rely on market research to gather the information necessary to successfully complete this stage. This is the first time in the PAL process that CDT requires sponsoring departments to provide a financial analysis for the project, including a comparison of the cost of not implementing a new IT system—that is, maintaining existing technology or manual processes—to the various alternatives.

Stage 3—Procurement Analysis. The third stage requires departments to identify the detailed requirements for the project based on the recommended alternative selected in Stage 2 and develop a solicitation—a request for information from vendors. The solicitation documents the project requirements, terms, and conditions.

Stage 4—Bid Analysis and Finalization of Project Details. The fourth stage requires departments to release the solicitation developed in Stage 3. Prospective vendors use the solicitation to develop their bid for an IT project. The department evaluates the bids that respond to the solicitation and selects a vendor. The Stage 4 planning document also outlines the final project details, including the project scope, schedule, cost, and resource needs. These project details serve as a baseline for monitoring the project’s progress and performance moving forward.

Completed PAL Process. Each stage in the PAL process builds off the analysis from the prior stage. Collectively, the planning documents from the four stages create a comprehensive plan for implementing the proposed IT project. Departments cannot begin their projects without receiving approval from CDT for each of the four stages. Because this process is still relatively new, it is uncertain how long it will take for sponsoring departments to complete the PAL process. In some cases, a department may be able to move through the four stages in a single fiscal year. In others cases, planning may extend across several fiscal years. The duration will likely be affected by (1) the sponsoring department’s capacity for planning IT projects, (2) the complexity of the project, and (3) CDT’s workload. Refer to Figure 1 for a visual depiction of the new project approval process.

Back to the Top

How Does the New Project Approval Process Address Shortcomings of Prior Process?

CDT designed the PAL process to improve the planning quality and increase the likelihood of success for IT projects undertaken by the state. The new process attempts to accomplish these goals by addressing some of the shortcomings of the FSR approval process.

Requires Collaboration Between Sponsor Department and CDT During Project Planning Stage. The PAL stages require departments to collaborate with CDT at various points during the planning process. Should CDT identify deficiencies during the planning process, it can communicate its concerns and recommended solutions early to sponsor departments. This structured and guided approach towards planning and approving IT projects may lead to a better planned project and bolster the likelihood of successful project implementation.

Emphasizes Business Analysis. The new project approval process emphasizes due diligence in defining the business case before an IT solution is selected. As discussed previously, projects approved via the FSR process often resulted in a mismatch between the programmatic needs and the functions available in the IT system. The new project approval process intends to address this issue by requiring a detailed business analysis in Stage 1. Departments are required to consider the programmatic needs that motivated the project proposal and the project objectives that most serve the interest of the affected programs and its constituents.

Approval Process Incorporates Knowledge From Working With Vendors. Unlike the FSR process, PAL incorporates procurement activities into the project approval process. When the project is fully approved under the new process, the administration and the Legislature should have a better understanding of, and confidence in, the project schedule and cost because it reflects the vendor selection. In this way, the new process not only could provide more reliable information, but also moves projects further along in the planning and development process relative to the FSR process.

Increases Likelihood That Final Project Aligns With Expectations. The new process should provide for more accurate cost and schedule baseline estimates because of the detailed analysis required by PAL, collaboration with CDT, and of the movement of procurement into the project approval process. Ultimately, it allows departments to refine their plans and analysis collaboratively with CDT to arrive at more accurate estimates and sound project plans at the time of project approval. With more accurate cost and schedule baselines there should be far fewer SPRs.

New Process May Save Time and Money in Long Term. Because of the hoped‑for higher quality upfront planning in the PAL process, departments may save time and money over the life of the IT project. Under the FSR process, departments often had to course correct when a major issue was discovered or the project fell behind schedule. Project staff were often redirected to problem solve the issue and then tasked with updating the project plan and securing approval from CDT through the SPR process. Diverting resources away from project execution activities and towards issue resolution and the SPR process can be costly and time consuming. A sounder project plan that mitigates the need for course corrections during project development could help departments realize the benefits of their IT systems more quickly and reduce project costs. Figure 2 provides a comparison of the new and former project approval processes.

Back to the Top

What are The Potential Trade‑offs and Implications of the New Approval Process?

While the potential benefits of PAL appear clear, the new project approval process comes with trade‑offs and implications of which the Legislature should be aware.

New Project Approval Process Will Likely Take Longer Upfront. The new project approval process will likely take longer for departments to complete than the former FSR approval process. This is a natural consequence of making the project approval process more robust with the expanded requirements for detailed upfront analysis. Departments may need to call upon a significant number of programmatic and technical experts to successfully complete the detailed analysis CDT requires for project approval. In some cases, departments may leverage the expertise of contractors for specific planning‑related activities, such as identifying system requirements. This is especially true for departments that have limited experience implementing IT projects.

Some Departments May Seek Budget Augmentation for Planning Effort. Under the FSR process, departments largely absorbed the cost of planning an IT project. However, since the PAL process requires more detailed analysis upfront and includes new activities (mainly procurement) that previously occurred after approval, some departments may request a budget augmentation to support the effort. These budget requests may seek staff resources and/or contract services. CDT requires departments to have an approved Stage 1 planning document before a department submits a budget request for any of the remaining three stages. Essentially, CDT requires departments to absorb the cost of Stage 1. The Governor’s 2017‑18 budget includes several proposals that request funds to support various stages of the new project approval process. (Please refer to the box below for examples of some of the budget requests to fund PAL‑related activities.)

PAL Process May Not Align With Budget Cycle. It is uncertain how long it will take departments to move through the entire PAL process. The duration depends on several factors, including the complexity of the proposed project and the sponsoring department’s experience planning IT projects. It is possible that the stages of the PAL process will not align with the state’s traditional budget process. For example, a department may receive approval from CDT for Stage 4 midway through a state fiscal year. The sponsoring department may not want to delay the project start for several months until a budget request could be secured for the following fiscal year. If a department anticipates that development activities could occur in the same fiscal year that Stage 4 is anticipated to be complete, the department could request funding to support development activities before the Stage 4 analysis is approved by CDT. In this scenario, however, the Legislature could be asked to approve funding for project design, development, and implementation without the benefit of a complete project plan. This could compromise the Legislature’s effective budgetary oversight of the project.

Measuring Project Success Remains Unclear. The PAL process should increase the quality of upfront planning and result in more accurate cost and schedule baseline estimates than the FSR process. However, PAL does not ultimately guarantee project success. This is because CDT has still not formally defined project success. Evaluating success can be complicated. For example, if a project runs over budget and takes longer to implement than anticipated, but the IT system works effectively for its users, is the project considered a success? Alternatively, if a project is completed on time and on budget, but the IT system does not effectively meet the needs of users, is the project considered a success? Evaluating project success likely involves a combination of quantitative measures and, harder‑to‑evaluate, qualitative measures.

Unclear How PAL Will Align With Other IT Process Changes. At the same time the state is implementing the PAL process for IT project approval, it is also making other IT process related changes. Most notably, departments are beginning to consider developing and deploying their IT projects in increments rather than all at once as one big project. The Legislature will need to make sure it understands how any new project development and deployment methods work with the PAL process.

Back to the Top

What are the Legislative Decision Points Under the New Project Approval Process?

The Legislature has two key decision points under the new project approval process: (1) whether to fund planning activities associated with the PAL process for proposed IT projects and (2) whether to fund project design, development, and implementation for projects ultimately approved by CDT. Additionally, like in the FSR process, the Legislature will retain its oversight role of approved projects, which typically includes decisions regarding future changes to the project. Figure 3 depicts the legislative decision points under the new project approval process and through the life of the project. We discuss the legislative decision points below.

Some Departments May Request Funding for Project Planning. As described earlier, in some cases, sponsoring departments may request funding to support the planning activities associated with the new project approval process. Departments may request funding from the Legislature for activities associated with all but the first stage of the PAL process. The project approval process may span across multiple state fiscal years, and the Legislature may see more than one budget request over the planning period for specific PAL activities. The Legislature will consider the merits of these requests through the traditional budget process. Some departments may continue to absorb the cost of planning IT projects. In these cases, the Legislature will not learn of the project until Stage 4 is complete.

Departments Will Request Funding for Project Design, Development, and Implementation. Once CDT approves the project (after Stage 4), the sponsor department will request funding to begin designing, developing, and implementing the IT project. This will be an opportunity for the Legislature to review the complete project plan and determine if the project objectives and approach have merit given the cost, schedule, and the Legislature’s own priorities. Typically, the Legislature approves funding for IT projects on a year‑by‑year basis. This provides assurances that the sponsoring department will return in a subsequent fiscal year with a status update and a request for additional funding. Should project challenges arise, the department would be accountable to the Legislature regarding how the challenge developed and steps to address and mitigate future challenges before additional funding was approved.

Legislative Role and Decision Points Following Project Approval. After a project has been approved and funded, the Legislature has a key oversight role regarding the state’s IT project portfolio. It can require periodic updates regarding the status of IT projects of key interest or projects that have experienced prior challenges. The Legislature also has the opportunity to review budget proposals associated with revised project plans—SPRs. Through this budget review process, the Legislature has the opportunity to assess if the project changes are reasonable. Near the conclusion of an IT project, the Legislature often considers requests for ongoing funding to support the maintenance and operation of the IT system.

Issues For Legislative Consideration

Although the PAL process has the potential to improve the quality of IT project implementation in theory, we raise a number of issues for the Legislature to consider as it exercises oversight of this new process.

In Some Cases, Funding for Planning May Have Merit . . . In some cases, departments may request significant funding to support the PAL process. While we discussed the potential benefits of providing resources in the planning effort—a sound project plan may make successful project implementation more likely—it is possible that the proposed project ultimately does not move forward into development and implementation and the funding does not actually lead to a project. A project may not move forward for several reasons, including because the department decided not to pursue the project, CDT did not approve all four stages of the PAL process, or the Legislature rejected the budget augmentation for project development and implementation.

The merits of providing funding for these project planning proposals should be determined on a case‑by‑case basis. When determining whether to support a request for planning funds or require a department to absorb the cost of planning a proposed IT project, the Legislature may want to consider such things as:

• The sponsoring department’s performance/experience planning and implementing IT projects in the past, which may inform how much support a department may need fulfilling the PAL planning requirements.
• The complexity of the proposed project—more complex projects typically require more complex planning.
• Whether the project is a priority for the Legislature.
• Whether the project is necessary to comply with federal laws or regulations, especially if noncompliance may jeopardize federal funding.
• Whether the Legislature is comfortable approving a budget request that ultimately may not result in a new IT system.

. . . And Gives Legislature an Opportunity to Weigh in Early. When sponsoring departments request funding for PAL‑related planning activities, it presents the Legislature with an early opportunity to weigh in on its own priorities for the project. Under the FSR process, the Legislature was not informed of a potential project until the planning process was complete. A request for planning funding for the PAL process provides the Legislature with an earlier opportunity to learn about projects that departments are considering. If the Legislature has certain priorities it would like reflected in the project, it could build in requirements as part of its budget approval that ensure the department considers those priorities. For example, the Legislature could require departments to consider specific alternatives during Stage 2, include certain functionality in the solicitation developed during Stage 3, or in other ways ensure the system planning reflects its priorities.

Legislature May Need to Build in Additional Oversight Methods. When the PAL process does not neatly align with the budget cycle, the Legislature may need to build in additional oversight methods. The planning documents developed during the four PAL stages offer the Legislature critical information necessary to evaluate the merits of the proposed IT project. The Legislature would need to consider the trade‑offs between authorizing project funding without a complete project plan versus delaying the implementation of a proposed IT project. If delaying implementation would have significant negative consequences, such as preventing a department from meeting a statutorily established deadline, the Legislature might consider other options that build in its own approval and oversight role. For example, the Legislature could consider using provisional budget bill language to withhold the portion of the authorized budget for project development until the department meets certain conditions, such as submitting an approved Stage 4 planning document.

Actual Benefits of New Project Approval Process Should Be Evaluated. Several years often elapse between project approval and full system implementation. This lag will make it difficult for CDT and the Legislature to determine early on if the proposed potential benefits of the PAL process are realized. Additionally, to fully assess if IT projects are implemented more successfully under the PAL process than under the prior FSR process, the evaluation of several projects that were approved and fully implemented since PAL went into effect in 2016 would be needed. An early indicator that PAL may not fully realize its anticipated benefits could be if SPRs continue to be common among IT projects approved through the PAL process. However, for now, the Legislature may want to direct CDT to report at budget hearings on: (1) whether there are additional changes to the project approval process that may further improve the likelihood for project success and (2) the quantitative and qualitative measures it will use to evaluate the effectiveness of PAL and project success. The latter could ensure that CDT, the Legislature, and IT system stakeholders can speak to the performance of IT projects using a common definition of success.

---