November 11, 2009

Try Before You Buy: Expanding Multi–Stage Procurements for Large IT Systems

Executive Summary

(Short video summary on our website or on YouTube)

The state has several vehicles for procuring information technology (IT) goods and services. However, it has largely limited itself to one traditional procurement approach for large automation projects. This approach, called the firm fixed price (FFP) procurement, creates a prescriptive process that gives equal footing to all vendors to help ensure open competition and reduce the state’s exposure to protests and potential lawsuits due to perceived vendor bias.

In many cases, the FFP approach may be the most appropriate strategy. However, blanket use of it for all large automation projects overlooks important differences among projects’ characteristics and needs that have sometimes resulted in significant project delays and cost overruns. For this reason, the state is beginning to look to the multi–stage procurement as an alternative approach for developing its more complex IT systems. The key feature of the multi–stage procurement approach is that it creates a collaborative environment for state and vendor staff as they work together to build a responsive solution to the state’s business needs.

Because a strategic and well–run procurement can mitigate some of the problems that develop later in a project, we recommend that the Legislature require state entities to include procurement strategies in project documents when they submit them for approval. These documents would indicate whether FFP, multi–stage, or another approach will be followed, and the justification for that choice. The Legislature’s interest in procurement will encourage all projects to carefully consider the best procurement approach for their particular needs early in the planning process. Our analysis indicates that an earlier evaluation of procurement strategy can reduce the risks of cost overruns and schedule delays while increasing the likelihood that complex systems integration projects will be successfully developed.

Introduction to IT Procurement

According to the Office of the State Chief Information Officer (OCIO), California’s IT budget for 2007–08 was an estimated $3 billion ($1.7 billion from the General Fund). This includes expenditures for IT staff salaries, equipment, facilities, as well as IT system development and maintenance. Contracts for IT purchases cost the state roughly $1 billion annually, a sizeable component of the state’s IT spending. This report examines some of the state’s procurement practices for acquiring IT products, with a focus on procurement for California’s large automation systems. It looks at traditional procurement practices and considers alternative approaches and the value in being flexible in procurements for large automation systems.

The State’s IT Project Approval Process

Generally, an IT project goes through a review and approval process that begins with the development of a feasibility study report (FSR). (This process is depicted in Figure 1.) An FSR includes a proposed system’s business justification, project management plan, benefits, risks, costs, and schedule, among other elements, and is submitted concurrently to the OCIO and the Department of Finance (DOF). The OCIO reviews a project on its business and technology merits while DOF reviews the project’s budget. When the OCIO and DOF approve an IT project, DOF and the department usually develop a budget proposal that is then sent through the legislative budget review process. The Legislature has the opportunity to evaluate the project, either approving or rejecting it through the actions it takes on the budget proposal. Typically, upon legislative approval, funds may be allocated and the procurement process for an IT project may formally begin.

Information Technology (IT) Project Approval Process

IT Procurement Strategy Depends on Cost and Scope

Basic IT Goods and Services. About 95 percent of the state’s IT goods and services are acquired using Leveraged Procurement Agreements (LPAs). These are arrangements which allow the state to buy directly from prequalified suppliers through contracts negotiated by the Department of General Services Procurement Division (DGS PD). Both IT equipment (such as monitors, servers, printers, hardware, and software) and IT services (such as Web design consulting, document conversion, and application development training) are typically acquired through LPAs. There are various LPAs and each individual procurement is capped at a different amount. For example, the California Multiple Award Schedules offer goods and services through contracts that DGS PD has assessed to be fair and reasonable and are under $500,000, while Master Agreements are contracts competitively bid by DGS and are under $1.5 million. The LPAs leverage the state’s buying power and help streamline the procurement process for state entities, reducing the overall cost, effort, and time for purchases. Nearly all IT products purchased through LPAs are completed within six months.

Complex Systems Integration Projects. While LPAs are adequate for the vast majority of the state’s IT purchases, another procurement vehicle is necessary when purchasing goods and services for complex systems integration projects. Generally, complex systems integration projects require software that must be integrated or configured to perform specific program and business functions for the state’s large automation systems. The Statewide Automated Welfare System, the Child Support Automated System, and the Automatic Collection Enhancement System are examples of large automation systems. Complex system integration projects can take years to complete, require a large supply of state and contracted staff resources, and costs almost always exceed LPA caps. For example, contracts for software and/or vendor services to configure software can range from the low millions to the high tens of millions. In some circumstances, the costs can even reach the hundreds of millions of dollars, as was the case for the Child Support Automated System, which required nearly $1 billion for prime vendor services.

While contracts for large systems integration projects make up only a tiny fraction of all IT contracts (about 5 percent), the state spending on them is sizeable (about 35 percent of the state’s IT contracting budget, or about $350 million annually). For complex systems integration projects where LPA caps are ordinarily exceeded, the state traditionally has followed an FFP procurement strategy for acquiring IT goods and services.

The Firm Fixed Price Procurement

Customary Steps in an FFP Procurement

All approved projects must first develop (often with the assistance of DGS PD or external consultants) a request for proposals (RFP) to elicit bids from interested vendors. The RFP includes, among other things, the business requirements, or the business goals that the proposed system must meet, and the technical requirements, or the technology standards and environment around which the proposed system must be designed. When these requirements are detailed and well–documented, potential bidders have more information so they may reasonably estimate the size, scope, complexity, and cost associated with developing an IT solution for the state. Vendors then submit proposals, which are to include a technical solution for system development along with an FFP for that solution. The DGS PD works with project staff to review and evaluate proposals using a set of criteria that are established in advance for the project and are included in the RFP. After evaluating the FFP proposals, DGS PD and project staff choose the proposal considered the “best value” for the state from among these proposals. (We note that the state can select IT goods and services based on a best value evaluation rather than purely on the lowest cost. This is an important distinction from other state–contracted goods and services where the “low bid” generally must be awarded the contract.) The FFP process is depicted in Figure 2.

Major Steps in a Firm Fixed Price Procurement

When to Use the FFP Approach

The U. S. Office of Management and Budget (OMB), through its Capital Programming Guide, has issued advice on when to consider using particular procurement approaches. The OMB guidance, along with best practices in private sector procurement, suggests there are certain conditions when an FFP approach ought to be considered. The office has concluded that the FFP approach typically offers the most benefits when (1) a product or proposed solution is available in the market for purchase or (2) system requirements can be well–documented. We elaborate on these criteria below.

Solution Is Available in the Market. When the solution is deemed to be available in the market, this means that any of a number of vendors could provide the solution and the services to implement it. Under these conditions, potential vendors can reasonably estimate the cost for their solution and offer a competitive FFP proposal. The state, in turn, can evaluate proposals largely on the basis of their cost. Ideally, this procurement would attract multiple competing vendors, each with the incentive to offer a competitive cost proposal, since each knows that other vendors could provide the market solution and win the bid with a lower price. In general, increased competition drives down the total price of system development for the state.

System Requirements Are Well–Documented. Many of California’s complex systems integration projects require a technology solution that is targeted to meet the needs of a particular department, agency, or state program—a product that often is not readily available in the market. However, for projects for which the state can document well the relevant business and technical requirements, utilizing the FFP approach often remains a good option. Under these circumstances, potential bidders have enough information from the detailed requirements to reasonably estimate the size, complexity, and costs for developing a solution and offer competitive proposals. The nearby text box about the VoteCal project provides an example of a system solution that is both available in the market and whose requirements are well–documented.

Strategic Application of the FFP Approach: The VoteCal Project

The Secretary of State’s (SOS) VoteCal project provides a good example of ideal conditions for using the firm fixed price (FFP) approach. The VoteCal project now under development will provide a statewide automated voter registration system as mandated by the Help America Vote Act (HAVA) of 2002 that interfaces with other state information technology systems to verify the identity of voters. This is a large and complex project, especially given that voter registration takes place in the 58 counties. California is one of the last states to develop a system to meet the federally mandated requirements. As such, SOS staff had the advantage of learning from other states’ successes and mistakes during the planning, development, and deployment of similar systems across the country. This contributed to SOS’s ability to build solid technical and business requirements in the request for proposals. The VoteCal project was able to attract multiple bidders who had participated in the development of HAVA systems in other states. Three submitted competitive FFP proposals, a good showing for such a complex systems integration project.

Advantages of the FFP Approach

There are some distinct advantages to using the FFP approach. It creates a prescriptive process that gives equal footing to all vendors. This enables a transparent and level playing field for bidders and helps ensure open competition. This typically reduces the state’s exposure to bid protests and potential lawsuits based on allegations of vendor bias. When feasible, the FFP approach increases competition and can decrease the overall costs for system development.

When the FFP Approach Is Risky

Despite its potential benefits in some circumstances, the standard FFP approach may not always be the ideal way to acquire IT goods and services. It often becomes problematic when the state cannot provide detailed information on requirements to vendors so that they can reasonably estimate the complexity, scope, and cost of a solution. Although it may be difficult to imagine that the state could not well document its own requirements, this is typically the case for complex systems integration projects. Such systems might be expected to meet multiple business and program goals; span departments, agencies, and multiple counties; and interface with other IT systems and programs. Additionally, the volume of data the proposed system will need to handle may be massive. Finally, new proposed systems may not exist to draw comparisons with similar projects in other jurisdictions for building requirements (as was the case with the Secretary of State [SOS] VoteCal system discussed in the earlier box).

Potential Consequences for The State Are Serious

When system requirements are perceived as vague or ambiguous, there are risks for the vendors that could lead to serious consequences for the state. For example, vendors may encounter difficulty in interpreting the RFP requirements. From the vendors’ perspective, having to “guess” how to meet the state’s needs adds risk to the project, and raises the possibility that their interpretation will differ (perhaps wildly) from the state’s expectations for system performance. Because of the risk that they may be held responsible for any problems, vendors may unnecessarily increase the cost of their proposals to account for potential issues that may surface later as they begin system development and learn more about the state’s actual needs. Additionally, this situation also creates a risk that a vendor who has been awarded a contract cannot meet the state’s needs. There are potentially serious consequences for the state should these risks materialize. Among the major potential risks are:

Unfortunately, under the traditional RFP process, many of these issues may not come to light until late into system development, after the state has invested a sizeable amount of dollars and effort (see examples below). At that point, the Legislature may have limited flexibility to alter the course of the project.

FFP Approach Mismatched to Complex Projects—State Examples

Below, we discuss two instances in which the state pursued the FFP approach on complex systems integration projects that either required complicated solutions not available in the market or had project requirements that were not easily defined. In both cases, the FFP approach failed to provide projects with a vendor that was able to complete the project on time and within budget.

The State Controller’s Office’s (SCO) 21st Century Project is intended to be a modern, fully integrated human resources (HR) management system that will replace several of the state’s antiquated HR management systems. Among other functions, the new system is to perform payroll, benefits administration, and employee self–service (a function that would give employees the ability to view and maintain their HR and payroll information).

After a 12–month procurement, the SCO received two proposals and, after an evaluation process, contracted with one of the vendors. Early in project development, the vendor began charging the state additional fees for work it claimed was outside the scope of the RFP—a claim SCO refuted. Additionally, the vendor swapped out experienced staff for less experienced staff and missed major milestones toward project completion. The state terminated the contract citing the vendor’s failure to meet contractual commitments. At that point, SCO had paid about $25 million (of the $70 million contract) in vendor payments and the project was two years behind schedule.

The Statewide Automated Child Support System was a federal– and state–mandated system that was intended to provide automated child support enforcement tracking and monitoring capability. Begun in the early 1990s with a $150 million budget, the state canceled the contract with the integration vendor in 1997, after spending over $100 million. This initial attempt was followed by a second attempt where the FFP approach was once again utilized. After more than two years in procurement, the state received only one viable bid for a $1.3 billion solution. Eventually, a statewide automated system was deployed at a total cost of $1.6 billion.

In these circumstances, the FFP approach may have provided an equal playing field for vendors, but it did not increase the total number of actual bids, screen out ill–qualified vendors, or assist the state in defining its business needs. An alternative procurement approach allowing more communication between the state and vendors could have shed light on the weaknesses (and strengths) of potential vendors and helped clarify vague or poorly written requirements prior to executing a contract and building the system.

An Alternative Approach—Multi–Stage Procurement

As discussed above, the FFP approach may pose serious risks to some complex systems integration projects. This suggests the state must be flexible and consider alternative procurement strategies. Currently, the state is investigating one such strategy called multi–stage procurement.

Customary Steps in a Multi–Stage Procurement

As its name suggests, a multi–stage procurement is a single procurement divided into multiple stages. (For simplicity, this report describes a two–stage procurement, although three or more stages are possible.) During stage one, the state releases an RFP to solicit proposals from interested vendors, much like under the FFP approach. However, rather than contract with one vendor, the state procures the services of two or three vendors, the actual number depending on the financial and staff resources of a project. These vendors will move on to stage two in which they each will be awarded a lump sum of dollars (an amount established previously in the RFP) to compete against each other. This competition is often referred to as a “bake–off.” The bake–off requires that each vendor build a smaller version of their proposed solution, called a “proof of concept” to (1) prove their understanding of the state’s business goals and (2) convince the state that theirs is the best solution. Competing vendors must also submit their proposals for developing the entire system during the end of the second stage. Each vendor’s proof of concept and proposal are evaluated and scored on criteria spelled out in the RFP. The vendor with the highest score during the bake–off “wins” the contract to build the complete system. See Figure 3 for a more detailed depiction of the process at each stage.

Major Steps in a Multi-Stage Procurement

Advantages of a Multi–stage Procurement

There are many potential advantages to a multi–stage procurement, which we describe in more detail below.

Increased Vendor Participation. Paying vendors “up front” a portion of what they will spend to build a prototype offsets some of their costs and may encourage greater vendor participation in the procurement process. This is particularly true for smaller vendors who may not otherwise have had the capital to participate. Greater vendor competition generally reduces state costs and can lead to more, and more diverse, proposals.

Prequalifications Sift Out Weaker Vendors. The multi–stage procurement sets up certain vendor criteria as a part of stage one, creating a kind of prequalification phase for vendors who wish to compete in the bake–off to potentially win the development contract. Only the most qualified vendors would typically receive high scores in stage one and move on to compete in the bake–off. (Although IT projects could in theory conduct prequalifications on potential vendors under the FFP approach, this generally does not occur due to the time and effort required of state staff.)

Enhanced Learning Opportunities. During the bake–off, vendor and project staff have the mutual opportunity to ask questions, raise concerns, and hammer out system requirements, a significant departure from the traditional FFP approach in which little two–way communication takes place because of procurement confidentiality rules. As a result, state staff are usually in a better position to assess and compare vendor capabilities and each vendor’s approach to meeting the state’s needs.

More Responsive Vendor Proposals. There are opportunities in the bake–off for vendors to learn about the state’s needs, potentially leading to more responsive and accurate proposals. The more accurate the technical and cost proposal, the less potential there is for unexpected costs and delays during actual system development.

Risks of a Multi–Stage Procurement

As with any procurement approach, the multi–stage procurement does carry risks that should be considered during a project’s initial planning.

Longer Procurement Schedule. The addition of stages can lengthen the total procurement schedule when compared to a traditional FFP approach. However, we note that having a shorter process “on paper” could be illusory. Spending more time planning during the early phases of procurement, as occurs under a multi–stage process, could prevent some of the problems that have traditionally burdened FFP procurements for complex systems integration projects. These problems have included delays due to revisiting poorly drafted or defined RFP requirements and their associated cost overruns.

More Upfront Costs for State. The state’s initial vendor payments could add considerably to initial development costs.

State Staff Could Be Spread Thin. Working with multiple vendors could strain state staff who are tasked with providing equal assistance to each vendor team to ensure a level playing field.

Vendor Withdrawal. One vendor could withdraw from the bake–off, potentially leaving the state with a one–vendor option at the end of procurement. This situation provides the state less leverage to negotiate proposal costs. We note that vendor withdrawal can also happen under the FFP approach. However, it is likely that the impact would be greater for a multi–stage approach because the state has invested more time and upfront dollars.

Single–Vendor Multi–Stage Procurements

The upfront costs of paying multiple vendors as part of the stage two bake–off competition could make it prohibitively expensive for some of the state’s smaller systems integration projects. However, some of these projects could still benefit from a multi–staged procurement approach using one vendor. Rather than contract with multiple vendors, the state could award a single vendor the opportunity to produce a prototype and subsequent proposal for the system development contract. In this type of procurement, the state and vendor would still have the opportunity to work closely with each other. At the end of stage two, if the state was satisfied with this vendor, it could proceed to award the contract. If, on the other hand, the learning experiences yielded information on the vendor or its approach that was negative, the state would not be bound to award the final contract to build the project.

Potential Concerns. Although the benefits of the multi–stage procurement are still in place under this modification, there are some potential disadvantages. First, the state could potentially end up with no vendor after an investment of time and money (although the state would own the vendor’s proof of concept). Another disadvantage is that the omission of the competitive aspect could result in a more costly proposal. The lack of multiple vendors means there is less pressure on each vendor to hold down costs.

Private and Public Multi–Stage Procurements

Private sector companies routinely use multi–stage procurements to acquire the best products for their needs relatively quickly and affordably. The federal government, through OMB’s Capital Programming Guide, likewise encourages the use of the multi–stage procurement and prototyping approach to acquire goods and services for government programs and IT systems. For example, the U. S. military often uses an incremental or staged approach for weapons and IT system development projects due to their large scale and costs. The U.S. Department of Health and Human Services has likewise utilized prototypes in a multi–stage procurement to work on developing a nationwide health information network.

Generally, multiple vendors are funded throughout the early stages to design prototypes, with more qualified vendors chosen to continue onto latter stages. Only one vendor will receive a contract to finish the product/system. (See the nearby boxes for further examples.)

While the multi–stage procurement approach has gained favor with the private sector and the federal government, it has been slow to catch on with state governments. In California, there is currently only one IT project using the multi–stage approach.

 

Netflix Prize: A Creative Example of a Private–Sector Multi–Stage Procurement

In 2006, the online DVD–rental service Netflix began a creative multiyear competition that sought a best movie recommendation algorithm or formula that would help the company better predict the movies viewers would like to see based on previously viewed movies. Such a formula would greatly increase movie rentals and thus boost the company’s profit margin. The winning entry received a grand prize of $1 million and Netflix intends to use the winning technology for its new “filtering” formula. Various teams entered formulas throughout the three years of competition. These formulas were scored on whether and to what degree they outperformed Netflix’s existing formula as well as each others’ formulas. Progress prizes of $50,000 were awarded in 2007 and 2008 to the best–performing teams until one team met the specified performance level (a 10 percent accuracy improvement from Netflix’s existing formula) in 2009. This process proved to be so successful that Netflix has already posted instructions for a Netflix Prize 2 worth another $1 million to further improve the algorithm.

A Department of Defense Multi–Stage Procurement

The Joint Strike Fighter program began in the late 1990s and would be the largest major aircraft effort in Pentagon history, replacing the nation’s aging fighter, strike, and ground attack aircraft. The Pentagon successfully used a multi–stage procurement approach to leverage key technologies for this acquisition which began with an initial phase in which three vendors submitted bids to participate in a competitive prototyping phase. Out of the initial phase, Lockheed Martin and Boeing were awarded individual contracts of $750 million each to develop prototype aircraft that met the requirements set forth by the U.S. government. After pilot testing the aircraft, defense officials selected the Lockheed F–35 prototype, whose technology outperformed Boeing’s X–32 prototype. The contract for the actual development was awarded to Lockheed in late 2001 and in spring 2009, the Secretary of Defense announced that the U.S. would buy 2,440 Joint Strike Fighters from Lockheed for about $80 million each.

Attempting the Multi–Stage Approach—A Second Try for the 21st Century Project

After the 21st Century project’s major setback with its original vendor, project staff opted to conduct a multi–stage procurement to acquire a new vendor. After reworking the RFP accordingly, including the creation of the criteria for each of two stages of the procurement, DGS released the RFP in spring 2009. Four reputable vendors submitted stage one proposals. After an evaluation process, the state awarded the top two scoring vendors $500,000 contracts each to participate in a stage two bake–off to work on a proof of concept and develop a final cost proposal. At the time this report was prepared, state staff had received and were evaluating the proposals with the intent of having the prime vendor on board by January 2010.

While the 21st Century procurement has not yet concluded, there are already signs that this approach is moving the project forward. One early indication is the increased vendor participation during stage one. Additionally, state staff found the increased communication with vendors throughout the procurement helped to define expectations and mutual responsibilities, and system requirements. Another positive indicator is that, at the end of stage two, project staff received two viable proposals, a good sign that both vendor teams felt they understood the requirements and business goals for the proposed system well enough to provide a reasonable solution.

Considering the Multi–Stage Approach—The Financial Information System of California. The Financial Information System of California (FI$Cal) is another project which intends to use the multi–stage procurement approach to acquire its software and prime vendor. When complete, FI$Cal is intended to fully integrate the state’s financial management system, interfacing with many other state IT systems. Currently, there are over 2,000 requirements which cover multiple financial functions within accounting, budgeting, and procurement. Project staff, recognizing the proposed system’s complexity, realized they could not provide the level of detail in every requirement for vendors to understand the sheer scope of the project, let alone assess the costs for system development to offer a reasonable cost proposal. Our analysis indicates that the decision to proceed with the multi–stage approach makes sense in this case because of its potential to drastically reduce vendor and state risks.

Encouraging Multi–Stage Procurements

The state has been reluctant to utilize alternative procurement methods, such as the multi–stage approach. While no state law explicitly prohibits California from conducting multi–stage procurements, vendors have expressed concern that Public Contract Code Section 10430, which restricts so–called “follow–on” contracts, casts a cloud over this procurement strategy. Specifically, the code states that no vendor who has provided consulting services to the state can later bid on a contract to provide the services they recommended. The purpose of Section 10430 is to prevent potential conflicts of interests—not to prevent multi–stage procurement. Although the state views these types of procurements as one process broken into multiple stages, many vendors have voiced concern that this interpretation is open to legal challenge. From their perspective, participation in these procurements could leave them vulnerable should a competing vendor sue the state on grounds that it had awarded a follow–on contract.

To address this issue, the Legislature adopted legislation as part of the 2009–10 budget package exempting the follow–on contract restrictions for multi–stage procurements. This change should create a friendlier environment for multi–stage procurements in the future. We will monitor vendor reaction to this change to see if it results in more multi–stage procurements.

Analyst’s Recommendation: Incorporate Procurement Approach into Legislative Review Process

Complex systems integration projects should not follow a one–size, fits–all approach. Instead, the unique characteristics of each project—its complexity, scope, size, and requirements—should help determine its procurement approach. As it reviews such projects, the Legislature should take into account various procurement options available to the state; their benefits, risks, and consequences; and the conditions under which one procurement approach would be more advantageous than another.

Establish Up–Front Legislative Review of Procurement Approach. In keeping with the approach outlined above, we recommend that the Legislature carefully consider each project’s procurement approach as part of the overall review process for IT proposals.

Currently, the Legislature focuses on an IT project’s business and program merits and weighs these against the estimated costs for building a system. The documents available for legislative review may include the FSR and/or accompanying budget requests. However, these documents do not as a rule include any discussion of a project’s procurement approach. Typically, project staff will develop a detailed procurement plan, which is submitted to DGS PD for review before the procurement process may begin. The Legislature, however, does not usually have an opportunity to review the procurement plans and weigh in on this decision.

Because a well–run procurement may mitigate many of the problems that could otherwise develop later in a project, we recommend the enactment of legislation requiring state entities to present the rationale for their procurement strategy for each IT project. For complex systems integration projects, the strategy should include consideration of the advantages and disadvantages of a multi–stage procurement approach. This strategy could be included in a project’s FSR and/or as part of the separate documents presented to justify budget requests to the Legislature. In other words, just as project documents provide the business reasons to justify a proposed new system, they should also include a high–level explanation for the chosen procurement approach.

Notification When Procurement Approach Would Change. To keep the Legislature up to date about procurement strategy, we further recommend that state agencies notify the Legislature whenever they modify the approved procurement approach for a complex systems integration project. This notification process could be implemented through the following proposed budget control language:

A state entity to which state funds are appropriated for an information technology project shall notify the Joint Legislative Budget Committee and the chairpersons of the appropriate policy committees of both houses of the Legislature of any changes to the project procurement approach as defined in the most recent legislatively approved project documents not more than 30 days after a formal decision has been made for that procurement change and before associated procurement documents are released to vendors.

This change would give the Legislature the opportunity to learn of any major procurement changes and to assess how they might affect a project.

 

Acknowledgments

This report was prepared by Erika Li and reviewed by Todd Bland. The Legislative Analyst's Office (LAO) is a nonpartisan office which provides fiscal and policy information and advice to the Legislature.

LAO Publications

To request publications call (916) 445-4656. This report and others, as well as an E-mail subscription service, are available on the LAO's Internet site at www.lao.ca.gov. The LAO is located at 925 L Street, Suite 1000, Sacramento, CA 95814.

Return to LAO Home Page