March 9, 2010

Hon. Kevin De León
Assembly Member, 45^th District
Room 2114, State Capitol
Sacramento, California 95814

Dear Assembly Member De León:

This responds to your request relating to California’s regulatory environment and AB 32 (Núñez), the Global Warming Solutions Act of 2006 (Chapter 488, Statutes of 2006). Specifically, you have asked that we analyze the methodologies, data, and reliability of the findings of two studies by Varshney and Associates. You noted in your request that:

The first study—Cost of State Regulations on California Small Businesses Study (September 2009)—concluded that California’s regulations of all types resulted in reduction in the gross state product (GSP) of $493 billion annually in lost output and $134,000 annually per small business.
The second study—Cost of AB 32 on California Small Businesses—Summary Report of Findings (June 2009)—concluded that AB 32 will cost California’s small businesses $183 billion in lost output each year.

In our response below, we summarize the methodologies and analyses contained in these two studies, discuss their findings, and provide our assessment of the analyses supporting their conclusions.

Legislative Analyst’s Office (LAO) Conclusion. Both of the two studies you have asked us to review have major problems involving both data, methodology, and analysis. As a result of these shortcomings, we believe that their principal findings are unreliable.

The First Study—Overall Cost of State Regulations

This study was written by Professors S. Varshney and D. Tootelian (V&T) of California State University, Sacramento. It was mandated by Chapter 232, Statutes of 2006 (AB 2330, Arambula). This measure, among other things, required the Office of Small Business Advocate to commission a study on the costs of regulations on small businesses that is parallel to the study on the impact of regulatory costs on small firms conducted by the United States Small Business Administration, to be completed no later than October 1, 2007.

Key V&T Study Findings. The V&T’s principal conclusions are that the total annual economic cost of all regulations in California amounts to a loss of $493 billion in GSP and 3.8 million jobs. It also concluded that regulatory costs are born almost entirely by small businesses, and in 2007 amounted to over $134,000 per firm.

V&T’s Methodology and Data

The basic methodology employed by V&T was to use the statistical tool of multiple linear regression analysis to attempt to isolate the contribution of different factors, including the regulatory environment, on California’s GSP. Specifically:

The V&T estimated a linear multiple regression equation to correlate states’ gross product (GSP in millions of current dollars) to states’ ordinal rankings for six different indexes developed by Forbes Magazine. These state indexes were described by Forbes as covering the topics of business costs, economic climate, growth prospects, labor characteristics, quality of life, and regulatory environment.
To estimate this equation, a combined time-series cross-section regression approach was used. This was done by pooling together the data for the six indexes for all 50 states for 2006 and 2007.
The estimated regression coefficient for the regulatory variable was then multiplied by its ordinal state ranking for California to arrive at an estimate of the total effect on California’s GSP due to the regulatory variable.
Specifically, California’s average ordinal ranking by Forbes for the regulatory variable in 2006 and 2007 was 40 (that is, the 40^th worst state) and V&T’s estimated regression coefficient on the regulatory variable was -4,424. The V&T concluded from this that each one unit of ordinal ranking on the regulatory variable reduces a state’s GSP by $4.4 billion, and thus, the total negative impact of the regulatory environment on California’s GSP is a minus $177 billion.
The V&T then arrived at its estimate of $493 billion for the total adverse effect on California GSP due to regulations by applying a “multiplier” effect of roughly 2.8 to account for various indirect and induced effects flowing from the above $177 billion figure. This multiplier effect was derived using a modeling software called IMPLAN. This model traces through and computes the multiplied effects on output, employment, income, and other economic variables that a direct economic shock or policy change has as it works its way through the economy over time.
Lastly, V&T’s estimate of $134,000 in lost output annually per California small business due to regulations was arrived at by dividing its estimated $493 billion total adverse effect of regulations on California’s GSP by 3.7 million, the number of small businesses in California estimated to have existed in 2006 by the United States Small Business Administration Officer of Advocacy.

LAO Comments

Our review of this study indicates that it contains a number of serious shortcomings that render its estimates of the annual economic costs of state regulations essentially useless. The most significant of these problem areas include the following.

Regulatory Environment Index Has Problems. The index used by V&T to rank a state’s regulatory environment is not well defined and has other shortcomings. The regulatory environment index is described by Forbes to include the following elements: regulatory and tort environment, business incentives, transportation-related factors, and bond rating. These components, in turn, included information from such sources as Pacific Research Institute and bond rating agencies. Representatives of Forbes indicated to us that there was no specific equation or formula available regarding how these rankings were arrived at, including exactly how the various factors involved were combined to determine a state’s regulatory index value. Rather, we were told this ranking was arrived at by Forbes staff through discussions and subjective evaluation. The key thing to note is that the regulatory environment index included things other than just business regulations per se, and that specific individual regulations were not identified nor was information about their benefits, effectiveness, and cost efficiency made available. Given this, we question whether the index used provides a reliable measure by which to identify California’s relative regulatory environment ranking or assess the economic affects of state regulations.
Regression Analysis Is Deficient. The regression analysis has a number of problems. The biggest is that the relative size of states was not taken into account by V&T in explaining interstate differences in GSP. One way to have done this would be to have focused on explaining GSP adjusted for a measure of the size of each state, such as by using per capita GSP. The authors indicated to us that they in fact tried this methodology, but got poor results and thus did not use them. Our own regressions indicate that when per capita GSP is used, the regression outcomes dramatically change and become questionable. For instance, the regression itself explains only a bit over 10 percent of interstate differences in per capita GSP, and the regression coefficient measuring the effect of the regulatory index on the economy changes its sign from negative to positive. The latter finding, which also has been noted by other economists who have reviewed the V&T study, is inconsistent with the hypothesis that a poorer ranking on the regulatory environment index hurts GSP. In addition, the equation’s low explanatory power suggests that either the variables hypothesized to affect GSP are not very influential, are not being measured correctly, and/or there are other more important variables that have been excluded.
Reliance on Ordinal Index Poses Special Difficulties. The Forbes regulatory environment index simply ranks states numerically with no information about how they score in terms of an actual numeric index value. In one case, a state could differ by just one rank position from another but have a much different regulatory climate. In a different case, a one position ranking difference in the index might represent very little difference in regulatory environment. This inherently limits the index’s ability to explain differences in economic performance in the regression analysis, even if one puts aside the other shortcomings noted above.
Application of Multiplier Is Inappropriate. As noted above, V&T used an estimated 2.8 multiplier effect to scale-up the estimated effect of the regulatory environment on GSP from $178 billion to $493 billion annually. Using multipliers is appropriate when estimating how an initial change in some type of spending or income will affect overall output after considering the impact of the new spending as it circulates throughout the economy and is re-spent, causing subsequent indirect and induced impacts on output over time. In this case, however, the multiple regression analysis already implicitly captures such interactions by focusing on explaining GSP itself. Thus, applying a separate multiplier effect is inappropriate.
Effect on Small Businesses Is Overstated. The V&T’s finding that the cost of regulations on small businesses amounts to roughly $134,000 per firm annually is overstated. Even if the direct cost of regulations is disproportionately borne by small businesses, as assumed by V&T, dividing $493 billion by the number of small businesses (3.7 million) is inappropriate and results in an overstatement. This is because the former number (which itself is overstated) is V&T’s estimate of the total economy-wide effect of regulations, including their indirect and induced impacts on consumers and firms other than just those parties on whom the regulations are initially imposed. In addition, some portion of V&T’s estimated $178 billion effect on GSP due to regulations prior to application of the multiplier is not ascribable to small businesses, but realistically would apply to more modest-sized and larger firms that are not small businesses—even if one is of the opinion that small businesses bear the brunt of regulatory costs.

The Second Study—Cost of AB 32 on Small Businesses

This study also was written by V&T. It was commissioned by the California Small Business Roundtable in March 2009 to examine the possible impact of AB 32 on the California economy, and specifically the impacts it will have on small businesses in California.

Key V&T Study Findings. The V&T’s principal conclusions are that the annual costs to small businesses of implementing AB 32 are likely to total $183 billion in reduced GSP and the equivalent of 1.1 million fewer jobs. The average annual cost of AB 32 per small business was estimated to be approximately $50,000.

V&T’s Methodology and Data

The general approach used by V&T in this study was to use certain assumptions about the direct effects of AB 32 on California consumers and businesses. The V&T then constructed three alternative scenarios regarding the effects of AB 32’s implementation through the Scoping Plan (SP) adopted by the California Air Resources Board (CARB), and used IMPLAN to estimate the associated economy-wide effects of the SP under each scenario. Specifically:

Scenario One was defined as a “minimum-impact” scenario. This scenario assumed that the annualized cost of implementing AB 32 is $24.9 billion as estimated in CARB’s SP, but did not include any costs that were not identified in the SP, such as various transition costs, investment costs, and research and development costs.
Scenario Two focused on the expected impact of the SP in terms of costs projected to be incurred by California consumers. This was predicated on the assumption that the costs of the SP to businesses would largely be shifted to consumers through the prices they pay for the goods and services they purchase. The V&T specifically assumed that the SP would increase costs to consumers by various amounts (based either on their own calculations or on information from other sources) in five areas: household costs, transportation costs, natural gas, electricity, and food. The V&T estimated that these added costs would amount to $3,857 annually per household or an increase of $52.2 billion (slightly over 6 percent) in total for California.
Scenario Three involved the expected impact on small businesses. This analysis focused on five areas of cost increases to businesses due to implementation of the SP: transportation, housing, food, fuels, and utilities. The V&T assumed that, given estimates from other research studies, costs in these five areas to small businesses would rise by at least 10 percent because of the SP. It also assumed that these five cost categories account for roughly 45 percent of all small business costs. Thus, it assumed that costs to small businesses due to the SP would rise by 4.5 percent. Based on an estimate that 2009 receipts to small businesses in California totaled $1.6 billion, V&T concluded that SP would raise costs to California small businesses by $63.9 billion.
The V&T then applied an IMPLAN multiplier of roughly 2.8 to the dollar effects noted above for each scenario to arrive at an estimated total reduction in California output due to the SP of $72 billion, $149 billion, and $183 billion, respectively, for the three scenarios.
The V&T also used the IMPLAN model to make estimates of the impacts of the SP on employment, labor income, and tax receipts. This analysis predicted, for example, that the three scenarios would result in job losses of about 430,000, 900,000, and 1.1 million, respectively.
The V&T’s estimate that the SP would cost California’s small businesses an average of $50,000 annually was arrived at by dividing the $183 billion total impact under Scenario 3 above by the estimate provided earlier of 3.7 million California small businesses.

LAO Comments

As with the first study above, our review of this second study indicates that it contains a number of serious shortcomings that render its estimates of the economic effects of AB 32’s proposed implementation through the SP highly unreliable. The most significant of these issue areas are as follows.

Scenario 1 Completely Disregards Any SP Savings. Regarding Scenario 1, Appendix G-1 of CARB’s SP does identify costs for the SP’s measures totaling $24.9 billion annually—the figure V&T used. However, the SP also identified savings due to the SP’s actions of $40.4 billion. Thus, the SP estimates that there would be net savings, not costs. The V&T note that the estimated savings are too speculative to include. While we have our own concerns about some of the SP’s savings estimates, not acknowledging that there are any savings seems to be an extreme position. Thus, we believe that V&T’s estimated costs of the SP are overstated, perhaps significantly.
Scenario 2 Cost Estimates Biased Upward. Regarding Scenario 2, the cost increases that V&T ascribe to consumers of $52.2 billion is over twice the amount of the CARB’s estimated implementation cost of the SP. We cannot reconcile these numbers. The likely reason for the majority of the discrepancy involves the various specific assumptions V&T used in building up their cost calculations. Several recent analyses by outside energy economists have documented in detail a variety of significant cost overstatements by V&T in this area. An example involves the housing category, where the authors assume AB 32 would add $50,000 to the cost of constructing a new home, based on the cost to outfit a “zero net emission” house that goes far beyond AB 32’s SP standards. Given the above, we believe V&T’s total consumer cost estimate of implementing AB 32 through the SP is upwardly biased.
Scenario 3 Cost Estimates Also Overstated. Regarding Scenario 3, many of the cost categories evaluated for small businesses are similar to those in Scenario 2 for consumers, and thus the problems noted above with Scenario 2’s calculations also largely apply to Scenario 3. Therefore, we also believe that Scenario 3 overstates the costs of the SP to small businesses.
Application of Multiplier Magnifies Biases and Raises Issues. To the extent V&T’s first-order cost estimates under the three scenarios above are overstated, application of the IMPLAN’s 2.8 multiplier almost triples the size of these overstatements. In addition, we have concerns about the appropriateness of directly injecting V&T’s first-order cost estimates into the IMPLAN framework, given that certain elements of the estimates appear to already incorporate various indirect and induced interactions within the economy. In addition, we have concerns about how well IMPLAN’s existing structure captures such things as the trade flows that would correspond to the specific consumption and investment activity associated with SP-related activities.

Should you have questions regarding this information, please feel free to contact David Vasché of my staff at (916) 319-8305.

Sincerely,

Mac Taylor
Legislative Analyst

Return to LAO Home Page