1
IMPROVING REGULATORY BENEFIT-COST ANALYSIS Susan E. Dudley & Brian F. Mannix
¨
To paraphrase Winston Churchill, benefit-cost analysis may be the
worst tool for policymaking, except for all the others that have been tried.1 If regulatory interventions in market transactions are to have any hope
of achieving desired outcomes, they must be based on an understanding of the tradeoffs associated with alternative actions. Every president since Jimmy Carter has recognized this and required regulatory agencies to analyze the benefits and costs of proposed regulations before they are issued. Across developed countries, benefit-cost analysis (BCA) is the principal public policy tool for laying out available information in a way that allows policy makers to make balanced, efficient regulatory decisions in the face of limited resources. However, BCA has limitations. Despite numerous advances in the field, a number of significant problems have arisen that challenge its legitimate use in informing and evaluating public policy decisions.
The barriers to improving BCA are both institutional and technical. Among the institutional factors constraining the sound application of BCA in regulatory matters are that (1) legislation is often either silent on its use, or explicitly prohibits it; (2) BCA is conducted by regulatory agencies who use it to advocate for, rather than objectively analyze, proposed new regulations; (3) efforts to counteract agencies’ parochial perspective have not been as effective as they could be; and (4) incentives for ex post evaluation of ex ante estimates of the benefits and costs of regulatory actions are lacking.
Technical barriers stem from the way agencies conduct regulatory BCA, which tends to systematically bias the results. In particular, (1) analysts often start with a presumption that economic markets are fragile and prone to failure, but that their regulatory solutions will work exactly as planned, and that private decision makers are subject to cognitive biases that regulators somehow do not exhibit; (2) they identify co-benefits without
¨ Susan E. Dudley is director of the George Washington University Regulatory Studies Center;
she served as Administrator of the White House Office of Information and Regulatory Affairs from 2007-2009. Brian F. Mannix is a research professor in the George Washington Regulatory Studies Center; he served as the Environmental Protection Agency’s Associate Administrator for Policy, Economics and Innovation from 2005 to 2009. This article was prepared for the Journal of Law & Politics symposium, Reining in the Administrative State, on October 21, 2016.
1 444 Parl. Deb. HC (5th ser.) (1947) col. 206–07 (UK).
Journal of Law & Politics [Vol. XXXIV:1 2
searching for corresponding co-costs; (3) they apply risk assessment methods that are fundamentally incompatible with BCA; and (4) retrospective review is analytically challenging.
This article briefly reviews the process by which regulations are developed in the United States and the role for BCA. It then examines the institutional and technical factors limiting the use of BCA as a tool for improving regulatory policy. It concludes with some recommendations.
I. U.S. REGULATORY PRACTICES
When issuing new regulations, federal agencies are constrained by their
enabling legislation,2 by the Administrative Procedure Act (APA), which requires agencies to provide public notice and seek comment before issuing new regulations,3 and by executive requirements for regulatory impact analysis (primarily BCA).4 Presidents from both parties for more than forty years have supported ex ante regulatory impact analysis to make agencies weigh the likely positive and negative consequences of regulations before they are issued.5
Executive Order (“E.O.”) 12,866,6 issued by President Bill Clinton in 1993, and reinforced by George W. Bush,7 Barack Obama,8 and Donald Trump,9 currently guides the development and review of regulations.10 It expresses the philosophy that regulations should (1) address a “compelling public need, such as material failures of private markets”; (2) be based on an assessment of “all costs and benefits of available regulatory alternatives, including the alternative of not regulating”; and (3) “maximize net benefits” to society unless otherwise constrained by law.11 It also assigns
2 See U.S. CONST. art. I, § 1 (“All legislative powers herein granted shall be vested in a Congress of the United States, which shall consist of a Senate and a House of Representatives.”).
3 Administrative Procedure Act, Pub. L. No. 79-404, 60 Stat. 237 (1946) (codified as amended in scattered sections of 5 U.S.C.).
4 Exec. Order No. 12,866, 58 Fed. Reg. 51,735 (Oct. 4, 1993) (“In deciding whether and how to regulate, agencies should assess all costs and benefits of available regulatory alternatives, including the alternative of not regulating.”).
5 See Susan E. Dudley, Improving Regulatory Accountability: Lessons from the Past and Prospects for the Future, 65 CASE W. RES. L. REV. 1027 (2015).
6 58 Fed. Reg. at 51,735 (Sept. 30, 1993). 7 OFFICE OF MGMT. & BUDGET, EXEC. OFFICE OF THE PRESIDENT, CIRCULAR A-4: REGULATORY
ANALYSIS (2003). 8 Exec. Order No. 13,563, 76 Fed. Reg. 3821 (Jan. 18, 2011). 9 Exec. Order No. 13,777, 82 Fed. Reg. 12285 (Feb. 24, 2017). 10 President Trump has imposed additional procedures, including the requirement that every new
regulation be offset by the removal of at least two existing regulations. See Exec. Order No. 13,771, 82 Fed. Reg. 9339 (Jan. 30, 2017).
11 Exec. Order No. 12,866, 58 Fed. Reg. at 51,735.
2018] Improving Regulatory Benefit-Cost Analysis
3
the Office of Information & Regulatory Affairs (OIRA) within the Office of Management and Budget (OMB) responsibility for reviewing executive branch agency proposed and final regulations before they are issued, along with supporting analyses.
A. Role of BCA in Regulatory Development
In the U.S. and most developed countries, BCA is considered an important aspect of ex ante regulatory impact analysis.12 OIRA explains that the purpose of the regulatory impact analysis (RIA)13 is to provide the public with a “careful and transparent analysis” of the effects of regulatory actions. It should include “an assessment and (to the extent feasible) a quantification and monetization of benefits and costs anticipated to result from the proposed action and from alternative regulatory actions.”14
The purpose of the RIA is to inform agency decisions in advance of regulatory actions and to ensure that regulatory choices are made after appropriate consideration of the likely consequences. To the extent permitted by law, agencies should proceed only on the basis of a reasoned determination that the benefits justify the costs (recognizing that some benefits and costs are difficult to quantify). Regulatory analysis also has an important democratic function; it promotes accountability and transparency and is a central part of open government.15
B. BCA in Practice
OIRA reports each year to Congress on the benefits and costs of the major rules it reviewed over the previous ten years. In its most recent final (2015) report, it states:
The estimated annual benefits of major Federal
regulations reviewed by OMB from October 1, 2004, to September 30, 2014, for which agencies estimated and monetized both benefits and costs, are in the aggregate between $216 billion and $812 billion, while the estimated
12 See ORG. FOR ECON. CO-OPERATION & DEV., INTRODUCTORY HANDBOOK FOR UNDERTAKING
REGULATORY IMPACT ANALYSIS (RIA) (2008), https://www.oecd.org/gov/regulatory-policy/44789472.pdf.
13 Note that in the U.S., the regulatory impact analysis document is frequently abbreviated as RIA, while the act of doing the analysis is not.
14 OFFICE OF MGMT. & BUDGET, EXEC. OFFICE OF THE PRESIDENT, REGULATORY IMPACT ANALYSIS: A PRIMER 2 (2011), https://www.whitehouse.gov/sites/default/files/omb/inforeg/regpol/circular-a-4_regulatory-impact-analysis-a-primer.pdf
15 Id.
Journal of Law & Politics [Vol. XXXIV:1 4
annual costs are in the aggregate between $57 billion and $85 billion, reported in 2001 dollars. . . . These ranges reflect uncertainty in the benefits and costs of each rule at the time that it was evaluated.16
Thus, according to this report, the benefits of regulations issued over the
last ten years are almost a factor of ten higher than the costs. However, these estimates represent only a fraction of promulgated rules, and “a closer examination reveals that the benefit figures are highly dependent on a few assumptions and that the ranges presented are unlikely to reflect the true uncertainty surrounding them.”17 It is significant that the reported benefits and costs are based on ex ante estimates developed by the agencies themselves before the regulations went into effect. OMB cautions that its “reliance on those estimates in this Report should not necessarily be taken as an OMB endorsement of all the varied methodologies used by agencies to estimate benefits and costs.” 18 OMB identifies several key uncertainties embedded in these estimates, including how regulations’ expected reduction in risks to life are valued and the numerous “assumptions used in projecting the health impact of reducing particulate matter.”19
Those caveats often get lost in public discourse, however, and the aggregate estimates are widely reported, without qualification, as evidence of the net benefits of federal regulatory activity.20
Why, after decades of practice, has BCA not lived up to its potential? The barriers to improving regulatory BCA are both institutional and technical. These are discussed in the next two sections.
II. INSTITUTIONAL BARRIERS TO IMPROVING BCA
Several institutional barriers limit the extent to which robust BCA
informs regulatory policy decisions. First, although presidents of both parties have long required agencies to base new regulations on BCA of
16 OFFICE OF MGMT. & BUDGET, EXEC. OFFICE OF THE PRESIDENT, 2015 REPORT TO CONGRESS ON
THE BENEFITS AND COSTS OF FEDERAL REGULATIONS AND AGENCY COMPLIANCE WITH THE UNFUNDED MANDATE REFORM ACT 1–2 (2015), https://obamawhitehouse.archives.gov/sites/default/files/omb/inforeg/2015_cb/2015-cost-benefit-report.pdf (internal footnotes omitted).
17 Susan E. Dudley, OMB’s Reported Benefits of Regulation: Too Good to Be True?, REGULATION, Summer 2013, at 26.
18 OFFICE OF MGMT. & BUDGET, EXEC. OFFICE OF THE PRESIDENT, supra note 14, at 14. 19 Id. 20 See, e.g., Richard Pierce, Jr., The Regulatory Budget Debate, 19 N.Y.U. J. LEGIS. & PUB. POL’Y
249 (2016).
2018] Improving Regulatory Benefit-Cost Analysis
5
alternative approaches, Congress continues to pass laws that do not permit explicit consideration of tradeoffs. Second, BCA is conducted by the agencies wishing to issue new regulations, and as a result, is often used as a tool for advocacy, rather than a neutral tool for analysis. Third, efforts to counter agencies’ parochial perspective, such as oversight and public comment, have not been fully effective. Fourth, the government rarely evaluates regulatory outcomes ex post to determine the accuracy of ex ante analyses. These are each discussed below.
A. Legislation Precludes BCA
Executive Order 12,866 requires agencies to apply BCA to examine alternative policy options, stating, “in choosing among alternative regulatory approaches, agencies should select those approaches that maximize net benefits (including potential economic, environmental, public health and safety, and other advantages; distributive impacts; and equity), unless a statute requires another regulatory approach.”21
This last caveat is significant. Legislation delegating regulatory authority to executive-branch agencies rarely includes explicit requirements for agencies to base their regulatory decisions on BCA. The Safe Drinking Water Act is a notable exception. The Toxic Substances Control Act was another exception, but with support from industry and non-governmental organizations alike, Congress amended it in 2016 to remove most of the benefit-cost balancing language.22 Most statutes are silent on whether regulations should be based on BCA,23 but some have been interpreted as precluding a weighing of costs against benefits.24
As a result, although E.O. 12,866 requires agencies to estimate both benefits and costs of regulations, especially those “economically significant” rules expected to have impacts of $100 million or more in a year,25 a small fraction of regulations each year actually includes such analysis. In 2014, for example, executive branch agencies issued 53 economically significant rules, of which only 13 were accompanied by full BCA. This count does not include independent regulatory agencies, such as the Securities and Exchange Commission, the Federal Communications
21 Exec. Order No. 12,866, 58 Fed. Reg. 51,735 (Oct. 4, 1993) (emphasis added). 22 Frank R. Lautenberg Chemical Safety for the 21st Century Act, Pub. L. No. 114-182, 130 Stat.
448 (2016). 23 Arthur Fraas, The Role of Economic Analysis in Shaping Environmental Policy, 54 L. &
CONTEMPORARY PROBLEMS 113, 114 (1991). 24 See Whitman v. Am. Trucking Ass’ns, 531 U.S. 457 (2001). 25 Exec. Order No. 12,866, 58 Fed. Reg. at 51,735.
Journal of Law & Politics [Vol. XXXIV:1 6
Commission, and the Consumer Financial Protection Bureau,26 which are not subject to E.O. 12,866 or OIRA oversight and rarely conduct even rudimentary BCA.27
B. BCA is Conducted by Regulatory Agencies
One important institutional barrier to better regulatory analysis is that RIAs are conducted by the agencies themselves, and agencies face incentives to demonstrate that the benefits of their desired actions exceed the costs.28 RIAs are often developed after decisions are made and used to justify, rather than inform, regulations. As noted above and discussed in the next section on technical barriers to better analysis, regulatory benefit estimates, in particular, are highly uncertain, relying on hypothetical models and numerous assumptions, which are rarely subjected to ex post evaluation for accuracy.29
C. Efforts to Counter Agencies’ Parochial Perspective Have Been Ineffective
OIRA’s role in reviewing agency regulations before they are published is an important one, and evidence suggests it contributes to higher quality analysis in executive-branch agencies compared to independent regulatory agencies.30 Nevertheless, as a reliable check on agencies’ analysis, it has drawbacks. First, OIRA has fewer than 50 analysts responsible for reviewing all of the significant regulations of the executive branch.31
26 After the recent decision in PHH Corp. v. CFPB, the CFPB is no longer an independent agency,
and may be subject to E.O. 12,866, although the president clearly has the discretion to exempt them. 839 F.3d 1 (D.C. Cir. 2016). See Susan Dudley, CFPB Court Ruling is a Victory for Individual Liberty, FORBES (Oct. 12, 2016, 11:51AM), http://www.forbes.com/sites/susandudley/2016/10/12/cfpb-court-ruling-is-a-victory-for-individual-liberty/.
27 OMB’s 2015 Report to Congress states: “Independent agencies still have challenges in providing monetized estimates of benefits and costs of regulation.” OFFICE OF MGMT. & BUDGET, EXEC. OFFICE OF THE PRESIDENT, supra note 16, at 32.
28 Stephen Breyer observed that “well-meaning, intelligent regulators, trying to carry out their regulatory tasks sensibly, can nonetheless bring about counterproductive results.” Breyer attributes this problem to a combination of public perceptions, congressional actions, and uncertainties inherent in understanding and predicting risks. These external factors exacerbate the problem of “tunnel vision,” a phrase he uses to describe how agencies single-mindedly pursue a particular goal to a point that “the regulatory action imposes high costs without achieving significant additional safety benefits.” STEPHEN BREYER, BREAKING THE VICIOUS CIRCLE: TOWARD EFFECTIVE RISK REGULATION 11 (1995).
29 Dudley, supra note 17, at 26–27. 30 Arthur Fraas & Randall Lutter, On the Economic Analysis of Regulations at Independent
Regulatory Commissions, 63 ADMIN. L. REV. (SPECIAL ISSUE) 213, 226–27 (2011). 31 Susan Dudley & Melinda Warren, Regulators’ Budget Reflects President Trump’s Priorities: An
Analysis of the U.S. Budget for Fiscal Years 1960 through 2018, 39 REGULATORS’ BUDGET REP. 1 (July 18, 2017),
2018] Improving Regulatory Benefit-Cost Analysis
7
Perhaps more importantly, as an office within the Executive Office of the President, in addition to offering “a dispassionate and analytical ‘second opinion’ on agency actions,”32 it is responsible for pursuing the president’s priorities. Often agencies eschew analysis and instead choose a politically popular option; the same political forces will likely have influence within the Executive Office of the President.
Public comment is a key accountability tool, but it often comes late in the regulatory development process after agencies have conducted analysis and crafted positions. Regulatory impact analyses are dense and complex documents, often running into the thousands of pages, making meaningful public comment and peer review difficult.
The legislature has not been effective at monitoring individual RIAs or holding agencies accountable for BCA; as noted above, while Congress debates the merits of proposed legislation that would require BCA, it continues to pass new legislation that precludes it.33 Confronted with a silent statute, the Supreme Court has found that agency balancing of benefits and costs may be unavailable;34 more recently, it has found it optional;35 and more recently still, it has found that a decision may be arbitrary under the APA if it does not take costs into consideration.36
D. Ex ante Estimates Are Not Evaluated ex post
Ex ante benefit and cost estimates are not verified with empirical data ex post, even though several executive orders (for example, 12,866; 13,563; and 13,610) direct agencies to evaluate existing regulations. These retrospective review guidelines have been met with limited success, largely because they did not change underlying incentives.37 Unlike other government programs that are reassessed each time their funds are appropriated, regulations, once created, tend to exist in perpetuity.38
https://regulatorystudies.columbian.gwu.edu/sites/regulatorystudies.columbian.gwu.edu/files/downloads/2018_Regulators_Budget-2017-07-18.pdf.
32 Fed. Regulatory Rev., 74 Fed. Reg. 8819 (Feb. 26, 2009). 33 Frank R. Lautenberg Chemical Safety for the 21st Century Act, Pub. L. No. 114-182, 130 Stat.
448 (2016). 34 See Whitman v. Am. Trucking Ass’ns, 531 U.S. 457 (2001). 35 Entergy Corp. v. Riverkeeper, Inc., 556 U.S. 208 (2009). 36 Michigan v. EPA, 135 S. Ct. 2699 (2015). 37 Reducing Unnecessary and Costly Red Tape through Smarter Regulations: Hearing Before the
Joint Econ. Comm., 113th Cong. 32–46 (2013) (statement of Susan E. Dudley, Director, The George Washington University Regulatory Studies Center).
38 Susan E. Dudley, Retrospective Evaluation of Chemical Regulations (Org. for Econ. Co-operation & Dev. Env’t Working Papers, No. 118).
Journal of Law & Politics [Vol. XXXIV:1 8
For new regulations, OIRA serves a gatekeeper role, which compels regulating agencies to present analysis consistent with executive order requirements if they wish to issue new rules. On the other hand, once a regulation is issued, the consequence of not conducting ex post analysis is less problematic from the agency’s perspective, in that the regulation will remain on the books.39
Compounding this asymmetric incentive structure is that regulated parties may be more motivated to prevent a potentially burdensome regulation from being implemented than to advocate for a regulation to be removed.40 Once a regulation is in place, it confers a competitive advantage on some parties, especially those who have already invested in compliance.41 Incumbents and other beneficiaries are thus less likely to support evaluation that may lead to changes or repeal.42
III. TECHNICAL BARRIERS TO IMPROVING BCA
BCA is a valuable tool for informing policy decisions when collective
action is necessary, but it is necessarily a static exercise, dependent upon assumptions and models of how the world would look in the future with and without a regulatory intervention. It is not a replacement for market processes, which are dynamic and responsive to diverse preferences and changing circumstances.43 Often, the numerous assumptions on which estimated benefits and costs depend are highly uncertain and not transparent to decision-makers or the public.44 Technical barriers to high quality BCA include a lack of attention to the compelling public need for government intervention in markets (the “market failure”), lack of objectivity in identification of benefits and costs, incompatibility of underlying risk assessments, and challenges in conducting retrospective analysis to corroborate estimates after regulations are implemented.
39 Sofie E. Miller & Susan E. Dudley, Regulatory Accretion: Causes and Possible Remedies, 67
ADMIN. L. REV. ACCORD 98, 111 (2016), http://www.administrativelawreview.org/wp-content/uploads/2016/03/MillerDudley_PublishedVersion-1.pdf.
40 Dudley, supra note 38, at 9. 41 Miller & Dudley, supra note 39, at 109–10. 42 U.S. agencies suggest that “environmental advocacy groups have constituted a weak barrier to
evaluation.” Nicholas Hart, Evaluation at EPA: Determinants of the U.S. Environmental Protection Agency's Capacity to Supply Program Evaluation (Aug. 31, 2016) (unpublished Ph. D. dissertation, George Washington University), https://pqdtopen.proquest.com/doc/1812333751.html?FMT=AI.
43 Friedrich A. von Hayek, The Use of Knowledge in Society, 35 AM. ECON. REV. 519, 530 (1945). 44 Susan E. Dudley & Marcus Peacock, Improving Regulatory Science: A Case Study of the
National Ambient Air Quality Standards, 24 SUP. CT. ECON. REV. 49 (2017).
2018] Improving Regulatory Benefit-Cost Analysis
9
A. Lack of Attention to Market Failure Executive orders are explicit that agencies’ RIAs should clearly
articulate the core problem that requires regulatory action. EO 12,866 states:
Federal agencies should promulgate only such
regulations as are required by law, are necessary to interpret the law, or are made necessary by compelling public need, such as material failures of private markets to protect or improve the health and safety of the public, the environment, or the well-being of the American people. In deciding whether and how to regulate, agencies should assess all costs and benefits of available regulatory alternatives, including the alternative of not regulating.45
The concept of “market failure” is an important one in regulation. The
market exchange of goods and services between willing buyers and sellers efficiently relies on price signals to allocate scarce resources to their highest and best use.46 Absent a clearly identified market failure, regulation and other forms of government intervention can disrupt those market forces and lead to misallocation of resources,47 inevitably resulting in negative net benefits. Thus, targeting a systemic problem rather than relying on anecdotes to support regulation is important. If a regulation is not based on a “compelling public need,” it is more susceptible to special-interest pressures. As the OECD observes, “[a] basic aspect of RIA is that it must be conducted with this ‘whole of society’ view in mind, rather than paying undue attention to impacts on individual groups that may be lobbying for regulation.”48
Yet many RIAs claim very large net benefits without identifying a compelling public need to justify the necessity of a collective solution imposed by the government.49 In some cases, the RIA does not explain
45 58 Fed. Reg. 51,735 (Oct. 4, 1993). 46 Susan E. Dudley et al., Consumer’s Guide to Regulatory Impact Analysis: Ten Tips for Being an
Informed Policymaker, 8 J. BENEFIT-COST ANALYSIS 187, 191 (2017). 47 As the OECD observes: “Identifying one or more significant sources of market failure provides
evidence of a potential case for regulation. However, regulation frequently fails to address the identified market failure effectively and efficiently. There is a risk that market failure may be supplanted (or compounded) by regulatory failure.” ORG. FOR ECON. CO-OPERATION AND DEV., supra note 12, at 7.
48 Id. at 6. 49 Sofie Miller & Brian Mannix, One Standard to Rule Them All: The Disparate Impact of Energy
Efficiency Regulations, in NUDGE THEORY IN ACTION 251, 251–55 (Sherzod Abdukadirov ed., 2016).
Journal of Law & Politics [Vol. XXXIV:1 10
why private markets are unable to reach solutions superior to government action. For example, the Environmental Protection Agency (EPA) and the Department of Transportation (DOT) estimate their joint fuel economy rules will have large negative costs (which suggests that they would be justified even if they caused environmental harm), because, according to their calculations, the fuel savings consumers will derive from driving more fuel-efficient vehicles will outweigh the increased purchase price.50
At the same time that RIAs are cavalier about articulating why markets are unable to respond to the problem identified, they blindly assume that regulators are unbiased and knowledgeable and that the regulation will work exactly as planned.51
For example, while DOT and EPA do not identify a material failure of private markets that would prevent consumers from reaping the huge cost savings described above absent government regulation, they do make heroic assumptions to arrive at those estimates. Their results depend heavily on assumptions about future energy prices and the choice of discount rate—a rate significantly lower than consumers reveal they use when making personal decisions. Their RIAs do not appear to appreciate other vehicle attributes which consumers might value. By looking at average prices and usage patterns and by applying a low discount rate, the regulators paradoxically conclude that by taking away consumers’ choices, they can make them better off.52
This appears to be a classic case of the “planner’s paradox,” where planned solutions always look better on paper than unplanned solutions because the planner sees only his “data, assumptions, biases, and understandings of the way the world works . . . All of the unseen difficulties with the planned solution—the data, assumptions, biases, and understandings of the world that turn out to be wrong—are invisible to the analyst because the data he considers are his own.”53
50 According to EPA & DOT, under the recently proposed standards for heavy-duty trucks, “the
buyer of a new long-haul truck in 2027 would recoup the investment in fuel-efficient technology in less than two years through fuel savings.” Press Release, Environmental Protection Agency, EPA, DOT Propose Greenhouse Gas and Fuel Efficiency Standards for Heavy-Duty Trucks (June 19, 2015), https://www.epa.gov/newsreleases/epadot-propose-greenhouse-gas-and-fuel-efficiency-standards-heavy-duty-trucks.
51 See Adam Smith, Utilizing Behavioral Insights (without Romance): An Inquiry into the Choice Architecture of Public Decision-Making, (Oct. 16, 2016) (Geo. Wash. Univ. Regulatory Studies Ctr. unnumbered working paper), https://regulatorystudies.columbian.gwu.edu/utilizing-behavioral-insights-without-romance-inquiry-choice-architecture-public-decision-making.
52 Dudley, supra note 17, at 26–30. 53 Brian Mannix, The Planner’s Paradox, REGULATION, Summer 2003 at 8, 8–9.
2018] Improving Regulatory Benefit-Cost Analysis
11
This problem is accentuated by the increasingly prevalent application of behavioral insights to justify regulation based on the cognitive biases and fallacies of individuals acting on their own behalf.54 RIAs rarely consider that government decision-makers may suffer from similar, if not more, problematic biases.55
Yet, almost by definition, regulatory policies substitute the judgment of government regulators for those of individuals, and it is easy to succumb to what Nobel laureate Friedrich Hayek called the “fatal conceit.”56 When agencies calculate large net benefits without being able to identify a material failure of private markets, and must depend instead on assumptions about consumer irrationality such that consumers cannot be trusted to make decisions in their own self-interest, those benefits should be viewed with skepticism.57
B. Lack of Objectivity in Identification of Benefits and Costs
As observed above, agencies have incentives to demonstrate that proposed regulations will have net benefits to society, which can lead to RIAs that look more like advocacy tools than neutral fact-finding documents. This is perhaps most evident in EPA air-quality regulations, where so-called “co-benefits” play an important role, while the concept of “co-costs” is never used. OMB reports that rules that reduce reductions in fine air particles (PM2.5) contribute between 61 and 80 percent of the estimated benefits of all federal regulations.58 Many of these benefits derive from ancillary reductions in PM2.5 that EPA expects will occur coincidentally from controls aimed at other pollutants. For example, in 2010, four regulations claimed 100 percent of their quantified benefits from ancillary reductions in PM2.5.
59 Three of those regulations targeted emissions of toxic air pollutants and the fourth established standards for sulfur dioxide. “In 2012, 99 percent of the reported benefits from EPA’s
54 Brian F. Mannix & Susan E. Dudley, The Limits of Irrationality as a Rationale for Regulation,
34 J. POL’Y ANALYSIS & MGMT. 705, 707, 711 (2015); see also Brian F. Mannix & Susan E. Dudley, Please Don’t Regulate My Internalities, 34 J. POL’Y ANALYSIS & MGMT. 715, 716 (2015).
55 See Niclas Berggren, Time for Behavioral Political Economy? An Analysis of Articles in Behavioral Economics, REV. AUSTRIAN ECON. 199 (2012); Smith, supra note 51.
56 See von Hayek, supra note 43, at 530. 57 Susan Dudley, Perpetuating Puffery: An Analysis of the Composition of OMB's Reported
Benefits of Regulation, 47 BUS. ECON. 165 (2012). 58 OFFICE OF MGMT. & BUDGET, EXEC. OFFICE OF THE PRESIDENT, 2012 REPORT TO CONGRESS ON
THE BENEFITS AND COSTS OF FEDERAL REGULATIONS AND UNFUNDED MANDATES ON STATE, LOCAL, AND TRIBAL ENTITIES 12 (2012).
59 Dudley, supra note 57.
Journal of Law & Politics [Vol. XXXIV:1 12
mercury and air toxics rule were co-benefits.”60 The mercury rule was vacated by the Supreme Court,61 and the agency’s reliance on co-benefits was questioned in the course of the litigation.62
In principle, a benefit-cost analysis should be “complete.” It should include all of the significant consequences of a policy decision: direct and indirect, intended and unintended, beneficial and harmful. In practice, all such analyses must to some degree fall short of completeness. The problem here is that agencies do not appear to be undertaking the search for benefits and costs objectively. On the benefit side of the equation, they quantify or list every conceivable good thing that they can attribute to a decision to issue new regulations, while on the cost side they only consider the most obvious direct and intended costs of complying with the regulation. Thus, in setting stringent utility-emissions standards, EPA dismisses risks associated with reduced electric reliability, the competitiveness of the U.S. economy in international trade, or the effect that higher electricity prices will have on the family budget. In establishing new fuel economy standards, EPA, DOT, and the Department of Energy use unrealistic assumptions to estimate consumer energy and fuel savings, without considering all the other complex factors that go into individual decisions about which car or appliance to buy.63
C. Risk Assessments Are Incompatible with BCA
When regulations are intended to reduce risks to human health or the environment, the BCAs will rely upon scientific risk assessments for critical inputs. In some applications, such as the Federal Emergency Management Administration’s (FEMA’s) evaluation of projects that mitigate damage from natural hazards, risk assessment and BCA are compatible. FEMA uses benefit-cost modules that have probabilistic risk assessments built in, complete with damage functions, to ensure that hazard-mitigation funds are spent cost-effectively.64
60 Dudley, supra note 14, at 28. 61 See Michigan v. EPA, 135 S. Ct. 2699, 2711–22 (2015). 62 Susan E. Dudley, Supreme Court’s EPA Mercury Ruling is a Victory for Common Sense
Regulation, THE CONVERSATION (June 30, 2015, 1:32PM), http://theconversation.com/supreme-courts-epa-mercury-ruling-is-a-victory-for-common-sense-regulation-44073.
63 Dudley, supra note 17, at 26–30. 64 Brian Mannix, Are Chemical Risk Assessment and Benefit Cost Analysis Compatible?, GEO.
WASH. UNIV. REGULATORY STUDIES CTR. (Jan. 6, 2016), https://regulatorystudies.columbian.gwu.edu/are-chemical-risk-assessment-and-benefit-cost-analysis-compatible.
2018] Improving Regulatory Benefit-Cost Analysis
13
However, practices for developing chemical risk assessments generally are not compatible with BCA because they explicitly strive to err on the side of precaution by overstating risk. 65 Rather than presenting probabilities and a range of outcomes that reflect uncertainties, chemical risk assessments often generate precise-sounding predictions that hide not only considerable uncertainty about the actual risk, but the reliance on deliberately biased inferences and assumptions for handling that uncertainty.66
Though these practices are intended to be precautionary and health-protective, when used in BCA they may have the opposite effect. That is due to the “health-wealth effect”: when consumers’ incomes go down, they will buy less of everything, including risk reduction.67 In the economy as a whole, if we spend a billion dollars’ worth of resources on anything, that is a billion dollars less that will be spent on everything else in the consumers’ market basket – including risk reduction. Estimates vary, but some empirical research suggests that a $15 million decrease in income is associated with the loss of an additional statistical life.68
The use of exaggerated risk estimates in BCA leads to policies that spend too much money to reduce those risks, which increases other risks because of this “health-wealth” effect. In fact, if a particular mortality risk is overestimated by a factor of ten or more, as is often the case with chemical risk assessments, then the income effect will not only diminish the expected benefits: it will end up killing more people than it saves.69
Whenever a predictive risk assessment relies on a long chain of inferences, there will be many opportunities to introduce a bias in the prediction of risk. Sometimes risk assessors may simply be designing their model to minimize false negatives, while paying a bit less attention to false positives. At every step, people may think they are just being cautious – and erring on the side of safety. But the cumulative result is just the
65 For example, EPA’s “Risk Assessment Principles and Practices” document states: “[s]ince EPA
is a health and environmental protective agency, EPA’s policy is that risk assessments should not knowingly underestimate or grossly overestimate risks. This policy position prompts risk assessments to take a more ‘protective’ stance given the underlying uncertainty with the risk estimates generated.” ENVTL. PROT. AGENCY, AN EXAMINATION OF EPA RISK ASSESSMENT PRINCIPLES AND PRACTICES 13 (2004).
66 See Dudley & Peacock, supra note 44, at 5. 67 Mannix, supra note 64. 68 Randall Lutter et al., The Cost-Per-Life-Saved Cutoff for Safety-Enhancing Regulations, 37
ECON. INQUIRY 599 (1999). 69 Mannix, supra note 64.
Journal of Law & Politics [Vol. XXXIV:1 14
opposite of what they intended. The exaggeration of chemical risks actually kills people.70
D. Ex post BCA Challenging
As noted above, agencies rarely conduct ex post evaluation to verify the accuracy of their ex ante analysis and assumptions. This is not only due to the poor incentives discussed above; retrospective review poses analytical challenges. Once a regulation is in place, it is difficult to accurately say what the outcome would have been without it.71 For example, would air emissions have increased directly with economic and population growth, or would technological change and citizen preferences have driven emissions lower?72 Measuring opportunity costs (what activities or innovations were foregone to achieve regulatory goals?) is difficult, and measuring regulatory benefits is often harder.73 We have no way of knowing, for example, how many important pharmaceuticals have not been discovered, because of the barriers to innovation presented by drug regulations.
IV. RECOMMENDATIONS
This section offers some modest recommendations to address the
institutional and technical problems facing BCA.
A. Institutional
1. Legislation requiring BCA Congress should legislatively require BCA and establish a standard that
new regulations consider a range of reasonable alternatives and attempt to maximize net benefits to society. Several bipartisan bills in recent Congresses would have codified the language of President Clinton’s Executive Order 12,866 and President Obama’s Executive Order 13,563.74 This would lend congressional support to the orders’ nonpartisan principles and guiding philosophy that before issuing regulations, agencies should identify a compelling public need, evaluate the likely effects of different regulatory approaches, and select the approach that provides the greatest
70 Id. 71 Dudley, supra note 38, at 7. 72 Jonathan Adler, The Fable of Federal Regulation: No, States Didn’t Ignore Environmental
Problems, PERC REP. (Dec. 10, 2004), http://www.perc.org/articles/fable-federal-regulation. 73 Dudley, supra note 38, at 7 74 Principled Rulemaking Act, S. 1818, 114th Cong. (2015).
2018] Improving Regulatory Benefit-Cost Analysis
15
net benefit for the country.75 Ideally, such a requirement would override authorizing statutes that ignore or explicitly prohibit analysis of tradeoffs.
2. Stronger legislative and judicial checks on agency decisions
Legislation could apply the Executive Order requirements to independent agencies and make compliance with them judicially reviewable.76 Judicial review could be valuable because agencies tend to take more seriously aspects of their missions that are subject to litigation, and might be particularly important for independent regulatory commissions, whose actions are not subject to OIRA review.77 “Like executive and congressional oversight, judicial oversight would likely make regulatory agencies more accountable for better decisions based on better analysis.”78
3. More vigorous and meaningful public input
Another way to shine more light on agency decisions and the supporting analysis would be through publication of advance notices of proposed rulemaking (ANPR) for high-impact rules. As noted above, regulatory-impact analyses are often developed after decisions are made and are used to justify, rather than inform, regulations. ANPRs could be valuable for soliciting input from knowledgeable parties on a range of possible approaches, data, models, etc., before particular policy options have been selected.79 These might include “back of the envelope” analyses that consider the effects of a wide range of alternatives.80 “Successful reforms might involve pre-rulemaking disclosure of risk-assessment information to engage broad public comment on the proper choice of studies, models, assumptions, etc. long before any policy decisions are framed, or ‘positions’ established.”81
75 Dudley, supra note 5, at 1044. 76 Dudley, supra note 5, at 1045. 77 Catherine M. Sharkey, State Farm “with Teeth”: Heightened Judicial Review in the Absence of
Executive Oversight, 89 N.Y.U. L. REV. 1589, 1620 (2014). 78 Reducing Unnecessary and Costly Red Tape through Smarter Regulations, supra note 37, at 8. 79 Susan E. Dudley & Kai Wegrich, Regulatory Policy and Practice in the United States and
European Union (Mar. 10, 2015) (Geo. Wash. Univ. Regulatory Studies Ctr. unnumbered working paper), http://regulatorystudies.columbian.gwu.edu/achieving-regulatory-policy-objectives-overview-and-comparison-us-and-eu-procedures.
80 Christopher Carrigan & Stuart Shapiro, What's Wrong with the Back of the Envelope? A Call for Simple (and Timely) Benefit-Cost Analysis, 11 REGULATION & GOVERNANCE 203 (2017).
81 Dudley, supra note 44.
Journal of Law & Politics [Vol. XXXIV:1 16
4. Require agencies to evaluate existing regulations before issuing new ones
While ex post evaluation has a long tradition in other areas (particularly in programs financed through the fiscal budget),82 it has received little attention in the regulatory arena, despite government guidelines requiring it.83 In essence, ex ante analyses are hypotheses of the effects of regulatory actions. Better regulatory evaluation would allow agencies and others to test those hypotheses against actual outcomes.84 This would not only inform decisions related to the benefits and costs of existing policy, but would provide feedback that would improve future ex ante analyses and future policies.85 To incentivize more robust evaluation of regulations once they are in effect, agencies could be required to test the validity of previous BCA predictions before commencing new regulation. As a condition for issuing new regulations, agencies could be required to present a robust framework for later evaluation and a commitment to gather necessary data.
5. An independent body may offer more objective retrospective review Rather than leaving the responsibility for retrospective evaluation with
regulatory agencies, Congress could assign an independent body responsibility for reviewing the accumulated stock of regulations and making recommendations to repeal rules or sets of rules.86 This model has the potential to address some of the accumulated regulatory burden87 and to improve regulatory evaluation. An independent third-party review would offer an objectivity that past efforts (which depend on regulatory agencies themselves to identify outmoded regulations) lacked,88 and “would likely
82 Stijn Smismans, Policy Evaluation in the EU: The Challenges of Linking Ex Ante and Ex Post
Appraisal, 6 EUR. J. RISK REG. 6, 12 (2015). 83 Joseph Aldy, Learning from Experience: An Assessment of the Retrospective Reviews of Agency
Rules and the Evidence for Improving the Design and Implementation of Regulatory Policy, ADMIN. CONF. U.S. (Nov. 18, 2014), https://www.acus.gov/report/retrospective-review-report. Aldy writes that federal regulatory agencies have a mixed record on ex post review, despite their “long track record of prospective analysis of proposed regulations that can address these questions.” Id. at 4.
84 Hypothesis testing is an essential element of the scientific method, and provides the feedback necessary to inform predictions. See Jose Wudka, What Is the “Scientific Method”?, U.C. RIVERSIDE (Sept. 24, 1998), http://physics.ucr.edu/~wudka/Physics7/Notes_www/node6.html.
85 Dudley supra note 38, at 27. 86See, e.g., S. 708, 114th Cong. (2015); S. 1683, 114th Cong. (2015). 87 Michael Mandel & Diana Carew, Regulator Improvement Commission: A Politically-Viable
Approach to U.S. Regulatory Reform, PROGRESSIVE POL’Y INSTITUTE (May 2013), http://www.progressivepolicy.org/wp-content/uploads/2013/05/05.2013-Mandel-Carew_Regulatory-Improvement-Commission_A-Politically-Viable-Approach-to-US-Regulatory-Reform.pdf.
88As Greenstone observed, “the process of self-evaluation is challenging for all organizations, as it requires complete objectivity. Indeed, history is unkind to organizations that fail to get outside reviews of their work.” Examining Practical Solutions to Improve the Federal Regulatory Process: Roundtable
2018] Improving Regulatory Benefit-Cost Analysis
17
identify reform opportunities agencies would miss.”89 Perhaps most importantly, institutionalizing a third-party review could improve review by motivating better data collection and more rigorous evaluation of whether risk management regulation is actually achieving its desired effect.90
6. Offset requirements could provide incentives for better BCA
To motivate retrospective evaluation of regulations, several countries have “initiated programs that require new regulatory costs to be offset by removal of existing regulatory burdens.”91 President Trump issued E.O. 13,771 in January 2017 requiring that “for every one new regulation issued, at least two prior regulations be identified for elimination, and that the cost of planned regulations be prudently managed and controlled through a budgeting process.”92
While such approaches do not explicitly balance the benefits of regulations against the costs93 and may not be appropriate for all types of regulations, they could motivate governmental and non-governmental agents to develop approaches to quantify the benefits and costs of regulations so they can trade off less-cost effective rules and retain those that are achieving their goals. Along these lines, sunset provisions could provide incentives for evaluation of regulations’ effects.
B. Technical
1. Agencies should consider how regulation will affect competition Agencies should be required to present evidence that they have
identified a material failure of competitive markets or public institutions that requires a federal regulatory solution, and provide an objective evaluation of alternatives (including the alternative of not regulating) and
Before the Subcomm. on Regulatory Affairs & Fed. Mgmt. of the S. Comm. on Homeland Sec. and Gov’t Affairs, 114th Cong. (2015) (statement of Michael Greenstone, Milton Friedman Professor of Economics, University of Chicago, Director, Energy Policy Institute at Chicago).
89 A Review of Regulatory Reform Proposals: Hearing Before the S. Comm. on Homeland Sec. & Gov’t Affairs, 114th Cong. (2015) (statement of Susan E. Dudley, Director, The George Washington University Regulatory Studies Center).
90 Dudley, supra note 38, at 26. 91 Susan E. Dudley, Can Fiscal Budget Concepts Improve Regulation?, 19 N.Y.U. J. LEGIS.&
POL’Y 259, 270 (2016). 92 Exec. Order No. 13,771, 82 Fed. Reg. 9339 (Jan. 30, 2017). 93 Richard J. Pierce, Jr., The Regulatory Budget Debate, 19 N.Y.U. J. LEGIS. & POL’Y, 249, 251
(2016).
Journal of Law & Politics [Vol. XXXIV:1 18
of the competitive and distributional impacts of different approaches.94 Whatever their particular mission, regulators need to be mindful that competition is the most important regulator of our economy.95
2. Agencies should recognize that behavioral insights apply to regulators
as well as those being regulated Regulations that derive most of their benefits from providing private
monetary gains that individuals can achieve without government intervention, such as fuel savings from driving energy efficient cars, should require a particularly demanding burden of proof.96 The analysis should be required to provide evidence that individuals behave irrationally (and do not learn) in the specific situation covered by the proposed regulation.97 It should also provide evidence that regulators are not subject to biases that may color their judgment of consumer welfare, and explain why they are better able to judge individuals’ preferences or be more faithful agents of individuals’ interests than the individuals themselves. 98
3. Agencies should examine benefits and costs in a consistent manner OMB recommends99 that “[b]enefits and costs of a regulation should be
assessed in a consistent manner,” yet agencies routinely count “co-benefits” without considering co-costs. As noted above, in principle, a benefit-cost analysis should account for all of the effects of a regulatory decision: indirect as well as direct, delayed as well as immediate, improbable as well as probable, unintentional as well as intentional. In practice, the analyst must define reasonable bounds on what to include. Agencies should define those boundaries in a way that not only produces a reasonably accurate and complete analysis, but also one that remains
94 Exec. Order No. 12,866, 58 Fed. Reg. 51,735 (Oct. 4, 1993). 95 Sofie E. Miller et al., Steps to Increase Competition and Better Inform Consumers and Workers
to Support Continued Growth of the American Economy, GEO. WASH. UNIV. REG. STUD. CTR. (May 12, 2016), https://regulatorystudies.columbian.gwu.edu/public-comment-national-economic-council-president%E2%80%99s-executive-order-13725-steps-increase.
96 Ted Gayer & W. Kip Viscusi, Overriding Consumer Preferences with Energy Regulations, 43 J. REG. ECON. 248, 264 (2013)..
97 Dudley et al. supra 46, at 192. 98 Brian F. Mannix & Susan E. Dudley, The Limits of Irrationality as a Rationale for Regulation,
34 J. POL’Y ANALYSIS & MGMT. 705, 707, 711 (2015); see also Brian F. Mannix & Susan E. Dudley, Please Don’t Regulate My Internalities, 34 J. POL’Y ANALYSIS & MGMT. 715, 716 (2015).
99 OFFICE OF MGMT. & BUDGET, EXEC. OFFICE OF THE PRESIDENT, supra note 16, at 54.
2018] Improving Regulatory Benefit-Cost Analysis
19
unbiased.100 The problem with ‘co-benefits’ is not that ancillary [effects] are being included, but that they are being included selectively.”101
4. Risk assessments should provide expected values and ranges based on
probabilistic analysis Because risk assessment necessarily involves assumptions and
judgments as well as pure scientific inputs, the government should establish procedures and incentives to make more transparent the effect different credible risk-assessment inputs and assumptions have on the range of plausible outcomes. This would make risk assessment more compatible with BCA, which is supposed to inform decision-makers of the expected value and range of the benefits and costs of different interventions.
5. Regulations should be designed to facilitate natural experimentation and
learning Ex ante RIAs necessarily depend on unverifiable assumptions and
models of how the world would be had the regulation never been implemented, and how responses to regulatory requirements will alter those conditions.102 To test those hypotheses, agencies should be required to design regulations from the outset in ways that allow variations in compliance.103 For example, EPA is required to review the National Ambient Air Quality Standards (NAAQS) every five years. Before it issued a new standard, it could be required to apply quasi-experimental techniques to gather and analyze epidemiology data and health outcome trends in different regions of the country and compare them against predictions.104 Regulations such as these lend themselves to quasi-experimental techniques to examine regulatory benefits and costs, because
100 Dudley et al. supra note 46, at 200. 101 Sofie E. Miller et al., Public Comment on OMB’s 2015 Draft Report to Congress on the Benefits
and Costs of Federal Regulation, GEO. WASH. UNIV. REG. STUD. CTR. (Dec. 15, 2015), https://regulatorystudies.columbian.gwu.edu/public-comment-omb%E2%80%99s-2015-draft-report-congress-benefits-and-costs-federal-regulations.
102 Dudley, supra note 38, at 5. 103 Dudley, supra note 38. 104 See Louis A. Cox, Public Interest Comment on The Environmental Protection Agency’s
Proposed Rule: National Ambient Air Quality Standards for Ozone, GEO. WASH. UNIV. REG. STUD. CTR. (Mar. 17, 2015), https://regulatorystudies.columbian.gwu.edu/sites/regulatorystudies.columbian.gwu.edu/files/downloads/TCox-NAAQS-ozone-2015.pdf; see also Francesca Dominici et al., Particulate Matter Matters, 344 SCIENCE 257 (2014).
Journal of Law & Politics [Vol. XXXIV:1 20
different areas of the U.S. must respond differently depending on their attainment status.
Copyright of Journal of Law & Politics is the property of Journal of Law & Politics and itscontent may not be copied or emailed to multiple sites or posted to a listserv without thecopyright holder's express written permission. However, users may print, download, or emailarticles for individual use.
