How Accurate are the Power Calculations Relied on by the SEC in its Regulatory Deliberations?

In two related decisions in Chamber of Commerce of the United States of America v. Securities and Exchange Commission (SEC), the District of Columbia Federal Court of Appeals ruled that the SEC had not fully complied with some provisions of the Administrative Procedures Act when it required that the boards of investment companies managing mutual funds have at least 75% of their membership and the Chairman be independent directors. In preparation for renewed rule making, the Office of Economic Analysis of the SEC prepared a Power Study to respond to an industry-sponsored report claiming that the returns of funds with independent boards and chairs are not superior to funds with boards dominated by management. The Power Study concluded that the available studies on the effectiveness of independent board members do not have sufficient statistical power to detect a meaningful difference in the returns of the two types of funds. This paper demonstrates that the method used by the SEC in their power calculation is not correct, unless a very restrictive condition that rarely occurs in practice holds. When the appropriate power formulas are used, the expected power of studies of the same size as the ones examined by the SEC is actually lower than the corresponding results of the SEC. Thus, the results in the paper actually strengthen the argument that the SEC is advocating. The relevance of both the SEC and the industry studies to the main issue in the case is also questioned in the discussion