The proposition that a prudent investor should be diversified is widely accepted if not incontrovertible for ordinary investors – investors who have no reasonable expectation of influencing company management or business policy. Indeed, fiduciary duty requires that trustees and other investment managers assure that their clients be diversified. But the idea of diversification is not well articulated or understood even by sophisticated investors. This article fills that gap by laying out the logic (and mathematics) of diversification for ordinary investors who invest in common stock. As shown here, diversification can eliminate almost all of the company-specific risk that goes with investing in equities without any sacrifice of expected return. The only risk that remains is the risk that the market as a whole may do better or worse than expected. Thus, it is often said that diversification is the only free lunch in the market. It follows that diversified investors who assume less risk because they are diversified will pay more for the shares in which they invest and will thus dictate market prices. It further follows that undiversified stock-picking investors assume more risk than necessary and thus pay too much for the stocks in which they invest. In other words, the market has eaten their free lunch. The logic of diversification explains why investors have flocked to index funds – which offer maximum diversification for minimum fees – and why almost half of all stock in US companies is held by such funds. But the idea of diversification alone does not explain how much to invest in which companies. Fortunately, the market provides the answer to this question, and the answer turns out to be indexing. Generally speaking, we can depend on individual companies to maximize returns for their own stockholders by seeking out the most profitable opportunities in which to do business. Since market price is a function of expected return (divided by the required rate of return), it follows that ordinary investors should invest their funds in proportion to the aggregate market value of investee companies because by doing so investors are assured that their money is spread evenly across all lines of business in the economy in proportion to prospects for return. Thus, ordinary investors should invest in index funds that hold shares in proportion to the market capitalization of all possible portfolio stocks. It turns out that the 500 (or so) largest US stocks account for more than 80% of total market value. And as it happens, the value-weighted version of the S&P 500 (SPX) comprises exactly that portfolio. It follows that investing in an index fund that tracks SPX is a good way – possibly the best way – achieve maximum diversification at the lowest possible expense. Thus, it is mostly coincidence that investors favor index funds that track SPX as opposed to other indices. SPX was designed to measure the market and not as a normative strategy for investing. But it turns out that SPX provides the best guide for how to allocate investment funds within a portfolio of US common stocks. In other words, if SPX did not exist, it would be necessary to invent it. Moreover, by so allocating funds, the expenses of investment management can be kept to a minimum, because following an affirmative strategy of indexing precludes expending any fund assets on company-specific research. To engage in such research would be a literal waste of assets – and thus a breach of fiduciary duty – since there is nothing fund managers can do with the fruits thereof without violating an announced strategy of indexing. Similarly, indexing keeps the expenses of trading at the fund level to a minimum because the only trading that is necessary or appropriate is portfolio balance trading (PBT) to keep fund holdings proportional to the market capitalization of portfolio companies.Finally, it is wrong to characterize indexing as a passive investment strategy or one by which index investors free-ride on the efforts of traditional stock-picking investors who engage in company-specific fundamental research. To the contrary, indexing magnifies the disciplinary effects of market prices on portfolio company management. Moreover, indexing by some (or many) investors creates trading opportunities for other investors since index investors effectively cede first mover advantages to investors who engage in company-specific research. Thus, there is a symbiotic relationship between index investors and activist investors – and indeed among all investors of diverse interests – that arguably makes the market more efficient than it would otherwise be (in the absence of indexing)
