New study says widely-used models can be particularly misleading in performance evaluation

CAPM / Alpha Theory 14 Jul 2008

It seems to have become a financial axiom that actively managed mutual funds fail to justify their fees.  Ergo, index funds are often proposed as the best way to lose the least amount of money.

But what if the underperformance of actively managed funds has been driven by their underlying strategy, not their stock-picking buffoonery?

Beneath the complexity of their recent paper on benchmark indices, that’s the question posed by Martijn Cremers and Antti Petajisto of Yale and Eric Zitzewitz of Dartmouth. (You may recall the names Cremers and Petajisto from their paper on active share – a new metric to measure active management.  See related posting.)

The researchers found that academic models aimed at isolating manager skill by adding new variables to the CAPM (such as the Fama/French and Carhart models) are a substantial weakness.  Instead, they propose using actual indices as variables in an equation to reveal manager skill.

If this sounds familiar, it’s because William Sharpe himself actually used real-life indices in his seminal 1992 paper on Style Analysis.  But as the authors of this report point out, most of the literature since has relied on more academic variables such as Fama and French’s Small Minus Big (SMB) market cap variable.

It turns out that the pure index-based models actually perform the best in terms of pricing and performance evaluation, and thus they would serve as a good replacement for the commonly used academic factor models.  These indices, argue the researchers, represent well-diversified portfolios that naturally qualify as proxies for systematic factor risk.

Continues the paper:

…even pure index funds tracking common benchmark indices would appear to have significant positive or negative skill.  Yet these indices represent broad, well-diversified, and passive portfolios which almost by definition should have zero alphas…these nonzero index alphas are symptoms of a deeper problem in the Fama-French and Carhart methodology.

So What?

More than an esoteric academic debate, this has some pretty major implications on the way we analyse and rank mutual funds.  In fact, it can completely reverse the conclusions drawn from traditional analysis techniques.

In performance evaluation applications, we verify that the benchmark alphas can also have a significant impact. When we sort mutual funds into groups across size and value dimensions, the Carhart model indicates that small-cap funds underperformed large-cap funds by 2.13% per year from 1996 to 2005. But this result arises from the fact that the four-factor alphas of the small-cap benchmark indices are on average an astounding 5.07% per year less than that of the large-cap indices. If instead we control for the benchmark index of a fund, the results are completely reversed, and we find that small-cap funds outperformed large-cap funds by 2.94%. These numbers are certainly large enough to affect manager selection, especially given that mutual fund alphas are generally so close to zero.

They find that the typical actively-managed fund may have underperformed due to its focus on small caps, not due to bad stock picking…

Controlling for exposure to the Russell 2000 entirely eliminates the difference between the index funds and the median expense actively managed funds, and controlling for the S&P 500 eliminates much of the remaining difference between the median and most expensive funds. Expensive actively managed funds have loaded more heavily on the Russell 2000 and less heavily on the S&P 500, while index funds and inexpensive actively managed funds have done the reverse.

The bottom line, according to the authors:

The Fama-French and Carhart models can be particularly misleading in performance evaluation due to the large alphas they assign to passive benchmark indices, and they generate unnecessarily noisy alpha estimates. But also in standard asset pricing tests we can improve cross-sectional explanatory power by replacing the SMB and HML factors with index factors…Our analysis of mutual fund returns reveals the dramatic impact that the benchmark alphas can have on inferences about performance.

In other words, according to this research, the performance of active management seems to depend on your perspective.

Be Sociable, Share!

One Comment

  1. ryan brown
    July 15, 2008 at 2:42 pm

    Practioners would immediately classify the Russell 2000 as the ideal of small-cap returns, instead of SMB. Two problems retail investors encounter when replicating HML or SMB, that I’ve never seen addressed by academic:

    1). They are equal weight indices (actually hybrids of EW and VW), and no retail index/etf is equal-weighted
    2). DFA, the most likely source to replicate these indices does not short, so you couldn’t get returns associated with “ML” and “MB” from the the Fama-French epicenter.

    One thing that’s surprising is how little momentum (“UMD”) has been emphasized as a security selection strategy in the ETF/Fund space.

Leave A Reply

← Can US retail investors "fend for themselves"? Harry Markowitz →