State Street MD on hedge replication model: “I was shocked the explanatory power was so high”

4 Comments

1. 

   Noël Amenc and Lionel Martellini

   May 13, 2013 at 9:06 am

   In the present article, a number of comments are made about a paper we published in the European Financial Management Journal (Amenc, N., L. Martellini, J.-C. Meyfredi and V. Ziemann, 2010, Passive hedge fund replication — Beyond the linear case, European Financial Management Journal, 16, 2, 191-210.) We feel that some of these comments deserve a response, as they might be misleading for the reader who is not overly familiar with the research.

   The contribution of our paper published in the European Financial Management Journal is to extend Hasanhodzic and Lo (2007) by assessing the out-of-sample performance of various non-linear and conditional hedge fund replication models. In a nutshell, our ambition was to improve the performance results in Hasanhodzic and Lo (2007) by considering more sophisticated models and we were quite surprised to find that going beyond the linear case does not necessarily enhance the replication power.

   The article “Hedge Fund Replication: A Re-examination Of Two Key Studies” claims that our paper “includes some very helpful analysis […] but whose conclusions are undermined by selective omission. For instance: Even though there was over two years of live data from replication indices that showed strong results with high correlation through the crisis, the authors neglect to include this and focus instead on re-doing the Lo analysis with the admittedly incomplete five factor set.” We indeed decided not to analyze live performance data from passive replication products, and this for two reasons. First of all, this was not the focus of our paper, which instead was (as mentioned above) to try and improve over linear replication models. Secondly, we did not feel that two years was a sufficiently long sample to allow for any meaningful statistical analysis. It is our belief that showing coherence in the scope of a research project paper, and avoiding meaningless statistical analysis, should be called something other than “selective omission.”

   A second criticism made in the article “Hedge Fund Replication: A Re-examination of Two Key Studies” is that we use a selection of useful factors for each strategy, as opposed to using a large identical set of factors for all strategies. This criticism is phrased as follows: “By starkly reducing the factor set, the authors essentially designed an experiment that was bound to fail.” Unfortunately, we have found that while including more factors improves the performance of replication models in-sample, it tends to hurt the out-of-sample performance of such models. In short, parsimony is a well-known necessary condition for out-of-sample robustness, and we feel it is misleading to claim that one only needs to add an increasingly large number of factors to generate satisfactory hedge fund replication performance.

   Finally, a comment is made that “It is difficult to read this paper without the sense that the authors, who are closely tied to the fund of hedge fund industry (and funded by Newedge), had a predetermined agenda.” We feel that this comment is out of place. EDHEC-Risk has always been known for publishing unbiased academic research and at no point in the research process did Newedge intervene to influence the results in any possible way. In the same way that we consider that professionals have the right to be taken seriously and to be criticized for what they write and not for what they are when they express scientific views, we believe that authors from the academic world deserve the same level of respect. If the only real argument for criticizing a research paper is to disparage the authors’ conduct with no evidence, then we do not think that this criticism is admissible. Just like we think it is logical to display the financial contributions to our research programs that the Institute receives from our sponsors (the authors are not beneficiaries), we also think it is logical for this concern for transparency and the sponsor’s desire to support transparent and independent research to be recognized and not denigrated.

   ---

2. 

   Andrew Beer

   August 15, 2013 at 12:05 pm

   Dear Professors Amenc and Martellini:

   My apologies for the delayed response. Thank you for taking the time to read and provide a critique of my note. I will respond in order:

   1. The reason I highlighted the live performance was that the extant replicators were largely successful at tracking industry returns in 2007-09. As a practitioner and researcher, this is valuable information. It was not a coincidence that the models had similar features — 24 month window length, 5-8 factors across major asset classes, etc. Those parameters were chosen because highly capable researchers at investment banks and asset management firms were conducting similar experiments and reached similar conclusions. Consequently, it seemed to me at the time that there was a lot of other research and live validation that a variant on the Hasandhozic/Lo experiment could work well. This seemed pertinent to me, but perhaps it wasn’t to you.

   2. We’ve found that more factors do in fact improve out of sample results, but up to a limit. Since I don’t have your research, I cannot comment on the specific results you refer to. I fully agree with the concept of parsimony, which we’ve embraced while other firms have added unnecessary factors for appearance of complexity. In addition — and this may simply be a semantic issue — I don’t see how going from 5 or 6 factors to one or two is an “extension” of the study. Instead of working to extend or modify the H/Lo factors, you seemed to scrap them entirely and start again with a much narrower pool — one or two factors for many sub strategies. At the time, I thought it was well established that one or two factor models didn’t work well. I still don’t understand this transition in your paper and it seems to me like you ended up with a very different study (perhaps “Can a two factor model explain hedge fund sub sector returns out of sample?”). As a practitioner, my interpretation of this was that the poor outcome was a foregone conclusion, and hence I questioned why it was included.

   3. I apologize for the insinuation. This was based on a rumor at the time of publication — I don’t remember the source, probably from one of the banks who of course had their own agenda. As a practical matter, though, I would argue that no research is unbiased. As a liberal arts major, I am trained to read into subtext, and my strong impression from reading your papers and the way they were structured was that you had a vested interest in arguing against what you describe as “passive” replication. If I misinterpreted this, then I apologize.

   If you would like to have a live debate on any of these issues, please feel free to contact me directly. And thank you again for taking the time to read and carefully respond to my submission.

   All the best,

   ADB

   ---

3. 

   Prof Jim Liew

   May 5, 2015 at 11:19 am

   That definitely would be fun to see a live debate! Did you guys do it?

   ---