A recent US study confirmed the finding that high school grades are the most accurate predictor of student success in the first year of college. This is immediately intuitive. In a similar way, the past performance of investment managers is often imagined to be a useful predictor of future returns. In our industry, intuition does not correspond with reality as a good period of performance is just as likely to be followed by a poor one. This would not be such a problem were it not for the fact that even some of the smartest investors tend to disregard the evidence and their own bitter experience, and treat track records as shorthand for skill. “If they had outperformed then they must be skilful, and vice versa”.

But given that reality is not that simple and track records have proven to be an unreliable predictor of future performance, is it any wonder that even sophisticated pension funds are left questioning if skill exists at all and, if it does, do they have the tools to find it? The extensive evidence from Inalytics’ database of managers shows that skill does in fact exist. This implies that it is the traditional methods used to find it that are failing the industry.

Accepting that managers have skill, but it is the tools used to find it that have let us down, requires a radical change in the way we go about evaluating managers. By turning away from track records, and towards an evidence based approach that analyzes every decision a manager takes to identify their real strengths and weaknesses, we have found the framework we are seeking. This methodology involves putting every investment decision under the microscope to obtain a clear picture of a manager’s DNA. In practice this means analyzing every purchase and sell, and every decision to include or exclude a stock from the portfolio on a daily basis, so as to evaluate how well they construct portfolios and time trades.

Identifying skill comes down to whether:

• Overweight positions in the portfolio perform well;
• Underweights perform badly;
• Buys outperform; or
• Sells underperform.

These are simple common sense metrics that establish whether the manager is typically correct when they are positive on a stock and if they also have the ability to identify poorly performing stocks and sell them before a period of poor performance. This process lays bare the facts: either the decisions added value on average or they did not – end of story, no waffle, no excuses. The fundamental point is that a true evaluation of skill comes from a rigorous and detailed analysis of every decision a manager takes so as to identify their real strengths and weaknesses and where the balance lies between them. These are insights that are not available from looking at track records alone.

Hit rates and the win loss ratios – a step up from track records
Shifting the focus from track record to decisions gets us closer to understanding the nature of investment skill, but it also creates the challenge of how to handle these hundreds of thousands if not millions of decisions in a way that provides real insights into a manager’s investment skills. Inalytics has previously introduced the concepts of hit rates and win loss ratios as an efficient and intuitive way of getting behind the track records. These two measures answer the basic yet profound questions of whether a manager gets more decisions right than wrong (hit rates) and whether the good decisions offset the poor ones (win loss ratios). Simple intuitive questions that investors have been asking for a long time, but up to now the industry has not had the answer. In this paper we show that the average hit rate for the portfolios in the Inalytics database was only 49.6% meaning that managers don’t get six out of 10 decisions right, as was previously assumed; the reality is much more pedestrian and disappointing at slightly less than half of all decisions taken. This disappointing outcome is offset however by an average win loss ratio of 102%, which implies that good decisions offset the poor decisions, albeit by a small margin. These numbers relate to the averages across all the decisions that managers take. However we have seen in a previous research paper that when managers have skill, it manifests itself in the positive or overweight decisions as shown Table 1.

This data illustrates one of the most well established phenomena in the world of behavioural economics, the endowment effect which suggests managers add value when they are positive on a stock and lose value when they are negative. Having identified that managers’ good decisions more than offset their poorer ones when they are positive on a stock the next question is how persistent is this phenomenon? The data indicates that if the ratio is favourable, or above 100 in any given month, then it is 55.6% likely to be above 100 the next month. This may seem on the face of it a relatively low probability, but given the extensive size of the population, the result is highly statistically significant. The complete data is shown in Table 2. These results demonstrate that these measures get closer to the heart of the question of whether a manager has skill and if it is persistent.

Identifying Skill
This process of identifying each decision a manager takes and systematically evaluating whether they were successful or not, elevates our understanding of the nature of investment skill. By standing back from the noise, and looking at the cold evidence, we can see what managers are good at and where their gaps lie.

Our experience, and the feedback from our clients, strongly suggests that this effort to identify and quantify investment skill is worthwhile. Investment managers, in turn, are able to really examine their strengths and weaknesses. Both are equally important if they want to improve and refine their investment process. We shall see in the following section how GLG uses the analysis to provide objective feedback to their managers and traders as part of the process of continual improvement.

Applying the methodology in practice
GLG uses the information primarily to provide an evidence-based analysis and objective feedback to the managers and traders. Due to its objective nature, the analysis is unambiguous and free from opinion. It highlights strengths and weaknesses of the investment process in order to maintain what the manager is good at and provides a focus to develop the weaker areas. In order to understand whether a manager has skill, every decision the manager takes must be examined.

There are two types of decisions that fund managers take which matter. Managers have a very simple choice. They either own a stock or they don’t.

• They either buy the stock, or increase their weighting in it; or
• They sell the stock, or decrease their weighting in it.

The practice of expressing investment convictions is referred to as conviction analysis. If we put these decisions under the microscope, then evaluating manager skill becomes transparent and objectively analysed through a quantitative, forensic process. This process has been applied to two of GLG’s portfolios.

Methodology
Our approach has involved using conviction analysis to look at the persistency of GLG’s decision-making over time. We compiled data on every stock in the two portfolios examined and their benchmarks on a monthly basis, and were able to make the following key examinations:

Contribution of conviction positive decisions
We examined the monthly alpha generated by the portfolios relative to their benchmark weightings, assigning each stock to one of five categories, heavily underweight, underweight, neutral, overweight, and heavily overweight (see Figs. 1 and 4). Overweight positions in the portfolio should perform well, and underweights perform badly.

Hit rates and win loss ratios
We examined the managers’ hit rates and win loss ratios, and compared them to our sample database of 215 traditional long only portfolios, which excludes hedge funds. The hit rate is defined as the number of winners in the portfolio as a percentage of the total number of observations. The win loss ratio compares the outperformance that comes from good decision making to the alpha lost from making poor decisions. The average hit rate for the portfolios in the Inalytics database was 49.6%. The average win loss ratio was 102%.

Sector breadth and style bias
We analyzed the portfolios across all sectors in the benchmark, and looked at style bias.

Manager timing
We made observations on the managers’ timing skills by examining four categories:

• Initiation: the purchase of a stock whose previous weighting inthe portfolio was zero.
• Scale up: the increased allocation of an existing holding.
• Closing: the complete elimination of a position.
• Scale down: the sale of part of an existing holding.

We examined the style of a manager’s implementation both compared to pre-trade returns over three and six months, and post trade performance over three, six, and 12 months.

The GLG European mandate
The GLG European mandate achieved very strong results, as demonstrated by the performance of its heavily overweight stocks (see Fig.1) which represented the highest positive conviction positions.

More impressively, all other parts of the portfolio also made positive contributions. The portfolio’s alpha generation has also been persistent; 70% of the months analysed in the heavily overweight category delivered positive contribution, as demonstrated in Fig.2. Like most managers however, the portfolio suffered some performance difficulties during the latter half of 2008, as their heavily overweight stocks lost value over that period.

Hit rate and win loss ratios
The hit rate for this manager has been 53.1%, and the win loss ratio is 120%, both well above the peer group average, although both have suffered since mid- 2008. The manager has a higher hit rate and win loss ratio than you would expect from chance alone.

Style bias
The portfolio has added substantial value from the large and small cap positions in the portfolio (see Table 3). Furthermore, over 50% of the total outperformance emanates from stocks that are not in the benchmark (see Fig.3). The portfolio has also demonstrated positive alpha across every sector of the benchmark.

Manager’s timing skills
As seen from Table 4, the manager made Initiations in stocks which had underperformed marginally in the three and six month period leading up to purchase, demonstrating that the manager has a slight contrarian tendency when initiating a holding. Initiated holdings’ relative performance was, on average, subsequently positive.

When scaling up however, the manager’s buying process was characterised by a strong momentum bias, tending to buy into a period of outperformance. Hit rates and win loss ratios for stocks that have been scaled up tend to be lower than those that have been initiated, and performance subsequently was negative on average. The manager demonstrates skill once again when closing and scaling down, selling out of stocks that subsequently underperform. For this portfolio, the selling process is more successful than the buying process. It is certainly more persistent.

The GLG Global mandate
As seen by Fig.4, the GLG Global mandate generated the bulk of its alpha from the highest conviction position decisions represented by the heavily overweight category. The heavily underweight category also added value, so value was added from two sides of the extreme. The portfolio also had a negative contribution from its neutral category, indicating that small cap stocks have outperformed the benchmark as an asset class over the period under review.

Hit rate and win loss ratios
The alpha was generated by a win loss ratio of 111%, which is in line with the peer group average and a hit rate of just slightly over 50%, which is above average of the other managers at 48.5%. The last three quarters reviewed saw a sharp downturn in results for the heavily overweight category, due to a hit rate of 42% and a win loss ratio of 79%. The portfolio has added considerable value from the large cap positions in the portfolio but also from small caps and off-benchmark positions. The portfolio has also demonstrated positive alpha across the majority of sectors, but it is by no means a clean sweep (see Fig.5). Energy and materials have contributed negatively, while information technology and financials have added the most value (see Table 5).

Manager’s timing skills
As seen from Table 6, the manager made initiations in stocks which had underperformed over the three and six month period prior to purchase, demonstrating that the manager has a contrarian bias when initiating a holding. Initiated holdings subsequently outperformed strongly. The manager is less successful when scaling up, with hit rates consistently less than 50%. The managers’ scaling up process was characterised by a momentum bias, although the results were mixed, implying stocks that were scaled up performed close to benchmark returns, on average. The manager’s selling categories have demonstrated extreme implementation styles. The manager tends to scale down stocks that are performing well in the portfolio whereas stocks subject to closing had previously underperformed quite sharply. Both strategies, though, were successful as stocks that were sold, either scaled down or closed, underperformed subsequent to the trade.

Simon Savage is a Portfolio Manager at GLG Partners LP. He has 18 years of experience in portfolio management and investment research. He joined GLG in 2004 as a co-manager of the GLG European Long-Short Fund. Simon previously worked for Morgan Stanley, Bear Stearns and Nomura. He has a BA in Physics from Oxford University.

Rick Di Mascio is Chief Executive and founder of Inalytics Limited. Prior to establishing the company, he held a number of senior positions within the industry, for companies including CINMan Limited, Goldman Sachs Asset Management and Olympus Capital Management. He is also Chairman of the T-Charter group.