VR Logo

Analyzing Wronged Equity Analysis

When the markets were bullish, equity analysis were working, but stopped when the markets turned bearish

I recently happened to have a chat with a youngster who works with a prominent national-level stock broker. His job is to deal with clients on the phone, give them advice about which stocks to buy and sell and execute the orders. The advice is based on whatever the research department would feed him. According to him, after January, the advice has become useless. “Our technical analysis stopped working,” was what he told me. When the markets were rising strongly, the research department’s analysis used to work and after that it stopped working. Presumably, the researchers and analysts were doing exactly what they were doing earlier, but the recommendations they made just stopped working.

Just a few days earlier, I had been told something identical by the Chief Investment Officer of a large mutual fund company. The CIO’s actual words were that ‘accuracy of their analytical models declined sharply’ but that’s just a fancy way of saying the same thing.

Why did things just stopped working? Here’s an example that may provide an interesting clue. Variations of this example are often used to explain the accuracy of various survey and testing methods. Imagine a medical test for some disease that is 99 per cent accurate. That is, it correctly identifies 99 per cent of sick people as having the disease. It also identifies 99 per cent of healthy people as being free from the disease. In the terminology of statistics, both the specificity and the sensitivity of the test is 99 per cent.

Now, let us apply this test to a population of one lakh in which this disease has an incidence of about half a per cent, meaning that 500 people suffer from it and the other 99,500 are free from it. The test’s accuracy level means that it will erroneously identify 1 per cent of the healthy people as sick and one per cent of the sick people as healthy. Therefore, of the 99,500 healthy ones, the test will identify 98,505 people as healthy and 995 as sick. Similarly, of the 500 people who are actually sick, 495 are correctly identified as sick and 5 are erroneously tagged as healthy. In all, 1,490 people have been identified as sick, of whom 995 are actually healthy! A test that is 99 per cent accurate has produced a result that is 66 per cent wrong in revealing sickness but almost completely accurate in revealing the lack of sickness!

This thought experiment can help you understand why so much equity analysis has stopped working. Substitute people for companies, sickness and health for good and bad stocks, and investment analysis or the diagnostic test and you’ll have the explanation.

Till January this year, perhaps 80 per cent of the stocks were healthy. Now, 20 per cent are healthy. And mind you, equity analysis is less (a lot less) accurate than 99 per cent. If the test above was 70 per cent accurate and was used in an 80:20 situation then its error rate would be above 70 per cent.

Many equity analysts are charlatans who just say any old thing that is needed to sell stocks. But even if they are perfectly sincere, and even if they believe in their methods, the accuracy of short-term equity predictions is hobbled by the inexorable arithmetic of chance.