NEW TECHNIQUES
Developing Indicators
Testing Signal Efficacy
by Jeff Parent
Here's how you can evaluate how effective your trading system is.
Not unlike new pharmaceutical drugs, buy and sell signals must be put through rigorous tests to determine their usefulness. Only when signals prove profitable repeatedly should the indicator being tested be incorporated into a trading system. But how can you quantify how effective your signal is? In this article I will suggest a standardized process to achieve this goal. The steps involved in this process are:
1. Test a similar batch of stocks under the same conditions and time periods.
2. Compile a composite for the batch for relative performance comparison.
3. Match the outcome of a random selection to the selection chosen by the indicator.
4. Test for statistical significance to determine if the indicator is tradable.
I once undertook an intensive quest for a magic, holy grail indicator. I didn't find one, but from all my research I did manage to devise a good high-volatility (range) predictor.
After a more detailed examination of the data, however, I made some unfortunate discoveries - unfortunate because they nullified all my previous work. The predictor selected low-priced stocks, and it made more selections in unusually volatile markets. This brought to light two significant relationships. One was the relationship between volatility and stock price: the typical percentage change in a $1 stock is higher than in a $50 stock. The second focused on relative versus absolute performance (buy signals are more accurate in rising markets). My naïve approach may not have resulted in an indicator, but it wasn't a wasted effort either. It helped me to expose some correlations and improved my analytical skills.
FIGURE 1: THE COMPOSITE INDICATOR. Split-adjusted price data from about 490 stocks qualified each day to be included in the index.
...Continued in the December 2002 issue of Technical Analysis of STOCKS & COMMODITIES
Excerpted from an article originally published in the December 2002 issue of Technical Analysis of STOCKS & COMMODITIES magazine. All rights reserved. © Copyright 2002, Technical Analysis, Inc.