20-day average and a neighborhood test: discard, don't tune

Idea in brief

John Bollinger, the creator of Bollinger Bands, explained in an interview how their team tests new systems. Bollinger deliberately picked a simple example: buy a stock on a close above the 20-day moving average, sell and go short on a close below it. There is always a position.

What comes next is not optimization but a check of the neighbors. Average lengths 16, 17, 18, 19, 21, 22, 23 and 24 are run. If they give roughly similar results, the system passes. If not, the whole system is discarded. Picking the best length or changing the rules until the neighborhood lines up is not allowed.

The card describes both a system and a protocol. The system can be replaced with any of your own that has one or two parameters, and the protocol stays the same. For most readers, the protocol is what matters.

Why it might work

Bollinger's logic: a rule based on a fundamental property of the market (trend, mean reversion, volatility) should not depend on an exact number. If 20 days works but 18 and 22 do not, the result came not from a market property but from a lucky match with a specific history. Such a system will fall apart at the first small change in the market.

Bollinger rejects optimization on principle. Together with rejecting leverage, Bollinger names this as what sets the team apart from most. Walk-forward is accepted as a working method, but not for Bollinger's own use: constant reoptimization over windows produces systems whose parameters change every few weeks. The goal is rules that work for years the way they were written.

A second heuristic from the same interview concerns the length of the description. A system that can be stated in two or three lines is worth testing. A system of 150 lines almost certainly is not. This system fits in one line.

A caveat: the protocol answers whether the system is overfitted. It does not answer whether it is profitable. Equally losing results on all nine lengths are also formally "similar". So the neighborhood test needs a separate quality criterion.

Rules

System (author)

MA = SMA(Close, 20)                              // Finetiq: the type of average is not named in the interview, we use a simple one

IF Close > MA THEN BUY AT NEXT BAR OPEN          // author: close above the average
IF Close < MA THEN SELL SHORT AT NEXT BAR OPEN   // author: close below, sell and go short
// Finetiq: execution at the next day's open, the author did not specify the entry moment
// always in the market: a buy closes the short, a sell closes the long
// the description has no stop, target or time exit
// size: Finetiq, a fixed fraction of capital without leverage (the author is against any leverage)

Neighborhood protocol (author)

Center    = 20
Neighbors = [16, 17, 18, 19, 21, 22, 23, 24]     // author

FOR N IN [Center] + Neighbors:
    backtest(MA = SMA(Close, N))                 // same data, period, costs and position size
    save metrics(N)

IF results(Neighbors) roughly similar to results(Center)
    PASS
ELSE
    DISCARD system                               // author: no optimization and no tuning

// Finetiq: the author gave no numeric criterion for "roughly similar". Starting rule for testing:
//   all 9 runs are profitable after costs
//   AND worst CAGR among neighbors >= 0.5 * CAGR(Center)
//   AND worst MaxDD among neighbors <= 1.5 * MaxDD(Center)
// Finetiq: after a failure, you cannot declare the best neighbor the new center and rerun the test

Time slices (author, no numbers)

// author: results on separate stretches of history must also be similar
Slices = split history into K equal stretches     // Finetiq: K = 4
FOR each Slice: metrics(Slice) with N = 20

IF at least one stretch is losing
   OR CAGR of the worst stretch < 0.25 * CAGR of the whole history     // Finetiq: starting threshold
    DISCARD system

// author: walk-forward and reoptimization over windows are not used

Addition: old systems as a ready-made out-of-sample (author)

Bollinger advises looking for ideas in old books. If the rules were published 20 years ago, all the time since publication is an honest out-of-sample that can be tested right away. The protocol is the same: the system exactly as written, the neighborhood of each parameter, time slices. As sources Bollinger names books by Garfield Drew, Welles Wilder ("New Concepts in Technical Trading Systems"), Charles Patel and Norman Fosback ("Stock Market Logic").

Parameters

What to test

1. The neighborhood itself. Nine runs, 16–24, on one instrument. Plot the metric as a function of average length. A flat plateau and a sharp peak at 20 lead to opposite conclusions.
2. Neighborhood on a basket. The same nine lengths on 20–50 stocks or futures. For each length, look at the median across instruments. A single stock passes the test by chance much more easily than a basket.
3. Which metric decides. Test the protocol on net profit, on Sharpe and on maximum drawdown. A system can be similar in profit and very different in drawdown. Decide in advance which metric gives the verdict, otherwise the choice of metric becomes a new form of overfitting.
4. Protocol versus optimization. Take the first half of history, find the best length from 5–100 and test its neighborhood. Then, on the second half, compare the systems that passed and failed the test. If those that passed are not better, the "similar" criterion is too soft.
5. Costs. A reversal on the crossing of price and the 20-day average happens often in a range. Run all nine lengths with commission and slippage. Short averages reverse more often, so costs hit the neighborhood unevenly.
6. Time slices. Split history into four stretches. Check whether the whole result rests on one year with a strong trend (for example, 2008 or 2020).
7. The short half. Bollinger trades US stocks and talks about their long-term growth. Compare the full system, longs only, and a "long above the average, out of the market below" variant.

Platform notes

TradingView (Pine Script)

• The TradingView strategy tester has no built-in parameter sweep. The nine runs have to be done by hand by changing the input. An alternative: compute nine virtual systems in one indicator and show their results in a table.
• With manual runs, it is easy to change the history period or costs between runs without noticing. Fix the date range, commission and slippage in strategy().
• By default process_orders_on_close = false: signal at the day's close, execution at the next day's open, as in the rules.
• strategy.entry in the opposite direction reverses the position, so no separate exit is needed.
• History length is limited by the subscription plan. The four time slices need long daily history on all nine runs.

MultiCharts and TradeStation (EasyLanguage)

• The 16–24 neighborhood can be run with the standard optimizer: an input for the average length, step 1. Look at the table of all runs, not the best row.
• Buy next bar at market and SellShort next bar at market reverse the position by themselves.
• For a basket of stocks: Portfolio Maestro in TradeStation, Portfolio Trader in MultiCharts. The protocol is then applied to the median across the basket.
• For futures in TradeStation, the daily bar closes at settlement and the 1440-minute bar at the last trade (Kevin Davey, The Algorithmic Advantage #036). The "close above the average" signal will differ between the two bar types. For stocks this does not matter.

MetaTrader 5 (MQL5)

• The tester has a slow complete search. For nine values, choose it rather than the genetic algorithm to get all the runs.
• Average: iMA(..., MODE_SMA, ...). Take the signal from the closed bar (index 1) and check for a new daily bar.
• The daily bar is built on the broker's server time. Short Sunday bars at brokers whose server is not on GMT+2/+3 add an extra "day" to the 20-day average.
• Stocks and indices in MT5 are most often CFDs. A swap fee is charged for every night in a position, and this system is always in the market. Include it in the test, otherwise the result is inflated.

Where the idea can break

• Bollinger described an example of the procedure, not Bollinger's own trading system. The interview gives no figures for it and no claim that it is profitable.
• The criterion for "roughly similar results" is not named. All thresholds in the card are ours. A criterion that is too soft lets overfitting through, one that is too strict discards everything.
• A neighborhood of four days in each direction is narrow. All nine lengths can benefit equally from quirks of the period. Time slices and a basket of markets protect against this, not the neighborhood itself.
• The neighborhood test does not protect against idea selection. If you try twenty different systems and keep the one whose neighborhood lined up, overfitting still happened, just one level higher.
• Always in the market on a 20-day average reverses often in a range. On stocks with a wide spread and on CFDs with swap fees, costs can eat the whole result.
• The author trades US stocks in a long rising market. The short half of the system goes against long-term growth in that market.