We have talked about the dangers of optimizing trading systems (forcing trading systems to conform to historical data), but, one of the subtler types of optimization occurs with portfolio selection. This happens when trading systems are only shown tested across a handful or a small number of markets (or occasionally just one market).

The trouble is that what is typically done is that most all the available markets get tested, and then only those that performed the best get shown in the portfolio. This is an tremendous blunder, because the markets that performed best historically are rarely the ones that continue to be the best. What traders end up with is something that just worked well historically.

To prevent this bias, we think that the most robust method to see a system test is across ALL the available markets. Some will argue that different markets should be traded different ways, and to that we say Rubbish. Markets are always changing, and a market that traded like market XYZ today will trade like market ZYX tomorrow. Until a system is robust enough to trade every market, it is probably a useless curve fit of the data.

For us in the commodities markets, we test roughly 80 markets. There are over 100 futures contracts that trade, but we do limit the choice to those that are liquid enough (have enough trading volume) to trade.

Testing this way does cause one problem. The difficulty is that traders can be trading numerous markets in the same sector at a given time. Investors will need to have some sector risk management mechanisms in place. We like to be certain that the risk in a given sector does not go beyond about 5% of the account equity.

If an individual creates a system that can effectively trade nearly EVERY commodity market and uses the identical rules for each market and gets tested over a long period, he may be on to something. Just keep in mind, the next time someone shows the results of a backtest ask him „How many markets does this test include?“ If the answer (for commodities) is less than about 70 or 80, then be on your guard that this may be curve fit results. Once again, curve fitting tends to produce systems that ONLY perform well in the past.

We test all the systems we make available across practically every tradable commodity market. We could easily enhance performance (historically) by simply cherry picking the best markets, but we realize this is misleading data.

Trading Systems Some trading systems like moving average systems do not know how much risk they are taking. Position sizing is determining HOW MANY contracts to trade when a trading system gets a signal. I do not think I am being unfairly biased; I’ve spent over 15 years investigating all types of trading systems.