PLEASE NOTE: I do not invite anyone who has anything to say about 'over-optimisation' to comment on this post. I am not requesting your opinion if it involves if/why/how this is over-optimisation.
Let's imagine that you have an algorithm with multiple (20 or so) inputs. The inputs are grouped into 'families', where each family represents an indicator. For example, the MACD family has 4 inputs, numbered 1 to 4. Each family has no less than 3 inputs.
Let's also assume that you have a simple backtesting/optimisation engine. You want to get the best output for a defined backtesting period (net profit, hit rate, drawdown etc). Using your backtesting engine, you are able to optimise a group of inputs by specifying the Start, Step, and Stop values. A sequential optimisation approach is taken by the optimisation engine, such that if you were optimising two parameters simultaneously (P1 and P2), then P1 would be set to its start value, and P2 would be scanned through against P1 using the steps. This would be repeated for each step of P1. For example, if P1start = 1, P1step = 1, P1end = 3, P2start = 5, P2step = 10 and P2end = 25, then the scan would proceed like so:
P1 = 1 , P2 = 5
P1 = 1, P2 = 15,
P1 = 1, P2 = 25,
P1 = 2, P2 = 5,
P1 = 2, P2 =15
Each backtest/optimisation iteration takes 1 minute to run. Let's assume that each individual optimisation should take no longer than two or three hours, so if you were optimising 10 inputs, which involved 5 families each with 2 inputs, and you did each family one at a time, then it would take in total between 10 and 15 hours for the whole optimisation.
There are two approaches (that I am aware of) that allow you to find optimum parameters, without waiting a very long time for the optimisation to finish.
Approach 1: Let's say your first family has 3 inputs. We choose only two of those inputs out of the three and optimise using 10 steps for each. In total, this is 100 runs. Once that optimisation is done, you choose the values for those parameters that came out best, and that is your new base program. Then you select one of the inputs that you just optimised, and run it against the one that you haven't yet optimised. Again, 10 x 10, so 100. Then you do your final 10 x 10 optimisation on the remaining pair. With this approach, you can start off fairly coarse (Start = 10, Step = 10, Stop = 100), and then repeat the whole process again with a finer optimisation (Start = x, Step = 5, Stop = y), until you get your optimum parameters. You then must repeat this for each family.
Approach 2: Let's say you are optimising a family with 4 inputs. In this approach, you optimise all the inputs for each family simultaneously. You start off very coarse, and progressively hone in with finer optimisations. For your first run, your scan values for all 4 inputs would be Start = 10, Step = 20, Stop = 100. This means 5 x 5 x 5 x5 = 125 runs. Then select your optimum, and repeat multiple times, each with a finer step size (eg 10, then 5, then 2 etc). You then must repeat this for each family.
From my experience, Approach 1 works better, but I haven't practiced Approach 2 enough to be sure.
My questions are:
1) Are both approaches capable of landing on the optimum parameters for a specified time period (can this be mathematically proven)?
2) If appropriate, which approach is superior and why?
3) Are there any other approaches that you are aware of, and are these better? If so, why?
I invite those with qualitative arguments and/or quantitative arguments to provide their thoughts.
Submitted October 08, 2020 at 06:18AM by ph0ph0