I will not talk about the philosophy of optimizing my algorithm, and what the backtest is for. In my opinion, it is necessary to optimize, but it is important to do it correctly. In my practice, backtesting plays an extremely small role for me when creating an algorithm, but still it's worth telling you about some aspects of this issue. First of all, I would like to show you how it all looks. Two examples:
in the first picture, you can see the default algorithm created for simple indicators, RSI 20 on top of SMA20. Buy when the indicator is close to 100, sell when it is close to zero. No filters and complications (this is necessary for this post). А table of results for 400 passes, from 5 to 100 with a step of 5 for each indicator is also shown as an example. In this table you can see that the number of negative results is quite small (a good example, no more).
The second picture shows an optimized and slightly different type of 3D graphics. On points, it is easier to understand the distribution of profitability depending on the choice of parameter. Accordingly, the green points are profit, red are loss.
A few rules I've worked out for myself:
1. Don't adjust your algorithm to the market. Profit on history does not guarantee income in real trading. Firstly, the machine selects parameters based on the already known history, which will not change, and secondly, we don’t not know how the market will change tomorrow. We are not even talking about global changes in trends, which is actually natural for any security. The technical side, the Commission amount, the price step, the cost of the price step, and other costs may also change. One way to get around the problem is to do backtesting and then forward testing. You can also generate random quotes for your paper and run the test on random quotes.
2. When analyzing the optimization results table, evaluate not only profitable parameters, but also unprofitable ones. It is not enough to observe positive statistics, because it may turn out that your algorithm is only effective in 1% of random parameters, and 99% of the statistics are negative. Therefore, it is best to either optimize the parameters in stages, or also conduct statistical analysis of the results.
3. Рarameter selection method for optimization. When creating an algorithm, I usually set the parameter myself. For example, realizing that the trading volume of 10,000 lots per hour is very small on a liquid ticker, I will not specify the selection of the parameter from 1 to 1,000,000, but will specify the value from 40,000 to 100,000 in increments of 5,000. If I understand that the average for a long period is important to me, then it would be stupid to optimize this parameter from 1 to 200, etc.
4. Don't believe, blindly, the results. When making optimization, you should use more than just statistics. For example, a 2% stop on a trade is not suitable for my risks. But I can analyze the results to assess whether I can change something and accept such risks (for example, if this way it is possible to reduce the number of losing trades in a row, and a more serious drawdown on the account, based on a series of losing trades).
5. The number of transactions should be sufficient for analysis. The most important thing for me is the number of transactions. YES, for long trends, when transactions are not made every month, it is difficult to collect statistics. And in this case, I usually take the algorithm, adapt it to local trends, and see how the system will in theory look like with a large number of transactions, at least 100 transactions per year. Otherwise, most machines can adjust statistics for 10 transactions — bought, sold and profit. And if the ticker is in a good trend, then do not get ahead of the algorithm, the script is bought and hold.
6. Close parameters, but distant results. If we observe that when a parameter is changed by one step, the results are very different, then most likely these parameters are not suitable for evaluating the quality and stability of the algorithm. For example, if the indicator period 70 gives a profit of $1000 and the period 65 gives a loss of $200, then the algorithm itself is very weak. It needs to be saturated with filters and modules for managing money and risks.
7. Diversify your trade with different variations of parameters. So, we have made a large number of backtests, and we can select a certain set of parameters that meet our requirements. Then you don't have to choose any specific parameters. You can run 2-5-10 scenarios, and observe the weak and strong zones of parameters, and it is possible to refuse any of them, and some will continue to work.
Share your observations and comments, as well as suggestions!) Thanks.
Submitted October 14, 2020 at 12:47AM by Saro_M_V