What is Backtesting?

Backtesting is the execution of a strategy or signal on historical data. You “pretend” that historical prices are happening “right now” and see how the strategy would work in such a market. To conduct a backtest, you need to select a strategy and the historical range on which you want to test this strategy. It is usually interesting to test a strategy over a long period of time (months and sometimes years).

For example, below are the balance and equity curves of the standard MA_ADVISOR (Moving Average Crossover) strategy, performed on the 2010 history of EUR/CHF (3.5 million ticks, 8 ticks per minute), applied on a 15-minute interval, with the number of periods for the fast moving average - 15 intervals, for the slow one - 55, without stops and limits, on a 1-hour chart. You can see all the "classic" problems with moving average crossover strategies: it loses in a slow market and only performs well in a strong trend.

The tabs related to tables and optimizer are empty.

Special Warning About Backtesting Results

Hypothetical or simulated performance results have certain limitations. Unlike actual financial results, simulated results do not represent actual trading. In addition, since trading operations were not executed, the results may be either overestimated or underestimated, since in reality there is the influence of certain market factors, for example, such as lack of liquidity. Programs that simulate trading are generally designed with profit in mind. There is no representation that any account will experience profits or losses similar to those shown.

Previously, we have already considered the issue of the mandatory stages of developing a trading strategy for working in the stock market. One of the most important stages is testing the strategy's performance on historical data - backtesting. Today we will talk about him.

What is this

In simple terms, backtesting involves running a trading strategy algorithm using historical financial data. The algorithm, having detected certain exchange events (“signals”), will generate orders to buy or sell financial instruments - these operations will have an associated income or loss.

The total amount of income or loss (profit and loss, P&L, PnL) for the time specified in the trading strategy will be an indicator of the success or failure of the algorithm.

There are several goals that trading software developers achieve using backtesting:

• Filtration- each strategy has certain performance and efficiency indicators that are included in it by the developer. Accordingly, any strategy that does not allow achieving the set goals must be “filtered out.”
• Modeling- with the help of backtesting, developers can test various market models (changes in liquidity conditions, transaction costs, order processing speed, channel delays, etc.) without the risk of losing real money.
• Optimization- by “running” a strategy on historical data, you can improve its performance in specific market situations.
• Functionality check- with the help of testing, the developer can understand whether errors were made when describing the strategy in the program code.

As you can see, backtesting is a useful tool for developers of financial systems, but it is not always possible to correctly test on historical data. The more high-frequency strategy must be implemented, the more difficult it is to correctly model the impact of certain market situations and parameters of a specific exchange platform on the overall performance of the system.

Misconceptions about backtesting

Well-known expert on stock trading, quant and developer of stock robots Michael Hulls-Moore, is convinced that novice developers of stock exchange systems often make mistakes when creating them due to certain misconceptions. In particular, the expert cites four such misconceptions:

Expecting equally good results in the future

A developer is often tempted to make changes to testing parameters to obtain more convincing results.

At the same time, if in the case of historical data it is possible to change something and accurately predict the result, then in “combat” mode the robot may not work as efficiently. It is necessary to measure the performance of the strategy for different values ​​of input parameters.

Using "future" data

In some cases, the creators of trading strategies include assumptions about the future state of the market in the data set. In case of errors in the code, incorrect calculation of the optimal parameters for the strategy, or incorrect use of extreme price values ​​(highs and lows), launching such a strategy on the real market may be unsuccessful (this is one of the most common reasons why strategies work more efficiently on historical data, than in real time).

Incorrect assessment of your psychological stability

When conducting tests, the developer sees the final performance of his algorithm. If over a certain period of time (say, a year or five years) the system makes a profit, then there is a great temptation to not pay attention to the deposit drawdowns (losses received) that happened along this path to success. It seems to people that they can easily survive the loss of 25% of their money (after all, the robot must recoup it later).

In practice, not everyone has the fortitude to survive such moments without making rash actions (and if the algorithm allows for the loss of 25% of money on history, then in reality such a situation is very likely), which often lead to even greater losses.

What parameters need to be taken into account

Developers of trading systems need to take into account many different parameters that can affect the final financial viability of a particular strategy.

Transaction costs

Novice traders often pay attention only to the performance of their algorithm directly on the market, but forget to take into account the associated costs, which can offset all the income received. The most obvious costs in this case will be transaction fees charged by the exchange and broker (ITinvest’s fees on some tariffs are approximately the same as those on the exchange).

Slippage and delays

Slippage is the difference in price between the price at which the trading robot intended to carry out the transaction and the one at which it actually took place. It takes time to “deliver” an order to the core of the exchange trading system. In the case of high-speed trading robots (HFT), every millisecond counts, during which the price may change slightly, making the trade less profitable (or unprofitable at all).

Some financial instruments are highly volatile (their prices change frequently), so when working with them it is necessary to make allowances for possible slippage.

Liquidity Impact

When dealing with relatively illiquid instruments, a trader must keep in mind the possible impact that the actions of his trading system will have on the market. If not many people are buying and selling a particular stock, then an order to buy a significant number of those shares can greatly change its price. To avoid such a situation, it is necessary to teach the robot to break transactions into a large number of small orders that cannot greatly affect the market.

Types of trading orders

The performance of a trading strategy is also influenced by what trading orders its developer plans to use to complete transactions. Most often, traders resort to market orders and limit orders.

The market order (“by market”) is executed immediately at the price of a financial instrument (share, futures, option, etc.) formed on the market at the current moment. Accordingly, if it is necessary to make a large transaction, for example, purchasing a large number of shares, the market order will result in to the fact that several transactions will occur at different prices - there may not be the required number of people on the market willing to sell shares at one price, then having bought all their shares, the robot will move on to the next offered price and so on.

Market orders are an aggressive tool - they will always be executed, while the final price of the transaction remains unknown to the trader.

Limit orders allow the robot to determine the worst price at which it makes sense to execute a trade. Such an order may remain unexecuted (if there were no people willing to sell or buy at the specified price on the market) or partially executed (there were not enough people willing), as a result of which it is considered a more passive means of making transactions.

Their advantage, of course, is the fact that the transaction price is predetermined. The list of currently placed orders of the Limit type is called the Order book and is displayed in trading terminals in a separate window.

When testing a strategy, it is important to pay attention to its behavior when using market and limit orders. If the order queue is not modeled correctly, the trading strategy may show worse results when working in real time compared to running on historical data.

Backtesting Tools

There are quite a large number of publicly available systems that can be used to test financial strategies:

• MS Excel- Microsoft Excel, familiar to everyone, can also be used to write mechanical trading systems. Most brokers allow you to connect this tool with their software products (downloading data and generating trading signals using VBA). The disadvantage of such a solution will be the low speed of work, but the advantage is that it is free and quick to implement simple strategies. Alternative - Open Office
• Matlab- a software environment designed to perform complex calculations. There are plugins for use in stock trading. With its help, you can create small scripts that nevertheless describe quite complex strategies. The downside is that the system is paid and not cheap. Alternatives for the Russian market TSlab and StockSharp. Traders also use MetaStock, Wealth-Lab and Omega products to create mechanical trading systems.
• C++/C#- programming languages ​​that are widely used in the financial world. Java and Scala are gradually gaining popularity.
• Built-in trading terminal tools c - some trading terminals have built-in tools for creating trading robots and backtesting strategies. The corresponding plugin can be installed in the SmartX terminal. To write robots, the scripting language TradeScript is used.

Window for back testing of a plugin for creating robots using TradeScript in the SmartX terminal

Conclusion

Backtesting is the most important stage in the development of a trading strategy, without which it is difficult to count on the adequate operation of a trading robot in the “combat” conditions of the real market. It is important to understand that the successful operation of a strategy on historical data does not guarantee equally good results when used in real trading in real time.

In addition to testing on historical data, developers should check the operation of the program in real time - this can be done using special test trading systems provided by exchanges and brokers. With the help of such risk-free systems with virtual money, you can debug the robot’s reaction to changing market conditions - usually data in such cases is provided by exchange platforms (with a delay or “thinned out”).

That's all for today, thank you for your attention. We will be happy to answer questions in the comments.

P.S. If you notice a typo or error, write a personal message and we will quickly correct everything.

3 back-testing

4 back-to-back testing

5 back-to-back testing

a methodology for testing software products that involves applying the same input data to two or more instances of a module or system, comparing the resulting output data (results) and analyzing inconsistencies if any are found. A similar approach is used when testing hardware products.

6 back-to-back testing

7 back-to-back test

8 back-to-back testing

9 back-to-back testing

10 back-to-back testing

11 back-cross testing

12 back-cross testing

Backcross testing, backcross testing

See also in other dictionaries:

back-testing- In general, the process of comparing predictions from a forecasting model to observable data. A model may be run using historical inputs after which the mode s forecast is compared to the actual outcomes observed for the forecasted period. In… …Financial and business terms

VaR back-testing- rizikos vertės nustatymas atliekant grįžtamąjį patikrinimą statusas Aprobuotas sritis kolektyvinio investavimo subjektų veikla apibrėžtis Rizikos vertės metodo tikslumo ir kokybės tikrinimo procedūra, kai lyginamas pagal metod ą gaunamos rizikos… … Lithuanian dictionary (lietuvių žodynas)

Testing cosmetics on animals- is a form of animal testing, intended to ensure the safety and hypoallergenic properties of the products for use by humans. Because of the harm done to the animal subjects, this testing is opposed by animal rights activists and others, and is... ... Wikipedia

Back injury- Back injuries often occur as a result in unfavorable pressures to the spine during day to day activity. Common thoughts on back injuries conclude that these are a result of incorrect lifting methods and posture. Both of these are correct,… … Wikipedia

Back-To-Back-Test- Dynamische Software Testverfahren sind bestimmte Prüfmethoden um beim Softwaretest Fehler in Software aufzudecken. Während bei statischen Verfahren die zu testende Software nicht ausgeführt wird, setzen dynamische Verfahren die Ausführbarkeit der … Deutsch Wikipedia

Back-to-back connection- Telecommunications= In telecommunications, a back to back connection is a directref 1 connection between either:#the output of a transmitting device and the input of an associated receiving device. When used for equipment measurements or testing… … Wikipedia

Animal testing- A white Wistar lab rat Description Around 50–100 million vertebrate animals are used in experiments annually. Subjects Animal testing, scien... Wikipedia

Animal testing on non-human primates- Image taken inside Covance Experiments involving non human primates (NHPs) include toxicity testing for medical and non medical substances; studies of infectious disease, such as HIV and hepatitis; neurological studies; behavior and cognition;… … Wikipedia

Ultrasonic testing- Step 1: The UT probe is placed on the root of the blades to be inspected with the help of a special borescope tool (video probe). Step 2: Instrument settings are input. Step 3: The probe is scanned over the blade root. In this case, an... ... Wikipedia

Flexible Architecture for Simulation and Testing- The FAST Project is a new hybrid hardware prototyping platform enabled by integrating a variety of hardware components on a printed circuit board (PCB) to implement Chip Multiprocessor (CMP) or Multiprocessor (MP) systems. The Flexible... ... Wikipedia

Databases are the inevitable part of such software products as online stores, banking system applications and so on. These can the databases of different types, for example, DB2, SQL, and others

In order to verify the correct work of database, it is performed server side testing which actually is backend testing.

Performing database testing, the specialists do not work with the graphical user interface (GUI), they proceed directly to analyzing the processes of execution of the request with specified parameters and receiving the correspondent answer from the server. With this purpose, JSON and XML can be used.

There are several phases of backend tests:

1. the delivery of SQL server specification;
2. the testing of design specification;
3. the implementation of the tests into the design with the help of SQL code.

Backend testing is the process of validation such database aspects as schemas, tables, indexes and keys, data storage procedure, the process of data duplication, server validation, etc. The tests are launched with the aim of preventing data corruption, deadlock, data loss and others. To check the work of databases, several types of tests are executed.

What are the types of databases testing?

• Non-functional testing
• Functional testing
• Structural testing

The special testing tools will help to simplify the verification procedure of the proper operation of databases. This can be TurboData, Data Factory, AETG, Data Generation and others.

What are the advantages of backend testing?

• Such a software testing ensures the full control over the test coverage.
• Backend checking promotes the detection of a larger number of potential bugs even at the early stages of the development.
• With the help of backend testing, the specialists execute more detailed checking of the system, as the applying of black bow method is not allowed for this testing type.

I will tell you how you can gain practical trading experience; all you need is free time.

In addition to an objective assessment of the results obtained when testing your trading strategy, you will get closer to the process of directly making trading decisions. This will help you a lot Back testing of a trading strategy. By the way, strategy testing is also possible in automatic mode, but I believe that traders who do not have serious trading experience should develop a trading strategy, and testing it would be useful in manual mode.

Let's look at an example of testing a trading strategy in a short time for a specific trading period. To conduct back testing we need:

Technical analysis program

Program for recording transactions and processing statistics

Algorithm of work, strategy, plan for making trading decisions.

Technical analysis program

There are a huge number of technical analysis programs, but I personally am familiar with the MultiCharts program. It is simple and accessible. There are a lot of more advanced programs, such as Omega, TC2000, TradeStation, which can be found on the Internet, but I am used to working with MultiCharts. In addition, you can find it on the Internet for free or download a shareware version, without any time limit for its use.

Download and install the program; we only need MultiCharts directly.

After installing the program for tracking quotes, we need the quotes themselves. They can be downloaded from the Finam broker’s website in the “Export Quotes” section.

Then, download in the format TIME, DATE, HIGH, OPEN, CLOSE, VOL, LOW, quotes of the asset you have chosen. In our example, we require quotes for RTS index futures for the last quarter, on a time period of 5 minutes (see Markets on the Russian Stock Exchange - RTS). It is possible to download quotes for your choice of all available instruments and for all available time periods. The remaining fields remain without any changes. We select the parameters and click on the button called “get file”.

We receive a text file with the quotes you need (Fig. below):

To upload quotes to the MultiCarts program, you need to open the QuoteManeger program, which acts as a source of historical data quotes for our MultiCharts. Then you need to add an exchange, or select it from the existing list. In this case, you should specify the trading parameters. To do this, go to the “Tools” menu, then “Exchanges & ECNs”

Now let's add a contract. The item according to the following scheme Symbol ->AddSymbol ->Manually.

Click OK. Next we edit the name of the tool. I recommend that you write the name of the instrument accepted on the exchange to avoid confusion. We check compliance on all Settings and Session, Stock, PriceScale tabs. If everything is in order, click OK!

A little directly about the process of importing historical data. In the QuoteManager window, right-click our tool and select ImportData -> ASCII

When you launch it, a new DataServerMode window will appear, you need to select Offline. Then add a new chart along this path File -> New -> Chart Window