imported>Jarble: /* Financial analysis */ paraphrasing

2024-08-25T01:42:12Z

Financial analysis: paraphrasing

New page

{{short description|Testing a predictive model on historical data}}
'''Backtesting''' is a term used in modeling to refer to testing a [[predictive modelling|predictive model]] on historical data. Backtesting is a type of [[retrodiction]], and a special type of [[Cross-validation (statistics)|cross-validation]] applied to previous time period(s).

== Financial analysis==
In the economic and financial field, backtesting seeks to estimate the performance of a strategy or model if it had been employed during a past period. This requires simulating past conditions with sufficient detail, making one limitation of backtesting the need for detailed historical data. A second limitation is the inability to model strategies that would affect historic prices. Finally, backtesting, like other modeling, is limited by potential [[overfitting]]. That is, it is often possible to find a strategy that would have worked well in the past, but will not work well in the future.<ref>{{cite web|title=Pseudo-mathematics and financial charlatanism. Notices of the American Mathematical Society, Volume 61, Number 5, pp. 458-471|url=https://www.ams.org/notices/201405/rnoti-p458.pdf|author=Bailey, Borwein, Lopez de Prado, Zhu (2014)}}</ref> Despite these limitations, backtesting provides information not available when models and strategies are tested on synthetic data.

Historically, backtesting was only performed by large institutions and professional money managers due to the expense of obtaining and using detailed datasets. However, backtesting is increasingly used on a wider basis, and independent web-based backtesting platforms have emerged. Although the technique is widely used, it is prone to weaknesses.<ref>{{cite web|title=Issues related to back testing|url=http://www.financialtrading.com/issues-related-to-back-testing/|author=FinancialTrading|date=2013-04-27}}</ref> [[Basel Accords|Basel financial regulations]] require large financial institutions to backtest certain risk models.

For a [[Value at risk|Value at Risk]] 1-day at 99% backtested 250 days in a row, the test is considered green (0-95%), orange (95-99.99%) or red (99.99-100%) depending on the following table:<ref>{{cite web| title=Supervisory framework for the use of "backtesting" in conjunction with the internal models approach to market risk capital requirements|url=https://www.bis.org/publ/bcbs22.pdf | publisher=Basle Committee on Banking Supervision |date=January 1996 | page=14}}</ref>
[[Image:Backtesting exceptions 1Dx250.png|backtesting exceptions 1Dx250|thumb|500x500px]]
{| class="wikitable alternance"
|+ 1-day VaR at 99% backtested 250 days
|-
! scope="col" | Zone
! scope="col" | Number exceptions
! scope="col" | Probability
! scope="col" | Cumul
|-
| rowspan="5" | Green
! scope="row" | 0
| 8.11%
| 8.11%
|-
! scope="row" | 1
| 20.47%
| 28.58%
|-
! scope="row" | 2
| 25.74%
| 54.32%
|-
! scope="row" | 3
| 21.49%
| 75.81%
|-
! scope="row" | 4
| 13.41%
| 89.22%
|-
| rowspan="5" | Orange
! scope="row" | 5
| 6.66%
| 95.88%
|-
! scope="row" | 6
| 2.75%
| 98.63%
|-
! scope="row" | 7
| 0.97%
| 99.60%
|-
! scope="row" | 8
| 0.30%
| 99.89%
|-
! scope="row" | 9
| 0.08%
| 99.97%
|-
| rowspan="3" | Red
! scope="row" | 10
| 0.02%
| 99.99%
|-
! scope="row" | 11
| 0.00%
| 100.00%
|-
! scope="row" | ...
| ...
| ...
|}
For a [[Value at risk|Value at Risk]] 10-day at 99% backtested 250 days in a row, the test is considered green (0-95%), orange (95-99.99%) or red (99.99-100%) depending on the following table:
[[Image:Backtesting exceptions 10Dx250.png|backtesting exceptions 10Dx250|thumb|500x500px]]
{| class="wikitable alternance"
|+ 10-day VaR at 99% backtested 250 days
|-
! scope="col" | Zone
! scope="col" | Number exceptions
! scope="col" | Probability
! scope="col" | Cumul
|-
| rowspan="9" | Green
! scope="row" | 0
| 36.02%
| 36.02%
|-
! scope="row" | 1
| 15.99%
| 52.01%
|-
! scope="row" | 2
| 11.58%
| 63.59%
|-
! scope="row" | 3
| 8.90%
| 72.49%
|-
! scope="row" | 4
| 6.96%
| 79.44%
|-
! scope="row" | 5
| 5.33%
| 84.78%
|-
! scope="row" | 6
| 4.07%
| 88.85%
|-
! scope="row" | 7
| 3.05%
| 79.44%
|-
! scope="row" | 8
| 2.28%
| 94.17%
|-
| rowspan="3" | Orange
! scope="row" | 9
| 1.74%
| 95.91%
|-
! scope="row" | ...
| ...
| ...
|-
! scope="row" | 24
| 0.01%
| 99.99%
|-
| rowspan="2" | Red
! scope="row" | 25
| 0.00%
| 99.99%
|-
! scope="row" | ...
| ...
| ...
|}

== Hindcast  ==
[[File:Hindcasting.jpeg|200px|thumb|right|Temporal representation of hindcasting.<ref>Taken from p.145 of [https://archive.org/details/TECA2004 Yeates, L.B., ''Thought Experimentation: A Cognitive Approach'', Graduate Diploma in Arts (By Research) dissertation, University of New South Wales, 2004.]</ref>]]

In [[oceanography]]<ref>{{cite web|title= Hindcast approach|url= http://www.oceanweather.com/research/HindcastApproach.html|publisher= OceanWeather Inc.|accessdate= 22 January 2013}}</ref> and [[meteorology]],<ref>{{cite journal|last= Huijnen|first= V. |author2= J. Flemming |author3= J. W. Kaiser |author4= A. Inness |author5= J. Leitão |author6= A. Heil |author7= H. J. Eskes |author8= M. G. Schultz |author9= A. Benedetti |author10= J. Hadji-Lazaro |author11= G. Dufour |author12= M. Eremenko|title= Hindcast experiments of tropospheric composition during the summer 2010 fires over western Russia|journal= Atmos. Chem. Phys.|year= 2012|volume= 12|issue= 9 |pages= 4341–4364|url= http://www.atmos-chem-phys.net/12/4341/2012/acp-12-4341-2012.html|accessdate= 22 January 2013|doi= 10.5194/acp-12-4341-2012|bibcode= 2012ACP....12.4341H |doi-access= free }}</ref> ''backtesting'' is also known as ''hindcasting'': a '''hindcast''' is a way of testing a [[mathematical model]]; researchers enter known or closely estimated inputs for past events into the model to see how well the output matches the known results.

Hindcasting usually refers to a [[computer simulation|numerical-model]] integration of a historical period where no observations have been [[data assimilation|assimilated]]. This distinguishes a hindcast run from a [[meteorological reanalysis|reanalysis]]. Oceanographic observations of [[salinity]] and [[temperature]] as well as observations of [[ocean surface wave|surface-wave parameters]] such as the [[significant wave height]] are much scarcer than meteorological observations, making hindcasting more common in oceanography than in meteorology. Also, since surface waves represent a forced system where the wind is the only generating force, [[wind wave model|wave hindcasting]] is often considered adequate for generating a reasonable representation of the wave [[climate]] with little need for a full reanalysis. Hydrologists use hindcasting for model stream flows.<ref>{{cite web|title= Guidance on Conducting Streamflow Hindcasting in CHPS|url= http://www.nws.noaa.gov/om/water/RFC_support/HEFS_doc/Streamflow_Hindcasting_Guidance_V3.pdf|publisher= NOAA|accessdate= 22 January 2013}}</ref>

An example of hindcasting would be entering [[climate]] forcings (events that force change) into a [[climate model]]. If the hindcast showed reasonably-accurate climate response, the model would be considered successful.

The [[ECMWF re-analysis]] is an example of a combined atmospheric reanalysis coupled with a wave-model integration where no wave parameters were assimilated, making the wave part a hindcast run.

==See also==
{{Div col}}
* [[customer foresight|Applied research (customer foresight)]]
* [[Black box|Black box model]]
* [[Climate]]
* [[ECMWF re-analysis]]
* [[Forecasting]]
* [[NCEP/NCAR reanalysis|NCEP re-analysis]]
* [[Economic forecast]]
* [[Retrodiction]]
* [[Statistical arbitrage]]
* [[Thought Experiment]]
* [[Value at risk]]
{{Div col end}}

==References==
{{reflist}}

[[Category:Tests]]
[[Category:Technical analysis]]
[[Category:Mathematical modeling]]
[[Category:Numerical climate and weather models]]
[[Category:Statistical forecasting]]

Backtesting - Revision history

imported>Jarble: /* Financial analysis */ paraphrasing