Distributional Fitting and Quantitative Risk Management
www.OSLRiskManagement.com | www.OSLAnalyticsAcademy.com
www.OSLRiskManagement.com | www.OSLAnalyticsAcademy.com
Distributional Fitting in Quantitative Risk Management
Limit of Liability/Disclaimer of Warranty
While the authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the authors endorse the information or services the organization, website, or product may provide or recommendations it may make. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or been removed between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
© All rights reserved - 2022. To learn how to obtain permission to reuse material, email info@oslriskmanagement.com
The authors have the permission of OSL Risk Management (www.oslriskmanagement.com) and Real Option Valuations (www.realoptionsvaluation.com) to use figures, layouts and results from Risk Simulator, Real Options SLS and BizStats (advanced analytical software for quantitative research) to provide recommendations and support the methodologies described throughout this document.
Version - OSLAA2022UK001
Corresponding Authors
Prof. Dr. Johnathan Mun
PhD, MS, MBA, BS, CQRM, FRM, CFC, MIFC
jcmun@realoptionsvaluation.com
- CEO, Real Options Valuation, Inc.
- Chairman, IIPER International
- Director, OSL Risk Management (UK)
- Full Professor, U.S. Naval Postgraduate School.
- Author | Inventor | Risk Specialist | Researcher
Dr. Elvis Hernandez-Perdomo
PhD(Fin), PhD(EngSc), MIF, MSc, CQRM, AFHEA, FEI
elvis.hernandez@oslriskmanagement.com
- Director and CEO, OSL Risk Management (UK)
- Academic Director, OSL Analytics Academy (UK)
- PhD in Finance | PhD in Engineering Science
- Assistant Professor and Executive Trainer
- Author | Inventor | Risk Specialist | Researcher
"When historical data exists, you risk models can increase accuracy and improve risk forecasts by selecting a statistical distribution that best fits a set of data"
Distribution fitting aims to predict the probability distribution function (Normal, Bernoulli, Poisson, Lognormal, Gamma, Weibull, Erlang, Exponential, Gumbel, and so on) that better represents a set of data points associated with a random variable (discrete or continuous). In quantitative risk management and empirical studies, it is crucial to define a distribution that accurately reflects the underline data, theory, expert information, and so on. If an analyst, engineer, or researcher, selects the wrong distribution, the final stochastic calculations, probabilistic scenarios, and risk forecasts will be wrong.
A quick mistake in quantitative risk management is assuming normality. Life is complicated and non-normal
Life would be great if quantitative risk managers could assume that data inputs in decision models were normally distributed. The normal distribution is the most frequently used distribution in financial risk management. Why? They only need average and standard deviation, and there are symmetric and very-low extreme values. However, life is complicated and non-normal. They cannot just look at the shape of a distribution of data points and assume that a given distribution is a good fit for a data set.
The critical question is, which distribution does an analyst or engineer use for a particular input variable in a model? What are the relevant distributional parameters? If no historical data exist, must the analyst make assumptions about the variables in question? Note that many probability distributions might also closely fit the observed empirical data. For this reason, understanding the phenomenon under study and determining the characteristics and distribution properties of the associated random variable are crucial in quantitative risk management and risk-based decision-making.
In this technical eBook, we will illustrate how distributional fitting works. It means that you will be able to objectively define which distribution fits the data best and provide decision support beyond the technical specifications. Observe that in risk management, selecting a good distribution can lead to reasonable predictions and more accurate results.
In quantitative risk management, stochastic decision-making and uncertainty analysis, it is essential to have a distribution that accurately reflects the data. However, the historical data need to be consistent, cleaned from errors (outliers, duplicates, missing points, and the like), and predictable to determine the best fitting distribution and relevant distributional parameters for the empirical data. As a result, an analyst or engineer can use the obtained distribution for a particular input variable or assumption in a risk-based decision-making model.
“When there is no data, how can we apply distributional fitting?”
If no historical data exist, one approach is using the Delphi method, where a group of experts are tasked with estimating the behaviour of each variable. For instance, a group of mechanical engineers can be tasked with evaluating the extreme possibilities of a spring coil’s diameter through rigorous experimentation or guesstimates. These values can be used as the variable’s input parameters (e.g., uniform distribution with extreme values between 0.5 and 1.2). When testing is not possible (e.g., market share and revenue growth rate), management can still make estimates of potential outcomes and provide the best-case, most-likely case, and worst-case scenarios, whereupon a triangular or custom distribution can be created.
Conversely, distributional fitting can be accomplished when reliable historical data are available. Assuming that historical patterns hold and that history tends to repeat itself, then historical data can be used to find the best-fitting distribution with their relevant parameters and, consequently, to better define the variables to be simulated.
“How do you know which distribution to use for a particular input variable in a model?”
Quantitative risk management professionals need to use statistical ranking methods (goodness of fit techniques) to estimate the parameters of the various distributions by comparing the empirical distribution with the theoretical distributions.
Once the estimation is complete, the goodness of fit helps determine which distribution fits the data best. There are also visual techniques (i.e., P-P plots or charts to compare the theoretical and empirical cumulative density functions) that can help with the selection process; they examine a histogram with the distribution overlaid and compare the empirical with the theoretical model. However, visual techniques do not guarantee to find the most suitable distribution nor provide quantitative evidence. In other words, a visual inspection for assessing normality does not guarantee that the distribution is normal.
To avoid making mistakes, risk modellers and data analysts rely on statistical methods to determine the distribution's parameters to identify the best distribution. Mainly, four parameters are used in distribution fitting, location, scale, shape, and threshold. However, it is essential to underline that not all parameters exist for each distribution. For example, a normal distribution has only two parameters, location (average) and scale (standard deviation).
"Given that you have a set of observations, determining the right probability distribution function reduces additional uncertainty on your risk models because of selecting a wrong input assumptions."
Traditionally, the statistical ranking methods used in the distributional fitting routines are the Chi-Square and Kolmogorov-Smirnov tests. The first is used to test discrete distributions and the latter, continuous distributions. Briefly, a hypothesis test coupled with the maximum likelihood procedure and internal optimisation routines are used to find the best-fitting parameters on each distribution tested. Finally, the results are ranked from the best fit to the worst fit.
Note that the null hypothesis (Ho) being tested is such that the fitted distribution is the same as the population from which the sample data to be fitted comes. Thus, when a distributional fitting method reports a p-value lower than a critical alpha level (typically 0.10 or 0.05), the distribution is wrong. Conversely, the higher the p-value, the better the distribution fits the data.
There are other distributional fitting tests, reporting error measures (MAPE% - Mean absolute percentage error) between the empirical and the theoretical distribution, such as the Anderson-Darling, Bayes-Schwarz, etc. However, these are very sensitive parametric tests and are highly inappropriate in Monte Carlo simulation distribution-fitting routines.
Therefore, due to their parametric requirements, these tests are most suited for testing normal distributions and distributions with normal-like behaviours (e.g., binomial distribution with a high number of trials and symmetrical probabilities) and less accurate results when performed on non-normal distributions.
"Risk Simulator not only provides a large number of distribution to model uncertainty, but also contain multiple statistical ranking methods for distributional fitting, everything in one solution"
To select the best fitting distribution, a maximum likelihood estimation method is used to determine the distribution's parameters from a set of data. This process must be run with great care and implemented across many distribution functions (i.e., normal, non-normal, discrete, and continuous).
Risk Simulator®, R, Python, and MATLAB® are the most advanced distribution fitting tools not only because of their speed but also because of their accuracy in simultaneously assessing many distribution functions and fitting tests. Nevertheless, Risk Simulator does not require programming skills and can be implemented using your existing excel models on single and multiple variables using various distributional-fitting tests. For example, Figure 1 shows a Risk Simulator screenshot with distributional-fitting tests and options for continuous and discrete variables.
Figure 1. Distributional Fitting Options in Risk Simulator
For further information, click on Risk Simulator | Analytical Tools | 11 Distributional Fitting (Single Variable) or 12 Distributional Fitting (Multiple Variable). Although Risk Simulator allows multiple distribution-fitting methods, the Kolmogorov-Smirnov and Chi-Square tests are better suited for normal and non-normal distribution. They are highly recommended in Monte Carlo simulation distribution-fitting routines, and risk quantification approaches.
Risk Simulator is one of the most powerful Excel Add-in Software and Quantitative Research tool, and now embedding BizStats with AI/Machine Learning Analytics, allows performing more than 300 quantitative methods and analytical tools for quantitative risk management, including Statistical Analysis, Regression Analysis, Data Analytics, Time Series, and Multivariate Analysis.
In this analytical approach, Risk Simulator allows determining the best fitting distribution function associated with a single variable (continuous or discrete) by using and comparing different statistical methods and ranking multiple distribution functions according to the computed statistic tests and p-values, or error measurement.
Risk Simulator provides a general example and data set (Risk Simulator | Example Models | 03 Data Fitting) to perform a distributional fitting on a single variable. However, do not hesitate to use different data sets and follow these steps:
Figure 2. Single-Variable Distributional Fitting
The intermediate report (Figure 3) and the final report (Figure 4) show the test statistics, p-values, theoretical statistics (based on the selected distribution), empirical statistics (based on the raw data), the original data (to maintain a record of the data used), and the assumption complete with the relevant distributional parameters (i.e., if you selected the option to generate assumption automatically and if a simulation profile already exists). The results also rank all the selected distributions and how well they fit the data.
In terms of result interpretation (Figure 4), for example, if the computed p-value, using the Kolmogorov-Smirnov test, is higher than a critical alpha level (typically 0.10 or 0.05), then the distribution is the suitable distribution. Roughly, the p-value can be seen as a percentage explained; that is, if the p-value is 0.9996 (Figure 4), then setting a normal distribution with a mean of 100.67 and a standard deviation of 10.4 explains about 99.96% of the variation in the data, indicating an excellent fit.
Figure 3. Distributional Fitting Result
Figure 4. Distributional Fitting Report – Single Variable
For fitting multiple variables, the process is relatively similar to fitting individual variables. The data and variables’ headings should be arranged in columns (i.e., each variable is organised by columns, see Figure 5). The same analysis and result interpretation are performed when fitting multiple variables as when single variables are fitted. The difference can be observed in the final report. Although it is impractical to produce each variable’s distributional rankings across all the distribution fitting methods, Risk Simulator shows the best distribution and the correlation matrix for the empirical data based on the selected methods. However, if the rankings are important, run the single-variable fitting procedure instead, on one variable at a time.
Therefore, Risk Simulator provides a general example and data set (Risk Simulator | Example Models | 03 Data Fitting) to illustrate a multiple-variable distributional fitting. However, do not hesitate to use different data sets and follow these steps:
Figure 5. Multiple-Variable Distributional Fitting
Based on the chosen distributional fitting methods, the final report (Figure 6) shows the selected distributions, including test statistics, p-values, theoretical and empirical statistics, correlation matrix (based on the raw data), and the distribution assumptions with the relevant parameters. For instance, in Figure 6, observe the computed p-values using the Kolmogorov-Smirnov and Chi-Square tests depending on the type of random variable (continuous or discrete). The higher the p-value (typically greater than 0.10 or 0.05), the more suitable the distribution is. In addition, the pair-wised correlation matrix indicates the linear associations among the variables used during the fitting distribution process. If some are significant, you must consider them in the Monte Carlo simulations and quantitative risk models.
Figure 6. Distributional Fitting Report – Multiple Variables
This eBook is written based on the latest publication of Johnathan Mun in Modeling Risk and Applied Analytics.
Mun, J. (2016) Modeling Risk | Applying Monte Carlo Risk Simulation, Strategic Real Options, Stochastic Forecasting, Portfolio Optimization, Data Analytics, Business Intelligence, and Decision Modeling.. IIPER Press.
Amazon Site - Get Your Book Now
OSL Analytics Academy in collaboration with International Institute of Professional Education and Research (IIPER®) offer the most complete Quantitative Research Methods (QRM) and Quantitative Risk Management (QRMBS) Training to learn how to perform Distribution Fitting, Monte Carlo Risk Simulation, Data Analytics, Decision Modelling, among other topics.
QRM Program | Wide range of statistical techniques available to analyse quantitative data and enhance your research skills. By the end if this course, you will have:
Know More About the QRMBS Here
QRMBS Executive Program | Enhance your risk modelling skills focused on Monte Carlo Risk Simulations and analytical tools applied to Business Models, Strategies, and Case Studies.
Prof. Dr. Johnathan Mun
PhD, MS, MBA, BS, CQRM, FRM, CFC, MIFC
jcmun@realoptionsvaluation.com
Dr. Johnathan C. Mun is the founder, chairman, and CEO of Real Options Valuation, Inc. (ROV), a consulting, training, and software development firm specializing in strategic real options, financial valuation, Monte Carlo risk simulation, stochastic forecasting, optimization, decision analytics, business intelligence, healthcare analytics, enterprise risk management, project risk management, quantitative research methods, and risk analysis located in northern Silicon Valley, California. ROV has partners around the world including Argentina, Beijing, Chicago, China, Colombia, Ghana, Hong Kong, India, Italy, Japan, Malaysia, Mexico City, New York, Nigeria, Peru, Puerto Rico, Russia, Saudi Arabia, Shanghai, Singapore, Slovenia, South Africa, South Korea, Spain, United Kingdom, Venezuela, Zurich, and others.
Dr. Mun is also the chairman of the International Institute of Professional Education and Research (IIPER), an accredited global organization staffed by professors from named universities from around the world that provides the Certified in Quantitative Risk Management (CQRM) and Certified in Risk Management (CRM) designations, among others. He is the creator of many powerful tools including Risk Simulator, Real Options SLS Super Lattice Solver, Modeling Toolkit, Project Economics Analysis Tool (PEAT), Credit Market Operational Liquidity Risk (CMOL), Employee Stock Options Valuation, ROV BizStats, ROV Modeler Suite (Basel Credit Modeler, Risk Modeler, Optimizer, and Valuator), ROV Compiler, ROV Extractor and Evaluator, ROV Dashboard, ROV Quantitative Data Miner, and other software applications, as well as the risk analysis training DVD.
He has over 21 registered patents and patents pending globally. He has authored over 26 books published by John Wiley & Sons, Elsevier Science, IIPER Press, and ROV Press, including multiple volumes of the Applied CQRM Series (IIPER Press, 20192020); Modeling Risk: Applying Monte Carlo Simulation, Strategic Real Options, Stochastic Forecasting, Portfolio Optimization, Data Analytics, Business Intelligence, and Decision Modeling, First Edition (Wiley, 2006), Second Edition (Wiley, 2010), and Third Edition (ROV Press, 2015); The Banker’s Handbook on Credit Risk (2008); Advanced Analytical Models: 250 Applications from Basel II Accord to Wall Street and Beyond (2008); Real Options Analysis: Tools and Techniques (2003, 2005); Real Options Analysis Course: Business Cases (2003); Applied Risk Analysis: Moving Beyond Uncertainty (2003); and Valuing Employee Stock Options (2004). His books and software are being used at over 350 top universities around the world.
Currently a risk, finance, and economics professor, Dr. Mun has taught courses in financial management, investments, real options, economics, and statistics at the undergraduate and the graduate MS, MBA, and PhD levels. He teaches and has taught at universities all over the world, and has chaired many graduate research MBA thesis and PhD dissertation committees. He also teaches weeklong Risk Analysis, Real Options Analysis, and Risk Analysis for Managers public courses where participants can obtain the CQRM designations on completion. He is a senior fellow at the Magellan Center and sits on the board of standards at the American Academy of Financial Management. He was formerly the Vice President of Analytics at Decisioneering, Inc., where he headed the development of options and financial analytics software products, analytical consulting, training, and technical support, and where he was the creator of the Real Options Super Lattice software.
He was a Consulting Manager and Financial Economist in the Valuation Services and Global Financial Services practice of KPMG Consulting and a Manager with the Economic Consulting Services practice at KPMG LLP. During his tenure at Real Options Valuation, Inc., Decisioneering, and KPMG Consulting, he taught and consulted on a variety of real options, risk analysis, financial forecasting, project management, and financial valuation issues for more than 100 multinational firms (current and former clients include 3M, Airbus, Boeing, BP, Chevron Texaco, Financial Accounting Standards Board, Fujitsu, GE, Goodyear, Microsoft, Motorola, Northrop Grumman, Pfizer, Timken, U.S. Department of Defense, U.S. Navy, Veritas, and many others). His experience prior to joining KPMG included being department head of financial planning and analysis at Viking Inc. of FedEx, performing financial forecasting, economic analysis, and market research. Prior to that, he did financial planning and freelance financial consulting work. Dr. Mun received a PhD in finance and economics from Lehigh University, where his research and academic interests were in the areas of investment finance, econometric modeling, financial options, corporate finance, and microeconomic theory. He also has an MBA in business administration, an MS in management science, and a BS in biology and physics.
He is Certified in Financial Risk Management, Certified in Financial Consulting, and Certified in Quantitative Risk Management. He is a member of the American Mensa, Phi Beta Kappa Honor Society, and Golden Key Honor Society as well as several other professional organizations, including the Eastern and Southern Finance Associations, American Economic Association, and Global Association of Risk Professionals. In addition, he has written many academic articles published in the Journal of Expert Systems with Applications; Defense Acquisition Research Journal; American Institute of Physics Proceedings; Acquisitions Research (U.S. Department of Defense); Journal of the Advances in Quantitative Accounting and Finance; Global Finance Journal; International Financial Review; Journal of Financial Analysis; Journal of Applied Financial Economics; Journal of International Financial Markets, Institutions and Money; Financial Engineering News; and Journal of the Society of Petroleum Engineers. Finally, he has contributed chapters in dozens of books and written over a hundred technical whitepapers, newsletters, case studies, and research papers for Real Options Valuation, Inc.
Dr. Elvis Hernandez-Perdomo
PhD(Fin), PhD(EngSc), MIF, MSc, CQRM, AFHEA, FEI
elvis.hernandez@oslriskmanagement.com
Dr. Elvis Hernandez-Perdomo is the cofounder, and Executive Director of OSL Risk Management Ltd., an IT-based consulting, executive training, and software customization firm specializing in strategic real options, financial and project valuation, Monte Carlo risk simulation, stochastic forecasting, optimization, decision making analytics, business intelligence, engineering risk services, enterprise risk management (ERM), project risk management, corporate governance, and risk analysis located in United Kingdom. OSL Risk Management has partners in Spain, Italy, Portugal, Germany, Ghana, Nigeria, South Africa, Saudi Arabia, and others. Dr. Hernandez is an associate director of OSL Consulting Engineering, a firm specializing in engineering services for oil and gas, petrochemicals, renewables, and energy companies.
He is also a Board Member of multiples risk management organizations like the International Institute of Professional Education and Research (IIPER), AFRisk Convention, and EURisk Convention. He is also a senior executive trainer for the Certified in Quantitative Risk Management (CQRM Accreditation), and a program director for multiples executive trainings in Project Risk Management, Multicriteria Decision-Making, Enterprise Risk Management, Risk Quantification and Real Options Analysis, Reliability Engineering and Operational Risk, Decommissioning Risk, Company and Project Valuations, Portfolio and Asset Management, Financial Risk Management, and others. Dr. Elvis is a Member of the Institute of Directors (IoD, UK) and of the Portuguese Economic Association, and has been Senior Consultant in several international engineering and energy projects in multiple countries across America, Europe, and Africa, and numerous industries (oil and gas, energy, logistics, telecommunications, financial and insurance, and SMEs).
Therefore, he holds public seminars on risk analysis and CQRM programs. He was a former Central Banker's Economist, Founder and Executive Director of a Latin-America IT-based consulting firm, and Senior Executive Consultant at Real Options Valuations, Inc. (ROV) working on real options, risk management, Monte Carlo Risk Simulation, optimization, and business intelligence analytics. Currently Dr. Hernandez-Perdomo is a risk and engineering lecturer and has taught courses in financial management, investments, research methods, real options, project management and statistics at the undergraduate and the graduate MS and PhD levels. Therefore, he is a Senior Executive Trainer in quantitative risk management at the Energy Institute and Institute of Risk Management in the United Kingdom.
During his tenure OSL Risk Management, Real Options Valuations, OSL Consulting Engineering, ONUVA Technologies, and Central Bank of Venezuela, he taught and consulted on a variety of real options, risk analysis, decision making, data analytics, project management, and financial valuation aspects a large number of international clients. For example, E.ON, Petrobras, Centrica, PDVSA, ECOPETROL, Saudi Telecom Company, Wood, INEOS, CRODA, Orsted, ADNOC, AECOM, ERYC, GasMeth, Huawei, AXA, KIER Group, KPMG, Deloitte, Ministry of Defence, MTN Ghana, Vanguard, Vodafone, Department of International Trade (DIT UK), Telefonica, Mitsubishi Chemicals, Saudi Aramco, and many others.
Dr. Elvis holds a PhD in Finance from the University of Hull (UK), and a PhD in Engineering Science, where his research and academic interests are Corporate Governance, Multicriteria Decision Analysis, Risk Management, Project Valuations and Real Options in Energy Projects. He also has an MS in Statistics and Operations Research (Distinction), an MS/MIF in Finance (Top Five), a BS in Economics (Honours - Magna-Cum-Laude), a Certified in Quantitative Risk (CQRM) by the IIPER (member of the prestigious AACSB), and an Associate in Business ERP by SAP Corporation. Dr Hernandez belongs to the Academy of Higher Education in the UK as an Associate Fellow. Furthermore, he has written many academic articles published in the Journal of the Operational Research Society, Journal of Reliability Engineering & System Safety, Applied Energy, Journal of Central Bank, Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering, Journal of Engineering, Partial Order Concepts in Applied Sciences (Springer International Publishing), and Journal of the International Council on Systems Engineering.
In quantitative risk management, determining a probability distribution function that reflects a data set requires theory, expert information, or the distributional fitting method. Selecting a wrong distribution can lead to flawed stochastic calculations, probabilistic scenarios, and risk forecasts. Thus, distributional fitting methods help predict the probability distribution function (normal, Bernoulli, Poisson, lognormal, gamma, Weibull, etc.) to represent better a random variable’s data set (discrete or continuous).
Some analysts and engineers can rely on visualisation tools to assess how “good” a distribution matches the data. However, many probability distributions might closely fit the empirical data, and it is essential to determine the underline characteristics of the random variable’s data. These characteristics are associated with the distribution properties and parameters (location, scale, shape and threshold) and a general understanding of the related phenomenon under study. So, statistical methods (i.e., Kolmogorov-Smirnov test, Chi-Square test, and the like) provide objective measures (p-values, statistics test, or error measures) to predict the probability distribution function, minimising the error between an empirical and theoretical distribution.
Finally, remember that when trying to find a distribution that best fits the data, note that predictions of occurrence based on fitted probability distributions are subject to uncertainty. It occurs not only because the observed data series may not be entirely representative due to random errors but also because changes in external conditions (crisis, disruptions, structural breaks, and so on) may cause changes in the probability of occurrence of the phenomenon.
www.OSLRiskManagement.com | www.OSLAnalyticsAcademy.com