top of page

COURSE OBJECTIVE


Recently, the number of models used in financial institutions has increased exponentially, particularly in the field of credit risk, such is the case of scoring models in admission, monitoring and recovery, machine learning models, IRB parameters, capital, correlations, stress testing and the recent IFRS 9 parameters, among many others.

​

This proliferation of models has benefits such as automation, efficiency and speed in decision making. However, they also have drawbacks, due to decisions made by the wrong models or used inappropriately.

 

Model risk, in the United States, is defined as the set of possible adverse consequences derived from decisions based on results and incorrect reports of models, or from their inappropriate use. The European regulator defines it as the risk related to the underestimation of own funds, for example, due to the use of the IRB.

 

The objectives of the course are the following:

 

  • Explain the definition and scope of the model risk, the best practices in terms of management, control, governance validation and quantification of the same. Know how COVID-19 impacts credit risk models and the model risk itself.

  • Explain the pioneering model risk directive SR 11-7 in the US, the recent internal model review directive, TRIM, in the European Union, EU, and other important model risk and validation directives such as the estimation directive of PD and LGD and treatment of EBA default exposures.

  • Explain the use of artificial intelligence for model validation.

  • Techniques are shown to achieve the automation of the Construction and Calibration of Models through Artificial Intelligence.

  • Indicate the best practices for validation of credit risk models of financial institutions.

  • Show model risk quantification techniques in credit scoring models, PD, LGD and EAD parameters and regulatory and economic capital.

  • Explore credit scoring validation techniques, and others such as discriminant power, stability tests and backtesting.

  • Offer a very significant number of econometric and machine learning methodologies to develop credit scoring, PD, LGD and EAD models under the IRB and IFRS9 approaches.

  • Explain methodologies to develop models of economic capital and stress testing.

  • Present validation techniques for economic and regulatory capital models.

  • Show a significant number of validation techniques for econometric models and time series used in stress testing.

  • Modeling the stress testing of the PD, LGD and transition matrices of consumer and corporate portfolios.

  • Show innovative stress testing validation techniques.

  • Explain and detect model risk in stress testing.

 

WHO SHOULD ATTEND?

​

This program is aimed at managers, analysts and credit risk consultants. Particularly, to professionals of model risk, model validation and model auditing. For a better understanding of the topics, it is recommended that the participant have knowledge of statistics. The course contains exercises in SAS, R and Excel.

fondo-azul-degradado-lujo-abstracto-azul-oscuro-liso-banner-estudio-vineta-negra.jpg
Time.png

Schedules:

​

​

  • Europe: Mon-Fri, CEST 16-19 h

 

  • America: Mon-Fri, CDT 18-21 h

​

  • Asia: Mon-Fri, IST 18-21 h

 

 

 

 

Price.png

Price: 6 900 €

Early Bird Price: 5.900 €

 

Ending 10 May

Level.png

Level: Advanced

Duration.png

Duration: 30 h

Material.png

Material: 

Presentations PDF

Exercises in R, Python, SAS and Excel

medico-utiliza-tecnologia-medica-robotica-e-innovadora-diagnosticar-examinar-cerebro-pacie

Comprehensive Guide to Model Risk

in Credit Risk Management

fondo-azul-degradado-lujo-abstracto-azul-oscuro-liso-banner-estudio-vineta-negra.jpg

AGENDA

Comprehensive Guide to Model Risk

in Credit Risk Management

​

Anchor 12

INTRODUCTION RISK MODEL

 

Module 1: Risk Management Model and Quantification

​

  • Model Risk Definitions

  • model risk management

  • Model Definition

  • Sources of Model Risk

    • ​Dating

    • Estimate

    • Use

  • Inventory of risk models

  • control methodology

  • Process and technology management

  • governance

  • Model lifecycle management

  • Model risk quantification

  • Quantitative risk management cycle model

  • source identification

  • Model risk mitigation

  • Model documentation

  • Model validation

  • Profile of model risk teams in financial institutions

  • Structure and organization chart

  • Main team activities

  • How to make an inventory of models?

  • COVID-19

    • Impact of COVID-19 on credit risk

    • Impact of COVID-19 on model risk

    • Main flaws in credit risk models

    • Generation of Post-COVID-19 credit risk models

  • Case study 1: European bank model risk

  • Case study 2: model risk in credit risk models

​

MODEL RISK IMPLEMENTATION

 

​Module 2: MODEL RISK PROCESS

​

  • Model validation process

  • Definition of the objective and methodology of the model

  • Review of model memories

  • validation plan

  • Validation conclusions

  • Model risk sub-risks:

    • Model risk in the data

    • Model risk in the methodology

    • Model risk in implementation

    • Risk in model results

  • Model governance

  • Controls at each sub-risk level

  • Analysis and validation of model documentation

  • Purpose of the model

  • Data

  • Design and methodology

  • Specification and estimate

  • Evidence

  • Implementation

  • monitoring

  • operational controls

  • reporting

  • Control panel

  • Solutions and technology necessary for model risk management

  • Case study 3: Model risk management for credit risk models, validation and documentation review

​

EU AND US MODEL RISK REGULATION

 

​Module 3: Model Risk Management Directive SR 11-7 in USA

​

  • Introduction

  • Scope and purpose

  • Model Risk Management

  • Model development, implementation and use

  • Model validation

  • Concept strength assessment and development testing

  • Permanent monitoring, verification of processes and Benchmarking

  • Analysis of the results, including Backtesting

  • Governance, Policies and Controls

  • Politics and procedures

  • Roles and responsibilities

  • Internal audit

  • model inventory

​

Module 4: Guide for the Targeted Review of Internal Models (TRIM) EU

​

  • Scope and objectives of the guide

  • General principles of internal models Roll-out and PPU

  • governance

  • Internal audit

  • internal validation

  • Model Usage

  • Management of model changes

  • Data quality

  • Third parties Participation

  • Credit risk

    • Scope of the credit risk guide

    • Data requirements

  • Probability of default (PD)

    • Structure of PD models

    • main drivers

    • Pool distribution

    • Rating Philosophy

    • Calculation of the default rate

    • Calculation of long-term mean PD

  • Loss Given Default (LGD)

  • Credit conversion factor (CCF)

  • Model-related conservatism margin

  • Review of estimates

​

MODEL RISK

​

Module 5: Validation of Models in practice

​

  • Lessons learned from the financial crisis on validation​

  • Validation Framework

  • Validation Definition

  • Validation principles

  • Roles and responsibilities

  • Scope and frequency

  • Validation Process

  • Internal Governance

  • Validation of IRB Models

  • Qualitative Validation

    • model design

    • data quality

    • Use Test

  • Quantitative Validation

    • backtesting

    • Discriminating Power

    • Stability Tests

  • Technological infrastructure

  • Required documentation

  • Internal validation department and team

  • Audit department and team

  • artificial intelligence

    • Autonomous validation, reconstruction and recalibration

    • Model Validation using Machine Learning

    • Artificial intelligence for model risk

    • Artificial intelligence to validate models​

​​

Module 6: Model Risk Management

​

  • Governance in model risk

    • Role of the board and senior management

    • Role of the risk department

    • Responsible and creators of the model

    • Model risk committees

    • Internal audit

    • Policy definition

  • Organization

    • Three lines of defense

    • Internal and external communication

  • Model Life Cycle Phases

    • ID

    • Model Inventory

    • Classification

    • Levels or "Tiering" according to materiality, sophistication and impact

    • Planning and development

    • internal validation

    • Documentation

    • pattern approval

    • Implementation and use of the model

    • Model monitoring

    • International best practice risk management model

  • Model Risk Mitigation

Data quality test

Validation and audit

Benchmarks

What-if sensitivity tests

Stress testing

backtesting

  • Model risk appetite

    • Model risk appetite statement

    • risk tolerance

  • Qualitative measurement of model risk

    • Creation of the Model Risk Scorecard

    • Definition of scale and ranges

    • International Scorecard Best Practices

  • Case study: Scorecard for model risk

​

MODEL RISK IN CREDIT RATING AND SCORING

 

​Module 7: Model risk in rating and scoring

​​

  • Dimension and use of materiality

  • Classification of scoring models by importance within the financial institution

  • Impact of the model on the entity

  • model dependency

  • Model limitations

  • Model governance

  • Documentation and review

  • Implementation

  • operational controls

  • Decision tree to assess rating and scoring models

  • Casuistry in expert Credit Rating models

  • Casuistry in statistical credit scoring models

  • Model risk in big data

  • Risk model by machine learning and black box

​

Construction of Credit Scoring and model risk analysis

 

Module 8: Advanced data validation

​

  • Data typology

  • transactional data

  • Unstructured data embedded in text documents

  • Social Media Data

  • data sources

  • Data review

  • Target definition

  • Time horizon of the target variable

  • Sampling

    • Random Sampling

    • Stratified Sampling

    • Rebalanced Sampling

  • Exploratory Analysis:

    • histograms

    • Q Q Plot

    • Moment analysis

    • boxplot

  • Treatment of Missing values

    • imputation

    • Delete

    • Keep

  • Advanced Outlier detection and treatment techniques

    • Z-Score

    • Mahalanobis Distance

  • Data Standardization

  • Variable categorization

    • Equal Interval Binning

    • Equal Frequency Binning

    • Chi-Square Test

  • binary coding

  • WOE Coding

    • WOE Definition

    • Univariate Analysis with Target variable

    • Variable Selection

    • Treatment of Continuous Variables

    • Treatment of Categorical Variables

    • Fisher Score

    • gini

    • Information Value

    • Pearson Correlation

    • Cramer Von Misses

    • Optimization of continuous variables

    • Optimization of categorical variables

  • Decision Trees

    • Segmentation

    • Expert Decision

      • Statistics

      • Decision Trees

      • K Means Clustering

      • Finite Mixture Model

      • Univariate Gaussian Mixture

      • Bivariate Gaussian Mixture

  • ​Exercise 1: Exploratory Analysis in R

  • Exercise 2: Detection and treatment of Outliers in R

  • Exercise 3: Missing imputation techniques in R

  • Exercise 4: Stratified and Random Sampling

  • Exercise 5: Analysis of the Weight of Evidence in Excel

  • Exercise 6: Univariate analysis in percentiles in R

  • Exercise 7: Continuous variable optimal univariate analysis in R

  • Exercise 8: KS, Gini and IV validation of each variable in R and Excel

  • Exercise 9: Optimizing categorical variables in R

  • Exercise 10: Univariate Analysis with decision trees in R

  • Exercise 11: Segmentation using K means Clustering in R

​

Module 9: Multivariate models and Machine Learning

 

  • Multivariate Models

    • Logistic regression

    • Cox Regression

    • Model Risk

  • ​Machine Learning

    • decision trees

    • neural networks

    • SVM

    • Ensemble Learning

    • bagging

    • Boosting

    • Random Forest

  • Model Risk in Machine Learning​

    • overfitting

    • Transparency

    • failed sampling

    • important variables

  • Exercise 12: Logistic Regression, stepwise method in R

  • Exercise 14: Piecewise Regression in Excel and R

  • Exercise 15: Decision trees in R

  • Exercise 16: Support Vector Machine in R

  • Exercise 17: Neural Networks in R

  • Exercise 18: Ensemble models in R

  • Exercise 19: Random Forest in R

  • Exercise 20: Bagging in R

  • Exercise 23: Comparison of models of discriminant power between models: Neural Networks, Logistic Regression, Panel Data Logistic Regression and Cox Regression

  • Exercise 24: Model Risk using Confidence Intervals of Logistic Regression Coefficients

​

​Module 10: Model Risk in the Scorecard

​

  • Scoring assignment

  • Scorecard Classification

    • Scorecard WOE

    • Binary Scorecard

    • Continuous Scorecard

  • Scorecard Rescaling

    • Factor and Offset Analysis

    • Scorecard WOE

    • Binary Scorecard

  • Reject Inference Techniques

    • cut-off

    • parceling

    • Fuzzy Augmentation

  • Advanced Cut Point Techniques

    • Cut-off optimization using ROC curves

  • Model risk by cut point decision

  • Model risk due to lack of data

  • Model Risk for not updating or recalibrating

  • Exercise 21: Construction of Scorecard in Excel

  • Exercise 22: Optimum cut-off point estimation in Excel and model risk by cut-off point selection

  • Exercise 23: Confusion matrix to verify Type 1 and Type 2 Error in Excel with and without variables

  • Exercise 24: Model risk in credit scoring due to not recalibrating on time

​

Validation of Credit Scoring models

​

​Module 11: Stability tests

​

  • Model stability index

  • Factor stability index

  • Xi-square test

  • K-S test

  • Exercise 25: Stability tests of models and factors

​

Module 12: Validation of traditional and Machine Learning models

​

  • Out of Sample and Out of time validation

  • Checking p-values in regressions

  • R squared, MSE, MAD

  • Waste diagnosis

  • Goodness of Fit Test

    • deviation

    • Bayesian Information Criterion (BIC)

    • Akaike Information Criterion

  • Multivariate Multicollinearity

  • cross validation

  • Error bootstrapping

  • Binary case confusion matrix

  • Multinomial case confusion matrix

  • Main discriminant power tests:

    • KS, ROC Curve, Gini Index, Cumulative Accuracy Profile, Kullback-Leibler Distance, Pietra Index, 1-Ph, Conditional Entropy, Information Value, Kendall Tau, Brier Score, Divergence

  • confidence intervals

  • Jackknifing with discriminant power test

  • Bootstrapping with discriminant power test

  • Kappa statistic

  • K-Fold Cross Validation

  • Traffic Light Analysis

  • Exercise 26: Cross validation in R

  • Exercise 27: Gini Estimation, Information Value, Brier Score, Lift Curve, CAP, ROC, Divergence in Excel

  • Exercise 28: Bootstrapping of R parameters

  • Exercise 29: Gini/ROC Bootstrapping in R

  • Exercise 30: Kappa estimation

  • Exercise 31: K-Fold Cross Validation in R

  • Exercise 32: Out of time traffic light validation (horizon 6 years) of Logistics and Machine Learning models

​

Automation of the Construction and Calibration of Models with Artificial Intelligence

 

Module 14: Build Automation and Calibration

​

  • What is modeling automation?

  • that is automated

  • Automation of machine learning processes

  • Optimizers and Evaluators

  • Modeling Automation Workflow Components

    • Summary

    • Indicted

    • Feature engineering

    • Model generation

    • Assessment

  • Hyperparameter optimization

  • Reconstruction or recalibration of credit scoring

  • Credit Scoring Modeling

    • Main milestones

    • Evaluation and optimization

    • Possible Issues

  • PD calibration modeling

    • Evaluation and optimization

    • backtesting

    • Discriminating Power

    • Stability Tests

  • Global evaluation of modeling automation

  • Implementation of modeling automation in banking

  • Technological requirements

  • available tools

  • Benefits and possible ROI estimation

  • Main Issues

  • Model Risk

  • Exercise 33: Automation of modeling and optimization and validation of credit scoring hyperparametry

​

Explainable Artificial Intelligence

​

Module 15: Explainable Artificial Intelligence XAI

​

  • Interpretability problem

  • model risk

  • Regulation of the General Data Protection Regulation GDPR

  • EBA discussion paper on machine learning for IRB models

    • 1. The challenge of interpreting the results,

    • 2. The challenge of ensuring that management functions adequately understand the models, and

    • 3. The challenge of justifying the results to supervisors

  • ​Black Box Models vs. Transparent and Interpretable Algorithms

  • interpretability tools

  • Shap, Shapley Additive explanations

    • Global Explanations

    • Dependency Plot

    • Decision Plot

    • Local Explanations Waterfall Plot

  • Lime, agnostic explanations of the local interpretable model

  • Explainer Dashboard

  • Other advanced tools

  • Exercise 34: XAI interpretability of credit scoring

MODEL VALIDATION

​

PD, LGD and EAD validation IRB and IFRS 9

 

Module 16: Directive on the estimation of PD and LGD IRB and defaulted exposures issued by EBA

​

  • European Directive on estimation of PD and LGD, and exposures in default

  • Why is it advisable to consider it in Latin America?

  • Parameter variability reduction

  • Homogenization of the calculation of PD and LGD

  • Implementation dates in European banks

  • Data quality

  • Representativeness of the data for the development of the model and for the calibration of the risk parameters

  • Human judgment for parameter estimation

  • Treatment of deficiencies and margin of conservatism (Moc)

  • PD estimation

  • Model development

  • Data requirement

  • Risk drivers and rating criteria

  • Treatment of external ratings

  • Rating philosophy

  • Pool Treatment

  • PD Calibration

  • Data requirement

  • One Year Default Rate Calculation

  • Calculation and use of the average observed default rate

  • Long-term default rate

  • Calibration of the long-term default rate

  • LGD estimation

  • Methodologies for estimating PD

  • Data requirement

  • Recoveries from collaterals

  • Model development

  • Risk drivers

  • Collateral Eligibility

  • Inclusion of collaterals

  • LGD Calibration

  • Definition of economic loss and realized loss

  • Treatment of commissions, interest and other withdrawals after default

  • Discount rate

  • Direct and indirect costs

  • long-term LGD

  • Long Term LGD Calibration

  • Estimation of risk parameters for exposures in default

  • Estimation and calibration of the Expected Loss Best Estimate

  • Estimation and calibration of LGD in-default

  • Application of risk parameters

  • Review of estimates

  • Accompanying Documents

  • impact assessment

  • Identification of the problem

  • Policy objectives

  • Baseline scenario

  • Options considered

  • Cost-benefit analysis

​

Module 17: PD templates for IRB and IFRS 9 approach

​

  • Introduction to Probability of Default

  • Default definition

  • Default Triggers

  • Effective and robust process to detect default

  • Technical defaults and technical default filters

  • Indispensable data model

  • Single Factor Analysis

  • Multifactor Analysis

  • Model Selection

  • Historical PS

  • Econometric and Machine Learning Models of PD

  • Risk factors that affect default

    • macroeconomic

    • idiosyncratic

  • PD Logistic Regression

  • PD COX Regression

  • PD Log-log Complementary

  • PD Logistic Regression Data Panel

  • Machine Learning to estimate PD

  • ​PD Calibration

  • Introduction to Calibration

  • Anchor Point Estimate

  • Mapping from Score to PD

  • Temporal structure of the PD

    • PS Marginal

    • PS Forward

    • Cumulative PD

  • Techniques for Mapping PD's to temporary structure

  • Vintages or vintages of PD

  • Bayesian PD

  • Expert Judgment

  • Prior and posterior distribution

  • Markov Chain Monte Carlo

  • probit model

  • ​Adjustment to the PD Economic Cycle

  • Introduction of Adjustment to the Economic Cycle

  • Directives on the economic cycle in the PD

  • PD Trough The Cycle (PD TTC) Models

  • Considerations of the Adjustment to the “Scalar Variable” approach cycle

  • PD on Low Default Portfolios

  • PD estimation without correlations

  • PD estimation with correlations

  • LDP Calibration Using CAP Curves

  • Bayesian PD estimation for LDP

  • Default correlation

  • Correlation of defaults and multiperiod

  • Neutral Bayesian and Conservative Bayesian

  • ​Transition and PD Matrices

  • Properties of transition matrices

  • Multi-year transition matrix

    • discrete time

    • continuous time

    • Generating Matrix

    • Exponential of a matrix

  • duration method

  • Cohort method

  • error management

  • IFRS 9 PD Modeling

  • IFRS 9 requirements

  • Probability Weighted

  • Forward Looking

  • Lifetime PD modeling

  • PD Forecasting Modeling

  • PD Point in Time Forecasting

  • Markov chains

  • Exercise 35: Calibration of PD with COX regression in R

  • Exercise 36: PD calibration with log-log complementary in R

  • Exercise 37: PD calibration with logistic model in R

  • Exercise 38: PD calibration logistic Bayesian regression

  • Exercise 39: Calibration of PD regression panel logistic data

  • Exercise 40: Calibration of PD Lasso Regression

  • Exercise 41: Calibration of PD Bayesian Probit regression in R

  • Exercise 42: Transition matrices in Excel and R

  • Exercise 43: Multinomial Regression to estimate PD Lifetime

  • Exercise 44: Multi stage Markov chains in R

​​​

Module 18: Backtesting PD IRB and IFRS9

 

  • Validation of the PD in IRB PIT and TTC

  • Validation of PD Lifetime and PD12m in IFRS 9

  • Backtesting PS

  • PD Calibration Validation

  • Hosmer Lameshow test

  • normal test

  • Binomial Test

  • Spiegelhalter test

  • Redelmeier Test

  • Traffic Light Approach

  • Traffic Light Analysis and PD Dashboard

  • PS Stability Test

  • Forecasting PD vs Real PD in time

  • Validation with Monte Carlo simulation

  • Exercise 45: Backtesting PD IRB and PD IFRS 9

  • Exercise 46: Forecasting Estimated PD and Actual PD in Excel

  • Exercise 47: Validation using Monte Carlo Simulation

 

Module 19: LGD Models for IRB and IFRS 9 Approach

 

  • ​BASEL III and EBA LGD

  • LGD estimation and calibration in practice

  • Default LGD estimation

  • LGD Econometric and Machine Learning Models

  • Advantages and disadvantages of LGD Predictive Models

  • Forward Looking models incorporating Macroeconomic variables

  • Parametric and non-parametric models and transformation regressions

  • Linear regression and Beta transformation

  • Linear Regression and Logit Transformation

  • Linear regression and Box Cox transformation

  • Logistic and Linear Regression

  • Logistic and nonlinear regression

  • Censored Regression

  • Generalized Additive Model

  • neural networks

  • SVM

  • Beta regression

  • Inflated beta regression

  • Fractional Response Regression

  • LGD for IFRS 9

  • Comparison of regulatory LGD vs. IFRS 9

  • LGD adjustments

  • Selection of Interest Rates

  • Allocation of Costs

  • floors

  • Treatment of collateral over time

  • Marginal LGD

  • PIT LGD

  • Loss Lifetime Concept

  • exposure treatment

  • Exercise 48: Logistic and linear regression LGD in R

  • Exercise 49: Neural Networks and SVM LGD

  • Exercise 50: Generalized Additived Model LGD in R

  • Exercise 51: Beta Regression Model LGD in R

  • Exercise 52: Long-term LGD calibration

  • Exercise 53: Inflated Beta Regression in SAS

  • Exercise 54: Comparison of the performance of the models using Calibration and precision tests

​

Module 20: LGD Backtesting

 

  • LGD backtesting in IRB

  • LGD Backtesting in IFRS 9

  • Accuracy ratio

  • absolute accuracy indicator

  • Confidence Intervals

  • transition analysis

  • RR Analysis using Triangles

  • Advanced LGD Backtesting with a vintage approach

  • Backtesting for econometric models

  • ROC, Gini and K-S curve

  • Exercise 55: Comparison of the performance of the models using Calibration and precision tests.

 

Module 21: EAD Models

 

  • Guidelines for estimating CCF

  • Guidelines for Estimating CCF Downturn

  • Temporal horizon

  • Transformations to model the CCF

  • Approaches to Estimating CCF

  • Fixed Horizon approach

  • Cohort Approach

  • Variable focus time horizon

  • CCF Econometric and Machine Learning Models

    • Linear regression

    • Logistic regression

    • Generalized Additive Model

    • neural networks

    • SVM

    • Beta regression

    • Inflated beta regression

    • Fractional Response Regression

  • EAD for IFRS 9

    • Comparison of regulatory EAD vs. IFRS 9

    • Adjustments in the EAD

    • Interest Accrual

    • CCF PIT Estimate

    • Modeling of available lifetime

  • Exercise 56: Estimation and adjustments for EAD IFRS 9 in excel and R

  • Exercise 57: Neural Networks and SVM CCF

  • Exercise 58: Generalized Additived Model CCF in R

  • Exercise 59: Beta Regression Model CCF in R

​

Module 22: EAD Backtesting

​

  • EAD validation

  • CCF validation

  • Backtesting EAD and CCF IRB

  • Backtesting of the EAD and CCF IFRS 9

  • r squared

  • Pearson coefficient

  • Spearman correlation

  • Validation using ROC, K-S and Gini

  • Exercise 60: Comparison of the performance of EAD models

​

IFRS 9 Expected Credit Loss (ECL) Validation

​

​Module 23: ECL Validation

​

  • Initial validation

  • Periodic validation

  • Monitoring

  • Main milestones of qualitative validation

    • Data quality

    • Default Definition

    • Relevance of the qualification process

    • Override Analysis

    • environmental dynamics

    • user test

  • Main milestones of quantitative validation

    • Samples used for validation purposes

    • Discriminating Power

    • population stability

    • Characteristic Stability

    • concentration analysis

    • Staging analysis

    • Parameter Calibration

    • ECL backtesting

  • Principle 5 – Validation of the ECL model in Basel III

    • Governance

    • Model inputs

    • model design

    • ​Model output/performance

  • Validation metrics

    • Bayesian/Akaike/Schwartz/Deviance information criteria

    • Receiver operating characteristic (ROC) curve or AUROC statistic

    • Lorenz curve, Gini coefficient, Kolmogorov-Smirnov test

    • T-tests, F-tests, Wald test, log likelihood test

    • RMSE, MAPE, MAD

    • R-squared, Adjusted R-squared

    • Out-of-sample testing

    • benchmarking

    • Population stability index (PSI)

  • Statistical problems

    • Sampling bias

    • Survivorship bias

    • Disproportionately high model weightsT

    • Autoregressive and lagged terms that do not capture macroeconomic effects

    • spurious correlation

    • Smoothing methods that alter data integrity

    • simple linear models in nonlinear relationships

​

Validation of Regulatory and Economic Capital for Credit Risk

​

Module 24: Economic Capital Models

​

  • Definition and Objective

  • Temporal horizon

  • Default Correlation

  • asset mapping

  • unexpected loss

  • Regulatory capital models

  • ASRF Economic Capital Models

  • Business Models

  • Multifactorial Models

  • Economic Capital for retail using Charge off

  • Dependency modeling using copulas

  • VaR estimation and Expected Shortfall

  • Economic capital management

  • Exercise 61: Default correlation matrix in SAS

  • Exercise 62: Correlation of default: consumer portfolios in R

  • Exercise 63: Correlation of assets with EMV and observable data in R

  • Exercise 64: Creditrisk + in R

  • Exercise 65: One-factor model in Excel and R

  • Exercise 66: Multifactorial Model in Excel

  • Exercise 67: T-student in Excel in Excel

  • Exercise 68: Copulas in R

​

Module 25: Economic capital validation

​

  • Regulatory Capital Models

  • Economic Capital Models

  • Validation of Credit Portfolio Models

    • Model Design

    • Model Output

    • Processes, data and test of use

  • Loss aggregation validation

  • Validation of Basel models

  • Testing distributions using Berkowitz test

  • Credit Loss Distribution

  • Simulation of the critical chi-square value

  • Berkowitz test in subportfolios

  • power assessment

  • Scope and limits of the test

  • Model risk in economic capital due to uncertainty

  • Exercise 69: Implementation of the Berkowitz test in economic and regulatory capital models

  • Exercise 70: Simulation of losses and model risk in regulatory and economic capital

​

Validation of Stress Testing Credit Risk

 

Module 26: Forecasting Models

​

  • Data processing

    • Non-Stationary Series

    • Dickey-Fuller test

    • Cointegration Tests

  • Econometric Models

    • ARIMA models

    • VAR Autoregressive Vector Models

    • ARCH models

    • GARCH models

    • Linear regression

  • ​Machine Learning Models

    • Supported Vector Machine

    • neural network

  • Exercise 71: Nonstationary Series and Cointegration Tests in R and Python

  • Exercise 72: macroeconomic variables with VAR in R

  • Exercise 73: Garch modeling market variables R

  • Exercise 74: Machine Learning SPV and NN modeling in Python

​​

Module 27: Validation of econometric models

​

  • Review of assumptions of econometric models

  • Review of the coefficients and standard errors of the models

  • Model reliability measures

  • Error management

  • not normal

  • heteroscedasticity

  • Outliers

  • autocorrelation

  • multicollinearity

  • Exercise 75: Measuring Logistic Regression Collinearity

​

Module 28: Stress Testing Consumer Credit Risk

​

  • Temporal horizon

  • Multi-period approach

  • Impact on P&L, RWA and Capital

  • Macroeconomic Stress Scenarios in consumption

  • Stress Testing of PD, LGD and EAD

  • Stress Testing of the Transition Matrix

  • Chage Off Stress Testing

  • ​Losses from new impaired assets

  • Losses on old impaired assets

  • ​Exercise 76: Stress Testing PD in Excel and R multifactorial model

  • Exercise 77: Stress Testing PD Multiyear Approach

  • Exercise 78: Stress test of PD and Autoregressive Vectors

  • Exercise 79: Net Charge Off Stress Test

  • Exercise 80: LGD Stress Test

​

Module 29: Stress Testing Credit Risk Portfolios Corporate

​

  • Stress Test Methodology for corporate portfolios

  • Creditmetrics and Transition Matrices

  • Credit Index and PD

  • PD simulation and transition matrices

  • Exercise 81: Corporate portfolio stress test

​​

Module 30: Stress Testing of ECL IFRS 9

 

  • Stress testing IFRS 9 and COVID-19

  • Pandemic scenarios applied to the ECL calculation

  • Stress Testing of IFRS 9 parameters

  • EBA Stress Testing 2021

  • Treatment of the moratorium

  • Possible regulatory scenarios

  • Impact on P&L

  • PIT starting parameters

  • PIT projected parameters

  • Calculation of non-productive assets and impairments

  • Changes in the stock of provisions

  • Changes in the stock of provisions for exposures phase S1

  • Changes in the stock of provisions for exposures phase S2

  • Changes in the stock of provisions of exposures phase S3

  • Sovereign Exposure Impairment Losses

  • Impact on capital

  • Internal Stress Testing Model for ECL IFRS 9

  • Exercise 82: Stress Testing of the ECL using matrices and time series R and Excel

​​

Module 31: Validation of Stress Testing

​

  • Validation of Stress Testing

  • Validation of the Best Case and adverse scenarios

  • Stationarity of variables

  • The signs of economically intuitive coefficients

  • Statistical significance of coefficients

  • confidence levels

  • residual diagnoses

  • Model performance metrics

    • goodness of fit

    • Risk classification

    • cumulative error measures

  • Industry Accepted Thresholds

  • Intuitive sort order

  • Generalized Cross Validation

  • Squared Correlation

  • Root Mean Squared Error

  • Cumulative Percentage Error

  • Akaike Information Criterion

  • Exercise 83: Validation tests of stress testing VAR vs MARS

bottom of page