CREDIT RISK INTERNAL MODELS

 

Module 1: Credit Risk Internal Models &

Model Risk in Machine Learning

​

• ECB guide to internal models July 2025

• Introduction and Supervisory Foundations

• Foreword & purpose of the ECB Guide.

• Legal framework: CRR, CRR2/CRR3, CRD, RTS/ITS.

• TRIM legacy and harmonisation of practices.

• Role of JSTs and supervisory investigations.
  Exercise: Map EU regulations to internal model approval steps.

•  Overarching Principles of Internal Models

• Group-wide implementation & consistency.

• Documentation requirements & model register.

• Data governance expectations (DORA, BCBS 239).

• Model risk management framework (definition, policies, lifecycle).

• Role of management body & senior management.
  Exercise: Build a model risk framework checklist.

• Internal Validation Principles

• Independence & proportionality of validation.

• Frequency of initial and annual validations.

• Separation of duties between development and validation.

• Resource & skill requirements.
  Case Study: Validation independence in a medium-sized bank.

•  Internal Audit and Oversight

• Scope of internal audit in internal models.

• Independence, resources, and reporting lines.

• Interaction with validation.

• Audit documentation and follow-up.

• Climate-Related and Environmental Risks

• ECB guidance on integrating climate/ESG in internal models.

• Materiality assessments.

• Examples of climate risk transmission channels.
  Exercise: Extend an LGD model with climate-related variables.

•  Machine Learning in Internal Models

• Supervisory stance on ML models (complexity, opacity, drift).

• Classification as “highly complex” or “dynamic” models.

• Governance and skill requirements across lines of defence.

• Data quality, bias, explainability, and ethics.
  Lab: Train an ML PD model (Python) & apply SHAP for explainability.

• Credit Risk Models – Governance and Validation

• Scope & roll-out of IRB models.

• Internal governance structures.

• Model use tests and limitations.

• Change management for IRB models.

• Data maintenance and documentation.
  Exercise: Create an IRB model governance flowchart.

•  Definition of Default

• Consistency in default definition.

• Days past due criterion.

• Unlikeliness to pay indicators.

• Return to non-defaulted status.

• Adjustments after definition changes.
  Case Study: Recalibrating PDs after default definition revision.

•  Estimation of Credit Risk Parameters

• Use of data for estimation.

• Probability of Default (PD): calibration, discriminatory power.

• Loss Given Default (LGD): downturn, discounting, recovery data.

• EAD/Conversion Factors: estimation, credit lines, derivatives.

• Margin of Conservatism (MoC): principles and application.

• Review and recalibration of estimates.

• Maturity calculation for non-retail exposures.
  Lab: Calibrate PD and LGD using historical data in Python/R.

• Model Risk Governance & ICAAP Integration

• Classification of model complexity/materiality.

• MoC and its impact on capital.

• Link between validation, internal audit, and ICAAP/ILAAP.

• Supervisory review process (SREP).

• Third-party involvement and outsourcing risks.
   

​

MODEL RISK 

 

​Module 2: Model Risk Process

​

• Model validation process

• Definition of the objective and methodology of the model

• Review of model memories

• validation plan

• Validation conclusions

• Model risk sub-risks:

  • Model risk in the data

  • Model risk in the methodology

  • Model risk in implementation

  • Risk in model results

• Model governance

• Controls at each sub-risk level

• Analysis and validation of model documentation

• Purpose of the model

• Data

• Design and methodology

• Specification and estimate

• Evidence

• Implementation

• monitoring

• operational controls

• reporting

• Control panel

• Solutions and technology necessary for model risk management

• Case study 3: Model risk management for credit risk models, validation and documentation review

​

EU and US Model Risk Regulation

 

​Module 3: Model Risk Management Directive SR 11-7 in USA

​

• Introduction

• Scope and purpose

• Model Risk Management

• Model development, implementation and use

• Model validation

• Concept strength assessment and development testing

• Permanent monitoring, verification of processes and Benchmarking

• Analysis of the results, including Backtesting

• Governance, Policies and Controls

• Politics and procedures

• Roles and responsibilities

• Internal audit

• model inventory

​

Module 4: Guide for the Targeted Review of Internal Models (TRIM) EU

​

• Scope and objectives of the guide

• General principles of internal models Roll-out and PPU

• governance

• Internal audit

• internal validation

• Model Usage

• Management of model changes

• Data quality

• Third parties Participation

• Credit risk

  • Scope of the credit risk guide

  • Data requirements

• Probability of default (PD)

  • Structure of PD models

  • main drivers

  • Pool distribution

  • Rating Philosophy

  • Calculation of the default rate

  • Calculation of long-term mean PD

• Loss Given Default (LGD)

• Credit conversion factor (CCF)

• Model-related conservatism margin

• Review of estimates​

​

Module 5: Validation of Models in practice

​

• Lessons learned from the financial crisis on validation​

• Validation Framework

• Validation Definition

• Validation principles

• Roles and responsibilities

• Scope and frequency

• Validation Process

• Internal Governance

• Validation of IRB Models

• Qualitative Validation

  • model design

  • data quality

  • Use Test

• Quantitative Validation

  • backtesting

  • Discriminating Power

  • Stability Tests

• Technological infrastructure

• Required documentation

• Internal validation department and team

• Audit department and team

• artificial intelligence

  • Autonomous validation, reconstruction and recalibration

  • Model Validation using Machine Learning

  • Artificial intelligence for model risk

  • Artificial intelligence to validate models​

​​

Module 6: Model Risk Management

​

• Governance in model risk

  • Role of the board and senior management

  • Role of the risk department

  • Responsible and creators of the model

  • Model risk committees

  • Internal audit

  • Policy definition

• Organization

  • Three lines of defense

  • Internal and external communication

• Model Life Cycle Phases

  • ID

  • Model Inventory

  • Classification

  • Levels or "Tiering" according to materiality, sophistication and impact

  • Planning and development

  • internal validation

  • Documentation

  • pattern approval

  • Implementation and use of the model

  • Model monitoring

  • International best practice risk management model

• Model Risk Mitigation

Data quality test

Validation and audit

Benchmarks

What-if sensitivity tests

Stress testing

backtesting

• Model risk appetite

  • Model risk appetite statement

  • risk tolerance

• Qualitative measurement of model risk

  • Creation of the Model Risk Scorecard

  • Definition of scale and ranges

  • International Scorecard Best Practices

• Case study: Scorecard for model risk

 

 

Quantification of Model Risk

​​

Module 7: Quantitative Measurement of Model Risk

​

• The importance of quantifying model risk

• Best international classifications

• Optimal and non-optimal models

• Categories of model risk:

• Data deficiency

• Model parameter uncertainty

• Inappropriate use of the model

• Model specification

• Change in the dynamics of the financial and economic environment

• Incorrect model implementation, misinterpretation of model output, and other errors

• Model risk aggregation

​

Module 8: Models for Quantifying Model Risk

​

• Why does this type of model risk occur?

• Examples

• Magnitude of model error related to human error

• Qualitative measurement of model risk

• Scorecard model

• Creation of the model risk scorecard

• Definition of scale and ranges

• International best practices for scorecards

• LDA approach to model risk measurement

• Loss Distribution Approach (LDA)

• Available data

• Scenario Based Approach (SBA)

• Scenario generation

• Scenario assessment

• Data quality and validation

• Parameter determination

• Frequency Distributions

• Severity Distributions

• Distribution of losses due to model risk

• Monte Carlo Simulation

• Panjer Approach

• FFT

• Bottom-up model uncertainty approach

• Sources of information

• Exercise 1: Estimation of model risk scorecard, establishment of scale and ranges.

• Exercise 2: Estimation of model risk using the following frequency and severity distributions:

• Frequency

  • Poisson and Negative binomial

• Severity

  • Lognormal, Gamma, Weibull, Inverse Gaussian

  • GDP EVT, G-H 4 parameter, Mixture of lognormals

  • Lognormal-EVT, Poisson-Gamma Bayesian Approach, Lognormal Partition and GDP

• Scenarios with Expert Criteria

• Exercise 3: Selection of the best distribution using Cramer Von Misses, AD, and KS goodness-of-fit tests

• Exercise 4: Comparison of model risk using Recursive Panjer, Fast Fourier Transformation, and Monte Carlo Simulation​

​

Model Risk in Credit Scoring

 

​Module 9: Model risk in Credit scoring

​​

• Dimension and use of materiality

• Classification of scoring models by importance within the financial institution

• Impact of the model on the entity

• model dependency

• Model limitations

• Model governance

• Documentation and review

• Implementation

• operational controls

• Decision tree to assess rating and scoring models

• Casuistry in expert Credit Rating models

• Casuistry in statistical credit scoring models

• Model risk in big data

• Risk model by machine learning and black box

​​

Module 10: Exploratory data analysis EDA and Binning 

​

• Exploratory data analysis

• Data typology

• transactional data

• Unstructured data embedded in text documents

• Social Media Data

• data sources

• Data review

• Target definition

• Time horizon of the target variable

• Sampling

  • Random Sampling

  • Stratified Sampling

  • Rebalanced Sampling

• Exploratory Analysis:

  • histograms

  • Q Q Plot

  • Moment analysis

  • boxplot

• Treatment of Missing values

  • imputation

  • Delete

  • Keep

• Advanced Outlier detection and treatment techniques

  • Z-Score

  • Mahalanobis Distance

• Data Standardization

• Binning

• Variable categorization

  • Equal Interval Binning

  • Equal Frequency Binning

  • Chi-Square Test

• binary coding

• WOE Coding

  • WOE Definition

  • Univariate Analysis with Target variable

  • Variable Selection

  • Treatment of Continuous Variables

  • Treatment of Categorical Variables

  • Gini and KS

  • Information Value

  • Optimization of continuous variables

  • Optimization of categorical variables

• ​Exercise 5: Exploratory Analysis in R

• Exercise 6: Detection and treatment of Outliers in R

• Exercise 7: Missing imputation techniques in R

• Exercise 8: Stratified and Random Sampling

• Exercise 9: Analysis of the Weight of Evidence in Excel

• Exercise 10: Univariate analysis in percentiles in R

• Exercise 11: Continuous variable optimal univariate analysis in R

• Exercise 12: KS, Gini and IV validation of each variable 

• Exercise 14: Optimizing categorical variables in R

• Exercise 15: Univariate Analysis with decision trees in R

• Exercise 16: Segmentation using K means Clustering in R

MACHINE LEARNING MODELS

​

Module 11: Machine Learning

​

• Unsupervised models​

• K Means

• Principal Component Analysis (PCA)

• Advanced PCA Visualization​​

• Supervised Models

• Ensemble Learning

• Bagging trees

• Random Forest

• Boosting

• Adaboost

• Gradient Boosting Trees

​

Deep Learning

 

Module 12: Deep Learning 

​

• Feed Forward Neural Networks

• Single Layer Perceptron

• Multiple Layer Perceptron

• Neural network architectures

• Activation function​

• Back propagation

  • Directional derivatives

  • gradients

  • Jacobians

  • Chain rule

  • Optimization and local and global minima

• Deep Learning Convolutional Neural Networks CNN​​

  • CNN for pictures

  • Design and architectures

  • convolution operation

  • filters

  • strider

  • padding

  • Subsampling

  • pooling

  • fully connected

  • Credit Scoring using CNN

  • Recent CNN studies applied to credit risk and scoring

• Deep Learning Recurrent Neural Networks RNN​​

  • Long Term Short Term Memory (LSTM)

  • Hopfield

  • Bidirectional associative memory

  • descending gradient

  • Global optimization methods

  • RNN and LSTM for credit scoring

  • One-way and two-way models​​

• Generative Adversarial Networks (GANs)

  • Generative Adversarial Networks (GANs)

  • Fundamental components of the GANs

  • GAN architectures

  • Bidirectional GAN

  • Training generative models

  • Credit Scoring using GANs

​​​

•  

Module 15: Generative AI​

​

• Introducing generative AI

• What is Generative AI?

• Generative AI Models

  • Generative Pre- trained Transformer (GPT)

• Text generation, Image generation, Music generation, Video generation

• Generating text

• Generating Code

• Ability to solve logic problems​

• Enterprise Use Cases for Generative AI

• Overview of Large Language Models (LLMs)

  • Transformer Architecture

  • Types of LLMs

  • Open-Source vs. Commercial LLMs

  • Key Concepts of LLMs

• Prompts

• Tokens

• Embeddings

• Model configuration

• Prompt Engineering

• Model adaptation

• Specifying multiple Dataframes to ChatGPT

• Exercise 17: Embeddings for words, sentences, question answers

• Exercise 18: Embeddings on Large Dataset

• Exercise 19: Advanced Prompting Techniques

• Exercise 20: Large Language Models in Credit Rating

​

Model risk in Credit Scoring​

​

Module 16: Multivariate models and Machine Learning

 

• Multivariate Models

  • Logistic regression

  • Cox Regression

  • Model Risk

• ​Machine Learning

  • decision trees

  • neural networks

  • SVM

  • Ensemble Learning

  • bagging

  • Boosting

  • Random Forest

• Deep Learning

  • CNN

  • LSTM

  • GAN

  • LLM

• Model Risk in Machine Learning​

  • overfitting

  • Transparency

  • failed sampling

  • important variables

• Exercise 21: Logistic Regression, stepwise method in R

• Exercise 22: Piecewise Regression in Excel and R

• Exercise 23: Decision trees in R

• Exercise 24: Support Vector Machine in R

• Exercise 25: Neural Networks in R

• Exercise 26: Ensemble models in R

• Exercise 27: Random Forest in R

• Exercise 28: Bagging in R

• Exercise 29: Credit scoring using deep learning CNN Python

• Exercise 30: Credit Scoring using Deep Learning LSTM Python

• Exercise 31: Credit Scoring using GANs Python

• Exercise 32: Comparison of models of discriminant power between models: Neural Networks, Logistic Regression, Panel Data Logistic Regression and Cox Regression

• Exercise 33: Model Risk using Confidence Intervals of Logistic Regression Coefficients

​

Module 14: Tuning Hyperparameters in Deep Learning

​

• Hyperparameterization

• Grid search

• Random search

• Bayesian Optimization

• Train test split ratio

• Learning rate in optimization algorithms (e.g. gradient descent)

• Selection of optimization algorithm (e.g., gradient descent, stochastic gradient descent, or Adam optimizer)

• Activation function selection in a (nn) layer neural network (e.g. Sigmoid, ReLU, Tanh)

• Selection of loss, cost and custom function

• Number of hidden layers in an NN

• Number of activation units in each layer

• The drop-out rate in nn (dropout probability)

• Number of iterations (epochs) in training a nn

• Number of clusters in a clustering task

• Kernel or filter size in convolutional layers

• Pooling size

• Batch size

• Interpretation of the Shap model

• Exercise 34: Tuning hyperparameters in Xboosting, Random forest and SVM models for credit scoring 

• Exercise 35: Tuning hyperparameters in Deep Learning model for credit scoring

​

​Module 17: Model Risk in the Scorecard

​

• Scoring assignment

• Scorecard Classification

  • Scorecard WOE

  • Binary Scorecard

  • Continuous Scorecard

• Scorecard Rescaling

  • Factor and Offset Analysis

  • Scorecard WOE

  • Binary Scorecard

• Reject Inference Techniques

  • cut-off

  • parceling

  • Fuzzy Augmentation

• Advanced Cut Point Techniques

  • Cut-off optimization using ROC curves

• Model risk by cut point decision

• Model risk due to lack of data

• Model Risk for not updating or recalibrating

• Exercise 36: Construction of Scorecard in Excel

• Exercise 37: Optimum cut-off point estimation in Excel and model risk by cut-off point selection

• Exercise 38: Confusion matrix to verify Type 1 and Type 2 Error in Excel with and without variables

• Exercise 39: Model risk in credit scoring due to not recalibrating on time​