from google.colab import drive
drive.mount('/content/drive')
import sys
sys.path.append("/content/drive/MyDrive/Colab Notebooks/instquality/")
import os
os.chdir("/content/drive/MyDrive/Colab Notebooks/instquality/")
import warnings
warnings.filterwarnings("ignore")
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
The complete codebase for this project is available at github.com/janjdarecki/instquality
Summary¶
This analysis asks which specific laws and institutions have the greatest influence on a country’s borrowing costs, as measured by the spread on a 10-year sovereign bond. While the literature shows that governance and political-institutional conditions matter, its reliance on broad indices makes it difficult to pinpoint which institutions matter most, whether they affect borrowing costs through levels or through changes, and how their influence unfolds over time. To address this, the project assembles a granular dataset of more than 100 institutional and macroeconomic variables from seven sources and evaluates them using complementary empirical exercises.
First, the incremental-signal analysis tests whether institutional and macro fundamentals help forecast next-year spreads beyond simple mean reversion. Regularised regressions (Lasso, Ridge, Elastic Net) are benchmarked against an autoregressive model. The results show that short-run spread movements are almost entirely driven by persistence, with fundamentals adding no marginal predictive content at one-year horizons. This confirms that most institutional and macroeconomic information is already priced in at short horizons and that changes in institutions rarely move spreads in the near term.
Second, the priced-in-levels analysis examines how much of today’s cross-country variation in sovereign spreads can be structurally explained by fundamentals. More than half of the dispersion is accounted for by institutional and macro variables. The three institutional variables that explain the largest share of current pricing levels are Monetary Freedom, the (Time) Cost of Importing and Exporting, and the Integrity of the Legal System.
To understand how these relationships evolve, both analyses are extended to horizons from one to ten years. For the incremental-signal regressions, short horizons remain persistence-dominated, but at medium-to-long horizons (roughly 4–10 years) additional signal from macro-institutional variables begins to emerge. For these regressions, Freedom of Foreigners to Visit - a measure of openness - becomes the only consistently important institutional variable.
Repeating the temporal regressions for the priced-in analysis show that structural legal quality — measured by judicial independence and the integrity of the legal system — emerge as the main long-run determinant of sovereign borrowing costs, dominating in importance in years 5-10 and consistently overshadowing other institutional factors.
Introducing the data¶
The dataset combines institutional‑quality and macroeconomic indicators from seven major international sources, covering 1960–2023. It brings together political, governance, and market‑freedom variables used as predictors of sovereign bond spreads.
A detailed overview of data coverage, quartile values, distributions, and pairwise correlations is available in the 01_load_and_visualise notebook. Data preparation, including handling of missing values, removal of sparsely populated columns, and engineering of lagged and averaged variables, was performed in 02_preprocess. For a full list of variables please refer to the functions/labels file.
| Source | Description | Period | Variables | Example Variables |
|---|---|---|---|---|
| EFW | Economic Freedom of the World — Fraser Institute | 1970–2021 | 79 | Judicial Independence, Freedom of Foreigners to Visit, Military Interference, Private Sector Credit, Monetary Freedom |
| FIW | Freedom in the World — Freedom House | 2006–2023 | 16 | Political Rights, Civil Liberties, Electoral Process, Rule of Law, Freedom of Expression |
| IEF | Index of Economic Freedom — Heritage Foundation | 1995–2023 | 13 | Property Rights, Government Integrity, Fiscal Health, Trade Freedom, Investment Freedom |
| P5D | Polity V Dataset — Polity Project | 1960–2020 | 17 | Democracy Score, Autocracy Score, Executive Constraints, Competitiveness of Participation, Regime Durability |
| PTS | Political Terror Scale | 1976–2022 | 6 | Political Terror Scale (Amnesty International, Human Rights Watch, State Department) |
| WGI | World Governance Indicators — World Bank | 1996–2022 | 36 | Voice & Accountability, Government Effectiveness, Regulatory Quality, Rule of Law, Control of Corruption |
| WB | World Bank (macro + institutional) | 1960–2023 | 46 | GDP Growth, Inflation (CPI), Current Account Balance, Central Government Debt, Statistical Capacity Index, Social Progress Index |
10‑Year Sovereign Spread over U.S. Treasuries (%) was constructed from OECD and World Bank bond yields (1960–2024) and serves as the unified dependent variable across all analyses, representing long‑term sovereign borrowing costs relative to a global risk‑free benchmark. More information on the target variable, including its construction, coverage, and descriptive statistics, can be found in the 01_load_and_visualise notebook.
Methods¶
The analysis uses a unified framework for regularised regression, implemented in functions/spec to estimate and interpret the relationship between institutional‑quality, macroeconomic variables, and sovereign bond spreads. The framework supports three model types — Lasso, Ridge, and Elastic Net — implemented with expanding‑window cross‑validation for time‑series consistency. The procedure automatically selects optimal regularisation parameters and can be applied in either an agnostic (unrestricted coefficients) or β=1 specification, the latter interpreting coefficients relative to mean‑reversion benchmarks. The analyses are executed in the 03_run_base_analysis and 04_run_temporal_analysis notebooks.
Feature preparation includes imputation, scaling, clustering, and correlation diagnostics. Highly correlated variable variants are grouped via hierarchical clustering (using a configurable correlation threshold), and only representative variables are retained to reduce redundancy. This ensures interpretability while allowing the model to capture cross‑sectional heterogeneity. Additional checks identify cross‑variable correlations above the threshold, primarily for transparency since regularisation already addresses multicollinearity.
To enhance interpretability, the framework integrates feature‑attribution and stability analysis. SHAP values quantify each variable’s contribution to predicted spreads, while stability selection evaluates how consistently each feature is chosen across repeated subsamples. Both are aggregated by core variable — merging lagged, differenced, and moving‑average variants into unified economic indicators. Model performance is evaluated using R², RMSE, and Diebold‑Mariano tests to assess whether fundamentals provide statistically significant incremental predictive power beyond mean reversion.
Results¶
Baseline incremental signal¶
This section asks whether institutional‑quality and macroeconomic fundamentals contain incremental forecasting power for next‑year sovereign spreads beyond simple mean reversion.
We first estimate an autoregressive benchmark model (t → t+1) to capture the persistence of spreads, then compare its out‑of‑sample accuracy with regularised regressions (Ridge, Lasso, Elastic Net) using macroeconmic and institutional quality (IQ) fundamentals as predictors.
Model performance is evaluated via R², RMSE, and Diebold‑Mariano tests for equal predictive accuracy.
# mean-reversion as benchmark
import pandas as pd
df = pd.read_pickle("saved/df.dat")
from functions.summary import show_simple_benchmark
show_simple_benchmark(df, 0.75)
# show_simple_benchmark(df, 0.8)
# show_simple_benchmark(df, 0.85)
Share of between-country variance in total variance: 90.72%
Mean-reversion benchmark results:
75–25 split: RMSE = 1.27, R² = 0.907, Share of between-country variance in total variance = 90.72%
80–20 split: RMSE = 1.22, R² = 0.908, Share = 91.36%
85–15 split: RMSE = 1.32, R² = 0.894, Share = 92.98%
Across all data splits, close to 90% of the variation in next‑year sovereign spreads is explained simply by the current spread. This highlights a very strong degree of persistence, consistent with the idea that sovereign credit premia adjust slowly over time. On average, the benchmark misses by roughly 1.2–1.3 percentage points (120–130 bps).
The stability of R² across multiple train–test splits confirms that this persistence is not an artefact of sample composition. The share of between‑country variance above 90% further indicates that much of the dispersion in spreads reflects structural cross‑country differences rather than time‑varying shocks.
This benchmark provides the starting point for the incremental signal analysis below, which tests whether institutional‑quality and macroeconomic variables can explain the unexpected component of next year's spread beyond what is captured by mean reversion. The subsequent section then examines how much of today’s spread level can be structurally explained by fundamentals repesented by macro and institutional quality variables (“priced‑in levels”).
from functions.summary import get_benchmark_stats, summarise_signal_regressions
import pandas as pd
df = pd.read_pickle("saved/df.dat")
benchmark_df_75 = get_benchmark_stats(df, 0.75)
benchmark_df_80 = get_benchmark_stats(df, 0.80)
benchmark_df_85 = get_benchmark_stats(df, 0.85)
benchmark_dfs = {
0.75: benchmark_df_75,
0.80: benchmark_df_80,
0.85: benchmark_df_85
}
summarise_signal_regressions(
"specs",
base_name="signal",
models=["ridge", "lasso", "elastic"],
benchmark_dfs=benchmark_dfs
)
================================================================================
INCREMENTAL SIGNAL REGRESSIONS — PERFORMANCE SUMMARY (75-25 split)
Benchmark R²: 0.9074, Benchmark RMSE: 1.2728
================================================================================
Model R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Incremental signal present DM_stat DM_p
Ridge 0.9085 1.2651 +0.11 -0.8 Rejected 1.03 0.303
Lasso 0.9090 1.2619 +0.16 -1.1 Rejected 1.00 0.319
Elastic Net (L1=0.75) 0.9087 1.2642 +0.12 -0.9 Rejected 1.00 0.316
Elastic Net (L1=0.5) 0.9086 1.2645 +0.12 -0.8 Rejected 1.08 0.281
Elastic Net (L1=0.25) 0.9086 1.2649 +0.12 -0.8 Rejected 1.08 0.281
================================================================================
INCREMENTAL SIGNAL REGRESSIONS — PERFORMANCE SUMMARY (80-20 split)
Benchmark R²: 0.9075, Benchmark RMSE: 1.2167
================================================================================
Model R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Incremental signal present DM_stat DM_p
Ridge 0.9063 1.2246 -0.12 +0.8 Rejected -0.82 0.414
Lasso 0.9079 1.2144 +0.04 -0.2 Rejected 0.16 0.873
Elastic Net (L1=0.75) 0.9075 1.2165 +0.00 -0.0 Rejected 0.02 0.982
Elastic Net (L1=0.5) 0.9073 1.2180 -0.02 +0.1 Rejected -0.12 0.908
Elastic Net (L1=0.25) 0.9071 1.2191 -0.04 +0.2 Rejected -0.24 0.814
================================================================================
INCREMENTAL SIGNAL REGRESSIONS — PERFORMANCE SUMMARY (85-15 split)
Benchmark R²: 0.8939, Benchmark RMSE: 1.3196
================================================================================
Model R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Incremental signal present DM_stat DM_p
Ridge 0.8919 1.3317 -0.20 +1.2 Rejected -1.02 0.307
Lasso 0.8936 1.3217 -0.03 +0.2 Rejected -0.12 0.906
Elastic Net (L1=0.75) 0.8933 1.3235 -0.06 +0.4 Rejected -0.26 0.791
Elastic Net (L1=0.5) 0.8930 1.3251 -0.09 +0.6 Rejected -0.42 0.677
Elastic Net (L1=0.25) 0.8928 1.3264 -0.11 +0.7 Rejected -0.54 0.587
The incremental‑signal exercise evaluates whether institutional quality (IQ) and macroeconomic variables contain predictive power for next‑year sovereign spreads beyond the mean‑reversion benchmark. Performance is benchmarked against the autoregressive baseline (R² approx 0.90), RMSE ≈ 1.2–1.3 p.p.).
Overall, the regularised models reproduce the benchmark’s accuracy almost exactly, but do not deliver statistically meaningful out‑of‑sample gains. This suggests that most of the information embedded in institutional and macro variables is already reflected in the current spread, reinforcing the strong persistence pattern documented earlier.
These findings set up the next section, which investigates the cross‑sectional (priced‑in level) determinants of spreads — assessing how much of the level of spreads can be structurally explained by fundamentals, even if they do not predict reversion dynamics.
Baseline priced-in levels (cross-sectional fundamentals)¶
Here we explore how much of the current level of sovereign spreads can be structurally explained by macroeconomic and institutional fundamentals.
Unlike the forecasting setup, these regressions are cross‑sectional: spreads and fundamentals are measured contemporaneously. We estimate regularised models (Ridge, Lasso, Elastic Net) and assess out‑of‑sample explanatory power (R², RMSE). SHAP importance values decompose which variables or categories contribute most to the priced‑in structure.
from functions.labels import all_labels
from functions.summary import summarise_priced_in_regressions
model_configs = [
("lasso", None),
("ridge", None),
("elastic", 0.25),
("elastic", 0.5),
("elastic", 0.75)
]
summarise_priced_in_regressions(
specs_dir="specs",
base_name="levels",
suffix="",
model_configs=model_configs,
all_labels=all_labels,
h=0
)
==========================================================================================
PRICED‑IN LEVEL REGRESSIONS — PERFORMANCE SUMMARY
==========================================================================================
Horizon Split Model R²_train R²_test RMSE_test
0 75‑25 Elastic Net (L1=0.25) 0.8492 0.5086 2.9377
0 75‑25 Elastic Net (L1=0.5) 0.8203 0.5097 2.9342
0 75‑25 Elastic Net (L1=0.75) 0.8002 0.5175 2.9110
0 75‑25 Lasso 0.7904 0.5146 2.9197
0 75‑25 Ridge 0.7440 0.5549 2.7959
0 80‑20 Elastic Net (L1=0.25) 0.8956 0.6021 2.5346
0 80‑20 Elastic Net (L1=0.5) 0.8793 0.6116 2.5042
0 80‑20 Elastic Net (L1=0.75) 0.8638 0.6030 2.5315
0 80‑20 Lasso 0.8828 0.6127 2.5007
0 80‑20 Ridge 0.8685 0.6231 2.4667
0 85‑15 Elastic Net (L1=0.25) 0.8918 0.6327 2.4706
0 85‑15 Elastic Net (L1=0.5) 0.8755 0.6389 2.4495
0 85‑15 Elastic Net (L1=0.75) 0.8600 0.6289 2.4831
0 85‑15 Lasso 0.8794 0.6307 2.4771
0 85‑15 Ridge 0.8748 0.6400 2.4458
* denotes variables selected in >80% of regressions (stability selection not performed for Ridge)
========================================================================================================================
LASSO — SHAP IMPORTANCE BY TRAIN‑TEST SPLIT (%)
========================================================================================================================
label Average 75‑25 80‑20 85‑15
Monetary Freedom (0–100) 12.69 21.07* 8.34* 8.65*
2E. Integrity of the Legal System (0–10) 9.11 21.18* 3.35* 2.81*
3B. Standard Deviation of Inflation (0–10) 7.44 3.97 9.30* 9.05*
4B(ii). Costs of Importing and Exporting (0–10) 6.15 9.50* 4.41* 4.55*
Current Account Balance (% of GDP) 5.47 9.23* 3.52* 3.67*
2D. Military Interference in Rule of Law and Politics (0–10) 4.56 11.36* 1.41* 0.91*
1B. Transfers and Subsidies (0–10) 2.88 0.00 4.18* 4.46*
Inflation (CPI, annual %) 2.84 5.73 1.34* 1.44*
5C(iii). Impartial Public Administration (0–10) 2.37 0.00 3.31* 3.81*
2B. Impartial Courts (0–10) 2.23 6.26 0.22 0.21
5A(ii). Private Sector Credit (0–10) 2.13 2.07 1.84* 2.47*
A. Electoral Process (0–12) 1.94 0.00 2.44* 3.39*
4D(i). Financial Openness (0–10) 1.92 0.00 2.82* 2.95*
2A. Judicial Independence (0–10) 1.77 0.00 2.68* 2.63
Real GDP Growth (annual %, constant prices) 1.69 0.00 2.75* 2.33*
5A(i). Ownership of Banks (0–10) 1.57 0.00 2.28* 2.44*
5C(i). Regulatory Burden (0–10) 1.54 0.00 2.44* 2.19*
Gender Disparity Index (0–10, higher = more equality) 1.44 0.00 2.22* 2.11*
1C. Government Investment (0–10) 1.25 0.00 1.85* 1.91*
Area 4. Freedom to Trade Internationally (0–10) 1.24 0.35 1.45* 1.91*
1E. State Ownership of Assets (0–10) 1.20 0.00 1.81* 1.78*
GDP per capita (constant LCU) 1.17 1.03 1.25* 1.22*
GDP Deflator (annual % change) 1.10 1.76 0.70* 0.85*
Mining Output (% of GDP) 1.02 0.07 1.46* 1.53*
4D(ii). Capital Controls (0–10) 1.00 0.00 1.42* 1.58*
========================================================================================================================
RIDGE — SHAP IMPORTANCE BY TRAIN‑TEST SPLIT (%)
========================================================================================================================
label Average 75‑25 80‑20 85‑15
Real GDP Growth (annual %, constant prices) 3.52 4.85 2.92 2.78
Monetary Freedom (0–100) 3.02 2.69 3.12 3.25
Current Account Balance (% of GDP) 2.57 3.29 2.27 2.14
5A(ii). Private Sector Credit (0–10) 2.46 2.43 2.28 2.68
1B. Transfers and Subsidies (0–10) 2.32 2.15 2.38 2.44
4B(ii). Costs of Importing and Exporting (0–10) 2.31 2.48 2.22 2.23
4D(iii). Freedom of Foreigners to Visit (0–10) 1.96 1.95 1.58 2.35
1E. State Ownership of Assets (0–10) 1.96 2.47 1.75 1.66
2A. Judicial Independence (0–10) 1.90 1.88 1.90 1.93
3B. Standard Deviation of Inflation (0–10) 1.75 1.33 1.91 2.01
2E. Integrity of the Legal System (0–10) 1.74 1.88 1.70 1.65
2B. Impartial Courts (0–10) 1.61 2.14 1.46 1.22
Political Terror Scale – State Department (1–5, higher = more terror) 1.58 1.67 1.54 1.52
Gender Disparity Index (0–10, higher = more equality) 1.57 1.50 1.66 1.56
Area 4. Freedom to Trade Internationally (0–10) 1.33 1.46 1.22 1.31
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 1.32 1.73 1.32 0.92
Area 1. Size of Government (0–10) 1.30 1.10 1.34 1.45
5D(ii). Business Permits (0–10) 1.29 1.13 1.46 1.28
Social Progress Index – Basic Human Needs (0–100) 1.25 1.74 1.05 0.95
5C(ii). Bureaucracy Costs (0–10) 1.24 0.72 1.50 1.51
3A. Money Growth (0–10) 1.22 1.23 1.14 1.29
5C(iii). Impartial Public Administration (0–10) 1.22 1.40 1.31 0.96
5A. Credit Market Regulation (0–10) 1.20 0.97 1.27 1.36
5B(ii). Hiring and Firing Regulations (0–10) 1.20 0.88 1.29 1.44
Business Freedom (0–100) 1.18 1.11 1.30 1.13
4D. Controls of Movement of Capital and People (0–10) 1.17 1.14 1.15 1.22
5C(i). Regulatory Burden (0–10) 1.16 1.16 1.23 1.08
4D(i). Financial Openness (0–10) 1.13 1.35 1.04 0.99
4A(iii). Tariff Rate Dispersion (0–10) 1.13 1.71 0.89 0.80
Central Government Debt (% of GDP) 1.11 0.99 1.11 1.23
Area 3. Sound Money (0–10) 1.08 0.97 1.12 1.15
Trade Freedom (0–100) 1.05 0.86 1.13 1.17
2H. Reliability of Police and Crime (0–10) 1.05 0.71 1.25 1.18
Fuel Exports (% of merchandise exports) 1.05 0.99 1.23 0.94
1C. Government Investment (0–10) 1.04 0.91 1.07 1.14
5D. Freedom to Enter Markets and Compete (0–10) 1.02 0.71 1.18 1.16
GDP per capita (constant LCU) 1.00 1.14 0.97 0.89
========================================================================================================================
EN (L1=0.25) — SHAP IMPORTANCE BY TRAIN‑TEST SPLIT (%)
========================================================================================================================
label Average 75‑25 80‑20 85‑15
Monetary Freedom (0–100) 6.65 7.93* 6.08* 5.93*
3B. Standard Deviation of Inflation (0–10) 4.82 4.93* 4.75* 4.77*
1B. Transfers and Subsidies (0–10) 3.95 4.72* 3.38* 3.74*
4B(ii). Costs of Importing and Exporting (0–10) 3.28 3.88* 2.86* 3.09*
Real GDP Growth (annual %, constant prices) 3.08 4.48* 2.39* 2.37*
Current Account Balance (% of GDP) 2.92 4.66* 1.96* 2.13*
5A(ii). Private Sector Credit (0–10) 2.67 2.96* 2.29* 2.76*
2A. Judicial Independence (0–10) 2.37 1.89* 2.64* 2.57*
2E. Integrity of the Legal System (0–10) 2.35 2.92* 2.05* 2.07*
4D(i). Financial Openness (0–10) 2.13 2.87* 1.71* 1.80*
Gender Disparity Index (0–10, higher = more equality) 2.11 2.47* 2.00* 1.87*
5C(iii). Impartial Public Administration (0–10) 2.04 2.56* 1.88* 1.68*
1E. State Ownership of Assets (0–10) 2.00 2.33* 1.85* 1.81*
5C(i). Regulatory Burden (0–10) 1.66 1.96* 1.58* 1.43*
Political Terror Scale – State Department (1–5, higher = more terror) 1.62 2.04* 1.38* 1.44*
5D(ii). Business Permits (0–10) 1.57 0.93* 2.08* 1.71*
Property Rights (0–100) 1.48 0.97* 1.83* 1.63*
1C. Government Investment (0–10) 1.44 1.61* 1.31* 1.40*
Area 4. Freedom to Trade Internationally (0–10) 1.44 1.71* 1.23* 1.38*
2D. Military Interference in Rule of Law and Politics (0–10) 1.43 2.56* 0.91* 0.83*
GDP per capita (constant LCU) 1.43 2.17* 1.16* 0.95*
2H. Reliability of Police and Crime (0–10) 1.41 0.89 1.71* 1.64*
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 1.36 1.88* 1.25* 0.94*
2B. Impartial Courts (0–10) 1.35 2.00* 1.12* 0.94*
Area 3. Sound Money (0–10) 1.34 1.06* 1.47* 1.50*
Fuel Exports (% of merchandise exports) 1.29 1.74* 1.25* 0.88*
5A(i). Ownership of Banks (0–10) 1.29 2.07* 0.86* 0.93*
5C(ii). Bureaucracy Costs (0–10) 1.26 0.00 2.17* 1.61*
4A(iii). Tariff Rate Dispersion (0–10) 1.23 2.28* 0.69* 0.71*
5A. Credit Market Regulation (0–10) 1.23 0.43 1.55* 1.72*
Mining Output (% of GDP) 1.13 0.97* 1.17* 1.25*
4B. Regulatory Trade Barriers (0–10) 1.09 1.51* 0.94* 0.81*
5B(ii). Hiring and Firing Regulations (0–10) 1.09 0.00 1.53* 1.74*
A. Electoral Process (0–12) 1.08 0.24 1.41* 1.60*
Regime Durability (years since last regime change) 1.07 1.24* 1.12* 0.84*
5B(iii). Flexible Wage Determination (0–10) 1.05 0.97* 1.09* 1.10*
========================================================================================================================
EN (L1=0.5) — SHAP IMPORTANCE BY TRAIN‑TEST SPLIT (%)
========================================================================================================================
label Average 75‑25 80‑20 85‑15
Monetary Freedom (0–100) 9.19 12.32* 7.74* 7.52*
3B. Standard Deviation of Inflation (0–10) 5.91 5.45* 6.18* 6.09*
Current Account Balance (% of GDP) 4.83 7.97* 3.15* 3.38*
4B(ii). Costs of Importing and Exporting (0–10) 4.68 5.89* 3.94* 4.20*
1B. Transfers and Subsidies (0–10) 4.35 4.32* 4.26* 4.48*
2E. Integrity of the Legal System (0–10) 3.52 5.73* 2.43* 2.41*
Real GDP Growth (annual %, constant prices) 3.41 4.63* 2.93* 2.68*
5A(ii). Private Sector Credit (0–10) 2.85 3.26* 2.32* 2.98*
2D. Military Interference in Rule of Law and Politics (0–10) 2.63 5.58* 1.23* 1.09*
5C(iii). Impartial Public Administration (0–10) 2.47 2.15* 2.58* 2.68*
4D(i). Financial Openness (0–10) 2.33 2.02* 2.44* 2.53*
1E. State Ownership of Assets (0–10) 2.07 2.30* 1.98* 1.94*
2A. Judicial Independence (0–10) 2.02 1.38 2.38* 2.31*
Gender Disparity Index (0–10, higher = more equality) 1.99 1.83* 2.12* 2.01*
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 1.77 3.13* 1.21* 0.98*
GDP per capita (constant LCU) 1.67 2.15* 1.51* 1.34*
5A(i). Ownership of Banks (0–10) 1.66 1.52 1.73* 1.74*
Area 4. Freedom to Trade Internationally (0–10) 1.63 1.72* 1.48* 1.68*
2B. Impartial Courts (0–10) 1.47 2.95* 0.82* 0.64*
Mining Output (% of GDP) 1.45 1.59* 1.32* 1.45*
A. Electoral Process (0–12) 1.44 0.00 1.84* 2.48*
GDP Deflator (annual % change) 1.44 2.30* 0.96* 1.06*
Political Terror Scale – State Department (1–5, higher = more terror) 1.43 1.84* 1.11* 1.35*
5C(i). Regulatory Burden (0–10) 1.38 0.27 2.01* 1.85*
Fuel Exports (% of merchandise exports) 1.37 1.85* 1.31* 0.96*
1C. Government Investment (0–10) 1.32 0.40 1.76* 1.81*
4A(iii). Tariff Rate Dispersion (0–10) 1.18 1.49* 0.98* 1.06*
Inflation (CPI, annual %) 1.17 1.83* 0.79* 0.88*
Area 3. Sound Money (0–10) 1.17 1.08 1.29* 1.15*
5D(ii). Business Permits (0–10) 1.12 0.00 1.94* 1.42*
2H. Reliability of Police and Crime (0–10) 1.09 0.00 1.64* 1.62*
Regime Durability (years since last regime change) 1.06 0.85* 1.35* 0.98*
4B. Regulatory Trade Barriers (0–10) 1.01 1.14 1.10 0.79
========================================================================================================================
EN (L1=0.75) — SHAP IMPORTANCE BY TRAIN‑TEST SPLIT (%)
========================================================================================================================
label Average 75‑25 80‑20 85‑15
Monetary Freedom (0–100) 11.66 15.50* 9.73* 9.74*
4B(ii). Costs of Importing and Exporting (0–10) 6.62 9.60* 5.10* 5.17*
Current Account Balance (% of GDP) 6.31 9.47* 4.58* 4.88*
2E. Integrity of the Legal System (0–10) 6.09 11.10* 3.81* 3.36*
3B. Standard Deviation of Inflation (0–10) 5.97 5.01* 6.76* 6.15*
2D. Military Interference in Rule of Law and Politics (0–10) 4.28 9.17* 2.07* 1.61*
1B. Transfers and Subsidies (0–10) 3.73 1.55 4.70* 4.94*
5C(iii). Impartial Public Administration (0–10) 2.92 1.68 3.36* 3.72*
5A(ii). Private Sector Credit (0–10) 2.79 3.12* 2.30* 2.95*
Real GDP Growth (annual %, constant prices) 2.60 2.39 3.08* 2.34
2B. Impartial Courts (0–10) 2.07 5.20* 0.49 0.53
GDP Deflator (annual % change) 1.96 3.00* 1.35* 1.52*
4D(i). Financial Openness (0–10) 1.94 0.00 2.90* 2.92*
1E. State Ownership of Assets (0–10) 1.85 1.40 2.12* 2.02*
A. Electoral Process (0–12) 1.68 0.00 2.10* 2.95*
5A(i). Ownership of Banks (0–10) 1.66 0.00 2.45* 2.53*
GDP per capita (constant LCU) 1.65 1.72* 1.67* 1.56*
Inflation (CPI, annual %) 1.55 2.46 1.03* 1.15*
5C(i). Regulatory Burden (0–10) 1.55 0.00 2.38* 2.27*
Area 4. Freedom to Trade Internationally (0–10) 1.53 1.24 1.56* 1.79*
Mining Output (% of GDP) 1.50 1.41* 1.52* 1.58*
1C. Government Investment (0–10) 1.48 0.00 2.18* 2.27*
Gender Disparity Index (0–10, higher = more equality) 1.39 0.03 2.15* 1.99*
Fuel Exports (% of merchandise exports) 1.33 1.79 1.32* 0.87*
2A. Judicial Independence (0–10) 1.31 0.00 2.12* 1.82*
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 1.31 3.90* 0.04 0.00
Oil & Gas Output (% of GDP) 1.12 2.13* 0.68* 0.55*
Area 3. Sound Money (0–10) 1.05 1.51 0.79 0.85*
4A(iii). Tariff Rate Dispersion (0–10) 1.03 0.00 1.45* 1.64*
============================================================================================================================================
FINAL AVERAGE SHAP IMPORTANCE BY MODEL (%)
============================================================================================================================================
Average LASSO RIDGE EN (L1=0.25) EN (L1=0.5) EN (L1=0.75)
Monetary Freedom (0–100) 8.64 12.69 3.02 6.65 9.19 11.66
3B. Standard Deviation of Inflation (0–10) 5.18 7.44 1.75 4.82 5.91 5.97
4B(ii). Costs of Importing and Exporting (0–10) 4.61 6.15 2.31 3.28 4.68 6.62
2E. Integrity of the Legal System (0–10) 4.56 9.11 1.74 2.35 3.52 6.09
Current Account Balance (% of GDP) 4.42 5.47 2.57 2.92 4.83 6.31
1B. Transfers and Subsidies (0–10) 3.45 2.88 2.32 3.95 4.35 3.73
Real GDP Growth (annual %, constant prices) 2.86 1.69 3.52 3.08 3.41 2.60
5A(ii). Private Sector Credit (0–10) 2.58 2.13 2.46 2.67 2.85 2.79
2D. Military Interference in Rule of Law and Politics (0–10) 2.58 4.56 0.00 1.43 2.63 4.28
5C(iii). Impartial Public Administration (0–10) 2.20 2.37 1.22 2.04 2.47 2.92
4D(i). Financial Openness (0–10) 1.89 1.92 1.13 2.13 2.33 1.94
2A. Judicial Independence (0–10) 1.87 1.77 1.90 2.37 2.02 1.31
1E. State Ownership of Assets (0–10) 1.82 1.20 1.96 2.00 2.07 1.85
2B. Impartial Courts (0–10) 1.75 2.23 1.61 1.35 1.47 2.07
Gender Disparity Index (0–10, higher = more equality) 1.70 1.44 1.57 2.11 1.99 1.39
5C(i). Regulatory Burden (0–10) 1.46 1.54 1.16 1.66 1.38 1.55
Area 4. Freedom to Trade Internationally (0–10) 1.43 1.24 1.33 1.44 1.63 1.53
GDP per capita (constant LCU) 1.38 1.17 1.00 1.43 1.67 1.65
1C. Government Investment (0–10) 1.31 1.25 1.04 1.44 1.32 1.48
5A(i). Ownership of Banks (0–10) 1.24 1.57 0.00 1.29 1.66 1.66
A. Electoral Process (0–12) 1.23 1.94 0.00 1.08 1.44 1.68
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 1.15 0.00 1.32 1.36 1.77 1.31
Inflation (CPI, annual %) 1.11 2.84 0.00 0.00 1.17 1.55
Mining Output (% of GDP) 1.02 1.02 0.00 1.13 1.45 1.50
Fuel Exports (% of merchandise exports) 1.01 0.00 1.05 1.29 1.37 1.33
The priced‑in level regressions test how far today’s sovereign spread levels can be explained by structural fundamentals. Across models and data splits, institutional and macroeconomic variables jointly account for about 50–63% of the cross‑sectional variance in spreads, with stable out‑of‑sample R² values around 0.5–0.6 and RMSE near 2.5–2.9 percentage points. Performance improves modestly with larger training samples, confirming the robustness of the underlying relationships.
Institutional quality variables play a key role in shaping the permanent component of sovereign risk. Monetary Freedom, Costs of Importing and Exporting and Integrity of the Legal System emerge as the most systematically selected features. Measures of state capacity and governance (Impartial Public Administration, Transfers and Subsidies, Military Interference) further contribute to explaining long‑run differences in perceived sovereign credibility.
Macroeconomic fundamentals complement this institutional structure. External balances, inflation stability and GDP growth all show consistent importance across models, suggesting that macro discipline is a priced dimension of sovereign spreads. Together, these results imply that while short‑term changes in spreads are largely unpredictable, cross‑country differences reflect persistent macro‑institutional foundations that markets incorporate into risk premia.
Temporal incremental signal¶
This section examines whether institutional‑quality and macroeconomic variables provide incremental predictive signal for sovereign spreads over longer forecast horizons (1–10 years) and how this signal evolves with time.
We re‑estimate the mean‑reversion benchmark and regularised models at each horizon to assess whether fundamentals' predictive relevance as persistence decays. Model performance is compared across horizons using out‑of‑sample R², RMSE, and Diebold‑Mariano tests, which classify each horizon as dominated by signal, noise, or equal predictive accuracy relative to the benchmark.
# compare performance across splits for each model
import pandas as pd
df = pd.read_pickle("saved/df.dat")
from functions.summary import plot_model_splits
plot_model_splits("benchmark", title_text="Benchmark mean-reversion model", df=df)
The benchmark mean‑reversion model shows strong short‑term persistence, with test R² values close to 0.9 for one‑year‑ahead spreads. Predictive power declines steadily with the forecast horizon, reaching around 0.4–0.5 by year 8–10, while RMSE rises from roughly 1.3 p.p. to around 3.0 p.p. over the same period.
Across test splits, results are broadly consistent but not identical. The 85‑15 split tends to retain slightly higher R² at intermediate horizons, while the 25% test split shows a smoother decline. These mild discrepancies suggest that the strength of mean‑reversion and persistence varies somewhat with sample composition and the balance between high‑ and low‑volatility periods.
Overall, the temporal pattern points to a slow yet measurable erosion of spread predictability over time: near‑term dynamics are dominated by persistence, while longer horizons increasingly reflect structural differences and accumulated shocks that are harder to forecast.
# incremental signal regressions across 1 to 10 year horizon
from functions.summary import get_benchmark_stats, print_temporal_signal_stats, plot_temporal_signal_summary
import pandas as pd
df = pd.read_pickle("saved/df.dat")
benchmark_df_75 = get_benchmark_stats(df, 0.75)
benchmark_df_80 = get_benchmark_stats(df, 0.80)
benchmark_df_85 = get_benchmark_stats(df, 0.85)
benchmark_dfs = {
0.75: benchmark_df_75,
0.80: benchmark_df_80,
0.85: benchmark_df_85
}
print_temporal_signal_stats(
specs_dir="specs",
base_name="signal",
models=["ridge", "lasso", "elastic"],
benchmark_dfs=benchmark_dfs
)
plot_temporal_signal_summary(
specs_dir="specs",
base_name="signal",
models=["lasso", "ridge", "elastic"],
benchmark_dfs=benchmark_dfs
)
---------------------------------------------------------------------------------------------------------------------------------------
75-25 TRAIN-TEST SET SPLIT
---------------------------------------------------------------------------------------------------------------------------------------
>>> RIDGE
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.9085 1.2651 +0.11 -0.8 Rejected 1.03 0.303
2 0.7969 1.8737 +0.26 -1.2 Rejected 1.32 0.186
3 0.7670 1.9231 +0.51 -2.1 Rejected 1.94 0.052
4 0.7328 2.0693 +0.94 -3.6 Accepted (SIGNAL) 2.68 0.007
5 0.6416 2.3935 +2.12 -7.0 Accepted (SIGNAL) 4.45 0.000
6 0.6033 2.5468 +4.39 -13.7 Accepted (SIGNAL) 5.94 0.000
7 0.5940 2.5702 +6.34 -19.4 Accepted (SIGNAL) 8.04 0.000
8 <0 >100 - - Accepted (NOISE) -7.40 0.000
9 <0 >100 - - Accepted (NOISE) -7.03 0.000
10 0.4689 2.9300 +9.60 -25.4 Accepted (SIGNAL) 7.75 0.000
>>> LASSO
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.9090 1.2619 +0.16 -1.1 Rejected 1.00 0.319
2 0.8012 1.8538 +0.69 -3.2 Rejected 0.91 0.363
3 0.7677 1.9203 +0.58 -2.4 Rejected 0.82 0.412
4 0.7318 2.0732 +0.84 -3.2 Rejected 1.73 0.084
5 0.5979 2.5354 -2.26 +7.2 Accepted (NOISE) -2.01 0.045
6 0.5798 2.6211 +2.04 -6.3 Accepted (SIGNAL) 3.65 0.000
7 0.5786 2.6188 +4.79 -14.5 Accepted (SIGNAL) 7.35 0.000
8 0.4400 3.0187 +3.99 -10.6 Accepted (SIGNAL) 6.46 0.000
9 0.3927 3.1333 +5.20 -13.1 Accepted (SIGNAL) 7.19 0.000
10 0.4281 3.0404 +5.52 -14.3 Accepted (SIGNAL) 6.22 0.000
>>> ELASTIC NET (L1=0.5)
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.9086 1.2645 +0.12 -0.8 Rejected 1.08 0.281
2 0.8002 1.8581 +0.60 -2.8 Rejected 0.94 0.345
3 0.7710 1.9068 +0.90 -3.7 Rejected 1.49 0.136
4 0.7443 2.0240 +2.10 -8.2 Accepted (SIGNAL) 3.11 0.002
5 0.6220 2.4582 +0.16 -0.5 Rejected 0.21 0.836
6 0.5840 2.6081 +2.46 -7.6 Rejected 1.29 0.196
7 0.5833 2.6040 +5.27 -16.0 Accepted (SIGNAL) 7.73 0.000
8 0.4671 2.9448 +6.70 -18.0 Accepted (SIGNAL) 8.63 0.000
9 0.4193 3.0639 +7.86 -20.1 Accepted (SIGNAL) 8.60 0.000
10 0.4499 2.9818 +7.70 -20.2 Accepted (SIGNAL) 7.07 0.000
---------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------
80-20 TRAIN-TEST SET SPLIT
---------------------------------------------------------------------------------------------------------------------------------------
>>> RIDGE
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.9063 1.2246 -0.12 +0.8 Rejected -0.82 0.414
2 0.7919 1.8057 +0.20 -0.8 Rejected 0.69 0.493
3 0.7462 1.9953 +0.35 -1.4 Rejected 1.04 0.298
4 0.7262 2.0802 +0.55 -2.1 Rejected 1.26 0.206
5 0.7058 2.1701 +0.89 -3.3 Rejected 1.51 0.132
6 0.6333 2.4518 +3.80 -12.4 Accepted (SIGNAL) 4.02 0.000
7 0.5831 2.6071 +6.53 -19.7 Accepted (SIGNAL) 6.64 0.000
8 0.5149 2.8108 +8.99 -24.9 Accepted (SIGNAL) 5.71 0.000
9 <0 >100 - - Accepted (NOISE) -7.09 0.000
10 <0 >100 - - Accepted (NOISE) -6.82 0.000
>>> LASSO
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.9079 1.2144 +0.04 -0.2 Rejected 0.16 0.873
2 0.7993 1.7731 +0.94 -4.1 Rejected 0.97 0.331
3 0.7478 1.9891 +0.50 -2.0 Rejected 0.65 0.513
4 0.7271 2.0771 +0.63 -2.4 Rejected 1.17 0.242
5 0.6886 2.2326 -0.83 +3.0 Rejected -0.61 0.540
6 0.6151 2.5118 +1.98 -6.4 Accepted (SIGNAL) 2.86 0.004
7 0.5688 2.6515 +5.10 -15.2 Accepted (SIGNAL) 6.63 0.000
8 0.4722 2.9319 +4.71 -12.8 Accepted (SIGNAL) 6.23 0.000
9 0.3922 3.1617 +5.74 -14.6 Accepted (SIGNAL) 6.93 0.000
10 0.4132 3.1112 +6.19 -16.0 Accepted (SIGNAL) 6.92 0.000
>>> ELASTIC NET (L1=0.5)
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.9073 1.2180 -0.02 +0.1 Rejected -0.12 0.908
2 0.7979 1.7792 +0.80 -3.5 Rejected 1.00 0.318
3 0.7506 1.9779 +0.78 -3.1 Rejected 1.16 0.245
4 0.7347 2.0479 +1.40 -5.3 Rejected 1.76 0.078
5 0.6996 2.1928 +0.27 -1.0 Rejected 0.30 0.763
6 0.6260 2.4760 +3.07 -10.0 Accepted (SIGNAL) 4.04 0.000
7 0.5732 2.6378 +5.55 -16.6 Accepted (SIGNAL) 6.90 0.000
8 0.4956 2.8660 +7.06 -19.4 Accepted (SIGNAL) 7.54 0.000
9 0.4180 3.0938 +8.32 -21.4 Accepted (SIGNAL) 7.93 0.000
10 0.4394 3.0409 +8.81 -23.0 Accepted (SIGNAL) 8.16 0.000
---------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------
85-15 TRAIN-TEST SET SPLIT
---------------------------------------------------------------------------------------------------------------------------------------
>>> RIDGE
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.8919 1.3317 -0.20 +1.2 Rejected -1.02 0.307
2 0.8105 1.7401 -0.74 +3.4 Accepted (NOISE) -2.36 0.018
3 0.8160 1.7159 -1.48 +7.0 Accepted (NOISE) -3.77 0.000
4 0.7427 2.0379 -1.32 +5.3 Accepted (NOISE) -2.35 0.019
5 0.6803 2.2877 -1.68 +6.1 Accepted (NOISE) -2.06 0.039
6 0.7352 2.1165 -1.33 +5.4 Rejected -1.26 0.208
7 0.7523 2.0397 +1.91 -7.7 Rejected 1.52 0.128
8 0.5635 2.7080 +8.27 -24.5 Accepted (SIGNAL) 3.82 0.000
9 0.4789 2.9752 +11.06 -30.1 Accepted (SIGNAL) 4.32 0.000
10 0.5514 2.7809 +13.56 -39.3 Accepted (SIGNAL) 4.24 0.000
>>> LASSO
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.8936 1.3217 -0.03 +0.2 Rejected -0.12 0.906
2 0.8190 1.7003 +0.11 -0.5 Rejected 0.10 0.920
3 0.8125 1.7322 -1.83 +8.7 Accepted (NOISE) -2.85 0.004
4 0.7430 2.0367 -1.29 +5.2 Accepted (NOISE) -1.98 0.047
5 0.6807 2.2864 -1.65 +6.0 Rejected -0.96 0.336
6 0.7392 2.1003 -0.93 +3.8 Rejected -0.93 0.352
7 0.7504 2.0475 +1.72 -6.9 Rejected 1.72 0.086
8 0.5367 2.7900 +5.58 -16.3 Accepted (SIGNAL) 2.22 0.026
9 0.4403 3.0836 +7.20 -19.2 Accepted (SIGNAL) 4.13 0.000
10 0.4772 3.0023 +6.14 -17.1 Accepted (SIGNAL) 4.93 0.000
>>> ELASTIC NET (L1=0.5)
---------------------------------------------------------------------------------------------------------------------------------------
horizon R²_test RMSE_test ΔR² vs Benchmark (p.p.) ΔRMSE vs Benchmark (bps) H1=Non-equal predictive accuracy (DM test) DM_stat DM_p
1 0.8930 1.3251 -0.09 +0.6 Rejected -0.42 0.677
2 0.8166 1.7117 -0.13 +0.6 Rejected -0.13 0.894
3 0.8155 1.7181 -1.52 +7.3 Accepted (NOISE) -2.58 0.010
4 0.7456 2.0262 -1.02 +4.1 Rejected -1.03 0.302
5 0.6836 2.2761 -1.36 +5.0 Rejected -1.55 0.121
6 0.7377 2.1065 -1.08 +4.4 Rejected -0.95 0.340
7 0.7514 2.0433 +1.82 -7.4 Rejected 1.70 0.089
8 0.5382 2.7855 +5.73 -16.8 Accepted (SIGNAL) 4.29 0.000
9 0.4457 3.0685 +7.74 -20.7 Accepted (SIGNAL) 5.12 0.000
10 0.4958 2.9483 +8.00 -22.5 Accepted (SIGNAL) 5.33 0.000
---------------------------------------------------------------------------------------------------------------------------------------
Across forecast horizons, the Diebold‑Mariano outcomes confirm three distinct predictive regimes. For short horizons (1–2 years), nearly all models fall into the equal‑accuracy category: incremental regressions track the mean‑reversion benchmark almost perfectly, with no additional predictive signal. At these horizons, spreads are dominated by persistence, and model complexity adds noise rather than insight.
At intermediate horizons (3–5 years), results become mixed: noise outcomes rise as models begin to deviate from mean reversion without yielding consistent forecast improvements. This stage reflects transitional dynamics — diminishing autocorrelation from the benchmark, but insufficient long‑term signal yet emerging from the fundamentals.
From six years onward, a more systematic signal appears. Predictive accuracy improves across regularised models, with the a strong concentration of signal at year 7 to 10, with year 9 showing a partial reversion toward noise. This alternating pattern suggests that fundamentals gradually gain explanatory strength over longer horizons, though the power is uneven and interacts with sample composition and horizon‑specific volatility. In summary, short‑term forecasts are persistence‑driven, the mid‑range is noisy and unstable, and long‑horizon models capture genuine but horizon‑sensitive structural signal.
# which variables provide incremental signal?
from functions.summary import summarise_temporal_signal_vars
from functions.labels import all_labels
summarise_temporal_signal_vars(
specs_dir="specs",
base_name="signal",
models=["ridge", "lasso", "elastic"],
all_labels=all_labels
)
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T4 (%)
======================================================================================================================================================
Label Average Lasso (–) Ridge (75/25) Elastic Net (L1=0.5) (75/25)
4D(iii). Freedom of Foreigners to Visit (0–10) 22.92 - 45.83 0.00
Monetary Freedom (0–100) 15.65 - 2.21 29.09
Real GDP Growth (annual %, constant prices) 13.88 - 1.95 25.81
Inflation (CPI, annual %) 7.33 - 0.28 14.38
Current Account Balance (% of GDP) 6.99 - 0.95 13.03
2C. Protection of Property Rights (0–10) 5.46 - 0.84 10.07
3B. Standard Deviation of Inflation (0–10) 2.33 - 1.00 3.65
Social Progress Index – Basic Human Needs (0–100) 1.78 - 3.55 0.00
Oil & Gas Output (% of GDP) 1.76 - 0.49 3.02
5C(ii). Bureaucracy Costs (0–10) 1.14 - 1.40 0.88
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T5 (%)
======================================================================================================================================================
Label Average Lasso (–) Ridge (75/25) Elastic Net (L1=0.5) (–)
4D(iii). Freedom of Foreigners to Visit (0–10) 25.80 - 25.80 -
Real GDP Growth (annual %, constant prices) 3.12 - 3.12 -
Monetary Freedom (0–100) 2.70 - 2.70 -
Social Progress Index – Basic Human Needs (0–100) 2.44 - 2.44 -
G. Personal Autonomy & Individual Rights (0–16) 1.54 - 1.54 -
1E. State Ownership of Assets (0–10) 1.50 - 1.50 -
3B. Standard Deviation of Inflation (0–10) 1.38 - 1.38 -
Civil Liberties (0–60) 1.29 - 1.29 -
4B(ii). Costs of Importing and Exporting (0–10) 1.28 - 1.28 -
5D(i). Market Openness (0–10) 1.28 - 1.28 -
F. Rule of Law (0-16) 1.26 - 1.26 -
4B. Regulatory Trade Barriers (0–10) 1.22 - 1.22 -
2C. Protection of Property Rights (0–10) 1.22 - 1.22 -
D. Freedom of Expression & Belief (0–16) 1.21 - 1.21 -
5D. Freedom to Enter Markets and Compete (0–10) 1.13 - 1.13 -
Business Freedom (0–100) 1.11 - 1.11 -
5C(ii). Bureaucracy Costs (0–10) 1.10 - 1.10 -
Area 3. Sound Money (0–10) 1.08 - 1.08 -
5A(ii). Private Sector Credit (0–10) 1.06 - 1.06 -
2B. Impartial Courts (0–10) 1.06 - 1.06 -
C. Functioning of Government (0–12) 1.06 - 1.06 -
Social Progress Index – Opportunity (0–100) 1.02 - 1.02 -
5D(ii). Business Permits (0–10) 1.01 - 1.01 -
Official Exchange Rate (local currency per USD, period average) 1.00 - 1.00 -
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T6 (%)
======================================================================================================================================================
Label Average Lasso (75/25, 80/20) Ridge (75/25, 80/20) Elastic Net (L1=0.5) (80/20)
Inflation (CPI, annual %) 53.30 100.0 0.22 59.67
GDP Deflator (annual % change) 13.56 0.0 0.35 40.33
4D(iii). Freedom of Foreigners to Visit (0–10) 9.85 0.0 29.56 0.00
Real GDP Growth (annual %, constant prices) 1.60 0.0 4.80 0.00
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T7 (%)
======================================================================================================================================================
Label Average Lasso (75/25, 80/20) Ridge (75/25, 80/20) Elastic Net (L1=0.5) (75/25, 80/20)
Inflation (CPI, annual %) 52.91 100.0 0.25 58.49
GDP Deflator (annual % change) 13.97 0.0 0.39 41.51
4D(iii). Freedom of Foreigners to Visit (0–10) 5.13 0.0 15.40 0.00
Real GDP Growth (annual %, constant prices) 1.87 0.0 5.60 0.00
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T8 (%)
======================================================================================================================================================
Label Average Lasso (75/25, 80/20, 85/15) Ridge (80/20, 85/15) Elastic Net (L1=0.5) (75/25, 80/20, 85/15)
Inflation (CPI, annual %) 43.68 72.50 0.28 58.25
GDP Deflator (annual % change) 14.72 1.98 0.43 41.75
Real GDP Growth (annual %, constant prices) 6.53 12.96 6.62 0.00
Monetary Freedom (0–100) 3.59 8.39 2.37 0.00
4D(iii). Freedom of Foreigners to Visit (0–10) 3.56 0.00 10.69 0.00
Oil & Gas Output (% of GDP) 1.34 3.70 0.31 0.00
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T9 (%)
======================================================================================================================================================
Label Average Lasso (75/25, 80/20, 85/15) Ridge (85/15) Elastic Net (L1=0.5) (75/25, 80/20, 85/15)
Inflation (CPI, annual %) 43.14 73.66 0.22 55.54
GDP Deflator (annual % change) 15.89 2.86 0.35 44.46
4D(iii). Freedom of Foreigners to Visit (0–10) 10.52 0.00 31.57 0.00
Real GDP Growth (annual %, constant prices) 4.21 7.41 5.22 0.00
Monetary Freedom (0–100) 2.44 5.65 1.67 0.00
Oil & Gas Output (% of GDP) 2.15 6.22 0.24 0.00
2C. Protection of Property Rights (0–10) 1.68 3.62 1.43 0.00
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T10 (%)
======================================================================================================================================================
Label Average Lasso (75/25, 80/20, 85/15) Ridge (75/25, 85/15) Elastic Net (L1=0.5) (75/25, 80/20, 85/15)
Inflation (CPI, annual %) 51.31 100.0 0.23 53.71
GDP Deflator (annual % change) 15.56 0.0 0.39 46.29
4D(iii). Freedom of Foreigners to Visit (0–10) 6.05 0.0 18.14 0.00
Real GDP Growth (annual %, constant prices) 1.85 0.0 5.56 0.00
==============================================================================================================
AVERAGE SHAP IMPORTANCE BY HORIZON (%)
==============================================================================================================
Horizon Average 4 5 6 7 8 9 10
Label
Inflation (CPI, annual %) 36.04 7.33 0.59 53.30 52.91 43.68 43.14 51.31
4D(iii). Freedom of Foreigners to Visit (0–10) 11.98 22.92 25.80 9.85 5.13 3.56 10.52 6.05
GDP Deflator (annual % change) 10.62 0.10 0.56 13.56 13.97 14.72 15.89 15.56
Real GDP Growth (annual %, constant prices) 4.72 13.88 3.12 1.60 1.87 6.53 4.21 1.85
Monetary Freedom (0–100) 3.78 15.65 2.70 0.72 0.82 3.59 2.44 0.57
2C. Protection of Property Rights (0–10) 1.48 5.46 1.22 0.42 0.51 0.60 1.68 0.46
Current Account Balance (% of GDP) 1.33 6.99 0.69 0.25 0.28 0.38 0.33 0.40
Social Progress Index – Basic Human Needs (0–100) 0.98 1.78 2.44 0.57 0.65 0.77 0.39 0.28
Oil & Gas Output (% of GDP) 0.87 1.76 0.48 0.13 0.11 1.34 2.15 0.14
3B. Standard Deviation of Inflation (0–10) 0.77 2.33 1.38 0.34 0.37 0.37 0.30 0.30
1E. State Ownership of Assets (0–10) 0.76 0.26 1.50 0.66 0.78 0.78 0.62 0.72
4B(ii). Costs of Importing and Exporting (0–10) 0.64 0.38 1.28 0.49 0.71 0.73 0.35 0.53
G. Personal Autonomy & Individual Rights (0–16) 0.58 0.55 1.54 0.41 0.55 0.42 0.36 0.25
2B. Impartial Courts (0–10) 0.56 0.19 1.06 0.39 0.55 0.58 0.50 0.68
5C(ii). Bureaucracy Costs (0–10) 0.52 1.14 1.10 0.28 0.24 0.31 0.30 0.29
The SHAP decomposition across horizons (T4–T10) shows a clear transition in what markets price as key sovereign risk drivers. At medium horizons (T4–T5), Freedom of Foreigners to Visit and Monetary Freedom dominate, together accounting for 30–40% of total importance. These variables capture institutional openness and policy credibility — signalling that, a few years out, markets still reward countries that maintain macro discipline and institutional accessibility. The prominence of Real GDP Growth and Protection of Property Rights at these horizons further supports a forward‑looking but still policy‑anchored pricing logic.
Beyond five years, the structure shifts markedly toward monetary credibility channels. Inflation becomes the single most important driver, with SHAP contributions exceeding 50% in several horizons (T6–T7, T10), while the GDP Deflator emerges as a complementary indicator of general price‑level credibility. The persistence of these two variables suggests that what ultimately sustains creditworthiness in the long run is the capacity to preserve internal nominal stability.
Meanwhile, the earlier leaders — Freedom of Foreigners to Visit and Monetary Freedom — decline in importance as horizons lengthen, indicating that openness and formal monetary frameworks gradually give way to deeper, outcome‑based measures of credibility. In other words, the market’s pricing focus evolves from institutional access and policy reputation to sustained performance on inflation and monetary stability — a shift from structural signalling to proven long‑term consistency in macro management.
Temporal priced-in levels (structural persistence of fundamentals)¶
This section examines how the relationship between fundamentals and sovereign spreads evolves over time by regressing future spread levels (1 to 10 years into the future) on current macroeconomic and institutional variables.
A slow decay in explanatory power indicates that spreads remain anchored in long‑lived structural traits consistently priced by markets, while a faster decay or shifting variable mix reflects an adaptive pricing structure in which the weight of fundamentals changes over time.
# priced-in regressions across splits
from functions.summary import print_temporal_stats, get_prediction_stats, get_benchmark_stats
import pandas as pd
df = pd.read_pickle("saved/df.dat")
specs_dir = "specs"
base_name = "levels"
suffix = ""
summary_df_75 = get_prediction_stats(specs_dir, base_name, suffix, 0.75)
summary_df_80 = get_prediction_stats(specs_dir, base_name, suffix, 0.80)
summary_df_85 = get_prediction_stats(specs_dir, base_name, suffix, 0.85)
benchmark_df_75 = get_benchmark_stats(df, 0.75)
benchmark_df_80 = get_benchmark_stats(df, 0.80)
benchmark_df_85 = get_benchmark_stats(df, 0.85)
print("75-25 TRAIN-TEST SET SPLIT")
print("-"*125)
print_temporal_stats(summary_df_75, benchmark_df_75)
print("80-20 TRAIN-TEST SET SPLIT")
print("-"*125)
print_temporal_stats(summary_df_80, benchmark_df_80)
print("85-15 TRAIN-TEST SET SPLIT")
print("-"*125)
print_temporal_stats(summary_df_85, benchmark_df_85)
75-25 TRAIN-TEST SET SPLIT ----------------------------------------------------------------------------------------------------------------------------- Test‑set R² by Horizon Test‑set RMSE by Horizon ----------------------------------------------------------------------------------------------------------------------------- model elastic net L1=0.5 lasso ridge mean reversion model elastic net L1=0.5 lasso ridge mean reversion horizon horizon 1 0.498 0.535 0.511 0.907 1 2.969 2.857 2.932 1.273 2 0.266 0.382 0.373 0.794 2 3.589 3.295 3.318 1.886 3 0.290 0.225 0.274 0.762 3 3.395 3.547 3.434 1.944 4 0.349 0.345 0.312 0.723 4 3.251 3.260 3.341 2.106 5 0.406 0.367 0.366 0.620 5 3.106 3.206 3.207 2.463 6 0.362 0.313 0.377 0.559 6 3.217 3.339 3.179 2.684 7 0.335 0.315 0.373 0.531 7 3.284 3.335 3.190 2.764 8 0.362 0.304 <0 0.400 8 3.218 3.360 >100 3.124 9 0.336 0.317 <0 0.341 9 3.282 3.329 >100 3.265 10 0.394 0.312 0.368 0.373 10 3.137 3.340 3.202 3.184 ----------------------------------------------------------------------------------------------------------------------------- 80-20 TRAIN-TEST SET SPLIT ----------------------------------------------------------------------------------------------------------------------------- Test‑set R² by Horizon Test‑set RMSE by Horizon ----------------------------------------------------------------------------------------------------------------------------- model elastic net L1=0.5 lasso ridge mean reversion model elastic net L1=0.5 lasso ridge mean reversion horizon horizon 1 0.577 0.631 0.557 0.908 1 2.613 2.441 2.674 1.217 2 0.472 0.529 0.534 0.790 2 2.919 2.757 2.743 1.814 3 0.326 0.369 0.531 0.743 3 3.299 3.191 2.752 2.009 4 0.418 0.427 0.310 0.721 4 3.065 3.042 3.337 2.101 5 0.427 0.423 0.395 0.697 5 3.042 3.052 3.126 2.203 6 0.387 0.310 0.368 0.595 6 3.146 3.338 3.194 2.576 7 0.368 0.355 0.373 0.518 7 3.195 3.227 3.182 2.804 8 0.378 0.298 0.356 0.425 8 3.169 3.366 3.225 3.060 9 0.331 0.310 <0 0.335 9 3.285 3.337 >100 3.308 10 0.395 0.357 <0 0.351 10 3.126 3.222 >100 3.271 ----------------------------------------------------------------------------------------------------------------------------- 85-15 TRAIN-TEST SET SPLIT ----------------------------------------------------------------------------------------------------------------------------- Test‑set R² by Horizon Test‑set RMSE by Horizon ----------------------------------------------------------------------------------------------------------------------------- model elastic net L1=0.5 lasso ridge mean reversion model elastic net L1=0.5 lasso ridge mean reversion horizon horizon 1 0.605 0.641 0.575 0.894 1 2.561 2.443 2.658 1.320 2 0.492 0.546 0.546 0.818 2 2.906 2.746 2.748 1.706 3 0.379 0.416 0.548 0.831 3 3.213 3.115 2.741 1.646 4 0.473 0.492 0.550 0.756 4 2.960 2.904 2.736 1.985 5 0.539 0.517 0.574 0.697 5 2.768 2.833 2.660 2.227 6 0.483 0.362 0.385 0.749 6 2.932 3.257 3.197 2.062 7 0.528 0.524 0.409 0.733 7 2.799 2.814 3.134 2.117 8 0.491 0.483 0.359 0.481 8 2.908 2.932 3.264 2.953 9 0.419 0.400 0.475 0.368 9 3.108 3.157 2.954 3.276 10 0.414 0.382 0.456 0.416 10 3.121 3.204 3.006 3.174 -----------------------------------------------------------------------------------------------------------------------------
# compare with benchmark across splits
from functions.summary import get_prediction_stats, get_benchmark_stats, plot_temporal_stats
plot_temporal_stats(summary_df_75, benchmark_df_75, 0.75)
plot_temporal_stats(summary_df_80, benchmark_df_80, 0.8)
plot_temporal_stats(summary_df_85, benchmark_df_85, 0.85)
Across all train–test splits, the temporal priced-in regressions display moderate out-of-sample explanatory power, with R² values generally in the 0.30–0.55 range depending on horizon. Mean reversion dominates at very short horizons, where the persistence of fundamentals is strongest and deviations from trend tend to close mechanically; its R² near 0.90 for horizon 1 and RMSE well below the regression models illustrates how powerful pure persistence is in the immediate term. However, this advantage fades quickly as the horizon lengthens, and by horizons 4–10 the regularised models converge toward similar performance while mean reversion deteriorates steadily. The elastic-net, lasso, and ridge estimators produce broadly comparable R² trajectories, with only mild improvements as the training sample expands (75–25 → 85–15). This pattern suggests that the priced-in component of fundamentals carries some predictive structure, but only to a moderate degree: the models improve upon naïve forecasts at medium horizons yet do not fully overcome the dominant persistence dynamics, as reflected in the still-modest test-set R² levels.
# which variables have the strongest structural persistence?
from functions.summary import summarise_temporal_priced_in_vars
from functions.labels import all_labels
summarise_temporal_priced_in_vars(
specs_dir="specs",
base_name="levels",
models=["lasso", "ridge", "elastic"],
all_labels=all_labels
)
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T0 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
Monetary Freedom (0–100) 8.30 12.69 3.02 9.19
3B. Standard Deviation of Inflation (0–10) 5.03 7.44 1.75 5.91
2E. Integrity of the Legal System (0–10) 4.79 9.11 1.74 3.52
4B(ii). Costs of Importing and Exporting (0–10) 4.38 6.15 2.31 4.67
Current Account Balance (% of GDP) 4.29 5.47 2.57 4.83
1B. Transfers and Subsidies (0–10) 3.19 2.88 2.33 4.35
Real GDP Growth (annual %, constant prices) 2.87 1.69 3.51 3.41
2D. Military Interference in Rule of Law and Politics (0–10) 2.72 4.56 0.97 2.63
5A(ii). Private Sector Credit (0–10) 2.48 2.13 2.46 2.86
5C(iii). Impartial Public Administration (0–10) 2.02 2.38 1.22 2.47
2A. Judicial Independence (0–10) 1.90 1.77 1.90 2.02
4D(i). Financial Openness (0–10) 1.79 1.92 1.12 2.33
2B. Impartial Courts (0–10) 1.77 2.23 1.61 1.47
1E. State Ownership of Assets (0–10) 1.74 1.20 1.96 2.07
Gender Disparity Index (0–10, higher = more equality) 1.67 1.45 1.57 1.99
Inflation (CPI, annual %) 1.49 2.83 0.47 1.17
Area 4. Freedom to Trade Internationally (0–10) 1.40 1.24 1.33 1.63
A. Electoral Process (0–12) 1.39 1.94 0.80 1.44
5C(i). Regulatory Burden (0–10) 1.36 1.54 1.16 1.38
5A(i). Ownership of Banks (0–10) 1.36 1.57 0.86 1.66
GDP per capita (constant LCU) 1.28 1.17 1.00 1.66
1C. Government Investment (0–10) 1.20 1.25 1.04 1.32
Political Terror Scale – State Department (1–5, higher = more terror) 1.16 0.48 1.57 1.43
Mining Output (% of GDP) 1.11 1.02 0.86 1.45
5D(ii). Business Permits (0–10) 1.08 0.82 1.29 1.12
GDP Deflator (annual % change) 1.04 1.10 0.57 1.44
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 1.03 0.00 1.32 1.77
Fuel Exports (% of merchandise exports) 1.01 0.62 1.05 1.37
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T1 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
Real GDP Growth (annual %, constant prices) 7.00 3.96 8.50 8.53
Monetary Freedom (0–100) 4.40 6.84 2.25 4.11
1B. Transfers and Subsidies (0–10) 3.44 4.29 2.01 4.02
2E. Integrity of the Legal System (0–10) 3.31 5.05 2.43 2.46
Current Account Balance (% of GDP) 3.11 3.91 3.23 2.20
2A. Judicial Independence (0–10) 2.94 1.82 2.27 4.73
4B(ii). Costs of Importing and Exporting (0–10) 2.90 3.45 2.60 2.64
1E. State Ownership of Assets (0–10) 2.32 1.84 3.14 1.98
5A(ii). Private Sector Credit (0–10) 1.97 1.82 2.13 1.95
3B. Standard Deviation of Inflation (0–10) 1.94 2.85 0.96 2.00
Area 3. Sound Money (0–10) 1.88 2.94 0.94 1.75
5C(i). Regulatory Burden (0–10) 1.82 2.40 1.15 1.90
GDP Deflator (annual % change) 1.81 3.01 0.51 1.91
1C. Government Investment (0–10) 1.71 2.20 1.15 1.78
Social Progress Index – Basic Human Needs (0–100) 1.52 1.39 1.97 1.19
Area 4. Freedom to Trade Internationally (0–10) 1.51 1.87 1.43 1.24
2H. Reliability of Police and Crime (0–10) 1.46 1.95 0.45 1.99
Inflation (CPI, annual %) 1.44 2.89 0.14 1.29
Political Terror Scale – State Department (1–5, higher = more terror) 1.43 1.03 1.43 1.84
Gender Disparity Index (0–10, higher = more equality) 1.37 1.32 1.01 1.77
Central Government Debt (% of GDP) 1.31 1.34 0.92 1.68
4D(i). Financial Openness (0–10) 1.25 1.17 1.23 1.34
Regime Durability (years since last regime change) 1.19 1.82 0.47 1.28
Fuel Exports (% of merchandise exports) 1.18 1.68 0.98 0.89
5C(iii). Impartial Public Administration (0–10) 1.17 1.31 1.50 0.70
GDP per capita (constant LCU) 1.17 0.21 0.84 2.45
4D(iii). Freedom of Foreigners to Visit (0–10) 1.15 0.52 1.78 1.14
5D(ii). Business Permits (0–10) 1.10 1.14 0.79 1.37
3A. Money Growth (0–10) 1.06 1.31 1.33 0.54
5C(ii). Bureaucracy Costs (0–10) 1.04 1.10 0.92 1.09
Business Freedom (0–100) 1.03 1.05 0.78 1.26
3D. Freedom to Own Foreign Currency Bank Accounts (0–10) 1.02 1.55 0.38 1.13
4A(iii). Tariff Rate Dispersion (0–10) 1.00 0.95 1.56 0.48
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T2 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
Real GDP Growth (annual %, constant prices) 33.79 60.63 7.28 33.45
4D(iii). Freedom of Foreigners to Visit (0–10) 4.92 0.97 11.61 2.17
2A. Judicial Independence (0–10) 2.83 1.07 2.21 5.22
1E. State Ownership of Assets (0–10) 2.59 0.84 2.80 4.13
2E. Integrity of the Legal System (0–10) 2.56 2.29 2.35 3.05
1B. Transfers and Subsidies (0–10) 2.32 1.87 1.65 3.45
4B(ii). Costs of Importing and Exporting (0–10) 2.32 1.92 2.54 2.49
Monetary Freedom (0–100) 2.21 1.92 1.85 2.86
Current Account Balance (% of GDP) 1.91 1.22 2.51 2.00
GDP per capita (constant LCU) 1.53 0.95 0.83 2.82
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 1.46 0.41 1.94 2.04
4D(i). Financial Openness (0–10) 1.16 0.73 1.30 1.45
Social Progress Index – Basic Human Needs (0–100) 1.14 0.58 2.41 0.42
5A(ii). Private Sector Credit (0–10) 1.02 0.80 1.58 0.68
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T3 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
Real GDP Growth (annual %, constant prices) 29.98 54.56 4.86 30.52
1E. State Ownership of Assets (0–10) 6.02 6.89 2.84 8.33
4D(iii). Freedom of Foreigners to Visit (0–10) 5.09 1.31 11.25 2.70
2E. Integrity of the Legal System (0–10) 4.41 3.51 2.54 7.18
2A. Judicial Independence (0–10) 4.24 3.69 2.22 6.82
1B. Transfers and Subsidies (0–10) 3.44 3.11 1.76 5.46
4B(ii). Costs of Importing and Exporting (0–10) 2.63 1.74 2.34 3.82
Current Account Balance (% of GDP) 2.60 1.64 2.05 4.12
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 2.51 0.98 2.02 4.52
Monetary Freedom (0–100) 1.63 1.06 1.34 2.48
GDP per capita (constant LCU) 1.57 1.34 0.85 2.52
Social Progress Index – Basic Human Needs (0–100) 1.56 0.50 2.79 1.39
4D(i). Financial Openness (0–10) 1.38 1.26 1.26 1.62
2B. Impartial Courts (0–10) 1.27 0.81 1.85 1.16
Area 1. Size of Government (0–10) 1.17 0.53 1.02 1.97
Regime Durability (years since last regime change) 1.15 1.06 0.52 1.86
3A. Money Growth (0–10) 1.06 0.60 1.25 1.34
3B. Standard Deviation of Inflation (0–10) 1.01 0.96 0.73 1.35
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T4 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
1E. State Ownership of Assets (0–10) 16.09 31.29 2.08 14.89
2A. Judicial Independence (0–10) 8.68 12.67 2.37 11.01
4D(iii). Freedom of Foreigners to Visit (0–10) 7.32 1.10 18.99 1.87
2E. Integrity of the Legal System (0–10) 6.84 8.92 2.31 9.29
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 5.46 6.94 2.13 7.30
Real GDP Growth (annual %, constant prices) 3.81 4.45 0.98 6.01
Current Account Balance (% of GDP) 2.71 2.44 1.66 4.02
2B. Impartial Courts (0–10) 2.51 2.38 1.87 3.27
1B. Transfers and Subsidies (0–10) 2.20 1.99 1.66 2.94
Social Progress Index – Basic Human Needs (0–100) 1.96 1.48 2.64 1.76
4B(ii). Costs of Importing and Exporting (0–10) 1.87 1.06 2.08 2.47
Mining Output (% of GDP) 1.72 2.24 0.64 2.28
Monetary Freedom (0–100) 1.71 2.45 1.06 1.63
Voice & Accountability – Estimate (−2.5 to +2.5) 1.66 2.83 0.67 1.47
Trade Openness (% of GDP, goods and services) 1.43 1.33 0.51 2.45
5B(iii). Flexible Wage Determination (0–10) 1.42 1.25 0.85 2.17
4D(i). Financial Openness (0–10) 1.39 1.11 1.28 1.79
3A. Money Growth (0–10) 1.39 1.09 1.10 1.98
1C. Government Investment (0–10) 1.30 1.22 0.48 2.20
Area 1. Size of Government (0–10) 1.22 0.42 0.77 2.47
5A(i). Ownership of Banks (0–10) 1.09 1.00 0.43 1.83
Gender Disparity Index (0–10, higher = more equality) 1.03 1.03 0.39 1.66
1D(ii). Top Marginal Income + Payroll Tax Rate (0–10) 1.02 0.70 1.30 1.05
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T5 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
2A. Judicial Independence (0–10) 13.39 21.51 2.35 16.31
2E. Integrity of the Legal System (0–10) 9.68 14.76 2.70 11.59
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 8.36 12.02 2.21 10.85
4B(ii). Costs of Importing and Exporting (0–10) 4.38 5.81 2.49 4.84
1E. State Ownership of Assets (0–10) 4.36 6.22 1.77 5.10
Mining Output (% of GDP) 3.62 4.72 1.08 5.07
4D(iii). Freedom of Foreigners to Visit (0–10) 3.56 0.45 9.71 0.52
Current Account Balance (% of GDP) 3.22 3.29 2.29 4.09
1D(ii). Top Marginal Income + Payroll Tax Rate (0–10) 3.12 2.58 1.89 4.90
5B(iii). Flexible Wage Determination (0–10) 2.83 2.98 1.40 4.12
Social Progress Index – Basic Human Needs (0–100) 2.55 2.81 2.97 1.88
Gender Disparity Index (0–10, higher = more equality) 2.16 2.28 0.53 3.66
1B. Transfers and Subsidies (0–10) 1.92 1.80 1.79 2.16
4D(i). Financial Openness (0–10) 1.55 1.09 1.57 2.00
3A. Money Growth (0–10) 1.50 1.49 1.32 1.69
5A(i). Ownership of Banks (0–10) 1.35 1.03 0.71 2.30
Real GDP Growth (annual %, constant prices) 1.34 1.19 0.69 2.15
1C. Government Investment (0–10) 1.33 1.44 0.58 1.96
5B(v). Cost of Worker Dismissal (0–10) 1.32 1.54 0.99 1.43
Voice & Accountability – Estimate (−2.5 to +2.5) 1.27 1.87 0.67 1.28
Trade Openness (% of GDP, goods and services) 1.12 1.34 0.59 1.42
4C. Black Market Exchange Rates (0–10) 1.07 1.56 0.28 1.37
5C(iii). Impartial Public Administration (0–10) 1.03 0.34 1.83 0.92
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T6 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
2A. Judicial Independence (0–10) 20.82 37.04 2.66 22.76
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 14.11 25.46 2.38 14.48
2E. Integrity of the Legal System (0–10) 14.09 24.27 2.79 15.20
4D(iii). Freedom of Foreigners to Visit (0–10) 5.85 0.00 17.54 0.00
Mining Output (% of GDP) 4.71 5.90 0.92 7.30
Current Account Balance (% of GDP) 2.57 0.25 1.52 5.93
1D(ii). Top Marginal Income + Payroll Tax Rate (0–10) 2.47 0.00 1.52 5.89
3A. Money Growth (0–10) 2.37 3.33 1.09 2.70
Social Progress Index – Basic Human Needs (0–100) 2.35 1.25 2.25 3.56
5B(iii). Flexible Wage Determination (0–10) 2.24 0.00 0.74 5.98
4B(ii). Costs of Importing and Exporting (0–10) 1.36 0.00 1.72 2.37
4C. Black Market Exchange Rates (0–10) 1.34 2.20 0.23 1.60
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T7 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
2A. Judicial Independence (0–10) 14.95 23.51 2.22 19.12
2E. Integrity of the Legal System (0–10) 10.41 12.89 3.35 15.00
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 9.84 14.32 2.55 12.65
4D(iii). Freedom of Foreigners to Visit (0–10) 4.27 0.81 10.96 1.04
Mining Output (% of GDP) 4.08 5.49 1.30 5.44
5B(iii). Flexible Wage Determination (0–10) 3.17 4.19 1.23 4.08
1D(ii). Top Marginal Income + Payroll Tax Rate (0–10) 3.11 3.49 1.80 4.04
Social Progress Index – Basic Human Needs (0–100) 2.96 3.06 2.66 3.16
Current Account Balance (% of GDP) 2.27 2.18 1.83 2.81
3A. Money Growth (0–10) 2.02 2.17 1.23 2.66
5B(v). Cost of Worker Dismissal (0–10) 1.85 2.27 0.94 2.34
Voice & Accountability – Estimate (−2.5 to +2.5) 1.53 2.48 0.49 1.63
Political Terror Scale – State Department (1–5, higher = more terror) 1.40 1.71 0.86 1.63
1E. State Ownership of Assets (0–10) 1.35 0.72 1.53 1.81
4D(i). Financial Openness (0–10) 1.32 1.15 1.53 1.29
Trade Openness (% of GDP, goods and services) 1.15 1.52 0.44 1.50
Gender Disparity Index (0–10, higher = more equality) 1.14 1.51 0.18 1.74
5C(ii). Bureaucracy Costs (0–10) 1.06 0.96 0.80 1.42
Area 2. Legal System & Property Rights (gender-adjusted, 0–10) 1.02 0.94 2.08 0.03
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T8 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
2A. Judicial Independence (0–10) 17.28 30.81 2.14 18.89
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 12.21 22.79 1.93 11.91
2E. Integrity of the Legal System (0–10) 11.39 16.84 2.99 14.34
Mining Output (% of GDP) 4.28 5.13 1.17 6.55
5B(iii). Flexible Wage Determination (0–10) 4.05 2.83 0.82 8.51
4D(iii). Freedom of Foreigners to Visit (0–10) 3.37 0.00 9.84 0.28
1D(ii). Top Marginal Income + Payroll Tax Rate (0–10) 2.85 1.67 1.53 5.36
3A. Money Growth (0–10) 2.56 3.92 1.05 2.72
Current Account Balance (% of GDP) 2.20 1.06 1.57 3.97
Social Progress Index – Basic Human Needs (0–100) 1.60 2.11 1.86 0.82
5B(v). Cost of Worker Dismissal (0–10) 1.29 1.44 1.02 1.40
Fuel Exports (% of merchandise exports) 1.02 0.47 0.56 2.04
2G. Regulation of Real Property (0–10) 1.00 0.54 0.35 2.12
Political Terror Scale – State Department (1–5, higher = more terror) 1.00 1.27 0.92 0.80
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T9 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
2A. Judicial Independence (0–10) 21.53 31.76 3.07 29.77
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 12.46 23.57 2.20 11.62
2E. Integrity of the Legal System (0–10) 10.68 12.42 2.80 16.81
4D(iii). Freedom of Foreigners to Visit (0–10) 5.70 0.00 17.09 0.00
Mining Output (% of GDP) 4.72 5.97 1.66 6.52
5B(iii). Flexible Wage Determination (0–10) 4.19 4.36 1.92 6.28
3A. Money Growth (0–10) 2.25 2.76 1.05 2.94
1D(ii). Top Marginal Income + Payroll Tax Rate (0–10) 2.07 1.15 1.76 3.31
Area 2. Legal System & Property Rights (gender-adjusted, 0–10) 1.61 2.19 1.66 0.99
Fuel Exports (% of merchandise exports) 1.57 1.57 0.79 2.35
2B. Impartial Courts (0–10) 1.42 0.00 1.73 2.53
5B(v). Cost of Worker Dismissal (0–10) 1.39 1.43 1.17 1.58
4D(i). Financial Openness (0–10) 1.22 1.09 1.44 1.14
Current Account Balance (% of GDP) 1.15 0.59 2.02 0.84
======================================================================================================================================================
AVERAGE SHAP IMPORTANCE BY MODEL FOR T10 (%)
======================================================================================================================================================
Label Average Lasso Ridge Elastic Net (L1=0.5)
2A. Judicial Independence (0–10) 17.31 28.10 2.84 20.99
5B(iii). Flexible Wage Determination (0–10) 7.31 11.39 2.14 8.39
4D(iii). Freedom of Foreigners to Visit (0–10) 5.55 0.00 16.64 0.00
Mining Output (% of GDP) 4.40 6.33 1.62 5.26
1D(ii). Top Marginal Income + Payroll Tax Rate (0–10) 3.62 4.54 1.46 4.87
5B(v). Cost of Worker Dismissal (0–10) 2.80 5.80 1.16 1.43
Current Account Balance (% of GDP) 2.76 3.11 2.21 2.95
4D(i). Financial Openness (0–10) 2.12 3.42 1.29 1.64
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 2.03 0.00 2.03 4.05
GDP per capita (current USD) 1.92 2.54 1.14 2.07
Fuel Exports (% of merchandise exports) 1.92 2.04 0.86 2.85
Trade Openness (% of GDP, goods and services) 1.66 3.25 0.45 1.28
3A. Money Growth (0–10) 1.63 1.74 0.99 2.17
5B(vi). Conscription (0–10) 1.44 3.61 0.50 0.21
5B(ii). Hiring and Firing Regulations (0–10) 1.38 2.87 0.99 0.27
2E. Integrity of the Legal System (0–10) 1.37 0.00 2.00 2.12
1B. Transfers and Subsidies (0–10) 1.24 1.01 1.47 1.23
Gender Disparity Index (0–10, higher = more equality) 1.10 2.71 0.34 0.24
Political Terror Scale – State Department (1–5, higher = more terror) 1.02 2.40 0.66 0.00
==============================================================================================================
AVERAGE SHAP IMPORTANCE BY HORIZON (%)
==============================================================================================================
Horizon Average 0 1 2 3 4 5 6 7 8 9 10
Label
2A. Judicial Independence (0–10) 11.44 1.90 2.94 2.83 4.24 8.68 13.39 20.82 14.95 17.28 21.53 17.31
Real GDP Growth (annual %, constant prices) 7.40 2.87 7.00 33.79 29.98 3.81 1.34 0.92 0.82 0.46 0.29 0.11
2E. Integrity of the Legal System (0–10) 7.23 4.79 3.31 2.56 4.41 6.84 9.68 14.09 10.41 11.39 10.68 1.37
Area 2. Legal System & Property Rights (no gender adjustment, 0–10) 6.39 1.03 0.83 1.46 2.51 5.46 8.36 14.11 9.84 12.21 12.46 2.03
4D(iii). Freedom of Foreigners to Visit (0–10) 4.33 0.84 1.15 4.92 5.09 7.32 3.56 5.85 4.27 3.37 5.70 5.55
1E. State Ownership of Assets (0–10) 3.40 1.74 2.32 2.59 6.02 16.09 4.36 0.87 1.35 0.84 0.52 0.73
Mining Output (% of GDP) 2.82 1.11 0.99 0.61 0.80 1.72 3.62 4.71 4.08 4.28 4.72 4.40
Current Account Balance (% of GDP) 2.62 4.29 3.11 1.91 2.60 2.71 3.22 2.57 2.27 2.20 1.15 2.76
5B(iii). Flexible Wage Determination (0–10) 2.54 0.78 0.97 0.55 0.45 1.42 2.83 2.24 3.17 4.05 4.19 7.31
4B(ii). Costs of Importing and Exporting (0–10) 2.02 4.38 2.90 2.32 2.63 1.87 4.38 1.36 0.74 0.75 0.43 0.47
1B. Transfers and Subsidies (0–10) 1.87 3.19 3.44 2.32 3.44 2.20 1.92 0.51 0.91 0.62 0.77 1.24
1D(ii). Top Marginal Income + Payroll Tax Rate (0–10) 1.84 0.41 0.51 0.61 0.49 1.02 3.12 2.47 3.11 2.85 2.07 3.62
Monetary Freedom (0–100) 1.83 8.30 4.40 2.21 1.63 1.71 0.33 0.15 0.30 0.26 0.27 0.52
3A. Money Growth (0–10) 1.57 0.87 1.06 0.61 1.06 1.39 1.50 2.37 2.02 2.56 2.25 1.63
Social Progress Index – Basic Human Needs (0–100) 1.56 0.60 1.52 1.14 1.56 1.96 2.55 2.35 2.96 1.60 0.61 0.35
4D(i). Financial Openness (0–10) 1.36 1.79 1.25 1.16 1.38 1.39 1.55 0.86 1.32 0.96 1.22 2.12
5B(v). Cost of Worker Dismissal (0–10) 1.14 0.74 0.94 0.85 0.41 0.74 1.32 0.25 1.85 1.29 1.39 2.80
Gender Disparity Index (0–10, higher = more equality) 1.06 1.67 1.37 0.70 0.54 1.03 2.16 0.65 1.14 0.87 0.39 1.10
3B. Standard Deviation of Inflation (0–10) 1.05 5.03 1.94 0.94 1.01 0.98 0.58 0.08 0.41 0.31 0.19 0.13
2B. Impartial Courts (0–10) 1.01 1.77 0.54 0.63 1.27 2.51 0.65 0.55 0.67 0.58 1.42 0.50
Trade Openness (% of GDP, goods and services) 0.91 0.47 0.90 0.34 0.41 1.43 1.12 0.95 1.15 0.93 0.70 1.66
Fuel Exports (% of merchandise exports) 0.84 1.01 1.18 0.67 0.44 0.21 0.29 0.32 0.60 1.02 1.57 1.92
5C(ii). Bureaucracy Costs (0–10) 0.82 0.92 1.04 0.96 0.95 0.82 0.99 0.36 1.06 0.52 0.77 0.60
1C. Government Investment (0–10) 0.82 1.20 1.71 0.76 0.86 1.30 1.33 0.32 0.72 0.33 0.30 0.17
GDP per capita (constant LCU) 0.80 1.28 1.17 1.53 1.57 0.99 0.37 0.25 0.29 0.31 0.28 0.75
5C(iii). Impartial Public Administration (0–10) 0.80 2.02 1.17 0.80 0.48 0.55 1.03 0.60 0.57 0.56 0.49 0.56
Political Terror Scale – State Department (1–5, higher = more terror) 0.79 1.16 1.43 0.49 0.28 0.35 0.45 0.32 1.40 1.00 0.82 1.02
5A(ii). Private Sector Credit (0–10) 0.76 2.48 1.97 1.02 0.61 0.41 0.41 0.19 0.28 0.35 0.26 0.39
Voice & Accountability – Estimate (−2.5 to +2.5) 0.76 0.09 0.35 0.60 0.65 1.66 1.27 0.15 1.53 0.99 0.37 0.74
Area 2. Legal System & Property Rights (gender-adjusted, 0–10) 0.72 0.23 0.79 0.73 0.61 0.55 0.56 0.67 1.02 0.66 1.61 0.52
5A(i). Ownership of Banks (0–10) 0.70 1.36 0.83 0.36 0.44 1.09 1.35 0.25 0.63 0.23 0.37 0.75
5C(i). Regulatory Burden (0–10) 0.69 1.36 1.82 0.95 0.40 0.25 0.30 0.22 0.95 0.47 0.60 0.25
Area 3. Sound Money (0–10) 0.64 0.94 1.88 0.91 0.57 0.94 0.38 0.33 0.30 0.32 0.26 0.22
Area 4. Freedom to Trade Internationally (0–10) 0.63 1.40 1.51 0.94 0.48 0.49 0.40 0.42 0.43 0.40 0.26 0.23
2D. Military Interference in Rule of Law and Politics (0–10) 0.62 2.72 0.60 0.36 0.33 0.34 0.37 0.46 0.51 0.52 0.32 0.26
4C. Black Market Exchange Rates (0–10) 0.57 0.48 0.59 0.33 0.59 0.57 1.07 1.34 0.68 0.33 0.20 0.06
Social Progress Index – Opportunity (0–100) 0.54 0.61 0.52 0.64 0.52 0.36 0.44 0.38 0.64 0.70 0.65 0.46
2H. Reliability of Police and Crime (0–10) 0.53 0.96 1.46 0.32 0.21 0.18 0.24 0.29 0.45 0.50 0.43 0.84
5C. Business Regulations (0–10) 0.51 0.26 0.36 0.40 0.38 0.59 0.64 0.47 0.82 0.74 0.49 0.41
The temporal priced-in regressions show that the factors explaining sovereign spread levels shift as the horizon extends. At T0–T1, the explanatory pattern is relatively broad: Monetary Freedom, the Cost of Importing and Exporting, and the Integrity of the Legal System account for most of the variation in current pricing levels, alongside contributions from inflation volatility and the current account balance. This indicates that today’s spreads reflect a mixture of macro conditions, trade frictions, and institutional features, with no single institutional family yet dominating.
From roughly T2 onward, the structure becomes more focused, and legal-system variables steadily take over. Judicial Independence, Integrity of the Legal System, and the broader Legal System & Property Rights index rise sharply in importance and remain consistently influential through T10. By mid-to-long horizons (T5–T10), these legal variables explain a large share of the cross-sectional variation, while macro variables and most trade-related measures fade. A few structural features (such as mining output or selected labour-market regulations) persist at the margins, but the overall pattern suggests legal-system quality becoming the main long-run determinant of sovereign borrowing costs.
The findings must however be framed in the context of the modest (and fading) explanatory power of the model, which usually accounts for roughly 35-55% of the variation after T2, depending on the horizon and train-test split.
# plot performance across splits
import pandas as pd
df = pd.read_pickle("saved/df.dat")
from functions.summary import plot_model_splits
plot_model_splits("lasso_levels", "Lasso priced-in level regressions")
plot_model_splits("ridge_levels", "Ridge priced-in level regressions")
plot_model_splits("elastic_levels", "Elastic L1=0.5 priced-in level regressions")
# export notebook to HTML
import os, re, subprocess, shutil, time
time.sleep(60)
notebook = "05_summary.ipynb"
html_name = "05_summary.html"
docs_dir = "docs"
title_text = "Project summary"
subprocess.run(
["jupyter", "nbconvert", "--to", "html", notebook],
check=True
)
with open(html_name, "r", encoding="utf-8") as f:
html = f.read()
html = re.sub(r"<title>.*?</title>", f"<title>{title_text}</title>", html, flags=re.DOTALL)
html = html.replace("<body>", f"<body>\n<h1>{title_text}</h1>\n<hr>\n")
with open(html_name, "w", encoding="utf-8") as f:
f.write(html)
os.makedirs(docs_dir, exist_ok=True)
shutil.move(html_name, os.path.join(docs_dir, html_name))
print(f"Exported and updated {os.path.join(docs_dir, html_name)}")