Playbook: Fair-Lending (Reg B / ECOA) Model Validation Using Synthetic Households

Scoping: which models are in fair-lending scope

Reg B covers credit transactions — extension, modification, and termination. Models that influence any credit decision are in scope: underwriting, pricing, line-assignment, account closure, collections strategy. Marketing-targeting models that pre-screen for credit offers are also in scope per recent guidance.

Not in scope: models that only target investment products without credit components. The line gets fuzzy for 'wealth-tech' products that include lending features (Pledged-asset lines, securities-backed loans). If lending is involved, treat the model as in scope.

Test population: the synthetic corpus structure

Validation requires a synthetic population with documented protected-class distributions. The corpus should reflect the population the model will encounter in production with intentional representation of protected classes (race / ethnicity, sex, age, marital status, national origin, religion).

Protected-class distributions for the corpus must be documented and justified. Two patterns work: (1) match ACS distributions for the geographic footprint of your business; (2) match an over-sampled distribution where each protected class has at least 1,000 households for statistical power. Pattern 2 is preferred for validation because the analysis has more power on the smaller classes.

Disparate-impact testing

Run the model against the synthetic corpus and measure outcome rates by protected class. The standard test is the four-fifths rule: the rate of favorable outcomes for any protected class should be at least 80% of the rate for the most-favored class. Below 80% is a presumption of disparate impact requiring justification.

Go beyond the simple four-fifths test by measuring: (1) approval-rate disparities, (2) pricing disparities (APR or fee structure given approval), (3) line-amount disparities given approval, (4) the error-rate of the model itself across classes (over-rejection on one class is itself an issue even if the approval rate hits four-fifths).

Where disparities exist, run the less-discriminatory-alternative analysis. Per Reg B and recent CFPB guidance, the existence of a less-discriminatory alternative that achieves comparable business goal is a defect. The analysis is structurally: try alternative models, measure both their disparate-impact and their business-objective performance, and identify any that strictly dominate the proposed model.

// Disparate-impact analysis sketch
const classes = ['white', 'black', 'hispanic', 'asian', '...'];
const outcomes = {};

for (const cls of classes) {
  const subCorpus = corpus.filter(
    h => h.demographics.race_ethnicity === cls
  );
  const decisions = subCorpus.map(h => model.decide(h));
  outcomes[cls] = {
    approval_rate: rate(decisions, 'approved'),
    avg_apr: avg(decisions.filter(d => d.approved).map(d => d.apr)),
    avg_line: avg(decisions.filter(d => d.approved).map(d => d.line))
  };
}

// Four-fifths rule
const max_rate = Math.max(...Object.values(outcomes).map(o => o.approval_rate));
for (const [cls, o] of Object.entries(outcomes)) {
  if (o.approval_rate / max_rate < 0.8) {
    flag('disparate_impact_approval', cls, o);
  }
}

Adverse-action explanation testing

Per ECOA and the 2022-2023 CFPB circulars, every adverse-action notice must list specific reasons that are accurate, consistent across customers with similar circumstances, and derived from the model's actual decision rationale.

For models with complex feature interactions or ML-derived non-linearities, this is hard. The validation must show that the adverse-action explanations: (1) reflect the actual model features driving the rejection, not a post-hoc generic list; (2) are consistent — same household, run the model twice, get the same explanation; (3) are intelligible — a customer reading the explanation can understand what would change the outcome.

Test by running the model against synthetic rejected households, generating the explanation per your production logic, and verifying each explanation against the three criteria. Reject any that fails — explanation generation is fixable, but only if discovered before production.

Steering & matched-pair analysis

Steering — channeling protected-class members to less-favorable products — is a fair-lending concern even when ultimate outcome rates are equivalent. Test by matched-pair analysis: synthetic households that match on income, credit profile, and other factors but differ on protected class. The model should produce indistinguishable outcomes for the matched pair.

The synthetic corpus is well-suited for this because matched pairs are constructible by holding all attributes constant and varying only the protected-class field. With real-data testing this is impossible (no two real customers match exactly); with synthetic data it's straightforward.

Documentation: the artifacts examiners request

Documentation is half the validation. The artifacts to produce include: (1) the validation methodology — the corpus structure, the test plan, the thresholds; (2) the validation results — disparate-impact tables, adverse-action explanation review, matched-pair findings; (3) the remediation log — for any disparities discovered, what was changed and what the post-change retest showed; (4) the ongoing monitoring plan — what's measured in production and how thresholds trigger reassessment.

This package goes to the model risk committee or equivalent governance. A subset goes to the regulator on examination. Recent OCC and CFPB exams have asked for this package as a default request.

Ongoing monitoring

Pre-deployment validation isn't sufficient. Per OCC bulletin on AI-in-credit, ongoing monitoring with documented thresholds is expected. The monitoring should track: (a) approval-rate disparities by protected class in production; (b) pricing-disparity by protected class; (c) adverse-action explanation accuracy by sampling; (d) input-data drift that might invalidate validation.

When monitoring trips a threshold, the response procedure should be documented: model retraining, model retirement, or compensating controls. Documented response prevents the monitoring from being theater.

Key takeaways

• Fair-lending validation has materially expanded post-2023 — disparate-impact, adverse-action explanations, less-discriminatory alternative, and ongoing monitoring are all expected.
• Synthetic corpora structured for protected-class testing make matched-pair analysis tractable in a way real-data testing can't be — same attributes, varying only protected class.
• Adverse-action explanation testing is the test ML-driven credit models often fail. Generic post-hoc explanations don't satisfy ECOA / CFPB; explanations must reflect actual feature drivers.
• Documentation is half the validation. The full package — methodology, results, remediation, monitoring — goes to model risk governance and to regulators on examination.

FAQ