Generating Reg BI Test Data: A Methodology for Compliance Teams
SEC examinations of Regulation Best Interest don't audit your policies — they audit individual recommendations made to individual clients. The supervisory system that catches the recommendation that should never have been made gets exercised by the cases on the edge of suitability, not by the modal client. That makes test-data design the single most under-invested part of most broker-dealer compliance programs. This guide walks through a defensible methodology for constructing a Reg BI test corpus, the specific scenarios it must exercise, and how synthetic data closes the gap that anonymized production data leaves.
Why anonymized production data fails for Reg BI testing
The first instinct of most compliance teams is to anonymize production data. The data is real, the distribution matches the firm's actual book, and the legal posture is straightforward.
The problem is exactly that the distribution matches the existing book. Reg BI examiners cite specific fact patterns — a 78-year-old with 60% concentration in a single equity, a recent inheritance recipient steered into illiquid alternatives, a client with deteriorating cognitive markers whose POA hasn't been updated. In any individual firm's production book of business, these cases exist in single digits. Some firms have zero. Sampling them out of an anonymized snapshot gives you a test set with no statistical power to exercise the supervisory system on the cases that actually drive enforcement.
The second problem is that production data inherits any supervisory failures already present in the book. If your supervisory system has been quietly missing a class of red flag for two years, the production data won't surface that miss in testing — the misses are already in the book.
The five Reg BI scenarios any test corpus must exercise
Build the corpus to exercise the following five fact patterns at calibrated frequencies. Each maps directly to enforcement-action and FINRA-notice fact patterns from the past three years.
- Concentrated single-equity holdings in clients age 75+ — the canonical senior suitability red flag
- Risk-tolerance mismatches where account allocation exceeds the client's documented risk score
- Cognitive-decline markers paired with outdated or missing POA documentation
- Recent-inheritance events (within 18 months) followed by recommendation of illiquid or alternative products
- Repeat unsuitable recommendations across the supervisory review threshold (multiple flagged transactions in a 12-month window)
Calibration: why 130 households is the right number
The instinct is to scale the test corpus to match production volume — 1,000+ households if the firm has 1,000+ clients. Resist this instinct.
For Reg BI testing, 130 households is sufficient if the population is calibrated to the relevant fact patterns. The supervisory system's logic is exercised by structural diversity, not by volume. A 130-household corpus deliberately weighted toward concentrated-senior cases (60% age 70+, with 25% age 80+, with concentrated holdings in roughly half) exercises the suitability-engine logic more thoroughly than 5,000 modal clients ever could.
A corpus of this size also fits the practical realities of compliance review. A senior compliance officer can walk an examiner through 130 households in a meaningful way; 5,000 households turn the demonstration into a query exercise that loses the auditor.
Documentation: the audit trail you need at examination
The examiner will ask three questions about your test corpus: where did the data come from, what does it exercise, and how often is it refreshed? Each answer must be documentable.
For source: synthetic data with archetype-driven generation provides a clean answer (no real individuals; structurally calibrated). For coverage: a corpus where each archetype maps to specific Reg BI fact patterns lets you point at a coverage matrix rather than a list of test cases. For refresh: an annual cycle aligned with FINRA interpretive-guidance updates and recent enforcement actions demonstrates the test corpus is a living artifact, not a one-time fixture.
The Methodology PDF that accompanies the WealthSchema Reg BI Suitability Audit Pack is structured exactly to answer these three questions — the field-by-field derivation, the archetype-to-scenario coverage matrix, and the annual refresh methodology are all documented for examination use.
Integration with the supervisory engine
The corpus is not the end of the work — it's the input. The supervisory engine has to actually consume it. Most engines accept structured fields with names compatible with the major broker-dealer data models. The WealthSchema schema deliberately uses field names that map cleanly to the four most common platforms; the Methodology PDF includes a field-mapping appendix.
// Pseudocode: filter the corpus for the canonical Reg BI red flag const flaggedCases = households.filter(h => h.members.some(m => m.age >= 75) && h.assets.concentration_pct > 0.25 && h.compliance.suitability_flags.length === 0 // Caught nothing yet ); // Each case in flaggedCases is a supervisory-system failure // surfaced by the test corpus. Investigate why your engine // didn't already flag the recommendation.
Key takeaways
- Anonymized production data fails Reg BI testing because the modal client doesn't exercise the suitability engine — the edge cases do.
- A 130-household corpus calibrated to the five canonical Reg BI fact patterns exercises supervisory logic more thoroughly than thousands of typical clients.
- Document three things for the examiner: data source (synthetic, no PII), coverage (archetype-to-scenario matrix), and refresh cadence (annual, tracked to interpretive guidance).
- Integration matters as much as the corpus itself — field-name compatibility with the firm's supervisory engine determines whether the test work happens or stalls in data prep.
FAQ
Will the SEC accept synthetic data for compliance demonstration?+
Synthetic data has been used for compliance demonstration purposes for years. The relevant question is whether the data is appropriate to the test — not whether the data is 'real'. For Reg BI scenarios where real-client edge cases are statistically rare, synthetic data is often more appropriate than anonymized production data because it can be calibrated to exercise the specific patterns examiners cite.
How often should the test corpus be refreshed?+
Annually is the right cadence for most firms. The refresh should track FINRA interpretive guidance updates from the prior 12 months and any new enforcement-action fact patterns. Major regulatory changes (a new SEC release, a substantial FINRA rule change) may warrant an off-cycle refresh.
Should I build the corpus in-house or buy?+
Building in-house is straightforward in concept and brutal in execution — the calibration work alone takes a senior compliance analyst 200+ hours, and maintaining the corpus through interpretive-guidance updates takes another 50+ hours per year. For most firms, buying a ready-made corpus and customising the test scenarios on top is the more economical path. The tradeoff is the customisation depth: a turnkey corpus serves 80% of test cases; a bespoke corpus serves 100%.
How does this corpus interact with the firm's actual supervisory data?+
The corpus runs through the supervisory engine the same way real data does — same fields, same routing logic, same review queue. The result is a test environment where the supervisory engineer or compliance officer can run end-to-end scenarios without exposure to real client data. After validation, the engine is then applied to real production data with confidence that the supervisory rules behave as intended.
What about FINRA Rule 2090 KYC and FINRA Notice 21-09 senior-client guidance?+
The corpus is calibrated against both. FINRA Rule 2090 KYC fields are present on every household. FINRA Notice 21-09 senior-client guidance — particularly the trust-relationship and POA-update factors — drives the calibration of the senior-cohort populations. The Methodology PDF documents the alignment.
Can I share the corpus with examiners during a review?+
Yes. The Data License explicitly permits use in examiner demonstrations and audit walk-throughs. Sharing the corpus directly with an examiner is permitted; redistributing the corpus to other firms or third parties is not.
Is the data sufficient for Reg BI Disclosure Obligation testing?+
The corpus focuses on the Care Obligation. For Disclosure Obligation testing — Form CRS, Form ADV, conflict disclosure — additional structures (the firm's actual disclosures, the client-acknowledgment timestamps, the supervised-person identifiers) are needed alongside the corpus. The corpus serves as the prospect side of the disclosure interaction; the firm-side disclosure data has to come from the firm's own systems.