A product thought exercise · Nathan Rozendaal
What can complaint patterns tell us?
I explored 2,000 public CFPB complaints to see how the details in a story relate to the outcome a company reports. Get It Back is a prototype for making those patterns easier to inspect.
The question I wanted to explore: could a model make a messy set of complaints useful, while showing where its conclusions stop? The scenarios below are fictional. The percentages are exploratory model outputs, not personal forecasts.
Explore the prototype
Choose a fictional scenario, then vary the assumptions to see how the model responds.
Preset details are illustrative. No personal complaint is collected or submitted.
What the model estimates
Change an assumption
Compare the scenario with a hypothetical variation. A higher estimate shows an association learned by the model; it does not show that changing the story causes a better outcome. Differences are in percentage points.
The scenario assumptions
These are editable analyst judgments, not verified facts or legal determinations. Changing them updates the model and the sample comparisons.
Related complaints in the sample
Real public narratives selected by similarity to the scenario. They illustrate variation in reported outcomes.
Patterns I explored in 2,000 complaints
Share of March 2026 bank, card and payment-app complaints labeled by the company as monetary relief. Groups overlap; these are descriptive comparisons.
What I take from the exercise
Turning unstructured stories into comparable features makes a dataset easier to explore. Putting the model beside the original complaints makes its assumptions easier to question.
The product challenge is how to present that evidence without making a percentage feel like a promise. A consumer-facing version would need further validation of its estimates and a clearer way to express uncertainty.
How this works, and its limits
The 2,000 complaints are a random sample of March 2026 bank, card and payment-app narratives from the CFPB's public narratives archive. Each was read by Jev, a judgment model from TypeSafe, on 13 questions: problem type, what the person asked for, how specific they were, whether they'd already disputed, and more. Those readings were matched against the outcome the company reported. The displayed estimates come from logistic models fitted to those features and company-reported outcomes. The prototype retains those models and the original sample.
The fictional scenarios use preset feature values; no new model calls are made. Outcomes are reported by companies, the sample includes only published narratives, and the analyst scores can be wrong. These associations do not establish causation. The estimates have not been established as reliable forecasts for an individual complaint, and the demo does not estimate a dollar recovery amount.