How KarroFin Automated Credit Scoring for 250K Borrowers

For: COO or head of credit at a mid-sized NBFC or digital lending startup who is manually underwriting hundreds of loan applications a week and knows the bottleneck is the process, not the people — but has no clear picture of what an automated credit pipeline actually looks like end-to-end before it goes to regulators

The lesson from building an automated credit scoring pipeline for KarroFin's 250,000 borrowers is unglamorous: the model was never the hard part. The hard part was the feature layer — turning messy bank statements, GST filings, UPI transaction logs, and telco data into a numeric matrix a model could actually consume, with lineage clean enough to defend in front of a compliance officer. Teams that skip this and jump straight to XGBoost end up with a scorecard that works in the notebook and collapses in production.

This post walks through what KarroFin's underwriting problem actually looked like, what the first attempt got wrong, the architectural decision that unlocked scale, and the metrics that moved. If you run credit at an NBFC or a digital lender and you're staring at a queue of manual applications every morning, this is the shape of the system you probably need to build.

The starting point: manual underwriting at breaking volume

KarroFin was a mid-market lender extending small-ticket credit — mostly working capital and consumer loans — to borrowers across tier-2 and tier-3 India. The credit team was doing what most NBFCs at that stage do: analysts pulling bank statements from email, running eyeballs over three months of transactions, cross-referencing bureau pulls, filling in a scorecard spreadsheet, and forwarding to a credit manager for approval.

The problems compounded predictably:

• Turnaround time from application to decision was measured in days, not minutes. Borrowers were dropping off to faster competitors.
• Two analysts scoring the same file could reach different decisions. Consistency was a policy document, not a reality.
• Volume was climbing faster than the team could hire. Every new analyst took weeks to become useful and months to trust with edge cases.
• There was no clean audit trail. When a loan went bad, reconstructing why it was approved required interviewing the analyst who did it — assuming they still worked there.

The leadership call was correct: automate. But the first version of "automate" that got scoped was, in retrospect, wrong.

What the team tried first — and why it stalled

The initial approach was the one most teams reach for. Pull the bureau score, pull a few derived variables from the bank statement (average balance, number of bounces, salary credit consistency), build a logistic regression on historical portfolio performance, and route anything the model wasn't confident about to a human. Classic champion-challenger setup.

It sort of worked. The model trained fine. AUC on a holdout was respectable. Then it hit production and three things happened.

First, the input data was garbage more often than anyone expected. Bank statement PDFs came in dozens of formats — SBI, HDFC, ICICI, cooperative banks, regional rural banks — and the parser the team was using worked on maybe 70% of them cleanly. The remaining 30% either failed silently or produced numeric fields that were subtly wrong. A misread decimal place would turn a ₹4,500 salary into ₹45,000. The model, dutifully, would approve.

Second, the feature definitions drifted. "Average monthly balance" meant one thing to the data scientist who built the model and something slightly different to the analyst who had been calculating it manually for two years. When performance looked off in the first month, nobody could tell if the model was miscalibrated or if the features it was seeing didn't match the features it was trained on.

Third, and most painfully, compliance had questions the team could not answer. Not "is the model biased" — that one they were ready for. Questions like: "For this specific rejected applicant, show me every input variable, the source document, the timestamp it was extracted, the version of the parser that extracted it, and the version of the scoring model that consumed it." That level of lineage did not exist.

The takeaway was clear. Rebuilding the model was not the priority. Rebuilding everything upstream of the model was.

The architectural call that unlocked it: a normalization layer as a first-class citizen

The decision that changed the trajectory of the project was to stop treating data ingestion and feature engineering as glue code and start treating it as the core product. Concretely, the pipeline was rebuilt around four decoupled layers:

1. Ingestion and document extraction

Every input source — bank statement PDFs, GST returns, bureau reports, UPI transaction exports, KYC documents, application form data — got its own extractor with an explicit schema and version number. Bank statement parsing in particular got a hybrid approach: a rules-based parser for the top 15 formats that covered the majority of volume, and an ML-based fallback (layout-aware document understanding) for the long tail. Every extracted field carried metadata: source document ID, page number, extraction confidence, parser version.

Critically, low-confidence extractions were flagged, not silently accepted. A statement where the parser wasn't sure about three transactions would route to a human reviewer whose only job was to correct those three fields — not re-underwrite the whole file. This is the sort of narrow, well-defined human-in-the-loop step that actually scales.

2. Canonical transaction ledger

All financial transactions — from bank statements, UPI logs, whatever — were normalized into a single canonical schema. One row per transaction, with fields for date, amount, direction, counterparty (as extracted), a category (assigned by a classifier), and a source reference. This one decision paid for itself repeatedly. Downstream features stopped caring where a transaction came from. New data sources could be added by writing an adapter that produced canonical rows.

The transaction categorizer itself was a small NLP model trained on labelled Indian bank narration strings — the notoriously cryptic UPI-VPA-REF-NUMBER strings that carry more signal than they look like they do. Getting counterparty extraction right on these narrations was worth more than a fancier scoring model.

3. Feature store with versioned definitions

Every derived feature — average monthly inflow, bounce ratio, salary consistency score, obligation-to-income ratio, seasonality of business inflows for self-employed borrowers — was defined once, in code, in a feature store. Every feature had an owner, a definition, and a version. When a model was trained, it recorded exactly which feature versions it used. When it scored an application in production, it pulled those same versions. The training-serving skew problem largely disappeared.

This is also where alternative data started to earn its keep. For thin-file borrowers with limited bureau history, features derived from UPI activity, GST filing regularity, mobile recharge patterns, and utility payment behavior filled in the picture. None of these are magic on their own. In combination, and cleanly engineered, they moved the needle on approval rates for segments the traditional scorecard couldn't touch.

4. Scoring, decisioning, and explanation

Only after the three layers above were solid did the actual scoring model matter. The team ended up with a gradient-boosted ensemble as the primary scorer, with a monotonicity constraint on key variables (a higher bounce ratio should never improve your score — obvious, but you have to enforce it) and SHAP-based explanations attached to every decision.

The decisioning layer sitting on top of the score was deliberately kept as a policy engine — plain rules, editable by the credit team, expressing things like "approve if score > X and DTI < Y and no bureau flag Z." This separation matters. The model gives a probability. Credit policy decides what to do with it. Policy changes far more often than models, and credit teams should not need a data scientist to change a cutoff.

Making it survive compliance review

Explainability was engineered in, not bolted on. Three things carried the most weight in the compliance conversation:

• Reason codes. Every rejected application produced a ranked list of the top factors driving the rejection, in plain language, derived from SHAP values mapped to human-readable feature descriptions. Not "feature_37 contributed -0.14" but "low average monthly balance over the last 90 days."
• Full lineage on every decision. Given an application ID, the system could reproduce, from source documents up to final decision, every input, every transformation, every model version. This is what compliance actually wants.
• Fairness testing built into the model release pipeline. Before a new model version could be promoted, it had to pass tests for disparate approval rates and score distributions across protected segments. Failed tests blocked deployment.

None of this is exotic. It is standard practice in mature ML systems. But for a lending team building their first automated pipeline, it is easy to underinvest here and expensive to retrofit.

What moved

The rebuilt pipeline scored 250,000 borrowers with a small credit ops team supervising exceptions rather than underwriting every file. The changes that mattered most operationally:

• Straight-through processing on a significant majority of applications — decisioned end-to-end without human touch, with turnaround measured in minutes rather than days.
• Human review focused on flagged exceptions: low-confidence extractions, borderline scores, policy-triggered manual review. Analysts stopped being throughput bottlenecks and started being quality controllers.
• Consistency: two applications with identical inputs produced identical decisions, every time. This alone changed how the risk team could reason about portfolio performance.
• Auditability: any decision, on any borrower, could be reconstructed on demand.

The AUC of the final model was better than the first attempt, but honestly, not dramatically so. Most of the value came from the plumbing.

Lessons that generalize

If you are the COO or head of credit at an NBFC staring at this problem, the things worth internalizing:

The feature pipeline is the product

You almost certainly have enough data to score borrowers accurately. It just doesn't live anywhere the model can reach yet. Budget for the ingestion, normalization, and feature store work as if it were the primary deliverable — because it is. The model is the last two weeks of a much longer project.

Human-in-the-loop only works if the loop is narrow

Sending an entire file back to an analyst for review defeats the purpose. Sending an analyst three specific transaction lines to correct, with the rest of the pipeline waiting on that single fix, is scalable. Design the exceptions, don't just permit them.

Explainability is an architecture decision, not a library

You cannot "add SHAP" at the end and call it explainable. Reason codes, lineage tracking, feature versioning, and fairness testing all need to be present from the first serious version of the system. Retrofitting these into a live model is far more expensive than building them in.

Separate the score from the policy

Your credit policy will change more often than your model. Keep them in different systems, with different owners. The data science team owns the score. The credit team owns the policy. Both should be able to move without waiting on the other.

Alternative data helps at the margin — plumbing helps everywhere

UPI patterns, GST regularity, telco data — these all add signal, especially for thin-file borrowers. But they add signal only if the pipeline to consume them is clean. If your bank statement parser is dropping 30% of statements, no amount of alternative data will save you.

How CodeNicely can help

The KarroFin build is the sort of engagement CodeNicely takes on when a lender has real volume, real data, and a real need to get out of manual underwriting without losing regulatory footing. The work spans document extraction pipelines, canonical data models, feature stores, model development, explainability tooling, and the boring-but-critical MLOps around versioning and lineage. If you want a closer look at similar work in the lending space, the Cashpo case study covers KYC and AI credit scoring in more depth, and the AI Studio page details the underlying capabilities.

The right starting point is usually not a model — it is a two-week audit of your current data flow. Where are the bank statements coming in? What formats? What's the parse success rate today? Where does feature calculation happen? What lineage exists? Most teams discover, uncomfortably, that they don't know the answers. That's the actual first sprint.

For teams operating in India specifically, the regulatory context around digital lending, DPDP, and RBI guidelines shapes several of these architecture calls. CodeNicely's India AI practice works within these constraints regularly.

Frequently Asked Questions

What data do you actually need to build an alternative credit scoring model?

At minimum: bureau data, bank statement transaction data (three to twelve months), application form data, and KYC. For thin-file borrowers, useful additional sources include GST filings for SME borrowers, UPI transaction patterns, telco and utility payment behavior, and device/behavioral data from your application flow. The quality of the parsing pipeline matters more than the exhaustiveness of the source list.

How do you make an AI credit scoring model explainable enough for a compliance review?

Three things: reason codes on every decision (top drivers in plain language, typically derived from SHAP values), end-to-end lineage from source document to final decision (with versioning of both features and models), and fairness testing built into your model release pipeline. Monotonicity constraints on key variables also help — regulators respond well to models where you can prove that worse behavior can never produce a better score.

Can we automate underwriting without replacing our credit analysts?

Yes, and you probably shouldn't try to eliminate them. The pattern that works is straight-through processing for the majority of applications, with analysts focused on exceptions: low-confidence data extractions, borderline scores, policy-flagged reviews, and portfolio quality monitoring. Their role shifts from underwriting every file to controlling the quality of the automated system.

How long does it take to build an automated lending platform of this scope?

It depends heavily on your current data infrastructure, the complexity of your product mix, and the regulatory scope. Rather than quote a generic number, it's more useful to scope the specific pieces — ingestion, feature layer, model, decisioning, explainability, MLOps. Talk to CodeNicely for a personalized assessment based on your current stack and volume.

What is the biggest mistake NBFCs make when automating credit decisioning?

Under-investing in the feature pipeline and over-investing in the model. Teams spend weeks tuning XGBoost and then discover that their bank statement parser fails on 30% of inputs, their feature definitions drift between training and serving, and their compliance team can't reconstruct why a specific applicant was rejected. Get the plumbing right first — the model is comparatively easy.