What happens when a bank connection breaks

On any given day, somewhere between 0.4% and 2% of our bank connections are broken. That number has never been zero and it never will be. Bank connections are the least reliable dependency we have, by an order of magnitude, and building around that assumption shapes a surprising amount of our architecture.

This post is about what "broken" means, how we notice, and the judgement call at the end of it.

Four failure modes, three of them silent

The obvious failure is an expired consent. Under PSD2, a user's authorization lapses every 90 days and must be renewed. This one is honest: the API returns a clear error, we know exactly what happened, and we know exactly what to tell the user.

The other three are worse.

Credential invalidation happens when someone changes their bank password. The connection doesn't error immediately; it just starts returning empty result sets, which is indistinguishable from a quiet month.

Partial sync is the one that keeps me up. The API returns 200 OK with a truncated transaction list. Nothing is wrong from the caller's perspective. You simply have fewer transactions than exist, and every downstream number — balances, categories, projections — is quietly wrong.

Stale data is the same problem with a timestamp. The aggregator serves a cached response from six hours ago and flags it accurately in a header that's easy to ignore.

The lesson we learned the hard way: an error you can see is a good day. Plan for the failures that look like success.

Detecting the silent ones

We run a reconciliation check on every sync. The principle is simple — the balance the bank reports should equal the previous balance plus everything that moved since.

def verify_sync(account, snapshot):
    expected = snapshot.opening_balance + sum(t.amount for t in snapshot.transactions)

    drift = abs(expected - snapshot.closing_balance)
    if drift < Decimal("0.01"):
        return SyncResult.CLEAN

    # A gap means transactions are missing, not that the maths is wrong.
    if drift > account.drift_threshold:
        return SyncResult.INCOMPLETE

    return SyncResult.SUSPECT

def verify_sync(account, snapshot):
    expected = snapshot.opening_balance + sum(t.amount for t in snapshot.transactions)

    drift = abs(expected - snapshot.closing_balance)
    if drift < Decimal("0.01"):
        return SyncResult.CLEAN

    # A gap means transactions are missing, not that the maths is wrong.
    if drift > account.drift_threshold:
        return SyncResult.INCOMPLETE

    return SyncResult.SUSPECT

def verify_sync(account, snapshot):
    expected = snapshot.opening_balance + sum(t.amount for t in snapshot.transactions)

    drift = abs(expected - snapshot.closing_balance)
    if drift < Decimal("0.01"):
        return SyncResult.CLEAN

    # A gap means transactions are missing, not that the maths is wrong.
    if drift > account.drift_threshold:
        return SyncResult.INCOMPLETE

    return SyncResult.SUSPECT

If the arithmetic doesn't close, we didn't receive everything. It doesn't tell us what is missing, but it tells us not to trust the sync — and that's enough to stop the bad data before it reaches a projection.

A SUSPECT result triggers a re-fetch with a wider date window. An INCOMPLETE result marks the account stale and freezes any derived figures until it resolves.

We also track a per-institution health score, a rolling seven-day success rate weighted toward recent attempts. When an institution drops below 85%, we stop scheduling optimistic background syncs against it and back off to a slower cadence. There's no point hammering an API that's having a bad week, and the aggregator charges us per call regardless of outcome.

The recovery ladder

Not every failure deserves the same response, so we escalate in four steps.

A silent retry comes first, with jittered backoff. Roughly 70% of failures resolve here, because roughly 70% of failures are transient.

If that fails, we fall back to a cached read. Projections keep working on data up to 24 hours old, with a stale marker in the interface. A slightly old number is more useful than no number.

If the connection is still broken after 24 hours, we degrade visibly. The account shows a "last updated" timestamp and drops out of the aggregate total rather than contributing a wrong figure to it.

Only then do we ask the user to reconnect.

The judgement call

That last step is where engineering ends and product judgement starts.

Every reconnection prompt costs something. It's friction, it's a small dent in confidence, and if you send enough of them people start ignoring all of them — including the one that matters. But not prompting means someone is looking at a number that's quietly wrong, which is the worse failure for a product whose entire proposition is that you can trust the number.

We landed on a rule that has more nuance than I'd like: prompt immediately if the account materially affects the projection, and batch the prompt into the weekly digest if it doesn't. A dormant savings account that hasn't moved in eight months doesn't warrant a push notification. Your current account does.

Nine months in, our reconnection completion rate is 74% within 48 hours. I'd like it higher. The remaining 26% is mostly people who connected an account once out of curiosity and never came back — which, honestly, is a fine outcome too.