Transaction Fraud Detection: Rules vs Machine Learning

The short answer

Rules (a rule engine) are explicit conditions like "if the amount is above X and the card country differs from the IP country, decline". They are transparent, quick to change and easy for auditors to read, but they only catch what someone has already described.

ML scoring is a model that estimates the probability of fraud from hundreds of features. It finds non-obvious combinations and adapts to new schemes after retraining, but it needs labeled data, monitoring and explanations.

In practice the hybrid wins: hard rules for obvious and mandatory cases, a model for risk estimation, and decision rules on top of the score.

Comparing the approaches

Which features are used

Features matter more than the choice of algorithm. Main groups:

• Transaction: amount, currency, merchant MCC, channel (e-commerce, POS, P2P), time of day.
• Velocity features: count and sum of operations per card, device and IP over the last 5 minutes, hour and day.
• Device and session: device fingerprint, new device or not, emulator, typing speed, password change right before payment.
• Geolocation: mismatch between card country, IP and shipping address, "impossible travel" between two operations.
• Customer behavior: deviation from their usual amount, typical merchants and active hours.
• Link graph: how many cards share one device or phone number, links to previously confirmed fraud.

Velocity and behavioral features require an online aggregate store (feature store, Redis, stream processing); otherwise the production model sees different data than it was trained on.

Real-time constraints

The authorization decision has to be made in milliseconds while the card scheme and the buyer are waiting. That means:

• Features are precomputed or updated incrementally, not queried from history at payment time.
• The model must be cheap at inference: gradient boosting is often more practical than deep networks.
• You need a fallback: if scoring times out, a base rule set applies instead of "approve everything".
• Heavy analysis (graph links, manual review) moves to post-authorization, after approval, with the option to hold shipping or withdrawal.

False positives

A declined honest payment is lost revenue and an annoyed customer. To keep the balance:

• Measure not only the share of fraud caught but also precision and the decline rate for good customers.
• Use three outcomes instead of two: approve, decline, or step up (3-D Secure, OTP, call, manual queue).
• Choose the score threshold by the cost of errors: missed fraud and a false decline cost different amounts in different segments.
• Test new rules in shadow mode: the rule logs its decision but does not block until its impact is measured.

A combined architecture

Payment -> Enrichment (device, geo, velocity from feature store)
        -> Block lists and mandatory rules (sanctions, limits)
        -> ML model: score 0..1 + reason codes
        -> Decision rules: threshold by segment, amount, channel
        -> approve / challenge / decline
        -> Decision logs -> labeling (chargebacks, complaints, analysts)
        -> Retraining and rule tuning

Key elements:

1. One decision engine where rules and the score live in the same logic and the same log.
2. Feedback loop: chargebacks and analyst decisions come back as labels. Account for delay: a chargeback can arrive weeks later.
3. Case management for analysts with decision reasons, customer history and action buttons.
4. Drift monitoring: score distribution, decline rate, quality on fresh labels.

Common mistakes

• Training only on approved transactions: operations declined by rules never get labels, so the model is blind to part of the fraud.
• Future leakage: features that were not yet known at payment time.
• Piling up rules without review: a year later nobody knows why half of them exist.
• One threshold for every segment and channel.