Preventing Customer Churn: How Machine Learning Helped a Financial Services Company Retain Customers

A large financial services firm was struggling with customer retention. Having operated in the financial space for decades, they had an extensive customer set of people with a wide range of account characteristics, ranging from small balances to multi- million dollar portfolios. The customer success team was tasked with preventing customers from withdrawing funds and closing accounts by taking early, preventative actions.  

The challenge they faced, was that they had no reliable way to tell who was at risk of closing an account. To compensate, the team relied on a set of heuristic rules built from the anecdotal experiences of individual sales representatives. Effectively, the customer success team (CST) was operating blind, spending efforts on accounts based on intuition rather than evidence. They wondered if there was a better, more data-driven way to approach the problem.  

The natural approach for a problem like this is a machine learning based classification algorithm to determine accounts which are likely to churn. As is often the case, however, the data was not in a usable state. Despite having operated for decades, this company had changed data platforms many times over the years. When Aimpoint Digital was initially approached with the problem, we were given data beginning in 2023 which, as of time of writing, was approximately 3.3 years of data. The issue was, the organization wanted to predict forward up to 12 months.

They also requested use of a rolling 12 month look back period of account history. For a single datapoint, that meant looking back 12 months to compile a feature, and then predicting 12 months forward. This reduced the usable training dataset from 3.3 years down to about 16 months of usable data. For this amount of time, with one year of training history available, a model would not be able to learn any long- term patterns. This left a few options: reduce the prediction horizon, reduce the length of the look back period, and Dive into the depths of old databases to resurrect some extra training data.

Because the 12-month prediction horizon was a business requirement, reducing the lookback period for an account would still leave us lacking in the data department. We set up a series of calls with the data storage teams to track down additional account history. The biggest obstacle emerged after securing account history: there was no transaction- level data log historically, only monthly snapshots. To solve this, we built a substantial data transformation pipeline that took an account’s snapshot, and, using rolling statistics, estimated transaction level behavior. Because this generated millions of records, TSpark and EMR were essential to processing it at scale. Once this completed, the effort yielded 15 years of account history to train on.

With data compiled across the modern datalake and historic data system, we had our training set ready. Running various hyperparameter configurations and modeling approaches, we were able to get the optimized approach.  

The trained model (XGBoost) achieved AUPRC of 0.94 and ROC-AUC of 0.986 on a held out test set of about 8k participants. The model was able to correctly identify ~95% of participants who withdrew with ~84% precision, a particularly strong performance showing.  

The precision-recall tradeoff at top-K (figure 3) is the shape you want for a triage/prioritization use case. Precision stays above 90% through the top 15% of scored samples and only degrades below 80% beyond top ~22%.

After developing a solid data engineering and science pipeline, this was pushed into production in the available AWS environment. Updated data lands in S3 weekly. A SageMaker Processing Job — packaged as a Docker container in ECR and run on an r5.4xlarge instance for its memory footprint — pulls the new data, runs the pipeline end-to-end, and writes ranked churn predictions back to S3. The workflow is orchestrated by Step Functions, triggered on a weekly cron via EventBridge, and finishes by dispatching a digest email of the highest-risk participants via SES.