AI-Based Tool Accelerates Data Ingestion for Financial Provider

A Fortune 500 leading provider of investment and business solutions was challenged with efficiently handling the ingestion, transformation, and publication of large amounts of data from multiple sources, including data from new acquisitions. CapTech collaborated with them to develop a solution to streamline these processes and ensure seamless data integration.

Faced with terabytes upon terabytes of data from multiple sources after several acquisitions, a Fortune 500 financial institution needed to improve the ingestion of the data and make it more actionable. Current manual methods required multiple teams working tirelessly for months. With more data from other institutions on the way, the financial institution wanted something that could be automated and scaled. Enhanced efficiency wasn’t the only driver for a change. They also wanted a solution that helped ease the strain placed on all of the workers involved in the data integration initiatives.

We collaborated with the financial provider to develop a metadata-driven ingestion pipeline. Leveraging our AI-based accelerator, Green Arrow, which automates source-to-target mapping and enables quick ingestion of new data into existing targets, we helped to create a pipeline intended to accelerate the data ingestion from third-party producers. The pipeline was designed to pull source data in, transform it, and publish it out to the business.

Green Arrow was used for multiple data-mapping processes, helping to organize the large volume of data from disparate sources and make it much more dynamically driven. Training Green Arrow involved minimal effort from the financial institution’s employees; once it had a reference on correctly processed data, Green Arrow was able to identify similar data patterns, which would be reviewed for accuracy.

The solution was able to ingest new data within five minutes, significantly improving the speed and efficiency of data integration. As a whole, data consolidation processes went from taking months to weeks.

The new AI-based solution also ensures data mapping processes are transparent by making decision-making processes repeatable. Delegating common data integration tasks also helped reduce the number of people needed, giving more employees the bandwidth to concentrate on more complex tasks. A time reduction like this completely transforms the feasibility equation, delivering capabilities that were once unattainable.