The Evolving Data Technology Landscape for Credit Cards

The Evolving Data Technology Landscape for Credit Cards

This is the third post in a series of blogs looking at how credit card issuers have the opportunity to transform marketing to become truly personalized. In the first blog, we discussed the need for personalized marketing and constraints to achieving it. In the second blog we explored the MarTech (marketing technology) landscape. In this blog we discuss challenges and solutions for building a data landscape that supports personalized marketing.

Underpinning the personalized marketing vision we’ve been describing is data management. I highlighted the “DM HUB” that fulfills this in the Gartner Transit Map. Though small in size, it’s large in importance: it serves as a centralized data source for customer data, partner data and campaign data that MarTech solutions need.


The “DM HUB” is easy to miss on the map but every MarTech solution runs through it

Traditionally, credit card issuers would consume account and transaction data from the credit card processors and associations via daily batch feeds that process overnight. With the increased capabilities to capture, transmit and store data, legacy processes are becoming overwhelmed. Credit card processors can now transmit transaction authorizations as they are happening. Organizations use social media in their marketing efforts. The amount and types of partner data available increases every day.

Enterprise data warehouses cannot keep up with the amount of data being captured and the speed at which organizations need to analyze and react to the data.

Luckily there are many options in the technical landscape to respond to this need:


Select options to consider when evolving your organization’s data technology landscape

Much like Gartner’s Transit Map, this is a pretty overwhelming set of options. To understand how these options give marketing teams greater personalization capabilities, we will first outline one example of an architecture that could be built from this set of options. Then, we will explain how this architecture enables a personalized marketing use case: sending an offer to a customer with an email triggered by credit card usage.



A data architecture example built with technical options from the previous graphic

On the left, this example shows a sample of data sources. While not an exhaustive list, it gives an idea of the types of data that credit card issuers can use.

A.  Credit card transactional data: available from the credit card processors in real time
B.  Customer activity on a corporate web site
C.  Direct mail responses
D.  Traditional inbound files: from credit card processors, business partners, or credit bureaus.

The middle of the graphic shows the technology components that service the data sources on the left.

E.  Spark: a data processing tool that can read from all those sources, transform the data into a consistent event format and land that data on Kafka.
F.  Kafka: a data log that organizes events by topic, maintains sequence and exposes those events to other processes.
G.  Micro Services: small programs built for a discreet purpose. An infrastructure can have many of these doing different things, but can also have many doing the same thing in parallel to process more events faster.

Moving to the right of the graphic, in this example, Micro Services read the events from the Kafka log and act on them.

H.  Write the data to Cassandra: Cassandra is a database organized in a traditional table and column format. This is where you go to look up all the information for a particular customer.
I.  Write the data to Hadoop: Hadoop is a data store that allows fast and inexpensive data storage in a flexible. format. This is used for very high volumes of data that are not needed in real time but have value for later analysis.
J.  Trigger an activity: a micro service can also trigger an activity such as sending an email to a customer.

On the far right, the last component shown is an analytics engine, R.

K.  R: a statistical analytics platform that allows for deep analysis of the data stored on Hadoop and the distilling of disparate data points into easily understood groupings or values. Examples include customer scoring and segmentation into personas. The results of the analysis are written to Cassandra for high speed access; for example, by a customer service representative.


To explain how the example architecture enables a specific use case, we will describe how the architecture will pick up on a customer’s credit card activity and trigger a real-time email with an offer personalized to that particular customer. For this use case, when the customer uses his card at a retail store, the credit card issuer wants to send him an email with a spend and get offer before he leaves the parking lot.



  1. The customer makes a purchase at a merchant. The processor pushes that authorization to the card issuer.
  2. Spark receives the message from the processor, transforms the message into an authorization event, and sets that event on the Kafka authorization topic.
  3. Kafka sequences that event chronologically with all the other authorizations and holds that data for future processes to act upon.
  4. There is a Micro Service monitoring for authorization events on Kafka. When it sees the authorization, it processes the business logic and determines that this authorization is meant to trigger an email. It creates the email message and hands it off to a mail service.
  5. The mail service delivers the message.
  6. The customer receives the message as an email with a spend and get offer.

In the use case, the time to process a single authorization from when Spark first saw it (#2) until the time the email server sent the message (#5) would be measured in seconds. This use case can scale to handle large data volumes by simply creating more instances of Spark, Kafka, Micro Services, etc. to handle more authorizations in parallel.

Outside of #1-6 for this use case, the authorization from #2 is also written to Cassandra (so a customer sevice representative could see the customer’s credit card activity) and to Hadoop to make the authorization available for future analytics by R.

The example architecture and use case here demonstrate a sample of the analysis and technology infrastructure that could be used to support credit card issuers in transforming their marketing to become truly personalized. As additional requirements and use cases evolve, additional data processes and Micro Services can be added without modifying or interrupting the existing processes. Not only can this solution scale to the data volume, it can also scale to the business.

Phone: 312-602-4000
222 W. Adams
Chicago, IL 60606
Show Buttons
Share On Facebook
Share On Twitter
Share on LinkedIn
Hide Buttons