The topic of microservices architecture comes up often in our discussions with clients. There’s a lot of interest but also confusion about how, when and why to use it. This is a very complex topic, so we’ve developed this microservices “miniseries” to provide some insight from our experiences and to get more discussion going in this area.
The first article in this series offered some background about microservices architecture: what it is, when and why it is beneficial, and some things to think about before jumping in. In this article, we tackle the topic of how to identify and evaluate opportunities to use microservices architecture and prepare for development.
In any discussion about microservices architecture, it helps to think in terms of a particular scenario or business issue. For the purposes of this article, we consider a large insurance organization. The insurer’s core systems serve a variety of functions: government, commercial and individual customer segments; claims processing; eligibility; profile management; and various others. The insurer is moving from their outdated systems by building a modern platform primarily to improve claims processing flexibility and performance as they look to expand regionally and to new business lines. This platform also needs high scalability and flexibility for portions of its operations without impacting others. With those goals in mind here’s the approach we would recommend to get started.
Define the business problem and required functionality
As with any technology-enabled business solutions, the starting point is a clear definition of the business problem and goals. In this case, the insurer wants to process claims faster and more accurately with less user intervention. This improvement has the impact of driving down cost in many other areas of the business. To clarify the problem, the organization’s process owners and system architects will need to discuss key challenges, current system constraints, and business needs and then use those as the foundation for shaping a solution.
At this stage, it is also helpful to identify broad features and functions required to inform initial architecture design, but it is not necessary to produce any detailed requirements documentation or fully formed user stories. That work will come later. Using our insurance example, the company determines that it needs a flexible, new claims processing algorithm to support auto-adjudication, as well as system capabilities for scaling to support bursts of peak claims processing volume of 100,000 claims per second.
Confirm the potential for using microservices architecture
Clarifying the problem helps the team to assess and validate the need to tackle the solution with a microservices architecture. Some issues may not be the best candidates, so these are some questions to ask at this stage:
- Do you need scalability and flexibility in certain functionality and its use of the system?
- Is the business problem you are trying to solve functionally complex and evolving?
- Is your organization capable of delivering the solution using this style of architecture?
- Do you have a resource, advisor, or partner with experience that you can tap in to help with the microservices architecture?
With the decision to move forward on the microservice path in hand the following steps will help prepare for efficient development.
Assemble the right team
Development using microservices architecture requires a new approach to organizing the resources than traditional system implementation projects. The people who own the functionality will need to be highly involved in building it. Therefore, it is important for business and technical resources to work closely from the outset.
Typically, a small focused team of business analysts, product managers, testers, operations, and developers with strategic perspectives and agile capabilities are best suited to advance the process from the point of defining the business need through planning and executing development. This smaller team can then serve as a seed to grow the system, disciplines, and architecture principals more broadly across the organization.
If you or your team is new to microservices architecture, it may make sense to supplement the team with a resource(s) familiar with the complexities of this approach.
Map the domain boundaries of the system
At a broad level, you will need to understand how key areas of the system are organized and connected. Although microservices isn’t strictly a domain-driven style of architecture, it helps to borrow from the domain-driven (DDD) design concepts. There are many great resources online to learn about DDD if you are new to the subject.
Start by breaking down functionality into domains and subdomains that align to the business. Identify which of those are core, supporting, and generic within the system. In our claims processing example, this may include lists for validating membership of people and their associated plans, as well as a provider information to validate that providers are in the network and approved for a particular plan.
Use visual diagramming to draw/map relationships within the domains and continue to further refine the model by iteratively focusing on smaller portions of the system. Outline the bounded contexts; those will translate well to service boundaries in your architecture. Here’s a simplified example domain diagram for our insurance example.
When going through this exercise, it is important to think about domains independently from current architecture. You want to “re-ground” your understanding of functional domain boundaries so you can start thinking about how each of them would be built independently with no technical constraints.
Analyze the interaction and communication patterns between the components
Together with business stakeholders, analyze interactions and communication patterns between bounded contexts. These are some questions to ask and answer:
- What is the nature of the relationship? How do they interact? In our claims processing example, the claims engine uses the provider information to determine whether the provider is approved for the patient’s particular health plan.
- How do components exchange data? Is it in real-time, batched, broadcast, direct usage?
- Is this one-way or two-way communication? Can one side of the interaction exist without the other?
- What is the volume or scale required within this interaction? How much does it fluctuate over time or in response to certain scenarios (for example, open enrollment periods for insurers, retail systems on Black Friday or finance markets after a jobs announcement)?
As you analyze the interactions, document answers to these questions. This will be useful in choosing the type of service and supporting technology components (e.g. queues, APIs, etc.) needed to build your system.
Identify and prioritize the services to be built
As you are working on the model, begin to list the services to be built that map to the domain. This list does not need to be comprehensive at this point; remember this is an iterative process and you’ll be continually refining this list throughout the implementation.
In doing so, think of each service as a full stack. Estimate the effort and business value at a high level and then prioritize the order in which to build elements of the solution. Ideally, the first step in the development process will address a low-risk and potentially high value element with clear domain context and easy integration to legacy architecture as needed.
Using our insurance example, the company will need to build microservices to support its core domain of claims processing. It will also have to create services in the supporting and generic domains.
Don’t be surprised if you identify many services to build. Keep in mind that the services are meant to be small and focused in their purpose; large enterprises may have 100s of services or more in their architecture. It’s important to start with a small number of services, released iteratively, gaining experience and velocity as you proceed. Once you have prioritized services, you can then begin creating a rough implementation plan.
With the approach of starting small and expanding incrementally, the insurance company decides to start by building a single microservices to support claims intake for the individual line of business. This service will be built to add claims into the system for further processing. It will need to do so at a very high scale.
Identify the classes and types of technology needed to solve the problem(s)
Now that you have a prioritized list of what you intend to build it’s time to think about the technology that will support it. This would include:
- The underlying microservice frameworks that supports and runs your services
- The messaging technology to support communication among services
- The frameworks, libraries, and programming language(s) you will use to create your services
- The data platforms, storage and database technologies for your microservices
- User interface (UI) frameworks and hosting technology
- Security frameworks and protocols needed to meet requirements
- Management, monitoring, and development operations technology that allow you to deploy and run your services
- The infrastructure, cloud platform, virtualization, containerization, and resource management technology that your services will run upon
At this point you don’t have to decide on any single technology or framework but it helps to narrow down choices. A microservices architecture should support a heterogeneous technology environment, however it’s practical to limit technology choices in certain categores. For example, you might limit your RDBMS choices to 2 technologies (e.g. SQL Server and PostgeSQL) based on available skills in your organization rather than choosing to support any/all RDBMS systems.
Build a list of the key technology needed and remember that this does not need to be comprehensive. If these technologies are new to the team you may have to reach out to partners, build some proof of concepts, and your research here.
Get ready for implementation
It’s very likely you may need to repeat these activities several times over a number of weeks to gain sufficient clarity on the design, functionality, architecture, technology and approach. Once you have done so, you will then be ready to start planning, designing, selecting technology and implementing your first microservices. Not surprisingly, this will present a host of other considerations—and we will address those in the next articles in this series.