Skip to main content

3 posts tagged with "api"

View All Tags

· 5 min read
Alvaro Jose

On our previous installments, we discussed the smells that can happen when splitting microservices, and the strategies that exist to make them as independent as possible. But how do we define boundaries? How do we define the process that our microservice is in charge off?

Event Storming

Event storming is a technique that is part of DDD. But, what is Event storming?, the definition on Wikipedia is:

A workshop-based method to quickly find out what is happening in the domain of a software program. The business process is "stormed out" as a series of domain events.

This process is run with stickies in a physical or digital board during a session, and requires the 'experts' on the process to be present to provide the context what/whom/how. The outcome is an understanding of the business process, not the technical one. To be able to separate them into different steps with clear responsibilities.

Step-By-Step Guide

let's do an example of how a company sets up our internet connection

Prepare a board and the people for the session

Event storming requires people to share a common view and brainstorm and discuss on it. This process takes to count time as a dimension. And has multiple types of stickies that can be used.
You can see an example board on the next image:

Regarding the Stickies, their color represent a specific meaning[1]:

  • Events (orange): Represent the factual events and anything that is relevant to a domain expert.
  • Commands (blue): These are requests to do something. They can originate from a user or system or by another event.
  • System (pink): These represent systems involved in the domain. They may issue commands or receive commands along with triggering events.
  • User (yellow): These are human users involved in the process. They may be a single person or a department/team.
  • Aggregate (tan): This is the first level of categorization and can be thought of as the “thing” that a group of events operates on.
  • Read Model (green): This represents data that may be critical for a user or system to decide.
  • Policy (gray): These represent standards or rules that may need to be executed, such as rules for a compliance policy.

Define the Events of your system

Events are the most important information of our board. They represent facts regarding the process and helps encapsulate the knowledge of the 'experts'.
As we mention before, time is a significant dimension. A process always happens in a period of time. Starting by organizing this 'things' that happen in a timeline is a good way to start.

In our example, you can see on the previous image we go from checking coverage, to creating a user, to creating a contract and connecting our user to the network.

Identify the Systems involved (Optional)

The intent of this step is to identify the existing systems and their interdependency. When we discuss systems, they can be internal or external.

In our example, all starts with the website, but soon enough it becomes apparent most of the process is taken care by the monolith.

This step is optional in the case you have a greenfield. Nevertheless, I highly recommend it if you are splitting a monolith.

Add the Actors

These are real people who are part of the process, they tend to be the starting point of a chain of events, or even on a manual process we are trying to automate the executors of the individual step.


In our case, the user is the one starting the process, but there needs to be a technician doing the last steps manually.

Connect the dots with Commands

Now we are left with events that are done by someone and take effect in parts of our system. But we are missing the cause and effect that made this look this way.

Commands allow exactly this, is a specific action or decision that will push our system into a certain direction.

Commands can be positive or negative actions, causing bifurcation and showing different cases that our system needs to cope with.

Define Bounded Context

now we are left to define where each of the sub-process that conform our system starts and ends. This is done by grouping the stickies with an enclosing and giving a noun + verb to it, as it's a sub-process and it evokes action.

Now you have a set of split actions that can become their microservices and provide part of the process independently.

Create Capabilities Matrix (Optional)

Now, with the bounded context, we can start defining the capabilities of our services. This is straightforward to express in a matrix.

ContextCapabilities
Network ManagementCheck coverage
Enable Network
3rd party Hardware management integration
User ManagementCreate User
User Email Verification
contract ManagementCreate Contract
User Email Verification
3rd party digital signature integration

Devise your Goal Architecture (Optional)

Knowing our current architecture, it's good to think where we want to go.
This is not only a technical challenge, but an organizational challenge due to Conway's law. If we would like to be successful in splitting a monolith our communication, meaning the teams structure involved, need to resemble this target state.

Define a plan on how to split the Monolith (Optional)

A change so big as the one shown on the previous image can be overwhelming for an organization and create a paralysis and doubts. It's always good to split the problem in steps to understand progress and be always on a better state. This will improve morale.

[1] https://www.capitalone.com/tech/software-engineering/event-storming-for-microservice-architecture/

· 3 min read
Alvaro Jose

On the previous installment of this series, we discussed the pitfalls that could happen when we split a monolith into microservices. In specific, we talked about creating what are called microliths.

Given that you have followed the recommendations on designing your domains correctly. Today we are going to elaborate on patterns to remove that synchronous communication in between 'microservices'. This will help our services to become more resilient.

The Patterns

Circuit Breakers

The most simple solution we can go for is called circuit breakers. As it implies, is just a piece of code that upon multiple request failed to a downstream service will fail silently and allow service to resume their normal behavior.

What are we solving and what are we letting unsolved:

  • ✔️ We don’t fail continuously if some other service fails.
  • ❌ We silently don’t finish the entire process requested.
  • ❌ We require all chain of dependencies to be called.
  • ❌ We force other services to scale to our needs.
  • ❌ Data is mutable, so errors will be propagated and not solvable.

Outbox Pattern

The next level in solving our microlithic issue is to decouple our services using Pub/Sub to exchange models in between services.
Our service will consume and store the necessary information to run the process locally, and will broadcast the outcome models. This will mean there will always be a strong consistency in the outbox, and eventual consistency on the service database (if it exists).

What are we solving and what are we letting unsolved:

  • ✔️ We don’t fail continuously if some other service fails.
  • ✔️ We always finish our process and promise the rest will be done.
  • ✔️ We just require our service to do what we promise.
  • ✔️ Fast services will be fast, and slow services can go slow.
  • ❌ Data is mutable, so errors will be propagated and not solvable.

Event Sourcing

The last level is event sourcing. The idea is to use the events that generated a specific state and not use the calculated state that a service can provide us.

This allows a higher resilience due to the immutability of the data. In this case, calculation issues of the past can be solved, as we can reprocess the entire set of events that took us to a certain state.

Conclusion and follow-ups

These are some of the patterns that can make our services more independent and resilient. Nevertheless, each of them has a different complexity, meaning it also affects the complexity of our code. For this, we need to make sure we use the correct tool for the job.

· 4 min read
Alvaro Jose

The Monolith

We have all at this point encounter the big service that jumpstarted the business. It's always good to find it or know it existed. It shows that there was an intent to not resolve every architectural problem before we even knew we had a business.

Nevertheless, it tends to outgrow itself and become more a pain than a solution. Some of these pains are:

  • We all work on the same code base, and conflicts and side effects start to happen.
  • You need to release the entire solution, even if different teams have different cycles.
  • There are code freezes to go through validation cycles.
  • It scales as a whole, not only the portion that has an increase in traffic.

Due to these pains, microservices were created. To give team/domain independence to create focused solutions on a business that has already been validated.

The Microservices

Let's start with a definition of a microservice:

Microservices are an architectural and organizational approach to software development where software is composed of small independent services that communicate over well-defined APIs. These services are owned by small, self-contained teams.

microservices

All sounds like flowers and happiness when we talk about microservice. Nevertheless, does microservices solve the entire issue by itself?

Have you encountered the next cases in a microservice architecture?

  • Before we release a new version, we need to sync deploys with another team.
  • Our application was down, but is not our issue.
  • Our service was working and scaling fine until the team X started using us.
  • And more…

What is happening?

Microliths

The smells mention before are caused by what Jonas Boner call Microliths, a great word for what is happening here.
Even if we think this are 'independent' services, synchronous communication can cause side effects we don't want:

  • There can be cascading events between your services.
  • Your domain boundaries are not clear because you don’t own the entire process.
  • Slow services are forced to scale by faster services requirements.
  • There is additional latency on the network calls.

What got lost in translation?

Having microliths comes from multiple misconceptions we have. Some of them are:

Domains != Resources

Every so often, when we divide the monolith, we think about domains being resources. Due to how we normally have divided API's and DB's as we think about splitting what already exists and not about extracting the processes being achieved.

When thinking about a microservice, we should think about what part of the process it is trying to solve, this will help us define good boundaries for our bounded context.

When we think in a process, data is secondary. The process will require different pieces of existing data to fulfill their capabilities, and it is ok for it to own its copy of what is needed to fulfill his mission.

Independence != Single Source

A single source of data does not mean independence, as whenever your software requires complementary data, it will have to acquire it from somewhere else, what means a direct dependency. This also affects boundaries as you must enter other team's domain.

If you strive for independence, copy the information you require for your process, even if it exists somewhere else.

Fast != Synchronous

Humans think that a direct response is always faster than sending out a message. While occasionally this is true, in microservices this could start a cascade of synchronous calls from one service to the next one, leaving our users in a timeout limbo.

Think if really your system requires calling others directly or if you can message them to start their process.

Resilience != Complete

Making sure the entire process has been completed, is normally confused by resiliency. Resiliency only refers to the capability to complete the process.
If we have well-defined contracts in between our pieces, we don't need to finish things synchronously, we can promise our users things will happen. And let our services do their work at their speed.

Conclusion and follow-ups

Are we doomed?

The answer is no, we are not doomed! We can design our services with the correct division using some DDD tooling and also use the correct tools to decouple our microservices.
Let's talk about this on the next chapters of this series.