microservicios, en qué lío me he metido

Microservicios,

en qué lío me he metido

by

MADRID · NOV 18-19 · 2016

Ernesto Hernández

@ehdez73

Miguel Ortega

@mortegarod

MADRID · NOV 18-19 · 2016

MADRID · NOV 18-19 · 2016

MADRID · NOV 18-19 · 2016

● 7 Countries● Over 50 carriers● >300 shipping services● >500% shipment grow● >2M shipments/year

MADRID · NOV 18-19 · 2016

GIVEN:

The team size, the requirements and the legacy systems that we had.

WHEN:

We tried to migrate to a microservices architecture.

THEN:

We found that this approach to microservices was the one that better suited to our needs.

Work in progress

MADRID · NOV 18-19 · 2016

New Idea

http://transformcustomers.com/category/business-transformation/

MADRID · NOV 18-19 · 2016

Our legacy old

Monolith

http://isijisan.blogspot.com.es/2015/03/el-escarabajo-pelotero.html

MADRID · NOV 18-19 · 2016

Fast growth

MADRID · NOV 18-19 · 2016

Our shiny and new

Monolith

https://mekubal.wordpress.com/2012/04/03/the-myth-of-monolithic-judaism/

KOKUDO

MADRID · NOV 18-19 · 2016

Our Monoliths

KOKUDO

MADRID · NOV 18-19 · 2016

Business needs!!! We need an API

MADRID · NOV 18-19 · 2016Sometimes you need to clean up

MADRID · NOV 18-19 · 2016

● Another monolith

● ESB orchestration

● DB coupling

● Fixed deploy dates

● Something distributed

● More choreography

● Different storages

● Deploy flexibility

MADRID · NOV 18-19 · 2016

It seems clear right?

Microservices

MADRID · NOV 18-19 · 2016

Not so fast...

2014 2015 2016

MADRID · NOV 18-19 · 2016but what the hell is a microservice?

CHALLENGE ACCEPTED

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 1

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 1

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 2

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 2

● ONE DATABASE● CODE STILL IN MONOLITH

FATAL

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 3

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 3

● ONE DATABASE● NO REAL MICROSERVICES● DEPLOY/CONFIGURATION HELL

ERROR

MADRID · NOV 18-19 · 2016

Turn

Around

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 4

MADRID · NOV 18-19 · 2016

Synchronous

Network call

MADRID · NOV 18-19 · 2016

Synchronous

Network call

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 4

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 4

● SYNCHRONOUS CALLS HELL● WHERE ARE MY JOINS?

warning

MADRID · NOV 18-19 · 2016

Asynchronous

MESSAGE BUS

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 5

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 5

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 5

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 5

● SHARED LIBRARY● NEW MONOLITHS

INFO

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 6

MADRID · NOV 18-19 · 2016

Packlinktechture Phase 6

● DUDE, WHERE ARE MY LOGS?

DEBUG

MADRID · NOV 18-19 · 2016

Correlation id

Network call{ correlation_id: 123, ... }

{ correlation_id: 123, ... }{ correlation_id: 123, ... }

{ correlation_id: 123, ... }

{ correlation_id: 123, ... }

{ correlation_id: 123, ... }

MADRID · NOV 18-19 · 2016

Logging stack

MADRID · NOV 18-19 · 2016

Logging stack

TRACE

MADRID · NOV 18-19 · 2016

● EVENT SOURCING● CQRS

MADRID · NOV 18-19 · 2016

The Platform

MADRID · NOV 18-19 · 2016

Docker Ecosystem FTW

● build once, run everywhere● easy scaling with docker-compose ● infrastructure scaling with swarm

MADRID · NOV 18-19 · 2016

Hashicorp Consul

- Key/Value- consul-template- Leader election- DNS - Already needed for docker swarm

MADRID · NOV 18-19 · 2016

Service discovery

MADRID · NOV 18-19 · 2016

How do I develop locally?

MADRID · NOV 18-19 · 2016

New old problems

● Onboarding new developers● Config management● Debugging● Data Bootstrapping

MADRID · NOV 18-19 · 2016

What do we need?

● Infrastructure Bootstrapping:○ up and running○ configuration

● Microservices Bootstrapping:○ up and running○ application data

MADRID · NOV 18-19 · 2016

How do we solved it?

● Infrastructure Bootstrapping.○ up and running○ configuration

● Microservices Bootstrapping:○ up and running○ application data

MADRID · NOV 18-19 · 2016

How do we solved it?

● Infrastructure Bootstrapping.○ up and running○ configuration

● Microservices Bootstrapping:○ up and running○ application data

MADRID · NOV 18-19 · 2016

Local development

MADRID · NOV 18-19 · 2016

Service discovery (review)

MADRID · NOV 18-19 · 2016

Service discovery (IDE)

MADRID · NOV 18-19 · 2016

Service discovery (remote)

MADRID · NOV 18-19 · 2016

Service discovery (all together)

MADRID · NOV 18-19 · 2016

Our (helpful) Dev Tools

MADRID · NOV 18-19 · 2016https://slack.com

MADRID · NOV 18-19 · 2016https://newrelic.com

MADRID · NOV 18-19 · 2016

Wiggumbot

https://wiki.jenkins-ci.org/display/JENKINS/GitHub+pull+request+builder+pluginhttp://docs.sonarqube.org/display/PLUG/GitHub+Plugin

MADRID · NOV 18-19 · 2016

Hound

https://github.com/etsy/hound

Etsy, Inc.

MADRID · NOV 18-19 · 2016

Asciidoctor

http://asciidoctor.org

MADRID · NOV 18-19 · 2016

Custom scripts

check-status.sh

compose-upgrade.sh

dashboards

checks

channels

metrics

Platform monitoring

MADRID · NOV 18-19 · 2016

Was it worth it?

MADRID · NOV 18-19 · 2016

# Shipments

MADRID · NOV 18-19 · 2016

# Deployments

2 → ~20 per week

MADRID · NOV 18-19 · 2016

It seems clear right?

Q&A

MADRID · NOV 18-19 · 2016

Thank you

MADRID · NOV 18-19 · 2016

WE ARE HIRING