How to choose the right technology for your project

From DZone (original). I agree with conclusion:

  • Choose what the team is familiar with.
  • Don’t be cool — play in your free time.
  • Choose the right tool for the job.
  • Prefer principles of good software design over the latest hype.
  • Don’t overload the number of languages.
  • Educate yourself & the team on foundations of computer science.

Microservice Monitoring

Following with the previous post, we were thinking about how monitor our microservices, and here is our proposal:

monitoring

  • Data Collector (fluentd). Instead of using the common ELK Stack (elasticsearch, logstash, kibana) or Elastic Stack, we substitute logstash for fluentd. There are some comparisons like this one. logstash has a limited on-memory queue and you would need to install a message brocker, such as redis, RabbitMQ or Apache Kafka, to facilitate a buffered publish-subscriber model or enable persistent queues with limitations.Each microservice logs through console and we’ll use the Docker fluentd logging driver to process these logs and send them to our elasticsearch.
  • Distributed tracing system (Zipkin). It helps gather timing data needed to troubleshoot latency problems in microservice architectures. It manages both the collection and lookup of this data. We’ll store the information in our elasticsearch.Each microservice will use Spring Cloud Sleuth to trace each request, maintain an unique identifier and keep track. Spring Cloud Sleuth will send the data to Zipkin.
  • Distributed Search and Analytics (elasticsearch). We’ll store our data in elasticsearch so we will be able to search whatever we want in our logs.
  • Explore, Visualize and Discover data (kibana). We’ll visualize our stored data using kibana.

Microservice Arquitectures

At UST Global we are reviewing some microservice architectures which will help us to support our customers. We selected these three:

Netflix OSS Architecture

We will use the Netflix OSS (Netflix Open Source Software) components:

netflix-oss-architecture

  • Configuration Service (Spring Cloud Config). Each Spring Boot application connects to the Configuration Service to get its configuration. Each application configuration is saved in a Version Control Service (GIT), in order to maintain a record of all modifications.
  • Gateway (Netflix/Zuul). The Gateway receives all the requests from outside and the gateway proxies them to the microservice depending on the configuration (what service and where it should be directed) and the registration (where the service is located).
  • Service Registration & Discovery (Netflix/Eureka). Each microservice self-registers allowing to others microservices invoke them after discovering.

Consul.io Architecture

We will use the following architecture:

consul-architecture

  • Configuration & Service Discovery (Consul.io). Each microservice connects to the Configuration Service to get its configuration. This server is also used to discover other microservices.
  • Gateway (nginx + Consul Template). The Gateway receives all the requests from outside and the gateway proxies them to the microservice depending on the registration (where the service is located). As alternative, we could use linkerd as the gateway.
  • Service Registration (Registrator). Each docker instance is registered by Registrator inside Consul.

If we would not use Consul as DNS for other services (like databases,…), we would not need Registrator. Instead of it, we would let microservices to register them-self through Consul.

API Gateway Architecture

We are trying Tyk and also made two pull request to reduce the Docker image sizes for Tyk Gateway and Tyk Dashboard. Tyk offers an API management platform with an API Gateway, API analytics, developer portal and API Management Dashboard.

This API Gateway has the service discovery feature, we would not need any proxy/gateway (nginx + Consul template or Netflix/Zuul):

tyk-architecture

  • Service Registration & Discovery (Consul.io). Each microservice connects to the Configuration Service to get its configuration. This server is also used to discover other microservices.
  • Gateway (Tyk). The Gateway receives all the requests from outside and the gateway proxies them to the microservice depending on the registration (where the service is located).

API Development: Design-First or Code-First?

I just read this post in DZone (this is the original) related to “API Development: Design-First or Code-First?” and I am not agree with the second phrase in “The design-first approach advocates for designing the API’s contract first before writing any code. This is a relatively new approach“.

People talks about API’s like if they were born recently but we integrate systems since long time ago, so I think we should design API’s for those integrations, don’t you think? You can ‘google’ for “WSDL-first” or “Contract-First” and you will get a lot of results about “design-first” when we built SOAP API’s long time ago.

I personally prefer “Design-first” for some reasons:

  • It’s not only designing any API, it’s about your “Company API Strategy“. It’s about how will people think about the way you show your “business”.
  • Server and client side are able to build at the same time; it’s not implementation aware.
  • If you adopt “Code-first”, you have to be aware about the name conventions for the classes you declare. For instance, if you use the “DTO” suffix convention, may be this “DTO” suffix will be exposed in your API and it sucks.

The last survivor

Disculpad por este post tan largo pero creo que merece la pena, al menos para mí.

Después de 2 años, 8 meses y 14 días, mi etapa en Yaap llega a su fin. Hasta ahora, siempre había tenido la oportunidad de despedirme de mis compañeros pero, en esta ocasión, soy el último superviviente 🙂 Así que aprovecho este medio para “despedirme” de todas las personas con las que he tenido la gran fortuna de trabajar.

Cierto es que, a lo largo de estos años, he trabajado en empresas donde había mucha calidad profesional y personal, pero en esta última etapa, en una start-up como Yaap, el ambiente ha sido distinto. Todos los departamentos están juntos, todos trabajando para sacar unos productos adelante con el máximo esfuerzo y siempre con buen ambiente y trabajando codo con codo con los compañeros, sean o no de tu mismo departamento.

Durante este tiempo, han pasado muchas personas por la Compañía, y de todas y cada una, me llevo gran recuerdo, incluso con las que haya podido tener altibajos (el camino no siempre ha sido de rosas), y por eso, me quedo con lo mejor, porque de todos he aprendido. Nunca había visto tantas buenas personas juntas en un mismo proyecto; y cuando me refiero a buenas personas, lo digo en todos los ámbitos.

Al final, me quedo con muchos amigos y espero tener la oportunidad algún día de poder volver a trabajar con ellos. Y si no, siempre quedará el Casa Tua 🙂

Un fuerte abrazo a todos

Trazabilidad en las peticiones

En este mundillo donde cada aplicación/servicio se ha de comunicar con otras tantas, es crucial mantener una trazabilidad entre los servicios, de forma que podamos revisar las trazas y distinguir las peticiones/respuestas, recuperando aquellas que se correspondan con una petición concreta.

Para poder correlar las peticiones, necesitamos un identificador único que se propague en todas las peticiones hacia el resto de aplicaciones/servicios y que éstos, para cada traza que escriban relacionada con esa petición, muestren el identificador en la traza.

Continue reading “Trazabilidad en las peticiones”