Ascend Money TrueMoney

Introduction and Scope

• Ascend Money is a regional FinTech organization providing innovative financial services in the ASEAN Region
• One Engineer works part-time in Zipkin Server and enabling the teams to use Zipkin and instrument their code by themselves
• Most of the instrumentation will be via Spring-Cloud-Sleuth and since most of the services will be migrated to Spring Boot
• We use EFK for logging and Micrometer for metrics

System Overview

• Ascend Thailand has ~300 engineers managing >200 services written primarily in Java (Spring legacy and Spring Boot)

• Services are spread in Multiple datacenters in both on-premise and cloud

• Service communication is generally over Http

• Messaging components:

  • RabbitMQ
  • Redis

• Platform:

  • Marathon/Mesos and AWS ECS (Current)
  • OpenShift and Google GCP (Currently migrating)

Goals

• Use Zipkin to understand the latencies caused and bottlenecks and the response time of various request during system failure
• Measure latency improvements before and after refactoring the services
• Replace custom correlation Ids and use the trace id for log correlation via sleuth
• Have all the services instrumented and use Zipkin when we migrate to OpenShift
• Understand the system architecture using Zipkin Dependencies and also identify non-conformant service communications that deviates from the design

Current Status (16th October, 2018)

• 50+ services using Zipkin in the non-production and 40+ in production. Mainly in the AWS
• By end of this year we are aiming to increase it to 100+ for mostly cutting across our critical paths
• In production, we have 6 instances of Zipkin collectors (3 for each Http and Kinesis collector)

We are reusing the Elastic Search clusters(10 Nodes) setup for Logging Infrastructure. And keep the span data for 14 days. Our current size of spans everyday is ranging from ~2-4GB.