Implementation of span2 format

[codefromthecrypt]

This is the implementation of #1499 Not all steps are sequential

Overview

This work supports a new data type which represents a single-host view of an operation. It will be represented in json w/ optional proto3 mapping.

Full description is in #1499, but the format is below:

{
  "traceId": 32lowerHexCharacters,
  "parentId": 16lowerHexCharactersOrAbsentForRoot,
  "id": 16lowerHexCharacters,
  "kind": enum(CLIENT|SERVER|PRODUCER|CONSUMER|Absent),
  "name": stringOrAbsent,
  "timestamp": uint53EpochMicrosOrAbsentIfIncomplete,
  "duration": uint53MicrosOrAbsentIfIncomplete,
  "localEndpoint": existingEndpointTypeOrAbsent,
  "remoteEndpoint": existingEndpointTypeOrAbsent,
  "annotations": [
    {"timestamp": uint53EpochMicros, "value": string},
    ...
  ],
  "tags": {
    string: string,
    ...
  },
  "debug": trueOrAbsent,
  "shared": trueOrAbsent
}

Implementation overview

As we are working in the brown field, various phases can work at their own pace. Tracer libraries should be able to opt-into this without a breaking change to user apis. Any storage or transport work should anticipate mixed formats, but allow the ability to strictly accept the new one. The UI is completely decoupled from this, as all read apis still use the old format. We may revisit such later.

Basic library support

Since the server code is written in java, a first step could be to write the java library

• Create a working java Span2 type Adds zipkin.internal.Span2 #1669
  • This allows it to coexist in the classpath with existing model
• Make a conversion utility between the original and new model from Span2 (splitting as necessary) Adds zipkin.internal.Span2Converter #1670
• Create a Span2 json codec Adds zipkin.internal.Span2Codec.JSON #1671
• Make a detecting json decoder which can tell the difference between a "simple span" list and a normal one Accepts Zipkin v2 Span format in all current transports #1684
  • This allows us to decode "simple spans" from existing transports
  • the localEndpoint field is the best signal you are in a "simple span" as it is required and not present formerly.
• create zipkin2 package which formalizes the Span2 library (so that other java libraries like reporter can use it directly)

Upload Api Definition

While new servers can detect based on content, we should make it possible for new sites to strictly use the new type.

• Decide whether to use a new path (/api/v2/spans) or a media type header used when not auto-detecting. Ex. application/vnd.zipkin.span2s+json Accepts Zipkin v2 Span format in all current transports #1684
• Update the zipkin api model with the Span2 type Adds openapi spec for POST /api/v2/spans including the json format zipkin-api#33
• investigate a proto3 file to match json encoding
  • probably better to not try to match json encoding as proto3 has quirks like wrapper types for null (needed for parentId and debug). Also, in process identifiers as uint64 probably still makes sense.

Storage integration

Once the type + codec library is in, we can start using it on the backend even if the incoming data is in the old format.

• Create a Call api to eliminate double-interfaces for sync and async Adds v2 Call interface based on okhttp and retrofit #1705
• Create SpanConsumer api for v2 Organizes v2 classes into corresponding internal packages #1702 Adds v2 Call interface based on okhttp and retrofit #1705
• Create SpanStore api for v2 Adds v2 SpanStore and a bridging adapter to v1 #1709
• At the very least, elasticsearch could use this model, under a type span2 Adds Elasticsearch 6.x support using Span2 model #1674
  • queries can be supported in compat mode to check both types
• StackDriver trace translation Uses latest zipkin library in support of simplified json format zipkin-gcp#43
• Azure Insights may also be able to use this type

Transport integration

Once the type + codec library is in, we can start using it for transport even if the storage layer uses the old model.

• Add support to http and kafka collectors. Accepts Zipkin v2 Span format in all current transports #1684
• Add support for AWS Kinesis, SQS senders/collectors. Adds support for json encoding, notably zipkin2 format zipkin-aws#48
• Add support for Azure Event Hub collectors. Accepts zipkin2 json span format openzipkin-attic/zipkin-azure#37
• Remove double-conversion on transport (span2 -> span -> span2) Removes double-conversion when collecting into Elasticsearch #1700
• Update brave and any other willing tracers
  • In java, the span type is a generic type parameter, so will have some code impact

Dependency linking integration

The Span2 type is a better fit for dependency links and can replace the internal type currently used.

• Make DependencyLinker use Span2 internally and remove the DependencyLinkSpan type Drops internal DependencyLinkSpan for Span2 #1673
• Release zipkin-dependencies that uses it
• Consider making DependencyLinker not internal anymore

[codefromthecrypt]

notified folks:

https://groups.google.com/d/msg/zipkin-user/IZr450uTISA/TjhUgsykAQAJ
https://twitter.com/zipkinproject/status/882140150632808448

[codefromthecrypt]

here's the first work on the java binding and codec #1651

[codefromthecrypt]

Update: all conversion tests work. I'm going to do a POC on elasticsearch to make sure it works. Once it does, I'll chop up #1651 into several smaller works

[codefromthecrypt]

@xeraa quick question. Is there an option besides spark to do a data migration?

Ex. I'd like to walk through documents grouped by traceID then write them back modified to a different index.

[xeraa]

@adriancole multiple options:

1. Reindex API: As long as you are using Elasticsearch 5.0+ and can use a query to address the right documents that's the easiest solution. You can also use scripting to change the documents:

POST _reindex
{
  "source": {
    "index": "source_index",
    "query": {
      "match": {
        "foo": "bar"
      }
    }
  },
  "dest": {
    "index": "destination_index"
  },
  "script": {
    "inline": "if (ctx._source.foo == 'bar') {ctx._version++; ctx._source.remove('foo')}",
    "lang": "painless"
  }
}

2. If you need something more complex, we normally rely on Logstash with Elasticsearch as the input and output and a filter with the selection and transformation in the middle.

[codefromthecrypt]

thx for the help @xeraa

[codefromthecrypt]

#1700 makes the write api span2 native. I'll make a later pull request for the read side. Notably, I'm not going to carry over the merging/clock skew adjustment logic currently copy-pasted into every impl. This simplifies the storage contract which is raw by default now. The merging/clock-skew can be done in an api decorator and/or javascript.

[codefromthecrypt]

#1705 hones the way we'll address callbacks (ex span consumption) and synchronous calls (ex span name lists). This will reduce the effort in implementing v2 interfaces

[codefromthecrypt]

draft of the v2 java read api #1709

[codefromthecrypt]

#1711 << elasticsearch uses spanstore2 api
#1710 << expose spanstore2 http api

[codefromthecrypt]

#1726 << finishes the v2 model (and codec ops required). This doesn't make a v2 "storage component" type, yet, as this can be done later

[codefromthecrypt]

#1729 << adds the v2 storage component

[codefromthecrypt]

openzipkin/zipkin-api#47 starts on the proto3 encoding

[codefromthecrypt]

think we're all done