Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal for language additions to OTTL #30800

Open
jsuereth opened this issue Jan 26, 2024 · 24 comments
Open

Proposal for language additions to OTTL #30800

jsuereth opened this issue Jan 26, 2024 · 24 comments
Labels
discussion needed Community discussion needed enhancement New feature or request pkg/ottl Stale

Comments

@jsuereth
Copy link
Contributor

jsuereth commented Jan 26, 2024

Component(s)

pkg/ottl

Is your feature request related to a problem? Please describe.

We were attempting to use OTTL to transform Google's structured logging format in GKE (see: https://cloud.google.com/logging/docs/structured-logging) to the current example data model in OpenTelemetry (see: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/data-model-appendix.md#google-cloud-logging).

Effectively, we're trying to parse a JSON log body and extract components into OTLP equivalents:

{
  "severity":"ERROR",
  "message":"There was an error in the application.",
  "httpRequest":{
    "requestMethod":"GET"
  },
  "times":"2020-10-12T07:20:50.52Z",
  "logging.googleapis.com/insertId":"42",
  "logging.googleapis.com/labels":{
    "user_label_1":"value_1",
    "user_label_2":"value_2"
  },
  "logging.googleapis.com/operation":{
    "id":"get_data",
    "producer":"github.com/MyProject/MyApplication",
     "first":"true"
  },
  "logging.googleapis.com/sourceLocation":{
    "file":"get_data.py",
    "line":"142",
    "function":"getData"
  },
  "logging.googleapis.com/spanId":"000000000000004a",
  "logging.googleapis.com/trace":"projects/my-projectid/traces/06796866738c859f2f19b7cfb3214824",
  "logging.googleapis.com/trace_sampled":false
}

In doing so we noticed a lot of friction in OTTL and duplicate expressions.

Here's a "simplified" (i.e. still missing some if/where statements, and new required built-in functions for span processing) version:

context: log
statements:
- set(body, ParseJSON(body["message"])) where (body != nil and body["message"] != nil)
- merge_maps(attributes, body["logging.googleapis.com/labels"], "upsert") where body["logging.googleapis.com/labels"] != nil
- delete_key(body, "logging.googleapis.com/labels") where (body != nil and body["logging.googleapis.com/labels"] != nil)
- delete_key(cache, "__field_0") where (cache != nil and cache["__field_0"] != nil)
- set(cache["__field_0"], body["logging.googleapis.com/httpRequest"]) where (body != nil and body["logging.googleapis.com/httpRequest"] != nil)
- delete_key(body, "logging.googleapis.com/httpRequest") where (body != nil and body["logging.googleapis.com/httpRequest"] != nil)
- set(cache["value"], cache["__field_0"])
- set(attributes["gcp.http_request"], cache["value"]) where (cache != nil and cache["value"] != nil)
- delete_key(cache, "__field_0") where (cache != nil and cache["__field_0"] != nil)
- set(cache["__field_0"], body["logging.googleapis.com/logName"]) where (body != nil and body["logging.googleapis.com/logName"] != nil)
- delete_key(body, "logging.googleapis.com/logName") where (body != nil and body["logging.googleapis.com/logName"] != nil)
- set(cache["value"], cache["__field_0"])
- set(attributes["gcp.log_name"], cache["value"]) where (cache != nil and cache["value"] != nil)
- delete_key(cache, "__field_0") where (cache != nil and cache["__field_0"] != nil)
- set(cache["__field_0"], body["logging.googleapis.com/severity"]) where (body != nil and body["logging.googleapis.com/severity"] != nil)
- delete_key(body, "logging.googleapis.com/severity") where (body != nil and body["logging.googleapis.com/severity"] != nil)
- set(cache["value"], cache["__field_0"])
- set(severity_text, cache["value"]) where (cache != nil and cache["value"] != nil)
- delete_key(cache, "__field_0") where (cache != nil and cache["__field_0"] != nil)
- set(cache["__field_0"], body["logging.googleapis.com/sourceLocation"]) where (body != nil and body["logging.googleapis.com/sourceLocation"] != nil)
- delete_key(body, "logging.googleapis.com/sourceLocation") where (body != nil and body["logging.googleapis.com/sourceLocation"] != nil)
- set(cache["value"], cache["__field_0"])
- set(attributes["gcp.source_location"], cache["value"]) where (cache != nil and cache["value"] != nil)
- delete_key(cache, "__field_0") where (cache != nil and cache["__field_0"] != nil)
- set(cache["__field_0"], body["logging.googleapis.com/spanId"]) where (body != nil and body["logging.googleapis.com/spanId"] != nil)
- delete_key(body, "logging.googleapis.com/spanId") where (body != nil and body["logging.googleapis.com/spanId"] != nil)
- set(cache["value"], cache["__field_0"])
- set(span_id, cache["value"]) where (cache != nil and cache["value"] != nil)
- delete_key(cache, "__field_0") where (cache != nil and cache["__field_0"] != nil)
- set(cache["__field_0"], body["logging.googleapis.com/trace"]) where (body != nil and body["logging.googleapis.com/trace"] != nil)
- delete_key(body, "logging.googleapis.com/trace") where (body != nil and body["logging.googleapis.com/trace"] != nil)
- set(cache["value"], cache["__field_0"])
- set(trace_id, cache["value"]) where (cache != nil and cache["value"] != nil)

Describe the solution you'd like

We'd like to propose a new expression-focused syntax for OTTL that would allow the previous OTTL to look like this:

on log
when log.body is aStringMessage(parsedJson(json))
yield log with {
  attributes: attributes with json["logging.googleapis.com/labels"] with {
    "gcp.log_name": json["logging.googleapis.com/logName"]
  },
  spanID: StringToSpanID(json["logging.googleapis.com/spanId"]),
  traceID: StringToTraceID(json["logging.googleapis.com/trace"]),
  severity_text: json["logging.googleapis.com/severity"],
  body: json with {
    "logging.googleapis.com/labels": nil,
    "logging.googleapis.com/logName": nil,
    "logging.googleapis.com/severity": nil,
    "logging.googleapis.com/sourceLocation": nil,
    "logging.googleapis.com/spanId": nil,
    "logging.googleapis.com/trace": nil,
    "logging.googleapis.com/labels": nil,
  },
}

At a high level we propose the following:

  • Supporting string-formatting literals, e.g. "I can reference {expression}s"
  • Add a type system for better "prior to evaluation" error messages, including the ability to
    get error messages without running the collector. (e.g. Go, Rust, Typescript)
  • Allow operations to operate against structural data, preferably with a JSON-like feel. (e.g. Jsonnet TypeScript, Dart)
    • Assign multiple values at the same time.
    • Have the visual structure mirrored in the code.
  • Provide List comprehensions (e.g. Typescript, Python, Kotlin) that simplifies operating against
    lists of "KeyValueList" i.e. Attributes.
    • This should dramatically reduce need for built-in functions to be worthwhile.
    • This should replace need for: limit, truncate_all, replace_*, keep_keys, delete_keys.
  • Pattern matching or "binding patterns", to simplify dealing with generic AnyValue attributes and log bodies, reducing the need to duplicate intent between where clause and statements.
  • (optional) - Only reserve new keywords in "stateful" context. (This would require migrating to a stateful lexer, but could help avoid reducing the available identifiers).

Describe alternatives you've considered

We investigated leveraging CEL or LUA for this purpose.

Unfortunately there are a few shortcomings we think this proposal would alleviate:

  • CEL lacks anonymous structural definitions. All key-value objects must be constructed as full types.
  • CEL + LUA lack pattern-matching syntax. We believe pattern matching, with explicit "guard" vs. "execution" interpretation can dramatically simplify many OTTL expressions.
  • CEL is inherently expression-based (returns a value) while OTTL is inherently imperative (mutates values)
  • LUA is a full fledged programming language and could require more care to limit runtime complexity.

Additional context

I have a fully implemented prototype "trans-piler" which can take this new syntax and backport OTTL statements from it. This prototype includes grammar suggestions and rationale.

I would like to consider whether OTTL should expand its expression prior to #28892.

@jsuereth jsuereth added enhancement New feature or request needs triage New item requiring triage labels Jan 26, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@TylerHelmuth
Copy link
Member

@jsuereth thanks for taking the time to dive into OTTL. There is a lot to comprehend here and it is going to take some time for me to ingest everything. I think this would be a very good topic for a Collector SIG meeting.

@TylerHelmuth
Copy link
Member

TylerHelmuth commented Jan 26, 2024

@jsuereth since this isnt a PR yet I'm gonna supply some feedback from https://github.com/jsuereth/ottl-proposal/blob/main/PROPOSAL.md here

feedback on the complaints section

inconsistent and surprising syntax limitations (parentheses cannot be safely added around any expression, Boolean values can't be used as conditions without appending == true, time values can't be stored in cache fields, etc)

I believe all boolean or mathematical expressions can have () safely added as long as the result is still a boolean/mathematical expressiong. Do you have an example where this is not working.

Boolean values can now be used in conditions directly: #20911. Do you have an example where this is not working?

See #26108 for the cache and time values.

Isn't set(attributes["key"], nil) the same as delete_key(attributes, "key")

Currently set is no-op when the value is nil: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/ottlfuncs/func_set.go. I agree that isn't clear enough.

Inconsistency in identifier lookups. trace_id.string and trace_id["string"] don't refer to the same thing even though body.string and body["string"] do.

We've recently added a feature so that indexing fields that cannot be index throws an error on startup. body.string and body["string"] don't refer to the same thing: the former returns the body as a string and the latter requires the body to be a map and returns the value in the map at "string". I'll argue our path's have been consistent with the use of . to separate segments and what can and cannot be indexed.

feedback on the proposal section

  • I'd absolutely love a type system. I've thought a little bit about how the context could tell the parse the type of a path and then how the parser could use that knowledge when creating the function, but I've never gotten far.
  • I don't understand the pattern matching section yet. If the goal is to have a common where clause there is [processor/transform] Allow common where clause #27830, although I recognize that this doesn't solve the problem directly in OTTL
  • At a brief glance I don't comprehend yet what these 2 expressions do, can you go into more detail?
    attributes: attributes with json["logging.googleapis.com/labels"] with {
     "gcp.log_name": json["logging.googleapis.com/logName"]
    },
    
    body: json with {
     "logging.googleapis.com/labels": nil,
     "logging.googleapis.com/logName": nil,
     "logging.googleapis.com/severity": nil,
     "logging.googleapis.com/sourceLocation": nil,
     "logging.googleapis.com/spanId": nil,
     "logging.googleapis.com/trace": nil,
     "logging.googleapis.com/labels": nil,
    },
    

@TylerHelmuth
Copy link
Member

TylerHelmuth commented Jan 26, 2024

For the comparison to existing OTTL syntax, I'd like to submit my attempt at parsing a JSON log to specification. I probably misinterpreted some stuff, so please let me know if I've done something wrong:

Input:

{
  "severity":"ERROR",
  "message":"There was an error in the application.",
  "httpRequest":{
    "requestMethod":"GET"
  },
  "times":"2020-10-12T07:20:50.52Z",
  "logging.googleapis.com/insertId":"42",
  "logging.googleapis.com/labels":{
    "user_label_1":"value_1",
    "user_label_2":"value_2"
  },
  "logging.googleapis.com/operation":{
    "id":"get_data",
    "producer":"github.com/MyProject/MyApplication",
     "first":"true"
  },
  "logging.googleapis.com/sourceLocation":{
    "file":"get_data.py",
    "line":"142",
    "function":"getData"
  },
  "logging.googleapis.com/spanId":"000000000000004a",
  "logging.googleapis.com/trace":"projects/my-projectid/traces/06796866738c859f2f19b7cfb3214824",
  "logging.googleapis.com/trace_sampled":false
}

transformprocessor config:

  transform:
    error_mode: ignore
    log_statements:
      - context: log
        statements:
          - merge_maps(cache, ParseJSON(body), "upsert") where IsMatch(body, "^\\{")

          - set(time, Time(cache["times"], "%Y-%m-%dT%H:%M:%S.%f%z")) where cache["times"] != nil

          - set(severity_number, SEVERITY_NUMBER_DEBUG) where cache["severity"] == "DEBUG"
          - set(severity_number, SEVERITY_NUMBER_INFO) where cache["severity"] == "INFO"
          - set(severity_number, SEVERITY_NUMBER_WARN) where cache["severity"] == "WARNING"
          - set(severity_number, SEVERITY_NUMBER_ERROR) where cache["severity"] == "ERROR"
          - set(severity_number, SEVERITY_NUMBER_FATAL) where cache["severity"] == "CRITICAL"

          - set(attributes["gcp.http_request"], cache["httpRequest"]) where cache["http_request"] != nil

          - set(span_id.string, cache["logging.googleapis.com/spanId"]) where cache["logging.googleapis.com/spanId"] != nil

          - replace_pattern(cache["logging.googleapis.com/trace"], "projects/.*/traces/([\\w\\d]+)", "$$1") where cache["logging.googleapis.com/trace"] != nil
          - set(trace_id.string, cache["logging.googleapis.com/trace"]) where cache["logging.googleapis.com/trace"] != nil

          - delete_key(cache, "times")
          - delete_key(cache, "severity")
          - delete_key(cache, "httpRequest")
          - delete_key(cache, "logging.googleapis.com/trace")
          - delete_key(cache, "logging.googleapis.com/spanId")
          - flatten(cache, prefix="gcp")
          - merge_maps(attributes, cache, "insert")

output:

Resource SchemaURL: 
ScopeLogs #0
ScopeLogs SchemaURL: 
InstrumentationScope  
LogRecord #0
ObservedTimestamp: 2024-01-26 23:21:16.374373 +0000 UTC
Timestamp: 2020-10-12 07:20:50.52 +0000 UTC
SeverityText: 
SeverityNumber: Error(17)
Body: Str({"severity":"ERROR","message":"There was an error in the application.","httpRequest":{    "requestMethod":"GET"},"times":"2020-10-12T07:20:50.52Z","logging.googleapis.com/insertId":"42","logging.googleapis.com/labels":{    "user_label_1":"value_1",    "user_label_2":"value_2"},"logging.googleapis.com/operation":{    "id":"get_data",    "producer":"github.com/MyProject/MyApplication",    "first":"true"},"logging.googleapis.com/sourceLocation":{    "file":"get_data.py",    "line":"142",    "function":"getData"},"logging.googleapis.com/spanId":"000000000000004a","logging.googleapis.com/trace":"projects/my-projectid/traces/06796866738c859f2f19b7cfb3214824","logging.googleapis.com/trace_sampled":false})
Attributes:
     -> logging.googleapis.com/spanId: Str(000000000000004a)
     -> times: Str(2020-10-12T07:20:50.52Z)
     -> logging.googleapis.com/sourceLocation: Map({"file":"get_data.py","function":"getData","line":"142"})
     -> httpRequest: Map({"requestMethod":"GET"})
     -> logging.googleapis.com/insertId: Str(42)
     -> logging.googleapis.com/labels: Map({"user_label_1":"value_1","user_label_2":"value_2"})
     -> message: Str(There was an error in the application.)
     -> logging.googleapis.com/operation: Map({"first":"true","id":"get_data","producer":"github.com/MyProject/MyApplication"})
     -> logging.googleapis.com/trace_sampled: Bool(false)
     -> log.file.name: Str(gcp.json)
     -> severity: Str(ERROR)
     -> logging.googleapis.com/trace: Str(projects/my-projectid/traces/06796866738c859f2f19b7cfb3214824)
     -> gcp.logging.googleapis.com/operation.id: Str(get_data)
     -> gcp.logging.googleapis.com/operation.producer: Str(github.com/MyProject/MyApplication)
     -> gcp.logging.googleapis.com/operation.first: Str(true)
     -> gcp.message: Str(There was an error in the application.)
     -> gcp.logging.googleapis.com/insertId: Str(42)
     -> gcp.logging.googleapis.com/labels.user_label_1: Str(value_1)
     -> gcp.logging.googleapis.com/labels.user_label_2: Str(value_2)
     -> gcp.logging.googleapis.com/trace_sampled: Bool(false)
     -> gcp.logging.googleapis.com/sourceLocation.file: Str(get_data.py)
     -> gcp.logging.googleapis.com/sourceLocation.line: Str(142)
     -> gcp.logging.googleapis.com/sourceLocation.function: Str(getData)
Trace ID: 06796866738c859f2f19b7cfb3214824
Span ID: 000000000000004a
Flags: 0
        {"kind": "exporter", "data_type": "logs", "name": "debug"}

I post this only for a more fair syntax comparison because I do not believe the one in the proposal is optimized.

@TylerHelmuth
Copy link
Member

I'll also admit that my knee-jerk reaction is to be defensive haha I really appreciate the time you've spent on this proposal and your desire to improve OTTl. I promise to keep an open mind!

@jsuereth
Copy link
Contributor Author

First, want to thank you for providing more optimal code for the use case! What I posted was our naive attempt with our current understanding of OTTL. It seems good that things can be simplified and I think some of our points were perhaps bugs that are now fixed as opposed to assumptions on syntax.

That said, I still think expanding available "expression" syntax and providing features like pattern matching or structural literals would be ideal to the types of programs you expect in OTTL.

I'm going to answer your questions over the next few days as I can and in separate comments, to hopefully have a good conversation and keep different points and features in separate conversation.

My proposal is here to help. While I stand behind all of it, we can try different pieces initially. I'm mostly looking to make sure when OTTL is stable it is scalable.

This isn't meant to be an attack on the current state, more a vision of where we can grow, and grow (hopefully) without breaking existing users.

@jsuereth
Copy link
Contributor Author

At a brief glance I don't comprehend yet what these 2 expressions do, can you go into more detail?

attributes: attributes with json["logging.googleapis.com/labels"] with {
 "gcp.log_name": json["logging.googleapis.com/logName"]
},
body: json with {
 "logging.googleapis.com/labels": nil,
 "logging.googleapis.com/logName": nil,
 "logging.googleapis.com/severity": nil,
 "logging.googleapis.com/sourceLocation": nil,
 "logging.googleapis.com/spanId": nil,
 "logging.googleapis.com/trace": nil,
 "logging.googleapis.com/labels": nil,
},

This is my idea of a merge operation. The second is meant to be a mechanism of deleting multiple values from the JSON body. x with y is a merge maps method with insert. It takes the right and overrides the left with the value. The nice part of having this be an expression that returns a value is that you can chain it as I do in the first example.

The first examples does the following:

  • inserts the nested key-value pairs from the JSON object found at key "googleapis.com/labels". These are expected to be string-string, but in my prototype parsejson returns the otlp AnyVal so you can basically assign that to attributes.
  • inserts one specific attribute called "gcp log_name" pulled from a different area of the JSON.

The ability to chain the merges keeps the syntax brief but also visually you can see the key-value pairs being set.

@jsuereth
Copy link
Contributor Author

jsuereth commented Jan 27, 2024

I don't understand the pattern matching section yet. If the goal is to have a common where clause there is [processor/transform] Allow common where clause #27830, although I recognize that this doesn't solve the problem directly in OTTL

Pattern matching is designed to do two things:

  • check some condition
  • provide named terms you can use that are guaranteed to be safe.

The goal in this addition is not just sharing the conditional but also gaining a term (name? Reference?) You can use that has some desired type or feature.

Example from scala:

case class Data(x: Int, y: String)

val x: Any = ...
x match {
  case Data(_, name) => ...here I can use "name" as a string...
}

Example from rust:

enum Option<T> {
  None,
  Some(T)
}

match opt_value {
  None => handle_none(),
  Some(value) => handle_value(value),
}

The idea here is that you can chain these patterns too to dramatically simplify both checking a condition and adding a new term.

For OTTL I imagine we'd use this to verify log body AnyVal have specific fields of specific types and give names to those fields we can use in the program.

On log
When body like { some_field: String(name_ill_use) }
set(attributes.new_label, name_ill_use)

Here the user selected name_ill_use. The interpreter will verify some_field exists on body and that it contains a string value. That can be done in the guard/Boolean where clause.

Hope that helps! Let me know if you need more details or examples from other languages.

@TylerHelmuth
Copy link
Member

TylerHelmuth commented Jan 27, 2024

My proposal is here to help. While I stand behind all of it, we can try different pieces initially. I'm mostly looking to make sure when OTTL is stable it is scalable. This isn't meant to be an attack on the current state, more a vision of where we can grow, and grow (hopefully) without breaking existing users.

Absolutely, I highlighting my own shortcomings haha we share the goal of making OTTL scalable and stable. Thanks again for thinking about this!

@TylerHelmuth
Copy link
Member

@jsuereth in the examples where is the variable json coming from?

@TylerHelmuth
Copy link
Member

@jsuereth I'd be really interested in seeing the examples from https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor#examples rewritten with this new syntax. I am curious how non-log/non-json statements look like and how they'll look in a yaml config.

@jsuereth
Copy link
Contributor Author

jsuereth commented Jan 30, 2024

@jsuereth in the examples where is the variable json coming from?

This the pattern matching component:

on log
when log.body is aStringMessage(parsedJson(json))
yield ...

aStringMessage is a function that takes in an AnyVal (log body) and returns Option<String>.
parsedJson is a function that takes in a String and returns Option<AnyVal> (i.e. parsed JSON).

The name json is the name the user has given this result. As you can see, this is an example of chaining "extractors" in the pattern matching.

@TylerHelmuth
Copy link
Member

Oh I see now. I was interpreting aStringMessage(parsedJson(json)) as a chain of function calls with json as the first input to parsedJson and then into aStringMessage, like if it was Go/Python/Java/.NET. Is the syntax you're proposing present in any other programming languages?

@crobert-1
Copy link
Member

I'm going to remove needs triage since this has been acknowledged and is being discussed.

@crobert-1 crobert-1 added discussion needed Community discussion needed and removed needs triage New item requiring triage labels Jan 31, 2024
@jsuereth
Copy link
Contributor Author

jsuereth commented Feb 5, 2024

Oh I see now. I was interpreting aStringMessage(parsedJson(json)) as a chain of function calls with json as the first input to parsedJson and then into aStringMessage, like if it was Go/Python/Java/.NET. Is the syntax you're proposing present in any other programming languages?

It is a chain of function calls, Scala is the language I'm borrowing from heavily here. Rust also has pattern matching with extraction, but doesn't expand to allow custom functions.

@jsuereth
Copy link
Contributor Author

jsuereth commented Feb 8, 2024

@jsuereth I'd be really interested in seeing the examples from https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor#examples rewritten with this new syntax. I am curious how non-log/non-json statements look like and how they'll look in a yaml config.

Tried to provide what examples might look like with proposed language features: https://github.com/jsuereth/ottl-proposal/blob/main/examples.md

Note: If you think it'd be worthwhile for me to tailor language feature proposals into components and what things might look like incrementally adding one feature at a time, let me know. E.g. there are examples I felt pattern matching helps the most, others list-comprehension.

While I do think all the language features would benefit OTTL, I'm happy to entertain a subset of this to contribute.

@TylerHelmuth
Copy link
Member

It is a chain of function calls, Scala is the language I'm borrowing from heavily here. Rust also has pattern matching with extraction, but doesn't expand to allow custom functions.

I am confused by the json entry in aStringMessage(parsedJson(json)). Is json and input or an output of that statement?

@jsuereth
Copy link
Contributor Author

jsuereth commented Feb 15, 2024

I am confused by the json entry in aStringMessage(parsedJson(json)). Is json and input or an output of that statement?

In the context of pattern matching, json is effectively an output.

We have a set of functions/methods called "extractors" which take an input and return Option[...]. The return value then is what's used.

So, for example, if i have a symbol log.body which is a String and a symbol parseJson which is a func (String) Option[Json], then in pattern matching I could use that function as follows:

match log.body {
   case parseJson(json) => // here I can use the term `json` as a `Json` type
   case value => // here I can use the term `value` as a string 
}

here log.body is the input to the "extractor" functions. The return value is what's in the () part of the function call, allowing you to chain MULTIPLE extractors.

Note: The simplest form of pattern matching just lets you extract structure arguments by name, but without any of these fancy function things. I think given JSON + Logging and the need to deal with AnyVal all the time, this feature can be useful. Particularly if adding new "patterns" is as easy as a function with optional return.

Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Apr 16, 2024
@jsuereth
Copy link
Contributor Author

I haven't had much time the past few months to pursue this (due to heavy investment in codegen / semantic-conventions / weaver).

I would like to repursue this with an aim and reducing the ticket/issue count on OTTL with targeted features (or just general OTTL language specification improvement).

@TylerHelmuth If you're amenable, maybe we can set up some time to chat? At a minimum I think given #32080 - The most important thing from this proposal would be to specify what . means formally and then leverage this to lift certain information about log transformations (e.g. understanding when new resources must be synthesized).

@jsuereth jsuereth removed the Stale label May 20, 2024
@evan-bradley
Copy link
Contributor

@jsuereth If you are able to attend, this would be great to discuss at a Collector SIG meeting. We have them every Wednesday at 12:00 EDT / 09:00 PDT. Here's a link to the meeting notes with a link to the Zoom room.

If this time doesn't work for you, I agree a call would be a good way to discuss our options here.

@jsuereth
Copy link
Contributor Author

@evan-bradley That time, unfortunately, conflicts with semconv tooling WG, so I'm unable to attend.

@jsuereth
Copy link
Contributor Author

jsuereth commented Jul 9, 2024

We discussed briefly at OTEL community days. I'll follow up with y'all after my summer vacation/travel have passed.

I think we can be highly focused on a few points of the proposal to make progress against key friction points.

Copy link
Contributor

github-actions bot commented Sep 9, 2024

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion needed Community discussion needed enhancement New feature or request pkg/ottl Stale
Projects
None yet
Development

No branches or pull requests

4 participants