Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Encoding Extension] Added module outline #23000

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ extension/asapauthextension/ @open-telemetry/collect
extension/awsproxy/ @open-telemetry/collector-contrib-approvers @Aneurysm9 @mxiamxia
extension/basicauthextension/ @open-telemetry/collector-contrib-approvers @jpkrohling @svrakitin
extension/bearertokenauthextension/ @open-telemetry/collector-contrib-approvers @jpkrohling @pavankrish123 @frzifus
extension/encodingextension/ @open-telemetry/collector-contrib-approvers @MovieStoreGuy
extension/headerssetterextension/ @open-telemetry/collector-contrib-approvers @jpkrohling @kovrus
extension/healthcheckextension/ @open-telemetry/collector-contrib-approvers @jpkrohling
extension/httpforwarder/ @open-telemetry/collector-contrib-approvers @atoulme @rmfitzpatrick
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ body:
- extension/awsproxy
- extension/basicauth
- extension/bearertokenauth
- extension/encoding
- extension/headerssetter
- extension/healthcheck
- extension/httpforwarder
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ body:
- extension/awsproxy
- extension/basicauth
- extension/bearertokenauth
- extension/encoding
- extension/headerssetter
- extension/healthcheck
- extension/httpforwarder
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/other.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ body:
- extension/awsproxy
- extension/basicauth
- extension/bearertokenauth
- extension/encoding
- extension/headerssetter
- extension/healthcheck
- extension/httpforwarder
Expand Down
10 changes: 5 additions & 5 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,11 @@ updates:
schedule:
interval: "weekly"
day: "wednesday"
- package-ecosystem: "gomod"
directory: "/extension/encodingextension"
schedule:
interval: "weekly"
day: "wednesday"
- package-ecosystem: "gomod"
directory: "/extension/headerssetterextension"
schedule:
Expand Down Expand Up @@ -1097,8 +1102,3 @@ updates:
schedule:
interval: "weekly"
day: "wednesday"
- package-ecosystem: "gomod"
directory: "/receiver/statsdreceiver"
schedule:
interval: "weekly"
day: "wednesday"
1 change: 1 addition & 0 deletions extension/encodingextension/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
72 changes: 72 additions & 0 deletions extension/encodingextension/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Encoding extension

MovieStoreGuy marked this conversation as resolved.
Show resolved Hide resolved
<!-- status autogenerated section -->
| Status | |
| ------------- |-----------|
| Stability | [development] |
| Distributions | [] |

[development]: https://github.com/open-telemetry/opentelemetry-collector#development
<!-- end autogenerated section -->

The encoding extension allows for other components to reference ingress/egress data formats
to allow for simplified operability.

_🚧 Under active development 🚧_

## Component Milestones

To help track what work needs to be done with this component, these are the currently active goals being
worked towards.

### Development

- CodecFactory: Allows other components to reference code by name and version
- Codec: Defining the expected interface for a codec
- Length Delimited Codec: Used fixed length payloads to represent a collection of data that can be merged together
- Add existing formats to the codec's libraries.
- Add support to the following components
- `file receiver`
- `file exporter`
- `kafka receiver`
- `kafka exporter`
- `kinesis exporter`
- Add codecs for open source formats, ie: `otlp`, `zipkin`, `jaeger`

Component support will be behind a feature gate to avoid change in behavior

### Alpha

- Adopt vendor formats as codecs
- Transition feature gate to default to using encoding extension
- Add dynamic mapping support for `avro`, `parquet`, and `gotemplate`
- Not fixed on these but I am aware that users are interested in these

### Beta

- Add support for remote schema management
- Using `AWS Glue`, `kafka schema registry`
- Performance break down of difference codecs (static, dynamic, remote)
- This is more to help provide information to users to expected performance impact.

### Stable

- All features complete
- Absent of bugs that impact core behaviours
- Dependencies can also be marked as stable

## Example configuration

```yaml
extensions:
encoding:

receivers:
kafka:
encoding:
name: zipkin/v1
format: proto
# ... other configuration values
```

[translators]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/translator
5 changes: 5 additions & 0 deletions extension/encodingextension/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

//go:generate mdatagen metadata.yml
package encodingextension // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/encodingextension"
31 changes: 31 additions & 0 deletions extension/encodingextension/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
module github.com/open-telemetry/opentelemetry-collector-contrib/extension/encodingextension

go 1.19

require go.opentelemetry.io/collector/component v0.81.0

require (
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/knadh/koanf v1.5.0 // indirect
github.com/knadh/koanf/v2 v2.0.1 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
go.opentelemetry.io/collector/config/configtelemetry v0.81.0 // indirect
go.opentelemetry.io/collector/confmap v0.81.0 // indirect
go.opentelemetry.io/collector/featuregate v1.0.0-rcv0013 // indirect
go.opentelemetry.io/collector/pdata v1.0.0-rcv0013 // indirect
go.opentelemetry.io/otel v1.16.0 // indirect
go.opentelemetry.io/otel/metric v1.16.0 // indirect
go.opentelemetry.io/otel/trace v1.16.0 // indirect
go.uber.org/atomic v1.10.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.24.0 // indirect
golang.org/x/net v0.11.0 // indirect
golang.org/x/sys v0.9.0 // indirect
golang.org/x/text v0.10.0 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
google.golang.org/grpc v1.56.1 // indirect
google.golang.org/protobuf v1.31.0 // indirect
)
Loading