Metrics #480

Half-Shot · 2019-05-22T20:50:01Z

This PR enables metrics support for the bridge. The way we are doing this is slightly weird. Rather than passing around a metrics instance to all of the handlers, I've just made a MetricPeg static class, which will host a IBridgeMetrics instance of your choosing. Personally, this feels more elegant.

Sorunome

Soru is a bit confused, are these metrics only in-memory? are they saved somehow? if yes, how? If it is just a json blob file thingy like old user and room store, wouldn't that introduce a significant bottleneck?

Sorunome · 2019-05-31T15:32:03Z

src/clientfactory.ts

+        // tslint:disable-next-line:only-arrow-functions
+        channel.send = function() {
+            MetricPeg.get.remoteCall("channel.send");
+            return channel.send.apply(channel, arguments);


doesn't this recurse? What if we call bindMetricsToChannel twice on a channel object, won't it do the metrics twice?

Yes hello I'm dumb, I wonder how we do this sanely.

Sorunome · 2019-05-31T15:32:58Z

src/bot.ts

@@ -389,6 +392,7 @@ export class DiscordBot {
            }
            const channel = guild.channels.get(room);
            if (channel && channel.type === "text") {
+                this.ClientFactory.bindMetricsToChannel(channel as Discord.TextChannel);


shouldn't we always bind the channel on message processing? Or even better, right when the message enters via the event listeners?

Nope, because we only call the send function when sending stuff that came via matrix. Though I guess we should also bind in the event listeners too.

Sorunome · 2019-05-31T15:34:18Z

src/bot.ts

@@ -766,14 +776,13 @@ export class DiscordBot {
        let rooms;
        try {
            rooms = await this.channelSync.GetRoomIdsFromChannel(msg.channel);
+            if (rooms === null) { throw Error(); }


new lines for if-condition due to {}?

I think I was being lazy, sorry.

Sorunome · 2019-05-31T15:37:52Z

src/metrics.ts

+    private matrixRequest: Histogram;
+    private requestsInFlight: Map<string, number>;
+    private bridgeGauges: IBridgeGauges = {
+        matrixGhosts: 0,


are these autofilled somehow?

Yes, by the bridge lib. Though some of them will remain 0

Half-Shot · 2019-05-31T15:44:15Z

Soru is a bit confused, are these metrics only in-memory? are they saved somehow? if yes, how? If it is just a json blob file thingy like old user and room store, wouldn't that introduce a significant bottleneck?

By default it blackholes the metrics unless you enable prometheus in the config. If enabled, the bridge will simply add another endpoint onto the bridge's http listener which will spit out a line seperated list of values which are kept in memory. A prometheus instance will then poll this endpoint every X seconds. It's not at all memory intensive since reads are done asyncronously and writes are just updating integer values.

Sorunome · 2019-05-31T15:54:50Z

Soru is a bit confused, are these metrics only in-memory? are they saved somehow? if yes, how? If it is just a json blob file thingy like old user and room store, wouldn't that introduce a significant bottleneck?

By default it blackholes the metrics unless you enable prometheus in the config. If enabled, the bridge will simply add another endpoint onto the bridge's http listener which will spit out a line seperated list of values which are kept in memory. A prometheus instance will then poll this endpoint every X seconds. It's not at all memory intensive since reads are done asyncronously and writes are just updating integer values.

ah, so you need to run another program (prometheus server) to read out the metrics? makes sense, then

…ppservice-discord into develop

Sorunome · 2019-06-04T16:23:09Z

why don't we bind this.clientFactory.bindMetricsToChannel(msg.channel as Discord.TextChannel); at the start of all those discord event listeners? (like edit, delete etc)

Signed-off-by: pacien <pacien.trangirard@pacien.net>

github: closes #487 Signed-off-by: pacien <pacien.trangirard@pacien.net>

fixing compilation with Typescript 3.5.1 github: closes #487 Signed-off-by: pacien <pacien.trangirard@pacien.net>

Fix compilation failure with TS 3.5.1

Override config with environment variables

Signed-off-by: pacien <pacien.trangirard@pacien.net>

github: closes #487 Signed-off-by: pacien <pacien.trangirard@pacien.net>

fixing compilation with Typescript 3.5.1 github: closes #487 Signed-off-by: pacien <pacien.trangirard@pacien.net>

…ord into develop

…iscord into hs/metrics

Half-Shot · 2019-06-13T10:54:11Z

💥

Half-Shot added 4 commits May 22, 2019 22:21

Add support for metrics

639623e

Dammit metrics.ts

c4ae6be

Enable the metrics

3d9a677

Wait for bridge to start first

4397227

Half-Shot force-pushed the hs/metrics branch from 42d0a66 to 4397227 Compare May 22, 2019 21:21

Half-Shot added 2 commits May 22, 2019 22:31

Expire old requests in metrics

663eebc

Store prefixes

d617346

Half-Shot mentioned this pull request May 22, 2019

Profile Caching #481

Merged

Sorunome and others added 3 commits May 26, 2019 22:14

Merge branch 'develop' into release/0.5.0

11a548f

Add support for remote send calls in metrics

2370b4d

Linting

95cb6af

Half-Shot force-pushed the hs/metrics branch from 99c2986 to 95cb6af Compare May 31, 2019 15:21

Add copyright header

3a90721

Sorunome requested changes May 31, 2019

View reviewed changes

Bind metrics safely

04f3007

turt2live and others added 4 commits June 2, 2019 13:49

Fix docker image for appservice bridge dependency

b0810a8

Merge branch 'travis/fix-docker' of https://github.com/t2bot/matrix-a…

a78de0b

…ppservice-discord into develop

Merge branch 'develop' into release/0.5.0

e57769e

Merge branch 'release/0.5.0'

c6e41d9

Sorunome and others added 8 commits June 4, 2019 19:30

fix version

0267969

fix some test assertion calls

8c09564

Signed-off-by: pacien <pacien.trangirard@pacien.net>

add config override using env variables

45d2bf8

Signed-off-by: pacien <pacien.trangirard@pacien.net>

add note for config override using env vars

9827b98

Signed-off-by: pacien <pacien.trangirard@pacien.net>

update p-queue lib

28ecd27

github: closes #487 Signed-off-by: pacien <pacien.trangirard@pacien.net>

disambiguate promise type

d02603c

fixing compilation with Typescript 3.5.1 github: closes #487 Signed-off-by: pacien <pacien.trangirard@pacien.net>

Merge pull request #489 from pacien/pqueue-ts-fix

831f922

Fix compilation failure with TS 3.5.1

Merge pull request #485 from pacien/config-from-env

90ea8c8

Override config with environment variables

Sorunome and others added 24 commits June 13, 2019 10:47

add support for animated discord guild icons

6ed4eb1

remove unneeded type setting

fda7d34

Merge branch 'soru/animated-guild-icons' into develop

d2642ef

fix some test assertion calls

a6cbac2

Signed-off-by: pacien <pacien.trangirard@pacien.net>

add config override using env variables

eb89fc1

Signed-off-by: pacien <pacien.trangirard@pacien.net>

add note for config override using env vars

37d76af

Signed-off-by: pacien <pacien.trangirard@pacien.net>

update p-queue lib

9a47c09

github: closes #487 Signed-off-by: pacien <pacien.trangirard@pacien.net>

disambiguate promise type

ef1872d

fixing compilation with Typescript 3.5.1 github: closes #487 Signed-off-by: pacien <pacien.trangirard@pacien.net>

Add non-warning to readme for installation

c3a142a

add support for animated discord guild icons

1b9d062

remove unneeded type setting

1391753

Merge branch 'develop' of github.com:Half-Shot/matrix-appservice-disc…

0fdb9dc

…ord into develop

Add support for metrics

980a1a2

Dammit metrics.ts

b336eda

Enable the metrics

bdc84f7

Wait for bridge to start first

c9cad35

Expire old requests in metrics

641edc7

Store prefixes

5f245e1

Add support for remote send calls in metrics

062d5b1

Linting

2243030

Add copyright header

cff2bc1

Bind metrics safely

55a1248

More binds

f64340b

Merge branch 'hs/metrics' of github.com:Half-Shot/matrix-appservice-d…

8405046

…iscord into hs/metrics

Half-Shot requested a review from Sorunome June 13, 2019 10:50

Sorunome approved these changes Jun 13, 2019

View reviewed changes

Add a note about metrics

c3eaaa8

Half-Shot merged commit b3a8e7a into develop Jun 13, 2019

jaller94 mentioned this pull request Dec 7, 2020

Add metrics to the config sample and schema #640

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Metrics #480

Metrics #480

Half-Shot commented May 22, 2019

Sorunome left a comment

Sorunome May 31, 2019

Half-Shot May 31, 2019

Sorunome May 31, 2019

Half-Shot May 31, 2019

Sorunome May 31, 2019

Half-Shot May 31, 2019

Sorunome May 31, 2019

Half-Shot May 31, 2019

Half-Shot commented May 31, 2019

Sorunome commented May 31, 2019

Sorunome commented Jun 4, 2019

Half-Shot commented Jun 13, 2019

Metrics #480

Metrics #480

Conversation

Half-Shot commented May 22, 2019

Sorunome left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Half-Shot commented May 31, 2019

Sorunome commented May 31, 2019

Sorunome commented Jun 4, 2019

Half-Shot commented Jun 13, 2019

💥