[Airflow] Remove metric_type mapping for 'airflow.scheduler.heartbeat' field and adjust the dashboard to visualize this

[ritalwar]

• Bug

What does this PR do?

This PR modifies the metric_type of airflow.scheduler.heartbeat from counter to gauge. Despite the field being labeled as a counter in the airflow documentation, the behavior of the values—consistently 1 or 2—suggests it aligns better with the characteristics of a gauge metric.
Additionally, dashboard is modified to accurately represent this data, with the aggregator function updated to use last_value.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

• Relates Choosing the right metric_type mapping and aggregator function for the Airflow dashboard's scheduler.heartbeat field in the context of TSDB. #7504

Screenshots

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-09-06T11:46:14.840+0000

• Duration: 14 min 27 sec

Test stats 🧪

Test	Results
Failed	0
Passed	2
Skipped	0
Total	2

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-08-28T06:24:48.070+0000

• Duration: 14 min 1 sec

Test stats 🧪

Test	Results
Failed	0
Passed	2
Skipped	0
Total	2

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (0/0)	💚
Files	100.0% (0/0)	💚
Classes	100.0% (0/0)	💚
Methods	25.0% (1/4)	👎 -75.0
Lines	100.0% (0/0)	💚
Conditionals	100.0% (0/0)	💚

[agithomas]

@lalit-satapathy / @ritalwar , I want to propose a third option - not to mention either gauge / counter following the conflicted behaviour we noticed and what is documented. Yes, this would lead to not able to benefit TSDB advantages. We can create a backlog item for metric_type mapping of this field. As we get more clarity, the change to map either to gauge or counter can be made. Thoughts?

[agithomas]

The advantage with the above approach is that the change (and thus the readme) don't contradict the official documentation of couchdb.

[ritalwar]

@lalit-satapathy / @ritalwar , I want to propose a third option - not to mention either gauge / counter following the conflicted behaviour we noticed and what is documented. Yes, this would lead to not able to benefit TSDB advantages. We can create a backlog item for metric_type mapping of this field. As we get more clarity, the change to map either to gauge or counter can be made. Thoughts?

I think we can go with this to avoid any conflicts and keep this issue open. However, let's keep this PR to stick with the 'last_value' as the aggregator function for this visualisation, as discussed. We'll put the metric_type mapping on hold for now. @lalit-satapathy , WDYT?

[ritalwar]

Following our discussion, I've removed the metric_type mapping for the 'scheduler.heartbeat' field and made the necessary updates to the dashboard, utilizing 'last_value' for visualizing this field's data. We can maintain the related issue as open to determine the appropriate metric_type mapping without conflicting with the documentation.

[agithomas]

LGTM!

[elasticmachine]

Package airflow - 0.3.1 containing this change is available at https://epr.elastic.co/search?package=airflow