feat: add E2E tests for cases that peers going offline

[BruceAko]

Description

Verifies that Scheduler can clear offline peer metadata when the host exits normally/abnormally by following steps:

1. Get the number of all hosts in the scheduler.
2. After a normal exit of a host, verify that the number of hosts in the scheduler decreases by 1.
3. After an abnormal exit of a host, verify that the number of hosts in the scheduler decreases by 1 after twice the announce host interval.

Related Issue

#3523

Motivation and Context

Earlier we have added functionality so that hosts send LeaveHost() requests to the Scheduler on normal exit and hosts on abnormal exit are also sensed by the Scheduler GC after a set amount of time and the metadata is cleared. Now, we are going to add E2E tests to verify that this functionality is correct.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation Update (if none of the other choices apply)

Checklist

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 75 lines in your changes missing coverage. Please review.

Project coverage is 51.66%. Comparing base (9cd6f41) to head (d49089d).

Files with missing lines	Patch %	Lines
scheduler/service/service_v2.go	0.00%	69 Missing ⚠️
scheduler/rpcserver/scheduler_server_v2.go	0.00%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3524      +/-   ##
==========================================
- Coverage   51.85%   51.66%   -0.20%     
==========================================
  Files         190      190              
  Lines       20488    20562      +74     
==========================================
- Hits        10625    10624       -1     
- Misses       9054     9128      +74     
- Partials      809      810       +1     

Flag	Coverage Δ
unittests	51.66% <0.00%> (-0.20%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
scheduler/metrics/metrics.go	8.24% <ø> (ø)
scheduler/rpcserver/scheduler_server_v2.go	1.75% <0.00%> (-0.09%)	⬇️
scheduler/service/service_v2.go	75.40% <0.00%> (-5.29%)	⬇️

... and 4 files with indirect coverage changes