fix CircleCi Setup Errors #3870
Conversation
.circleci/config.yml
Outdated
| - run: | ||
| name: Run functional tests (edge / https) VOD (Static MPD) | ||
| command: | ||
| node test/functional/runTests.js --selenium=remote --reporters=junit --app=remote --browsers=edge --protocol=https --streams="VOD (Static MPD)" |
There was a problem hiding this comment.
Set default to Chrome please
| @@ -133,6 +133,88 @@ jobs: | |||
| - store_test_results: | |||
| path: test/functional/reports | |||
|
|
|||
There was a problem hiding this comment.
Remove functional-tests-all if not required anymore
.circleci/config.yml
Outdated
| name: Run functional tests (edge / https) LIVE (Dynamic MPD) | ||
| when: always # run tests even if some previous tests failed | ||
| command: | ||
| node test/functional/runTests.js --selenium=remote --reporters=junit --app=remote --browsers=edge --protocol=https --streams="LIVE (Dynamic MPD)" |
There was a problem hiding this comment.
Is it possible to define a variable for the browser? Instead of writing edge or chrome multiple times define that globally?
There was a problem hiding this comment.
Yes, jobs can be made configurable
https://circleci.com/docs/2.0/reusing-config/
i suggest defining ONE generic functional test job and then run it with params like:
- functional-test
browsers: "edge"
protocol: "https"
streams: "Live Low Latency"
- functional-test
browsers: "chrome"
protocol: "https"
streams: "Live Low Latency"
.circleci/config.yml
Outdated
| command: | ||
| node test/functional/runTests.js --selenium=remote --reporters=junit --app=remote --browsers=edge --protocol=https --streams="LIVE (Dynamic MPD)" | ||
| - run: | ||
| name: Run functional tests (edge / https) Live Low Latency |
There was a problem hiding this comment.
I assume the different tasks are processed sequentially. Is it possible to run them in parallel? Do we need to split this to separate jobs then?
There was a problem hiding this comment.
agree, we should use jobs instead of sequential steps! Again, this can be done very elegant using configurable jobs. Jobs ran in a workflow are executed in parallel
| @@ -158,5 +240,7 @@ workflows: | |||
| only: | |||
| - development | |||
| jobs: | |||
| - functional-tests-all | |||
There was a problem hiding this comment.
@vrosenberg if functional-tests-all job is removed and substituted by others then the "dead" configuration should be removed.
There was a problem hiding this comment.
see this comment as obsolete in favor to rework using configurable jobs
.circleci/config.yml
Outdated
| - functional_precondition | ||
| - checkout | ||
| - dependencies_setup | ||
| - build_unit_test_steps | ||
| - functional_test_setup |
There was a problem hiding this comment.
let's create a new re-usable command that includes this 5 steps and use the command. i believe it's better since the steps are copied already 3 times
e.g.
commands:
functional_steps:
steps:
- functional_precondition
- checkout
- dependencies_setup
- build_unit_test_steps
- functional_test_setup
and then use...
functional-tests-without-DRM:
executor: dashjs-executor
steps:
- functional_steps
- run:
wdyt @vrosenberg ?
There was a problem hiding this comment.
Sounds good, it reduces redundancy and increases readability
.circleci/config.yml
Outdated
| name: Run functional tests (edge / https) LIVE (Dynamic MPD) | ||
| when: always # run tests even if some previous tests failed | ||
| command: | ||
| node test/functional/runTests.js --selenium=remote --reporters=junit --app=remote --browsers=edge --protocol=https --streams="LIVE (Dynamic MPD)" |
There was a problem hiding this comment.
Yes, jobs can be made configurable
https://circleci.com/docs/2.0/reusing-config/
i suggest defining ONE generic functional test job and then run it with params like:
- functional-test
browsers: "edge"
protocol: "https"
streams: "Live Low Latency"
- functional-test
browsers: "chrome"
protocol: "https"
streams: "Live Low Latency"
.circleci/config.yml
Outdated
| command: | ||
| node test/functional/runTests.js --selenium=remote --reporters=junit --app=remote --browsers=edge --protocol=https --streams="LIVE (Dynamic MPD)" | ||
| - run: | ||
| name: Run functional tests (edge / https) Live Low Latency |
There was a problem hiding this comment.
agree, we should use jobs instead of sequential steps! Again, this can be done very elegant using configurable jobs. Jobs ran in a workflow are executed in parallel
* VideoModel: fix returned current time and seeking state according to current seeking value * VideoModel: fix returned current time and seeking state according to current seeking value * VideoModel: fix returned current time and seeking state according to current seeking value
|
The functional tests connect to a BrowserStack session. In the current plan it only allows for 5 parallel automated testing sessions. I would implement a combination of parallelizing 5 jobs, which include multiple steps within in order to spread out the workload and reduce the time for test execution. |
|
updated PR is here |
Multiple Browserstack Sessions for functional tests in order to not exceed 2h timeout limit
Tests split into multiple CircleCI Jobs to not exceed 3h timeout on current circleci plan