Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Colab Notebook Integration Tests #26690

Merged
merged 11 commits into from
May 18, 2023
Merged

Colab Notebook Integration Tests #26690

merged 11 commits into from
May 18, 2023

Conversation

svetakvsundhar
Copy link
Contributor

Fixes #26604.

This PR adds a script to test the cells output, and compare it to a .spec file that will need to be provided in Colab submission PRs. This PR also adds a presubmit check to ensure this on Jenkins.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI.

@svetakvsundhar svetakvsundhar marked this pull request as draft May 14, 2023 23:03
@codecov
Copy link

codecov bot commented May 14, 2023

Codecov Report

Merging #26690 (bbda4d0) into master (44a17cb) will decrease coverage by 0.13%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master   #26690      +/-   ##
==========================================
- Coverage   81.07%   80.94%   -0.13%     
==========================================
  Files         469      471       +2     
  Lines       67199    67629     +430     
==========================================
+ Hits        54483    54745     +262     
- Misses      12716    12884     +168     
Flag Coverage Δ
python 80.94% <0.00%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...dks/python/apache_beam/testing/notebooks/runner.py 0.00% <0.00%> (ø)
...ks/python/apache_beam/testing/notebooks/testlib.py 0.00% <0.00%> (ø)
sdks/python/setup.py 0.00% <ø> (ø)

... and 59 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@AnandInguva
Copy link
Contributor

@svetakvsundhar
Copy link
Contributor Author

Run Python_Colab PreCommit

@svetakvsundhar svetakvsundhar marked this pull request as ready for review May 15, 2023 17:55
@svetakvsundhar
Copy link
Contributor Author

R: @liferoad
R: @tvalentyn

@github-actions
Copy link
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

@liferoad
Copy link
Collaborator

R: @damccorm

.test-infra/jenkins/job_PreCommit_Python_Colab.groovy Outdated Show resolved Hide resolved


def _skip_test(spec):
if IS_BEAM_DEV and "sql" in str(spec).lower():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this?

sdks/python/setup.py Outdated Show resolved Hide resolved
@github-actions github-actions bot removed the infra label May 18, 2023
Copy link
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just had one more nit, otherwise LGTM

sdks/python/setup.py Outdated Show resolved Hide resolved
Copy link
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I'll merge once CI completes

@damccorm damccorm merged commit dbefe4f into apache:master May 18, 2023
lostluck pushed a commit to lostluck/beam that referenced this pull request May 18, 2023
* Draft

* remove upper bound on 'nbformat' and 'nbconvert'

* remove upper bound on 'nbformat' and 'nbconvert'

* lint

* setup.py dependency

* move files

* move files

* change test trigger

* edits

* edits

* edits
Dippatel98 pushed a commit to Dippatel98/beam that referenced this pull request Jun 5, 2023
* Draft

* remove upper bound on 'nbformat' and 'nbconvert'

* remove upper bound on 'nbformat' and 'nbconvert'

* lint

* setup.py dependency

* move files

* move files

* change test trigger

* edits

* edits

* edits
bullet03 pushed a commit to akvelon/beam that referenced this pull request Aug 11, 2023
* Draft

* remove upper bound on 'nbformat' and 'nbconvert'

* remove upper bound on 'nbformat' and 'nbconvert'

* lint

* setup.py dependency

* move files

* move files

* change test trigger

* edits

* edits

* edits
cushon pushed a commit to cushon/beam that referenced this pull request May 24, 2024
* Draft

* remove upper bound on 'nbformat' and 'nbconvert'

* remove upper bound on 'nbformat' and 'nbconvert'

* lint

* setup.py dependency

* move files

* move files

* change test trigger

* edits

* edits

* edits
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request]: Colab notebooks automated Integration Tests
5 participants