Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce output data format versioning #2653 #2682

Merged
merged 11 commits into from
Sep 14, 2021

Conversation

AyanSinhaMahapatra
Copy link
Contributor

Add output data format version numbers to the headers and version help
text. Introduce new command line option to switch to the new experimental
data format.

Signed-off-by: Ayan Sinha Mahapatra ayansmahapatra@gmail.com

Add output data format version numbers to the headers and version help
text. Introduce new command line option to switch to the new experimental
data format.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Modifies tests to ignore addition of output data format in headers.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Renames flag to --future-format and updates help text test.
Adds future format version.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Copy link
Contributor Author

AyanSinhaMahapatra commented Sep 3, 2021

@pombredanne could you look into this why the tests with latest_from_pip are failing like here. These are failing as bin/pip install --force-reinstall --upgrade -e .[dev] replaces commoncode 21.8.31 with 21.8.27, which doesn't have the header changes, and hence the failures.

If I change this to 21.8.31, this happens in the test:

ERROR: Could not find a version that satisfies the requirement commoncode>=21.8.31 (from scancode-toolkit[dev]) (from versions: 20.9, 20.9.30, 20.10, 20.10.8, 20.10.20, 21.1.14, 21.1.21, 21.4.28, 21.5.12, 21.5.25, 21.6.10, 21.6.11, 21.7.23, 21.8.27)
ERROR: No matching distribution found for commoncode>=21.8.31

What am I doing wrong here

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@@ -249,6 +251,13 @@ def validate_depth(ctx, param, value):
'the starting directory. Use 0 for no scan depth limit.',
help_group=cliutils.CORE_GROUP, sort_order=301, cls=PluggableCommandLineOption)

@click.option('--future-format',
is_flag=True,
help='Output the next experimental data format, for JSON and YAML output.'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use future and not experimental, here and elsewhere


# See https://github.com/nexB/scancode-toolkit/issues/2653 for more information
# on the data format version
__output_format_version__ = '1.1'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use 1.0

@pombredanne
Copy link
Contributor

21.8.31

You need to push 21.8.31 to PyPI too IMHO

Copy link
Contributor

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO you could have fewer changes in the diffs if def check_json_scan(..., ignore_headers=True) was updated to be def check_json_scan(..., include_headers=False) this way few changes are needed in the tests overall... and only a few tests that test headers will need fixing.

@AyanSinhaMahapatra
Copy link
Contributor Author

You need to push 21.8.31 to PyPI too IMHO

You have to add me at https://pypi.org/project/commoncode/ btw

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra AyanSinhaMahapatra force-pushed the 2653-output-data-format-version branch 2 times, most recently from 72ab891 to a93fd62 Compare September 6, 2021 13:44
Replaces ignore_header with check_header attribute with default False,
to minimise diff and ignore headers by default.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
See #2653 (comment)

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@pombredanne pombredanne changed the title Introduce output data format versioning #2653 [WIP] Introduce output data format versioning #2653 Sep 7, 2021
Copy link
Contributor

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Merging!

@pombredanne pombredanne merged commit d8d272a into develop Sep 14, 2021
@pombredanne pombredanne deleted the 2653-output-data-format-version branch September 14, 2021 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants