CDPD-12081 Upgrade gcs connector to v1.9.17 from upstream in HDP 3.1-maint #8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Change on 2018/11/05 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=220160326
Change on 2018/11/05 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=220208182
Apparently, somebody created "does-not-exist" GCS bucket that caused one of the tests (that expects that this bucket does not exist!) to fail. Change on 2018/12/05 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=224261022
Change on 2018/12/05 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=224262503
Change on 2018/12/10 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=224891096
Change on 2018/12/10 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=224916690
Change on 2018/12/13 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=225450649
…ation Change on 2018/12/14 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=225564622
------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=225887390
Change on 2018/12/18 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=226058870
…ns which are accessed from production code.
These annotations are being violated; they are being removed to allow the introduction of an ErrorProne check which highlights such errors.
More information: []
Tested:
TAP --sample for global presubmit queue
[]
Change on 2018/12/19 by ghm <ghm@google.com>
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226249949
Change on 2018/12/20 by dagang <dagang@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=226362653
Change on 2018/12/20 by dagang <dagang@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=226393561
Change on 2018/12/28 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=227151680
Fixes: GoogleCloudDataproc#22 Change on 2019/01/03 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=227718811
Change on 2019/01/03 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=227762104
Customers report their jobs frequently fail because of failed rename and delete operations, but it's impossible to diagnose them, because nothing is logged by default. Change on 2019/01/16 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=229592005
Change on 2019/01/25 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=230956801
Change on 2019/01/25 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=230994471
* use Markdown formatting instead of HTML formatting * added link to search for all questions tagged with `google-cloud-dataproc` on SO
In Mockito 2, if a method expects a primitive type, but an any(<Primitive>.class) matcher is used in its place, it will throw an error. To prepare for this upcoming breakage, change
all existing any(<Primitive>.class) matchers to use the correct any<Primitive>() matcher.
For more information see []
Tested:
TAP --sample for global presubmit queue
[]
Change on 2019/02/04 by tvanderlippe <tvanderlippe@google.com>
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232301764
This will prevent Spark jobs from hanging after all work is done. Fixes GoogleCloudDataproc#150: GoogleCloudDataproc#150 Change on 2019/02/11 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=233437145
Change on 2019/02/13 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=233877650
…S (HTTP 429 reponse). Fixes GoogleCloudDataproc#151
…tream.fast.fail.on.not.found.enable' is true
Fixes GoogleCloudDataproc#144 Change on 2019/04/26 by dagang <dagang@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=245520498
…f it wasn't initialized before.
Change on 2019/05/13 by idv <idv@google.com>
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247978825
…Status` methods Note: this is essentially the same change as in [] that triggered omg/12873 in the past, but it has feature flag that turns off it by default and tests that assert number of GCS requests when parallelism is enabled. In the worst case `getFileStatus` method can make up to 3 sequential requests to GCS to get implicit directory status. After moving implicit directory repair from list to delete/rename operations this worst case could be more frequent than before, because there higher chance to encounter implicit non-repaired directory: GoogleCloudDataproc#156 This CL adds an option to execute these GCS requests in parallel which could reduce latency by up to 3 times. Change on 2019/05/13 by idv <idv@google.com> ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=248044068
Conflicts: bigquery/pom.xml gcs/pom.xml util-hadoop/pom.xml util/pom.xml
mukund-thakur
changed the title
|
I believe https://github.com/hortonworks/bigdata-interop/commits/v1.9.17-upgrade is a copy of 1.9.17 upstream with the single patch from the HDP-3.1-maint branch applied on it? If so, +1. LGTM. (Not walking through the diff) |
|
Yes it is the copy of v1.9.17 with one extra build related commit. |
sidseth
approved these changes
May 15, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Upgrading gcs connector to V1.9.17