-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added support for AWS Sigv4 for UrlLib3. #547
Conversation
Signed-off-by: dblock <dblock@amazon.com>
2a50056
to
a732d44
Compare
Codecov Report
@@ Coverage Diff @@
## main #547 +/- ##
==========================================
+ Coverage 70.63% 70.66% +0.02%
==========================================
Files 83 83
Lines 7857 7877 +20
==========================================
+ Hits 5550 5566 +16
- Misses 2307 2311 +4
|
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
c9f113f
to
fb9e1b9
Compare
@harshavamsi would appreciate your CR since you did support for the requests connection class. |
Signed-off-by: dblock <dblock@amazon.com>
fb9e1b9
to
f8f2b62
Compare
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
b709e40
to
8d5aeb0
Compare
Signed-off-by: dblock <dblock@amazon.com>
# Modifications Copyright OpenSearch Contributors. See | ||
# GitHub history for details. | ||
# | ||
# Licensed to Elasticsearch B.V. under one or more contributor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this license here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this code was refactored out of test_connection.py
, so let's keep it.
guides/auth.md
Outdated
@@ -9,24 +9,24 @@ OpenSearch allows you to use different methods for the authentication via `conne | |||
|
|||
## IAM Authentication | |||
|
|||
Opensearch-py supports IAM-based authentication via `AWSV4SignerAuth`, which uses `RequestHttpConnection` as the transport class for communicating with OpenSearch clusters running in Amazon Managed OpenSearch and OpenSearch Serverless, and works in conjunction with [botocore](https://pypi.org/project/botocore/). | |||
Opensearch-py supports IAM-based authentication via `RequestsAWSV4SignerAuth` and `Urllib3AWSV4SignerAuth`, which use `RequestHttpConnection` and `Urllib3HttpConnection` respectively, as the transport classes for communicating with OpenSearch clusters running in Amazon Managed OpenSearch and OpenSearch Serverless, and works in conjunction with [botocore](https://pypi.org/project/botocore/). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we recommend Urllib3AWSV4SignerAuth
here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've clarified the documentation in cb76874.
self.signer = AWSV4Signer(credentials, region, service) | ||
|
||
def __call__(self, method: str, url: str, body: Any) -> Dict[str, str]: | ||
return self.signer.sign(method, url, body) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
RequestsAWSV4SignerAuth
returns prepared request while Urllib3AWSV4SignerAuth
returns the headers, should we keep them consistent?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The requests
library expect a callable that derives from requests.auth.AuthBase
and receives a request
object. It's called inside the requests library.
The urllib3 library doesn't support such an interface. It actually splits the client constructor and exposes a perform_request
method. That's where we have the method, URL, and body, and call auth. There's no "request" object here.
We could make a similar interface to AuthBase
for urllib3, but it cannot be the same interface because that one exists in the requests library. Since these signer classes aren't interchangeable at all, they don't need to derive from the same class, and I already moved the common parts into AWSV4Signer
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That makes sense, thank you!
…uestHttpConnection. Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
@VachaShah I wrote some more tests and addressed your comments. LMK if there's anything else needed for this one |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Added a guide on making raw JSON REST requests. (opensearch-project#542) Signed-off-by: dblock <dblock@amazon.com> Added document lifecycle guide & sample code. Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Updated CHANGELOG Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Added support for AWS Sigv4 for UrlLib3. (opensearch-project#547) * WIP: Added support for AWS Sigv4 for UrlLib3. Signed-off-by: dblock <dblock@amazon.com> * Refactored common implementation. Signed-off-by: dblock <dblock@amazon.com> * Added sigv4 samples. Signed-off-by: dblock <dblock@amazon.com> * Updated CHANGELOG. Signed-off-by: dblock <dblock@amazon.com> * Add documentation. Signed-off-by: dblock <dblock@amazon.com> * Use the correct class in tests. Signed-off-by: dblock <dblock@amazon.com> * Renamed samples. Signed-off-by: dblock <dblock@amazon.com> * Split up requests and urllib3 unit tests. Signed-off-by: dblock <dblock@amazon.com> * Rename AWSV4Signer. Signed-off-by: dblock <dblock@amazon.com> * Clarified documentation of when to use Urllib3AWSV4SignerAuth vs. RequestHttpConnection. Signed-off-by: dblock <dblock@amazon.com> * Move fetch_url inside the signer class. Signed-off-by: dblock <dblock@amazon.com> * Added unit test for Urllib3AWSV4SignerAuth adding headers. Signed-off-by: dblock <dblock@amazon.com> * Added unit test for signing to include query string. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com> Remove support for Python 2.x. (opensearch-project#548) Signed-off-by: dblock <dblock@amazon.com> Fixed guide & added link in USER_GUIDE.md Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Added support for AWS Sigv4 for UrlLib3. (opensearch-project#547) * WIP: Added support for AWS Sigv4 for UrlLib3. Signed-off-by: dblock <dblock@amazon.com> * Refactored common implementation. Signed-off-by: dblock <dblock@amazon.com> * Added sigv4 samples. Signed-off-by: dblock <dblock@amazon.com> * Updated CHANGELOG. Signed-off-by: dblock <dblock@amazon.com> * Add documentation. Signed-off-by: dblock <dblock@amazon.com> * Use the correct class in tests. Signed-off-by: dblock <dblock@amazon.com> * Renamed samples. Signed-off-by: dblock <dblock@amazon.com> * Split up requests and urllib3 unit tests. Signed-off-by: dblock <dblock@amazon.com> * Rename AWSV4Signer. Signed-off-by: dblock <dblock@amazon.com> * Clarified documentation of when to use Urllib3AWSV4SignerAuth vs. RequestHttpConnection. Signed-off-by: dblock <dblock@amazon.com> * Move fetch_url inside the signer class. Signed-off-by: dblock <dblock@amazon.com> * Added unit test for Urllib3AWSV4SignerAuth adding headers. Signed-off-by: dblock <dblock@amazon.com> * Added unit test for signing to include query string. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com> Remove support for Python 2.x. (opensearch-project#548) Signed-off-by: dblock <dblock@amazon.com> Fixed guide & added link in USER_GUIDE.md opensearch-project#3 Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com>
* WIP: Added support for AWS Sigv4 for UrlLib3. Signed-off-by: dblock <dblock@amazon.com> * Refactored common implementation. Signed-off-by: dblock <dblock@amazon.com> * Added sigv4 samples. Signed-off-by: dblock <dblock@amazon.com> * Updated CHANGELOG. Signed-off-by: dblock <dblock@amazon.com> * Add documentation. Signed-off-by: dblock <dblock@amazon.com> * Use the correct class in tests. Signed-off-by: dblock <dblock@amazon.com> * Renamed samples. Signed-off-by: dblock <dblock@amazon.com> * Split up requests and urllib3 unit tests. Signed-off-by: dblock <dblock@amazon.com> * Rename AWSV4Signer. Signed-off-by: dblock <dblock@amazon.com> * Clarified documentation of when to use Urllib3AWSV4SignerAuth vs. RequestHttpConnection. Signed-off-by: dblock <dblock@amazon.com> * Move fetch_url inside the signer class. Signed-off-by: dblock <dblock@amazon.com> * Added unit test for Urllib3AWSV4SignerAuth adding headers. Signed-off-by: dblock <dblock@amazon.com> * Added unit test for signing to include query string. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com> Signed-off-by: roma2023 <romasaparhan19@gmail.com>
Description
AWSV4SignerAuth
in favor ofRequestsAWSV4SignerAuth
Urllib3AWSV4SignerAuth
in the documentation since that transport is faster.Testing
AOS
AOSS
Issues Resolved
Closes #546
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.