-
Notifications
You must be signed in to change notification settings - Fork 151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Storage: Capture relevant headers to blob properties during download #24
Comments
Thank you Tres! |
@william-silversmith The headers needed to do this task are buried several layers deep. I'm wondering if a better solution than trying to guess which ones to map onto the blob's properties, we might be better off designing for your usecase by supplying a
An example callback implementation for your usecase might be something like: def _update_blob_content_encoding(blob, response):
if blob.content_encoding is None: # latch
blob.content_encoding = response.headers.get('Content-Encoding') |
If we don't use a callback, then the relevant headers to start with would seem to be:
|
I almost never need more exotic headers than the ones you listed. If it's easier to only expose those headers, that's fine with me. If you want to go for a more general solution, I am fine with a (synchronous) callback. Thanks again! |
Any word on when this might drop? Thanks for your help! |
Hi Tres, I've made some changes to my own branch of this repo and they seem to work. I tried making a PR but the button was greyed out. Here's a link to the changes in case y'all are interested. In the mean time, I'll direct my repos to use the forked version. |
Unfortunately, it seems PyPI won't let me use a direct reference to github. It would be extremely convenient if some version of this could get merged. |
Figured out the PR submission issue. |
…ds (#204) Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-storage/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [x] Ensure the tests and linter pass - [x] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) Fixes #24 🦕 This PR autopopulates the following fields for non-chunked downloads based on the server header response: ``` blob.content_encoding blob.content_type blob.cache_control blob.storage_class blob.content_language blob.md5_hash blob.crc32c ```
…ds (googleapis#204) Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-storage/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [x] Ensure the tests and linter pass - [x] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) Fixes googleapis#24 🦕 This PR autopopulates the following fields for non-chunked downloads based on the server header response: ``` blob.content_encoding blob.content_type blob.cache_control blob.storage_class blob.content_language blob.md5_hash blob.crc32c ```
…ds (googleapis#204) Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-storage/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [x] Ensure the tests and linter pass - [x] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) Fixes googleapis#24 🦕 This PR autopopulates the following fields for non-chunked downloads based on the server header response: ``` blob.content_encoding blob.content_type blob.cache_control blob.storage_class blob.content_language blob.md5_hash blob.crc32c ```
Residual from googleapis/google-cloud-python#9003.
@william-silversmith notes that even with
raw_download
enabled, he is unable to detect thecontent_type
of a downloaded blob without performing an additionalreload
request, which is prohibitive for his usecase at scale. E.g.:Potentially even...
The text was updated successfully, but these errors were encountered: