[S3] Respect chunk_size in download_object_as_stream#2168
Open
Sanjays2402 wants to merge 1 commit into
Open
Conversation
download_object_as_stream and download_object_range_as_stream passed the hardcoded module-level CHUNK_SIZE to response.iter_content instead of the chunk_size supplied by the caller, so a user-specified chunk size was silently ignored. Forward chunk_size (falling back to CHUNK_SIZE) as the sibling drivers already do. Closes apache#1798
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1798
Description
download_object_as_streamanddownload_object_range_as_streamin the S3driver passed the hardcoded module-level
CHUNK_SIZEtoresponse.iter_content()instead of thechunk_sizegiven by the caller, so auser-specified chunk size was silently ignored. They now forward
chunk_size(falling back to
CHUNK_SIZE), matching the backblaze_b2 and cloudfilesdrivers. Added a regression test.
Status
Checklist (tick everything that applies)