Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 DeleteObjects throws S3Client.makeMetaRequest has invalid options; Operation name must be set for MetaRequestType.DEFAULT #826

Closed
Sowik opened this issue Aug 28, 2024 · 0 comments
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@Sowik
Copy link

Sowik commented Aug 28, 2024

Describe the bug

Since this was added - awslabs/aws-c-s3#439 I am getting IllegalArgumentException - S3Client.makeMetaRequest has invalid options; Operation name must be set for MetaRequestType.DEFAULT when I try to delete objects.

Expected Behavior

I expect the DeleteObjects to work (like it did before version 0.30.0) and not throw IllegalArgumentException - S3Client.makeMetaRequest has invalid options; Operation name must be set for MetaRequestType.DEFAULT

Current Behavior

java.util.concurrent.CompletableFuture@a2ac4b[Completed exceptionally: java.lang.IllegalArgumentException: S3Client.makeMetaRequest has invalid options; Operation name must be set for MetaRequestType.DEFAULT.]

((IllegalArgumentException)((AltResult)executeFuture.result).ex).stackTrace = {StackTraceElement[130]@14934} 
 0 = {StackTraceElement@14936} "software.amazon.awssdk.crt.s3.S3Client.makeMetaRequest(S3Client.java:148)"
 1 = {StackTraceElement@14937} "software.amazon.awssdk.services.s3.internal.crt.S3CrtAsyncHttpClient.execute(S3CrtAsyncHttpClient.java:162)"
 2 = {StackTraceElement@14938} "software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.doExecuteHttpRequest(MakeAsyncHttpRequestStage.java:204)"
 3 = {StackTraceElement@14939} "software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.executeHttpRequest(MakeAsyncHttpRequestStage.java:151)"
 4 = {StackTraceElement@14940} "software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$1(MakeAsyncHttpRequestStage.java:104)"
 5 = {StackTraceElement@14941} "java.base/java.util.concurrent.CompletableFuture.uniAcceptNow(CompletableFuture.java:757)"
 6 = {StackTraceElement@14942} "java.base/java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:735)"
 7 = {StackTraceElement@14943} "java.base/java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2214)"
 8 = {StackTraceElement@14944} "software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.execute(MakeAsyncHttpRequestStage.java:100)"
 9 = {StackTraceElement@14945} "software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.execute(MakeAsyncHttpRequestStage.java:65)"
 10 = {StackTraceElement@14946} "software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)"
 11 = {StackTraceElement@14947} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallAttemptMetricCollectionStage.execute(AsyncApiCallAttemptMetricCollectionStage.java:62)"
 12 = {StackTraceElement@14948} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallAttemptMetricCollectionStage.execute(AsyncApiCallAttemptMetricCollectionStage.java:41)"
 13 = {StackTraceElement@14949} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.attemptExecute(AsyncRetryableStage.java:144)"
 14 = {StackTraceElement@14950} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:136)"
 15 = {StackTraceElement@14951} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.execute(AsyncRetryableStage.java:95)"
 16 = {StackTraceElement@14952} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage.execute(AsyncRetryableStage.java:79)"
 17 = {StackTraceElement@14953} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage.execute(AsyncRetryableStage.java:44)"
 18 = {StackTraceElement@14954} "software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)"
 19 = {StackTraceElement@14955} "software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)"
 20 = {StackTraceElement@14956} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.execute(AsyncExecutionFailureExceptionReportingStage.java:41)"
 21 = {StackTraceElement@14957} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.execute(AsyncExecutionFailureExceptionReportingStage.java:29)"
 22 = {StackTraceElement@14958} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallTimeoutTrackingStage.execute(AsyncApiCallTimeoutTrackingStage.java:64)"
 23 = {StackTraceElement@14959} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallTimeoutTrackingStage.execute(AsyncApiCallTimeoutTrackingStage.java:36)"
 24 = {StackTraceElement@14960} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallMetricCollectionStage.execute(AsyncApiCallMetricCollectionStage.java:49)"
 25 = {StackTraceElement@14961} "software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallMetricCollectionStage.execute(AsyncApiCallMetricCollectionStage.java:32)"
 26 = {StackTraceElement@14962} "software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)"
 27 = {StackTraceElement@14963} "software.amazon.awssdk.core.internal.http.AmazonAsyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonAsyncHttpClient.java:215)"
 28 = {StackTraceElement@14964} "software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.invoke(BaseAsyncClientHandler.java:288)"
 29 = {StackTraceElement@14965} "software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.doExecute(BaseAsyncClientHandler.java:227)"
 30 = {StackTraceElement@14966} "software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.lambda$execute$1(BaseAsyncClientHandler.java:80)"
 31 = {StackTraceElement@14967} "software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.measureApiCallSuccess(BaseAsyncClientHandler.java:294)"
 32 = {StackTraceElement@14968} "software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.execute(BaseAsyncClientHandler.java:73)"
 33 = {StackTraceElement@14969} "software.amazon.awssdk.awscore.client.handler.AwsAsyncClientHandler.execute(AwsAsyncClientHandler.java:49)"
 34 = {StackTraceElement@14970} "software.amazon.awssdk.services.s3.DefaultS3AsyncClient.deleteObjects(DefaultS3AsyncClient.java:3387)"
 35 = {StackTraceElement@14971} "software.amazon.awssdk.services.s3.DelegatingS3AsyncClient.lambda$deleteObjects$21(DelegatingS3AsyncClient.java:2598)"
 36 = {StackTraceElement@14972} "software.amazon.awssdk.services.s3.DelegatingS3AsyncClient.invokeOperation(DelegatingS3AsyncClient.java:10099)"
 37 = {StackTraceElement@14973} "software.amazon.awssdk.services.s3.DelegatingS3AsyncClient.deleteObjects(DelegatingS3AsyncClient.java:2598)"

Reproduction Steps

I am using it with Spring Webflux.

S3 Async Client Config:

public class S3AsyncClientConfiguration {

	@Bean
	public S3AsyncClient s3AsyncClient(RegionProperties regionProperties, AwsCredentialsProvider credentialsProvider,
			AwsProperties awsProperties, S3Properties s3Properties) {

		return S3AsyncClient.crtBuilder()
				.credentialsProvider(credentialsProvider)
				.region(Region.of(Objects.requireNonNull(regionProperties.getStatic())))
				.endpointOverride(awsProperties.getEndpoint())
				.forcePathStyle(s3Properties.getPathStyleAccessEnabled())
				.checksumValidationEnabled(s3Properties.getChecksumValidationEnabled())
				.build();
	}
}

Service:

public class AwsCloudService  {

	private final S3AsyncClient s3AsyncClient;

	public Mono<Void> deleteData(String bucket, String blobPath) {

		Delete delete = Delete.builder().objects(ObjectIdentifier.builder().key(blobPath).build()).build();

		DeleteObjectsRequest deleteObjectsRequest = DeleteObjectsRequest.builder().bucket(bucket).delete(delete).build();

		CompletableFuture<DeleteObjectsResponse> future = s3AsyncClient.deleteObjects(deleteObjectsRequest);

		return Mono.fromFuture(future).doOnSuccess(unused -> log.info("Deleted data from path {}.", blobPath)).then();
	}

}

I know I am using DeleteObjects to delete just one object but this is done for a completely different reason, unrelated to the issue.

Possible Solution

If I am calling DeleteObjects correctly, could the reason be from S3CrtAsyncHttpClient.

    private static S3MetaRequestOptions.MetaRequestType requestType(AsyncExecuteRequest asyncRequest) {
        String operationName = asyncRequest.httpExecutionAttributes().getAttribute(OPERATION_NAME);
        if (operationName != null) {
            switch (operationName) {
                case "GetObject":
                    return S3MetaRequestOptions.MetaRequestType.GET_OBJECT;
                case "PutObject":
                    return S3MetaRequestOptions.MetaRequestType.PUT_OBJECT;
                default:
                    return S3MetaRequestOptions.MetaRequestType.DEFAULT;
            }
        }
        return S3MetaRequestOptions.MetaRequestType.DEFAULT;
    }

This method has cases for GetObject and PutObject that populate it with a MetaRequestType different than DEFAULT.

Additional Information/Context

No response

aws-crt-java version used

0.30.0 or higher

Java version used

21

Operating System and version

Windows 11

@Sowik Sowik added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Aug 28, 2024
@Sowik Sowik closed this as completed Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Development

No branches or pull requests

1 participant