Closed
Description
When using the minio client (and thus also when using the Go SDK), I have some files that just can't be uploaded when using the default settings. The only way I can upload them is by changing the --part-size, The default is 16MiB, but when I change it to 20 it works. I have disabled multipart completely for now to prevent any issues.
As far as I know there is nothing special to these files, most files like these upload just fine. It's a PDF file and it's 286697813 bytes. I use mc put from the local filesystem to copy them into Azure via s3proxy.
The error code that I get back from s3proxy is:
<?xml version='1.0' encoding='UTF-8'?><Error><Code>BadDigest</Code><Message>Bad Request</Message><RequestId>4442587FB7D0A2F9</RequestId></Error>
The error that is in the s3proxy logs is:
│ [s3proxy] E 10-24 17:56:15.230 S3Proxy-Jetty-4835 o.j.h.h.BackoffLimitedRetryHandler:88 |::] Cannot retry after server error, command is not replayable: [method=org.jclouds.azureblob.AzureBlobClient. │
│ public abstract void org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[test, test, AAKYEA==, [content=true, contentM │
│ etadata=[cacheControl=null, contentDisposition=null, contentEncoding=null, contentLanguage=null, contentLength=16777216, contentMD5=null, contentType=application/unknown, expires=null], written=false │
│ , isSensitive=false]], request=PUT https://url.blob.core.windows.net/test/test?comp=block&blockid=AAKYEA%3D%3D HTTP/1.1] │
│ [s3proxy] E 10-24 17:56:15.288 S3Proxy-Jetty-4826 o.j.h.i.JavaUrlHttpCommandExecutorService:93 |::] error after writing 15597568/16777216 bytes to https://url.blob.core.windows.net │
│ /test/test?comp=block&blockid=AAJxAA%3D%3D │
│ org.gaul.shaded.org.eclipse.jetty.io.EofException: Early EOF │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpChannelOverHttp.markEarlyEOF(HttpChannelOverHttp.java:287) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpChannelOverHttp.earlyEOF(HttpChannelOverHttp.java:266) │
│ at org.gaul.shaded.org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1634) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpConnection.parseRequestBuffer(HttpConnection.java:414) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpConnection.parseAndFillForContent(HttpConnection.java:348) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpChannelOverHttp.parseAndFillForContent(HttpChannelOverHttp.java:129) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpChannelOverHttp.produceContent(HttpChannelOverHttp.java:115) │
│ at org.gaul.shaded.org.eclipse.jetty.server.AsyncContentProducer.produceRawContent(AsyncContentProducer.java:470) │
│ at org.gaul.shaded.org.eclipse.jetty.server.AsyncContentProducer.nextTransformedContent(AsyncContentProducer.java:357) │
│ at org.gaul.shaded.org.eclipse.jetty.server.AsyncContentProducer.isReady(AsyncContentProducer.java:277) │
│ at org.gaul.shaded.org.eclipse.jetty.server.BlockingContentProducer.nextContent(BlockingContentProducer.java:111) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpInput.read(HttpInput.java:287) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpInput.read(HttpInput.java:272) │
│ at com.google.common.io.ByteStreams.read(ByteStreams.java:929) │
│ at com.google.common.io.ByteStreams.readFully(ByteStreams.java:797) │
│ at com.google.common.io.ByteStreams.readFully(ByteStreams.java:781) │
│ at org.gaul.s3proxy.ChunkedInputStream.read(ChunkedInputStream.java:56) │
│ at org.gaul.s3proxy.ChunkedInputStream.read(ChunkedInputStream.java:70) │
│ at com.google.common.hash.HashingInputStream.read(HashingInputStream.java:68) │
│ at com.google.common.io.ByteStreams$LimitedInputStream.read(ByteStreams.java:742) │
│ at java.base/java.io.FilterInputStream.read(Unknown Source) │
│ at org.jclouds.io.ByteStreams2.copy(ByteStreams2.java:6
6E22
8) │
│ at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:302) │
│ at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:175) │
│ at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:66) │
│ at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:97) │
│ at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91) │
│ at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
│ at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45) │
│ at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156) │
│ at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123) │
│ at jdk.proxy2/jdk.proxy2.$Proxy72.putBlock(Unknown Source) │
│ at org.jclouds.azureblob.blobstore.AzureBlobStore.uploadMultipartPart(AzureBlobStore.java:450) │
│ at jdk.internal.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) │
│ at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) │
│ at java.base/java.lang.reflect.Method.invoke(Unknown Source) │
│ at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:50) │
│ at jdk.proxy2/jdk.proxy2.$Proxy73.uploadMultipartPart(Unknown Source) │
│ at org.gaul.s3proxy.S3ProxyHandler.handleUploadPart(S3ProxyHandler.java:2851) │
│ at org.gaul.s3proxy.S3ProxyHandler.doHandle(S3ProxyHandler.java:748) │
│ at org.gaul.s3proxy.S3ProxyHandlerJetty.handle(S3ProxyHandlerJetty.java:80) │
│ at org.gaul.shaded.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) │
│ at org.gaul.shaded.org.eclipse.jetty.server.Server.handle(Server.java:563) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501) │
│ at org.gaul.shaded.org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287) │
│ at org.gaul.shaded.org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) │
│ at org.gaul.shaded.org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) │
│ at org.gaul.shaded.org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) │
│ at org.gaul.shaded.org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421) │
│ at org.gaul.shaded.org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390) │
│ at org.gaul.shaded.org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277) │
│ at org.gaul.shaded.org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199) │
│ at org.gaul.shaded.org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411) │
│ at org.gaul.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) │
│ at org.gaul.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) │
│ at org.gaul.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) │
│ at java.base/java.lang.Thread.run(Unknown Source)
All the blocks that it send are 16800342 bytes, except for the last one, which is 1487296 bytes. All other blocks are uploaded fine, except for the last one, which results in the error above.
Any idea what's going on here? I'm going to see if I can debug this some more tomorrow.
Other files that are failing are of sizes:
291995023, 286683989, 286904511, 287128205, 304781589, 293607881
I'm running the version from this commit: 356c12f