Core: Seeing unexpected exception during upload to GCS with Java client #2770
Description
Thanks for stopping by to let us know something could be better!
PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Please run down the following list and make sure you've tried the usual "quick fixes":
- Search the issues already opened: https://github.com/googleapis/java-storage/issues
- Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform
If you are still having issues, please include as much information as possible:
Environment details
- Specify the API at the beginning of the title. For example, "BigQuery: ...").
General, Core, and Other are also allowed as types - OS type and version: Linux x64 docker container unsure of specifics
- Java version: 11
- version(s): GCS Library version 2.38.0
Steps to reproduce
- Happens intermittently on upload.
Code example
final long ret =
isArchive ? uploadFile.length() : FileUtils.sizeOfDirectory(uploadFile);
if (isArchive) {
BlobId blobId = BlobId.of(s3BucketName, s3Path);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId)
.setStorageClass(infrequentAccess
? com.google.cloud.storage.StorageClass.NEARLINE
: com.google.cloud.storage.StorageClass.STANDARD).build();
storage.createFrom(blobInfo, uploadFile.toPath());
} else {
try (Stream<Path> stream = Files.walk(uploadFile.toPath())) {
forkJoinPool.submit(() ->
stream.parallel().filter(x -> !x.toFile().isDirectory())
.forEach(filePath -> {
try {
uploadFile(filePath.toFile(), s3BucketName,
Path.of(s3Path, uploadFile.toPath()
.relativize(filePath)
.toString())
.toString(),
infrequentAccess, true, sse);
} catch (Exception e) {
throw new RuntimeException(e);
}
})).get();
}
}
return ret;Stack trace
com.google.cloud.storage.StorageException: Client side data loss detected. Attempt to append to a resumable session with an offset higher than the backend has
|> PUT https://storage.googleapis.com/upload/storage/v1/b/prd-us-psc-lc-shared-infra-indexes-gcs-green/o?name=class%3Ddefault/org%3DN88FD8JR/partition%3Dd/type%3Draw_current/tstamp%3D0/size%3D0/partition-55&uploadType=resumable&upload_id=AD-8ljvvtif0xx7yz_pwkGcUQbPSH3b96EmAPBDcvZNDJHFHoZOTV6D9F5vM6qeP29LraFm2k5T2_APXqhM8V3ctwROvtRb6F4ALqoQ_L6mZD-Qd2A
|> content-range: bytes 1003487232-2090235391/1086748160
|> x-goog-gcs-idempotency-token: e5ebfb44-2ad2-4a60-a45a-9e4c121af017
|
|< HTTP/1.1 400 Bad Request
|< content-length: 37
|< content-type: text/plain; charset=utf-8
|<
|< Failed to parse Content-Range header.
|
at com.google.cloud.storage.ResumableSessionFailureScenario.toStorageException(ResumableSessionFailureScenario.java:305) at com.google.cloud.storage.ResumableSessionFailureScenario.toStorageException(ResumableSessionFailureScenario.java:133)
at com.google.cloud.storage.JsonResumableSessionPutTask.call(JsonResumableSessionPutTask.java:210)
at com.google.cloud.storage.JsonResumableSession.lambda$put$0(JsonResumableSession.java:81)
at com.google.cloud.storage.Retrying.lambda$run$0(Retrying.java:102)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.storage.Retrying.run(Retrying.java:99)
at com.google.cloud.storage.JsonResumableSession.put(JsonResumableSession.java:68)
at com.google.cloud.storage.StorageImpl.createFrom(StorageImpl.java:272)
at com.google.cloud.storage.StorageImpl.createFrom(StorageImpl.java:235)
External references such as API reference guides
- ?
Any additional information below
Case 53758543 opened with support, they suggested I also open an issue here.
Following these steps guarantees the quickest resolution possible.
Thanks!