[
https://issues.apache.org/jira/browse/JCLOUDS-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734550#comment-16734550
]
Andrew Gaul commented on JCLOUDS-1482:
--------------------------------------
This is a consequence of how multipart/SLO uploads work in Swift. To create a
large file, it clients create one or more parts then upload a manifest. Users
can download the manifest which reassembles the parts on the server side.
Unfortunately both the manifest and parts exist in the same namespace.
JCLOUDS-1285 suggests storing the parts in a separate container.
> Uploading a file to a Swift container create two items inside it: one of type
> BLOB and other of type FOLDER.
> -------------------------------------------------------------------------------------------------------------
>
> Key: JCLOUDS-1482
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1482
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
> Affects Versions: 2.1.1
> Reporter: Biswa Ranjan Ray
> Priority: Major
> Labels: openstack-swift
>
> *Usecase*: I'm trying to upload a file(107kb) to an OpenStack Swift container
> using the api BlobStore putBlob(String container, Blob blob, PutOptions
> options). Below code snippet demonstrates the way I get the BlobStoreContext
> and upload a file.
> {code:java}
> //To get the BlobStoreContext for OpenStack Swift
> private BlobStoreContext getBlobStoreContext() {
> final Properties override = new Properties();
> override.put(KeystoneProperties.KEYSTONE_VERSION, "3");
> override.put(KeystoneProperties.SCOPE, "project:" +
> swiftConfig.getProject());
>
> final String credential = swiftConfig.getPassword();
> final String identity = swiftConfig.getUserDomain() + ":" +
> swiftConfig.getUsername();
> final String authUrl = swiftConfig.getAuthUrl();
>
> return ContextBuilder.newBuilder("openstack-swift")
> .endpoint(authUrl+"/v3")
> .credentials(identity,credential)
> .overrides(override)
> .buildApi(BlobStoreContext.class);
> }
> //To upload a file into a container of an objectstore e.g AWS S3, OpenStack
> Swift
> public String uploadFile(String bucketName, byte[] bytes, String fileName) {
> BlobStore blobStore = getBlobStoreContext().getBlobStore();
> boolean isBlobExist = blobStore.blobExists(bucketName, fileName);
> if(!isBlobExist){
> //creating payload
> Payload payload = new ByteArrayPayload(bytes);
> //adding user metadata to the blob
> Map<String,String> userMetadata = new HashMap<String,String>();
> userMetadata.put("description", "sample content");
> //creating Blob
> Blob blob = blobStore.blobBuilder(fileName)
> .payload(payload)
> .userMetadata(userMetadata)
> .build();
> String etag = blobStore.putBlob(bucketName, blob,
> *PutOptions.Builder.multipart(true)*);
> String message = fileName+" uploaded successfully with etag "+etag;
> return message;
> }
> }
> {code}
> *Issue:* Once the upload process is complete, I found two items created
> inside the container. One is of type *BLOB* and the other is of type *FOLDER*
> as shown below:
> TestFile.DOCX
> TestFile.DOCX/slo/1546583793.081000/109256/33554432/00000000
> The actual issue is with the below line where I use multipart:
> String etag = blobStore.putBlob(bucketName, blob,
> *PutOptions.Builder.multipart(true)*);
> However the above code works fine in AWS S3 and adds only one item of type
> BLOB into the required container.
> *Debug:* I debug the code to find the following:
> # step into putBlob(bucketName, blob, *PutOptions.Builder.multipart(true)*)
> # step into RegionScopedSwiftBlobStore > putBlob(String container, Blob
> blob, PutOptions options).
> # step into putMultipartBlob(String container, Blob blob, PutOptions
> overrides). Here i found the *contentLength:* 109256(file size), *partSize:*
> 33554432, *part: 0*
> # **step into initiateMultipartUpload(String container, BlobMetadata
> blobMetadata, long partSize, PutOptions options). Here I found the
> *contentLength:* 109256(file size), *uploadId*:
> TestFile.DOCX/slo/1546583793.081000/109256/33554432
> # step into MultipartUpload.create(container, blobMetadata.getName(),
> uploadId, blobMetadata, options)
> # step into new AutoValue_MultipartUpload(containerName, blobName, id,
> blobMetadata, putOptions). Here onwards I can't debug any further.
> *Analysis*: I observed that if I use SwiftApi instead of the traditional
> BlobStore then I get the desired result as it creates only one item of type
> BLOB inside the container. But since we are evaluating a java based third
> party library to develop a multi cloud application and so far we found
> Jclouds is the right choice so we would like to use portable apis e.g
> BlobStore to perform CRUD operations instead of using provider specific apis
> e.g SwiftApi.
> Any help would be greatly appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
