We use cookies to give you the best experience on our website. If you continue to browse, then you agree to our privacy policy and cookie policy. (Last updated on: June 24, 2019).
Unfortunately, activation email could not send to your email. Please try again.
Syncfusion Feedback

Upload Large File to Azure Storage using Chunk Upload...

Thread ID:

Created:

Updated:

Platform:

Replies:

150303 Dec 31,2019 04:01 PM UTC Jan 23,2020 02:56 PM UTC ASP.NET Core - EJ 2 2
loading
Tags: File Upload
Gich
Asked On December 31, 2019 04:06 PM UTC

Hi, am struggling to upload large file to azure blob by use of Chunk Upload. But it fails. What it does it only upload a chunk to the azure blob storage. If I upload a file of 10 MB, in the blob, I can only see 34.3KB. Can you help me please? What am I missing? Below is the exact code and I have attached my solution as Zip. 

index.cshtml

@{
    ViewData["Title"] = "Home Page";
}

Game Upload


@{
    var asyncSettings = new Syncfusion.EJ2.Inputs.UploaderAsyncSettings { SaveUrl = "https://localhost:44357/api/Default/Save", RemoveUrl = "https://localhost:44357/api/Default/Remove", RetryCount=5, RetryAfterDelay=3000, ChunkSize=102400  };
}


Server-Side Scripts


 [HttpPost]
        [Route("Save")]
        public async Task Save(IList chunkFile, IList UploadFiles)
        {
            long size = 0;
            try
            {
                string blobName = "mytest1.zip";
                List b = new List();

                // for chunk-upload
                foreach (var file in chunkFile)
                {
                  
                    size += file.Length;
                    var ms = new MemoryStream();
                    file.OpenReadStream().CopyTo(ms);
                    byte[] Value = ms.ToArray();
                    b.Add(Value);
                }
                IEnumerable result = Enumerable.Empty();
                foreach (byte[] bytes in b)
                {
                    result = result.Concat(bytes);
                }
                byte[] newArray = result.ToArray();

                var storageCredentials = new StorageCredentials("", "");
                var cloudStorageAccount = new CloudStorageAccount(storageCredentials, true);
                var cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
                var container = cloudBlobClient.GetContainerReference("filo");
                await container.CreateIfNotExistsAsync();
                var newBlob = container.GetBlockBlobReference(blobName);
                newBlob.StreamWriteSizeInBytes = 6 * 1024 * 1024;
                // This sets the size of the blocks to use when you do a Put Blob and it breaks it into blocks to upload because the file is larger than the value of SingleBlobUploadThresholdInBytes.
                // By default, this is 4MB(4 * 1024 * 1024).

                newBlob.StreamMinimumReadSizeInBytes = 6 * 1024 * 1024;

                //set the blob upload timeout and retry strategy
                BlobRequestOptions options = new BlobRequestOptions();
                options.ServerTimeout = new TimeSpan(0, 180, 0);
                options.RetryPolicy = new ExponentialRetry(TimeSpan.Zero, 20);

                //get the file blocks of 2MB size each and perform upload of each block
                HashSet blocklist = new HashSet();
                List bloksT = GetFileBlocks(newArray).ToList();
                foreach (FileBlock block in GetFileBlocks(newArray))
                {
                    await newBlob.PutBlockAsync(
                         block.Id,
                         new MemoryStream(block.Content, true), null,
                         null, options, null
                         );

                    blocklist.Add(block.Id);

                }
                //commit the blocks that are uploaded in above loop
                await newBlob.PutBlockListAsync(blocklist, null, options, null);

                Console.WriteLine("Done");
            }
            catch (Exception e)
            {
                // Response.Clear();
                Response.StatusCode = 204;
                Response.HttpContext.Features.Get().ReasonPhrase = "File failed to upload";
                Response.HttpContext.Features.Get().ReasonPhrase = e.Message;
            }

        }



Attachment: svc_69be2b70.zip

Berly Christopher [Syncfusion]
Replied On January 14, 2020 11:42 PM UTC

Hi Gich, 
 
Greetings from Syncfusion support. 
 
We are facing some issues while saving the file in Azure storage. Currently we are checking the issues with high priority. So, we will check and update the further details in three business days(21st January 2020). We appreciate your patience until then. 
 
Regards, 
Berly B.C 


Gopi Govindasamy [Syncfusion]
Replied On January 23, 2020 02:56 PM UTC

Hi Gich,

Thanks for your patience.

We have validated your reported scenario for large file upload to Azure storage. By default, Azure file storage not merge the file to same location and multiple chunk file stream. We suggest to use session for  save the chunk file. Once complete all chunk file, you can get the session based on session key and save to Azure location. We have achieved your scenario for save the chunk file to Azure location. Please find the code snippet and sample for your reference.

Save.cs

       [HttpPost]

        [Route("Save")]

        public async Task Save(IList chunkFile)

        {

            try

            {

                foreach (var file in chunkFile)

                {

                    var httpPostedChunkFile = HttpContext.Request.Form.Files["chunkFile"];

                    var chunkIndex = HttpContext.Request.Form["chunk-index"];

                    var totalChunk = HttpContext.Request.Form["total-chunk"];

                    using (var fileStream = file.OpenReadStream())

                    {

                        if(Convert.ToInt32(chunkIndex) <= Convert.ToInt32(totalChunk))

                        {

                            var streamReader = new MemoryStream();

                            fileStream.CopyTo(streamReader);

                            var byteArr = streamReader.ToArray();

                            var content = new byte[] { };

                            if (HttpContext.Session.Get("streamFile") != null)

                            {

                                content = HttpContext.Session.Get("streamFile").Concat(byteArr).ToArray();

                            } else

                            {

                                content = byteArr;

                            }

                            HttpContext.Session.Set("streamFile", content);

                        }

                        if(Convert.ToInt32(chunkIndex) == Convert.ToInt32(totalChunk)-1)

                        {

                            var fileArray = HttpContext.Session.Get("streamFile");

                            var storageCredentials = new StorageCredentials("", "");

                            var cloudStorageAccount = new CloudStorageAccount(storageCredentials, true);

                            var cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

                            var container = cloudBlobClient.GetContainerReference("filo");

                            CloudBlockBlob blockBlob = container.GetBlockBlobReference(httpPostedChunkFile.FileName);

                            using (FileStream fileStreams = new FileStream(httpPostedChunkFile.FileName, FileMode.Create))

                            {

                                for (int i = 0; i < fileArray.Length; i++)

                                {

                                    fileStreams.WriteByte(fileArray[i]);

                                }

                                fileStreams.Seek(0, SeekOrigin.Begin);

                                HttpContext.Session.Remove("streamFile");

                                await blockBlob.UploadFromStreamAsync(fileStreams);

                            }

                        }

                    }

                }

            }

            catch (Exception e)

            {

                . . .

            }

        }


Startup.cs

public void ConfigureServices(IServiceCollection services)

        {

         . . .    

 

            services.AddSession(options => {

                options.IdleTimeout = TimeSpan.FromMinutes(30);

            });

. . .

        }

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)

        {

. . .

   app.UseSession();

. . .

}

 

Sample Link:  https://www.syncfusion.com/downloads/support/forum/150303/ze/svc598742026

 

Regards,

Gopi G.

 


CONFIRMATION

This post will be permanently deleted. Are you sure you want to continue?

Sorry, An error occured while processing your request. Please try again later.

Please sign in to access our forum

This page will automatically be redirected to the sign-in page in 10 seconds.

Warning Icon You are using an outdated version of Internet Explorer that may not display all features of this and other websites. Upgrade to Internet Explorer 8 or newer for a better experience.Close Icon

Live Chat Icon For mobile
Live Chat Icon