mardi 5 mai 2015

Process azure blobs and download them via ASP.NET

I am running a website on an Azure Web App (standard tier S1) where you can download files from a blob storage in different formats (xml, json).

I am inserting these files (html, text, images as base64 string) as block blobs into a container. Right now I am downloading all blobs and convert the result into a desired format, zip it and offer this as download on my ASP.NET website.

I see a problem here that the whole download and zip process runs inside the web app which could take a while and also reach the memory limit (1.75 GB on S1) when I am downloading, converting and zipping a lot of files. I am expecting more than 100.000 files per container with a resulting container size of 10 GB.

What would be the best way to offer a download of the whole container in a specific format (xml, json) which also has been zipped?

Possible solutions I have found that could help are:

  • Appending all blobs of the container to a "result blob" in the desired format. This would double the storage requirements of course http://ift.tt/1dLOQKn
  • Zipping files directly to the output stream http://ift.tt/1ADfY3g

Thanks

Aucun commentaire:

Enregistrer un commentaire