Azure batch resourceFiles limitation

Question

_{Thursday, February 26, 2015 3:51 AM}

I encountered an error "Total size of resourceFiles cannot be more than 32768 characters" when trying to add some resource files of the start task.

Is it a hard limitation that the length of serialized string of ResourceFiles property couldn't exceed 32768? Is there any way to bypass it?

All replies (1)

_{Thursday, February 26, 2015 5:42 PM ✅Answered}

Hi Charles,

Right now the best trick to handle this is to zip your resource files before you upload them to the storage account, and then unzip them when your task starts.  There are a few advantages to doing this:

Advantages:

1. Resource file upload and download should be quicker, because the data is compressed
2. You don’t hit the limit on resource file property length which you are seeing (you can have an arbitrarily huge resource file list and it will still end up as 1 zipped file).

Disadvantages:

1. Right now, we don’t provider a helper or way to unzip your files on your behalf on the VM, so you need to include the unzipping process on the command line of all of your tasks – something like “Unzip.exe <myZipFilename> && MyExeName.exe <cmdLineArgs>”

You can see these stackoverflow articles for a list of possible ways to unzip from the command line:

http://stackoverflow.com/questions/1021557/how-to-unzip-a-file-using-the-command-line

http://stackoverflow.com/questions/17546016/how-can-you-zip-or-unzip-from-the-command-prompt-using-only-windows-built-in-ca

7zip or a powershell script seem promising, or if you’re more comfortable with C# you can write a quick tool using System.IO.Compression.ZipFile:

        public static void ExtractToDirectory(string zipFilePath, string destinationDirectoryPath)
        {
            ZipFile.ExtractToDirectory(zipFilePath, destinationDirectoryPath);
        }

Thanks and hope this helps,
-Matt