C# read binary data in small chunks

Question

_{Tuesday, January 30, 2018 9:57 PM}

Hi, I am using the below code to read binary data that needs to be processed. The binary data size that's being read into the memory keeps growing lately. I am looking at options to read data in smaller chunks into the memory process it and free it. Could you please guide to re-write it accordingly.

public static byte[] ReadFully (Stream stream)
{
    byte[] buffer = new byte[32768];
    using (MemoryStream ms = new MemoryStream())
    {
        while (true)
        {
            int read = stream.Read (buffer, 0, buffer.Length);
            if (read <= 0)
                return ms.ToArray();
            ms.Write (buffer, 0, read);
        }
    }
}

Thank you in advance.

SQLEnthusiast

All replies (4)

_{Tuesday, January 30, 2018 10:05 PM ✅Answered}

Why not using Stream.CopyTo()?

_{Tuesday, January 30, 2018 11:14 PM}

Hi, I re-wrote the code and get the error: Maximum request length exceeded.

        private byte[] ToByteArray(Stream stream)
        {
            int initialLength = 32768;
            byte[] buffer = new byte[initialLength];
            long read = 0;

            int chunk;
            while ((chunk = stream.Read(buffer, (int)read, buffer.Length - (int)read)) > 0) //Error: Maximum request length exceeded.
            {
                read += chunk;
                if (read == buffer.Length)
                {
                    int nextByte = stream.ReadByte();

                    if (nextByte == -1)
                    {
                        return buffer;
                    }

                    byte[] newBuffer = new byte[buffer.Length * 2];
                    Array.Copy(buffer, newBuffer, buffer.Length);
                    newBuffer[read] = (byte)nextByte;
                    buffer = newBuffer;
                    read++;
                }
            }
            byte[] ret = new byte[read];
            Array.Copy(buffer, ret, read);
            return ret;
        }

Thank you.

SQLEnthusiast

_{Wednesday, January 31, 2018 6:38 AM}

Hi, @Stefan Hoffman, Thanks for your response. Even after making changes to the code using CopyTo() I get Error: Maximum request length exceeded.

public static byte[] ReadFully(Stream stream)
{
    using (MemoryStream ms = new MemoryStream())
    {
        stream.CopyTo(ms);
        return ms.ToArray();
    }
}

Regards

SQLEnthusiast

_{Thursday, August 9, 2018 9:59 AM}

What if I'm using .NET 3.5? There no such kind of possibility.
I'm using the same kind of code but length of stream very big (gigabytes) new byte[4447329319].
How to do in this case? Should I split length to several bytes[] and use one after another?