Share via

IO Exception when sending large size of bytes using HttpWebRequest

  • Article

Question

Monday, November 4, 2019 4:41 PM

We got this IO exception when sending relatively bigger size of bytes to WebAPI.

The IO exception happened when sending larger size of bytes from httpwebrequest.

We are writing pure .net C# code to call a web API by using HttpWebRequest.

The code is very similar to this page:

https://stackoverflow.com/questions/9145667/how-to-post-json-to-a-server-using-c

code :

 var httpWebRequest = (HttpWebRequest)WebRequest.Create("http://url");
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";

using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
    string json = "{\user\:\test\," +
                  "\password\:\bla\}";

    streamWriter.Write(json);
}

var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
    var result = streamReader.ReadToEnd();
}

We just using Json.net lib to convert object to json string. 

When I changed to use real large size of bytes, it throws exception as following: 

Exception details:

System.IO.IOException was unhandled
  HResult=-2146232800
  Message=Unable to write data to the transport connection: An established connection was aborted by the software in your host machine.
  Source=System
  StackTrace:
       at System.Net.Sockets.NetworkStream.MultipleWrite(BufferOffsetSize[] buffers)
       at System.Net.Security._SslStream.StartWriting(SplitWritesState splitWrite, SplitWriteAsyncProtocolRequest asyncRequest)
       at System.Net.Security._SslStream.ProcessWrite(BufferOffsetSize[] buffers, SplitWriteAsyncProtocolRequest asyncRequest)
       at System.Net.TlsStream.MultipleWrite(BufferOffsetSize[] buffers)
       at System.Net.PooledStream.MultipleWrite(BufferOffsetSize[] buffers)
       at System.Net.ConnectStream.InternalWrite(Boolean async, Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
       at System.Net.ConnectStream.Write(Byte[] buffer, Int32 offset, Int32 size)
       at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
       at System.IO.StreamWriter.Write(String value)
       at testREST_referral.Class1.Main(String[] args) in C:\Users\user.name\Documents\test\testREST_ref\testRESTl\testREST_ref\Program.cs:line 179
       at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
       at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException: 
       ErrorCode=10053
       HResult=-2147467259
       Message=An established connection was aborted by the software in your host machine
       NativeErrorCode=10053
       Source=System
       StackTrace:
            at System.Net.Sockets.Socket.MultipleSend(BufferOffsetSize[] buffers, SocketFlags socketFlags)
            at System.Net.Sockets.NetworkStream.MultipleWrite(BufferOffsetSize[] buffers)
       InnerException: 

All replies (15)

Monday, November 4, 2019 7:52 PM

You are not using try/catch for the StreamWriter.Write Method. You should do that before asking for assistance. That exception will be more useful. Look at the inner exception(s) too if there is one or more. If that exception (or those exceptions) do not help then post that here.

Sam Hobbs
SimpleSamples.Info

Monday, November 4, 2019 8:05 PM

Hello HappyMaggie,

I have not seen this one before so I did some digging. Looks like this is a pretty generic error so you will need to dive a bit deeper to determine the cause. 

When you say a large number of bytes, what is that? Is there a duration related to this? I am assuming the code works against the same target endpoint for smaller payloads and this error is only happening with large payloads.

Here are some things to investigate:

Cheers, Jeff

Monday, November 4, 2019 8:35 PM

Thank you Jeff. Yes, it is only happen on large size payload. It passed small size on payload.

Yes, I put KeepAlive = true.

Also, it looks like not related to firewall or virus scanner, since I was able to pass large payload by using the SOAP web API. This failure is happened on REST web API.  

I will add tracing at a lower level as you suggested.

I am using HttpWebRequest. I am reading some doc from Microsoft and it seems httpClient is better version than HttpWebRequest? 

Also, I am trying to use HttpClient.PutAsync to test. 

thanks

M.

Monday, November 4, 2019 8:39 PM

Hello HappyMaggie,

I have not seen this one before so I did some digging. Looks like this is a pretty generic error so you will need to dive a bit deeper to determine the cause. 

When you say a large number of bytes, what is that? Is there a duration related to this? I am assuming the code works against the same target endpoint for smaller payloads and this error is only happening with large payloads.

Here are some things to investigate:

Cheers, Jeff

My suggestion is how the experts would dig in. I think you are distracting HappyMaggie from a very productive path.

Sam Hobbs
SimpleSamples.Info

Monday, November 4, 2019 8:45 PM

The problem is on the ASP.NET WebAPI service-side and not on the HTTP client-side.

https://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/

Monday, November 4, 2019 9:01 PM

Thank you SA924x. 

I need to clarify something here:

1) I am writing a pure .net code to call a web API which is not in .net ( in oracle service bus)

2) If I used SOAP message API provided at OSB side, it passed large size of payload.

3) If I used REST message API provided at OSB side, for small payload, I can see it worked. But when the payload is bigger, I cannot see the message even hit the OSB. The caller side ( .net code) caught exception as I indicated before ( IO exception). 

4) Also, it is stranger that I can see once the same code with large payload worked once!

Thanks

M.

Monday, November 4, 2019 9:21 PM

Try KeepAlive set to false (you might get lucky).

Also, your post below says you are connecting to the Oracle Service Bus. Do you have any logs indicating that a connection was rejected due to message size limitation?

Are these files over 10MB? You might find that the service bus is just protecting itself. If you don't need to parse within the service bus you might want to look at transferring large files in smaller batches and reassembling downstream.

Cheers, Jeff

Monday, November 4, 2019 9:41 PM

Thank you Jeff. 

There is no inbound message for OSB at all when there is an exception from .net side. 

The passed size is 26kb. Once over that size, we got failure at .net side.

Yes, it is intermittent. It is really difficult to see why this exception happened. The problem is at .net side, not OSB.

Monday, November 4, 2019 9:41 PM

The problem is on the ASP.NET WebAPI service-side and not on the HTTP client-side.

https://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/

Are you saying that the exception reported in the original post is not occurring in the code that was posted in the original post? If the exception is occurring in the posted code then the exception should be caught by it.

Sam Hobbs
SimpleSamples.Info

Monday, November 4, 2019 11:18 PM

1) I am writing a pure .net code to call a web API which is not in .net ( in oracle service bus)

Then an Oracle forum is where you should be posting to. The default data receive settings  may be set to a default settings or are not large enough that may need to be adjusted. 

In general, I don't think  the error  has anything to do with the client side, but rather,  it's on the  Oracle service-side.

Tuesday, November 5, 2019 12:02 AM

Hello,

Try the following.

var httpWebRequest = (HttpWebRequest)WebRequest.Create("http://url");
httpWebRequest.ContentType = "application/json";
httpWebRequest.KeepAlive = false;
httpWebRequest.Method = "POST";
httpWebRequest.ProtocolVersion = HttpVersion.Version10;
httpWebRequest.ServicePoint.ConnectionLimit = 1;

using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
    string json = "{\"user\":\"test\"," +
                  "\"password\":\"bla\"}";

    streamWriter.Write(json);
}

var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
    var result = streamReader.ReadToEnd();
}

Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

NuGet BaseConnectionLibrary for database connections.

StackOverFlow

Tuesday, November 5, 2019 12:33 AM

hello HapyMaggie,

26kb is not that large so I am surprised. I would not be too quick to target the client (.net side). Most of these type of errors stem from the server (or a firewall/load balancer) trying to protect itself.

These errors often require looking at the lower level message exchange (hence why I suggested putting the network tracing on). You might be surprised at the amount of communication that happens establishing the connection.

Cheers, Jeff

Tuesday, November 5, 2019 4:51 PM

Thank you Jeff. I enabled trace. That gave me more information. 

The error is still showing on this line

          request.ContentLength = arr.Length;

            Stream dataStream = request.GetRequestStream();
            dataStream.Write(arr, 0, arr.Length);        // ==> exception here
            dataStream.Close();

exception thrown: 'System.IO.IOException' in System.dll
System.Net Error: 0 : [15100] Exception in AppDomain#11478475::UnhandledExceptionHandler - Unable to write data to the transport connection: An established connection was aborted by the software in your host machine..
   at System.Net.ConnectStream.InternalWrite(Boolean async, Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
   at System.Net.ConnectStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at TestUploadFiles.Class1.Main(String[] args) in C:\Users\test\Documents\testFiles\Program.cs:line 91
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
The program '[23692] TestUploadFiles.vshost.exe' has exited with code -1 (0xffffffff).

Tuesday, November 5, 2019 6:30 PM

if I use another way to send bytes, it just silently failed

ystem.Net.Sockets Verbose: 0 : [19764] Exiting Socket#51220585::Send() -> Int32#277
System.Net Verbose: 0 : [19764] Exiting HttpWebRequest#56584411::GetRequestStream() -> ConnectStream#47629573
System.Net Verbose: 0 : [19764] Entering ConnectStream#47629573::Write()
System.Net Verbose: 0 : [19764] Data from ConnectStream#47629573::Write
System.Net Verbose: 0 : [19764] 00000000 : 53 79 73 74 65 6D 2E 42-79 74 65 5B 5D          : System.Byte[]
System.Net.Sockets Verbose: 0 : [19764] Entering Socket#51220585::MultipleSend()
System.Net.Sockets Verbose: 0 : [19764] Exiting Socket#51220585::MultipleSend() 
System.Net Verbose: 0 : [19764] Exiting ConnectStream#47629573::Write() 
System.Net Verbose: 0 : [19764] Entering ConnectStream#47629573::Close()
System.Net.Sockets Verbose: 0 : [19764] Entering Socket#51220585::Send()
System.Net.Sockets Error: 0 : [19764] Socket#51220585::UpdateStatusAfterSocketError() - ConnectionAborted
System.Net.Sockets Error: 0 : [19764] Exception in Socket#51220585::Send - An established connection was aborted by the software in your host machine.
System.Net.Sockets Verbose: 0 : [19764] Exiting Socket#51220585::Send() -> Int32#0
System.Net Verbose: 0 : [19764] Exiting ConnectStream#47629573::Close() 
System.Net Verbose: 0 : [19764] Entering HttpWebRequest#56584411::GetResponse()
System.Net.Sockets Verbose: 0 : [19764] Entering Socket#51220585::Receive()
System.Net.Sockets Error: 0 : [19764] Socket#51220585::UpdateStatusAfterSocketError() - ConnectionAborted
System.Net.Sockets Error: 0 : [19764] Exception in Socket#51220585::Receive - An established connection was aborted by the software in your host machine.
System.Net.Sockets Verbose: 0 : [19764] Exiting Socket#51220585::Receive() -> Int32#0
System.Net.Sockets Verbose: 0 : [19764] Entering Socket#51220585::Dispose()
System.Net Error: 0 : [19764] Exception in HttpWebRequest#56584411:: - The underlying connection was closed: An unexpected error occurred on a receive..
System.Net Error: 0 : [19764] Exception in HttpWebRequest#56584411::GetResponse - The underlying connection was closed: An unexpected error occurred on a receive..
Exception thrown: 'System.Net.WebException' in System.dll
The thread 0x7ed0 has exited with code 0 (0x0).
The thread 0x4d34 has exited with code 0 (0x0).

Tuesday, November 5, 2019 8:07 PM

The defacto is to use HTTPClient().

https://johnthiriet.com/efficient-post-calls/#