Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Question
Friday, November 7, 2014 10:32 PM | 1 vote
Hi there,
I am a beginner in networking programming. Starting with a basic TCP/IP server and client application,
TCP/IP Server puts a TCP Listener object waiting for TCP Client. Once TCP Client tries to connect with TCP Server, TCP Server accepts the clients and goes in a loop to start receiving data from TCP Client (it receives 4 valid strings from TCP Client before exiting the loop). As TCP client sends the data, TCP server receives the data and display it upon the page.
Now, this code works fine when I use it upon my development machine using the local ip address (127.0.0.1) but when I try this code where TCP Server uses a public hosting server at internet, it gets issues.
While at a public network, client can connect with server and it shows me that while debugging that client is connected with server but when I send data from client to server, server never receives data. I have tried with different ports as well, but do not seem to work fine. Please advice.
Tayyab
All replies (11)
Monday, November 17, 2014 3:00 PM ✅Answered
One question though, with my existing code (without proper Protocol or Message Framing).
I feel that it should show me something as server should receive data (in whatever order). Like after the variable “requestCount” reaches the value of 4, it breaks the loop and display the received data(unordered, unsorted, incomplete etc). So this code should show me the whatever received data. Like some examples are given as following
- M—e--ss--a
- Mess—age—F—ro—m
- Message—From—Clien—t
But this code doesn’t show me anything and goes in infinite loop (that implies that server didn’t receive anything from the client) even though it is connected at given port (checked via debugging).
But this code works fine at development machine, problems comes when i put it at internet where server can connect with client but doesn't receive anything.
TL;DR: Multiple buffers sent by the client may be combined and observed as a single receive buffer by the server. And this is more likely in an internet scenario than a localhost scenario.
Let's say your 4 "strings" are sent by the client discretely -- and by that I mean that each one was sent using a different "Send" call, with a different send buffer. Let's say those four strings are the following:
- "MESSAGE"
- "FROM"
- "THE"
- "CLIENT"
You've correctly understood that the individual message may arrive fragmented. So if you were to concatenate your received data into a single string, and separate it with "--" each time you process all the data in your receive buffer on the server (as per our previous examples in this thread) then it's true that you might observe this final string
- "MESS--AGE--FR-OM--THE--C--LIE--NT--"
This shows that you understand that messages can be fragmented, or divided up. But what I fear you do not understand is that they can also be coalesced, or merged together. So you might also end up with any of these:
- "MESS--AGEF--ROMTHECLI--ENT--" (Notice characters from different strings are concatenated.)
- "M--E--S--S--A--G--E--F--R--O--M--T--H--E--C--L--I--E--N--T--" (this is technically possible, but preposterously unlikely)
- "MESSAGEFROMTHECLIENT--" (This is an important case)
- "MESSAGE--FROMTHECLIENT--" (This is actually the most likely case!)
Notice in the last two cases that you actually have processed FEWER THAN 4 receive buffers on the server. This means that, although you have received all the bytes you are ever going to receive, you haven't reached the condition that will break you out of your loop. The requestCount is still less than 4.
To understand why this is the case it might help to imagine how a postal service or courier service works. Let's say you want to send 4 packages to your friend. You go to the courier's office and you present your first package labelled "A". Then you leave. The courier has a large truck waiting to deliver your package. So far there's only one box, but it goes on the truck. And now the courier service has a problem and has to make a strategic decision.
- Do they tell the truck to leave immediately with a truck carrying only one box? That's very inefficient. But it is the absolutely only way to get the package to its destination in the least amount of time possible. If they decide to wait at all, they are delaying delivery of the package. If they want to uphold their reputation as being a fast courier, they have to leave immediately.
- Do they wait until the truck is full? This is highly efficient, because they only move the truck when it's full of packages, but they risk delaying packages a very long time, or perhaps even indefinitely if nobody sends any more packages. If the truck is never full, it never leaves. If the truck is very large, they may go out of business having never delivered their first package!
- Do they wait a small or even random amount of time, hoping to "split the difference" between the first two strategies? For example, any non-empty truck may leave at midnight, so that we know the delivery time will be delayed at most 24 hours. Not bad, but if the truck doesn't fill up, packages are getting delayed on average 12 hours. Packages sent just before midnight may arrive in a few hours, while packages sent in the morning seem to take 24 hours to arrive.
Case 1 is an important case because we have to consider what happens if you, as quickly as you can, try to send 4 packages.
- You give box "A" to the courier.
- You go back to your car and get box "B" then give box "B" to the courier.
- You go back to your car again and get box "C", and give it to the courier.
- You go back to your car yet again, and get box "D", and give it to the courier.
The courier doesn't know you're going back to the car. So as soon as they get box "A", they load it on the truck and send it off. But you very soon show up with box "B". The bad news now is that the truck is gone. They put box "B" in storage and wait for the truck to come back. This likely is the same fate for box "C" and box "D". They are all sitting in storage waiting for the truck to come back. When it does, the courier loads boxes "B" "C" and "D" on to the truck and immediately send it off again.
Your friend observes one truck arrive carrying box "A". Then some time later observes a second truck arrive carrying boxes "B", "C", and "D". Two trucks total - four packages total.
Obviously the exact details of what happens depends on how fast the courier's delivery vehicle is, how many of them are in service, and how much can be loaded onto each vehicle, as well as traffic conditions, mechanical issues, weather, logistics of storage bays, and the exact timing of each package delivery and vehicle trip. Trucks may move often, but take the package to a hub where it travels by plane or by ship. The last mile may even occur on foot, as it does with letter carriers (the so called "mailman").
Now let's get back to your example. With a "localhost" implementation, the delivery truck is EXTREMELY fast and large because it doesn't have to go anwhere. In fact the truck doesn't even need wheels because it doesn't go anywhere. It's basically just a glorified storage room all by itself. As soon as a package is put in storage, it has arrived because the destination is the same as where it started. So with this kind of scenario, each time you send a package, it arrives immediately. And your server would observe:
- "MESSAGE--FROM--THE--CLIENT--" (localhost scenario)
(NOTE: That the above scenario is how you were magically able to avoid dealing with message framing with your localhost connection. By the happy coincidence that when send buffers are small, receive buffers are empty, and the machine is not busy, the send buffer boundaries may coincide with the receive buffer boundaries. This is a fragile set of conditions though, and you'll find that your app is flakey and you'll get random things happening if you rely on this, because it is not guaranteed to occur.)
If we switch to the internet scenario, where the trucks are actually quite small, and take a non-trivial amount of time to deliver the package, then you're much more likely to have that first truck leave immediately, and then everything else go on the second truck and you'll observe this:
- "MESSAGE--FROMTHECLIENT--" (idle internet scenario)
If the network was idle when that first message was delivered then that's what you'll see. But if the network was actually busy when the first message was sent (due to other network traffic on the system) then that's the same as when the truck is "out for delivery" when you send your first package. The very first package sits in storage until the truck gets back. You can probably send all 4 packages before the truck gets back so you'll see:
- "MESSAGEFROMTHECLIENT--" (busy internet scenario)
(Incidentally, breaks within in the message will more likely occur when the traffic is heavy or the messages are long.)
Does any of this help make it more clear? It certainly sounds like you're starting to get it. I'm hoping this closes the gap in your understanding.
The main takeaway here is that the contents of multiple messages can be combined (or divided into pieces; but never reordered). Picture the delivery truck metaphor to remind yourself how this strategy applies.
If you're looking for additional reading, consider reading about Nagle's algorithm.
Saturday, November 8, 2014 3:42 AM
I'm concerned about the way you describe this: "it receives 4 valid strings from TCP Client before exiting the loop"
You don't receive strings, you just receive bytes. What makes the strings valid or invalid?
I'm concerned that this is may be a message framing issue. Are you aware that a TCP is an ordered stream of bytes, and not an ordered stream of messages? Send buffers are not indivisible messages, they are just ordered bytes, which may be delivered individually or together, or in smaller groups, or delayed until later, bundled with some or all of the bytes of other messages. Please look at this explanation of mine.
Can you describe the application protocol you are using? In other words, can you describe the format of data that you are sending over the TCP connection in each direction? Would your protocol still make sense if it were delivered one byte at a time?
In a loopback scenario, you're much more likely to have a receive buffer be exactly the same as the contents of the send buffer. In an internet scenario, the data will be chopped up differently. If this is, indeed, a message framing issue, then you may actually be avoiding some shortcomings of your protocol implementation due to the way send buffers are delivered on the loopback adapter (which tends to be immediately, and entirely, provided the receiver is responsive - but that's just an implementation quirk, and not a guarantee of TCP and is absolutely not going to happen when dealing with real networks.)
If you've failed to frame your messages in a robust way, then you'll see errors when the network starts chopping up and re-bundling your send buffers into different groupings.
Saturday, November 8, 2014 9:53 AM
I wonder whether this is as simple as being an iis configuration issue.
http://stackoverflow.com/questions/3188618/enabling-net-tcp
Did you do any configuration on iis or just install your stuff?
Or firewall maybe.
If you haven't already, download Fiddler and take a look at what it says.
It's free.
http://www.telerik.com/download/fiddler
Saturday, November 8, 2014 4:02 PM
I am using tcp here if you won't like to compare.
chanmm
chanmm
Saturday, November 8, 2014 10:18 PM
Thank you man for your reply. I found the given link very informative. For clearer picture i am posting my code here.
Server Code
TcpListener serverSocket = new TcpListener(47764);
TcpClient clientSocket = default(TcpClient);
string strResult = "";
int counter = 0;
protected void Page_Load(object sender, EventArgs e)
{
try
{
Response.Write("Server Started");
serverSocket.Start();
counter += 1;
clientSocket = serverSocket.AcceptTcpClient();
Response.Write("Client No:" + Convert.ToString(counter) + " started!");
//After Accepting a client, It initiate a new object and send the
//client object to listen data
handleClinet client = new handleClinet();
strResult = client.startClient(clientSocket, Convert.ToSting(counter));
Response.Write(strResult);
clientSocket.Close();
serverSocket.Stop();
Response.Write("Server Stopped");
}
catch (Exception exp)
{
Response.Write(exp.Message);
serverSocket.Stop();
}
finally
{
}
}
public class handleClinet
{
TcpClient clientSocket;
string clNo;
public string startClient(TcpClient inClientSocket, string clineNo)
{
this.clientSocket = inClientSocket;
this.clNo = clineNo;
return doChat();
}
private string doChat()
{
int requestCount = 0;
byte[] bytesFrom = new byte[10025];
string dataFromClient = "";
Byte[] sendBytes = null;
string serverResponse = null;
string rCount = null;
requestCount = 0;
bool blnDataArrived = false;
//It goes in an infinite loops (unless it gets four network streams having data) and change them //to a string before exiting the loop
while ((true))
{
try
{
NetworkStream networkStream = clientSocket.GetStream();
//Here it checks if network stream has data available
if (networkStream.DataAvailable)
{
//If it has data, it increase the counter by one unless value of four is reached
requestCount = requestCount + 1;
//Here it read the data from stream into bytes and then converts into string
networkStream.Read(bytesFrom, 0, (int)clientSocket.ReceiveBufferSize);
dataFromClient += "--"+ System.Text.Encoding.ASCII.GetString(bytesFrom);
//Here it checks, if it got four valid strings (having data), it sets the flag and //exit the function to display the received data.
if (requestCount > 3)
{
blnDataArrived = true;
}
if (blnDataArrived) return dataFromClient;
}
}
catch (Exception ex)
{
}
}
}
}
Client Code
System.Net.Sockets.TcpClient clientSocket = new System.Net.Sockets.TcpClient();
NetworkStream serverStream = null;
int intCount = 0;
private void Form1_Load(object sender, EventArgs e)
{
try
{
lblInformation.Text = "Client Started";
clientSocket.Connect("127.0.0.1", 47764);
lblInformation.Text += "Client Socket Program - Server Connected";
}
catch (Exception exp)
{
MessageBox.Show(exp.Message);
}
}
//Code for Button, upon clicking this button, client sends the data to server. At local computer, when I cilck this button 4 times, system received four strings and display at screen but when I put this code at public server, it doesn’t receive any data
private void btnSend_Click(object sender, EventArgs e)
{
try
{
NetworkStream serverStream = clientSocket.GetStream();
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(intCount + " Message from Client$");
serverStream.Write(outStream, 0, outStream.Length);
serverStream.Flush();
intCount++;
}
catch (Exception exp)
{
}
finally
{
}
}
Tayyab
Saturday, November 8, 2014 10:23 PM
@chanmm does it work when you put your application at internet ??
Tayyab
Sunday, November 9, 2014 11:05 PM | 1 vote
Yep. That's a message framing problem.
Did you read the link I sent you?
For example: If a client performs 2 discrete sends, First time with the data HELLO and a second send with the data WORLD, then the server might process three discrete receives, The first one receiving the data HEL then the second receiving LOWO then a third with the data RLD.
...or any other regrouping of the bytes so long as the order of the bytes is the same. The client essentially sent HELLOWORLD and the server receives HELLOWORLD.
THIS IS THE ONE THING YOU NEED TO UNDERSTAND: The notion that the beginning and ending of the send buffers being is somehow being conveyed to the receiver or preserved in any way by the TCP protocol is appealing to beginners and may seem logical and intuitive, but it is simply not true.
Just because you put a bunch of bytes into a send buffer doesn't mean that they will be received together. Also just because you send two buffers separately doesn't mean they will be received separately, they may be coalesced into a single buffer or divided up.
Your message protocol needs to make sense even when the send buffers' bytes are concatenated or broken into pieces.
To make this work and to divide up the stream of bytes into messages this we usually either make use of a delimiter such as a special reserved character like a comma or a new-line character. Or we write a fixed-size message length field into the stream. For example if we use exactly 1 byte to write out the length of the message, that follows it, then we can send messages up to 255 bytes in length. If we use 2 bytes for the size field, then we can send messages up to 65535 bytes in length.
When client and server agree on a way to make sense of the bytes they send back and forth, that's called a protocol. When you agree on how to package up and discrete messages into a stream of ordered bytes and how to take a stream of ordered bytes and break them apart into discrete messages, that's called message framing.
Choosing a protocol is not optional. Message framing is not optional.
Are you following me?
Specifically in your example, I notice that each time you receive some bytes, you increment a "requestCount". This perfectly embodies your misunderstanding of TCP. Just because you received some bytes, doesn't mean you have a complete request, you may have only part of one, with the remaining bytes still "in flight" to be processed in a subsequent receive call.
Also you append the contents of the receive buffer to a "dataFromClient" string, along with a "--" delimiter. This is wrong. If you send "HELLO" you may find that your "dataFromClient" string contains "--HE--LL--O" after processing three receives.
You need to concatenate bytes received by the server and process them byte at a time until you determine that you have a complete message. Then you can process the bytes of that message in its entirety. For example, you can use a protocol where you send a 0x00 (zero) byte between command strings. This is relatively easy to parse on the receiving side to combine or break up the receive buffers up into the original strings you sent. You just keep putting bytes into a buffer until you see a 0x00. then process the bytes you have so far as a message, then clear the buffer and resume parsing received bytes. This is just one example way of message framing -- with a single byte delimiter. You could also choose to use a 0xFF byte as the delimiter. This may be a little more robust because 0xFF isn't a valid ASCII character and you seem to have chosen an ASCII encoding for your strings.
I would be remiss not to warn you that UTF-8 may encode single characters as more than one byte, and that may fall on a receive buffer boundary -- so you have to be super careful how you reassemble UTF-8 encoded string bytes. This is where message length fields are better than delimiters as a protocol choice.
Saturday, November 15, 2014 9:00 PM
HI There,
I am immensely thankful for detailed answer along with explanation.
Your answer totally makes sense to and after reading your answer, I revisited the link you sent me earlier along with some articles regarding message framing and I found that I was doing it in a beginner way that needs to be changed.
I am going to work upon that along with a protocol and message framing mechanism and hopefully it will work.
One question though, with my existing code (without proper Protocol or Message Framing).
I feel that it should show me something as server should receive data (in whatever order). Like after the variable “requestCount” reaches the value of 4, it breaks the loop and display the received data(unordered, unsorted, incomplete etc). So this code should show me the whatever received data. Like some examples are given as following
- M—e--ss--a
- Mess—age—F—ro—m
- Message—From—Clien—t
But this code doesn’t show me anything and goes in infinite loop (that implies that server didn’t receive anything from the client) even though it is connected at given port (checked via debugging).
But this code works fine at development machine, problems comes when i put it at internet where server can connect with client but doesn't receive anything.
I am not sure if I conveyed my point well enough OR even if my point is a valid one but if it is, I would appreciate if you can share your input regarding my point.
Regards
Tayyab
Tuesday, November 18, 2014 8:55 PM
Hi there,
Your time and details regarding the mentioned scenario are much appreciated.
I will definitely spend more time with protocol and message framing and will try to put the communication in order.
One more question outside the current context, Do you think that if the specified port (to connect) is closed via firewall, it can reject the incoming data even though it allowed the client to connect with server ??
Tayyab
Wednesday, November 19, 2014 3:23 AM
One more question outside the current context, Do you think that if the specified port (to connect) is closed via firewall, it can reject the incoming data even though it allowed the client to connect with server ??
No. I don't think that at all. If the firewall allowed the connection then it wouldn't reject the data. If the port is blocked then it wouldn't allow the connection.
[Technically, proxies and VPNs could mess with traffic however they want though. They may allow a connection and then drop it when it fails to adhere to a specific protocol.]
Monday, December 1, 2014 8:29 PM
Hi there,
I just want to thank you for guidance. Based upon your suggestions, i made sure that i am sending and receiving the data in measured lengths. While doing so, i came across the fact that sending and receiving byte buffers should be exactly of same size as i saw this error at server end.
Specified argument was out of the range of valid values. Parameter name: sizeExit
After observing this error, i fixed the length of sent and received data and it worked like a charm... :-)
I definitely need to work more for message framing and protocols and gonna do that.
Meanwhile, i am looking forward to receive data from a GPS device that should send data via TCP/IP at a specific port. I tried the same program with this device but it didn't work.
Do you have any particular suggestion for this.
Thank you again man. God bless u.
Tayyab