Hi guys, I've been experimenting with some tcp/ip socket programming on windows using winsock2. Just trying to learn it so I've read some tutorials on msdn and saw some videos on youtube, etc. I think I understand how the basics of it work. So I got some very simple server/client programs to compile and learned about the blocking/non-blocking situation. After searching/experimenting around some more I seem to understand the select function and it's kind of queue it manages to handle multiple clients in one thread.
So I have a very simple server/client par which handles sending mouse positions from one client to the others, it seems to work for the little that I tested it, of course many things are done wrong since it's just some quick experimentation to learn about it.
Still, I was wondering, maybe some expert folks out there can give me some advice about which direction I should go from here, what resources I can use to learn the proper way of doing this kind of networking for games.
I have a couple questions about it:
As I understand so far, the recv funcion seems to block (freeze and wait for messages until one is received) unless told otherwise.
The way we tell a connection is to be non-blocking seems to be with the ioctlsocket function using the FIONBIO flag. The select function is then supposed to handle the non-blocking conversation using the FD_ZERO, FD_SET and FD_CLR macros. That is all good (assuming I even got this part correctly) and it seems to work on the server side. However, once I openend a window on the client side and called the recv function to get data from the server the recv call on the client side would still block resulting in the window not responding (as I understand since the message loop from windows were not getting processed anymore). So I tried to find an answer to what I have been doing wrong but couldn't find anything after some time searching. At this point I remembered about the multithreading episode of Handmade Hero and I thought it would be a solution to just launch a thread to call recv and that way it could block but the main thread could still process windows messages and all is good (remember we're talking about the client here, the server still uses select and just one thread). It does work but my question is should the recv function on client side really be blocking when I'm saying the socket is non-blocking? I suspect I might have done something wrong in this code. The full source code is at the end.
I have one other question which is possibly quite abstract and noob-like. So I get we send bytes back and forward. My first thought was to make a struct with the network information I wanted to send but after a quick google it seems that is a bad idea because different endianess and packing of the structs. My question is: what is a good model for sending meaningful game data over tcp/ip from clients to the sever and from server to clients? I could think of some really simple solutions like prefixing numbers with some code like mx100,my200 for mouse positions and stuff like that, but it doesn't seem like a very good approach. So I am hoping there are some experience folks out there who can tell me where to look and what to read from here so I can continue to experiment with this stuff. Anyway, that is it for now, thanks guys!
Full noob source code:
server.cpp
client.cpp:
So I have a very simple server/client par which handles sending mouse positions from one client to the others, it seems to work for the little that I tested it, of course many things are done wrong since it's just some quick experimentation to learn about it.
Still, I was wondering, maybe some expert folks out there can give me some advice about which direction I should go from here, what resources I can use to learn the proper way of doing this kind of networking for games.
I have a couple questions about it:
As I understand so far, the recv funcion seems to block (freeze and wait for messages until one is received) unless told otherwise.
The way we tell a connection is to be non-blocking seems to be with the ioctlsocket function using the FIONBIO flag. The select function is then supposed to handle the non-blocking conversation using the FD_ZERO, FD_SET and FD_CLR macros. That is all good (assuming I even got this part correctly) and it seems to work on the server side. However, once I openend a window on the client side and called the recv function to get data from the server the recv call on the client side would still block resulting in the window not responding (as I understand since the message loop from windows were not getting processed anymore). So I tried to find an answer to what I have been doing wrong but couldn't find anything after some time searching. At this point I remembered about the multithreading episode of Handmade Hero and I thought it would be a solution to just launch a thread to call recv and that way it could block but the main thread could still process windows messages and all is good (remember we're talking about the client here, the server still uses select and just one thread). It does work but my question is should the recv function on client side really be blocking when I'm saying the socket is non-blocking? I suspect I might have done something wrong in this code. The full source code is at the end.
I have one other question which is possibly quite abstract and noob-like. So I get we send bytes back and forward. My first thought was to make a struct with the network information I wanted to send but after a quick google it seems that is a bad idea because different endianess and packing of the structs. My question is: what is a good model for sending meaningful game data over tcp/ip from clients to the sever and from server to clients? I could think of some really simple solutions like prefixing numbers with some code like mx100,my200 for mouse positions and stuff like that, but it doesn't seem like a very good approach. So I am hoping there are some experience folks out there who can tell me where to look and what to read from here so I can continue to experiment with this stuff. Anyway, that is it for now, thanks guys!
Full noob source code:
server.cpp
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | #define WIN32_LEAN_AND_MEAN #include <windows.h> #include <stdio.h> #include <ws2tcpip.h> int WinMain( HINSTANCE Instance, HINSTANCE PrevInstance, LPSTR CmdLine, int ShowCmd) { AllocConsole(); freopen("CONOUT$", "w+", stdout); WSADATA WsData; WORD Ver = MAKEWORD(2, 2); int WsOK = WSAStartup(Ver, &WsData); if(WsOK != 0) return -1; SOCKET ListeningSocket = socket(AF_INET, SOCK_STREAM, 0); if(ListeningSocket == INVALID_SOCKET) return -2; sockaddr_in Hint; Hint.sin_family = AF_INET; Hint.sin_port = htons(54000); Hint.sin_addr.S_un.S_addr = INADDR_ANY; u_long Mode = 0; ioctlsocket(ListeningSocket, FIONBIO, &Mode); bind(ListeningSocket, (sockaddr *)&Hint, sizeof(Hint)); listen(ListeningSocket, SOMAXCONN); fd_set MasterSet; FD_ZERO(&MasterSet); FD_SET(ListeningSocket, &MasterSet); while(true) { fd_set CopySet = MasterSet; int SocketCount = select(0, &CopySet, 0, 0, 0); for(int i=0; i<SocketCount; ++i) { SOCKET Socket = CopySet.fd_array[i]; if(Socket == ListeningSocket) { printf("Client connected.\n"); SOCKET ClientSocket = accept(ListeningSocket, 0, 0); int NonBlocokingRequestResult; u_long NonBlockingMode = 0; NonBlocokingRequestResult = ioctlsocket(ClientSocket, FIONBIO, &NonBlockingMode); if(NonBlocokingRequestResult == SOCKET_ERROR) { printf("Error: %d.\n", WSAGetLastError()); } FD_SET(ClientSocket, &MasterSet); char *WelcomeMessage = "Welcome to the test server!"; send(ClientSocket, WelcomeMessage, strlen(WelcomeMessage) + 1, 0); } else { char BufferIn[4096]; ZeroMemory(BufferIn, 4096); int BytesIn = recv(Socket, BufferIn, 4096, 0); if(BytesIn <= 0) { printf("Client disconnected.\n"); closesocket(Socket); FD_CLR(Socket, &MasterSet); } else { for(int i=0; i<MasterSet.fd_count; ++i) { SOCKET SocketOut = MasterSet.fd_array[i]; if(SocketOut != ListeningSocket && SocketOut != Socket) { printf("CLIENT> %s\n", BufferIn); send(SocketOut, BufferIn, BytesIn, 0); } } } } } } WSACleanup(); return(0); } |
client.cpp:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 | #define WIN32_LEAN_AND_MEAN #include <windows.h> #include <windowsx.h> #include <stdio.h> #include <string.h> #include <stdlib.h> #include <conio.h> #include <process.h> #include <ws2tcpip.h> int ShouldClose = false; int MousePosX; int MousePosY; int MouseLeftDown; int LastSentMousePosX; int LastSentMousePosY; char DataReceived[4096] = {}; char DataSent[1024] = {}; SOCKET Socket; DWORD NetworkThreadID; DWORD NetworkThreadProc(LPVOID LParam) { DWORD Result = 0; ZeroMemory(DataReceived, 4096); int BytesReceived = 1; while(BytesReceived > 0) { BytesReceived = recv(Socket, DataReceived, 4096, 0); if(BytesReceived < 0) { int Error = WSAGetLastError(); } else if(BytesReceived > 0) { printf("SERVER> %s\n", DataReceived); } else { printf("Disconnected.\n"); } Sleep((1.0f/30)*1000); } return(Result); } LRESULT MainWindowProc( HWND Window, UINT Message, WPARAM WParam, LPARAM LParam) { LRESULT Result = 0; switch(Message) { case WM_LBUTTONDOWN: case WM_LBUTTONUP: { MouseLeftDown = (Message == WM_LBUTTONDOWN); } break; case WM_MOUSEMOVE: { MousePosX = GET_X_LPARAM(LParam); MousePosY = GET_Y_LPARAM(LParam); } break; case WM_CLOSE: case WM_DESTROY: { ShouldClose = true; } break; default: { Result = DefWindowProc(Window, Message, WParam, LParam); } break; } return(Result); } int WinMain( HINSTANCE Instance, HINSTANCE PrevInstance, LPSTR CmdLine, int ShowCmd) { AllocConsole(); freopen("CONOUT$", "w+", stdout); char *IpAddress = "127.0.0.1"; int Port = 54000; WSAData Data; WORD Version = MAKEWORD(2, 2); int WsResult = WSAStartup(Version, &Data); if(WsResult != 0) return -1; Socket = socket(AF_INET, SOCK_STREAM, 0); if(Socket == INVALID_SOCKET) { WSACleanup(); return -2; } sockaddr_in Hint; Hint.sin_family = AF_INET; Hint.sin_port = htons(Port); inet_pton(AF_INET, IpAddress, &Hint.sin_addr); int ConnResult = connect(Socket, (sockaddr *)&Hint, sizeof(Hint)); if(ConnResult == SOCKET_ERROR) { closesocket(Socket); WSACleanup(); return -3; } WNDCLASSA WindowClass={}; WindowClass.style = CS_OWNDC|CS_HREDRAW|CS_VREDRAW; WindowClass.lpfnWndProc = MainWindowProc; WindowClass.hInstance = Instance; // WindowClass.hCursor; WindowClass.lpszClassName = "MainWindowClassName"; RegisterClassA(&WindowClass); HWND Window = CreateWindowExA( 0, WindowClass.lpszClassName, "Client for Networking Exp 2", // WS_OVERLAPPEDWINDOW, WS_OVERLAPPEDWINDOW|WS_VISIBLE, CW_USEDEFAULT, CW_USEDEFAULT, 400, 400, 0, 0, Instance, 0 ); HANDLE NetworkThread = CreateThread( 0, 0, NetworkThreadProc, 0, CREATE_SUSPENDED, &NetworkThreadID ); ResumeThread(NetworkThread); while(!ShouldClose) { MSG Message = {}; while(PeekMessageA(&Message, 0, 0, 0, PM_REMOVE)) { TranslateMessage(&Message); DispatchMessageA(&Message); } if(MouseLeftDown) { if(MousePosX != LastSentMousePosX || MousePosY != LastSentMousePosY) { LastSentMousePosX = MousePosX; LastSentMousePosY = MousePosY; ZeroMemory(DataSent, 1024); char XValueStr[32]; char YValueStr[32]; itoa(MousePosX, XValueStr, 10); itoa(MousePosY, YValueStr, 10); strcat(DataSent, XValueStr); strcat(DataSent, ","); strcat(DataSent, YValueStr); ZeroMemory(DataReceived, 4069); int SendResult = send(Socket, DataSent, strlen(DataSent) + 1, 0); if(SendResult == SOCKET_ERROR) { } } } Sleep((1/30.0f)*1000); } TerminateThread(NetworkThread, 0); closesocket(Socket); WSACleanup(); return(0); } |
Edited by Rafael Abreu
on
Reason: Initial post
I think you are misunderstanding what does "nonblocking" mean for socket. It is a flag for OS to know whether recv function should wait for data or not. It is not about server informing client whether it should block or not. Each side need to set their own socket to nonblocking mode themselves.
In basic form there are three ways how you deal with sockets in asynchronous (non blocking) way:
1) polling mechanism - you set sockets to nonblocking mode and then simply call recv in a loop. As you could imagine this will waste performance if there is nothing else to do in a loop.
2) using select - do not use nonblocking mode, but you check which sockets are read to be read/written (with timeout 0) with "select" call and if there are any, you can safely read/write to them and recv/send calls won't block.
3) multithreading - do not use nonblocking mode, but just call recv from separate thread. Only then you need to deal with correctly passing read data back to your main or other worked threads.
You can see basic forms of these approaches in this article (includes example code): https://unixism.net/2019/04/linux...cations-performance-introduction/ It also shows POSIX specific way with fork'ing (not relevant to Windows)
Once you will want to do something more advanced, for performance reasons, you cannot do any of above approaches. They won't scale for large amount of clients or data. Then you need to use IOCP (I/O Completion Ports) on Windows, or epoll on Linux, or kqueue on BSD. These are all mechanisms how to efficiently receive data form large amount of sockets and to dispatch to threads for processing it. On newer Linux'es there is now alternative called io_uring. Here is basic tutorial on it.
Sending structures on socket is completely OK way to do it. I have worked on multiple successful commercial products that does this. If you are worried about big-endian vs little-endian, you should ask yourself - do you want to really support big-endian platform? Which platform is that? There are not many live hardware out there that is big endian. But yeah, if you need to handle it, you'll need to take extra care about it. Still - memcpy'ing structures will still work, all you'll need to do is reverse bytes for int types on receive side (or sender side - up to you).
As for padding/packing of structs - again question whether you need to support something that behaves so differently? You just need to know what compiler is doing on platforms you are compiling code, and how to make compiler to do what you want. It's all doable.
Most common alternative is to do custom serialization directly on bytes. Don't send strings as text strings. For example, if you have two integers to send, you just write 4 and 4 bytes always as little endian (if you care about endianness). Then receiving side will always read 4 and 4 bytes to get the message. This is basically part of you defining protocol how to parties will talk to each other. It is similar concept how file formats indicate what do they contain - for example, how bmp or png file tells you it's width and height. It has header that describes data coming after the header.
In basic form there are three ways how you deal with sockets in asynchronous (non blocking) way:
1) polling mechanism - you set sockets to nonblocking mode and then simply call recv in a loop. As you could imagine this will waste performance if there is nothing else to do in a loop.
2) using select - do not use nonblocking mode, but you check which sockets are read to be read/written (with timeout 0) with "select" call and if there are any, you can safely read/write to them and recv/send calls won't block.
3) multithreading - do not use nonblocking mode, but just call recv from separate thread. Only then you need to deal with correctly passing read data back to your main or other worked threads.
You can see basic forms of these approaches in this article (includes example code): https://unixism.net/2019/04/linux...cations-performance-introduction/ It also shows POSIX specific way with fork'ing (not relevant to Windows)
Once you will want to do something more advanced, for performance reasons, you cannot do any of above approaches. They won't scale for large amount of clients or data. Then you need to use IOCP (I/O Completion Ports) on Windows, or epoll on Linux, or kqueue on BSD. These are all mechanisms how to efficiently receive data form large amount of sockets and to dispatch to threads for processing it. On newer Linux'es there is now alternative called io_uring. Here is basic tutorial on it.
Sending structures on socket is completely OK way to do it. I have worked on multiple successful commercial products that does this. If you are worried about big-endian vs little-endian, you should ask yourself - do you want to really support big-endian platform? Which platform is that? There are not many live hardware out there that is big endian. But yeah, if you need to handle it, you'll need to take extra care about it. Still - memcpy'ing structures will still work, all you'll need to do is reverse bytes for int types on receive side (or sender side - up to you).
As for padding/packing of structs - again question whether you need to support something that behaves so differently? You just need to know what compiler is doing on platforms you are compiling code, and how to make compiler to do what you want. It's all doable.
Most common alternative is to do custom serialization directly on bytes. Don't send strings as text strings. For example, if you have two integers to send, you just write 4 and 4 bytes always as little endian (if you care about endianness). Then receiving side will always read 4 and 4 bytes to get the message. This is basically part of you defining protocol how to parties will talk to each other. It is similar concept how file formats indicate what do they contain - for example, how bmp or png file tells you it's width and height. It has header that describes data coming after the header.
Edited by Mārtiņš Možeiko
on
Ok so I see I was indeed misunderstanding how it works. I will experiment more with different ways of handling sockets. All the links you provided seem very good and interesting. I will come back to them until I can make sense of it all.
About sending structs over sockets it seems I was too hasty to discard it because of something I read randomly, so it's great news it is a viable option because it feels good to me to have a structure of what the data looks like. I realise I was too vague about the issues that could have when I said something about endianess and packing, those were just things that I read could be a problem but weren't really my thoughts so I see now I should just test it and see if they become a problem on my own.
I see what you mean about sending a header and information that follows a protocol I can establish, that makes a lot of sense.
So yea, I will continue to mess around with this stuff some more.
Again, I will continue to read on the info you sent me until it all sinks in properly. Thanks a lot for taking the time to help me out!
About sending structs over sockets it seems I was too hasty to discard it because of something I read randomly, so it's great news it is a viable option because it feels good to me to have a structure of what the data looks like. I realise I was too vague about the issues that could have when I said something about endianess and packing, those were just things that I read could be a problem but weren't really my thoughts so I see now I should just test it and see if they become a problem on my own.
I see what you mean about sending a header and information that follows a protocol I can establish, that makes a lot of sense.
So yea, I will continue to mess around with this stuff some more.
Again, I will continue to read on the info you sent me until it all sinks in properly. Thanks a lot for taking the time to help me out!
I also have a very basic question. I've been trying to figure this stuff out too. Does the select() function work for udp applications too. I've only ever seen it used in tcp, so I couldn't tell.
I've been wanting to reverse engineer rollback netcode as a personal project.
I've been wanting to reverse engineer rollback netcode as a personal project.
Yes, it does work.
thank you