Hi all,
I was wondering if someone might do a Networking guide. The Beej's Networking Guide isn't that great and a little hard to follow. LibCurl does make things a little easier or pcap. Getting in networking applications or just learning how to craft a network packet, or implement the OSI model for tcp/ipv4 seems daunting as there is no easy to follow guides that really explain how or what is going on.
I was wondering if someone might do a Networking guide. The Beej's Networking Guide isn't that great and a little hard to follow. LibCurl does make things a little easier or pcap. Getting in networking applications or just learning how to craft a network packet, or implement the OSI model for tcp/ipv4 seems daunting as there is no easy to follow guides that really explain how or what is going on.
My (limited) experience from school is with handling raw bsd sockets. For something like that, there are so many subtleties and possible configurations that I pretty much needed to use a textbook as a reference whenever doing something. I know textbooks can be pretty heavy, but the one I used was actually really helpful.
It was called TCP/IP Sockets in C: A Practical Guide For Programmers. It's relatively short (~130 pages), has lots of code examples and reference material, and you can find a .pdf of it pretty easily just by googling (or pick it up for $20-$30 on Amazon).
Alternatively, you can find all of the official code samples online for free: http://cs.ecs.baylor.edu/~donahoo/practical/CSockets2/textcode.html
It was called TCP/IP Sockets in C: A Practical Guide For Programmers. It's relatively short (~130 pages), has lots of code examples and reference material, and you can find a .pdf of it pretty easily just by googling (or pick it up for $20-$30 on Amazon).
Alternatively, you can find all of the official code samples online for free: http://cs.ecs.baylor.edu/~donahoo/practical/CSockets2/textcode.html
It depends on what level you want to do networking.
If it's just sending tcp/udp - beej's networking guide is good for that.
If you want higher level interface to do http[s] requests - use libcurl or something platform-specific (like NSURLSession or WinINet).
If it's just sending tcp/udp - beej's networking guide is good for that.
If you want higher level interface to do http[s] requests - use libcurl or something platform-specific (like NSURLSession or WinINet).
IRC might remember this last weekend I went through WinSock2 and had some problems being misguided by my intuitions about how should 'naturally' work. I read Beej and the Window's documentation, which did help. But what helped the most was going through http://gafferongames.com/networking-for-game-programmers/
It's focused on games, but it's a fun place to start. There's a code repository somewhere else online since his links are dead on that site.
It's focused on games, but it's a fun place to start. There's a code repository somewhere else online since his links are dead on that site.
One of the understated skills to acquire when you start with networking is how to do async IO.
Doing that lets you delay having to deal with threads and introduces you to what are essentially event queues.
Doing that lets you delay having to deal with threads and introduces you to what are essentially event queues.
Beej's is probably the way to go for application programming... but before you try to learn from that just fire up Wireshark and just look at some traffic. I think that is a good way to get an idea of what is going on on the "wire", you may also want to try a TCP/IP book for network technicians/administrators, (not for programmers) as they will have a broader focus and might help get a bigger picture of what is going on.
The basic networking code will be pretty much the same on all OSes but, when you go beyond the basics and try to do things like Async IO, or use protocols other than TCP/UDP, you have to look at APIs for each OS, their APIs and best practices can be very different, even between Linux and OS X for example...
If you are determined to get into router and switch development for OSI Layers 2 and 3, then you should probably learn FPGA programming, as a µ-controller alone isn't going to be very efficient at routing between many interfaces at the same time.
For each layer you are planning to implement you need to read the RFCs for the protocols you want to implement, and all their revisions. 1980's era TCP/IP is actually very easy to implement, but modern specifications are much more complicated to get right, and even OS's today don't agree on the best way to implement them (and thats how we have OS Fingerprinting...)
Implementing a networking stack for modern networking is going to be a pretty tough task, there really isn't anyway around it... There are a great many things that an OS does that won't really be covered in any basic networking guide.
The basic networking code will be pretty much the same on all OSes but, when you go beyond the basics and try to do things like Async IO, or use protocols other than TCP/UDP, you have to look at APIs for each OS, their APIs and best practices can be very different, even between Linux and OS X for example...
If you are determined to get into router and switch development for OSI Layers 2 and 3, then you should probably learn FPGA programming, as a µ-controller alone isn't going to be very efficient at routing between many interfaces at the same time.
For each layer you are planning to implement you need to read the RFCs for the protocols you want to implement, and all their revisions. 1980's era TCP/IP is actually very easy to implement, but modern specifications are much more complicated to get right, and even OS's today don't agree on the best way to implement them (and thats how we have OS Fingerprinting...)
Implementing a networking stack for modern networking is going to be a pretty tough task, there really isn't anyway around it... There are a great many things that an OS does that won't really be covered in any basic networking guide.
Edited by David Butler
on
One think I would like to do is use communicate over the network using the MAC address of the 'server' since I don't know what the IP will be. I don't want to make it fixed since I want the embedded device to just be able to connect on any network and find the 'server' based on the MAC address. So far I haven't much on how to do networking on just the ethernet layer.
You cannot use MAC addresses for that.
MAC addresses are not visible outside of "local" network (after router).
Instead you use DNS for that. DNS allows you to put string in your embedded device and then you can change IP address to what it resolves as often as you wish.
But if you want this to work on local network typically it is implemented with broadcasting. Your device can broadcast packet to whole local network saying "hey is there a server here?". And if server sees this message, it can connect to device and say "Yes? I'm here on address X, what do you want?".
MAC addresses are not visible outside of "local" network (after router).
Instead you use DNS for that. DNS allows you to put string in your embedded device and then you can change IP address to what it resolves as often as you wish.
But if you want this to work on local network typically it is implemented with broadcasting. Your device can broadcast packet to whole local network saying "hey is there a server here?". And if server sees this message, it can connect to device and say "Yes? I'm here on address X, what do you want?".
Edited by Mārtiņš Možeiko
on
So I found that the boost::aiso libraries have a C++ style networking API. the only issue so far is that there is no clear guide or example on how to use them. There is just man pages which I guess with some time can be figured out how to piece all of it together. Would there be any advantages of using the C way of sockets vs using boost?
Edited by Taz
on
boost::asio also uses sockets underneath. There is no other way to access network functionality than sockets. The only disadvantage of boost::asio is that it will make your code to depend on huge part of boost library. Which is a lot of code. But its perfectly reasonable to use OS socket API directly to achieve exactly same functionality as boost::asio.
Croepha
Beej's is probably the way to go for application programming... but before you try to learn from that just fire up Wireshark and just look at some traffic. I think that is a good way to get an idea of what is going on on the "wire", you may also want to try a TCP/IP book for network technicians/administrators, (not for programmers) as they will have a broader focus and might help get a bigger picture of what is going on.
...
I'll add this playlist on youtube https://www.youtube.com/playlist?...LowKtXNTBypH19whXTVoG3oKSuOcw_XeW
I've watched few days ago, it's a fun and interesting intro to networking. It doesn't go into any programming, but I really enjoyed it and I think you might too!