← Back to index

Ideas for a Network API

Finalspace
#16556
Post your ideas for a good C-based network api, which covers everything you need to create any kind of network application but with focus of user-control - meaning that the user provides the arrays/buffers and the api wont allocate anything.

So if you have a good idea, please post it here. Thanks.

The best suitable idea will be used for FPL ;)
Edited by Finalspace on
ratchetfreak
#16557
The network events notification must be able to integrate seamlessly into the gui event notification. However it should also be possible to make another thread deal with the result.

Something like continuation queue is a good choice, with each command you also pass the queue that you want notified when the operation is done. With option to make it a GUI event or wait right then and there.
Edited by ratchetfreak on
Mārtiņš Možeiko
#16560
My main suggestion would be to NOT do simple wrapper around BSD sockets. Do a bit more. Create all API nonblocking/async. Most difficult part will be dns resolving. On Win32 it is built in, but on Linux you'll need to do more code (don't use GNU libc specific functions, so it is portable). Allow user to do sync requests by explicit wait function, and/or what ratchetfreak suggests - integrate with gui event loop. Don't do high level functions ("send object position"), allow to send/receive raw bytes.
Edited by Mārtiņš Možeiko on
Finalspace
#16570
mmozeiko
My main suggestion would be to NOT do simple wrapper around BSD sockets. Do a bit more. Create all API nonblocking/async. Most difficult part will be dns resolving. On Win32 it is built in, but on Linux you'll need to do more code (don't use GNU libc specific functions, so it is portable). Allow user to do sync requests by explicit wait function, and/or what ratchetfreak suggests - integrate with gui event loop. Don't do high level functions ("send object position"), allow to send/receive raw bytes.


For *nix my plan is to implement everything in POSIX standard functions without using any extensions, but i expect that there will be differences between win32, linux and unix.

Regarding blocking vs non-blocking. I am not sure what the best way is. Normally i want the caller to decide how to use it.
I think having two apis: Blocking and Non-Blocking would be most suitable. Non-blocking would be handled on separated threads with bare minimum cache structures and blocking will most likely be just a wrapper around network functions.

And there will be no high-level functions such as "send object position". But there will be functions for sending/receiving raw bytes and integral data-types such as int16_t, uint32_t, etc. and maybe two functions for sending/receiving UTF-8 strings + conversion functions from system byte-order to network-byte order and vice versa.

A few examples (Blocking): 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
// Does not do any byte-order conversion
// Data wont be send in multiple packets. When it does not fit it will be cut of.
size_t fplSocketReadRaw(const fplSocket *socket, const size_t size, uint8_t *targetBuffer);
size_t fplSocketWriteRaw(const fplSocket *socket, const size_t size, const uint8_t *sourceBuffer);

// Does not do any byte-order conversion
// Bytes will be send/received in multiple packets if needed (Datagram limitation)
// Appends a header automatically, so you dont have to deal with limitations
size_t fplSocketReadBytes(const fplSocket *socket, const size_t size, uint8_t *targetBuffer);
size_t fplSocketWriteBytes(const fplSocket *socket, const size_t size, const uint8_t *sourceBuffer);

// Does byte-order conversion automatically
int16_t fplSocketReadS16(const fplSocket *socket);
int32_t fplSocketReadS32(const fplSocket *socket);

// Does byte-order conversion automatically
size_t fplSocketWriteS16(fplSocket *socket, const int16_t value);
size_t fplSocketWriteS32(fplSocket *socket, const int16_t value);

// Does byte-order conversion automatically
size_t fplSocketWriteS16(fplSocket *socket, const int16_t value);
size_t fplSocketWriteS32(fplSocket *socket, const int16_t value);

// Does byte-order conversion automatically (Wide <-> UTF-8)
// String will be send/received in multiple packets if needed (Datagram limitation)
size_t fpSocketWriteString(fplSocket *socket, const char *source, const size_t sourceLen);
size_t fplSocketReadString(fplSocket *socket, const char *maxTargetLen, char *target);

// Resolving / Lookup
bool fplResolveIPAddressV4ByHostname(const char *inHostname, fplNetworkIPAddressV4 *outAddress);
bool fplResolveIPAddressV6ByHostname(const char *inHostname, fplNetworkIPAddressV6 *outAddress);
bool fplResolveHostnameByIPAddressV4(const fplNetworkIPAddressV4 *inAddress, const size_t maxOutHostnameLen, char *outHostname);
bool fplResolveHostnameByIPAddressV6(const fplNetworkIPAddressV6 *inAddress, const size_t maxOutHostnameLen, char *outHostname);

// Better resolving / lookup
typedef enum fplNetworkIPAddressType {
  fplNetworkIPAddressType_V4 = 0,
  fplNetworkIPAddressType_V6,
} fplNetworkIPAddressType;

typedef struct fplNetworkIPAddress {
  union {
    char[32] v4;
    char[128] v6;
  };
  fplNetworkIPAddressType type;
} fplNetworkIPAddress;

bool fplParseIPAddress(const char *source, const size_t sourceLen, fplNetworkIPAddress *outAddress);
bool fplResolveIPAddressByHostname(const char *inHostname, fplNetworkIPAddress *outAddress);
bool fplResolveHostnameByIPAddress(const fplNetworkIPAddress *inAddress, const size_t maxOutHostnameLen, char *outHostname);


ratchetfreak
The network events notification must be able to integrate seamlessly into the gui event notification. However it should also be possible to make another thread deal with the result.

Something like continuation queue is a good choice, with each command you also pass the queue that you want notified when the operation is done. With option to make it a GUI event or wait right then and there.


At the moment the window event system in blocking only, so integrating network events would not work for time-critical events such as receiving data, but it could optionally support one-time events such as connecting/disconnecting events. But i dont want to add a full caching queue. The caller should be responsible for that.

The only thing cached should be just the connection state with socket/hostname/ip-address informations.
Edited by Finalspace on
ratchetfreak
#16572


If you support reading individual ints there better be a buffer backing it so you aren't reading byte at a time from the OS.

Finding a way to do endian conversion separate from the socket may be a better solution.
Mārtiņš Možeiko
#16573
I agree, don't bother with endian conversion. Let user deal with it, this is not socket responsibility. Does your file API also deals with endian conversion? No. Then socket API also should not.

Having noblocking sockets does not require separate thread. Its an unnecessary overhead. Normal read/write functions can easily be nonblocking by themselves. Only "difficult" thing, as I said before, is resolving host.

Also what is this - "Appends a header automatically"? Are you basically creating new protocol? That means it won't be possible to use your socket API for example to do HTTP request. Or implement IRC client. Or interoperate with any other non-FPL-based endpoint. I would say that is bad choice.
Randy Gaul
#16589
I heavily agree with the idea to make your API deal with raw buffers the user can place arbitrary data into. There are two different problems here: protocol, and transport. You want to provide a transport layer -- get bytes from computer A to B with a good API. Then leave the protocol itself up to the user, or implement another higher level layer that consumes your transport to implement something specific (like HTTP message passing, or something else).
Finalspace
#17151
Randy Gaul
I heavily agree with the idea to make your API deal with raw buffers the user can place arbitrary data into. There are two different problems here: protocol, and transport. You want to provide a transport layer -- get bytes from computer A to B with a good API. Then leave the protocol itself up to the user, or implement another higher level layer that consumes your transport to implement something specific (like HTTP message passing, or something else).


Sorry for the very very late answer, but this will be exactly what i will do in FPL: A good low-level network API and nothing more.
← Back to index