handmade.network » Forums » Learning DB Performance/Behavior
Kelimion
Jeroen van Rijn
235 posts
3 projects

A big ball of Wibbly-Wobbly, Timey-Wimey _stuff_

#12141 Learning DB Performance/Behavior
3 months, 2 weeks ago

There's another crypto library that may be of interest if libsodium isn't to your taste, libTomCrypt. I don't have any particular experience with this one, but it's in the public domain and well stocked feature-wise.
mmozeiko
Mārtiņš Možeiko
1477 posts
1 project
#12142 Learning DB Performance/Behavior
3 months, 2 weeks ago

tomcrypt is okay-ish. It works, but is a bit ugly. I don't like it much, only maybe as a reference code. And its too low-level - like OpenSSL or mbedtls crypto is. Implementing good crypto protocol on top if it will require decent understanding of crypto. libsodium is better in this way - it hides all the low-level crypto primitives behind nice to use and hard to get wrong API.

For lower level crypto libraries parts of BearSSL is very nice: https://bearssl.org/ It's not yet reached 1.0 version, but author pays a lot of attention to get all the minor details right.
msmshazan
Shazan Shums
162 posts

Some day I will make quality software.
Programming FTW.

#12143 Learning DB Performance/Behavior
3 months, 2 weeks ago

This is not a crypto related issue but when i was compiling mbedTls i got this as compile warnings:

build.bat
main.cpp
mbedTLS.lib(aes.obj) : MSIL .netmodule or module compiled with /GL found; restarting link with /LTCG; add /LTCG to the link command line to improve linker performance
Generating code
Finished generating code
CTIME: 7.074 seconds (encrypt.ctm)


I know i can get rid of it by adding /LTCG to link command line but what does it mean and what is causing this.

And i downloaded libsodium from https://download.libsodium.org/doc/

But it contained 3 versions of the binaries called static,ltcg,dynamic.. Which ones to use?

Just being curious on programming....
Kelimion
Jeroen van Rijn
235 posts
3 projects

A big ball of Wibbly-Wobbly, Timey-Wimey _stuff_

#12144 Learning DB Performance/Behavior
3 months, 2 weeks ago

It means the object file was creating with [url=https://msdn.microsoft.com/en-us/library/0zza0de8.aspx]the /GL flag set[url], using whole program optimisation. You need Link Time Code Generation /LTCG to have the linker put in the pre- and post-amble to the functions. They're not generated when you use /GL so they can be better optimized during link time.

For libsodium the same applies for the one called ltcg. Which one to use is of course entirely up to you. Should it be built in, should it be a dll? If you want to build it in, do you want to use whole program optimisation?
msmshazan
Shazan Shums
162 posts

Some day I will make quality software.
Programming FTW.

#12161 Learning DB Performance/Behavior
3 months, 2 weeks ago

Here 's an issue with using libsodium and the docs don't help.

Is this usage correct?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
#include "main.h"
#include <sodium.h>

int main(int argc,char* argv)
{
    if (sodium_init() == -1) {
        printf("Libsodium init failed.");
    }else{
        printf("Success.\n");
    }
     
    char * in = "HJKSALIWDJDKDSLSL";
    char * key = "pswd";
    char out[1024] = {};
    printf("input:%s\n",in);
    crypto_stream_aes128ctr((unsigned char *)&out[0],1024,(const unsigned char *)in,(const unsigned char *)key);
    printf("output:%s\n",out);
    return 0;
}

Just being curious on programming....
mmozeiko
Mārtiņš Možeiko
1477 posts
1 project
#12162 Learning DB Performance/Behavior
3 months, 2 weeks ago Edited by Mārtiņš Možeiko on June 8, 2017, 5:06 p.m.

Are you sure you are reading correct docs? I think following example is pretty clear how to use library: https://download.libsodium.org/do...phy/authenticated_encryption.html

crypto_secretbox_easy to encrypt and crypto_secretbox_open_easy to decrypt.
crypto_secretbox_easy accepts in arguments ciphertext, plaintext, plaintext size, nonce (IV) and key in that order.
crypto_secretbox_open_easy accepts in arguments plaintext, ciphertext, ciphertext size, nonce and key in that order.

And don't use KEY as PASSWORD. That's a huge security issue. If you have password, you need to derive KEY with some key stretching function. libsodium API for this is here: https://download.libsodium.org/doc/key_derivation/
msmshazan
Shazan Shums
162 posts

Some day I will make quality software.
Programming FTW.

#12163 Learning DB Performance/Behavior
3 months, 2 weeks ago

My bad not reading docs correctly.

But docs use Xsalsa20 but i want to use AES is there any way to do this.

Just being curious on programming....
mmozeiko
Mārtiņš Možeiko
1477 posts
1 project
#12164 Learning DB Performance/Behavior
3 months, 2 weeks ago

Its possible, but libsodium only supports AES on CPU's that have it hardware accelerated: https://download.libsodium.org/do...key_cryptography/aes-256-gcm.html

The thing about libsodium is that it wraps away all those low-level crypto primitives like AES or authentication. And it offers easy to use and hard to get wrong high-level API. If you know what you are doing and want to implement crypto scheme yourself and use directly low-level crypto primitives then libsodium is not really for you.
msmshazan
Shazan Shums
162 posts

Some day I will make quality software.
Programming FTW.

#12165 Learning DB Performance/Behavior
3 months, 2 weeks ago Edited by Shazan Shums on June 8, 2017, 5:52 p.m.

Now my issue is how to do SQL database reads,crypto functions and long time taking functions in an immediate mode based app.
How should I do multi threading ?
Example code would be nice.

Just being curious on programming....
mmozeiko
Mārtiņš Možeiko
1477 posts
1 project
#12166 Learning DB Performance/Behavior
3 months, 2 weeks ago Edited by Mārtiņš Možeiko on June 8, 2017, 7:23 p.m.

Look for episodes where Casey implemented asset loading in separate thread. Once asset is ready, the main thread could use the textures or sounds. In your case put sql execution in thread, and once results are ready, use them in main thread. All the code for this is available if you bought HandmadeHero, or just watch videos - you can see all necessary functionality being implemented in videos.
msmshazan
Shazan Shums
162 posts

Some day I will make quality software.
Programming FTW.

#12179 Learning DB Performance/Behavior
3 months, 2 weeks ago Edited by Shazan Shums on June 10, 2017, 11:26 a.m.

Thanks, for the help.
I tried creating a db file using it and it works.
But how can i prevent multiple task queueing for example when a person double clicks it may do the same task twice.

Just being curious on programming....
msmshazan
Shazan Shums
162 posts

Some day I will make quality software.
Programming FTW.

#12180 Learning DB Performance/Behavior
3 months, 2 weeks ago

Now there is a issue i have made a Callback like this ,

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
internal PLATFORM_WORK_QUEUE_CALLBACK(InitilizeSqlite)
{
    char Buffer[256]; 
    wsprintf(Buffer, "Thread: %u\n", GetCurrentThreadId());
    OutputDebugStringA(Buffer);
    if(IsSqliteThreadSafe()){
       wsprintf(Buffer,"Sqlite API is thread Safe.");
    }else{
       wsprintf(Buffer,"Sqlite API is not thread Safe.");
    }
    OutputDebugStringA(Buffer);
    Data = CreateOrOpenDatabase("Database.db");  
}


I wanted to get the pointer to the database as data but I don't know how to do it.
How can I get the Data pointer back to main thread.

Just being curious on programming....
Kelimion
Jeroen van Rijn
235 posts
3 projects

A big ball of Wibbly-Wobbly, Timey-Wimey _stuff_

#12182 Learning DB Performance/Behavior
3 months, 2 weeks ago Edited by Jeroen van Rijn on June 10, 2017, 4:10 p.m.

Well, I did send you an email a couple of days ago that you're not obligated to use sqlite3_exec and a callback. You can also use sqlite3_prepare and then step through the results with sqlite3_step, and then use sqlite3_finalize when all rows have been returned.

That simplifies things considerably. How you then synchronise between your sqlite worker thread and your other treads is entirely up to you.
msmshazan
Shazan Shums
162 posts

Some day I will make quality software.
Programming FTW.

#12183 Learning DB Performance/Behavior
3 months, 2 weeks ago

Thank's for responding @Jeroen van Rijn,

I'm not using sqlite's exec function and callbacks.

The function shown is the work queue callback. I just want a way to get a pointer or some data from the worker thread to the main thread.

I know you can insert pointer to data to the work entry function so it can do work on the data I want
get back data.

Or is it a problem because the data is stack allocated or something.

Just being curious on programming....
Kelimion
Jeroen van Rijn
235 posts
3 projects

A big ball of Wibbly-Wobbly, Timey-Wimey _stuff_

#12184 Learning DB Performance/Behavior
3 months, 2 weeks ago

Ah, right.

Like Martins suggested, you could take a look at how Handmade Hero has set up its worker threads and parcels out work. OBBG has a similar work queue. Both implementations were streamed at the time and archived on Youtube. You can use HMH's episode guide to search for 'thread' or 'worker thread' or something along those lines to find the relevant bits.

How you return the data is up to you. It's hard to comment without knowing more about how you structured your program. Like what does the code that calls this callback look like. Do you have the caller allocate and own the memory or the worker. Who's in charge of freeing that memory again.

In this case it's probably simple because you have a handle to an sqlite database, so you'd have the thing that calls the callback expect a return value of that type, and then you'd return the handle. Then when that's done it can close the sqlite database using that same handle. (At least from what I can make out just by looking at that snippet. The other end of the equation is missing).

But, if I were you I would watch how Casey and Sean did their worker thread systems, you'd get a feel for some of the possible solutions.