Handmade Network»Forums
Amin Mesbah
13 posts / 1 project
Safely Exiting a Windows Process with Multiple Running Threads
Edited by Amin Mesbah on Reason: allman

The minimal example below demonstrates something I find surprising about Windows process termination when there is a running secondary thread.

When we don't explicitly call ExitProcess() in main(), the assert in second_thread() fails. It seems as though shared_data.value is being uninitialized as part of process termination while second_thread() is still running.

What I would expect is that return 0 will result in a call to ExitProcess(0) in the CRT code. That's what Raymond Chen said in 2010, anyway. However this doesn't appear to be the case in Windows 10.

My questions

  1. Has the behavior of return 0 in a win32 program changed since Raymond Chen's post, or am I misunderstanding something?
  2. Is explicitly calling ExitProcess(0) a reasonably robust way to terminate all secondary threads during program termination. Are there advantages to instead using the technique mentioned in this MSDN article?
  • Create an event object using the CreateEvent function.
  • Create the threads.
  • Each thread monitors the event state by calling the WaitForSingleObject function. Use a wait time-out interval of zero.
  • Each thread terminates its own execution when the event is set to the signaled state (WaitForSingleObject returns WAIT_OBJECT_0).

Example Program

// Compiled with:
//     clang version 18.1.2
//     Target: x86_64-pc-windows-msvc
//     Thread model: posix
//     InstalledDir: C:\Program Files\LLVM\bin
//     clang-cl .\win32_process_exit_thread_error.c -o "win32_process_exit_thread_error.exe" /std:c11 /MT /WX /Zi /link /DEBUG:FULL /SUBSYSTEM:CONSOLE /WX

// Set this to a non-zero value to trigger the assert
#define EXIT_WITHOUT_CALLING_EXIT_PROCESS 1

#include <assert.h>
#include <stdint.h>

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

#define VALUE 9999

struct SharedData
{
    uint64_t value;
};

DWORD WINAPI second_thread(LPVOID lpParam)
{
    struct SharedData *shared_data = (struct SharedData *)lpParam;
    while (1)
    {
        if (shared_data->value != VALUE)
        {
            assert(0);
        }
    }
}

int main(void)
{
    struct SharedData shared_data =
    {
        .value = VALUE,
    };

    CreateThread(NULL, 0, &second_thread, &shared_data, 0, NULL);

    Sleep(160);

#if !EXIT_WITHOUT_CALLING_EXIT_PROCESS
    ExitProcess(0);
#endif

    return 0;
}

Main Thread State During Failed Assert

main_thread_at_assert_fail.png

Mārtiņš Možeiko
2568 posts / 2 projects
Safely Exiting a Windows Process with Multiple Running Threads

Assert like that most likely fails because your shared data is stored on stack of main thread. When main function returns, it goes into CRT shutdown code - but those functions also use stack space, overwriting your shared data value with garbage. Thus your thread will trigger the assert before CRT is able to call ExitProcess. CRT source code comes with VS installation, you can use VS debugger to step outside main() function and see what it does, and how it actually gets to ExitProcess().

Terminating process with explicit ExitProcess call is fine. It will properly shut down all the threads. But you must be sure they are not doing something that is bad to interrupt. Like if some thread is saving file, then sudden ExitProcess may interrupt it and file will be not saved properly.

Usually you want some kind of signal, be it win32 event, or semaphore, or futex, or whatever to be signaled for all threads - many options to choose from. Then threads exit their callback, and main thread waits on all thread handles with WaitForSingle/Multi objects before actually terminating process.

Amin Mesbah
13 posts / 1 project
Safely Exiting a Windows Process with Multiple Running Threads
Edited by Amin Mesbah on Reason: clarify example description
Replying to mmozeiko (#30050)

but those functions also use stack space, overwriting your shared data value with garbage

This is indeed exactly what was happening. Thanks!

Usually you want some kind of signal, be it win32 event, or semaphore, or futex, or whatever to be signaled for all threads - many options to choose from.

I tried out some different ways of implementing this signalling. Here's the minimal set of examples I came up with, which uses an Event, a Condition Variable, and WaitOnAddress() to coordinate termination of three respective groups of sub threads:

// Three different ways of signalling groups of sub threads to self-terminate.

#include <assert.h>
#include <stdint.h>
#include <stdio.h>

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

// For WakeByAddressAll, WaitOnAddress.
#pragma comment(lib, "synchronization")

#define THREAD_WAIT_TIMEOUT_MS 100

static uint32_t should_terminate_initial_value = 0;

struct SharedData
{
    uint32_t should_terminate;
    CONDITION_VARIABLE cv;
    SRWLOCK srw;
    HANDLE event;
};

struct ThreadData
{
    struct SharedData *shared;
    uint32_t thread_id;
};

DWORD WINAPI thread_notified_with_event(LPVOID lpParam)
{
    struct ThreadData *data = (struct ThreadData *)lpParam;
    struct SharedData *shared = data->shared;
    printf("thread[%d]: start\n", data->thread_id);

    while (1)
    {
        if (WaitForSingleObject(shared->event, THREAD_WAIT_TIMEOUT_MS) == WAIT_OBJECT_0)
        {
            break;
        }
    }

    printf("thread[%d]: end\n", data->thread_id);
    return 0;
}

DWORD WINAPI thread_notified_with_condition_variable(LPVOID lpParam)
{
    struct ThreadData *data = (struct ThreadData *)lpParam;
    struct SharedData *shared = data->shared;
    printf("thread[%d]: start\n", data->thread_id);

    while (1)
    {
        AcquireSRWLockShared(&shared->srw);

        if (shared->should_terminate)
        {
            ReleaseSRWLockShared(&shared->srw);
            break;
        }

        if (!SleepConditionVariableSRW(&shared->cv, &shared->srw, THREAD_WAIT_TIMEOUT_MS, CONDITION_VARIABLE_LOCKMODE_SHARED))
        {
            assert(GetLastError() == ERROR_TIMEOUT);
        }

        ReleaseSRWLockShared(&shared->srw);
    }

    printf("thread[%d]: end\n", data->thread_id);
    return 0;
}

DWORD WINAPI thread_notified_with_address(LPVOID lpParam)
{
    struct ThreadData *data = (struct ThreadData *)lpParam;
    struct SharedData *shared = data->shared;
    printf("thread[%d]: start\n", data->thread_id);

    while (1)
    {
        AcquireSRWLockShared(&shared->srw);

        if (shared->should_terminate)
        {
            ReleaseSRWLockShared(&shared->srw);
            break;
        }

        ReleaseSRWLockShared(&shared->srw);

        if (!WaitOnAddress(&shared->should_terminate, &should_terminate_initial_value, sizeof(shared->should_terminate), THREAD_WAIT_TIMEOUT_MS))
        {
            assert(GetLastError() == ERROR_TIMEOUT);
        }
    }

    printf("thread[%d]: end\n", data->thread_id);
    return 0;
}

int main(void)
{
    struct SharedData shared_data =
    {
        .should_terminate = should_terminate_initial_value,
        .event = CreateEvent(NULL, TRUE, FALSE, NULL),
    };

    InitializeSRWLock(&shared_data.srw);
    InitializeConditionVariable(&shared_data.cv);

    HANDLE subthreads[] =
    {
        CreateThread(NULL, 0, &thread_notified_with_event,              &(struct ThreadData){.thread_id = 0, .shared = &shared_data}, 0, NULL),
        CreateThread(NULL, 0, &thread_notified_with_event,              &(struct ThreadData){.thread_id = 1, .shared = &shared_data}, 0, NULL),
        CreateThread(NULL, 0, &thread_notified_with_event,              &(struct ThreadData){.thread_id = 2, .shared = &shared_data}, 0, NULL),
        CreateThread(NULL, 0, &thread_notified_with_condition_variable, &(struct ThreadData){.thread_id = 3, .shared = &shared_data}, 0, NULL),
        CreateThread(NULL, 0, &thread_notified_with_condition_variable, &(struct ThreadData){.thread_id = 4, .shared = &shared_data}, 0, NULL),
        CreateThread(NULL, 0, &thread_notified_with_condition_variable, &(struct ThreadData){.thread_id = 5, .shared = &shared_data}, 0, NULL),
        CreateThread(NULL, 0, &thread_notified_with_address,            &(struct ThreadData){.thread_id = 6, .shared = &shared_data}, 0, NULL),
        CreateThread(NULL, 0, &thread_notified_with_address,            &(struct ThreadData){.thread_id = 7, .shared = &shared_data}, 0, NULL),
        CreateThread(NULL, 0, &thread_notified_with_address,            &(struct ThreadData){.thread_id = 8, .shared = &shared_data}, 0, NULL),
    };

    size_t subthreads_count = sizeof(subthreads) / sizeof(subthreads[0]);

    printf("main: sleep\n");
    Sleep(2000);

    printf("main: signal termination\n");

    AcquireSRWLockExclusive(&shared_data.srw);
    shared_data.should_terminate = 1;
    ReleaseSRWLockExclusive(&shared_data.srw);

    SetEvent(shared_data.event);
    WakeAllConditionVariable(&shared_data.cv);
    WakeByAddressAll(&shared_data.should_terminate);

    WaitForMultipleObjects(subthreads_count, subthreads, TRUE, INFINITE);
    printf("main: subthreads terminated\n");

    for (size_t i = 0; i < subthreads_count; ++i)
    {
        CloseHandle(subthreads[i]);
    }

    return 0;
}