Just finished watching Martin's Boston lecture about errors and compilers. Very useful lecture, but I have a couple of questions:

1. On MVSC, when you enable /analyze for _Printf_format_string_, the compilation time gets extremely long. Not only that, but suddenly a bunch of dump warnings pop up (I'm already using /W4). Is the expected default behavior? I can manually disable all those warnings but is there any way to stop the compilation time from getting long? I could be wrong but it seems like /analyze also doesn't play nicely with #pragma warning(push, 0). It keeps complaining about something in "Windows.h" so I tried disabling it just for external code like in the video but nothing seems to change.
2. Address Sanitizer is a great tool but it doesn't seem to work well with debugger. I'm currently using raddbg, so maybe Visual Studio would have better support. Starting the same exe in the debugger somehow is way slower than just starting from the shell. It complains about some missing sanitizer DLLs so I must start raddbg with PATH including the build tool. When ASAN catches something, I'm always at some deep call stack and need to navigate back up. Am I doing something wrong? Also, Is there any way to visualize ASAN in the debugger?

/analyse turns the static code analyzer On. The static code analyzer will check your code for errors at compile time so it's normal that it's slowing down compile time a lot. It's checking for different types of error than the regular compiler does. I don't think it's meant to be On all the time, I personally use it from time to time to check my code.

I don't know if you can disable warnings from the static code analysis but I wouldn't do it. A lot of the time you will get warnings that probably don't illustrate an error in some case but do in other. So you need to look at the warning and see if its valid or not in that particular case. It's sometimes painful to have those warnings but I don't think there is a way to avoid it. For code that you didn't write (system headers / libs) don't worry about it, just skip them (unless you have a way to fix them and get the fix to be incorporated).

I don't know much about the address sanitizer as I've only used it for a few days and didn't got any error so far. It's also slower for me, but it's expected as using the sanitizer will add instrumentation to your code and that takes time. The requirement to run vcvarsall.bat also got me at first, and is annoying as I didn't used to have my debugger run with it (or RenderDoc) and I have to remember to disable it when I want to have somebody test a build. I'm assuming that if you use VisualStudio you'll get better visualization of the address sanitizer, as it's often the case with debug stuff from msvc.

One of the issue I also have with the address sanitizer is that when I try to step into some functions, I end up stepping into the address sanitizer code and getting back out to the correct location is annoying. But I'm also assuming that Visual Studio probably handles that better.

Yeah, it's pretty normal for it to take more time - as it is doing more things. Usually you don't do that part of your normal build, but "static checks" build. How often - that's up to you and your project specifics to decide.

windows.h should not produce any warnings on /W4 and /analyze. See here: https://godbolt.org/z/MEzesr1jq

Are you maybe using some old MSVC version? Otherwise maybe you're doing strange build?

To disable analyzer warnings in your code, you do them with same #pragma warning, just put them under ifdef that is active when analyzer is enabled:

#if defined(_PREFAST_)
#pragma warning ( disable : 6297 )
#endif

long long f(int x)
{
    return x << 31; // this will produce C6297
}

As for address sanitizer - it depends on dll's that are "development" only. They are not redistributed or installed globally in system. So you need to run it under vcvars environment setup. No need to manually change path, just run it under same env as your build. That happens automatically if you run under VS debugger. As for remedy - make sure you're on latest versions, it had a ton of fixes to support asan. Asan support in debugger is not automatic, debugger must know about it and support it properly. If you're on latest version and have an problem you can reproduce, open an issue in remedybg, I'm sure that will be fixed. I use VS debugger, that works fine with asan because it is updated every time something changes in asan.

Not sure what "visualize ASAN" in debugger means, but when problem happens you either look at call stack, or look at asan output - there it dumps where problem happened and reasons for it (where memory was allocated, is it overread/underread, etc...)

windows.h should not produce any warnings on /W4 and /analyze

Yeah, it's not "Windows.h", it's winnt.h(3454) : warning C28301: No annotations for first declaration of '_mm_clflush'. So maybe this file gets included by "Windows.h" or some other window files, I don't care.

To disable analyzer warnings in your code, you do them with same #pragma warning, just put them under ifdef that is active when analyzer is enabled:

1. Is _PREFAST_ defined when /analyze is on?
2. I tried #pragma warning (push, 0) and it didn't work. Presumably, because /analyze is different from your normal warning levels. Rather than manually disable each warning, aren't there different levels for /analyze? I literally only care about _Printf_format_string_.

As for remedy - make sure you're on latest versions, it had a ton of fixes to support asan

I said I'm using raddbg, not remedybg.

Not sure what "visualize ASAN" in debugger means, but when problem happens you either look at call stack, or look at asan output - there it dumps where problem happened and reasons for it (where memory was allocated, is it overread/underread, etc...)

I meant all this information is present in the terminal and not the debugger. I want to view the shadow bytes region in the memory view, highlight a memory region with different colors depending on whether it's poisoned or not, etc. I'm not familiar with raddbg and currently don't use VS so I don't know if either of those currently have better support for this.

One of the issue I also have with the address sanitizer is that when I try to step into some functions, I end up stepping into the address sanitizer code and getting back out to the correct location is annoying. But I'm also assuming that Visual Studio probably handles that better

Yeah, this and the wrong call stack when it crashed are making me crazy. Is there any way to fix it?

It's also slower for me, but it's expected as using the sanitizer will add instrumentation to your code and that takes time

But starting the same executable (with ASAN on) in the debugger is noticeably slower to me than in the terminal, so I don't know whether or not ASAN detects if it's currently being debugged and does extra things there.

Oh, sorry I misread raddbg. In raddbg support for asan probably is in worse state. I think some things were fixed there, but I'm pretty sure many more are still missing. Just carefully verify what it is actually showing you. Asan relies on structured exceptions a lot to catch bad accesses and uses them during the initialization, so it matters how debugger implements them.

So if you're using VS or Remedy, can you fix any of these things:

1. When ASAN is triggered, does the call stack in your function or ASAN function?

2. Does F11 keep stepping into some ASAN's jump table?

3. Do any of them have a special graphical window/view specifically for ASAN?

4. Asan relies on structured exceptions a lot to catch bad accesses and uses them during the initialization, so it matters how debugger implements them

   Can you go a little deeper on structured exceptions? What are those? I thought ASAN was implemented directly into the compiler, which means all the checking and reports are embedded directly in your code. How does a debugger fit into all of this? How bad is raddbg compared to VS/Remedy? Has raddbg just not supported ASAN yet, or just the basic (performance implication)? Are there some github branches I can switch to?

Regarding _Printf_format_string_ and /analyze:

1. Because on MSVC you must put the annotation before the argument, while on GCC it must be placed before the function, how can you abstract this to a cross-platform macro?

2. Does analyze have different levels/groups (like warnings)?

3. Seeing that the only thing that I care about analyze is _Printf_format_string_ (some other warnings are useful but I can occasionally check them like you suggested, while printf checking is something I always want to have), what do you think about this approach:

   void my_printf_(char* fmt, ...);
   #define my_printf(fmt, ...) ((1 || printf(fmt, __VA_ARGS__)), my_printf_(fmt, __VA_ARGS__))
   

   This is cross-platform (you don't have to put different annotations in different places) and the warning message will be the same (because it's just printf). The downside is the printf could (maybe?, on MSVC -O2 still produces a warning) get eliminated, depending on the optimization level, and you get no warning.

When you run in VS, you get this:

Source code stops on line that triggers asan, but you can see call stack into ASAN internals.

F11 will step into ASAN internals, but you can disable that with /JMC compiler argument.

There is no special view for ASAN. ASAN just dumps the output in console output and debugger output (pretty much the same thing). You can redirect it to file output with asan options env variable.

Structured exceptions are part of Win64 ABI. ASAN does the checking in source code, but for many things it cannot do that efficiently. So it relies on setting up special memory pages that are inaccessible and relies on CPU to fault on them, thus giving control to exception that debugger can catch.

I have no idea how well remedy or raddbg works for asan, I don't really use them.

For printf - you make two macros. One for MSVC other for GCC, and just make the other one defined to nothing on other compiler.

Afaik analyze does not have levels, its just one setting.

Typical way to reuse printf for format checking is to put it in sizeof - as that guarantees it won't be compiled to anything at runtime, even debug builds, as sizeof evaluates to constant. See here: https://developercommunity.visualstudio.com/t/support-format-specifier-checking-on-user-defined/843652#T-N10148740

F11 will step into ASAN internals, but you can disable that with /JMC compiler argument.

It's become much more usable, but now the debugger steps into other weird functions (or sometimes __CheckForDebuggerJustMyCode )

Structured exceptions are part of Win64 ABI. ASAN does the checking in source code, but for many things it cannot do that efficiently. So it relies on setting up special memory pages that are inaccessible and relies on CPU to fault on them, thus giving control to exception that debugger can catch.

So if I'm not using any debugger then what happens? When you said special memory pages, does it mean ASAN checking for basic things like whether a page is committed/reserved or not? If a debugger doesn't handle these exceptions well could it have performance problems? Can you give me some resources about what kind of exceptions ASAN throws (or just on structured exceptions in general)?

I have no idea how well remedy or raddbg works for asan, I don't really use them.

I thought because you worked at RAD Game Tools (Epic) you would know how raddbg works better than most.

For printf - you make two macros. One for MSVC other for GCC, and just make the other one defined to nothing on other compiler.

Like this

#if MSVC
#define CHECK_PRINTF_FUNC(...)
#define CHECK_PRINTF_ARG _Printf_format_string_
#else
#define CHECK_PRINTF_FUNC(pos) __attribute__((format(printf, pos, pos + 1)))
#define CHECK_PRINTF_ARG
#endif

void my_printf(CHECK_PRINTF_ARG const char* fmt, ...) CHECK_PRINTF_FUNC;

Can the __attribute__((format(printf, 1, 2))) be put before the argument? If it can then I could collapse down to a single macro.

I pretty much use only VS debugger because other debuggers are lacking too many features I use.

If you run program without debugger with ASAN enabled, then it will just crash. Because nobody will handle the exception. It's not much different what will happen when you dereference pointer to NULL or invalid memory.

SEH exceptions from user point of view are explained here: https://learn.microsoft.com/en-us/cpp/cpp/structured-exception-handling-c-cpp?view=msvc-170 From runtime point of view here: https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64?view=msvc-170

Enabling ASAN will give reduced performance regardless of how exceptions work. ASAN features are not free. Officially they say it is ~2x slowdown: https://clang.llvm.org/docs/AddressSanitizer.html#introduction But it very much depends on how much exactly program accesses memory. It can be more, it can be less.

__attribute__((format...)) is function attribute, not argument attribute. You can put it only on functions.

I pretty much use only VS debugger because other debuggers are lacking too many features I use.

Can you share some of the features that VS has but RedemyBG doesn't (except NatVis)?

Enabling ASAN will give reduced performance regardless of how exceptions work. ASAN features are not free. Officially they say it is ~2x slowdown: https://clang.llvm.org/docs/AddressSanitizer.html#introduction But it very much depends on how much exactly program accesses memory. It can be more, it can be less.

Yes, I know that turning on ASAN will be slower. The problem is if the debugger (raddbg in this case) is present, it will be substantially slower than just running it normally (both cases have ASAN turned on). So I wondered if this is the expected behavior or if it's just that raddbg is still very early access. Do you have the same problem on VS?

Another thing I forgot to ask is when accessing poison memory that was allocated by VirtualAlloc, I get this error AddressSanitizer: nested bug in the same thread, aborting. If you use malloc instead or hit other bugs like stack-after-use, it'll print out the report and the shadow bytes just fine. When I tested this yesterday (before posting the question), it worked fine. But for some reason, this popped up today.

Minidump debugging. Symbol server support. Custom pdb locations (I think remedy added this only very recently). Parallel Stacks window. Natvis is important for me too.

When I debug code with ASAN enabled it usually is debug build anyway (so already slow). I don't really pay attention to how much exactly slower it is.

That kind of error message typically means bug in asan. Normally it should not happen. Make sure you use MSVC asan, as clang's asan on Windows often is quite buggy. They do fix the issues, but often it takes quite time for bug to be fixed.

Before using ASAN, what I usually do to "poison" a region of memory is to have an end-of-page allocation. An empty page will follow the allocation to catch overflow bugs and when you want to free the memory you decommit it instead to catch use-after-free bugs. ASAN is just better at this because the region doesn't need to be aligned to a page boundary (4KB), and you can still poison extra bytes at the start and end to catch both underflow and overflow bugs (unlike a page allocator where you can't only choose one). There's no free meal so where's the catch? Is ASAN just way slower or has a higher memory overhead than a page allocator? Granted, ASAN would probably waste less space for small allocations than the page allocator (because everything must be at least a page).

Yes, enabling ASAN makes your code slower. And requires a lot more memory.
See https://clang.llvm.org/docs/AddressSanitizer.html#limitations

I'm unsure how ASAN costs more memory than a page allocator. Because a shadow byte accounts for 8 normal bytes, so at best it costs you a 1/8th more memory-wise, right (not counting the stack size)?

On 64-bit platforms AddressSanitizer maps (but not reserves) 16+ Terabytes of virtual address space

What do maps-but-not-reserves mean on Windows? I thought with VirtualAlloc you could only use MEM_COMMIT or MEM_RESERVE.

every allocation gets put in its own page with surrounding guard pages to catch buffer overflows

And allocations are not reused to catch use-after-free