I was experimenting with writing a PDB parser that would use mmaping to read the pages of MSF streams as if they are contiguous.
The thing about memory mapping on windows is that there doesn't seem to be a way to reserve a range of virtual memory pages and then fill that range up with multiple mmaps. Just reserving pages with VirtualAlloc still makes MapViewOfFileEx fail.
The way magic ring buffer does it is it reserves some memory to get a pointer to first free range of pages, then frees that memory and uses that pointer as the base for mmaps. Which is dicey, because if another thread does virtual alloc it might get those pages before they get mmaped. That's why the magic ring buffer does several attempts of this process and if all of them fail it gives up.
I tried using placeholder pages, but it seems like the only way you can use placeholders, the size of mmaped region should exactly match the size of the placeholder, which means I won't be able to map uncontiguous file pages contiguously in memory.
I was wondering whether anyone knows how that can be done and if so, how can it be done
Edited by bumbread
on
I tried using placeholder pages, but it seems like the only way you can use placeholders, the size of mmaped region should exactly match the size of the placeholder
VirtualFree with MEM_PRESERVE_PLACEHOLDER flag can split whole placeholder into separate parts - I'm 100% sure that allows to map individual pages from file in any order over placeholder regions, regardless of page order in file. That's what "magic ring buffer" example code in VirtualAlloc2 is doing.
Edited by Mārtiņš Možeiko
on
Are you familiar with RawPDB (https://github.com/MolecularMatters/raw_pdb) ?
It uses memory mapped files in the examples (https://github.com/MolecularMatters/raw_pdb/blob/main/src/Examples/ExampleMemoryMappedFile.cpp) and the library itself provides low level raw access (hence the name :-)) to the different MSFs streams.