Accelerating Debug Runs, Part 1: _NO_DEBUG_HEAP

(A more appropriate but even-less-catchy title might have been ‘accelerating runs from the debugger‘. As elaborated below, these two are not strictly equal).

A common notion is that debug builds can and should carry as much debugging overhead as one can possibly scram in – after all, the point in debug builds is exactly this, debug, and you should never care about their performance. After too many cases of slow-to-the-extent-of-utterly-unworkable builds, I respectfully disagree. In this and the next post, a few techniques to make debug builds run faster are laid out.

Introducing the Windows Debug Heap

As many, many, have already discovered – the WDH is a big deal as far as performance goes, and yet MSDN is unusually terse about it. The HeapSetInformation page says:

When a process is run under any debugger, certain heap debug options are automatically enabled for all heaps in the process. These heap debug options prevent the use of the LFH. To enable the low-fragmentation heap when running under a debugger, set the _NO_DEBUG_HEAP environment variable to 1.

And in some arcane corner of the WinDBG documentation:

Processes that the debugger creates (also known as spawned processes) behave slightly differently than processes that the debugger does not create.

Instead of using the standard heap API, processes that the debugger creates use a special debug heap. You can force a spawned process to use the standard heap instead of the debug heap by using the _NO_DEBUG_HEAP environment variable or the -hd command-line option.

(While the latter was written for windbg, everything except the –hd switch holds equally for VS).

What are these ‘certain heap debug options’? What is the price in performance? Can the WDH be avoided altogether? Stay tuned.

Creating and Avoiding the WDH

The debugger itself calls IDebugClient5::CreateProcess2 which creates a debuggee process with WDH by default, and in general – I suspect – has more power over the debuggee than a normal CreateProcess would. The WDH creation can be bypassed by specifying DEBUG_CREATE_PROCESS_NO_DEBUG_HEAP in the options argument, and the MS debuggers do exactly that when the aforementioned environment variable _NO_DEBUG_HEAP exists and is set to 1.

(Note: somehow I suspect CreateProcess with the DEBUG_PROCESS flag is not enough for the created debuggee to run under the WDH, but I didn’t check).

You can set this environment variable either globally for the machine (as I do) or in a specific debug session via the project properties:

What the WDH Does

  1. The only documented effect is disabling the LFH – which makes sense, as these are different heap layouts and thus are mutually exclusive. You do lose some speedups by dropping the LFH but by and large this is a negligible factor compared to the others.
  2. On every allocation the memory manager initializes every allocated DWORD to 0xbaadfood, and on every deallocation sets the memory to 0xfeeefeee – in addition to some bookkeeping just after the allocated chunk. Here’s the normal view:

And here’s the view with _NO_DEBUG_HEAP=1:

These magic numbers can help in some debugging scenarios – use of uninitialized heap memory, and usage after free – but truth be told, they rarely do. Here are some more details. Most of the extra time, however, is not spent there.

  1. On every memory operation, the WDH walks the heap and checks for integrity! To observe, add some corruption:

And run:

Now run again with _NO_DEBUG_HEAP set to 1 – and watch the assertion vanish.

Err, this stuff actually sounds useful. Sure I should I disable it?

For regular C++ applications – beyond a doubt, yes.

the CRT delivers identical functionality, on top of the windows debug heap, with different magic numbers: 0xcdcdcdcd for fresh allocations and 0xdddddddd for freed memory. If you leave the WDH on you’re initializing memory chunks – and worse, checking heap integrity – twice for each allocation. In regular development scenarios WDH is just empty, very expensive overhead.

By ‘regular’ C++ programs I mean those that don’t do anything fancy with the heap and just stick to the built in CRT heap. You can overload new/delete, as long as your overloads eventually call the shipped new/debug/malloc/free, or some dbg/aligned siblings.

One potential argument in favour of leaving the WDH on is that unlike the CRT debug heap the WDH is operational in release builds also, but (1) it is disabled for any launch outside a debugger anyway, (2) in the extremely unlikely case that you’d require memory integrity checks but don’t want to run a debug build, I would suggest just editing your debug configurations to include optimizations. (add /O2).

Oh, and in our applications setting _NO_DEBUG_HEAP=1 accelerated some runs by a factor of 10. Nough said.

Debugging Memory Corruption II

Some years ago I shared a trick that let’s you call _CrtCheckMemory from the debugger anywhere, without re-compilation.   The updated (as of VS2013) string to type at a watch window is:

{,,msvcr120d.dll}_CrtCheckMemory()

Let’s expand on that today, in two steps.

Checking memory on every allocation

The CRT heap accepts a neat little flag, called: _CRTDBG_CHECK_ALWAYS_DF.  Here’s how it used:

int main()
{
// Get current flag
int tmpFlag = _CrtSetDbgFlag(_CRTDBG_REPORT_FLAG);

// Turn on corruption-checking bit
tmpFlag |= _CRTDBG_CHECK_ALWAYS_DF;

// Set flag to the new value
_CrtSetDbgFlag(tmpFlag);

int* p = new int[100]; // allocate,
p[101] = 1;   // corrupt,    and…

int* q = new int[100];  // BOOM! alarm fires here

}

Testing for corruption on every allocation can tangibly slow down your program, which is why the CRT allows testing only every N allocations, N being 16, 128 or 1024.  Usage adds half a line of code – pasted from MSDN:

// Get the current bits
tmp = _CrtSetDbgFlag(_CRTDBG_REPORT_FLAG);

// Clear the upper 16 bits and OR in the desired frequency
tmp = (tmp & 0x0000FFFF) | _CRTDBG_CHECK_EVERY_16_DF;

// Set the new bits
_CrtSetDbgFlag(tmp);
}

Note that testing for corruption on every memory allocation is nothing like testing on every memory write – the alarm would not fire at the exact time of the felony, but since your software allocates memory (even indirectly) very often – this will hopefully help narrow down the crime scene quickly.

Checking memory on every allocation – from the debugger

You might reasonably want to enable/disable these lavish tests at runtime.

The debug flags are stored in {,,msvcr120d}_crtDbgFlag, and the numeric value of _CRTDBG_CHECK_ALWAYS_DF is 4, so one might hope that these lines would enable and disable these intensive memory tests:

image

Alas, this doesn’t work – _CrtSetDbgFlag contains further logic that routes the input flags further to internal variables. The easiest solution is to just call it:

image

First two lines enable, last two lines disable.  If you’re running with non default flags, the actual values you’d see might be different.

Debugging Handle Leaks

This is all well documented stuff and I won’t go into details – it’s here mostly for self reference (3rd time I had to chase this down in google).

Steps are:

(1) Install WDK to integrate the WinDbg engine with VS (not strictly necessary, but very convenient).

(2) Attach to the debugee via ‘User Mode’ transport:

image

(3) Continue execution, and break at the spot where the handle count is at ‘reference’ value.

(4) At the ‘Debugger Immediate Window’ type ‘!htrace –enable’

(5) Continue execution and break at a point where the handle count is supposed to be at reference value but isn’t.

(6) At the ‘Debugger Immediate Window’ type ‘!htrace –diff’.

 

The offending stack[s] should be visible at the debugger immediate window.  If you get garbage, there’s a good chance you’re debugging a 32bit process on a 64bit machine.

Setting a Watch on Wide Registers in VS

General-purpose registers can be watched from the watch window pretty much as regular variables:

image

(the ‘$’ prefix is optional, but is recommended by MS – probably as means to minimize clashes with code variables.)

It is less known that you can set similar watches on SSE registers.  The direct approach doesn’t work:

image

- probably since the expression evaluator doesn’t have built in 128 bit types.  You can, however, set watches for specific portions of a wide register. First set a watch on a single float, with ‘xmm00’-like syntax (2nd number indicates the 32-bit slot to watch):

image

And next, you can watch 64-bit portions as doubles, with ‘xmm0dh’-like syntax:  ‘d’ stands for double, and l/h specifies high/low halves to watch.

image

This syntax went non-official after VS2003 (!). Up until VS2012 you could also watch 32-bit fractions of SSE registers as ints (some instructions use that) with ‘xmm0il’ syntax. This was mentioned in a Connect answer from 2009– but broke in VS2013.  From the VS2003 link it seems sometime around 2003 you could set similar watches to MMX registers, with ‘mm00’ like syntax.  Never seen it work on any VS version I used.  Maybe there’s similar syntax that enables AVX registers watch, hiding somewhere?   I don’t even have an AVX-enabled computer to guess on.


Update: the int watch (‘xmm0il’) syntax seems to be broken only for x64 builds.

 

VS2012 Migration #3: autoexp and NoStepInto Replacements

In the past I blogged quite a few times about two immensely useful albeit mostly-unofficial debugger features: watch modification via autoexp.dat, and step-into modification via NoStepInto registry key. A long while ago I raised two suggestions at MS UserVoice, to invest in making these two semi-hacks into documented, supported features. The first suggestion got some traction, and is officially implemented in VS2012. The 2nd suggestion went mostly ignored – but nevertheless, there’s a new and better – though still undocumented – way to skip functions while stepping.

NatVis files

The Natvis (native-visualizers) file format is the shiny new replacement for autoexp.dat. It is well documented, and although still quite rough around the edges – bugs are accepted and treated, which means that for the first time it is actually supported. The new apparatus comes with several design advantages:

  1. It seems to be better isolated and not to crash the IDE so much,
  2. New visualizer debugging facilities are built in,
  3. Separate customized visualizers can be kept in separate files, allowing easier sharing (e.g., library writers can now share distribute .natvis files with their libraries).
  4. Natvis files can be placed at per-user locations.

It isn’t that much fun rehashing the syntax – being official and all – but I will include here a custom mfc-containers natvis, similar to the autoexp section I shared a while back

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <!--from afxwin.h -->
  <Type Name="CArray&lt;*,*&gt;">
    <AlternativeType Name="CObArray"></AlternativeType>
    <AlternativeType Name="CByteArray"></AlternativeType>
    <AlternativeType Name="CDWordArray"></AlternativeType>
    <AlternativeType Name="CPtrArray"></AlternativeType>
    <AlternativeType Name="CStringArray"></AlternativeType>
    <AlternativeType Name="CWordArray"></AlternativeType>
    <AlternativeType Name="CUIntArray"></AlternativeType>
    <AlternativeType Name="CTypedPtrArray&lt;*,*&gt;"></AlternativeType>
    <DisplayString>{{size = {m_nSize}}}</DisplayString>
    <Expand>
      <Item Name="[size]">m_nSize</Item>
      <Item Name="[capacity]">m_nMaxSize</Item>
      <ArrayItems>
        <Size>m_nSize</Size>
        <ValuePointer>m_pData</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>

  <Type Name="CList&lt;*,*&gt;">
    <AlternativeType Name="CObList"></AlternativeType>
    <AlternativeType Name="CPtrList"></AlternativeType>
    <AlternativeType Name="CStringList"></AlternativeType>
    <AlternativeType Name="CTypedPtrList&lt;*,*&gt;"></AlternativeType>
    <DisplayString>{{Count = {m_nCount}}}</DisplayString>
    <Expand>
      <Item Name="Count">m_nCount</Item>
      <LinkedListItems>
        <Size>m_nCount</Size>
        <HeadPointer>m_pNodeHead</HeadPointer>
        <NextPointer>pNext</NextPointer>
        <ValueNode>data</ValueNode>
      </LinkedListItems>
    </Expand>
  </Type>
  
  <Type Name="CMap&lt;*,*,*,*&gt;::CAssoc">
    <AlternativeType Name="CMapPtrToWord::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapPtrToPtr::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapStringToOb::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapStringToPtr::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapStringToString::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapWordToOb::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapWordToPtr::CAssoc"></AlternativeType>
    <AlternativeType Name="CTypedPtrMap&lt;*,*,*&gt;::CAssoc"></AlternativeType>
    <DisplayString>{{key={key}, value={value}}}</DisplayString>
  </Type>

  <Type Name="CMap&lt;*,*,*,*&gt;">
    <AlternativeType Name="CMapPtrToWord"></AlternativeType>
    <AlternativeType Name="CMapPtrToPtr"></AlternativeType>
    <AlternativeType Name="CMapStringToOb"></AlternativeType>
    <AlternativeType Name="CMapStringToPtr"></AlternativeType>
    <AlternativeType Name="CMapStringToString"></AlternativeType>
    <AlternativeType Name="CMapWordToOb"></AlternativeType>
    <AlternativeType Name="CMapWordToPtr"></AlternativeType>
    <AlternativeType Name="CTypedPtrMap&lt;*,*,*&gt;"></AlternativeType>
    <DisplayString Condition="(m_nHashTableSize &gt;= 0 &amp;&amp; m_nHashTableSize &lt;= 65535">{{size={m_nHashTableSize}}}</DisplayString>
    <Expand>
      <Item Name="num bins">m_nHashTableSize</Item>
      <ArrayItems>
        <Size>m_nHashTableSize</Size>
        <ValuePointer>m_pHashTable</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>

  <Type Name="CMap&lt;*,*,*,*&gt;">
    <AlternativeType Name="CMapPtrToWord"></AlternativeType>
    <AlternativeType Name="CMapPtrToPtr"></AlternativeType>
    <AlternativeType Name="CMapStringToOb"></AlternativeType>
    <AlternativeType Name="CMapStringToPtr"></AlternativeType>
    <AlternativeType Name="CMapStringToString"></AlternativeType>
    <AlternativeType Name="CMapWordToOb"></AlternativeType>
    <AlternativeType Name="CMapWordToPtr"></AlternativeType>
    <AlternativeType Name="CTypedPtrMap&lt;*,*,*&gt;"></AlternativeType>
    <DisplayString>{Hash table too large!}</DisplayString>
  </Type>
  

  <Type Name="ATL::CAtlMap&lt;*,*,*,*&gt;">
    <AlternativeType Name="ATL::CMapToInterface&lt;*,*,*&gt;"/>
    <AlternativeType Name="ATL::CMapToAutoPtr&lt;*,*,*&gt;"/>
    <DisplayString>{{Count = {m_nElements}}}</DisplayString>
    <Expand>
      <Item Name="Count">m_nElements</Item>
      <ArrayItems>
        <Size>m_nBins</Size>
        <ValuePointer>m_ppBins</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>
  <Type Name="ATL::CAtlMap&lt;*,*,*,*&gt;::CNode">
    <DisplayString Condition="this==0">Empty bucket</DisplayString>
    <DisplayString Condition="this!=0">Hash table bucket</DisplayString>
  </Type>
</AutoVisualizer>

Visualizing Map is a bit tricky, and I didn’t take the time yet to look deep into it – but the file is hopefully useful as it is. To use, just save the text as, say, MfcContainers.natvis, either under %VSINSTALLDIR%\Common7\Packages\Debugger\Visualizers (requires admin access), or under %USERPROFILE%\My Documents\Visual Studio 2012\Visualizers\ .

NatStepFilter Files

- are the new and improved substitute for the NoStepInto registry key. While there are some online hints and traces, the natstepfilter spec is yet to be introduced into MSDN – or even the VC++ team blog. For now you can watch the format specification, along with some good comments, at the %VSINSTALLDIR%\Xml\Schemas\natstepfilter.xsd near you, or even better – inspect a small sample at %VSINSTALLDIR%\Common7\Packages\Debugger\Visualizers\default.natstepfilter.

The default.natstepfilter is implemented by Stephen T. Lavavej, and is very far from complete – both because of regex limitations and because of decisions not to set non-overridable limitations on users:

“Adding something to the default natstepfilter is a very aggressive move, because I don’t believe there’s an easy way for users to undo it (hacking the file requires admin access), and it may be surprising when the debugger just decides to skip stuff.”

I can think of several ways for users to override .natstepfilter directives (never mind stepping-into via assembly, how about setting a plain breakpoint it the function you wish to step into?) – and so I don’t agree with that decision. Still I hope the default rules would improve alongside the documentation. We mostly avoid STL, so I had no need to customize .natstepfilter’s yet – I’ll be sure to share such customizations if I do go there.

Caveat

Both improvements, natvis and natstepfilter files, do not work for debugging native/managed mixed code, which sadly renders them unusable for most of our code. While this behavior is documented – I would hardly say it is ‘by design’. It does seem to irritate many others, so there is hope – as Brad Sullivan writes that MS are-

“… working on making everything just work in a future release of Visual Studio.”

Entry Point Not Found, and other DLL Loading Problems

Occasionally I come across DLL load problems:

The verbosity of the error messages varies greatly. In their raw form these include at least the DLL name, but as various frameworks come into play (for the error message above, it’s .net) – native exceptions are caught and re-thrown, and more often than not helpful information is lost on the way.

Turns out there’s a built in way to get verbose windows-loader output: the Show Loader Snaps flag. The easiest way to mark it is with the gflags utility, bundled with debugging tools for windows:

Under the hood, it merely adds a FLG_SHOW_LDR_SNAPS flag (0x00000002), to a DWORD value in the relevant IFEO registry key. This in turn causes Windows Loader to set the _ShowSnaps variable in the ntdll copy specific to the named process.

And now, behold the new and shiny loader trace (dumped to the debugger output window):

…    

2724:245c @ 11813487 – LdrpFindOrMapDll – RETURN: Status: 0x00000000

2724:245c @ 11813487 – LdrpLoadImportModule – RETURN: Status: 0x00000000

2724:245c @ 11813487 – LdrpLoadImportModule – RETURN: Status: 0x00000000

2724:245c @ 11813487 – LdrpLoadImportModule – RETURN: Status: 0x00000000

2724:245c @ 11813487 – LdrpSnapThunk – WARNING: Hint index 0x70a for procedure “?Revert@CStreamMemory@@UAGJXZ” in DLL “YaddaYadda.dll” is invalid

2724:245c @ 11813487 – LdrpSnapThunk – ERROR: Procedure “?Revert@CStreamMemory@@UAGJXZ” could not be located in DLL “YaddaYadda.dll”

First-chance exception at 0x77321d32 (ntdll.dll) in Strategist.exe: 0xC0000139: Entry Point Not Found.

Bam! There’s the offending DLL and the offending imported function, right there in the debugger.

Like many other useful features – it is documented, but very low on discoverability. Which is a fancy way of saying you can find it only if you already know exactly what you are looking for. I personally got around to it after digging around in ntdll assembly (just like Matt Pietrek, 14 years ago), trying to get to a string containing the name of an offending DLL.

The windows-copycat-opensource ReactOS source gives a nice view of the internal usage of this flag – called ShowSnaps in their source. The ‘snapping’ verb in this context refers to one of the actions performed by the loader: after rebasing the loaded DLL in the loading process memory space, the DLL’s exported function addresses are updated and must be copied to the importing process (or other dll) Import Address Table. This – in this context – is called snapping, and that’s where the extra tracing is hooked.

_DllMain@12 already defined

We recently faced this linkage error:

error LNK2005: _DllMain@12 already defined in MSVCRT.lib(dllmain.obj)

Searching gives ~36K results as of July 2012, many of which seem high quality (StackOverflow, MS, CodeProject etc.), and I was certain it would be a simple matter of finding a fix online and blindly applying it. However it seems the root cause in our particular case wasn’t covered yet (AFAIK), and it seems worthwhile to document.

The MS KB article teaches that this is a linkage order problem – MFC libs must be linked before the CRT ones – but none of the fixes the article proposes worked. We did have one build configuration which was successful and one which failed with the above LNK2005 (Release – but it really doesn’t matter) so I dumped two /VERBOSE linker outputs for the two configurations and diffed them. After some admittedly tedious inspection, an interesting difference came up – these lines were dumped only in the successful build:

Found __afxForceUSRDLL

Referenced in Stdafx.obj
Loaded mfcs100d.lib(dllmodul.obj)

The symbol name implies that it is intended to force some linkage, and including it seems to have the beneficial effect of loading the mfc lib mfcs100d.lib.  Indeed, searching reveals the following lines in dllmodul.cpp:

#ifdef _X86_
extern "C" { int _afxForceUSRDLL; }
#else
extern "C" { int __afxForceUSRDLL; }
#endif

and the following in afx.h:

// force inclusion of DLLMODUL.OBJ for _USRDLL
#ifdef _USRDLL
#pragma comment(linker, "/include:__afxForceUSRDLL")
#endif

So it turns out there’s a single condition that governs the linkage to the MFC library mfcs100/d (the one containing DllModul.obj, which exports _afxForceUSRDLL), and that condition is – _USRDLL being defined.   Our linking project was indeed a dll and somehow the default _USRDLL preprocessor macro was missing from it – restoring the definition fixed the linkage.

So bottom line, if you get a ‘DllMain@12 already defined’ linkage error for a dll, here’s another thing to try: make sure _USRDLL is defined in your project C++ property sheets.