VS2012 Migration #3: autoexp and NoStepInto Replacements

In the past I blogged quite a few times about two immensely useful albeit mostly-unofficial debugger features: watch modification via autoexp.dat, and step-into modification via NoStepInto registry key. A long while ago I raised two suggestions at MS UserVoice, to invest in making these two semi-hacks into documented, supported features. The first suggestion got some traction, and is officially implemented in VS2012. The 2nd suggestion went mostly ignored – but nevertheless, there’s a new and better – though still undocumented – way to skip functions while stepping.

NatVis files

The Natvis (native-visualizers) file format is the shiny new replacement for autoexp.dat. It is well documented, and although still quite rough around the edges – bugs are accepted and treated, which means that for the first time it is actually supported. The new apparatus comes with several design advantages:

  1. It seems to be better isolated and not to crash the IDE so much,
  2. New visualizer debugging facilities are built in,
  3. Separate customized visualizers can be kept in separate files, allowing easier sharing (e.g., library writers can now share distribute .natvis files with their libraries).
  4. Natvis files can be placed at per-user locations.

It isn’t that much fun rehashing the syntax – being official and all – but I will include here a custom mfc-containers natvis, similar to the autoexp section I shared a while back

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <!--from afxwin.h -->
  <Type Name="CArray&lt;*,*&gt;">
    <AlternativeType Name="CObArray"></AlternativeType>
    <AlternativeType Name="CByteArray"></AlternativeType>
    <AlternativeType Name="CDWordArray"></AlternativeType>
    <AlternativeType Name="CPtrArray"></AlternativeType>
    <AlternativeType Name="CStringArray"></AlternativeType>
    <AlternativeType Name="CWordArray"></AlternativeType>
    <AlternativeType Name="CUIntArray"></AlternativeType>
    <AlternativeType Name="CTypedPtrArray&lt;*,*&gt;"></AlternativeType>
    <DisplayString>{{size = {m_nSize}}}</DisplayString>
    <Expand>
      <Item Name="[size]">m_nSize</Item>
      <Item Name="[capacity]">m_nMaxSize</Item>
      <ArrayItems>
        <Size>m_nSize</Size>
        <ValuePointer>m_pData</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>

  <Type Name="CList&lt;*,*&gt;">
    <AlternativeType Name="CObList"></AlternativeType>
    <AlternativeType Name="CPtrList"></AlternativeType>
    <AlternativeType Name="CStringList"></AlternativeType>
    <AlternativeType Name="CTypedPtrList&lt;*,*&gt;"></AlternativeType>
    <DisplayString>{{Count = {m_nCount}}}</DisplayString>
    <Expand>
      <Item Name="Count">m_nCount</Item>
      <LinkedListItems>
        <Size>m_nCount</Size>
        <HeadPointer>m_pNodeHead</HeadPointer>
        <NextPointer>pNext</NextPointer>
        <ValueNode>data</ValueNode>
      </LinkedListItems>
    </Expand>
  </Type>
  
  <Type Name="CMap&lt;*,*,*,*&gt;::CAssoc">
    <AlternativeType Name="CMapPtrToWord::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapPtrToPtr::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapStringToOb::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapStringToPtr::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapStringToString::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapWordToOb::CAssoc"></AlternativeType>
    <AlternativeType Name="CMapWordToPtr::CAssoc"></AlternativeType>
    <AlternativeType Name="CTypedPtrMap&lt;*,*,*&gt;::CAssoc"></AlternativeType>
    <DisplayString>{{key={key}, value={value}}}</DisplayString>
  </Type>

  <Type Name="CMap&lt;*,*,*,*&gt;">
    <AlternativeType Name="CMapPtrToWord"></AlternativeType>
    <AlternativeType Name="CMapPtrToPtr"></AlternativeType>
    <AlternativeType Name="CMapStringToOb"></AlternativeType>
    <AlternativeType Name="CMapStringToPtr"></AlternativeType>
    <AlternativeType Name="CMapStringToString"></AlternativeType>
    <AlternativeType Name="CMapWordToOb"></AlternativeType>
    <AlternativeType Name="CMapWordToPtr"></AlternativeType>
    <AlternativeType Name="CTypedPtrMap&lt;*,*,*&gt;"></AlternativeType>
    <DisplayString Condition="(m_nHashTableSize &gt;= 0 &amp;&amp; m_nHashTableSize &lt;= 65535">{{size={m_nHashTableSize}}}</DisplayString>
    <Expand>
      <Item Name="num bins">m_nHashTableSize</Item>
      <ArrayItems>
        <Size>m_nHashTableSize</Size>
        <ValuePointer>m_pHashTable</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>

  <Type Name="CMap&lt;*,*,*,*&gt;">
    <AlternativeType Name="CMapPtrToWord"></AlternativeType>
    <AlternativeType Name="CMapPtrToPtr"></AlternativeType>
    <AlternativeType Name="CMapStringToOb"></AlternativeType>
    <AlternativeType Name="CMapStringToPtr"></AlternativeType>
    <AlternativeType Name="CMapStringToString"></AlternativeType>
    <AlternativeType Name="CMapWordToOb"></AlternativeType>
    <AlternativeType Name="CMapWordToPtr"></AlternativeType>
    <AlternativeType Name="CTypedPtrMap&lt;*,*,*&gt;"></AlternativeType>
    <DisplayString>{Hash table too large!}</DisplayString>
  </Type>
  

  <Type Name="ATL::CAtlMap&lt;*,*,*,*&gt;">
    <AlternativeType Name="ATL::CMapToInterface&lt;*,*,*&gt;"/>
    <AlternativeType Name="ATL::CMapToAutoPtr&lt;*,*,*&gt;"/>
    <DisplayString>{{Count = {m_nElements}}}</DisplayString>
    <Expand>
      <Item Name="Count">m_nElements</Item>
      <ArrayItems>
        <Size>m_nBins</Size>
        <ValuePointer>m_ppBins</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>
  <Type Name="ATL::CAtlMap&lt;*,*,*,*&gt;::CNode">
    <DisplayString Condition="this==0">Empty bucket</DisplayString>
    <DisplayString Condition="this!=0">Hash table bucket</DisplayString>
  </Type>
</AutoVisualizer>

Visualizing Map is a bit tricky, and I didn’t take the time yet to look deep into it – but the file is hopefully useful as it is. To use, just save the text as, say, MfcContainers.natvis, either under %VSINSTALLDIR%\Common7\Packages\Debugger\Visualizers (requires admin access), or under %USERPROFILE%\My Documents\Visual Studio 2012\Visualizers\ .

NatStepFilter Files

- are the new and improved substitute for the NoStepInto registry key. While there are some online hints and traces, the natstepfilter spec is yet to be introduced into MSDN – or even the VC++ team blog. For now you can watch the format specification, along with some good comments, at the %VSINSTALLDIR%\Xml\Schemas\natstepfilter.xsd near you, or even better – inspect a small sample at %VSINSTALLDIR%\Common7\Packages\Debugger\Visualizers\default.natstepfilter.

The default.natstepfilter is implemented by Stephen T. Lavavej, and is very far from complete – both because of regex limitations and because of decisions not to set non-overridable limitations on users:

“Adding something to the default natstepfilter is a very aggressive move, because I don’t believe there’s an easy way for users to undo it (hacking the file requires admin access), and it may be surprising when the debugger just decides to skip stuff.”

I can think of several ways for users to override .natstepfilter directives (never mind stepping-into via assembly, how about setting a plain breakpoint it the function you wish to step into?) – and so I don’t agree with that decision. Still I hope the default rules would improve alongside the documentation. We mostly avoid STL, so I had no need to customize .natstepfilter’s yet – I’ll be sure to share such customizations if I do go there.

Caveat

Both improvements, natvis and natstepfilter files, do not work for debugging native/managed mixed code, which sadly renders them unusable for most of our code. While this behavior is documented – I would hardly say it is ‘by design’. It does seem to irritate many others, so there is hope – as Brad Sullivan writes that MS are-

“… working on making everything just work in a future release of Visual Studio.”

Entry Point Not Found, and other DLL Loading Problems

Occasionally I come across DLL load problems:

The verbosity of the error messages varies greatly. In their raw form these include at least the DLL name, but as various frameworks come into play (for the error message above, it’s .net) – native exceptions are caught and re-thrown, and more often than not helpful information is lost on the way.

Turns out there’s a built in way to get verbose windows-loader output: the Show Loader Snaps flag. The easiest way to mark it is with the gflags utility, bundled with debugging tools for windows:

Under the hood, it merely adds a FLG_SHOW_LDR_SNAPS flag (0×00000002), to a DWORD value in the relevant IFEO registry key. This in turn causes Windows Loader to set the _ShowSnaps variable in the ntdll copy specific to the named process.

And now, behold the new and shiny loader trace (dumped to the debugger output window):

…    

2724:245c @ 11813487 – LdrpFindOrMapDll – RETURN: Status: 0×00000000

2724:245c @ 11813487 – LdrpLoadImportModule – RETURN: Status: 0×00000000

2724:245c @ 11813487 – LdrpLoadImportModule – RETURN: Status: 0×00000000

2724:245c @ 11813487 – LdrpLoadImportModule – RETURN: Status: 0×00000000

2724:245c @ 11813487 – LdrpSnapThunk – WARNING: Hint index 0x70a for procedure “?Revert@CStreamMemory@@UAGJXZ” in DLL “YaddaYadda.dll” is invalid

2724:245c @ 11813487 – LdrpSnapThunk – ERROR: Procedure “?Revert@CStreamMemory@@UAGJXZ” could not be located in DLL “YaddaYadda.dll”

First-chance exception at 0x77321d32 (ntdll.dll) in Strategist.exe: 0xC0000139: Entry Point Not Found.

Bam! There’s the offending DLL and the offending imported function, right there in the debugger.

Like many other useful features – it is documented, but very low on discoverability. Which is a fancy way of saying you can find it only if you already know exactly what you are looking for. I personally got around to it after digging around in ntdll assembly (just like Matt Pietrek, 14 years ago), trying to get to a string containing the name of an offending DLL.

The windows-copycat-opensource ReactOS source gives a nice view of the internal usage of this flag – called ShowSnaps in their source. The ‘snapping’ verb in this context refers to one of the actions performed by the loader: after rebasing the loaded DLL in the loading process memory space, the DLL’s exported function addresses are updated and must be copied to the importing process (or other dll) Import Address Table. This – in this context – is called snapping, and that’s where the extra tracing is hooked.

_DllMain@12 already defined

We recently faced this linkage error:

error LNK2005: _DllMain@12 already defined in MSVCRT.lib(dllmain.obj)

Searching gives ~36K results as of July 2012, many of which seem high quality (StackOverflow, MS, CodeProject etc.), and I was certain it would be a simple matter of finding a fix online and blindly applying it. However it seems the root cause in our particular case wasn’t covered yet (AFAIK), and it seems worthwhile to document.

The MS KB article teaches that this is a linkage order problem – MFC libs must be linked before the CRT ones – but none of the fixes the article proposes worked. We did have one build configuration which was successful and one which failed with the above LNK2005 (Release – but it really doesn’t matter) so I dumped two /VERBOSE linker outputs for the two configurations and diffed them. After some admittedly tedious inspection, an interesting difference came up – these lines were dumped only in the successful build:

Found __afxForceUSRDLL

Referenced in Stdafx.obj
Loaded mfcs100d.lib(dllmodul.obj)

The symbol name implies that it is intended to force some linkage, and including it seems to have the beneficial effect of loading the mfc lib mfcs100d.lib.  Indeed, searching reveals the following lines in dllmodul.cpp:

#ifdef _X86_
extern "C" { int _afxForceUSRDLL; }
#else
extern "C" { int __afxForceUSRDLL; }
#endif

and the following in afx.h:

// force inclusion of DLLMODUL.OBJ for _USRDLL
#ifdef _USRDLL
#pragma comment(linker, "/include:__afxForceUSRDLL")
#endif

So it turns out there’s a single condition that governs the linkage to the MFC library mfcs100/d (the one containing DllModul.obj, which exports _afxForceUSRDLL), and that condition is – _USRDLL being defined.   Our linking project was indeed a dll and somehow the default _USRDLL preprocessor macro was missing from it – restoring the definition fixed the linkage.

So bottom line, if you get a ‘DllMain@12 already defined’ linkage error for a dll, here’s another thing to try: make sure _USRDLL is defined in your project C++ property sheets.

Viewing types, part 3: Exceptions

Last time a way was shown to use internal RTTI mechanics to view C++ type names without direct debugger aid. There is one place in particular where such type names carry substantial information, and that is c++ exceptions. These can also require some additional treatment.

Native SEH exceptions are characterized – and can be caught – by a code. In contrast, C++ exceptions are characterized and caught by a type. So if you are debugging already within a catch-block (which is after some hefty compiler and runtime machinery already kicked in), you already have a direct view of the caught type, and need no extra trickery.

Where things get delicate is when a C++ exception goes uncaught in any C++ catch block, and is either caught in an SEH __except clause or goes completely uncaught and (hopefully) included in a core dump for you to analyze.  C++ exceptions, which are implemented on top of SEH ones, all utilize the code E06D7363 (standing for Exception, ‘M’ ‘V’ ‘C’ in ascii) and so cannot be distinguished by code. Some low-level trickery is in order.

The Old New Recipe

Sometime in 2010 Raymond Chen posted a recipe on how to view types of such exceptions (i.e.: originating in VC++, viewed in a context of SEH). Briefly, once you obtained an EXCEPTION_RECORD, do as follows:

Take Parameter 2 [of the exception record, O.S.] and go to the fourth DWORD and treat it as a pointer. (On 64-bit systems, you have to add this value to the HINSTANCE passed as Parameter 3 to convert it to a pointer.)

Next, go to the second DWORD and treat it as a pointer. (Again, on 64-bit systems, it’s really an offset from the HINSTANCE.)

Next, go to the second DWORD and treat it as a pointer. (64-bit systems: you know the drill.)

Finally, skip over the first two void*s and the rest is the class name.

And here’s a shameless cut & paste of his diagram:

EXCEPTION_RECORD
+----------+
| E06D7363 |
+----------+
|  ~~~     |
+----------+
|* ~~~     |
+----------+
|* ~~~     |
+----------+
| 3 or 4   |
+----------+
|* ~~~     |
+----------+
|*Object   |
+----------+     +---+
|*       ------> |~~~|
+----------+     +---+
|*HINSTANCE|     |~~~|
+----------+     +---+
                 |~~~|
                 +---+    +---+
                 | -----> |~~~|
                 +---+    +---+    +---+
                          | -----> |~~~|
                          +---+    +---+    +----------+
                                   | -----> |*   ~~~   |
                                   +---+    +----------+
                                            |*   ~~~   |
                                            +----------+
                                            |Class name|
                                            +----------+

Decorate with type names

If you’re thinking there’s gotta be a better way, you’d be right. Dumping VC++ intrinsic compiler types by compiling an empty file with /d1reportAllClassLayout – as done for type analysis – reveals a few types that directly relate to exception handling:

class _s__CatchableType    size(28):
+---
0    | properties
4    | pType
8    | _PMD thisDisplacement
20    | sizeOrOffset
24    | copyFunction
+---
class _s__CatchableTypeArray    size(4):
+---
0    | nCatchableTypes
4    | arrayOfCatchableTypes
+---
class _s__ThrowInfo    size(16):
+---
0    | attributes
4    | pmfnUnwind
8    | pForwardCompat
12    | pCatchableTypeArray
+---

Fitting these into Raymond’s diagram gives -

EXCEPTION_RECORD
+----------+
| E06D7363 |
+----------+
|  ~~~     |
+----------+
|* ~~~     |
+----------+
|* ~~~     |
+----------+
| 3 or 4   |
+----------+
|* ~~~     |
+----------+
|*Object   |  _s__ThrowInfo 
+----------+     +---+
|*       ------> |~~~|
+----------+     +---+
                 |~~~|
                 +---+
                 |~~~|  _s__CatchableTypeArray
                 +---+    +---+
                 | -----> |~~~|    _s__CatchableType
                 +---+    +---+    +---+
                          | -----> |~~~|   _TypeDescriptor
                          +---+    +---+    +----------+
                                   | -----> |*   ~~~   |
                                   +---+    +----------+
                                     .      |*   ~~~   |
                                     .      +----------+
                                     .      |Class name|
                                            +----------+

Into the debugger

Now for a toy – but real – example in VC. Take this code:

__try
{
throw std::runtime_error("adsf"); // throw a C++ exception
}
__except (EXCEPTION_EXECUTE_HANDLER ) // catch as a native exception
{
__debugbreak(); // can we identify the C++ type of the thrown exception?
}

Here’s an initial inspection of the exception record in the watch window:

image

NumberParameters being 3 indicates that this is an x86 system. The interesting parts are the contents of ExceptionInformation. Forget about parameter 1 for a minute, and let’s inspect parameter 2 – which by the analysis above should be of type _s__ThrowInfo:

image

There seem to already exist multiple indications of the thrown C++ type name, as circled in red, with no need to go further down Raymond’s recipe. However this is only due to the fact that our toy code throws a standard type for which symbols are available. If such is the kind exception you’re facing – you can happily stop here. If you’re facing a custom type – follow this game a bit further.   For some reason VS properly recognizes the type of arrayOfCatchableTypes but does not expand it, so let’s do it ourselves:

image

And the name is available as part of the TypeDescriptor, as for a regular C++ type (bottom red rect). Undname.exe it – if the decorated name isn’t obvious enough – and view the type name as appeared in the source code.

Bonus: An Alternate (Arguably Better) Recipe

As Raymond notes, parameter 1 of the ExceptionInformation (red rect ‘1’ above) is usually a pointer to the C++ object being thrown. It might contain much more information that it’s type name (well, at least if it’s not a custom type it should be accessible to you), and as seen in the previous post – an object contains its own route to the TypeDescriptor and hence the type name.  Here’s how it looks in the watch window:

image

Raymond notes that parameter 1 usually points to the thrown object, but -

…sometimes there is some junk in front that you have to skip over. Once you figure out what it is, you can dump it. (I haven’t bothered trying to figure out exactly how much; I just dump bytes and figure out the correct start of the object by inspection.)

You can easily do that from within VS too: if the expression in the example

  (_s__RTTICompleteObjectLocator**)(*(int*)0x003bf75c-4)

(containing the direct parameter 1, 0x003bf75c) expands to rubbish, start increasing the pointer until you arrive at meaningful struct contents. This seems a faster way to get to the information of interest since you get to the thrown object and its type name in a single search.

Viewing types, part 2: the manual way

Compiling with /d1reportAllClassLayout, even an empty file, dumps many compiler-intrinsic types. Six of these are interesting in the present context:  _PMD, _TypeDescriptor, _s__RTTIBaseClassDescriptor2, _s__RTTIBaseClassArray, _s__RTTIClassHierarchyDescriptor, and _s__RTTICompleteObjectLocator.

Most of this machinery exists solely for multiple inheritance, and virtual inheritance in particular – which modern languages eschew and is considered (I hope) a painful legacy by all current C++ professionals, and so I won’t go further in. Go here for an excellent survey on these deep internals of multiple inheritance and RTTI.

Turns out that for viewing types in sane (== single inheritance) code, two of these intrinsic types suffice:

1> class _TypeDescriptor    size(8):
1>      +—
1>   0    | pVFTable
1>   4    | spare
1>   8    | name
1>      +—
1>
1> class _s__RTTICompleteObjectLocator    size(20):
1>      +—
1>   0    | signature
1>   4    | offset
1>   8    | cdOffset
1>  12    | pTypeDescriptor
1>  16    | pClassDescriptor
1>      +—

The gem mentioned above tells us that an RTTICompleteObjectLocator pointer is placed right before the vftable. (This struct is responsible for locating the adjoint vftable within a larger vftable in the case of multiple inheritence, where method dispatching requires adjustor thunks, dynamic-casts require traversal of the full inheritance tree and things get hairy in general.  Not another word on that, I swear).

RTTICompleteObjectLocator contains a pointer to a TypeDescriptor, which happily points to a name!   To view this in the debugger, take a typed pointer (say, pb -sticking with the setup from last time), and type at a watch window

(_s__RTTICompleteObjectLocator**)((int*)pb->__vfptr – 1)

Here’s what you’d see:

image

That is (naturally) a decorated name, so pop open a VS command prompt and call undname.exe with the 0×2000 flag (decipher types) and that name string, omitting the leading ‘.’ :

image

And now, for something completely useful.

All the above may seem like a very roundabout – albeit cool – way of seeing what the debugger already shows, but this does present a very useful trick:

What if you don’t have symbols?

It is very easy to recognize dispatching of virtual functions (call on a register argument), and typically a few instructions before that $ecx would be populated with the ‘this’ pointer.  Now, you can know the name of the type whose methods you call, even without symbols!   (assuming the code was built with RTTI on).

E.g., around this virtual method call:

image

Type this at a watch window:

(_s__RTTICompleteObjectLocator**)(*(uintptr_t*)$ecx-sizeof(int*))

(some complexity added to conform also to x64 systems, thanks @Martin Ridgers!), and you’ll be greeted be a magnificent view:

image

Viewing types, part 1: The Normal way

Suppose C derives from B, which derives from A:

class A
{
public:
virtual void A_virt() {}
};

class B : public A
{
public:
virtual void B_virt() {}
};

class C : public B
{
public:
virtual void C_virt() {}
};

Now create A-pointers which point to children types, B and C, and watch them in VS:

image

The types in square brackets are the most derived types of the referenced objects. How does the debugger know them?

It can’t be pdb-voodoo, as evidently dynamic types are runtime only info. This info has to lurk somewhere in memory accessible at run time to the debugger.

Now where would be a good place to store such info?  We (wearing compiler designer hats for a second) could embed type info into every object instance, but that would be a gross duplication. We need a place that holds info common to all instances of the same class.

Such as, say, a virtual table.

- which is exactly where compiler writers do put run-time-type-info (RTTI).

Evidence (1): Watch the type name at the __vfptr value (circled in red above).

Evidence (2): mess with the __vfptr directly. Change pb->__vfptr to contain C::vftable:

image

And watch pb’s type miraculously change to C in the watch window:

image

Evidence (3): delete all virtual methods from A, B, and C, and watch the most-derived-type info disappear.

Evidence (4): compile without RTTI,

image

and watch the most-derived-type data disappear again.

So the debugger uses RTTI information, that is somehow attached to the vtable, to display type information. If the debugger can, perhaps we can do it ourselves when we must?…

Blank Variable Watch, or OMF Errors

During a debugging session I faced a weird situation where the code compiled and ran just fine, yet some class members appeared blank at the watch window, and others showed -

CXX0033: error in OMF type information

Skip to the bottom line: there’s a compiler switch, /Yl, that’s specifically tailored to address this symptom. In my project, the issue was solved by adding  /YlSomeFunctionIUse to the stdafx.cpp compiler command line (in the project that defined, not consumed, the blank symbols).

The root cause seems to be a clash of two intended behaviors: (1) Debug info of defined symbols is embedded in the PCH object module (~= .obj file) itself, (2) when a source file refers to the PCH but does not directly use any of the functions defined in it, the PCH object module is dropped from that compilation unit altogether, and thus relevant debug info is lost.

Quoting the /Yl msdn page:

An error can occur when you store the precompiled header in a library, use the library to build an object module, and the source code does not refer to any of the functions the precompiled header file defines.

- so it might be that the symptom is revealed only when the PCH is used in a build of a static library.

This KB article says this behaviour is by design, which seems weird to me. I see no justification for dropping the debug info along with unused function definitions. I’d gladly pay the price of some PDB bloat, to avoid having to use arcane, hidden compiler switches just to be able to debug properly.

BTW, WTF is OMF?

It really stands for [Relocatable] Object Module Format, a relic of ~20 years (which goes to show how old this debugger code is, really). It’s an object-file format designed by Intel in the 70s, and used by MS prior to adoption of their own COFF flavour sometime in the early 90s. The spec is still around, but seems untouched since 1993. The industry probably gave up on the idea of using different vendors for different parts of the compiler-linker-loader tool chain, and so standardization efforts have halted since.

StepOver Revisited

Andy Pennell exposed in 2004 (and I mentioned in 2009) a very useful undocumented VC feature: when you wish to avoid stepping into nagging functions (ctors, refcounts, whatever), you can specify them in the StepOver registry key. It supports RegExp’s and some extra syntax, so it was very convenient to specify entire framework classes and namespaces (ATL, MFC, std etc) and save many clicks during debugging.

As the case often turns out for undocumented goodies, the feature started showing some cracks in VS2010. Here is one very noticeable such crack.

When you have two or more VS instances running, they’re likely to compete over access to system resources. As of VS2010, when instances compete over the registry key -

HKCU\Software\Microsoft\VisualStudio\10.0_Config

…there’s a good chance it would be duplicated, with a suffix containing one of the instances’ process ID:

image

However, since access to the original key had failed – the new copy is not an exact duplicate and probably just contains some defaults (which brings up the question of why duplicate it in the first place, but I assume there’s a valid reason).

So there you have it – once such a race occurs, one of the VS instances would be blind to all the customizations you had put into 10.0_Config\NativeDE\StepOver, and you’d be back wasting dozens of clicks stepping in and out of std::shared_ptr copy ctors.

This seems easily fixable, but since it was an underground feature to start with – testing it was probably never part of any requirement for VS2010. Hope it makes it back some day.

Breaking on Data Read

You’re probably familiar with Data Breakpoints, and rightfully so: It’s extremely useful to know where a value changes. But did you know that with a little help VS can break when a value is used?

Usage

By ‘little help’ I mean some code. Plenty of free implementations are available: 1, 2, 3, 4. All are very similar (one notable difference discussed below), but I’m used to the first and will use it below.

First, a toy example:

#include "Breakpoint.h" 
...
CBreakpoint g_bp;
…
void Whateva()
{
int a = 3, b;
g_bp.Set(&a, 4, CBreakpoint::Read);
b = a; // g_bp breaks!
g_bp.Clear();
}

The link does mention that you can call CBreakpoint::Clear() from a QuickWatch window (== from anywhere the Expression Evaluator lives, for that matter). What's even more useful - you can call CBreakpoint::Set() from the debugger with a minor additional cast. While debugging the code above, evaluate the following in any watch window:

g_bp.Set(&a, 4, (CBreakpoint::Condition)3)

image 

Internals (well, some, anyway)

Both read and write breakpoints are implemented via debug registers: special registers on a x86 CPU which trigger an 'int 1' interrupt ('debug step') whenever a pre-specified virtual address is accessed. Debug Register Dr7 is set to activate any hardware breakpoint, Dr0-Dr3 determine the type (11b means read/write).

All implementations linked above modify the debug registers via SetThreadContext. The documentation includes a grave warning, that only implementations 3 & 4 seem to respect:

Do not try to set the context for a running thread; the results are unpredictable. Use the SuspendThread function to suspend the thread before calling SetThreadContext.

However I've never had issues with implementation 1, so I assume in practice usage of SetThreadContext with this particular mask (CONTEXT_DEBUG_REGISTERS) is safe.

This usage makes one wonder – are debug registers indeed part of a thread context? Are they reset on every context switch?

The intel manuals, vol 3A, section 16.4.2 details the contents of DR7:

The debug control register (DR7) enables or disables breakpoints and sets breakpoint conditions ... The flags and fields in this register control the following things:

• L0 through L3 (local breakpoint enable) flags (bits 0, 2, 4, and 6) — Enables (when set) the breakpoint condition for the associated breakpoint for the current task. When a breakpoint condition is detected and its associated Ln flag is set, a debug exception is generated. The processor automatically clears these flags on every task switch to avoid unwanted breakpoint conditions in the new task.

Oh dear. Are hardware breakpoints indeed that useless? Are they indeed blind to reads/writes by other threads?

Well obviously, no. It's a 1 minute test to set a HW-Bp, modify its address from a different thread and watch it trigger.

It all boils down to a nuance in x86 terminology: tasks are not threads. Windows does not use the task context switching hardware apparatus that x86 offers, so it really is an OS decision whether to store debug registers per thread – and the obvious choice seems to store them per process. That is probably the reason calling SetThreadContext with CONTEXT_DEBUG_REGISTERS mask is safe also for non-suspended threads.

g_dwLastErrorToBreakOn: Watching Errors on VS Revisited

Raymond Chen posted about SetLastError recently, and an interesting discussion ensued. One comment in particular caught my eye:

The easiest way to catch a specific last error value in debugger is to set ntdll!g_dwLastErrorToBreakOn to that value.

A good while back I needed to break when such a LastError is set, and dug up all sorts of hacks to do so – breaking at SetLastError, setting data breakpoint on the thread-env-block-error, and the like. Beyond being cumbersome and plain ugly such breakpoint tricks can be very slow, and give a lot of false positives.

Seems the Win32 folks had similar needs and I was glad to discover they formed a better (undocumented, but still) solution. The authoritative source seems to be a 2007 post from Microsoft’s Daniel Pearson:

..Hiding inside of kernel32′s address space is a global variable called g_dwLastErrorToBreakOn. It turns out that SetLastError checks the value of this variable and if it’s non-zero, calls DbgBreakPoint if the two [values] match.

It’s a zero-overhead trick, and is very easy to do in Visual Studio: make sure kernel32.dll symbols are loaded, then type in a watch window –

(int*){,,kernel32.dll}_g_dwLastErrorToBreakOn

- and edit the referenced int:

image

Pearson notes two changes introduced in Vista:

(1) Up until XP, only Win32 API implemented in KERNEL32.DLL actually used SetLastError (and so tested g_dwLastErrorToBreakOn) – other dll’s used to set the error value via RtlSetLastWin32Error. Since Vista all Win32 API which set an error do so with SetLastError, so the g_dwLastErrorToBreakOn is much more reliable.

(2) Since Vista, g_dwLastErrorToBreakOn moved to NTDLL.DLL, so the VS usage should be changed to -

(int*){,,ntdll.dll}_g_dwLastErrorToBreakOn

It’s interesting to note that ntdll.dll does contain a separate instance of g_dwLastErrortToBreakOn also on XP machines:

image

But I verified that this value is never read, on calls into both kernel32 and ntdll.