On API-MS-WIN-XXXXX.DLL, and Other Dependency Walker Glitches

Dependency walker is the tool of choice for static dependency analysis of native binaries (it has some dynamic analysis too, but that niche at least has some alternative solutions). It is in a rather sorry state, however – development seems to be abandoned since more or less 2005, and it is unanimously described as aging. As a prominent example of dependency walker analysis failures, try to run it on itself:

It seems the dependencies it is able to resolve are a negligible minority of the overall dependencies – and the interwebs are full of similar reports. The DLLs falsely reported as missing are all strangely named and unfamiliar, and the explanations given as SO answers range from a vague ‘some internal OS stuff’ to hypotheses about delay loads and side-by-side assemblies.

To the best of my understanding, as of March 2016 DependencyWalker 2.2 resolves side-by-side manifests very well and has no trouble with delay loads. I’m aware of only two dependency scenarios where it falls short, but unfortunately they are ubiquitous.

1: Compatibility Shims

Maybe more on that in another post. But –

2: Api Sets

…are the main issue.

Scarcely mentioned on MSDN:

An API Set is a strong name for a list of Win32 APIs … you should think of an API Set’s name as just a unique character string, and not as a dll name … API Sets rely on operating system support in the library loader … the library loader performs a runtime redirection of the reference…

Don’t be alarmed if that still sounds opaque. Brief history, as I understand it:

Sometime in the Vista dev cycle an effort referred to as MinWin began: essentially, smart people started moving functionality around in hope of simplifying the OS architecture. To protect the myriad components from breaking during a change, the ultimate solution was called in: an extra layer of indirection. This level is exactly Api Sets.

For example, the API set “api-ms-win-core-fibers-l1-1-1.dll” is an ‘atom’ of functionality encompassing the 5 APIs FlsAlloc, FlsFree, FlsGetValue, FlsSetValue and IsThreadAFiber (it is an untypically small such ‘atom’). All applications that consume fiber functionality declare dependency on this API set, and thereby become insensitive to the exact location of implementation (that might change between OS releases). During load time, the OS searches somewhere and automagically routes the calls from api-ms-win-core-fibers-l1-1-1.dll to wherever they happen to be implemented in this OS version.

One could argue that API sets now serve the original intended role of DLLs and that the architecturally clean solution is to have each API set implemented in its own DLL, but I’m sure this tradeoff has performance implications that I cannot even begin to quantify.

Some Internals

API sets are very partially documented, and the load-time mechanism that properly routes the calls – even less so. One could start by inspecting the shipped apiset.h (documented to be authored by the venerable Arun Kishan in Sep-2008), and learn that the key call is to the undocumented ApiSetResolveToHost. It is called from LoadLibrary, typically through a call stack such as –

ntdll.dll!_ApiSetResolveToHost@20()  + 0xf bytes
ntdll.dll!_LdrpApplyFileNameRedirection@28()  + 0x35 bytes
ntdll.dll!_LdrpLoadDll@24()  + 0xae bytes
ntdll.dll!_LdrLoadDll@16()  + 0x74 bytes
KernelBase.dll!_LoadLibraryExW@12()  + 0x120 bytes

The actual per-OS-version redirection data lies in a special file called ApiSetSchema.dll. Its technically a DLL (conforms to the PE spec), but not an executable one – the redirection data lies in a specialized section called .apiset, mentioned at the apiset.h macros. Sebastien Renaud did some spectacular reversing work and described the layout of the redirection data it contains.

Full(er) Redirection Table

In principle one could – and hopefully someday would – use Renaud’s work to create a community-maintained version of dependency walker, but until that day we can get by with the aforementioned built-in loader logging: whenever ShowSnaps is raised the loader spits out many hundreds of messages like –

3e30:02b8 @ 370478046 – LdrpPreprocessDllName – INFO: DLL api-ms-win-core-rtlsupport-l1-2-0.dll was redirected to C:\WINDOWS\SYSTEM32\ntdll.dll by API set

Running a few applications and filtering the results, I arrived at the table dumped below. I’ll update it as time permits – but if you have some dependency you don’t understand you can follow the same steps (well, for apps you can run, anyway): raise ShowSnaps for your app and inspect the output to see where the ApiSet I missed really routes to. If you do, please comment here so I can correct the table.

API Set Routes to…
api-ms-win-appmodel-state-l1-2-0.dll kernel.appcore.dll
api-ms-win-core-apiquery-l1-1-0.dll ntdll.dll


Edit (May 2016):

I won’t be listing the API sets here, as it turns out Geoff Chappell already took upon himself to maintain a list of API set redirections, along with versions and a very nice survey of the underlying apparatus, and actually a link to an MS patent describing the apparatus (if you’re able to decipher such descriptions).

Advertisements
This entry was posted in Win32. Bookmark the permalink.

8 Responses to On API-MS-WIN-XXXXX.DLL, and Other Dependency Walker Glitches

  1. John Schroedl says:

    I have wondered what happened to dependency walker as well. I will need to do some more investigating about api sets. Thanks for posting this, I’m surprised I haven’t heard about api sets before now.

  2. Alois Kraus says:

    Very interesting. Api sets are also a new thing to me. DependencyWalker still works good enough for me since ApiSets are not something any “normal” application will link to. If you are only searching for build inconsistencies it is still sufficient. But if you are after bugs in OS patches then things could get interesting …

    • Ofek Shilon says:

      Thanks. I agree Dependency walker is still useful, just even more so after you’re aware of its shortcomings.

  3. The redirection targets are already changing in some releases. E.g. the Nano Server does not even have to have kernel32.dll (it can be, for backward compatibility, added via “Reverse Forwarders”).

  4. Ofek,
    I for years just thought to myself “There are simply some DLL’s Dependency Walker can’t resolve”… now I have more information. Thanks :). More information than I needed, but gives me some rope with which to hang myself.
    I also decided to start a WordPress Blog, as a result of having to login to post this thank you :).
    Thanks for that too I suppose :).

    • Ofek Shilon says:

      Thanks. Did you have to sign in with a wordpress account to be able to comment?? If so, that surprises me (although of course I’m glad that resulted in a new blog)

  5. Pingback: First blog post – rossyoungbloodblog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s