Table of contents
In my professional career, I have seen most of us use Visual Studio for debugging but not many of the other debuggers that come for free. You may want such a debugger for many reasons, for example, on your home PC which you do not use for development but on which a certain program crashes from time to time. From the stack dump, you can figure out if IE crashed because of a third party plug-in.
I did not find any good quick starters for WinDbg. This article discusses WinDbg with examples. I assume you know the basic concepts of debugging – stepping in, stepping out, breakpoints and what it means to do remote debugging.
Note that this is meant to be a Getting Started document, which you can read and start using WinDbg. To know more about specific commands, consult the WinDbg documentation. You can use the commands presented in this document with any debugger provided by Microsoft, e.g. from the Command window of Visual Studio .NET.
This article is based on WinDbg 6.3.
This is the first of a series of articles on debugging. In my next article, I shall explain how to write debugger extension DLLs.
A brief overview of the Windows debuggers that you can download for free from here:
- KD – Kernel debugger. You want to use this to remote debug OS problems like blue screens. You want it if you develop device drivers.
- CDB – Command-line debugger. This is a console application.
- NTSD – NT debugger. This is a user-mode debugger that you can use to debug your user-mode applications. Effectively, this is Windows-style UI added to CDB.
- Windbg – wraps KD and NTSD with a decent UI. WinDbg can function both as a kernel-mode and user-mode debugger.
- Visual Studio, Visual Studio .NET – use the same debugging engine as KD and NTSD and offer richer UI than WinDbg for debugging purposes.
| Feature |
KD |
NTSD |
WinDbg |
Visual Studio .NET |
| |
|
|
|
|
| Kernel-mode debugging |
Y |
N |
Y |
N |
| User-mode debugging |
|
Y |
Y |
Y |
| Unmanaged debugging |
Y |
Y |
Y |
Y |
| Managed debugging |
|
Y |
Y |
Y |
| Remote debugging |
Y |
Y |
Y |
Y |
| Attach to process |
Y |
Y |
Y |
Y |
| Detach from process in Win2K and XP |
Y |
Y |
Y |
Y |
| SQL debugging |
N |
N |
N |
Y |
WinDbg is a debugger that wraps NTSD and KD with a better UI. It provides command-line options like starting minimized (-m), attach to a process by pid (-p) and auto-open crash files (-z). It supports three types of commands:
- regular commands (e.g.: k). The regular commands are to debug processes.
- dot commands (e.g.: .sympath). The dot commands are to control the debugger.
- extension commands (e.g.: !handle) – these are custom commands that you can add to WinDbg; they are implemented as exported functions in extension DLLs.
PDB files are program database files generated by the linker. Private PDB files contain information about private and public symbols, source lines, types, locals and globals. Public PDB files do not contain types, local and source line information.
Doing remote debugging using WinDbg is easy and can be done in one of a number of ways. In the following, ‘debugging server’ is the debugger running on the machine where you’d like to debug; ‘debugging client’ is the debugger controlling the session.
Server will display who all are connected from which servers and commands executed. You can quit the server by issuing ‘qq’; or quit the client using File->Exit. You’d need to belong to the “Debugger Users” user group and the server has to allow remote connectivity if you want to remote-debug.
The section “Enabling Postmortem Debugging” in the WinDbg documentation discusses this well. In short, you can set WinDbg as the default JIT debugger by running Windbg –I. This sets the registry key HKLM\Software\Microsoft\Windows NT\CurrentVersion\AeDebug to WinDbg. To set WinDbg as the default managed debugger, you’d need to set these registry keys explicitly:
HKLM\Software\Microsoft\.NETFramework\DbgJITDebugLaunchSetting to 2 HKLM\Software\Microsoft\.NETFramework\DbgManagedDebugger to Windbg.
With the JIT setting, WinDbg will be launched if an application throws an exception while not being debugged and does not handle the exception itself.
All these debuggers support 64-bit debugging on AMD64 and IA64.
WinDbg 6.3+ supports managed debugging, with the Whidbey .NET CLR. There is a good discussion on managed debugging in the documentation. Remember that there are no PDBs with managed code since managed code is compiled to ILASM; the debugger talks to the CLR to query extra information.
Points to note:
You can set a breakpoint at a managed code function only after it has been invoked at least once; because that is when it is JIT-compiled to ASM code. Keep in mind:
- Complications with function addresses and hence breakpoints:
- The CLR can discard compiled code, so function addresses may change.
- The same code may be multiply compiled if multiple app domains do not share the code. If you set a breakpoint, it gets set for the app domain of the current thread.
- Specialization of generics can cause multiple addresses for the same function.
- Complications with data layout and hence data inspection:
- The CLR may change data layout arbitrarily at runtime, so field offsets in a structure may change over time.
- Type information is loaded only on first use, so you may not be able to inspect a data field if it has not been used yet.
- Complications with debugger commands:
- When tracing through managed code, you would pass through chunks of runtime code like the JIT compiler code because you stepped into a function for the first time, or, when transitioning from managed to unmanaged code.
You can debug a service just as any other application using WinDbg, both after starting the service by attaching to the service process, and, by using WinDbg as a JIT debugger and programmatically calling DbgBreakPoint or DebugBreak, or an ASM int 3 on x86.
A debugger gets notified of each exception twice – it is notified the first time before the application gets a chance to handle the exception (‘first chance exception’); if the application does not handle the exception, the debugger is given a chance to handle the exception ( ‘second-chance exception’). If the debugger does not handle a second-chance exception, the application quits.
.lastevent, or, !analyze –v will show you the exception record and stack trace of the function where the exception occurred.
You can also use the .exr, .cxr and .ecxr commands to display the exception and context records. Note also that you can change the first-chance handling option for an exception using the sxe, sxd, sxn and sxi commands.
Debugger extensions are DLLs that you can hook up with a debugger to execute custom commands from within the debugger. There are certain functions that a DLL needs to implement and some requirements that a DLL needs to meet in order to qualify as an extension DLL. In the next article, we shall learn how to write an extension DLL yourself. The bang (!) commands are commands executed from your extension DLLs. Note that extension DLLs are loaded in the process space of the debugger.
You can take snapshot information of a process using the dump facility. A mini-dump is usually small, unless you take a full-memory minidump (.dump /mf). It is useful to dump handle information also, as .dump/mfh. A mini-dump contains information about all threads including their stacks and list of loaded modules. A full dump contains more information, like that of the process heap.
If your Windows OS crashes, it dumps the physical memory contents and all process information to a dump file, configured through System->Control Panel->Advanced->’Startup and Recovery’. It is also possible to take dumps of any live process by breaking into it. You can also take a dump of any process (.dump) that terminates abnormally by configuring WinDbg as a JIT debugger. Note that figuring out bugs in the code from a crash dump could be an involved process.
To analyze a dump, follow these steps:
Step 1: In WinDbg, File->’Open Crash Dump’, and point to the dump file
Step 2: WinDbg will show you the instruction your app was executing when it crashed.
Step 3: Set your symbol path and source path properly. If you cannot match symbols, you could have a hard time figuring out control flow. If you can match the symbols to source code of the appropriate version, it should be easy to figure out the bug at this point. Note that private symbol files have line number information and will blindly show the line in your source code without further checks; if your source is not version-matched properly, you’d not see the correct source code matching the assembly code. If you have public PDB files, you’ll see the last public function (on the call stack) that was invoked.
Note that debugging drivers or managed code is much different. Refer to [2] for debugging techniques for device drivers.
Symbol Files and Directories
You need symbols in order to be able to do effective debugging. Symbol files could be in an older COFF format or the PDB format. PDBs are program database files and contain public symbols. These debuggers allow you to mention a list of URIs where they would look for symbols for loaded binaries.
OS symbols are usually installed in the %SYSTEMDIR%Symbols directory. Driver symbols (.DBG or .PDB files) are usually in the same folder as the driver (.sys file). Private symbol files contain information about functions, local and global variables, and line information to correlate assembly code to source code; symbol files that are usually made available to customers are public symbol files – these files contain information about public members only.
You can set symbol directories through File->Symbol File Path, or using .sympath from the WinDbg command window. To add reference to a symbol server on the web, add:
SRV*downstream_store*http://msdl.microsoft.com/download/symbols
to your .sympath, thus:
.sympath+ SRV*c:\tmp*http://msdl.microsoft.com/download/symbols
Where c:\tmp is the download_store where necessary symbols will be downloaded and stored. Note that this particular symbol server exposes public symbols only.
The debugger matches information like filename, timestamp and checksum when matching a PDB with a binary (DLL or exe). If you have symbol information, you’d be able to see function names and their arguments in your call stack. If the binaries and PDBs are from your application, you’d additionally have information about private functions, local variables and type information.
The sympath can consist of multiple URIs. Sympath is initialized from the _NT_SYMBOL_PATH system environment variable.
You can set source code directories through File->Source File Path, or using .srcpath from the WinDbg command window. If you set source code directories, the debugger will pull up matching source code based on line number information from the PDB files during debugging.
- Set soft breakpoints using the
bp commands or using the toolbar breakpoint icon.
- Set hard breakpoints using code like
DbgBreakPoint() or KdBreakPoint().
- Use tracing routines
DbgPrint, KdPrint, OutputDebugString to print out to the WinDbg output window, from debugger extension DLLs.
The help file that comes with the WinDbg installation documents commands well, but the following basic commands should get you started:
| Feature |
Command |
What Does it Do |
Example / Comments |
See Also Related Commands |
| |
|
|
|
|
| Stack trace |
K, KB x |
Displays stack trace of current thread (x frames). Kb causes the display to include the first three parameters passed to each function. |
|
KP, Kp, or KV |
| Frame |
.frame X |
|
|
|
| Register watch |
R |
Displays register set. reax – displays the eax register. |
|
|
| Step |
t |
Trace = Step into (F11) |
|
|
| |
p |
Step over (F10) |
|
|
| |
Step out |
Shift + F11 |
|
|
| Disassemble |
u |
Unassemble next few instructions |
|
|
| |
u <start_address> |
Unassemble instructions at start_address |
|
|
| |
u <start_address>
<end_address>
|
Unassemble instructions from start_address till end_address |
|
|
| Breakpoints |
Bl |
List breakpoints. |
|
|
| |
be, bd, bc |
Enable / disable / clear breakpoint. |
|
|
| |
bp |
Set a breakpoint. |
|
|
| |
bu |
Set unresolved breakpoint. Breakpoint is resolved by symbolic name, not absolute address. Use this to set breakpoint at a function whose containing module has not yet been loaded. |
bu foo |
|
| |
|
|
|
|
| Comment |
* |
Ignores the command |
* Hello World |
|
| Continue |
G <address_X / symbol> |
Go. Resumes execution until address_X |
|
|
| |
GH |
Go, exception handled |
|
|
| |
GN |
Go, exception not handled |
|
|
| Quit |
Q |
|
|
|
| Dumping data |
dv |
Display local variables. |
You need private symbols. |
|
| |
Dd <address> |
Display dword values at specified address. |
To see value of an int, DD <addr> L1 |
|
| |
Ds, da (ASCII), du (Unicode) |
Dump string |
|
|
| |
Dt [dt module!typedef adr] |
Dump type. Will dump the contents of the memory using typedef as a template. |
|
|
| Change / Edit Values |
Eb (byte), ed (dword), ea (ASCII), eu (Unicode) |
Edit value of a variable |
|
|
| List modules |
lm |
List loaded modules |
|
Lmi, lml, !dlls |
| Threads |
~ |
Lists all threads |
|
|
| Command on thread n |
~n<command> |
Switch to a specific thread by thread-id and execute a command on the thread. |
~2kb (second thread’s stack) |
|
| |
|
|
|
|
| Search for a symbol in a module |
X module!<pattern> |
|
X blah!*foo* |
|
| Dump |
.dump |
|
|
|
| Source line display |
.lines |
Turns on source code display |
|
|
| |
ln adr |
Will show the symbol nearest to that location. |
|
|
Note:
- There is no “step out” (Shift+F11). You have to find the return address on the stack manually and use “g adr”. You can find this address by using “k”. If you know the function uses ebp frames you can use “g poi(ebp+4)” to step out.
- To inspect local variables:
- Use the “dv” command.
- Then use the “dt <
variablename>” command.
- Note: you may not see correct values if values are stored in registers or due to FPO.
| Feature |
Command |
What Does it Do |
Example / Comments |
See Also Related Commands |
| |
Vertarget |
Shows information about the system on which you are debugging. |
|
|
| Data breakpoint (hardware bp) |
Ba
[ba r/w/e size adr]
|
Sets a data breakpoint. You can break on read/ write/ execute attempt of a memory location. |
ba w4 adr |
|
| Exceptions |
.lastevent |
Displays last exception record |
|
|
| Exceptions |
Sx, Sxe, sxd, sxn, sxi exception_X |
Enable/ disable/ notify-only/ ignore first chance exception /event exception_X. Example of event: module unload/ thread creation. |
|
|
| Display type |
Dt |
Shows struct and field values. |
Dt x; // x: int
Dt myStruct; // struct myStruct
Dt myStruct myVar1; // shows myStruct.myVar1 |
|
| Reload symbols |
.reload |
Reloads symbols using the symbol path you would have set. |
|
|
| Source lines |
l+l, l+o, l+s, l+t |
Source line options |
|
|
| |
.ecxr |
If you had an exception, switches context to faulting context. |
|
|
| |
.quit_lock |
|
|
|
| |
; |
Command separator |
|
|
| |
? |
Evaluate expression |
|
|
| |
| |
Display process information |
|
|
| |
.chain |
Lists all loaded debugger extensions. |
|
|
| |
.echo <string> |
Echo/ print any string |
Echo xyz |
|
| |
.exr <address_x> |
Display exception record at x. |
|
|
| |
.cxr <address_x> |
Display context record at x. |
|
|
| |
.trap |
Dump a trap frame. |
|
|
- !help – help for WinDbg extension commands.
- !load, !unload – to load and unload debugger extension DLLs.
- !handle – displays information about handles owned by processes.
- !peb - shows the PEB (process environment block) including DLL information.
Attached is a sample application with these example functions:
- Example1: Program appears hung because a thread waits indefinitely on a critical section that another thread acquired and then exited without releasing.
- Example2: Exception: division by zero.
- Example3: Execute a command every time a breakpoint is hit.
- Example4: Exception: null pointer access
- Example5: Exception: double deletion
- Example6: Exception: stack overflow due to infinite recursion
- Exception: Array out-of-bound access
- Exception: Deleted pointer access
- Exception: Stack underflow
Please note that:
- How can I list all symbols exported by a module?
x <module>!*
- How can I find help for a specific command?
.hh <command>, or <command> /?
- I want a certain application x.exe to run always under WinDbg. How can I configure this?
Create a key named x.exe under “HKLM\Software\Microsoft\Windows NT\currentversion\image file execution options” and add a new string value “Debugger” to it; set its value to the path of windbg.exe.
- I want to do something every time a breakpoint is hit. How can I do that?
The bp command accepts a list of commands as argument that you can execute every time a breakpoint is hit. Example:
bp WindbgEx1!Example3+0x3d "dd [ebp-0x14] L1; .echo hello world;g"
(ref. attached code)
prints the value of a local variable in each iteration of function Example3.
- Can I put a breakpoint that is triggered only once?
Yes:bp /1
- Can I set a breakpoint such that it will start hitting only after k-1 passes?
Yes, bp <address> k
- WinDbg documentation [from Microsoft]
- “The Windows 2000 Device Driver Book” – Art Baker, Jerry Lozano
