(untagged)

Writing a Debugger - Part 2: The Debug Loop

Duncan Edwards Jones

0.00/5 (No votes)

21 Oct 2003

Part 2 in the series showing how to write a debugger in VB

Introduction

Windows NT (and successive operating system versions) expose a set of API functions and structures for debugging a running process. This article shows how these can be accessed from Visual Basic (version 5 or 6). It is recommended that the article Inside the executable: an introduction to the Portable Executable format for VB programmers be read in conjunction with this article, and the source code attached to it has the code for this article included.

Getting a Process to Debug

There are two ways to get a process to debug. Either:

attach a debugger to a process that is already running

or:

start a new process with a debugger attached to it.

Starting a New Process with a Debugger Attached

To start a process, you can use the CreateProcess API call:

Private Declare Function CreateProcess Lib _
    "kernel32" Alias "CreateProcessA" (ByVal lpApplicationName As String, _
                                       ByVal lpCommandLine As String, _
                                       ByVal lpProcessAttributes As Long,_
                                        ByVal lpThreadAttributes As Long,_
                                        ByVal bInheritHandles As Long, _
                                       ByVal dwCreationFlags As ProcessCreationFlags, _
                                       ByVal lpEnvironment As Long, _
                                       ByVal lpCurrentDirectory As String, _
                                       lpStartupInfo As STARTUPINFO, _
                                       lpProcessInformation As PROCESS_INFORMATION) _
                                       As Long

In addition to the functionality of the Shell command, this allows you to specify additional flags that affect how the process is created:

Public Enum ProcessCreationFlags
    DEBUG_PROCESS = &H1
    DEBUG_ONLY_THIS_PROCESS = &H2
    CREATE_SUSPENDED = &H4
    DETACHED_PROCESS = &H8
    CREATE_NEW_CONSOLE = &H10
    NORMAL_PRIORITY_CLASS = &H20
    IDLE_PRIORITY_CLASS = &H40
    HIGH_PRIORITY_CLASS = &H80
    REALTIME_PRIORITY_CLASS = &H100
    CREATE_NEW_PROCESS_GROUP = &H200
    CREATE_UNICODE_ENVIRONMENT = &H400
    CREATE_SEPARATE_WOW_VDM = &H800
    CREATE_SHARED_WOW_VDM = &H1000
    CREATE_FORCEDOS = &H2000
    CREATE_DEFAULT_ERROR_MODE = &H4000000
    CREATE_NO_WINDOW = &H8000000
End Enum

In order to start the process with an attached debugger, you specify the flags DEBUG_PROCESS + DEBUG_ONLY_THIS_PROCESS.

Attaching a Debugger to an Existing Process

To attach a debugger to a process which is already running, you need to obtain a handle to it and then attach a debugger using the DebugActiveProcess API call:

Private Declare Function DebugActiveProcess Lib "kernel32" _
                                   (ByVal dwProcessId As Long) As Long

The Debug Loop

Once you have attached your debugger to the process, you need to go into a debug loop. This consists of waiting for a debug event, processing the event when it comes in and then allowing the debugee to continue.

Waiting for a Debug Event to Occur

To wait for a debug event, you call the WaitForDebugEvent API call:

Private Declare Function WaitForDebugEvent Lib "kernel32" _
                                       (lpDebugEvent As DEBUG_EVENT_BUFFER, _
                                       ByVal dwMilliseconds As Long) As Long

This will return TRUE when a debug event has occurred and fill out the DEBUG_EVENT_... structure, which depends on what event occurred but always starts with a DEBUG_EVENT_HEADER:

Private Type DEBUG_EVENT_HEADER
    dwDebugEventCode As DebugEventTypes
    dwProcessId As Long
    dwThreadId As Long
End Type

Processing the Debug Event

How you deal with a debug event depends, naturally enough, on what event occurred. The event types are:

Public Enum DebugEventTypes
    EXCEPTION_DEBUG_EVENT = 1&
    CREATE_THREAD_DEBUG_EVENT = 2&
    CREATE_PROCESS_DEBUG_EVENT = 3&
    EXIT_THREAD_DEBUG_EVENT = 4&
    EXIT_PROCESS_DEBUG_EVENT = 5&
    LOAD_DLL_DEBUG_EVENT = 6&
    UNLOAD_DLL_DEBUG_EVENT = 7&
    OUTPUT_DEBUG_STRING_EVENT = 8&
    RIP_EVENT = 9&
End Enum

EXCEPTION_DEBUG_EVENT

This debug event is thrown whenever an exception occurs in the application being debugged. For example, if there was code in that application that was attempting to divide by zero then you would get an EXCEPTION_DEBUG_EVENT. The buffer that is passed back for this event is:

Public Enum ExceptionCodes
    EXCEPTION_GUARD_PAGE_VIOLATION = &H80000001
    EXCEPTION_DATATYPE_MISALIGNMENT = &H80000002
    EXCEPTION_BREAKPOINT = &H80000003
    EXCEPTION_SINGLE_STEP = &H80000004
    EXCEPTION_ACCESS_VIOLATION = &HC0000005
    EXCEPTION_IN_PAGE_ERROR = &HC0000006
    EXCEPTION_INVALID_HANDLE = &HC0000008
    EXCEPTION_NO_MEMORY = &HC0000017
    EXCEPTION_ILLEGAL_INSTRUCTION = &HC000001D
    EXCEPTION_NONCONTINUABLE_EXCEPTION = &HC0000025
    EXCEPTION_INVALID_DISPOSITION = &HC0000026
    EXCEPTION_ARRAY_BOUNDS_EXCEEDED = &HC000008C
    EXCEPTION_FLOAT_DENORMAL_OPERAND = &HC000008D
    EXCEPTION_FLOAT_DIVIDE_BY_ZERO = &HC000008E
    EXCEPTION_FLOAT_INEXACT_RESULT = &HC000008F
    EXCEPTION_FLOAT_INVALID_OPERATION = &HC0000090
    EXCEPTION_FLOAT_OVERFLOW = &HC0000091
    EXCEPTION_FLOAT_STACK_CHECK = &HC0000092
    EXCEPTION_FLOAT_UNDERFLOW = &HC0000093
    EXCEPTION_INTEGER_DIVIDE_BY_ZERO = &HC0000094
    EXCEPTION_INTEGER_OVERFLOW = &HC0000095
    EXCEPTION_PRIVILEGED_INSTRUCTION = &HC0000096
    EXCEPTION_STACK_OVERFLOW = &HC00000FD
    EXCEPTION_CONTROL_C_EXIT = &HC000013A
End Enum

Public Enum ExceptionFlags
    EXCEPTION_CONTINUABLE = 0
    EXCEPTION_NONCONTINUABLE = 1   '\\ Noncontinuable exception
End Enum

Private Type DEBUG_EXCEPTION_DEBUG_INFO
    Header As DEBUG_EVENT_HEADER
    ExceptionCode                                        As ExceptionCodes
    ExceptionFlags                                       As ExceptionFlags
    pExceptionRecord                                     As Long
    ExceptionAddress                                     As Long
    NumberParameters                                     As Long
    ExceptionInformation(EXCEPTION_MAXIMUM_PARAMETERS)   As Long
    dwFirstChance As Long
End Type

The exception flags tell you if it is possible to resume from the exception or not.

CREATE_THREAD_DEBUG_EVENT

This event occurs whenever a new thread is created by the debugee application. The buffer that is passed in is:

Private Type DEBUG_CREATE_THREAD_DEBUG_INFO
    Header As DEBUG_EVENT_HEADER
    hThread As Long
    lpThreadLocalBase As Long
    lpStartAddress As Long
End Type

This gives you a thread handle (for thread control API calls) and the base address and start address of the thread in the debugee process which is useful for analyzing the memory of that application.

CREATE_PROCESS_DEBUG_EVENT

This event occurs when the process is created. The buffer passed in is:

Private Type DEBUG_CREATE_PROCESS_DEBUG_INFO
    Header As DEBUG_EVENT_HEADER
    hfile As Long
    hProcess As Long
    hThread As Long
    lpBaseOfImage As Long
    dwDebugInfoFileOffset As Long
    nDebugInfoSize As Long
    lpThreadLocalBase As Long
    lpStartAddress As Long
    lpImageName As Long
    fUnicode As Integer
End Type

You can use the file handle passed in as part of this buffer to find the different parts of the process (imports section, exports, debug information, etc.) as per this article.

EXIT_THREAD_DEBUG_EVENT

This event occurs when a thread exits. The buffer passed in is:

Private Type DEBUG_EXIT_THREAD_DEBUG_INFO
    Header As DEBUG_EVENT_HEADER
    dwExitCode As Long
End Type

The exit code is whatever the thread set it to but is usually set to be non zero if an error caused the thread exit.

EXIT_PROCESS_DEBUG_EVENT

This event occurs when the process exits. The buffer passed in is:

Private Type DEBUG_EXIT_PROCESS_DEBUG_INFO
    Header As DEBUG_EVENT_HEADER
    dwExitCode As Long
End Type

The exit code is whatever the process set it to but is usually set to be non zero if an error caused the thread exit. You should stop the debug loop after you receive this event.

LOAD_DLL_DEBUG_EVENT

This event occurs when the application being debugged loads a dynamic link library. The buffer passed in is:

Private Type DEBUG_LOAD_DLL_DEBUG_INFO
    Header As DEBUG_EVENT_HEADER
    hfile As Long
    lpBaseOfDll As Long
    dwDebugInfoFileOffset As Long
    nDebugInfoSize As Long
    lpImageName As Long
    fUnicode As Integer
End Type

You can use the file handle passed in as part of this buffer to find the different parts of the DLL (imports section, exports, debug information, etc.) as per this article.

UNLOAD_DLL_DEBUG_EVENT

This event occurs when the process being debugged unloads a DLL it had loaded. The buffer passed in is:

Private Type DEBUG_UNLOAD_DLL_DEBUG_INFO
    Header As DEBUG_EVENT_HEADER
    lpBaseOfDll As Long
End Type

And you can use the lpBaseOfDll value to identify which DLL was unloaded.

OUTPUT_DEBUG_STRING_EVENT

This event occurs when the debugee calls the API call OutputDebugString to send debugging information to a debugger (where one is attached). The buffer passed in is:

Private Type DEBUG_OUTPUT_DEBUG_STRING_INFO
    Header As DEBUG_EVENT_HEADER
    lpDebugStringData As Long
    fUnicode As Integer
    nDebugStringLength As Integer
End Type

And you can read the string from the debugee using the ReadProcessMemory API call.

RIP_EVENT

This occurs if your process being debugged dies unexpectedly. The buffer passed in is:

Private Type DEBUG_RIP_INFO
    Header As DEBUG_EVENT_HEADER
    dwError As Long
    dwType As Long
End Type

Resuming the Debugee

Once you have extracted the information you need form the debug event, you need to resume the debugee so that it can continue running. To do this, you call the ContinueDebugEvent API call:

Public Enum DebugStates
    DBG_CONTINUE = &H10002
    DBG_TERMINATE_THREAD = &H40010003
    DBG_TERMINATE_PROCESS = &H40010004
    DBG_CONTROL_C = &H40010005
    DBG_CONTROL_BREAK = &H40010008
    DBG_EXCEPTION_NOT_HANDLED = &H80010001
End Enum

Private Declare Function ContinueDebugEvent Lib "kernel32" _
                                       (ByVal dwProcessId As Long, _
                                       ByVal dwThreadId As Long, _
                                       ByVal dwContinueStatus As DebugStates) As Long

Further Development

To expand on this framework and create a full debugger requires the ability to walk the memory and stack of the process being debugged and also to set breakpoints. I hope to get to this in the next article.

History

21^st October, 2003: Initial post

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here