Introduction Into Windows Anti-Debugging

Introduction

In recent times, tools for use in reverse engineering have flourished. There are plenty of resource sites for those who are interested in the field, and the field is very much worth the time invested in it. I found that learning C++ while introducing myself to reverse engineering and assembly language really helped me to understand how code works, and improved my C/C++ coding and my ASM coding at the same time. However, reverse engineering also has a darker side. Crackers are individuals who use their knowledge of reverse engineering to reverse another programmer's code, often to decode how a serial is processed or to remove a protection from a trial. Naturally, a pioneer will want to protect their investment; this can be done with tools such as Themida, Execryptor, Armadillo, and even a protection system coded by a CodeProject resident Jim Charles named Eagle Protector. This article is meant to inform individuals of some anti-debugging techniques, and is not meant to be all-inclusive, nor does it explore some of the more complex routines that commercial protectors use.

Background

An individual reading this should have a solid understanding of ASM, how computers handle memory, the Win32 Debugging API, and at least some knowledge of Windows internals. This code most likely will not work on any *nix platform due to the fundamental differences of the Operating Systems. Any other knowledge in the field of reverse engineering is also a plus. One great thing about learning and implementing anti-debugging is that you also develop your reversing skills, which is a great plus to anyone interested in the field. Along with the other mentioned subjects, an interested reader should also be familiar with the tools used for binary application reversing such as OllyDBG, WinDBG, SoftICE, IDA Pro, and others. Here are some links to some information that is important for readers to be familiar with before reading the following text:

• Windows Debugging API
• Assembly Language
• Computer Memory

IsDebuggerPresent

This is probably the simplest method for debugger protection. This function is part of the Win32 Debugging API, and its reference page can be found here: MSDN.

if(IsDebuggerPresent())
{
    MessageBox(NULL, TEXT("Please close your debugging application" + 
               " and restart the program"), 
               TEXT("Debugger Found!"), 0);
    ExitProcess(0);
}
// Normal Code Here....

IsDebuggerPresent Using the PEB

The IsDebuggerPresent function is actually a wrapper around this code. It directly access the PEB for the process and reads a byte value that signifies if the process is being debugged.

char IsDbgPresent = 0;
__asm {
     mov eax, fs:[30h]
     mov al, [eax + 2h]
     mov IsDbgPresent, al
}

if(IsDbgPresent)
{
    MessageBox(NULL, TEXT("Please close your debugging " + 
               "application and restart the program"), 
               TEXT("Debugger Found!"), 0);
    ExitProcess(0);
}
// Normal Execution

CheckRemoteDebuggerPresent

This is another Win32 Debugging API function; it can be used to check if a remote process is being debugged. However, we can also use this as another method for checking if our own process is being debugged. For those of you who wonder how this function differs, CheckRemoteDebuggerPresent calls the NTDLL export NtQueryInformationProcess with the SYSTEM_INFORMATION_CLASS set to 7 (ProcessDebugPort). The MSDN reference for the function can be found here: MSDN.

BOOL IsDbgPresent = FALSE;
CheckRemoteDebuggerPresent(GetCurrentProcess(), &IsDbgPresent);
if(IsDbgPresent)
{
        MessageBox(NULL, TEXT("Please close your debugging" + 
                   " application and restart the program"), 
                   TEXT("Debugger Found!"), 0);
        ExitProcess(0);
}
// Normal Execution

NtQueryInformationProcess

Instead of calling CheckRemoteDebuggerPresent an individual could also make the call to NtQueryInformationProcess process theirself. MSDN does not encourage the use of any NtxXx function because of the possibility of its behavoir changing, therefore some consideration may want to be made about using the function before committing to using it. The MSDN reference for this function can be found here: MSDN.

// Function Pointer Typedef for NtQueryInformationProcess
typedef unsigned long (__stdcall *pfnNtQueryInformationProcess)(IN  HANDLE, 
        IN  unsigned int, OUT PVOID, IN ULONG, OUT PULONG);

 
// ProcessDebugPort
const int ProcessDbgPort = 7;
 
// We have to import the function
pfnNtQueryInformationProcess NtQueryInfoProcess = NULL;
 
// Other Vars
unsigned long Ret;
unsigned long IsRemotePresent = 0;
 
HMODULE hNtDll = LoadLibrary(TEXT("ntdll.dll"));
if(hNtDll == NULL)
{
    // Handle however.. chances of this failing
    // is essentially 0 however since
    // ntdll.dll is a vital system resource
}
 
NtQueryInfoProcess = (pfnNtQueryInformationProcess)
   GetProcAddress(hNtDll, "NtQueryInformationProcess");
if(NtQueryInfoProcess == NULL)
{
    // Handle however it fits your needs but as before,
    // if this is missing there are some SERIOUS issues with the OS
}
 
// Time to finally make the call
Ret = NtQueryInfoProcess(GetCurrentProcess(), ProcessDbgPort, 
      &IsRemotePresent, sizeof(unsigned long), NULL);
if(Ret == 0x00000000 && IsRemotePresent != 0)
{
    // Debugger is present
    MessageBox(NULL, TEXT("Please close your debugging " + 
               "application and restart the program"), 
               TEXT("Debugger Found!"), 0);
    ExitProcess(0);
}

NtGlobalFlag

NtGlobalFlag is a DWORD value inside the process PEB. This value contains many flags set by the Operating System that affects the way the process runs. When a process is being debugged, the flags FLG_HEAP_ENABLE_TAIL_CHECK (0x10), FLG_HEAP_ENABLE_FREE_CHECK(0x20), and FLG_HEAP_VALIDATE_PARAMETERS(0x40) are set for the process, and we can use this to our advantage to identify if our process is being debugged.

unsigned long NtGlobalFlags = 0;

__asm {

    mov eax, fs:[30h]
    mov eax, [eax + 68h]
    mov NtGlobalFlags, eax
}


if(NtGlobalFlags & 0x70)
// 0x70 =  FLG_HEAP_ENABLE_TAIL_CHECK |
//         FLG_HEAP_ENABLE_FREE_CHECK | 
//         FLG_HEAP_VALIDATE_PARAMETERS
{
    // Debugger is present
    MessageBox(NULL, TEXT("Please close your debugging " + 
               "application and restart the program"), 
               TEXT("Debugger Found!"), 0);
    ExitProcess(0);
}
// Normal execution

Points of Interest

There is a lot more to anti-debugging in Windows, several methods such as using CloseHandle or OutputDebugStringA are patched in Windows Vista, and therefore didn't make it into the article. I do plan to write other articles on other anti-debug techniques, many methods available are difficult to implement in inline assembler alone, and would need to be coded in another ASM flavor/compiler in order to be implemented more efficiently.

References

• www.securityfocus.com - Great place, and originally where I found many of these techniques.

History

• September 18, 2008 - Original revision.