Secure Function Pointer and Callbacks in Windows Programming

• Download Sample 1: C function pointer and callback functions source - 7.64 KB
• Download Sample 2: C++ Static and Non-Static function pointer source - 7.06 KB
• Download Sample 3: Dynamic linked Library (DLL) function pointer source - 8.87 KB
• Download Sample 4: Dynamic linked Library function pointer Client source - 6.86 KB
• Download Sample 5: Unsafe function pointer Sample source - 7.43 KB
• Download Sample 6: Secure function pointer Sample source - 7.08 KB

Introduction

Function pointer is a pointer to function like pointer to variable. The user can call the function pointer after dereference with required parameters. The function pointer can be implemented using C or C++ language. Function pointer is useful for late binding, implements callbacks and passes a function as an argument to another function. This article explains the usage of function pointer and callback in Windows application programming Interface (API).

C++ supports the following two types of function pointers:

• Static member function pointers
• Non-static member functions pointers

C supports functions pointer like C++ Static member function. The application can define the variable name for address of the function and call using that variable name.

// define a function pointer and initialize to NULL
void (*FunctionPointer)(int, int, int) = NULL;                        // C

The C function pointer can declare with variable arguments for unknown number of arguments.

int (*ptr)(...);

The pointer to static member function in C++ is the same as pointer to c functions.

// define a function pointer and initialize to NULL
int (TMyClass::*pt2Member)(float, char, char) = NULL;                // C++
int (TMyClass::*pt2ConstMember)(float, char, char) const = NULL;     // C++

The developer can use typedef for declare function pointer. The typedef definition can be used for Initializing Function Pointers and call the function using function pointer.

// declare typedef for array of integer and integer arguments
typedef void (*FunctionPointer)(int arr[], int);

//Define the function pointer
FunctionPointer fpdisplay = NULL;

// Assign Display function address to function pointer and call the function
fpdisplay = &Display;
	if(fpdisplay)
		(*fpdisplay)(arr,9);

The Don Clugston’s Member Function Pointers and the Fastest Possible C++ Delegates article explain more depth in C++ member function pointer.

Calling Convention

Calling convention is used for passing the argument to function and cleaning the stack from memory. The functions pointer declaration should match with actual function calling convention. Microsoft C/C++ compiler supports the __cdecl, __clrcall, __stdcall, __fastcall and __thiscall calling convention. The __stdcall calling convention is used to call Win32 API functions. The __cdecl is default calling convention for C and C++ programs. The __cdecl calling convention creates larger executable than __stdcall. It requires each function to include clean up code.

The WinDef.h defines the calling convention macros. The CALLBACK, WINAPI, APIPRIVATE and PASCAL are __stdcall calling convention. The function and function pointer should use the same calling convention. If the developer doesn’t mention explicitly, the Microsoft C/C++ compiler takes __decl calling convention. But, if the developers use any win32 API, they should use __stdcall calling convention.

Callbacks

The callback function is a function pointer that is passed to another function and second function can call function through argument function pointer. The second function call directly to function pointer without knowing about the argument function. Every callback function has its own prototype. Because, the function name, return type and arguments are different for each function. All the C++ member functions have internal this pointer for access members of the class. So, the developer has no need to use this pointer for making use of callback function in C++. But, when we use Non-Member function and static function, we must use call back function explicitly. The reason these two would seem the same is because C does not have references, so you cannot send a reference.

Difference between Function Pointer and Callback Functions

No	Function Pointer	Callback
1	Function pointer is a pointer which points to the function.	Callback functions are function pointers passed as parameters of function, which can be called back when an event occurs.
2	Example vtable is array of function pointers	Callback function to be called when thread ends.
3	Function pointer is an address of function	Callback functions are passed function pointer as an argument and caller would callback if something happens.

Sample 1 explains C function pointer and callback functions. The BubbleSort sorts the array of elements and Display displays the array of integers. The BinarySearch and LinearSearch are two callback functions which are used for searching the element. The Binary search will work with only sorted elements and takes less time (better performance) compared to Linear search. The Linear search takes with sorted or unsorted array of elements.

/****************************************************************
 The calling function takes a single callback as a parameter.
*******************************************************************/
void Search(int (*SelectSearch)(int arr[], int, int,int),int arr[],
	int value,int left,int right) {

	int result = SelectSearch(arr,value,left,right);
	if(result == -1)
		printf("Search element not found!\n");
	else
		printf("Element %d found in %d position!\n", value,result);

}

The above function uses function pointer as an argument and uses function pointer for invoking the appropriate search function. The main function calls BinarySearch after sorting the unsorted input elements.

Secured CRT

The “Secured string handling” explains the basics about secured C Run time library string handling function. The secured CRT also provides a set of secured algorithm functions. The C run time algorithm uses callback functions for implementing compares elements.

No	function	Secured Version	Description
1	bsearch	bsearch_s	Binary search
2	_lfind	_lfind_s	Linear search
3	_lsearch	_lsearch_s	Linear search which is added to array if not found
4	qsort	qsort_s	Quick sort

The above sorting and searching functions compare function pointer will take three arguments. This C Run-Time Library functions use __cdecl calling convention. The Comparison function will take two parameters. The first parameter is pointer to the key for the search and the second parameter is a pointer to the array element to be compared with the key.

The secure version Comparison function will take three arguments. The argument compare is a pointer to a user-supplied routine that compares two array elements and returns a value specifying their relationship. The secured version compare will take Context pointer too. The context parameter makes it easier to perform thread-safe sorts. Instead of using static variables that must be synchronized to ensure thread safety, pass a different context parameter in each sort. The compare function may invoke one or more times during quick sort. MSDN has an example for the above secured version of CRT algorithms.

Visual Studio 2008 C Run time library run on Windows 2000, Windows XP, Server 2003 and Vista. Visual Studio 2010 C Run time library supports Windows XP with SP2, SP3, and Windows Server 2003 with SP1, Server 2003 with SP2, Vista, Windows Server 2008 and Windows 7. The Secured C Run time library comes along with Visual Studio. If the developer wants C Runtime library, they can download from Microsoft websites [12,13].

List of Function Pointer and Callback Function in Windowing

Windows application programming Interface (API) uses many places function pointer and callback functions. (This article discusses User Interface application Windows API only. Windows may use function pointer and callback function in other windows programming API too.) Every Windows application uses WNDCLASS/WNDCLASSEX structure for registering the Windows application using RegisterClass/RegisterClassEx Windows API. It contains window class information. The callback function registered as part of windows registration process. WNDCLASS/ WNDCLASSEX structure registers the Windows callback function WndProc in lpfnWndProc parameter. The CreateWindow function creates a window which specifies the window class (WNDCLASS/EX), window title, window style, and size of the window.

ATOM MyRegisterClass(HINSTANCE hInstance)
{
	WNDCLASSEX wcex;
	wcex.cbSize = sizeof(WNDCLASSEX);

	wcex.style			= CS_HREDRAW | CS_VREDRAW;
	wcex.lpfnWndProc	= WndProc;
	wcex.cbClsExtra		= 0;
	wcex.cbWndExtra		= 0;
	wcex.hInstance		= hInstance;
	wcex.hIcon		= LoadIcon(hInstance, 
|				MAKEINTRESOURCE(IDI_SIMPLEWIN32));
	wcex.hCursor		= LoadCursor(NULL, IDC_ARROW);
	wcex.hbrBackground	= (HBRUSH)(COLOR_WINDOW+1);
	wcex.lpszMenuName	= MAKEINTRESOURCE(IDC_SIMPLEWIN32);
	wcex.lpszClassName	= szWindowClass;
	wcex.hIconSm		= LoadIcon(wcex.hInstance, 
				MAKEINTRESOURCE(IDI_SMALL));

	return RegisterClassEx(&wcex);
}

The WindProc is the function used in WNDCLASS structure. The WndProc is called for all windows messages and handles all the messages. If it doesn’t find any message in WndProc, it calls the default window procedure. The DefWindowProc function calls the default window procedure to provide default processing for any window messages that an application does not process.

LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)

The GetClassLong function is used to retrieve WNDCLASS information for a window class and The SetClassLong function replaces the specified 32-bit (long) value at the specified offset into the extra class memory or the WNDCLASSEX structure for the class to which the specified window belongs. The S/GetClassLong function nIndex GCL_WNDPROC replaces the address of the window procedure associated with the class.

The following Windows API functions use callback and function pointer. The following table discusses only the function and purpose for the function.

DialogBox	DialogProc	Application-defined callback function uses processes messages sent to a modal or modeless dialog box.
Messages and Message Queues	SendAsyncProc	Application-defined callback function used with the SendMessageCallback function.
	SendMessageCallback	Send the specified message to a window or windows.
Timer	TimerProc	Application-defined callback function that processes WM_TIMER messages.
Window Classes	WNDCLASS/EX	lpfnWndProc points to the window procedure for handling all the windows messages.
Window Procedures	WindowProc	Application-defined function that processes messages sent to a window. WindowProc is a placeholder for the application-defined function name.
	CallWindowProc	The CallWindowProc function passes message information to the specified window procedure.
Window Properties	EnumProps	Enumerates all entries in the property list of a window by passing them, one by one, to the specified callback function.
	EnumPropsEx	Enumerates all entries in the property list of a window by passing them, one by one, to the specified callback function.
	PropEnumProc	Application-defined callback function used with the EnumProps function.
	PropEnumProcEx	Application-defined callback function used with the EnumPropsEx function.
Windows	EnumChildProc	Application-defined callback function used with the EnumChildWindows function.It receives the child window handles and it is a placeholder for the application-defined function name.
	EnumChildWindows	Enumerates the child windows that belong to the specified parent window by passing the handle to each child window, in turn, to an application-defined callback function.
	EnumThreadWindows	Enumerates all nonchild windows associated with a thread by passing the handle to each window, in turn, to an application-defined callback function.
	EnumThreadWndProc	Application-defined callback function used with the EnumThreadWindows function. It receives the window handles associated with a thread.
	EnumWindows	Enumerates all top-level windows on the screen by passing the handle to each window, in turn, to an application-defined callback function.
	EnumWindowsProc	Application-defined callback function used with the EnumWindows or EnumDesktopWindows function. It receives top-level window handles.

Dynamic Linked Libraries

Dynamic linked library is a windows executable file which contains a set of functions. DLL loads only once in memory and process can refer for executing the functions available in DLL. If no process refers DLL, it unloads from memory. DLL can be loaded implicitly or explicitly. The explicit DLL uses function pointer for loading the functions in memory. The implicit linking uses the library (.lib) file and loads the DLL when the executable starts loads in the memory. The explicit linking uses the DLL directly and loads the DLL when necessary using LoadLibrary (Ex) family functions.

The explicit linking has the following advantages over implicit linking:

• Explicit linking is useful when the application does not know the name of a DLL that it will have to load until run time.
• Implicit linking requires .lib, .h and .dll files for loading the DLL. But, explicit linking requires only DLL file.
• The application fails when implicit linking DLL fails to load during startup. But, explicit linking loads the DLL using LoadLibrary and it can handle and continue.
• If the application contains many DLLs, the Implicit linking will take a lot of time for loading the application. But, implicit linking loads the DLL when necessary.

The processes call LoadLibrary/LoadLibraryEx for loading the DLL in memory. MFC application uses AfxLoadLibrary for loading the explicit linking DLL. If the MFC application uses AfxLoadLibrary function for loading the DLL, it should call AfxFreeLibrary for unloading an extension DLL. If the function is able to load the DLL successfully, it returns the HANDLE of the function. If the DLL is already loaded in memory, it increments the reference count and returns the HANDLE. If it is not able to find an entry point, it simply returns NULL. Once he/she gets the HANDLE of the DLL, the user can call GetProcAddress function for retrieving the address of an exported function and run the function or variable. It takes DLL module handle which returns from LoadLibrary and name of the function or function export ordinal.

The function pointer doesn’t have any compile time check. The typedef for function pointer is useful for check type-safety in function prototypes of the exported functions. If the DLL is built with a module definition (.def) file, the GetProcAddress can use ordinal number. It is slightly faster if the DLL contains many exported DLL functions.

typedef int (__cdecl *FUNCTIONPROCADD)(int,int);

// do
FUNCTIONPROCADD ProcAdd;
BOOL bResult, bSuccess = FALSE;

// Get a handle to the DLL module.
HINSTANCE hinstLib = LoadLibrary(TEXT("winadd.dll"));

// If the handle is valid, try to get the function address.
if (hinstLib != NULL)
{
    ProcAdd = (FUNCTIONPROCADD) GetProcAddress(hinstLib, "add");

    // If the function address is valid, call the function.
    if (NULL != ProcAdd)
    {
        bSuccess = TRUE;
        (ProcAdd) (3,6);
    }

    // Free the DLL module.
    bResult = FreeLibrary(hinstLib);
}

// If unable to call the DLL function, use an alternative.
if (!bSuccess)
    printf("Message printed from executable\n");	

Secured Pointer Encoding

We have discussed the function pointer and callback functions using function pointer as parameter. We are living in an insecure world. The buffer overrun vulnerability gives an option to the attacker to attack the code and change the normal flow. The attacker may even run some other application as part of our application function flow. When the program calls the function, it creates a new stack frame with function parameters, return address, Frame pointer, Exception Handler frame and local variables. The good coding practice recommends making use of a local variable or passing by reference instead of global variable. If it requires, make use of only immutable variables or pointers. The function pointer declares globally for access the functions. If the function pointer declares as global, the function pointer is alive till the application terminates.

Microsoft secured Lifecycle (5.0) recommends for encode long lived pointers. The insecure pointer leads to buffer overrun attacks. The global scope function pointer and global pointer are long lived pointers. The long lived pointers are insecure pointers. The developer or designer must identify long lived pointers. Microsoft Windows provides the EncodePointer() and DecodePointer() function for encrypt and decrypt pointers using a secret technique for the given process. If the attackers try to overwrite a pointer, the encoded pointer doesn’t allow for attacks. The EncodePointer()/DecodePointer() functions are available from Windows XP SP2 and above for client operating system and Windows Server 2003 SP1 for server Operating system. The EncodePointer()/DecodePointer() functions are windows functions and Microsoft uses those functions for internal to CRT and may use other APIs too for encoding the global pointers. Windows uses those functions for encode unhandled exception filter addresses, long lived function pointers, long running heap allocation memory, kernel to perform indirect calls, etc.

The encode function does XOR operation with random number. The EncodePointer/DecodePointer function use application’s process information block and EncodeSystemPointer/DecodeSystemPointer uses value for each system reboot. Michael Howard's Web Log explains about the algorithm for XOR encoding operation. The Michael Howard's[4, 5] Web Log explains more about algorithm uses for XOR Operations.

Assume, search is a function and FunctionPointer declared typedef for function pointer. It lives for a long time in memory and reclaims the memory when terminating the application. The search function may take a long time to execute and the function pointer is declared as global. The attacker may attack this function pointer using buffer overrun attacks. The attacker can change the return address and add to execute some shell code or some other application. The EncodePointer takes PVOID windows data type and returns the same pointer to Void (PVOID) data type. Once the pointer is encoded, the attacker cannot call or change the global function pointer address. The developer can encode and decode the function pointer like the following syntax:

// search function
void Search() {
  // do something
}

typedef void (*FunctionPointer )(void);

// do something

FunctionPointer fp1 = (FunctionPointer)EncodePointer(&Search);
  // Do something
  // Do something
  FunctionPointer fp2 = (FunctionPointer)DecodePointer(fp2);

  if (fp2)
     (*fp2)();

The EncodePointer returns encoded pointer value. The attacker can’t attack the encoded pointer. But, the developer can decode the pointer using DecodePointer function with an encoded pointer. Michael Howard[4, 5] doesn’t recommend for use all global function pointers because it may affects the application performance.

The encode pointer used in crt\src\tidtable.c CRT file. This files use the OS mechanism for encoding pointers to prevent hijacking. The EncodePointer/ DecodePointer supports only for X86 machine. The tidtable.c gives _encode_pointer/_decode_pointer function for encode and decode CRT functions. But, those functions are undocumented and don’t export by CRT DLLs. The following _encode_pointer/_decode_pointer are defined in internal.h for internal helper functions for CRT.

/* internal helper functions for encoding and decoding pointers */
void __cdecl _init_pointers();
_CRTIMP void * __cdecl _encode_pointer(void *);
_CRTIMP void * __cdecl _encoded_null();
_CRTIMP void * __cdecl _decode_pointer(void *);

Examples

• Pointers internal to NT Heap
• Vectored exception handler pointers
• Long lived function pointers

Conclusion

The function pointer is an important concept in C and C++ programming. Windows application programming interface (Windows API) uses function pointer and callback function noisily. So, a Windows programmer should be aware of function pointer and callback function whether using or not using it in programming. The Microsoft Secured development Lifecycle (SDL) also recommends and provides EncodePointer/ EncodeSystemPointer and DecodePointer/ DecodeSystemPointer function for secure pointer operations. The pointer might be a function pointer or variable pointer. The insecure pointer leads to buffer overrun attacks. The C/C++ doesn’t support array bounds checking when initializing or moving data between functions. The buffer overrun attacks can override the function return address. When the function returns instead of jump back to calle, it jumps to attacker code. The stack overflow can use for change program flow or gain the privileges for operating system environment. The heap overrun involves for heap memory allocation functions which allocated memory by application.

Summary

1. Assign NULL for delete objects. It avoids "dangling" pointers.
2. Encode and Decode long lived static or global pointers. The encode pointer uses different one for each process. So, the attacker can’t predict the encode logic for buffer overrun attacks.
   1. Windows messages and run time dynamic-link libraries use function pointer heavily. So, use Encode and Decode pointer for secure pointer access.
   2. Load dynamic-link libraries (DLLs) must load DLLs and execute the function securely.
3. If application uses STL, check safe functions availability.
4. Follow Microsoft Secured development life cycle (SDL) 5.0 guidelines

Points of Interest

1. Microsoft Security Development Life Cycle
2. MSDN Visual C++ 2008
3. MSDN Magazine
4. Protecting against Pointer Subterfuge (Kinda!)
5. Protecting against Pointer Subterfuge (Redux)
6. Defeating compiler level buffer overflow protection
7. Microsoft Security Development Lifecycle (SDL) – Process Guidance
8. Function pointer tutorial
9. MSDN
10. Windows ISV Software Security Defenses
11. Member Function Pointers and the Fastest Possible C++ Delegates
12. Microsoft Visual C++ 2008 Redistributable Package (x86)
13. Microsoft Visual C++ 2010 Redistributable Package (x86)
14. How to obtain the Visual C++ 6.0 run-time components

History

• 1^st May, 2011: Initial version