Accessing Windows API with Plain x64 Assembly Language

• Download executable - 124.7 KB
• Download project - 346.1 KB

Introduction

This tool is a simple 64 bit file manager for Windows featuring a 100% assembly language source code. The tool shall be a sample for using advanced techniques in assembly language environment, like native 64 bit, object orientation and unwind ability for stack frames.

Wait, Assembly Language? Isn't It Dead Already?

No, it's not. While programming languages are getting more and more high level, assembly language is still the base of each and every modern device, smart phone, tablet, desktop or server. If you understand assembly language, each other language will be just a new set of syntax elements and a couple of interesting new concepts.

There is probably no useful reason to do a large project in assembly, but every programmer should at least have an idea of what is going on under the hood.

How Will Assembly Language Be Able to Use Modern Techniques as OOP?

Most of the common programming paradigms are not a property of the language, but a way how to write source code. The language can give you support to avoid mistakes and make writing easily, however you can still do it on your own.

This project features a fully object orientated style, multi threaded working and all the Windows features for x64 programs.

How to Work with Assembly Language?

Visual Studio 2013 contains the MASM (Microsoft Macro Assembler) which will be able to assemble the source files out of the box. If you encounter any problems with the project, please don't hesitate to contact.

For better layout of the ASM sources, we use a VStudio plug in for formatting the source code. This will become a sub project in near future (hopefully). Currently, the plug in is not worth a public release.

Using the Code

Some interesting code snippets:

//
// x64 procedure stack frame
//

actSetProgress	PROC	FRAME
		LOCAL	hwndList:QWORD	; progress list
		LOCAL	xItem:LVITEM	; new list item
		LOCAL	txBuffer [128]:WORD	; command name
		LOCAL	xRange:PBRANGE	; progress bar range

		push		rbp
		.pushreg	rbp
		push    	r12
		.pushreg	r12
		push		r15
		.pushreg	r15
		mov		rbp, rsp
		.setframe	rbp, 0

		sub		rsp, 8 + 2 * 128 + sizeof LVITEM + sizeof PBRANGE
		sub		rsp, 48
		.allocstack	48 + 8 + 2 * 128 + sizeof LVITEM + sizeof PBRANGE
		.endprolog

aspDone:	mov		rax, 0

		add		rsp, 48
		add		rsp, 8 + 2 * 128 + sizeof LVITEM + sizeof PBRANGE

		pop		r15
		pop		r12
		pop		rbp
		ret		0

		align		4

Procedure starts with PROC FRAME, since MASM cannot declare parameters in x64 mode nothing follows.

Afterwards local variables are declared, with appropriate data types. Used registers should be pushed and popped manually and declared to be so with .pushreg keyword. Space for variables must be allocated manually too and also space for any call usage inside this procedure (sub rsp, 48 in this example). Stack usage is declared with .allocstack. Since all calls are register based, stack must be corrected on return and use ret 0. Finally, the header end with .endprolog.

You must care about alignment of stack to be 8 bytes. Code should be aligned on 4.

Calling a register based function:

//
// register based call with more than 4 parameters
// first four paramters in registers, others on stack
// but keep space for first four parameters anyway
//

	lea	rax, [r15.VIEW_PARAM.txFontName]
	mov	[rsp + 104], rax
	mov	dword ptr [rsp + 96], DEFAULT_PITCH OR FF_DONTCARE
	mov	dword ptr [rsp + 88], DEFAULT_QUALITY
	mov	dword ptr [rsp + 80], CLIP_DEFAULT_PRECIS
	mov	dword ptr [rsp + 72], OUT_DEFAULT_PRECIS
	mov	dword ptr [rsp + 64], DEFAULT_CHARSET
	mov	dword ptr [rsp + 56], FALSE
	mov	dword ptr [rsp + 48], FALSE
	mov	eax, dword ptr [r15.VIEW_PARAM.unItalic]
	mov	dword ptr [rsp + 40], eax
	mov	eax, dword ptr [r15.VIEW_PARAM.unWeight]
	mov	dword ptr [rsp + 32], eax
	mov	r9d, 0
	mov	r8d, 0
	mov	edx, 0
	mov	ecx, dword ptr [r15.VIEW_PARAM.unFontSize]
	call	CreateFont

Registers rcx, rdx, r8 and r9 hold first four parameters. If more are required, they are placed on the stack. Keep space for first 64 bit parameters, as the procedure can write parameters back on stack if required.

Doing Basic Object Orientation

Each object has to be a small allocated memory block. It contains a list of the methods it defines (vtable), which will be always the first member in block. If you share this table between objects, it represents the difference between class and object. If you change methods in list, it be some kind of overload or inheritance.

Generate a new object via handmade new operation:

buttonNew	PROC	FRAME

	...

	; -- allocate and set vtable --

	call		GetProcessHeap
	test		rax, rax
	jz		btnExit

	mov		r8, sizeof CLASS_BUTTON
	mov		edx, HEAP_ZERO_MEMORY
	mov		rcx, rax
	call		HeapAlloc
	test		rax, rax
	jz		btnExit

	lea		rcx, [rax.CLASS_BUTTON.xInterface]
	mov		[rax.CLASS_BUTTON.vtableThis], rcx

	; -- fill in method pointers --

	lea		rdx, btnInit
	mov		[rcx.CLASS_BUTTON_IFACE.pfnInit], rdx
	lea		rdx, btnLoadConfig
	mov		[rcx.CLASS_BUTTON_IFACE.pfnLoadConfig], rdx
	lea		rdx, btnSaveConfig
	mov		[rcx.CLASS_BUTTON_IFACE.pfnSaveConfig], rdx
	lea		rdx, btnGetRect
	mov		[rcx.CLASS_BUTTON_IFACE.pfnGetRect], rdx
	lea		rdx, btnRender
	mov		[rcx.CLASS_BUTTON_IFACE.pfnRender], rdx

Objects size is allocated at process heap. The table with method pointers resists inside object data. All pointers are written to the table and the table pointer is filled into first position.

Sample class definition:

CLASS_BUTTON_IFACE	struc
 pfnInit		dq	?	; init button
 pfnLoadConfig		dq	?	; load configuration
 pfnSaveConfig		dq	?	; save configuration
 pfnGetRect		dq	?	; get buttons rect
 pfnRender		dq	?	; render single button rectangle
CLASS_BUTTON_IFACE	ends

CLASS_BUTTON		struc
 vtableThis		dq	?	; objects methods
 pxApp			dq	?	; parent object
 unCmd			dq	?	; command to execute
 txText			dw	32 dup(?)	; text on button
 txParam		dw	DEF_PATH_LENGTH dup(?)	; parameters for command
 xParams		VIEW_PARAM		; view parameters
 unShortcut		dq	?	; keyboard shortcut
 unCol			dq	?	; horizontal position
 unRow			dq	?	; vertical position
 txSection		dw	64 dup(?)	; button configuration section
 xInterface		CLASS_BUTTON_IFACE		; button interface
CLASS_BUTTON		ends

If the class is defined this way and kept properly aligned to 8 bytes, it will be fully compatible to Visual Studio C++ classes, so you can even interact with a C++ object or call assembly methods from C++.

Calling an object method (MS style, aka STDMETHODCALL):

//
// calling a method on an object
//
	lea	r9, txTitle
	mov	r8, [r12.CLASS_APP.unLang]
	mov	rdx, IDS_ABOUT_TITLE
	mov	rcx, r12
	mov	rax, [r12.CLASS_APP.vtableThis]
	call	[rax.CLASS_APP_IFACE.pfnLoadString]

Object has to be in rcx register, rax holds the vtable pointer, and the call goes to wanted method inside vtable.

History

• October 2016: First release