Introduction
This paper describes how to stop a run-away process. There are a few common reasons to do this: you have a bug in a device driver that can’t be shutdown in a normal way, to disable malware long enough to update and run anti-virus tools, a third party application was so badly written it is malfunctioning and it won’t close from the task manager.
Under Windows, a process has a message pump and it can receive commands. When it is well behaved, the application will close if it receives certain commands. In extreme cases, it may not respond or in the case of malware been designed to ignore these commands. A process will end when it has no more threads left, and that is a somewhat extreme way to close the process.
This article explores some of the methods known to the author, there are likely more but this should serve as a good introduction to the topic.
Risk Reward
Processes run for a reason. Stopping them abruptly could cause a disk sector to be corrupted or the PC to behave strangely or lock up. When the PC is already behaving strangely due to malware or badly written software, there is a sliding scale of which methods are appropriate ranging from politely asking the application to stop doing things that are done in extreme circumstances.
Shutdown Methods
CloseMainWindow
The System.Diagnostics.Process object has a CloseMainWindow() method that sends a windows manager close message to the applications message pump. This is the … polite way of asking an application to close. The application may or may not have been written to automatically close, or may jam while attempting to close, or ignore the message. The application may not have a MainWindowHandle yet, there is a time during an application creation that it is running but hasn’t asked for a window handle yet. Some processes without a UI might not have a window handle.
Kill
Using the System.Diagnostics.Process Kill() function is akin to using the TaskManager to end a process. It is the only way to end a process that doesn’t have a user interface. And if the process is accessing data, the process might not have a chance to clean up before closing. Killing a process may cause a loss of data. For example, killing a malware in the process of rewriting a key system DLL may cause the DLL to be corrupt, or it could keep the malware from deleting all of your files. Kill will not work if the process has turned itself into a system critical process (there are at least two ways it could do that with either RtlSetProcessIsCritical or NtSetInformationProcess in ntdll.dll).
Terminate Process
This is likely more extreme than Kill, MSDN describes that it might leave shared DLL’s with global data in an intermediate state because the process ends abruptly. It’s described as halting threads and requesting all I/O cancelled. In theory, unless the process has made blocking kernel calls that don’t cancel the process will stop. There are a few cases (API blocking on a kernel call that won’t return) where this call won’t work.
SuspendThread
This call doesn’t actually stop a process, but it will turn it rather safely into a process that can’t execute that can then be killed. The System.Diagnostics.Process API can be used to get the thread IDs which can be used to get a thread handle (with appropriate permissions) and then be used to suspend the thread. The advantage to this technique is the process is alive but rendered harmless. In the case of Malware, where multiple processes monitor each other and relaunch as needed, this may be enough to disable the Malware, and then either attempt to kill and remove or update anti-virus protection.
Priority and Affinity
Every processor has a priority associated with it, and an affinity for processor core. As it relates to the topic at hand, this means a process might be set to run very fast, all the time on all processing cores, and it would be easier to disable / kill if it was asked to go to a single core and run at a very low priority. The process priority class is a number, and the boost value + thread value + OS settings determines the next priority value used when scheduling the process. The affinity number is a bitmask of which processors may be used (1 sets it to the first core). This is great for buggy applications that consume so much CPU resources, they stop responding to user input, and for nudging malware to a state it can be removed. The demonstration code has an option to attempt to set these things before attempting to kill the process. A malicious application may still get around this by always setting its priority and affinity in a critical non-sleeping thread loop.
System Critical (Undocumented)
The OS keeps track of which processes are “critical” and which ones are not, killing a “critical” process may either be impossible or “blue screen” the PC. To make the process non-critical, there are two techniques: NtSetInformationProcess, and RtlSetProcessIsCritical. Both are ntDLL.dll calls that are undocumented / unsupported. There is some information out there but not much. The demonstration code attempts to use NtSetInformationProcess to remove the critical flag of an application.
TerminateThread
This is a call that kills a thread in an unsafe manner. By killing all threads of a process, the process will exit once all pending IO complete. Because the threads die at a very low unclean level, using this call is a last resort. In the authors' experience, it is a bad, very very very bad idea to use this call. It has some unpredictable affects. The process may enter such an unstable state that the OS cannot schedule the process to run but also be unable to close it. It can cause pending I/O to behave erratic, and it can also “blue screen” the PC. It was in a DLL the author was forced to debug that killed runaway USB driver reads, and occasionally crashed / locked up the PC, became unstoppable, and blue screened. After rebooting, the PC was fine.
With disk encryption, root kits, registry modification, viruses in disk forks, altering administrator’s privileges after taking control, etc., sometimes the choice is to use this call, or not use the PC for a few months till an anti-virus / anti-malware vendor provides a solution.
That is the only reason why the call was provided in the demo and the only case the author can see the call as reasonable.
SW Architecture of the Demonstration Application
The software takes the approach of a scatter gun aimed at a process or processes. Often, what doesn’t work the first time may work after a different technique has been tried and the OS has attempted to reschedule the process. Some bad processes monitor each other and might only be stopped after other processes have been killed in short order. The software runs in a loop making several attempts at each process per iteration. Certain key things were done.
Try…Catch
Every major call is wrapped, most of the calls that affect a process or thread can throw exceptions and the application goal is to keep moving, and try a different technique.
Permute
Every technique is tried but the starting point in the order is varied per iteration, this gives each technique a fresh start at a fresh list of processes.
By Process Name or Module Full Name
Some processes have a unique name, others may happen to have the same name as a needed process. Some processes have paths that can be typed in, others may not (resource paths, very long paths, paths with strange characters). The application can be run to handle either case.
Simple
This is a simple demonstration application, the UI is simplistic but functional. It is sufficient but by no means artistic nor provide all the finesse needed for all theoretical situations. It is likely that if more functionality is needed other techniques or specialized code involving the registry, code injection, etc. is also needed to deal with a specific malware threat or a highly unusual application.
