A Handy Idle-Processing Class for Long Procedures

• Download source files - 13.1 KB

Introduction

This article will introduce a convenient class to handle situations where long procedures must be handled without causing your application to freeze. It is well suited when implementing a thread is not the most desired approach to the problem.

Background

In 1995, when trying to develop a simulator application for Windows, one of the hurdles we had to deal with was long processes. It was no problem creating methods to do the processing, but once you entered the loop, the main application thread no longer processes UI messages until the loop was complete and control was returned. If this was anything more than a few seconds, it quickly became a problem, and users would hopelessly tap on the keys or click the mouse in an attempt to abort. At the time, threads were not an option in the design, so another answer had to be found.

The Problem

When handling long processes, most often, they are the result of intensive looping, and often loops within loops. Once you enter the loop, you don't exit the loop until the entire process is finished. Sometimes, this process takes minutes or even hours to complete. During the whole time, no messages can be processed by the application and window buttons, controls, toolbars and keys become quite useless. To abort, you need to process the message from one of these, and that requires that at some time, the application must return to the message loop.

Why Not Threads?

1. Often, program code does not need to run in complete autonomous thread. Rather, what is needed is some means to simply 'defer' the execution until the application is less busy. This keeps the primary thread running crisp and responsive.
2. It is very often the case that code must run a certain section without intervention (like a CRITICAL section in a thread). Since tasks are running in the primary thread, there is no intervention and you do not need to define these locks.
3. If you need access to resources or need to paint, you can do it in a thread, but in MFC, it is a messy affair since you must attach and detach windows and resources to keep the internal tables in check. You do not need to do this with idle task object.

The Solution

In the early development work, we developed everything directly with the Windows API. This meant that we also had to create the message loop. Since we had to return to the message loop, the solution lay in the loop itself. Rather than creating a large outer loop to the process, the polling already provided by the message queue was used instead to provide outer support for the long procedure. All that was needed was to hook into it and perform a loop test to decide when we were finished. C++ application objects were written to manage the message loop and its loop task processing. Smaller objects called 'tasks' were created to hook into the loop whenever it was needed.

When MFC was implemented with our software in 2002, the main application class provided the same basic support in the virtual method called OnIdle(). This was nice as it meant the code originally created to do this could be removed and all that was now required was to introduce the hook code into the idle cycles instead and leave the MFC message loop undisturbed. This was particularly attractive because it meant that we could use virtually the same code when the application was recently ported to use MFC. In addition, since we did not resort to multiple message processing points, all the window messaging could remain under the control of MFC.

Even though an idle processing call is provided, it's not very convenient. There is no nice way to hook into it and detach from it. What is needed is a structure and a simple object to allow developers to create sets of code that can be added and removed in a convenient fashion. This way, blocks of execution can be 'packaged' up, processed with a clean interface that is not unlike a thread object. The class that was created for this purpose is called ZTask. Everything in our libraries is prefixed by a Z but you can change it to anything you like. This 'task' object forms the basis of this discussion.

To use the idle processing class, all that is required is a simple change to the derived CWinApp class (in this case called MyApp). Once all the tasks have indicated their completion, this method will return false and cause the application to sleep until the next message arrives. This is the only change required in the application to enable task processing.

//-< OVERRIDE >-------------------------------------------------------------
// OnIdle()
//
// When the message queue is empty, this is time to work on stuff in the
// background. These background activities, or Tasks are called to execute
// one cycle of the work and then return to the queue.
//-------------------------------------------------------------------------
BOOL        MyApp::OnIdle(LONG count)
   {
   // From the MSDN C/C++ docs on the logical OR (||) operation, this code is 
   // sensitive to "short-circuit evaluation" of the C/C++ logical AND (&&) 
   // and OR (||) operators. This means super::OnIdle() is guaranteed to 
   // execute first and the latter is only called if the first returns FALSE.

   return CWinApp::OnIdle(count) || ZTask::executeTasks();
   }

Once tasks are added to the application, through specific methods in the source code, the application then managed the objects automatically. The tasks perform a single pass for each idle call and continue until all the tasks are complete. All that is required is to create these objects to do your work and get them started by adding them to the application.

The Task Class

Now that we have support in the application, defining the work to be done was clean and easy. A tasking object called ZTask was created to package up individual tasks into nice bundles. The class has virtual methods that are called when it is first added, each time an idle call is made, and one when it is removed. The method for processing a single iteration of the task is called execute(). It must be overridden when in the derived task. The other two are optional. The execute() method will continue to be called in the idle loop until you return true. Once true is returned, the task object will be cleaned up and deleted if necessary. All you need to do is decide how much processing you want to perform in a single pass. This will affect the responsiveness of the application.

//----------------------------------
// Class methods
//----------------------------------
public:
   static   bool           addTask(ZTask *task);
   static   bool           removeTask(ZTask *task);
   static   bool           executeBlocking(ZTask *task);

Two response methods are provided that are called at the moment when the taskAdded/Removed() to/from the queue for processing. These methods are called immediately when that occurs and allows an opportunity to set up the long process. Anything that must be initialized is inserted in these methods so that it is ready to go once the execute method is called. The added/removed methods are called only once, so in a file read for example, this would typically be used first to open and verify the file to be read.

The static method addTask() is responsible for setting up the task and getting it ready for execution. This involves performing a sanity check to make sure we are running in the application thread and ensuring that the task is not a child (i.e., top level). By default, they are all top-level.

//==============================================================================
// Add/Remove registered tasks
//------------------------------------------------------------------------------
bool        ZTask::addTask(ZTask *task) 
   { 
   ASSERT(AfxGetThread() == AfxGetApp());   // Application thread context only!

   // If the parent is already executing in blocking mode, then
   // we must execute the child task in blocking mode as well.
   // Short circuit the method at this point by returning.
   //
   ZTask *parent = task->pParent;
   if (parent != nullptr && parent->bBlocking)
      {
      return ZTask::executeBlocking(task);
      }

   // Add the task to the main execution list, but do not add the task
   // if it is already there! By merging the task with the list it 
   // will return false if it is found and skip over.
   //
   ASSERT(parent == nullptr || 0 <= vTasks.indexOf(parent));   // Sanity check!
   if (vTasks.mergeElement(task))
      {
      // If a task has a parent, then by design the parent must wait
      // for the child task. If so interrupt the parent by incrementing
      // the parent lock count.
      //
      if (parent != nullptr)
         {
         parent->interrupt();
         parent->vChildren << task;
         }

      // Ensure that the tasking is not locked for add/remove. New tasks
      // should not be added while nested in another add/remove action.

      //ASSERT(!bAdding);       // Check the call stack!
      bAdding = true;

      task->bEscape = task->taskAdded();
      if (task->bEscape)
         TRACE("\n[%s] Bad initial state, terminating task."
               , task->getName());

      // If no tasks are present. the idle processing 
      // may be inactive. Post a dummy message to kick 
      // start an initial OnIdle() method call.

      CWnd *main_wnd = AfxGetMainWnd();
      if (main_wnd != nullptr && vTasks.size() == 0)
         main_wnd->PostMessage(WM_KICKIDLE);

      bAdding = false;
      return true;
      }

   return false;
   }

The task object itself has three virtual methods that it used to perform the task.

//----------------------------------
// Virtual methods
//----------------------------------
protected:
   virtual  bool           taskAdded();
   virtual  void           taskRemoved();
   virtual  bool           execute();

Each of these response methods have a default implementation that will attempt to run code from an object implementing a simple interface. At this point, let's ignore the runnable object and just consider creating a derived task. The virtual method will be called right at the point the task is added and removed. I look at the task as a 'sandwich' where the bun is the start/stop activity and the meat is the process execution.

//-< VIRTUAL >------------------------------------------------------------------
// taskAdded()/Removed()
//
// The task has been added to the execute list, no action by default. These
// methods are called prior to the first execution and after the last execution.
// If specific initial/final actions are required, they can be overridden.
//------------------------------------------------------------------------------
bool        ZTask::taskAdded()
   {
   // Some tasks may not have derived implementations, so this method would
   // not be overridden. In this case the default action is execute methods
   // from an attached object implementing IRunnable.
   //
   if (pRunnable != nullptr)
      pRunnable->initialize(false);

   return false;              // No errors.
   }

void        ZTask::taskRemoved()
   {
   // Some tasks may not have derived implementations, so this method would
   // not be overridden. In this case the default action is execute methods
   // from an attached object implementing IRunnable.
   //
   if (pRunnable != nullptr)
      pRunnable->finalize();
   }

Since tasks are executed in the idle processing of the primary thread of the application, it is not be predictive when the task will be done. If the task is created on the heap using new, then it must also be deleted. Like threads, the task object has a flag that can be set to tell it to delete itself automatically when the task is done, so you don't have to. When the task completes, it is placed on a recycle list where its deletion is deferred until the next idle cycle occurs. This is handled automatically by the static method executeTasks(), which as described above, is called inside the OnIdle(). All the action happens here.

//------------------------------------------------------------------------------
// executeTasks()
//
// When the message queue is empty, the is time to work on stuff in the
// background. These background activities, or Tasks are called to execute
// one cycle of the work and then return to the queue.
//------------------------------------------------------------------------------
bool        ZTask::executeTasks()
   {
   // If there is anything in the recycling bin, delete those
   // objects before proceeding with any other tasks.
   //
   int n = ZTask::vRecycle.size();
   while (n--)
      delete ZTask::vRecycle[n];
   vRecycle.removeAll();

   // If the base class idle work is done, then the task count
   // will have dropped to zero. If so, then there are no more
   // tasks and return. On return it will signal OnIdle() that
   // no more idle cycles are required.
   //
   int task_count = ZTask::vTasks.size();
   if (task_count == 0)
      {
      return false;            // All done with tasks!
      }

   // Otherwise check if reached the end of the
   // task list an start over at the top.

   else if (iIndex >= task_count)
      {
      bNextIdle = bDoIdle;    // Establish critical polling.
      iIndex = 0;             // Start from the top of the list.
      bDoIdle = false;        // Begin next polling check.
      }

   // If we are at the top of the list, sort the list of tasks
   // based on the priority levels, this is useful for one-shot
   // tasks where some are preferred to be executed first, even
   // though they are added later to the queue. 

   if (bSort==true && iIndex == 0)
      {
      vTasks.sort(comparePriority);
      }

   // Get the next task in the list. Verify it exists.

   ZTask *task = ZTask::vTasks[iIndex];
   if (task == nullptr)
      {
      ASSERT(task != nullptr);   // Sanity check!
      return false;              // Something went wrong.
      }

   // If the task is active, then its lock count
   // will be zero. Check this before processing.

   if (!task->iLockCount || task->iStepCount)
      {
      if (task->iStepCount)
         task->iStepCount--;

      // If the task is ready, call the execute
      // method. Enclose in a try/catch block to
      // prevent a throw from aborting the program.

      try
         {
         if (task->ready() && !task->bEscape)
            {
            task->bEscape = task->execute();
            }
         }

      catch (CException *x)
         {
         x->ReportError();
         x->Delete();

         task->bEscape = true;
         }

      // If there is a termination request or a task returns true and 
      // no children are present, remove it from the task list,
      // causing the list to collapse  down to the  next task.

      if ((task->bTerminate==true || task->bEscape==true) && task->iLockCount==0)
         {
         ZTask::removeTask(task);
         iIndex--;         // Collapse indexer.
         }

      // Otherwise check if this task is marked as critical
      // polling. This will enforce idle cycles from the OS
      // on the next pass of the list.

      else
         bDoIdle = bDoIdle || task->bCritical;
      }

   iIndex++;      // Increment to the next task.

   // Returns 'true' if there is critical processing flagged
   // from the last pass down the list. If not it will be
   // 'false' and this procedure will not be called until
   // the next window message has been processed, freeing up
   // the CPU.

   return bNextIdle;
   }

There are a number of houskeeping and verification steps being done here, each is designed to deal with specific details of the cycling. In general, they are mostly for internal functions. The virtual execute() method, unlike taskAdded()/taskRemoved(), is called repeatedly until it return a value of ‘true’, signalling that it has completed its processing and the task is now ready for removal. To make this work cooperatively and keep the UI responsive, the outer loop of the long procedure is removed and replaced by repeated calls to execute() until such point it tells the caller to terminate.

//-< VIRTUAL >------------------------------------------------------------------
// execute()
//
// Override this method to insert the main body of execution for the task.
// Since a task is hooked into the idle time of the message queue, this
// method is called in a single idle cycle and thus comprises a single
// iteration. The execute action should be chosen such that it does not block
// the application.
//------------------------------------------------------------------------------
bool        ZTask::execute()
   {
   // When sub-classing, this method should be overridden to extend
   // the behaviour and include process activities. Otherwise use an
   // attached object implementing IRunnable.
   //
   if (pRunnable)
      return pRunnable->run();

   ASSERT(0);  // Oops! You must override this or nothing useful will happen!
   return true;
   }

When creating and using the tasks, it is now very simple and easy to do. Since the task now provides the outer loop, all you need to decide is how you want to break up the long procedure in your overridden task execution.

Using Tasks

Using the objects depends a lot on your needs. If you want to invoke a task more than once, as in the case of an updater, you may want to construct the object embedded with the auto-delete flag set to false. When you need to start it, you may make a call like this:

ZTask::addTask(this->pUpdater);

When it has completed (by returning true), it will automatically be removed again. We use this strategy to update backing stores for components. If you want to create a task and then forget about it, you would likely create it dynamically and then start it right away. The auto-delete flag would be turned on in this situation so that the object is cleaned up once it is finished. The call might look like this.

ZTask::addTask(new MyTask());

When creating a derived task class, you will need to implement the execute() method as a minimum. The taskAdded()/taskRemoved() are optional and if not implemented, will do the default behavior. For myself, I generally always implement all three virtual methods for completeness.

Dividing up the execute() into smaller pieces is simply a judgment call. I like to set it up so that each iteration is typically 100ms or less, so that any user input will be acted upon in less that this. For example, if I have a large ASCII file read task, often I will read and process a single line or small group at a time, keeping track of where I was in each iteration. Typically, I will use a progress bar to indicate its current position giving me an idea how it is progressing.

Blocking Execution

Of course, this article is about not blocking the application, but sometimes it must be the case. I have a situation where the same code does not block in one situation, but then must block in others. In addition, I really like to test out my code by running it in a blocking mode to make sure it behaves as expected prior to hooking it to the idle queue. For completeness I have include and addition static method executeBlocking().

//==============================================================================
// executeBlocking()
//
// Execute a task, blocking until done. Normally this is not intuitive since
// the whole point of having a task is *not* to block. In some situations it
// is useful for a task to execute this way for both development testing and
// to provide the option without re-write of the task.
//
//------------------------------------------------------------------------------
bool        ZTask::executeBlocking(ZTask *task) 
   {
   if (task != nullptr)
      {
      task->bBlocking = true;    // Blocking flag must be set here.

      // Call the initialization method. If it returns true, then the
      // initial state is bad and we must exercise an escape action
      // from the task. Finalization only occurs if initialization
      // is successful.
      //
      task->bEscape = task->taskAdded();
      if (task->bEscape == false)
         {
         do {
            if (task->iStepCount)
               task->iStepCount--;

            if (task->ready() && !task->bEscape)
               {
               task->bEscape = task->execute();
               }

            } while (!task->bEscape);
         }

      else
         {
         TRACE("\n[%s] Bad initial state, terminating task."
               , task->getName());
         }

      task->taskRemoved();

      // Like thread objects, check flag to have the object put in 
      // the recycling bin. We use a recycle bin so that the object 
      // can linger until the next idle cycle. Lingering allows the
      // task to be accessed after this call to retrieve state
      // information and error messages.
      //
      if (task->bAutoDelete)
         {
         ZTask::vRecycle << task;
         }

      return true;
      }

   return false;
   }

This method will run the code in the same pattern as the executeTasks(), but without idle cycle slicing. It still uses the recycle list to destruct, but waits until the next idle cycle. This 'lingering' can be handy, since the object is still valid after the call and can be queried for state and variable information.

For example, I use task to package up component factories that generate UI devices when an XML document element is processed. The code looks like this:

Scopes::Instrument::Factory factory(this);
factory.setInput(&element);
ZTask::executeBlocking(&factory);

c = factory.pInstrument;

This example uses a factory task that is scoped locally, but it can equally be created on the heap and setting the auto delete flag so that it is later recycled. At any point, the very same task can be hooked into the idle queue by switching out the executeBlocking() for addTask(). Very convenient for proofing the code.

One-Shots

One-Shots are pieces of code that need to be executed asynchronously, but only require a single pass. This is typically what happens when messages are processed such as painting or button clicks. A task object can also be used in exactly this fashion. The big advantage here is all the code associated with the task can be packaged together rather than distributed through window messages. All the task does is return true on the first pass. It is then removed right away by the application object. If the task is embedded as a member of another class, it is not auto-deleted and can be reused over and over by simply adding back to the application whenever needed.

In our applications, one-shot tasks are used extensively as update objects for trees and backing stores for controls and graphs. We do not want these activities to consume CPU time until the message queue is quiet, thus retaining responsiveness in the application. Since potentially dozens of these objects may be present on the display surface, interaction with them is triggering many update tasks that are queued until the application is idle. These objects are part of a library of tools and so they must be able to add the tasks automatically as needed, and without additional coding.

Here is a one-shot task that handles a package of data received at a communication port.

ZTask::addTask(new Comm::ExchangeHandler(Comm::Exchange::validate((CPtr)lparam)));

The communication handler is created, passed a communications object, and then handed off to the task processing where it executes as one-shot. Once it completes, the task is placed on the recycle bin and cleaned up on the next idle cycle. A lot is going on here, but I am able to invoke it from a single line of code. This creates a setup that is very maintainable and extensible.

One of the side benefits I have found with this approach is, it never runs while a window is scrolling, so it will not interfere with user interaction. A lengthy process can be quite noticeable when interacting with the UI as it becomes choppy with threads. Using this, the UI always has priority and the tasks will wait for user.

Notes

This class utilized a Vector class that is described in another article that I have already posted. You can easily replace this with a CTypedPtrList if you do not wish to use the Vector class. We utilize our own collection classes for maintenance reasons. The code provided here is well documented and should be easy to follow.

Let me know if you have any ideas in the comments below. Cheers!