reuse: An Object Recycling C++ Class Library

• Download source code - 16.6 KB

Introduction

In this article, I cover the reuse class library: the interface to the library, and the implementation of the object pool that powers it. With this class library, you can have the benefits of database connection pooling for your own resource-intensive classes.

The reuse Class Library

The class library resides in one file, reuse.h, with three classes: reusable, pool, and reuse.

reusable

The first class is the base class reusable:

class reusable
{
protected:
	reusable(const std::wstring& initializer = L"")
		: m_initializer(initializer)
	{}

public:
	virtual void clean() {}
	virtual bool cleanInBackground() const { return false; }
	virtual std::wstring initializer() const { return m_initializer; }

private:
	std::wstring m_initializer;
};

You can build class libraries around the reusable base class, implementing cleanInBackground() and clean() in different ways in your derived types.

But you don't have to.

As long as your class implements cleanInBackground(), clean(), and initializer(), you don't need to work with the reusable base class. This is because the object pool is a templated class working with any type of object, not only types derived from reusable.

pool

The main class is pool:

template <class T>
class pool
{
public:
	/// <summary>
	/// Declare a pool object for a type with given object count limits
	/// </summary>
	/// <param name="constructor">What object should be created based on the initializer?
	/// </param>
	/// <param name="maxInventory">How many objects can the pool hold 
	/// before objects put for recycling are dropped (deleted)?</param>
	/// <param name="maxToClean">How many objects can be in queue 
	/// for cleaned before objects are dropped (deleted)?</param>
	pool
	(
		const std::function<T* (const std::wstring&)> constructor,
        const bool allowBackgroudCleanup = true, 
		const size_t maxInventory = 1000U, 
		const size_t maxToClean = 1000U
	);

	/// <summary>
	/// Get a reuse object to get an object from the pool
	/// and automatically return it back to the pool
	/// </summary>
	/// <param name="initializer">Initializer for the object to return</param>
	/// <returns>
	/// An object for gaining access to a pooled object
	/// </returns>
	auto use(const std::wstring& initializer = L"")
	{
		return reuse<T>(*this, initializer);
	}

private:
	/// <summary>
	/// Get an object for a given initializer string
	/// Objects are created using initializer strings, used, then put() and clean()'d
	/// and handed back out
	/// </summary>
	/// <param name="initializer">Initializer for the object to return</param>
	/// <returns>Pointer to a new or reused object</returns>
	T* get(const std::wstring& initializer);

	/// <summary>
	/// Hand an object back to the pool for reuse
	/// </summary>
	void put(T* t);

	/// <summary>
	/// Thread routine for cleaning up objects in the background
	/// </summary>
	void cleanup();
};

To use a pool, you construct one with a constructor function that returns an object of type T, and limits on the size of the pool's internal resource usage.

Then you call use() with the initializer string to pass to your constructor function. use() returns a reuse<T> object that you hold onto to make use of a pooled object, and the reuse<T> object returns the object to the pool when you're done with it.

The constructor function you provide can return anything compatible with the template class T. It can decide on the type of object returned by way of processing the initializer string. This allows for a type-safe yet dynamic object pool, where you can use inheritance if you want to, but it is not required.

reuse

reuse<T> is handed back by pool<T>::use() for access to an object in the pool:

template <class T>
class reuse
{
public:
    /// <summary>
    /// Declaring an instance of this class gets a T object from the pool
    /// When this reuse object goes out of scope, the T object is returned to the pool
    /// </summary>
    /// <param name="pool">pool to get objects from and put objects back</param>
    /// <param name="initializer">string to initialize the object</param>
    reuse(pool<T>& pool, const std::wstring& initializer = L"")
      : m_pool(pool)
    {
        m_t = m_pool.get(initializer);
    }

    /// <summary>
    /// Cannot copy because of pointer owned by other
    /// Moving is okay
    /// </summary>
    reuse(const reuse& other) = delete;

    /// <summary>
    /// Cannot assign because of pointer owned by other
    /// Moving is okay
    /// </summary>
    reuse& operator=(const reuse& other) = delete;

    /// <summary>
	/// Move constructor to carry along the object pointer
	/// without an unnecessary get() / put() pair of pool calls 
	/// </summary>
	reuse(reuse&& other)
		: m_pool(other.m_pool)
		, m_t(other.m_t)
	{
		other.m_t = nullptr;
	}
	
	// Free the object back to the pool
	~reuse()
	{
		m_pool.put(m_t);
	}

	/// <summary>
	/// Access the pooled object
	/// </summary>
	T& get() { return *m_t; }

private:
	pool<T>& m_pool;
	T* m_t;
};

pool implementation

The pool implementation revolves around get(), put(), and cleanup().

get()

Special consideration is given for use cases that have no interest in initializer strings. A bucket (vector<T*>) is reserved for this use case.

When initializer strings are used, there is a bucket dedicated to each unique initializer string. This allows for multiple initializer strings per pool, like different connection strings for different databases, all using the same type of database connection class, or different database connection classes with a common base class. This can be a dynamic solution to your problem.

T* get(const std::wstring& initializer)
{
	if (m_keepRunning)
	{
		std::unique_lock<std::mutex> lock(m_bucketMutex);

		// Consider using the null bucket used with empty initializer strings
		if (initializer.empty()) 
		{
			if (!m_unBucket.empty())
			{
				T* ret_val = m_unBucket.back();
				m_unBucket.pop_back();
				m_size.fetch_add(-1);
				return ret_val;
			}
		}
		else // we're using initializer strings
		{
			// Find the bucket for the initializer string
			// See if a matching bucket exists and has objects to hand out
			const auto& it = m_initBuckets.find(initializer);
			if (it != m_initBuckets.end() && !it->second.empty())
			{ 
				T* ret_val = it->second.back();
				it->second.pop_back();
				m_size.fetch_add(-1);
				return ret_val;
			}
		}
	}

	// Failing all of that, including whether we should keep running,
	// construct a new T object with the initializer
	return m_constructor(initializer);
}

put()

When an object is to be recycled before being reused, its clean() function must be called.

It's up to the object whether clean() should be called immediately, or in the background.

/// <summary>
/// Hand an object back to the pool for reuse
/// </summary>
void put(T* t)
{
	if (t == nullptr)
		return;

	if (m_keepRunning)
	{
		if (t->cleanInBackground()) // queue up the object for background cleaning
		{
			std::unique_lock<std::mutex> lock(m_incomingMutex);
			if (m_incoming.size() < m_maxToClean)
			{
				m_incoming.push_back(t);
				m_incomingCondition.notify_one();
				return;
			}
		}
		else // clean up and directly add to the right bucket
		{
			if (m_size.load() < m_maxInventory)
			{
                t->clean(); 

				const std::wstring& initializer = t->initializer();
				std::unique_lock<std::mutex> lock(m_bucketMutex);
				if (initializer.empty())
					m_unBucket.push_back(t);
				else
					m_initBuckets[initializer].push_back(t);
				m_size.fetch_add(1);
				return;
			}
		}
	}

	// Failing all of that, including whether we should keep running, drop the object (delete)
	delete t;
}

cleanup()

Background cleanup processing gets objects to clean, cleans them, then adds them to the inventory for reuse.

/// <summary>
/// Thread routine for cleaning up objects in the background
/// </summary>
void cleanup()
{
    // Bail early on cleanup if we're not to clean in the background
    if (!m_allowBackgroudCleanup)
    {
        m_doneCleaning = true;
        return;
    }

	while (m_keepRunning)
	{
		// Get something to clean
		T* t;
		{
			std::unique_lock<std::mutex> lock(m_incomingMutex);
			m_incomingCondition.wait_for
			(
				lock,
				10ms,
				[&] { return !m_incoming.empty() || !m_keepRunning; }
			);
			if (m_incoming.empty() || !m_keepRunning)
				continue;

			t = m_incoming.back();
			m_incoming.pop_back();
		}

		// Delete the object if our shelves are full
		if (m_size.load() >= m_maxInventory)
		{
			delete t;
			continue;
		}

		// Clean it
		t->clean();

		// Add the object to the right pool
		std::wstring initializer = t->initializer();
		std::unique_lock<std::mutex> lock(m_bucketMutex);
		if (initializer.empty())
			m_unBucket.push_back(t);
		else
			m_initBuckets[initializer].push_back(t);
		m_size.fetch_add(1);
	}
	
	// All done.
	m_doneCleaning = true;

Example reuse Library Usage

reuse Unit Test

The unit test shows library usage and covers synchronous and async cleanup:

#include "CppUnitTest.h"

#include "../reuse/reuse.h"

#include <thread>

using namespace Microsoft::VisualStudio::CppUnitTestFramework;

namespace reuse
{
	std::atomic<int> test_class_count = 0;

	class test_class : public reusable
	{
	public:
		test_class(const std::wstring& initializer, const bool cleanInBackground)
			: reusable(initializer)
			, m_cleanInBackground(cleanInBackground)
		{
			test_class_count.fetch_add(1);
		}

		~test_class()
		{
			clean();
			test_class_count.fetch_add(-1);
		}

		virtual void clean()
		{
			data = "";
		}
		virtual bool cleanInBackground() const { return m_cleanInBackground; }

		void process()
		{
			data = "914";
		}

		std::string data;

	private:
		const bool m_cleanInBackground;
	};

	TEST_CLASS(reusetests)
	{
	public:
		TEST_METHOD(TestReuse)
		{
			// Test cleaning in the foreground and background
			for (int run = 1; run <= 4; ++run)
			{
				std::vector<bool> should_clean_in_bgs{ true, false };
				for (bool should_clean_in_bg : should_clean_in_bgs)
				{
					// Create our pool with a constructor function
					// that passes in how to do the cleaning
					pool<test_class>
						pool
						(
							[&](const std::wstring& initializer)
							{
								return new test_class(initializer, should_clean_in_bg);
							},
                            should_clean_in_bg
					    );

					// NOTE: We'll hold onto a pool pointer in p so after we're done using 
					// the object, we can inspect the object 
                    // as it sits in the pool's inventory.
					test_class* p = nullptr;
					{
						// Use the pool to get a test_class object
						auto use = pool.use(L"init");

						// Access the test_class object inside the reuse object
						test_class& obj = use.get();
						p = &obj; // stash off the pointer for later

						// See that the object is clean
						Assert::AreEqual(std::string(), obj.data);
						Assert::AreEqual(std::wstring(L"init"), obj.initializer());

						// Use the object
						obj.process();
						Assert::AreEqual(std::string("914"), obj.data);
					}
					if (should_clean_in_bg) // wait for cleanup
						std::this_thread::sleep_for(1s);

					// See that the object is clean in the pool, ready for reuse
					Assert::AreEqual(std::wstring(L"init"), p->initializer());
					Assert::AreEqual(std::string(), p->data);
				}
			}
		}
	};
}

reuse Performance Profile

The performance profile program uses a SQLite wrapper class - 4db, cut waaaay down - to compare time spent creating new database connections every time, versus reusing connections:

#include "../reuse/reuse.h"

#include "db.h"

#include <chrono>
#include <iostream>

// Define our SQLite (well, 4db) reusable wrapper
class sqlite_reuse : public reuse::reusable
{
public:
	sqlite_reuse(const std::wstring& filePath)
		: reuse::reusable(filePath)
		, m_db(filePath)
	{}

	fourdb::db& db() { return m_db; }

	// defaults to no-op clean and foreground (no-op) clean
	// and initializer() for free
private:
	fourdb::db m_db; // our resource-intense cargo
};

typedef std::chrono::high_resolution_clock high_resolution_clock;
typedef std::chrono::milliseconds milliseconds;

int wmain(int argc, wchar_t* argv[])
{
	if (argc < 2)
	{
		std::cout << "Usage: <db file path>" << std::endl;
		return 0;
	}

	std::wstring db_file_path = argv[1];

	size_t loopCount = 1000;

	std::string sql_query = "SELECT tbl_name FROM sqlite_master WHERE type = 'table'";

	for (int run = 1; run <= 10; ++run)
	{
		// Do the standard DB connect / query / close
		{
			std::cout << "Traditional: ";
			auto start = high_resolution_clock::now();
			for (size_t c = 1; c <= loopCount; ++c)
			{
				// This one-liner does it all
				fourdb::db(db_file_path).exec(sql_query);
			}
			auto elapsedMs = std::chrono::duration_cast<milliseconds>
                             (high_resolution_clock::now() - start);
			std::cout << elapsedMs.count() << "ms" << std::endl;
		}

		// Pool the database connection and reuse it
		{
			std::cout << "Pooled: ";
			auto start = high_resolution_clock::now();
			{
				// Create the pool
				reuse::pool<sqlite_reuse>
					pool
					(
						[](const std::wstring& initializer)
						{
							return new sqlite_reuse(initializer);
						}
					);
				for (size_t c = 1; c <= loopCount; ++c)
				{
					// This even longer one-liner does it all
					pool.use(db_file_path).get().db().exec(sql_query);
				}
			}
			auto elapsedMs = std::chrono::duration_cast<milliseconds>
                             (high_resolution_clock::now() - start);
			std::cout << elapsedMs.count() << "ms" << std::endl;
		}
	}

	return 0;
}

We're comparing the cost of:

fourdb::db(db_file_path).exec(sql_query);

with:

pool.use(db_file_path).get().db().exec(sql_query);

Run 1,000 times, creating a new connecting each time and using it costs 400ms. Reusing the database connection only costs 30ms.

Conclusion

I've shown how easy the reuse class library is to integrate and use, and what dramatic improvement in performance object recycling can bring.

History

• 22^nd February, 2022: Initial version