Summary
First, this article should not be construed to be encouraging the sharing of objects between asynchronous threads. Indeed, we view the sharing of objects between threads as an often problematic but sometimes unavoidable practice. Generally, when designing concurrent code we recommend favoring the "isolation + asynchronous messages" paradigm when practical. That said, when you do share objects between threads you're going to want to do it as safely as possible.
The scenarios when an object is shared between threads in C++ can be divided into two categories - a "read-only" one where the object is never modified, and a "non-read-only" one. Scenarios in the non-read-only category are going to require an access control mechanism.
Note that in C++, the fact that an object is declared const does not guarantee that it won't actually be modified due to the possibility of the object having "mutable" members. Sometimes those mutable members are "protected" (by a mutex or equivalent) making them "thread safe". But sometimes they are not. (Particularly in older code.) So extra vigilance may be called for when determining whether or not an object could be modified.
So first let's consider the general scenario where the programmer wants to allow for the shared object to be modified, and for the possibility that the shared object has unprotected mutable members. For these scenarios, you can use mse::TAsyncSharedReadWriteAccessRequester<> as demonstrated in the following example:
#include "mseasyncshared.h"
#include <future>
#include <list>
#include <random>
#include <iostream>
#include <ratio>
#include <chrono>
#include <string>
int main()
{
std::default_random_engine rand_generator1;
std::uniform_int_distribution<int> udist_0_9(0, 9);
const size_t num_tasks = 10;
const size_t num_digits_per_task = 10000;
const size_t num_digits = num_tasks * num_digits_per_task;
class CObj1WithUnprotectedMutable {
public:
std::string text() const {
m_last_access_time = std::chrono::system_clock::now();
return m_text1;
}
void set_text(const std::string& text) {
m_last_access_time = std::chrono::system_clock::now();
m_text1 = text;
}
std::chrono::system_clock::time_point last_access_time() {
return m_last_access_time;
}
private:
std::string m_text1 = "initial text";
mutable std::chrono::system_clock::time_point m_last_access_time;
};
class B {
public:
static size_t num_occurrences(
mse::TAsyncSharedReadWriteAccessRequester<CObj1WithUnprotectedMutable>
obj1_access_requester, const char ch, size_t start_pos, size_t length) {
auto obj1_readlock_ptr = obj1_access_requester.readlock_ptr();
auto end_pos = start_pos + length;
assert(end_pos <= obj1_readlock_ptr->text().length());
size_t num_occurrences = 0;
for (size_t i = start_pos; i < end_pos; i += 1) {
if (obj1_readlock_ptr->text().at(i) == ch) {
num_occurrences += 1;
}
}
return num_occurrences;
}
};
auto obj1_access_requester = mse::make_asyncsharedreadwrite<CObj1WithUnprotectedMutable>();
std::string rand_digits_string;
for (size_t i = 0; i < num_digits; i += 1) {
rand_digits_string += std::to_string(udist_0_9(rand_generator1));
}
obj1_access_requester.writelock_ptr()->set_text(rand_digits_string);
std::list<std::future<size_t>> futures;
for (size_t i = 0; i < num_tasks; i += 1) {
futures.emplace_back(std::async(B::num_occurrences, obj1_access_requester, '5',
i*num_digits_per_task, num_digits_per_task));
}
size_t total_num_occurrences = 0;
for (auto it = futures.begin(); futures.end() != it; it++) {
total_num_occurrences += (*it).get();
}
}
mse::TAsyncSharedReadWriteAccessRequester<> automatically protects the shared object from being accessed while it's being modified in another thread. (Although that's not really an issue in this simple example.)
But because of the possibility of the shared object having unprotected mutable members, out of prudence mse::TAsyncSharedReadWriteAccessRequester<> does not, by default, allow for simultaneous access, even through readlock_ptrs.
But sometimes you might really want to allow for simultaneous read operations. For those situations you can use mse::TAsyncSharedObjectThatYouAreSureHasNoUnprotectedMutablesReadWriteAccessRequester<> (and mse::make_asyncsharedobjectthatyouaresurehasnounprotectedmutablesreadwrite<>()). It has the same interface as mse::TAsyncSharedReadWriteAccessRequester<>, but an unwieldy name to help remind users of the prerequisite for using it.
And lastly, a common scenario is the simple one where only read access is required and the programmer has reliably determined that the shared object has no unprotected mutable members. In this case you can get away with not having any access control mechanism. For this scenario you can use mse::TReadOnlyStdSharedFixedConstPointer<> which is basically just a thin wrapper around (and publicly derived from) an std::shared_ptr that tries to ensure that the shared object is const and make clear to others reading the code the intended purpose of the pointer. (To share an object that will not be modified between threads.)
#include "mseasyncshared.h"
#include <future>
#include <list>
#include <random>
#include <iostream>
#include <string>
int main()
{
class CObj1WithNoMutables {
public:
CObj1WithNoMutables(const std::string& text) : m_text1(text) {}
std::string text() const {
return m_text1;
}
void set_text(const std::string& text) {
m_text1 = text;
}
private:
std::string m_text1 = "initial text";
};
class B {
public:
static size_t num_occurrences(const std::shared_ptr<const CObj1WithNoMutables> obj1_shptr,
const char ch, size_t start_pos, size_t length) {
auto end_pos = start_pos + length;
assert(end_pos <= obj1_shptr->text().length());
size_t num_occurrences = 0;
for (size_t i = start_pos; i < end_pos; i += 1) {
if (obj1_shptr->text().at(i) == ch) {
num_occurrences += 1;
}
}
return num_occurrences;
}
};
std::string rand_digits_string;
for (size_t i = 0; i < num_digits; i += 1) {
rand_digits_string += std::to_string(udist_0_9(rand_generator1));
}
auto obj1_roshfcptr = mse::make_readonlystdshared<CObj1WithNoMutables>(rand_digits_string);
std::list<std::future<size_t>> futures;
for (size_t i = 0; i < num_tasks; i += 1) {
futures.emplace_back(std::async(B::num_occurrences, obj1_roshfcptr, '5',
i*num_digits_per_task, num_digits_per_task));
}
size_t total_num_occurrences = 0;
for (auto it = futures.begin(); futures.end() != it; it++) {
total_num_occurrences += (*it).get();
}
}
Off to the races
Roughly speaking, a "race condition" is a situation where a program or code's results can vary as a function of the relative execution timing of concurrent threads. For example, say you have an integer variable, x, with an initial value of say, 5, and two threads. Let's suppose one of those threads will increment x by 1, and the other one will double x. So the value of x might end up as either 11 or 12, depending on which thread gets to x first. Right?
A "data race" is a situation where one or more asynchronous threads are allowed to access a piece of memory while it's being modified (by another thread). We consider data races to be a specific case of race conditions, but others choose to exclude data races from their definition of race condition. Any communication between asynchronous threads is potentially sufficient for a race condition to occur. Data races, on the other hand, require that a piece of memory be "shared" by multiple asynchronous threads. That is, each thread has direct access to the memory for some common period of time.
Data race bugs officially result in "undefined behavior". "Undefined behavior" here being basically a euphemism for "potential consequences of the most severe kind", including invalid memory access and, in some scenarios, remote code execution. In particular, data race bugs have the ability to cause an object to be accessible in an inconsistent state. For example, if we consider an std::vector, we can imagine that it contains a pointer to some allocated memory, some kind of integer indicating the number of elements contained, and another one indicating the capacity of the allocated memory, in terms of number of elements. (To the user) it should always be the case that the number of elements contained is less than or equal to the capacity. (Relationships that should always be true like this are called "invariants".) If ever this relationship is observed (by the user) to be false, the std::vector would be considered to be in an "inconsistent" or "corrupt" state. In the case of std::vector, possibly resulting in invalid memory access.
Invalid memory access can be considered one of, if not the most severe types of bugs in C++. (Although in theory, all "undefined behavior" bugs are equivalent.) They are particularly bad, because in practice they can cause your program to stop executing, expose sensitive data stored in otherwise inaccessible memory, or even allow for arbitrary code execution and the compromisation of the host environment. So it is often a priority to reduce or eliminate the possibility of invalid memory access, even if it comes at some cost (in terms of performance, flexibility, "bug hiding", increasing the likelihood of other less severe bugs, etc.). To that end, while it may not be possible to prevent all race condition bugs, we'd like to be able to reduce or eliminate the ones that unavoidably risk invalid memory access.
These include data race bugs, but also a slightly more general set we might call "object race" bugs. Data race bugs involve direct access of shared memory, while "object race" bugs also include indirect access of shared memory via any part of a shared object's interface, including member and friend functions and operators. The idea here is that objects that maintain consistent internal state generally allow their functions and operators to temporarily change the internal state to an inconsistent one, as long as consistency is restored before the function or operator returns. "Object race" bugs include bugs where one thread is allowed to access a shared object while it has been temporarily put in an inconsistent state by a member function or operator executing in another asynchronous thread.
If we consider the main factors that make a bug problematic
i) frequency/unpredictability of occurrence
ii) severity of consequences
iii) difficulty in reproducing/debugging
iv) ability to evade detection during testing
we note that "object race" bugs, as a category, achieves the superfecta - all four factors apply in spades - making it perhaps the worst class of bugs in all of computer programming.
Which is why it's best to avoid sharing objects between asynchronous threads when practical, and spare no safety mechanism when not.
So consider how std::shared_ptr can be used to address "object lifespan" bugs. That is, bugs where you (attempt to) access an object after it's been deallocated. If you were to hypothetically write a program such that all objects are allocated via std::make_shared and accessed through (appropriate) std::shared_ptrs only, then you would be essentially guaranteed to have no "object lifespan" bugs. Right? Because the std::shared_ptrs do two things - they provide access to the object, and they control when the object will be deallocated. So they ensure that the object is not deallocated until after they (all permanently) stop providing access to the object.
Well, a similar technique can be used to ensure that a shared object is not accessed while another thread is modifying it. In order to safely access an object, a thread needs to "own" an appropriate (write or read) "access lock" on the object. Well, imagine smart pointers that control when their target objects are deallocated (i.e. have "ownership" of the object's lifespan) in the same way that std::shared_ptrs do, but in addition, also control when the access lock is released (i.e. have "ownership" of the access lock). Let's call these smart pointers "writelock_ptr"s and "readlock_ptr"s, as appropriate. They won't release their access lock until after they stop providing access to the shared object (i.e. when they are destroyed).
So restricting the access of shared objects to accesses through writelock_ptrs and/or readlock_ptrs only will ensure that the accesses are safe.
While similar, there are differences in the way writelock_ptrs and readlock_ptrs are used compared with std::shared_ptrs. In particular, if you expect to need future access to a target object, you might consider storing an std::shared_ptr for later use. In contrast, you would not do that with writelock_ptrs and/or readlock_ptrs because as long as they exist, they hold a lock on the shared object, possibly preventing other threads from accessing the object. In general, you want to minimize the lifespan of writelock_ptrs and readlock_ptrs in order to maximize the availability of the shared object. This calls for a separate helper object we'll call an "access requester". Access requesters do not provide direct access to the shared object or hold any locks on it. They do (try to) provide writelock_ptrs and/or readlock_ptrs upon request. So if you expect to need future access to a shared object, rather than holding on to a writelock_ptr or readlock_ptr you can just use an access requester to reacquire one whenever you need. (See the first example in the "Summary" section and/or download the accompanying source code for a more comprehensive example.)
The mutable member matter
A peculiarity of C++ is that a const object is not necessarily guaranteed to be unmodifiable, because, for example, C++ permits "mutable" members. Now, it's always been understood that while the mutable keyword may be used to subvert the mechanics of const, the semantics of const should be preserved. That is, an object declared const, should, to the user (of its public interface), behave as if it were const even if under the hood private mutable members are actually being modified. But not until the arrival of C++11 did, by convention, the preservation of const semantics include notions of thread safety - the ability to be accessed simultaneously by asynchronous threads without issue. So the problem is that there are a bunch of legacy objects with mutable members out there that you might want to share between threads, but cannot be assumed to be safely shareable when declared const.
And it's not clear that this is an issue with just legacy objects. The current convention is that objects with mutable members should preserve thread safety as part of preserving const semantics, but doing so often has a cost in terms of performance and scalability. (Scalability because, for example, thread safety mechanisms often require locking cpu cache lines thus holding up other threads that need to access the same cache lines.) So when using such an object in a context where it is not being shared between threads, often a (small but) unnecessary performance cost is being paid. Now there are those of us that don't mind seeing the establishment of a convention that sacrifices a little performance in the name of safety. But you have to wonder how reliably a performance obsessed programming community is going to adhere to the rule when it's not clear that the trade-off is even necessary.
Consider an alternative convention where classes with mutable members are compelled to clearly indicate (in their name, for example) whether or not they can be safely shared between threads. And additionally, encourage any class that is not safely shareable (for performance reasons) to be provided as a pair with a compatible safely shareable version of the class. It's probably better to have the author positively indicate that their class is (or is not) safe to share rather than have the user just assume it.
And this being C++, the mutable keyword is not the only hole in the enforcement of const. For example, you may declare a const instance of a class with the intent of sharing it between threads. But if this class happens to be sort of a "compound" class that contains references to "child" objects (or "indirect members", if you will), the constness is not propagated to those child objects leaving them vulnerable to race condition and data race bugs. Such child objects can also be considered "mutable members" and with respect to safety, should be treated as such.
int main()
{
class CObjWithIndirectMember {
public:
CObjWithIndirectMember() : m_string1(*(new std::string("initial text"))) {}
~CObjWithIndirectMember() {
delete (&m_string1);
}
void set_string2(const std::string& string_cref) const {
m_string2 = string_cref;
}
void set_string1(const std::string& string_cref) const {
m_string1 = string_cref;
}
mutable std::string m_string2 = "initial text";
std::string& m_string1;
};
const CObjWithIndirectMember const_obj_with_indirect_member;
const_obj_with_indirect_member.m_string2 = "new text";
const_obj_with_indirect_member.m_string1 = "new text";
}
So, unfortunately, in C++ there is really no general way to ensure that an object will not be modified, and consequently no general way to ensure that simultaneous access to a shared object will be safe. So the prudent policy would be to permit simultaneous access only in cases where the shared object is simple enough that it is universally apparent that there is no issue with potentially mutable members, or in cases where the object provides explicit positive indication that it can be shared safely. And in either case, if you are going to permit simultaneous access, the code should clearly indicate it, preferably by using a type specifically dedicated for the purpose (and that may facilitate extra compile time safety). (For example, prefer a type like mse::TReadOnlyStdSharedFixedConstPointer over using std::shared_ptrs directly.)
More shared objects, more problems
So we've introduced data types that, among other things, protect the consistency of a shared object's internal state. But if you're sharing multiple objects, the consistency of any relationship (aka invariant) between those objects is not automatically protected. So while we recommend avoiding the practice of sharing objects between threads when practical, we would even more strongly advise avoiding the practice of sharing multiple interdependent objects between threads. But when you do have to do it, you should be thinking about operations on those objects as part of transactions that need to be executed atomically.
#include "mseasyncshared.h"
#include <future>
#include <list>
#include <random>
#include <ratio>
#include <chrono>
int main()
{
class CAccount {
public:
void add_to_balance(double amount) {
m_balance += amount;
m_last_transaction_time = std::chrono::system_clock::now();
}
double balance() const {
return m_balance;
}
private:
double m_balance = 0.0;
std::chrono::system_clock::time_point m_last_transaction_time;
};
class B {
public:
static bool nonatomic_funds_transfer(
mse::TAsyncSharedReadWriteAccessRequester<CAccount> source_ar,
mse::TAsyncSharedReadWriteAccessRequester<CAccount> destination_ar, const double amount)
{
if (source_ar.readlock_ptr()->balance() >= amount) {
source_ar.writelock_ptr()->add_to_balance(-amount);
destination_ar.writelock_ptr()->add_to_balance(amount);
return true;
}
else {
return false;
}
}
static bool atomic_funds_transfer(
mse::TAsyncSharedReadWriteAccessRequester<CAccount> source_ar,
mse::TAsyncSharedReadWriteAccessRequester<CAccount> destination_ar, const double amount)
{
auto source_writelock_ptr = source_ar.writelock_ptr();
auto destination_writelock_ptr = destination_ar.writelock_ptr();
if (source_writelock_ptr->balance() >= amount) {
source_writelock_ptr->add_to_balance(-amount);
destination_writelock_ptr->add_to_balance(amount);
return true;
}
else {
return false;
}
}
};
auto bobs_account_access_requester = mse::make_asyncsharedreadwrite<CAccount>();
auto bills_account_access_requester = mse::make_asyncsharedreadwrite<CAccount>();
auto barrys_account_access_requester = mse::make_asyncsharedreadwrite<CAccount>();
bobs_account_access_requester.writelock_ptr()->add_to_balance(100.0);
bills_account_access_requester.writelock_ptr()->add_to_balance(200.0);
barrys_account_access_requester.writelock_ptr()->add_to_balance(300.0);
std::future<bool> bob_to_bill_res = std::async(B::atomic_funds_transfer,
bobs_account_access_requester, bills_account_access_requester, 10.0);
std::future<bool> bill_to_barry_res = std::async(B::atomic_funds_transfer,
bills_account_access_requester, barrys_account_access_requester, 20.0);
std::future<bool> barry_to_bob_res = std::async(B::atomic_funds_transfer,
barrys_account_access_requester, bobs_account_access_requester, 30.0);
bool all_transfers_were_executed = (bob_to_bill_res.get() && bill_to_barry_res.get()
&& barry_to_bob_res.get());
}
Details for keeners
The standard library recognized the need to allow for the locking of a mutex multiple times by a single thread, so they provide std::recursive_mutex. They also recognized the need for a mutex that supports shared locking and locking attempts that expire after a timeout period, so they provide std::shared_timed_mutex. But of course for the general case you'd want a mutex that supports all of these features - an std::recursive_shared_timed_mutex. But frustratingly, (at the time of this writing) the standard library provides no such mutex. Perhaps because it's not obvious how to implement such a type in a way that is optimized for performance and memory footprint.
Anyway, we can't wait on the standard library, so we provide the mutex - mse::recursive_shared_timed_mutex. It's the mutex that's called for when implementing a general solution for shared objects like ours. And if you find yourself needing such a mutex for other purposes, it's there in the header file.
Speaking of locking attempts that expire after a timeout period, while we don't use them in the examples, the "access requester" types do support the functionality:
auto access_requester = mse::make_asyncsharedreadwrite<std::string>("some text");
auto writelock_ptr1 = access_requester.try_writelock_ptr();
if (writelock_ptr1) {
}
auto readlock_ptr2 = access_requester.try_readlock_ptr_for(std::chrono::seconds(10));
auto writelock_ptr3 = access_requester.try_writelock_ptr_until(std::chrono::steady_clock::now()
+ std::chrono::seconds(10));
Curb your enthusiasm (for perilous parallel programming practices)
When programming for today's multiprocessing architectures it can just seem convenient and natural to share objects between concurrent threads. And it can be easy to overlook the fact that the practice of sharing objects between threads brings with it a fundamentally different and more problematic kind of bug than we've been used to dealing with.
A bug in the control of access to a shared object can result in any part of that object (or anything referenced by the object) being modified at any time. It's the "any time" aspect that is different and particularly problematic. This means that if your program crashes, or an assert fails, even if you have a full stack trace and memory dump, it still may not be possible to deduce the sequence that led to the failure state.
And as an extra whammy, it is not rare for this type of bug to be impractical to reproduce. The position of not being able to deduce or reproduce the steps that occurred to arrive at the failure state is a disturbing one.
But not one that we, as a species, are unfamiliar with. For example, the ancient Egyptians were unable to deduce the sequence of events that caused occasional catastrophic flooding of the Nile. Nor could they reproduce the phenomenon on demand. They dealt with the situation by making ritual offerings to a ram-headed deity. Maybe that'd help with our data race bugs too. :)
And maybe some extra prudence when sharing objects between threads wouldn't hurt either.
