Smart observers to use with unique_ptr

unique_ptr<A>  apA(new A) 
SomeObject.m_pA = apA.get(); 

Now having had to declare a raw pointer and breach the safety of our unique_ptr by calling its .get() method we know full well that we are leaving smart pointer safety land and could be getting into trouble. We know not to call delete on SomeObject's m_pA member but can we rely on m_pA to remain valid or at least test as null if it isn't?

observable_unique_ptr<A> apA(new A); 
SomeObject.m_rA = apA; 

apA = NULL; //deletes the object 

if(SomeObject.m_rA) //tests as zero because it is informed about the deletion
    SomeObject.m_rA->DoSomething(); //doesn't get called

The very specific gaurantee of observer_ptr is that it will test as zero if the object has been deleted. So we merely need to test the observer_ptr before using it. In this case it will test as zero and DoSomething() will not be called. There is no need to put limits on what can be allowed nor to create complex architectures to keep all observers informed. You might have noticed that the initialisation of the observer_ptr m_rA is a direct assigment from the owner pA with no call to .get(). That is because with observer_ptr we never leave smart pointer safety land.

You can also use one observer_ptr to initialise another....

SomeOtherObject.m_rA = SomeObject.m_rA;

....and you can happily proliferate observer_ptrs throughout your code without creating any maintenance burden. Just test them before use. You don't do this with raw pointer aliases because it would become a nightmare keeping them all informed about object deletions.

observable_unique_ptr

observable_unique_ptr<T, D=default_delete<T>> is a typedef contraction (using directive) of 

unique_ptr<T, observable<T, D=default_delete<T>>>

It is a unique_ptr that has been prepared for being observed by observer_ptr<T>s by declaring it with the observable<T, D=default_delete> pseudo deleter. This does not do the deletion, it merely detects it so it can notify the hidden observer infrastructure. It calls the passed in deleter D, by default default_delete<T>, to do the deletion.

The observable pseudo deleter carries an overhead of one pointer variable, making observable_unique_ptr immediately twice the size of an unobservable unique_ptr .

observable_unique_ptr behaves exactly like a unique_ptr, however, like any unique_ptr carrying a custom deleter, it cannot be initialised with make_unique<T>() . Instead you have to use make_observable<T>() .

make_observable - free function

observable_unique_ptr<T> make_observable<T>(...).

Returns an observable_unique_ptr owning a new object  of type T

observable_unique_ptr<T> = make_observable<T>();

If you want to use your own custom deleter instead of default_delete then you will have to write your own observable make function to work with it - see the section on custom deleters.

observer_ptr

observer_ptr<T> is an entirely new smart pointer presented here.

It is a smart observer of objects held by observable_unique_ptr<T>.  Its key characteristics are:

• It cannot be used to delete the object it references
• it will test as zero if the object has been deleted by its owner. 

observer_ptr carries a pointer to the object it references and also a further pointer to a separate indicator of its validity, making it immediately twice the size of a raw pointer.

Construction and assignment

An observer_ptr can be constructed unitialised. That is to say reading as NULL.

observer_ptr<A> rA;

and can be constructed from or assigned by:

• an observable_unique_ptr
• another observer_ptr
• or NULL

observable_unique_ptr<A> apA=make_observable<A>();
rA = apA; //from an observable_unique_ptr

observer_ptr<A> rA2 = rA; //from another observer_ptr

rA = NULL; //from NULL - stops observing the object

It cannot be assigned or constructed from

• a raw pointer
• or make_observable<T>() nor make_unique<T>()

rA = new A; //ERROR will not compile

rA = make_observable<A>(); //ERROR will not compile

The only smart pointer that can be constructed from observer_ptr is another observer_ptr. You cannot construct an observable_unique_ptr from an observer_ptr.

observable_unique_ptr<T> apT = make_observable<T>();
observer_ptr<T> rT = apT;
observable_unique_ptr<T> apT2 = rT; //ERROR will not compile

Grammar of interaction

These direct assignment rules create a compiler enforced grammar of interaction between observer_ptr and observable_unique_ptr which ensures that you will have to make quite an effort to use them incorrectly. Because it is based on facilitating direct assignment only for correct moves, it results in the most easily written code being the correct code. You can only naturally get it right.

Dot methods

• T* get() returns the pointee as a raw pointer
• void release() an alternative to assigning NULL to zero the observer.

enable_observable_this

enable_observable_this is an add on base class that provides a method to return an observer_ptr referencing the 'this' pointer that can be called from within the class definition. 

It is intrusive on your class definitions so it is not the preferred way of making objects observable by observer_ptr. Nevertheless it can be the best way for a class to encapsulate the initialisation of external observing references as may be required for its proper operation.

get_observer_of_this method

observer_ptr<T> get_observer_of_this(this) 

Returns an observer_ptr referencing the this pointer.

class MyClass : public enable_observable_this
​{

    MyClass(observer_ptr<MyClass>& obs) //initialisation in constructor
    {
        obs = get_observer_of_this(this);
    }
};

Note that in the example above, get_observer_of_this(this) has been called in the constructor of the class, the natural place to ensure correct initialisation. However if you are familiar with the shared ownership equivalent std::enable_shared_from_this you will know that calling its shared_from_this() method in the constructor of your class will throw a run-time exception. This is because shared_from_this() checks at run-time if the object is correctly owned and construction happens before ownership is determined.

get_observer_of_this(this) will never thow an exception and will always return a valid observer_ptr even when called in a constructor. This is because correct ownership is ensured at compile time by the following restriction on how types inheriting enable_observable_this can be created and owned.

Object creation and ownership

Objects of classes inheriting enable_observable_this can only be created by make_observable<T>() or a custom deleter equivalent and can only be owned by an observable_unique_ptr<U> where U also inherits enable_observable_this.

This ensures at compile time that they will not be created or owned in a way that could leave the this pointer unsafe to observe. It does mean that you will not be able to declare them as owned in any other way, even as static variables:

class A : public enable_observable_this
{
};

A a; //ERROR - will not compile

observable_unique_ptr<A> apA = make_observable<A>(); //OK

Polymorphic ownership

enable_observable_this takes no type template and can be added once at any point in your class hierarchy but due to the ownership rules you will not be able to transfer ownership from a type inheriting enable_observable_this to a base class that doesn't inherit enable_observable_this.

class A
{};
class B : public A, public enable_observable_this
{​};

observable_unique_ptr<A> apA = make_observable<B>(); //ERROR will not compile

The implication for polymorphic ownership is that the common owning base class for classes inheriting enable_observable_this must also inherit enable_observable_this even if it doesn't itself call get_observer_of_this(this).

To summarise: 

The point in your hierarchy at which you inherit enable_observable_this (if you use it) should be your common base class for polymorphic ownership.

zero_observers method

For completeness a method is also to provided to explicitly zero all observers from within the definition of classes inheriting enable_observable_this.

<span style="font-size: 14.6667px;">void</span> zero_observers()

​This will also zero any observers that have been taken externally from the observable_unique_ptr that holds the object.

free functions

make_observable - described above

observable_unique_ptr<T> make_observable<T>(...).

zero_observers

zero_observers(main_thread_ptr);
zero_observers(worker_thread_ptr);
swap(main_thread_ptr, worker_thread_ptr);

move_to_unobservable

You will not be able to use std::move() to transfer ownership from an observable_unique_ptr to a non-observable unique_ptr because that could leave observers orphaned and unsafe. For this you will need to use:

observable_owner_ptr<T> apT(new T);
observer_ptr<T> rT=apT;
unique_ptr<T> apT2 = move_to_unobservable(apT); //zeroes all observers because apT2 can't support them
if(rT) //tests as zero
   rT->DoSomething();

static_pointer_cast

For completeness, a static casting function is provided for explicitly converting an observer_ptr to one of a more derived class

observer_ptr<more_derived_class> static_pointer_cast<more_derived_class>            (observer_ptr<less_derived_class>& ptr)

Casting from more derived to less derived is carried out implicitly and requires no explicit casting.  

Overhead

As already stated, observer_ptr and observable_unique_ptr are immediately twice the size of a raw pointer. 

minimum size = 2  raw pointers

Additionally an observable_unique_ptr and all observer_ptrs referencing it are connected by a seperately allocated DWORD of heap memory. 

typical average size = between 2 and 2.5 raw pointers

The seperately allocated DWORD of heap memory may remain attached to an observer_ptr whose object has been deleted and to an observable_unique_ptr that no longer has any observers. 

worst case maximum size = 3  raw pointers 

The use of... 

enable_observable_this forces your class to have a vtable 

...if it doesn't already have one.

There is a small amount of code execution in use but none of it involves iteration, recursion or walking of arrays or lists. It is all quite direct. The heap allocation of the seperately allocated DWORD is also likely to return quickly - a single DWORD cannot be hard to find.

Using the code

To make use of observer_ptr you simply have to include the downloaded file observer_ptr.h

#include <memory> //for unique_ptr

#include "observer_ptr.h"

It is not wrapped in a namespace. You can do that yourself if you find it necessary or helpful.

#include <memory> //for unique_ptr

namespace xnr{
#include "observer_ptr.h"
}

I suggest you write a bit of experimental code to try it out and then set about seeing how you can apply it in your code.

Deal with the observers you know about

You can start by dealing with the observers that you already know about in your code because you have either limited what is allowed so as not to undermine them or you have written code to keep them up to date on deletions. Convert them from raw pointers to observer_ptr. This will force you to convert their owners from unique_ptr to observable_unique_ptr. Now you can remove those arbritrary limitations on what can be allowed and/or remove all the support code that was trying to keep observers informed of deletions.

Look for the ones you didn't know about

Next you can look for the observers that you might be unaware of and do the same. Any global or class member that is a raw pointer is a candidate for this . You probably did make sure that they would not get caught out but it could also be that you have only been avoiding disaster by a whisker of good luck. Some of these raw pointers may be back references to a parent or static siblings in which case you should be able to use a C++ reference instead and that will clarify the situation and eliminate them from your enquiries. 
N.B. See the discussions below regarding arguments to functions and local variables that are pointers.

Make more liberal use of observers in your designs.

Finally you can alter your design approach to make more liberal use of trouble free observing references in the form of observer_ptr <font color="#111111" face="Segoe UI, Arial, sans-serif"><span style="font-size: 14px;">Allowing one thing to hold a direct reference to something else is a useful construct. We should not be having to back off from it.</span></font>

Function and method arguments

During a function or method call, any object passed in cannot be deleted except by an action initiated within the call itself. In most cases that means that there is no need to pass it in as an observer_ptr. A raw pointer or reference is good enough and does not create any unecessary linkage to a smart pointer system.

The two execptions are:

• There is a possibility that an action initiated within the call that may delete the passed in object.
• The purpose of the call is to set up an observer_ptr to point at the passed in object.

In these cases the function should take the object as an observer_ptr by value. This will allow it to be called passing either the owner (unique_ptr) or an observer of it (observer_ptr).

void MyFunc(observer_ptr<T> rT);

observable_unique_ptr<T> apT = make_observable<T>();
observer_ptr<T> rT = apT;

MyFunc(apT); //OK

MyFunc(rT); //OK

Here is an example of a function that could initiate an action leading to deletion of the passed in object and which is rescued from disaster by the use of observer_ptr:

void ShootInFoot(observer_ptr<Foot> rFoot)
{
   if(rFoot)
      rFoot->DoSomething():

   MSG msg;
   if(PeekMessage(&msg, NULL, NULL, NULL, PM_NOREMOVE)
      DispatchMessage(&msg);  //Foot shooting may happen here

   if(rFoot)   //check we still have foot
      rFoot->DoSomethingElse();
}

Local variables that are raw pointers or references

Local variables that are raw pointers or raw C++ references are similar to arguments passed into functions in that the object they point at is often gauranteed to exist throughout their lifetime. It would not be sensible to use observer_ptr in those contexts. However particular care is needed when referencing elements of collections with local variables. The following is a very efficient way of working with elements of a collection.

vector<T> v;

//fill vector
//...

T& t=v[3];
t.DoSomething();
t.DoSomethingElse();

but the following code is unsafe

T& t=v[3];
t.DoSomething();
t.DoSomethingElse();

v.push_back();

t.DoSomeMore();

The addition of a new element to the collection may cause the entire collection to be re-allocated elsewhere, leaving t invalid. Using a pointer instead and testing would not help because testing it would give a false result

T* pT=v[3];
pT->DoSomething();
pT->DoSomethingElse();

v.push_back();

if(pT) //unreliable test - pT could be invalid and still non-zero
   pT->DoSomeMore(); //CRASH

This can be solved using observer_ptr but this requires that your collection holds observable_unique_ptrs to the objects. So we can rewrite it as

vector<observable_unique_ptr<T>> v;

//fill vector
//...

observer_ptr<T> rT=v[3];
rT->DoSomething();
rT->DoSomethingElse();

v.push_back();

if(rT) //references the object which has not moved
   rT->DoSomeMore(); //ok

This is a very useful and safe solution. If the vector has moved its elements to a new address then rT will remain valid because it is a reference to the object pointed to by the observable_unique_ptr (which has not moved), not the observable_unique_ptr itself (which may have moved). Furthermore if an operation is performed that results in the element it references being deleted then rT will test as null.

However this solution has forced you to change how your objects are held, increased the overhead and has made your code more ugly. Under certain circumstances you may prefer to solve your problem by scoping your use of local pointers and references more tightly so they cannot persist outside of the code in which they are safe:

vector<T> v;

//fill vector
//...

{  //new scope
   T& t=v[3];   //initialise within tight scope
   t.DoSomething();
   t.DoSomethingElse();
}  //close scope before performing operation on collection

v.push_back();  //array may move but its v[3] element has not been removed

{  //new scope
   T& t=v[3];  //initialise new reference within tight scope
   t.DoSomeMore();
} //close scope to prevent any leakage of reference t into further code

Before you seek to replace local varaibles that are raw pointers or references with observer_ptr, you should seek to scope those local variables so they cannot persist invalid.

Custom deleters

If you want to provide your own custom deleter then you can pass it in to observable_unique_ptr as the second template parameter, exactly as you would with unique_ptr  

observable_unique_ptr<T, my_deleter<T>> apT(my_get_new_object<T>());

which expands (through the using directive) to...

unique_ptr<T, observable<T, my_deleter<T>>> apT(my_get_new_object<T>());

Custom make function

You will not be able to use make_observable<T>() to initialise it and will have to write your own make function that creates the object in a way tht is compatible with your custom deleter.

template<class T, class... _Types>
inline observable_unique_ptr<T, my_deleter<T>>
make_observable_my_way(_Types&&... _Args)
{
    return observable_unique_ptr<T, my_deleter<T>>
    (
        my_get_new_object<T> (_STD forward<_Types>(_Args)...)
   );
}

Accomodating types inheriting enable_observable_this

If you want to be able to use your custom deleter with classes that inherit enable_observable_this then there are two more steps that you need to make:

• Instead of passing your object creation function the class type T, pass it superclassed as to_be_held_by_observable_unique<T>. 

template<class T, class... _Types> 
inline observable_unique_ptr<T, my_deleter<T>>
make_observable_my_way(_Types&&... _Args)
{
    return observable_unique_ptr<T, my_deleter<T>>
    (
        my_get_new_object<to_be_held_by_observable_unique<T>> (_STD forward<_Types>(_Args)...)
   );
}

• Register your creation function as a friend of to_be_held_by_observable_unique<T> by defining FRIENDS_OF_HELD_BY_OBSERVABLE_UNIQUE immediately before including oberver_ptr.h as follows:

#define FRIENDS_OF_HELD_BY_OBSERVABLE_UNIQUE \
 template<class T, class... _Types>  friend T* my_get_new_object(_Types&&... _Args);

#include "observer_ptr.h"

If you want to use more than one custom deleter then simply extend the list of friends defined...

#define FRIENDS_OF_HELD_BY_OBSERVABLE_UNIQUE \
 template<class T, class... _Types>  friend T* my_get_new_object(_Types&&... _Args); \
 template<class T, class... _Types>  friend T* my_other_get_new_object(_Types&&... _Args);

#include "observer_ptr.h"

The application of to_be_held_by_observable_unique<T> can be made at any point at which you are able to make it in the call stack leading to the object creation function. However the functions that are made friends of to_be_held_by_observable_unique<T> must be the functions that directly create the object and call its constructor. Even if those functions are buried in read only library code you can still declare them as friends in this way without touching the library code.

How it works

The mechanism

oberver_ptr carries a pointer to the object it references and also a pointer to a seperately allocated ref_counted_validity_flag (1 DWORD of memory) that indicates the validity of the object. The observable deleter installed in observable_unique_ptr also carries a pointer to the same flag. That is to say an owner and all observers of it point at the same ref_counted_validity_flag. The essential mechanism is that the owner marks the flag as invalid when it deletes the object and the observers check it for validity before accesing the pointee.

The top bit of the ref_counted_validity_flag indicates both validity (valid if set) and that it is referenced by an owner. The remaining bits are a count of how many observers are referencing it. 

The ref_counted_validity_flag must persist in memory as long as anything still may refer to it and may be required long after the object and its owner have gone out scope. For this reason it is not destroyed by the owner but instead self destructs when the count of remaining references to it falls to zero. 

In the following diagram an object has been created and is owned by an observable_unique_ptr. That sets the 'pointer to object' member of unique_ptr referred to as pT. The pointer to ref_counted_validity_flag member of the observable deleter, pRC ,remains initially NULL.

When the first observer_ptr is intialised to point at the owner, its  pT and pRC members are initialised from those of the owner but if the pRC member of the owner is NULL then a ref_counted_validity_flag is created for the pRC members of both owner and observer to point at. The initial value of the ref_counted_validity_flag is 0x80000001 - top bit set and a count of 1 observer.

The next observer to reference the owner will increment the observer count making the value 0x80000002 as shown in the diagram. The diagram shows the steady state during the life of the object. When an observer_ptr is used, the pRC member is used to check the top bit of the ref_counted_validity_flag that it points at. It finds it to be set and therefore returns the pointee (its pT member) to be used.
 

The next diagram shows what happens when the object is deleted. The unique_ptr calls the observable deleter to do the deletion which it does by calling the passed in deleter (by default default_delete) but before that, it uses its pRC member to unset the top bit of the ref_counted_validity_flag and then zeroes its pRC member which disconnects it from the ref_counted_validity_flag. It doesn't mater now if the observable_unique_ptr falls out of scope. The observer_ptrs themselves have not changed except that their pRC members now point at a ref_counted_validity_flag that has been marked invalid. This will tell the observer_ptr to ignore its pT member when it is next used.

When one of the observer_ptrs is tested (all use of them involves testing first) it will first use its pRC member to test the ref_counted_validity_flag. When it finds that it is invalid, it decrements its observer count and zeroes its pRC member (disconnecting itself from the ref_counted_validity_flag) and finally returns NULL. From now on, that observer_ptr will test as null simply because its pRC member is null. There is no need to zero its pT member because the NULL pRC will ensire that pT is never accesed.

When the last observer_ptr is tested, its decrement of the observer count in the ref_counted_validity_flag reduces it value to zero. This causes  the ref_counted_validity_flag to self destruct.

All of the smart pointers now test immediately as NULL without reference to any ref_counted_validity_flag.

It is also possible that both observer_ptrs could have been zeroed while the owner still holds a valid object. This will leave the owner pointing at a  ref_counted_validity_flag that isn't needed anymore expect that the owner still points at it. 

The ref_counted_validity_flag will destroy itself when the object is deleted and unsetting the top bit reduces the compound count to zero.

Design considerations

The design had to accomodate the fact that the smart pointers themselves may move in memory, even though the objects they point at do not. This means that observable_unique_ptr and observer_ptr can hold pointers to their  ref_counted_validity_flag but the ref_counted_validity_flag cannot hold pointers back to observable_unique_ptr or observer_ptr, they can become invalid if the smart pointers are moved in memory. It is for this reason that:

1. observer_ptrs cannot be directly informed of object deletion. Instead, they have to refer to the ref_counted_validity_flag when next used.
2. observable_unique_ptrs cannot be directly informed that no observers remain and the ref_counted_validity_flag is no longer needed. Instead, the ref_counted_validity_flag has to remain in memory so that the observable_unique_ptr can point at it until the observable_unique_ptr itself is zeroed.

The impact of enable_observable_this

The enable_observable_this add-on base class also carries a pRC member that may point to a ref_counted_validity_flag. This has to be there so that get_observer_of_this(this) can properly set up the observer_ptr that it returns. The complication is that any class inheriting enable_observable_this can only be held by an observable_unique_ptr and this also carries a pRC member. We have a duplication. Not only does this waste memory but if we run with it then we have to develop stategies to synchronise them.

It is not possible for class inheriting enable_observable_this to access the pRC member of its owner but it is possible for the owner to  access the pRC member of the class object if it knows that it inherits enable_observable_this.

This problem is resolved by a some compile time type selection that creates a different version of the observable deleter for classes inheriting enable_observable_this which does not carry a pRC member and causes the pRC member of the class object to be used instead.

Having two possible implementations of the observable deleter and two ways of holding a pointer to the ref_counted_validity_flag depending on the type of T adds complexity to the code but the type selection it is sufficiently well encapsulated so as not to cause confusion, as you will see in the description of code that follows.

How it is coded

We can more or less start at the top of observer_ptr.h and work towards the bottom. This is best read in conjuction with complete access to observer_ptr.h as I will only repeat small fragments in the narrative. For brevity I will use the term observable_unique_ptr<T> even though it is not defined until the end of the file and throughout the code is expressed in full as unique_ptr<T, observable<T>>

Each of the following headings represent a section of of observer_ptr.h. They are:

observable deleter, 
observer_ptr, 
enable_observable_this, 
public free functions, 
observable_unique_ptr

//----------------------------------observable deleter---------------------------

The first item in the section is the _PRIVATE_observable class which hides the internals of the observable deleter from the public interface.

This begins with the defiition of the ref_counted_validity_flag class. It consists of just one unsigned int member whose top bit indicates validity, the remaining bits representing the observer count. It has the following public methods to simplify and control its use:

//sets top bit only
 inline ref_counted_validity_flag()
     : m_compound_count(1 << top_bit_power_of_2);

 // tests if top bit is set
 inline bool get_valid() const ;

 //unsets top bit and self destructs if result is zero
 void mark_invalid() ;

 //increments observer count
 inline void add_weak() ;

 //decrements observer count and self destructs if result is zero
 inline void release_weak() ;

Also in the _PRIVATE_observable class are two alternative definitions for the observable deleter:

• observable_with_pRC<T, D> which deals with normal types (not inheriting enable_observable_this) and has an m_pRC member.
• observable_for_observable_this<T, D> which deals with types inheriting enable_observable_this and has no data members

observable is defined as one or the other according to the type of T just after the end of the _PRIVATE_observable class  in the public namepace with a using directive:

//observable<T, D> - Chooses correct implementation of observable deleter for type T
template <class T, class D = default_delete<T>> 
using 
    observable = typename conditional
    <//condition
     is_base_of < enable_observable_this, T >::value,
     //type if true and type if false
        _PRIVATE_observable::observable_for_observable_this<T, D>,
        _PRIVATE_observable::observable_with_pRC<T, D>
    > ::type;

On deletion the unique_ptr will call the observable deleter's operator()(T* p) method:

• observable_with_pRC implements this as

//Invalidate RC and call passed in deleter to do delete
        void operator()(T* p)
        {
            if (m_pRC)
            {    
                m_pRC->mark_invalid();
                m_pRC = NULL;
            }
            D()(p); 
        }

• and observable_for_observable_this implements it as 

//just calls passed in deleter to do delete 
        inline void operator()(T* p)
        {
            //Do nothing
            D()(p); //just call the passed in deleter
        }

Other features of the alternative implementations for the observable deleter that require comment are:

• They both use private inheritance of the passed in deleter

template <class T, class D = default_delete<T>>
struct observable_with_pRC
        : private D  //no implicit conversion to D

This prevents automatic implicit conversion from observable<T, D> to its base class D<T> which in turn prevents ownership from being transferred from unique_ptr<T , obervable<T, D<T>>> to unique_ptr<T , D<T>>. This is to prevent std::move() from transferring ownership from an observable owner to a non-observable owner which would leave observers orphaned. It obliges you to use move_to_unobservable() instead.

• observable_with_pRC provides a constructor that takes type D , the passed in deleter, as an argument

//conversion permits move from unique to observable_unique
        inline observable_with_pRC(const D d) : m_pRC(NULL)    
        {}

This allows automatic implicit conversion from  D<T> to observable<T, D> which would not happen by default. It allows std::move() to transfer ownership from a non-observable owner to an observable owner which is not a problem, no information is lost.

• They both have a polymorphic constructor

//required for polymorphism
       template<class U, class D2,
           class = typename enable_if<
               //check types are convertable
               is_convertible<U *, T *>::value
                //also check passed in deleters are convertable
                && is_convertible<D2, D >::value,
            void>::type>
        inline observable_with_pRC(const observable_with_pRC<U, D2>& odi)
            : m_pRC(odi.m_pRC)
        {}

This is required in deleters and can be found in default_delete. The difference here is that it also has to check that the passed in deleters (that do the deletion) are convertable.

Also the fact that the two alternative deleters observable_with_pRC and observable_for_observable_this are in no way related to each other ensures that there can never be any conversion between classes that inherit enable_observable_this and those that don't

//------------------------observer_ptr-------------------------------

This section also starts with a private class _PRIVATE_observer_ptr to hide its internals. Within this private class are two structs rc_source_in_observable_deleter and rc_source_in_observable_this that provide alternative implementations of a method that returns a reference to the m_pRC member being used and a function that calls whichever has been type selected according to the type of T.

//Alternative RC sources
    struct rc_source_in_observable_deleter
    {
        template <class T, class D> 
        inline static ref_counted_validity_flag*& get_ref_pRC
            (unique_ptr<T, observable<T, D> >const& ptr)
        {
            return ptr.get_deleter().m_pRC;
        }
    };
    struct rc_source_in_observable_this 
    {
        template <class T, class D> 
        inline static ref_counted_validity_flag*& get_ref_pRC
            (unique_ptr<T, observable<T, D> >const& ptr)
        {
            return ptr->m_pRC;
        }
    };
    
    //function to get RC source
    template <class T, class D>
    inline static ref_counted_validity_flag*& get_ref_pRC
        (unique_ptr<T, observable<T, D> >const& ptr)
    {
        return typename conditional
            < //condition
             is_base_of < enable_observable_this, T >::value,
             //type if true and type if false
                rc_source_in_observable_this,
                rc_source_in_observable_deleter
            > ::type::get_ref_pRC<T, D>(ptr);
    }

get_ref_pRC() must return a reference to m_pRC because there may be a need to change the value of it.

When the class type does not inherit enable_observable_this,

• a reference to the m_pRC of the observable deleter ptr.get_deleter().m_pRC is returned

but when the class type inherits inherit enable_observable_this,  

• a reference to the m_pRC of the class object ptr->m_pRC is used.

Also within _PRIVATE_observer_ptr is observer_ptr_base, the untemplated base class for observer_ptr<T>. It holds the m_pRC member of observer_ptr<T>, assures that it is properly intialised and destroyed and provides a set of methods for operations associated with it that can be called by observer_ptr<T>. These wrap both calls to methods of the ref_counted_validity_flag and changes to the value of the m_pRC member selected and returned by get_ref_pRC(). This reduces the amount of templated code (the suff of code bloat) that is generated by observer_ptr<T>. Its methods are:

//Autonomous operations on its own ref_counted_validity_flag 
    inline observer_ptr_base() ;
    inline ~observer_ptr_base();
        
 //Called operations on its own ref_counted_validity_flag
    void _release() const ;
    bool _check_valid_ref() const;
        
 //Called operations that accept another ref_counted_validity_flag
    void _point_to_observable_owner
        (ref_counted_validity_flag*& pRC_class);
    void _point_to_ref
        (ref_counted_validity_flag*& pRC_src);

observer_ptr_base also provides a definition of a null_ref_ptr class that will only be known to itself and observer_ptr. It is used in observer_ptr as an argument for construction and assignment to NULL

After the _PRIVATE_observer_ptr class is the public definition of observer_ptr<T>. This holds a m_pT member, the pointee, and also, by virtue of inheriting observer_ptr_base, it holds a m_pRC member.

The key methods that gives observer_ptr is special quality are of course the boolean test

inline operator const bool() const
{
    return _check_valid_ref();
}

and the deference operator

T* const operator->() const //can throw exception
{
    if (_check_valid_ref())
        return m_pT;
    throw _PRIVATE_observer_ptr::null_dereference_exception();
}

in line with this the comparison methods call its get() method to get at the pointee

inline T* get() const
{
    return (_check_valid_ref()) ? m_pT : NULL;
}

The construction and assignment methods determine the behaviour of observer_ptr and require some comment - for brevity we will just look at the constructors:

• From NULL

inline observer_ptr(null_ref_ptr* pNull)
        : m_pT(NULL)
    {}

With what kind of argument do you accept a NULL? We want to be able to explicitly null an observer_ptr but we don't want to be able to initialise it with any other number nor any other pointer value. By taking a pointer to a type that nobody else can know about, null_ref_ptr, the only acceptable value that can be passed to it is NULL (the only pointer value that is typeless).

• From another observer_ptr

inline observer_ptr(observer_ptr const & ptr)
    {
        if (m_pT = ptr.get()) //assignment followed by test
            _point_to_ref(ptr.m_pRC);
    }

template <class U>
    inline observer_ptr(observer_ptr<U> const & ptr)
    {
        if (m_pT = ptr.get()) //assignment followed by test
            _point_to_ref(ptr.m_pRC);
    }

This allows the proliferation of observers from observers, something you can readily do with observer_ptr. The first is the copy constructor which must be defined expilcitly to prevent the compiler from generating its own defaults. The second is polymorphic. 

• From observable_unique_ptr

template <class U, class UDel = std::default_delete<U>>
    inline observer_ptr
        (unique_ptr<U, observable<U, UDel>>const& ptr)
    {
        if (m_pT = ptr.get()) //assignment followed by test
            _point_to_observable_owner
                (_PRIVATE_observer_ptr::get_ref_pRC<U, UDel>(ptr));
    }

template <class U, class UDel>
        observer_ptr(unique_ptr<U, UDel>const&& ptr) = delete;

This makes use of rValue reference discrimination in the same way as unique_ptr but with the opposite logic. You can construct an observer_ptr from an observable_unique_ptr but not if it is going out of scope. The deleted constructor that takes only an rValue reference && will catch observable_unique_ptrs that are going out of scope. This allows an observer_ptr to be initialised by an observable_unique_ptr but not by the temporary observable_unique_ptr returned by functions such as std::move() and make_observable<T>() that are transferring ownership.

observer_ptr also has private constructor that is only called by its friend class enable_observable_this and friend function static_pointer_cast<T>(). Its purpose it to allow those components to directly set its m_pT and m_pRC members.

inline observer_ptr(T* pT, ref_counted_validity_flag*& pRC)
        : m_pT(pT)
    {
        _point_to_observable_owner(pRC);
    }

//----------------enable_observable_this (intrusive)------------------------

In this case we will skip past the private class _PRIVATE_observable_this and look first at the definition of the publicly available enable_observable_this add-on base class. Here it is in full:

//Add on base class giving smart observer of the this pointer
class enable_observable_this
    : protected _PRIVATE_observable_this
{
    friend struct _PRIVATE_observer_ptr::rc_source_in_observable_this;
private:        
    mutable ref_counted_validity_flag* m_pRC;
    
    //pure vf forces use of complete_observable_this<T> for creation
    virtual void unused(hidden h) = 0; 
protected:
    inline enable_observable_this() : m_pRC(NULL)
    {}
    inline ~enable_observable_this()
    {
        if (m_pRC)
            m_pRC->mark_invalid();
    }
    
    //methods only available within class definition
    template <class U> observer_ptr<U> get_observer_of_this(U* const pThis)
    {
        if (NULL == m_pRC)
            m_pRC = new ref_counted_validity_flag;
        //calls observer_ptr private constructor
        return observer_ptr<U>(static_cast<U*>(this), m_pRC);
    }
    
    void zero_observers()
    {
        if (m_pRC)
            m_pRC->mark_invalid();
        m_pRC = NULL;
    }
};

Its operation is quite simple. It carries a m_pRC member (pointer to ref_counted_validity_flag). get_observer_of_this() creates a ref_counted_validity_flag for it to point at if it doesn't already exist and the destructor will mark the ref_counted_validity_flag invalid if there is one. There is also a method to zero all observers without deleting the object.

It also has a pure virtual function:

virtual void unused(hidden h) = 0; 

This is the key to enforcing the limitations on creation and ownership that apply to types that inherit enable_observable_this. Having declared a pure virtual function, it cannot be instantiated unless a derived class provides an implementation of that function. Furthermore you cannot provide that function in the classes that you derive from enable_observable_this because hidden is a private type that you don't have access to. It is this that forces you to superclass types that inherit enable_observable_this with to_be_held_by_observable_unique<T> for object creation

to_be_held_by_observable_unique<T> is defined just below enable_observable_this 

//to_be_held_by_observable_unique<T> - Superclass for object creation
template <class T>
using to_be_held_by_observable_unique = 
   typename conditional
   < //condition
    is_base_of < enable_observable_this, T >::value
    && 
    !is_base_of //guard against multiple application
     <_PRIVATE_observable_this::complete_observable_this<T>, T>
        ::value,
    //type if true and type if false
        _PRIVATE_observable_this::complete_observable_this<T>,
        T //has no effect on types not inheriting enable_observable_this
   > ::type;

Its definition is conditional on the type of T. If the type inherits enable_observable_this then the type is defined as complete_observable_this<T> defined in the _PRIVATE_observable_this class otherwise it is simply defined as T 

The  _PRIVATE_observable_this class hides the definition of the private hidden struct.

struct hidden{};

and the complete_observable_this<T> class

    //to_be_held_by_observable_unique<T> selects this for enable_observable_this types
    template <class T> class complete_observable_this
        : public T
    {
        //implementation of pure vf declared in enable_observable_this
        void unused(hidden h)
        {}

        //friend functions
        template<class T,
        class... _Types>
            friend typename enable_if < !is_array<T>::value,
            unique_ptr<T, observable<T>> > ::type make_observable
            (_Types&&... _Args);

        template<class T>
        friend typename enable_if<is_array<T>::value && extent<T>::value == 0,
            unique_ptr<T, observable<T>> >::type make_observable
            (size_t _Size);

        FRIENDS_OF_HELD_BY_OBSERVABLE_UNIQUE
   
        //private constructor can only be called by friends
        template<class... _Types>
        complete_observable_this(_Types&&... _Args)
            : T(_STD forward<_Types>(_Args)...)
        {}
    
    };

which provides an implementation of the unused(hidden h) pure virtual function.

implementation of pure vf declared in enable_observable_this
        void unused(hidden h)
        {}

It doesn't do anything and it never gets called but it allows the compiler to instantiate the class. 

Then it defines the two active forms of make_observable<T>() as friends and any creation function you have defined in the macro FRIENDS_OF_HELD_BY_OBSERVABLE_UNIQUE. This is important because the all of its members are private including its constructor, so only those friend functions will be able to create objects of this type.

//-------------------------public free functions---------------------------------------

make_observable<T>(...) is modelled on make_unique<T>(...) except that new is passed the type superclassed by to_be_held_by_observable_unique<T> and it returns an observable_unique_ptr instead of a plain unique_ptr.

//make_observable<T>() - adapted from make_unique<T>()
template<class T, class... _Types> inline
typename enable_if<!is_array<T>::value,
unique_ptr<T, observable<T>> >::type make_observable(_Types&&... _Args)
{
    return unique_ptr<T, observable<T, default_delete<T>>>
        (new to_be_held_by_observable_unique<T>(std::forward<_Types>(_Args)...));
}

zero_observers() is declared a friend of _PRIVATE_observer_ptr so that it can use its private get_ref_pRC<T, D>(ptr) function

//zero_observers() - Zeroes all observers of owner passed in
template<class T, class D = std::default_delete<T>>
void zero_observers
(unique_ptr<T, observable<T, D> >& ptr)
{
    if (_PRIVATE_observer_ptr::get_ref_pRC<T, D>(ptr))
    {
        _PRIVATE_observer_ptr::get_ref_pRC<T, D>(ptr)->mark_invalid();
        _PRIVATE_observer_ptr::get_ref_pRC<T, D>(ptr) = NULL;
    }
}

move_to_unobservable() is made unavailable for types inheriting enable_observable_this. It simply calls zero_observers() before making a unique_ptr from a pointer released by the owner.

//move_to_unobservable() -Transfer from observable to non-observable
template<class T, class D = std::default_delete<T>>
    //disallow if T inherits enable_observable_this
    typename enable_if<!is_base_of<enable_observable_this, T>::value,
unique_ptr<T, D > >::type move_to_unobservable
    (unique_ptr<T, observable<T, D> >& ptr)
{
    zero_observers(ptr);
    return unique_ptr<T, D >(ptr.release());
}

static_pointer_cast<T>() will compile if T will survive a static cast. It is declared a friend of observer_ptr and uses its private constructor to form an observer_ptr to return.

//static_pointer_cast() - Static cast function for observer_ptr<T>
template<class T, class U>
observer_ptr<T> static_pointer_cast(observer_ptr<U>& ptr)
{
    //calls observer_ptr private constructor
    return observer_ptr<T>(static_cast<T*>(ptr.m_pT), ptr.m_pRC);
}

//--------------------------------observable_unique_ptr-----------------------

Finally observable_unique_ptr is defined by a using directive at the end of observer_ptr.h to clarify that nothing else in the file makes use of this definition and you can use an alternative name without breaking anything.

//observable_unique_ptr contraction defined here
template <class T, class D = std::default_delete<T>>
using
    observable_unique_ptr = unique_ptr < T, observable<T, D> >;