Python objects’ immutability is defined by their types. Knowing which data types are mutable, which are not and the behavior when using mutable objects in certain situations is critical to avoid writing bug code.
The third article of the Python vs C++ Series is about immutability – an object cannot be modified after it is created.
(Note that the Python code in the series assumes Python 3.7 or newer.)
Const and Constexpr in C++
C++ supports two notions of immutability: const and constexpr. To declare an object is immutable, we use either const or constexpr when defining an object. Of course, there are more details than this when we consider the immutability in C++, but in general, all objects are mutable by default.
void main()
{
int nonConstVariable = 0; const int constVariable = 0; constexpr int secondsPerHour = 60 * 60; }
This article uses the word mutable in general terms, so don’t be confused with the C++ mutable keyword, which allows a class member to be changeable even if its class instance is const or allows a class member modified by a const method.
class MyClass
{
public:
int variable1 = 0;
mutable int variable2 = 0;
};
void main()
{
const MyClass myClass; myClass.variable2 = 10; myClass.variable1 = 10; }
Mutability and Immutability in Python
Unlike C++, where every object is mutable by default, Python objects’ immutability is determined by their type. The list below summarises some common mutable and immutable data types.
(See Built-in Types for more detail.)
What Does Immutable Mean in Python?
When we are new to Python, we might think everything is mutable in Python because we can update whatever objects we want. For example, the following code will work without an issue.
variable = 10
variable = "string"
variable = 2.0
variable = [1, 2, 3]
However, the meaning of updating objects is different between immutable objects and mutable objects. When assigning an object to a variable, we can think that the variable is a named pointer pointing to the object (we will talk more about this in the Copy Assignment section). If the object is immutable when we update the variable, we actually point it to another object, and Python’s garbage collection will recycle the original object if it is no longer used. On the contrary, if a variable points to a mutable object, the mutable object will be modified when we update the variable.
We can use a built-in function id to verify if an object we updated is still the same object. The id function returns the identity (the object’s address in memory) of an object. The following example shows how immutability behaves in Python and how we use the id function to check objects’ identities. Also, we use the hex function to convert the id output to hexadecimal format, so it looks more like a memory address.
integer = 10
print(f"integer: {integer}; address: {hex(id(integer))}")
integer = 20
print(f"integer: {integer}; address: {hex(id(integer))}")
string = "hello"
print(f"string: {string}; address: {hex(id(string))}")
string = "world"
print(f"string: {string}; address: {hex(id(string))}")
list_var = [1, 2, 3]
print(f"list_var: {list_var}; address: {hex(id(list_var))}")
list_var.append(4)
print(f"list_var: {list_var}; address: {hex(id(list_var))}")
dict_var = {"key1": "value1"}
print(f"dict_var: {dict_var}; address: {hex(id(dict_var))}")
dict_var["key2"] = "value2"
print(f"dict_var: {dict_var}; address: {hex(id(dict_var))}")
We can see that the address of variables integer and string is different before and after updating them, which means these two variables point to new objects (20 and world respectively) after updating. On the contrary, list_var and dict_var are mutable, so their addresses remain the same before and after updating them. Therefore, they still point to the same objects.
Why Knowing Python Objects’ Mutability and Immutability Is Essential?
That’s because if we are blind to Python objects’ immutability, we may be surprised by their behavior, and sometimes working with mutable objects without care leads to bugs. The following subsections will discuss a few scenarios that may not be intuitive for people from C++ backgrounds.
Use Mutable Value as Default Function Parameters
Python’s default argument feature allows us to provide default values for function arguments when defining a function. However, if our default values are mutable type, their behavior may not be desired. See an example below.
def my_function_1(parameter: List = []) -> None:
parameter.append(10)
print(parameter)
In this example, we define a function (my_function_1) and use an empty list as the default value. And then we try to call this function without providing a parameter several times.
my_function_1()
my_function_1()
my_function_1()
If we run the code, we will notice that the parameter keeps the value (i.e., 10) we append every time. Therefore, in the third time, the output becomes [10, 10, 10]. The behavior is actually similar to that we define a static variable in a function in C++ – the static variable is initialized only one time and holds the value even through function calls.
How to Avoid This Situation?
If using mutable type, such as list, is necessary (which is common), we should use None as the default value and Optional for type checking (Using Optional tells type checkers the value could be either None or the desired type). This way guarantees the parameter is new whenever we call the function without providing a parameter. See an example below.
def my_function_2(parameter: Optional[List] = None) -> None:
if parameter:
parameter.append(10)
else:
parameter = [10]
print(parameter)
This time, if we call the function (my_function_2) without providing a parameter several times, the parameter will not hold the value from the previous call.
my_function_2()
my_function_2()
In short, do not use mutable type as the default value of a function parameter. If the parameter needs to be mutable type, use None as the default value.
The Behavior of Class Variable
The second scenario happens when we use a mutable variable as a class variable. A class variable in Python is shared by all instances. The scenario is similar to the C++ class variable (i.e., declare a class variable with static). To define a class variable in Python, define a variable inside the class but outside any method. In the example below, we define a class (MyClass) with a mutable variable (mutable_member), an immutable variable (immutable_member), and a couple of instance variables.
from typing import List
class MyClass:
mutable_member: List = []
immutable_member: int = 0
def __init__(self) -> None:
self.immutable_instance_variable: int = 0
self.mutable_instance_variable: List = []
Since class variable is shared by all instances, if the variable is a mutable type, the change of the variable will affect all instances of the class. Before we update the class variables, let’s check their value and memory address.
print(f"MyClass.mutable_member: {MyClass.mutable_member}")
print(f"MyClass.mutable_member address: {hex(id(MyClass.mutable_member))}")
print(f"MyClass.immutable_member address: {hex(id(MyClass.immutable_member))}")
class1 = MyClass()
print(f"class1.mutable_member: {class1.mutable_member}")
print(f"class1.mutable_member address: {hex(id(class1.mutable_member))}")
print(f"class1.immutable_member address: {hex(id(class1.immutable_member))}")
class2 = MyClass()
print(f"class2.mutable_member: {class2.mutable_member}")
print(f"class2.mutable_member address: {hex(id(class2.mutable_member))}")
print(f"class2.immutable_member address: {hex(id(class2.immutable_member))}")
Here, we can see both mutable_member and immutable_member of class1 and class2 point to the same objects, which are the same as MyClass.
Now, let’s update the mutable class variable, and print out their values.
class1.mutable_member.append(10)
print(f"class1.mutable_member: {class1.mutable_member}")
print(f"class2.mutable_member: {class2.mutable_member}")
The update affects all instances of MyClass.
How about the immutable class variable?
The behavior is a little bit different when we update an immutable class variable. Now, let’s update the immutable_member from class1, and print out the address of both instances class1 and class2.
class1.immutable_member = 20
print(f"class1.immutable_member: {class1.immutable_member}")
print(f"class1.immutable_member address: {hex(id(class1.immutable_member))}")
print(f"class2.immutable_member: {class2.immutable_member}")
print(f"class2.immutable_member address: {hex(id(class2.immutable_member))}")
The output shows that class1.immutable_member no longer binds to the MyClass.immutable_member.
If we create a new MyClass instance class3, its class variables are still bound to MyClass’ class variables.
class3 = MyClass()
print(f"class3.immutable_member: {class3.immutable_member}")
print(f"class3.immutable_member address: {hex(id(class3.immutable_member))}")
print(f"MyClass.immutable_member address: {hex(id(MyClass.immutable_member))}")
Hence, class variables are shared by all instances. For a mutable class variable, if we modify it, the change will affect all instances, whereas, for an immutable class variable, if we change it from a class instance, the variable of this instance no longer binds to the original class variable.
Also, it is worth mentioning that instance variables are unique to each class instance regardless of its mutability.
class1.mutable_instance_variable.append(30)
print(f"class1.mutable_instance_variable: {class1.mutable_instance_variable}")
print(
f"class1.mutable_instance_variable address: "
f"{hex(id(class1.mutable_instance_variable))}"
)
print(f"class2.mutable_instance_variable: {class2.mutable_instance_variable}")
print(
f"class2.mutable_instance_variable address: "
f"{hex(id(class2.mutable_instance_variable))}"
)
Copy Assignment
Using the assignment operator (e.g., x = 10) in Python does not create copies of objects. Instead, it establishes a binding between the variables and the objects. The behavior is not a problem when working with immutable objects. The assignment operator will create a new binding between the variable and the new target for immutable objects. We can verify it using the is operator to check if two objects are the same object (we can also use the id function as we did in the previous examples). The following example shows the assignment operation behavior on immutable objects and the usage of the is operator.
my_string = "hello"
my_string_copy = my_string
print(my_string is my_string_copy)
my_string_copy += " world"
print(my_string is my_string_copy)
print(f"my_string: {my_string}")
print(f"my_string_copy: {my_string_copy}")
Copy Module and Shallow Copy
We know copy assignment only creates a binding between the object and the target, so how do we create a copy of an object? Python provides a copy module that offers shallow and deep copy operations. The following example uses copy.copy function to create a copy (value_copy) from a mutable object (value).
import copy
value = [1, 2, 3]
print(hex(id(value)))
value_bind = value
print(hex(id(value_bind)))
value_copy = copy.copy(value)
print(hex(id(value_copy)))
value_copy.append(4)
print(value_copy)
print(value)
From this example, the value_copy variable is independent of the value variable. Also, worth mentioning that the copy.copy function still creates a binding between the object and the target for an immutable object.
Shallow copy on mutable compound objects or immutable compound objects contains mutable compound objects
Shallow copy happens in C++ when coping with a compound object (e.g., a class) containing pointers; the copy copies the pointers but not the objects the pointers point to (See the diagram below).
Therefore, the object that the pointers of Object A and Object B point to become shared.
Since the copy.copy function in Python performs shallow copy, the situation mentioned above happens when copying a compound object with mutable compound objects, whether the top-level compound object is mutable or immutable. The following example demonstrates the shallow copy scenario.
import copy
compound_object = {"key1": 123, "key2": [1, 2, 3]}
print(hex(id(compound_object)))
compound_object_copy = copy.copy(compound_object)
print(hex(id(compound_object_copy)))
print(hex(id(compound_object["key2"])))
print(hex(id(compound_object_copy["key2"])))
compound_object_copy["key2"].append(4)
print(compound_object_copy)
print(compound_object)
Use copy.deepcopy function to perform a deep copy
If we want to perform deep copy, we should use copy.deepcopy instead. See the following example.
import copy
compound_object = {"key1": 123, "key2": [1, 2, 3]}
print(hex(id(compound_object)))
compound_object_copy = copy.deepcopy(compound_object)
print(hex(id(compound_object_copy)))
print(hex(id(compound_object["key2"])))
print(hex(id(compound_object_copy["key2"])))
compound_object_copy["key2"].append(4)
print(compound_object_copy)
print(compound_object)
Conclusion
Python objects’ immutability is defined by their types. Knowing which data types are mutable, which are not and the behavior when using mutable objects in certain situations is critical to avoid writing bug code.
(All the example code is also available at mutable_immutable_and_copy_assignment.)
