Python vs C++ Series: Development Environment Setup and Virtual Environment

This article is part of the Python vs. C++ Series. Unlike the previous articles focusing on language differences, this article talks about the setup of development environments. Different languages have different toolsets and, of course, different ways to set up their working environments. When we start learning a new language, the first thing we ask is probably how we set up a working environment. If we are already familiar with another language, that experience could speed up our learning process. C++ was my primary language when I started learning Python. I was hoping someone could tell me how to start working with Python using the concept or terminology I already knew. That’s why I wrote this article; hopefully, it will benefit others. Although this article is written for C++ background, it could also be helpful for people who want to know the setup of the Python working environment.

(Note that the Python code in the series assumes Python 3.7 or newer.)

C++ Development Environment Setups

People may have different preferences and working styles, but as C++ programmers, the bare minimum may include the following:

1. Install the toolchain, such as GCC or MSVC, based on the platform and the C++ version.
2. Install dependent libraries, e.g., Boost Library.
3. Start writing the code.

Python Development

Not surprisingly, Python has a similar workflow with an additional step – setting up a virtual environment.

1. Install Python based on the choice of Python version.
2. Set up a virtual environment.
3. Install dependent libraries.
4. Start writing the code.

Virtual Environment

When we think of virtual environments, virtual machines (e.g., VirtualBox) and containers (e.g., Docker) are probably the first things that come to our minds. However, Python virtual environment is a much simpler mechanism than a virtual machine and a container. The concept of the Python virtual environment is a mechanism allowing the creation of environments with various Python versions and their own independent set of installed Python packages in the environments’ site directories. It is nothing but a directory that contains the Python interpreter, libraries, and their installed packages and allows us to have an isolated environment.

Why Do We Need a Virtual Environment?

Consider the scenario in which we work on multiple projects, each with different dependencies or versions. In C++, working on this scenario could be unpleasant, especially when the versions are incompatible. If we don’t have multiple machines, we may need virtual machines or use containers to have isolated environments for each project. But in Python, we can leverage the virtual environment mechanism to have isolated environments for each project, which is much cheaper than virtual machines or containers.

How to Use It?

Since Python 3.3, the language has natively supported the virtual environment. This session will demonstrate the primary usage of Python virtual environment by creating a virtual environment called myenv in Linux.

• Create a Virtual Environment

  Shell

  user:~$ python -m venv myenv

  This command will create a folder named myenv and install its Python binary and default packages in the folder (the virtual environment can be any location with any preferred name).

• Activate the Virtual Environment

  After the virtual environment is created, we can activate it by invoking the activate script.

  Shell

  user:~$ source ./myenv/bin/activate

  Once the virtual environment is activated, the name of the virtual environment will show on our shell.

  Shell

  (myenv) user:~$

  The text (myenv) in the command prompt indicates that the current shell is in the myenv environment. Once we are in the virtual environment, the Python interpreter will be pointed to the one in the virtual environment. We can verify it by running the which python command, showing that the Python interpreter is now in the myenv folder.

  Shell

  (myenv) user:~$ which python 
  /home/user/myenv/bin/python

  Similarly, if we install a Python package, the package will be installed into myenv’s site directory, /home/user/myenv/lib/python3.7/site-packages/ in this example.

• Deactivate the Virtual Environment

  Once we are done with our coding or want to exit the virtual environment, we can use the deactivate command to leave the virtual environment. After the virtual environment is deactivated, our shell becomes normal.

  Shell

  (myenv) user:~$ deactivate 
  user:~$

• Tear Down the Entire Virtual Environment

  Since a Python virtual environment is just a folder, if we no longer need it or want to start it over, we can do so by deleting the virtual environment folder, myenv, in this case – no leftover.

How Does It Work?

When executing the python -m venv myenv command, the command creates a folder named myenv and creates a symbolic link of the Python binary that we perform the command to the myenv folder. The command also copies other related files from the Python interpreter that we perform the command to the myenv folder. In other words, the virtual environment is a copy of the Python environment that we use to create the virtual environment. For example, if our system has Python 3.7 and we use it to create a virtual environment called myenv, the virtual environment myenv is just a clone of the system Python 3.7.

When we activate it, the activate script updates the shell by appending the path of the virtual environment to the PATH (if Linux) variable. The following code snippet is the portion of the activate script that exports the path.

…
VIRTUAL_ENV="/home/user/myenv"
export VIRTUAL_ENV

_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
…

Therefore, we can use the following picture to summarize the concept of the Python virtual environment.

When we need to install a package, whether pip or other package installers, the package will be installed into the myenv/lib/python3.7/site-packages/ folder in our example. Thus, we can use virtual environments to manage the dependencies for each project without conflicting with others.

Notes of Python Virtual Environment

Although the Python virtual environment is very convenient, there are things we need to be aware of when working with virtual environments.

First, the isolation only happens on the Python level. Unlike virtual machines that provide an entire virtualized computer system, Python virtual environment is just a copy of an existing Python with its site directories (i.e., the place we install Python packages). If our code or the Python package has external dependencies such as C/C++ libraries, the Python virtual environment cannot isolate them and will use whatever is available in the system.

Second, when we create a virtual environment, its Python version will be the same as the Python binary we use to run the create command. For example, if our system only has Python 3.7, we can only create a virtual environment that has Python 3.7. Therefore, if we want to create a virtual environment with Python 3.9, we need to use Python 3.9 to create a virtual environment that has Python 3.9.

However, installing multiple versions of Python in our system is a pain and hard to switch between. Fortunately, many tools try to simplify the work. One of the most popular solutions is pyenv. Pyenv allows us to install multiple Python versions and provides an easy way to switch between them (Check Simple Python Version Management: pyenv for more information). With that, our workflow of creating a working environment becomes the following:

1. Use pyenv to switch to the desired Python version.
2. Use the desired Python version to create a virtual environment.
3. Activate the virtual environment.
4. Install the dependencies into the virtual environment.

Third, since a virtual environment is just a directory with a clone of the Python interpreter and its site directories, creating a virtual environment within a project is not recommended, especially if a project is managed by a source control system such as Git. Instead, it is common to create virtual environments under a common location, for example, .venv.

Package Installation

It’s unlikely that we write our program using only the language’s standard library. We often leverage other libraries to build our program. In C++, we usually go to the library’s website, download it, and install it. And the way to install a library depends on the library – use an installer if provided, build it ourselves, or even copy it to the appropriate location (e.g., /usr/local/lib/) manually. In Python, the most common place to find a library is PyPI, and the most common way to install a Python library is the pip tool.

PyPI stands for Python Package Index, a repository of software for the Python language. It can be seen as a central place where host Python projects. So, we can find most Python open source from PyPI.

Although there are many ways to install Python packages, pip (Package Installer for Python) is the de facto and recommended tool to install and manage Python packages. Out of the box, pip connects to PyPI, so installing a Python package can be as simple as the following (use NumPy as an example).

$ python -m pip install numpy

The command will download the NumPy package from PyPI and install it into the site-packages folder of the Python environment, e.g., myenv/lib/python3.7/site-packages/.

Note that we may see many documents or package installation instructions, say using pip to install a package without python -m in the command. For example:

$ pip install numpy

Although this is valid, using python -m with pip is recommended. The reason is that the -m argument (see -m for more detail) tells the Python interpreter to run the pip as the main module. That means when using python -m with pip, we execute the pip located at the exact location as the Python interpreter we specify, e.g., myenv/bin/. Without using python -m, the pip to be used will depend on the PATH of our systems. Therefore, it doesn’t ensure which pip will be used.

Another note of pip is that pip can also be configured to connect a private or a local repository. We can still use pip if we have our repository for hosting Python packages. More details about installing Python packages can be found at Installing Packages — Python Packaging User Guide.

Conclusion

The significant difference between Python and C++ (and probably other languages) is the use of virtual environments, and we should always use virtual environments for Python developments.