Managing C++ Projects with Conan and CMake

• Download source - 2.8 KB

C++ developers are a self-reliant bunch. We manage our memory at a low level, handle our own garbage collection, and never sacrifice performance for safety. The same holds true for managing dependencies. If a project requires a complex web of interdependent libraries, most C++ developers will write makefiles instead of relying on weird dependency-management tools.

When I first heard about Conan, I decided it was unnecessary. But as my dependency tree became a dependency forest, I became more interested. Could Conan really download and install libraries automatically? With specific versions? In a specific order? And handle dependencies among the dependencies?

Conan can do all of these things, and the goal of this article is to show how CMake and Conan can be combined together. I'll start by providing a basic introduction to CMake and then show how Conan can be used to manage packages for CMake builds.

1. Overview of CMake

When I work on embedded projects or Linux-only executables, I prefer the simplicity of GNU makefiles. For anything more complex, I use CMake. With CMake, it's straightforward to build multiple target configurations for multiple operating systems. It's not easy to learn, but its capabilities more than make up for the inconvenience.

CMake is freely available for download here. For most projects, using CMake involves three steps:

1. Define build instructions in CMakeLists.txt.
2. Execute the cmake command to read CMakeLists.txt and generate a buildsystem.
3. Execute the cmake --build command to build the target from the buildsystem.

This section walks through these steps and shows how they can be used to build a simple project named hello_cmake.

1.1 Writing CMakeLists.txt

Instructions in CMakeLists.txt are provided with statements calle commands. There are three points to know about commands:

• Every command has a name followed by zero or more arguments in parentheses.
• A command's arguments are separated by spaces instead of commas.
• Control flow relies on start/end commands instead of curly braces: if()/endif(), while()/endwhile(), foreach()/endforeach(), and so on.

I can't speak for all developers, but the commands in my CMakeLists.txt files tend to perform four main steps:

1. Set project-wide properties.
2. Define variables.
3. Create targets.
4. Set target properties.

This discussion presents the commands needed for each of these steps. Then I'll explain how CMake uses these commands to build targets.

Part 1: Project-Wide Properties

In all my projects, CMakeLists.txt begins with the same two commands:

• cmake_minimum_required(...) — sets the minimum version of CMake that can be used to build the project
• project(...) — defines project properties including its name and programming language (optional)

If you look in the hello_cmake project provided in this article, you'll find that the CMakeLists.txt file starts with the following two lines:

# Set the minimum version of CMake
cmake_minimum_required(VERSION 3.12)

# Set the project's name and properties
project(hello_cmake_project LANGUAGES CXX)

The project command has three arguments, but only the first is needed. This identifies the project's name as hello_cmake_project.

The argument CXX is optional. It's preceded by the LANGUAGES keyword to identify that it's the project's programming language. This combination of keywords and arguments can be confusing. But in general, keywords are usually capitalized and arguments usually aren't.

Part 2: Variables

The set command defines a variable that can be used throughout the build file. The simplest form of this command accepts two arguments: the variable's name and the variable's value.

To demonstrate, the following command creates two variables that identify directory names: one named SOURCE_DIR and one named INCLUDE_DIR. The value of SOURCE_DIR is src and the value of INCLUDE_DIR is include.

# Create a variable named SOURCE_DIR
set(SOURCE_DIR src)

# Create a variable named INCLUDE_DIR
set(INCLUDE_DIR include)

Once a variable is defined, its value can be accessed as ${NAME}, where NAME is the variable's name. As a result of the preceding code, the src directory can be accessed as ${SOURCE_DIR} and the include directory can be accessed as ${INCLUDE_DIR}.

Part 3: Targets

A target is a file to be constructed by the build. CMake supports two types of targets: executables and libraries. The command that defines an executable target is add_executable and the command that defines a library target is add_library.

In the hello_cmake project, the build file defines an executable target named hello_cmake and associates it with the main.cpp source file in the SOURCE_DIR folder. This is accomplished with the following command:

# Define a target executable
add_executable(hello_cmake ${SOURCE_DIR}/main.cpp)

Because of this command, hello_cmake will be the name of the executable produced by the build. It will also be the target's identifier in the commands that set target-specific properties.

Part 4: Target Properties

After a target is defined, a build file can set its properties in a number of ways. One method is to call set_target_properties with keywords that identify the properties to be set. Another method is to call commands that set individual properties. Five of these commands are given as follows:

• target_compile_options(target, ...) — set compile options
• target_include_directories(target, ...) — set directories containing header files
• target_link_directories(target, ...) — set directories containing libraries
• target_link_libraries(target, ...) — identify libraries required by the target
• target_link_options(target, ...) — set options for linking the required libraries

The hello_cmake project doesn't have any libraries, so the only properties that need to be set are the compile options and the include folder (given by the INCLUDE_DIR variable). This is accomplished with the following code:

# Set compile options for the target
target_compile_options(hello_cmake PUBLIC -Wall -std=c++17)

# Set the include directories for the target
target_include_directories(hello_cmake PUBLIC ${INCLUDE_DIR})

In both commands, the second option can be set to one of three values that identifies the command's scope:

• PUBLIC — the property is set for the given target and any target that uses the given target
• PRIVATE — the property is set only for the given target
• INTERFACE — the property is set for targets that use the given target, but not the given target

I tend to use PUBLIC in all my build files, and I'm sure the day will come when I regret this.

1.2 Generating the Buildsystem

After you've installed CMake, you'll be able to execute the cmake utility on the command line. This accepts several options and flags, but it's most important role involves reading CMakeLists.txt and generating a set of files and folders. CMake's documentation refers to these files and folders as a buildsystem.

Generating a buildsystem for the hello_cmake project requires five steps:

1. Download example_code.zip to your computer and decompress the archive.
2. Open a terminal or command prompt and change to the hello_cmake directory.
3. Create a build directory with the command: mkdir build
4. Change to the build directory with the command: cd build
5. Generate the buildsystem with the command: cmake ..

When the last step is performed, CMake will create the buildsystem inside the build folder. On a Linux/macOS system, the folder will contain one folder and three files:

• Makefile — a GNU makefile that builds the hello_cmake executable
• cmake_install.cmake — defines the install process (as opposed to the build)
• CMakeCache.txt — sets name/value pairs used during the build process
• CMakefiles — contains additional CMake files

The generated files depend on your operating system and CMake version, and most developers don't need to know what they are.

1.3 Building the Target

The final step of the build is the easiest. If the buildsystem has been created successfully and you're in the directory containing the generated files, you can perform the build with the following command:

cmake --build .

This tells CMake to look for a suitable makefile in the current directory and execute its instructions. If you run this command in the hello_cmake/build folder, CMake will compile and link the hello_cmake executable.

2. Understanding Packages

The preceding discussion explained how to build a trivial C++ project, but real-world projects have libraries. To account for these dependencies, the CMakeLists.txt file needs to be updated.

If a library file is available at a given location, it's easy to tell CMake how to link it into the target. For example, suppose the target foo needs the bar library, which is located in the baz folder. The following commands tell CMake how to link the library into the target:

# Identify the libraries required by the target
target_link_libraries(foo, bar)

# Identify the location of the target's libraries
target_link_directories(foo, baz)

Rather than keep track of where libraries are located, you can use packages. A CMake package contains header files and libraries, as well as CMake-specific files that provide metadata. Using packages provides three main advantages:

1. modularity - a package's libraries and header files are always kept separate
2. reusability - no hard-coded dependency locations, so CMakeLists.txt can be used with different operating systems
3. version management- it's easy to keep track of package versions and update them as needed

When you access dependencies in packages, the CMakeLists.txt file needs to be changed in three ways:

• The find_package command tells CMake to search for a package.
• The target_link_libraries command should specify that the library should be accessed through its package.
• The target_include_directories and target_link_directories aren't necessary for libraries and header files contained in a package.

When CMake executes find_package, it looks through a series of directories to find the given package. For example, the following command tells CMake to look for a package named mypack, and to produce an error if it can't be found:

find_package(mypack REQUIRED)

If a library should be accessed through a package, the second argument of target_link_libraries should identify both the package and the library. For example, the following command tells CMake to look for the library bar in the package mypack when building the target foo:

target_link_libraries(foo, mypack::bar)

For a more practical example, look at the CMakeLists.txt file in the read_json project. The code in this project depends on the JSON library provided by Niels Lohmann on his Github page. The name of the target executable is read_json and the name of the library is nlohmann_json, so the CMakeLists.txt file contains the following commands:

# Tells CMake to find the package named nlohmann_json
find_package(nlohmann_json REQUIRED)

# Tells CMake to access the nlohmann_json library in the nlohmann_json package
target_link_libraries(read_json nlohmann_json::nlohmann_json)

The first command tells CMake to search for the nlohmann_json package and produce an error if it can't be found. The second command tells CMake to find the nlohmann_json library in the nlohmann_json package and to link it into the build of the read_json target.

This raises an important question. The read_json project folder doesn't contain any libraries, so how can CMake find the package? The answer involves Conan.

3. Managing Packages with Conan

I've encountered a handful of C++ package management tools, including Spack and vcpkg. But I've found Conan the easiest of them. Like CMake, it's freely available for download. The main site is conan.io and the installation instructions are at https://docs.conan.io/2/installation.html.

Also like CMake, Conan is accessed through a utility that runs on the command line. The utility's name is conan, and it can be used in a number of commands. Table 1 lists six of them.

Table 1: Conan Commands

Command	Description
conan version	Provide version information
conan profile	Configure the system's profile
conan remote	Access Conan repositories
conan download	Download a package without installing
conan install	Install packages from a recipe
conan build	Install dependencies and run the build method

The first command, conan version, is the simplest. This prints the Conan version, the path to the Conan utility, and information about the Python installation.

3.1 Profile Configuration

To manage dependencies, Conan needs information about the operating system and the installed build tools. This information is called a profile, and you can create a profile for your system with the following command:

conan profile detect

When this runs, Conan creates a file containing parameters about the development system. On my Linux system, the file's name is default and it's stored in the ~/.conan2/profiles folder. Its content is given as follows:

[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=13
os=Linux

On my Windows system, running conan profile detect creates a file named default in the C:\Users\username\.conan2\profiles folder. Its content is given as follows:

[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=14
compiler.runtime=dynamic
compiler.version=194
os=Windows

Because this information is present, Conan knows what type of packages it needs to install and manage.

3.2 Downloading Packages

The primary advantage of using Conan is that it downloads, installs, and keeps track of packages so that developers don't have to. Conan downloads packages from remote repositories, and the following command lists the available repositories:

conan remote list

When I execute this, it produces the following result:

conancenter: https://center.conan.io [Verify SSL: True, Enabled: True]

This indicates that Conan can download packages from ConanCenter, which is Conan's default repository. This stores a wide range of packages for different operating systems, and you can explore this repository at https://conan.io/center.

You can test the download process by running conan download, which requires the package's name followed by -r and the name of the repository. For example, the following code downloads the gtest package from ConanCenter:

conan download gtest -r conancenter

If Conan can find the package, it will store its files in the .conan2 folder in your home directory. The name of the package folder starts with the package name followed by a unique numeric identifier.

3.3 Installing Packages

Rather than use conan download, it's more common to list required packages in a file called a recipe and then run conan install. This command reads the recipe, downloads packages into the ~/.conan2/p folder, and generates files needed to interface with the build system.

Conan recognizes two types of recipes:

• conanfile.txt — simple text file, provides information in sections
• conanfile.py — Python file, can be used for sophisticated package management operations

This discussion focuses on conanfile.txt, which is simpler but less flexible than conanfile.py. This file lists information under one or more sections, with each section name surrounded in square brackets. In the read_json project, the content of conanfile.txt is contained in three sections:

[requires]
nlohmann_json/3.11.3

[generators]
CMakeDeps
CMakeToolchain

[layout]
cmake_layout

The first section, requires, identifies the name and version of each package that needs to be installed. In this case, the project needs Version 3.11.3 of the nlohmann_json package, so the requires section contains nlohmann_json/3.11.3.

The second section, generators, contains keywords that tell Conan which files should be created during the installation process. In this example, there are two keywords:

1. CMakeDeps — creates a file named xyz-config.cmake for each dependency, where xyz is the package's name
2. CMakeToolchain — creates a file named conan_toolchain.cmake, which CMake can access during the build

The last section, layout, tells Conan about the directory structure it should use when generating files. If this is set to cmake_layout, Conan will create a build folder and generate files to facilitate CMake builds.

3.4 Example Conan/CMake Project

The second example project in example_code.zip is read_json. This parses JSON text using capabilities provided in the nlohmann_json package. For this reason, the CMakeLists.txt file contains the following command:

find_package(nlohmann_json REQUIRED)

The project also has a file named conanfile.txt, which lists nlohmann_json as a required package. When Conan performs installation, it will read this file, download and install the nlohmann_json package, and generate files needed for the CMake build. The command that launches in the installation is as follows:

conan install .

Because conanfile.txt specifies a CMake layout, Conan will create a folder named build, a subfolder named Release, and a subsubfolder named generators. The build/Release/generators directory contains several generated files, such as nlohmann_json-config.cmake, which tells CMake how to access the nlohmann_json package installed by Conan.

Once Conan finishes installing packages and generating files for CMake, the following command (executed in the build folder) will tell CMake to generate build files:

cmake .. -DCMAKE_TOOLCHAIN_FILE="Release/generators/conan_toolchain.cmake" -DCMAKE_BUILD_TYPE=Release

In this command, the CMAKE_TOOLCHAIN_FILE option tells CMake to read the conan_toolchain.cmake file in the Release/generators directory. After this command executes, the project can be built with the following command:

cmake --build .

If this completes successfully, the build will produce an executable named read_json. When run, the executable will parse JSON text and print a portion of the data to standard output.

4. Accessing Conan and CMake in Python

The preceding discussion explained how to configure Conan by writing the conanfile.txt recipe. This is fine for simple operations, but for greater flexibility, I recommend adding methods to the second type of Conan recipe: conanfile.py.

This section explains how to access Conan and CMake in conanfile.py, but before we get into the code, it's a good idea to install the required library:

pip install conan

This section explains how to access capabilities of both libraries. We'll look at XYZ...

But first, it helps to start with a simple example.

4.1  Simple Example

Earlier, I explained how the sections of conanfile.txt (requires, generator, and layout) tell Conan how to manage the build process. conanfile.py provides the same information, as shown in the following code:

from conan import ConanFile
from conan.tools.cmake import CMake, CMakeDeps, CMakeToolchain, cmake_layout

class SimpleExample(ConanFile):
    settings = 'os', 'compiler', 'build_type', 'arch'

    def requirements(self):
        self.requires('nlohmann_json/3.11.3')

    def generate(self):
        deps = CMakeDeps(self)
        deps.generate()
        tc = CMakeToolchain(self)
        tc.generate()

    def layout(self):
        cmake_layout(self)

    def build(self):
        cmake = CMake(self)
        cmake.configure()
        cmake.build()

This code creates a class named SimpleExample that inherits from ConanFile. It starts by defining a variable named settings and then defines four methods:

1. requirements - the dependencies that need to be installed
2. generate - generate necessary files for the build
3. layout - set the directory structure for generated files
4. build - perform the build

The first three methods should look familiar because they contain the same information as the sections of conanfile.txt. The last method accesses CMake to build the project. With conanfile.py in place, you can install dependencies by executing conan install . and then run the build by executing conan build . at a command line.

4.2  Creating and Uploading Packages

An earlier section explained what packages are and how they can be downloaded and installed with Conan. One advantage of using conanfile.py instead of conanfile.txt is that you can easily create new packages and upload them to a remote repository.

The general process of creating a package can be defined by adding the following methods to conanfile.py:

• The source method downloads and modifies source files.
• The requirements method identifies packages that need to be installed.
• The generate method creates files needed for the build process.
• The configure and config_options methods set variables used during the build.
• The build method launches the build process.
• The package method identifies the files to be included in the package.
• Provide package metadata in the package_info method.

If these methods are defined, you can create a new package by executing conan new in the directory containing conanfile.py.

After you've created a package, you can upload it to a remote repository by executing conan upload. This accepts a -r flag that identifies the remote repository. As an example, the following command uploads a package named my_package to a repository named my_repo:

conan upload my_package -r=my_repo

Keep in mind that you can obtain a list of available repositories by executing the command conan remote list. You can obtain more information by reading Conan's official documentation.

History

This article was first uploaded on August 6, 2024. Added content on September 2, 2024.