Code documentation is important, but maintaining the source code and its documents separately is challenging. If we can generate the document based on the source code, or at least based on the code comments, we have a better chance to keep the document up to date.
Introduction
As a developer, we all know the importance of code documenting: Good code is not only self-explanatory but also well-documented. However, we also struggle with the difficulty to keep documents up to date, especially if we maintain the source code and its documents separately. If we can generate the document based on the source code, or at least based on the code comments, we have a better chance to keep the document up to date.
Sphinx is a tool to build documents from the code. It supports many programming languages, but it is widely used in Python projects, including the official Python website. The official website of Sphinx provides abundant useful information and reference. However, for those who try to use Sphinx for the first time, the official website may be a little bit overwhelming. At least, that is my experience when I tried to use Sphinx for the first time. Therefore, I wrote this article, and hopefully, this article could provide a simple and straightforward tutorial for a newbie of Sphinx.
Plot
This tutorial uses a simple Python project (Sample Project) to demonstrate how to use Sphinx to generate HTML-based documents. The Sample Project is a simple binary search tree and binary tree traversal implementation. It is well documented by following NumPy style docstrings. The main purpose of the Sample Project is not only to be a sample code for this Sphinx tutorial, but also to demo how NumPy style docstrings translate to a real document via Sphinx.
The Sample Project can be downloaded from my Github.
$ git clone https://github.com/shunsvineyard/python-sample-code.git
The generated documents look like the picture below:
Assumptions and Requirements
This tutorial is based on the following software:
Note: Sphinx can run on both Linux and Windows.
How to Use Sphinx?
Sphinx uses reStructuredText as its markup language. The process of Sphinx generating documents is like:
Project source code (Python or other supported languages) ->
reStructuredText files -> documents (HTML or other supported format)
Sphinx provides two command-line tools: sphinx-quickstart and sphinx-apidoc.
sphinx-quickstart sets up a source directory and creates a default configuration, conf.py, and a master document, index.rst, which is to serve as a welcome page of a document.sphinx-apidoc generates reStructuredText files to document from all found modules.
In short, we use these two tools to generate Sphinx source code, i.e., reStructuredText files, and we modify these reStructuredText files, and finally use Sphinx to build nice documents.
Workflow
The same as software needs a developer’s maintenance, writing a software document is not a one-time job. It needs to be updated when the software changes. The workflow of using Sphinx can be seen as the following:
The picture demonstrates the basic workflow of using Sphinx, and the details of each step is illustrated in the following subsections.
Prepare
Before we start using Sphinx, we need to set up our working environment.
On Linux
user@ubuntu:~$ python3 -m venv sphinxvenv
user@ubuntu:~$ source sphinxvenv/bin/activate
(sphinxvenv) user@ubuntu:~$ git clone https://github.com/shunsvineyard/python-sample-code.git
(sphinxvenv) user@ubuntu:~$ cd python-sample-code/
(sphinxvenv) user@ubuntu:~/python-sample-code$ pip install -r requirements.txt
On Windows
c:\Workspace>python -m venv sphinxenv
c:\Workspace>sphinxenv\Scripts\activate
(sphinxenv) c:\Workspace>git clone https://github.com/shunsvineyard/python-sample-code.git
(sphinxenv) c:\Workspace>cd python-sample-code
(sphinxenv) c:\Workspace\python-sample-code>pip install -r requirements.txt
Now, we have the Sample Project and working environment for the Sphinx demo. Because the Sample Project already contains the docs folder, we need to delete it.
The layout of the Sample Project after we delete the docs folder looks like:
python-sample-code
├── LICENSE
├── README.rst
├── binary_trees
│ ├── __init__.py
│ ├── binary_search_tree.py
│ ├── binary_tree.py
│ └── traversal.py
├── pytest.ini
├── requirements.txt
├── setup.py
└── tests
├── __init__.py
├── conftest.py
├── test_binary_search_tree.py
└── test_traversal.py
Step 1: Use sphinx-quickstart to Generate Sphinx Source Directory with conf.py and index.rst
Assume we want to put all the document related files in the docs directory. So, we begin by creating a Sphinx documentation directory, docs. Then, we go to the docs directory and run sphinx-quickstart.
On Linux
(sphinxvenv) user@ubuntu:~/python-sample-code$ mkdir docs
(sphinxvenv) user@ubuntu:~/python-sample-code$ cd docs/
(sphinxvenv) user@ubuntu:~/python-sample-code/docs$ sphinx-quickstart
On Windows
(sphinxenv) c:\Workspace\python-sample-code>mkdir docs
(sphinxenv) c:\Workspace\python-sample-code>cd docs
(sphinxenv) c:\Workspace\python-sample-code\docs>sphinx-quickstart
Once we run sphinx-quickstart, it asks a few questions about this project. Following are the example answers for these questions:
Welcome to the Sphinx 2.2.0 quickstart utility.
Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).
Selected root path: .
You have two options for placing the build directory for Sphinx output.
Either, you use a directory "_build" within the root path, or you separate
"source" and "build" directories within the root path.
> Separate source and build directories (y/n) [n]: y
The project name will occur in several places in the built documentation.
> Project name: Sample Project
> Author name(s): Author
> Project release []: 0.1.0
If the documents are to be written in a language other than English,
you can select a language here by its language code. Sphinx will then
translate text that it generates into that language.
For a list of supported codes, see
https://www.sphinx-doc.org/en/master/usage/configuration.html
> Project language [en]:
Creating file ./source/conf.py.
Creating file ./source/index.rst.
Creating file ./Makefile.
Creating file ./make.bat.
Finished: An initial directory structure has been created.
You should now populate your master file ./source/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.
At the end of the sphinx-quickstart, it shows how to build the documents.
You should now populate your master file ./source/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.
If we do make html here, Sphinx will generate the default documents which contains nothing about the Sample Project. The preview of the output can be viewed at:
https://htmlpreview.github.io/?https://github.com/shunsvineyard/shunsvineyard/blob/master/use-sphinx-for-python-documentation/step1_output/index.html (the preview link above is powered by https://github.com/htmlpreview/htmlpreview.github.com)
Note: Sphinx is not a tool that offers fully automatic documents generation like Doxygen. sphinx-quickstart only generates some default files such as index.rst and conf.py with basic information answered by a user. Therefore, we need to do some work to make the documents real.
After running sphinx-quickstart, the layout of the project looks like:
python-sample-code
├── LICENSE
├── README.rst
├── binary_trees
│ ├── __init__.py
│ ├── binary_search_tree.py
│ ├── binary_tree.py
│ └── traversal.py
├── docs
│ ├── Makefile
│ ├── build
│ ├── make.bat
│ └── source
│ ├── _static
│ ├── _templates
│ ├── conf.py
│ └── index.rst
├── pytest.ini
├── requirements.txt
├── setup.py
└── tests
├── __init__.py
├── conftest.py
├── test_binary_search_tree.py
└── test_traversal.py
Note that Makefile is for Linux and make.bat is for Windows.
sphinx-quickstart generates few files, and the most important one is conf.py which is the configuration of the documents. Although conf.py serves as a configuration file, it is a real Python file. The content of conf.py is Python syntax.
Using Sphinx to generate a document is highly configurable. This section demonstrates the most basic configurations: the path to the project source code, theme for the documents, and adding extensions.
Set the Path to the Project
To make Sphinx be able to find the project, we need to uncomment these three lines:
And update the path to the project:
import os
import sys
sys.path.insert(0, os.path.abspath('../..'))
Select the Theme
Sphinx provides many built-in themes. The default is alabaster.
html_theme = 'alabaster'
In this tutorial, we change it to bizstyle.
html_theme = 'bizstyle'
More themes and their configurations can be found at https://www.sphinx-doc.org/en/master/usage/theming.html.
Add an Extension for NumPy Style
The Sample Project uses NumPy style for docstrings. Therefore, we need to add the extension (napoleon) for parsing NumPy style docstrings.
extensions = [
'sphinx.ext.napoleon'
]
This extension (napoleon) supports NumPy and Google style docstrings and provides several configurable features. For the Sample Project, since we use NumPy style docstrings, we should disable Google style.
napoleon_google_docstring = False
Other settings for napoleon can be found at https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html#module-sphinx.ext.napoleon
The complete conf.py example can be found at https://github.com/shunsvineyard/python-sample-code/blob/master/docs/source/conf.py.
Besides, Sphinx has many built-in extensions and also supports custom extension. To learn more, please visit https://www.sphinx-doc.org/en/master/usage/extensions/index.html
Now, we have the basic configuration for our project. Next, we use sphinx-apidoc to generate reStructuredText files from the Sample Project source code.
sphinx-apidoc is a tool for automatically generating reStructuredText files from source code, e.g., Python modules. To use it, run:
sphinx-apidoc -f -o <path-to-output> <path-to-module>
-f means force overwriting of any existing generated files.-o means the path to place the output files.
Complete usage of sphinx-apidoc is at https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html.
On Linux
(sphinxvenv) user@ubuntu:~/python-sample-code/docs$ sphinx-apidoc -f -o source/ ../binary_trees/
On Windows
(sphinxvenv) shunsvineyard@remote-ubuntu:~/python-sample-code/docs$
sphinx-apidoc -f -o source/ ../binary_trees/
In the Sample Project, sphinx-apidoc generates two files, binary_trees.rst and modules.rst. The layout of the project looks like the following:
python-sample-code
├── LICENSE
├── README.rst
├── binary_trees
│ ├── __init__.py
│ ├── binary_search_tree.py
│ ├── binary_tree.py
│ └── traversal.py
├── docs
│ ├── Makefile
│ ├── build
│ ├── make.bat
│ └── source
│ ├── _static
│ ├── _templates
│ ├── binary_trees.rst
│ ├── conf.py
│ ├── index.rst
│ └── modules.rst
├── pytest.ini
├── requirements.txt
├── setup.py
└── tests
├── __init__.py
├── conftest.py
├── test_binary_search_tree.py
└── test_traversal.py
The other important file sphinx-quickstart generates is index.rst. index.rst is the master document which is served as a welcome page and contains the root of the ‘’table of contents tree’’ (toctree). The toctree initially is empty after sphinx-quickstart creates index.rst.
.. toctree::
:maxdepth: 2
Add the modules to the index.rst
The generated modules.rst contains all the modules. In this case, it only has binary_trees. So we need to add the modules.rst to index.rst.
To add document to be listing on the welcome page (index.rst), do:
.. toctree::
:maxdepth: 2
modules
Note: When you add another reStructuredText file, use the file name without extension. If there is a hierarchy of the file, use forward slash ‘’/’’ as directory separators.
Add the README.rst to index.rst
Since the Sample Project already has a readme file, README.rst, at the top level of the project, we can add it to the welcome page of the document.
- Create a readme.rst file under docs/source and add the line
.. include:: ../../README.rst. (See https://github.com/shunsvineyard/python-sample-code/blob/master/docs/source/readme.rst) - Add the readme to the index.rst, so it can be included in the welcome page.
After these two steps, the index.rst looks like:
.. Sample Project documentation master file, created by
sphinx-quickstart on Sun Sep 15 20:47:59 2019.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Sample Project's documentation!
==========================================
.. toctree::
:maxdepth: 2
readme
modules
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
See https://raw.githubusercontent.com/shunsvineyard/python-sample-code/master/docs/source/index.rst for the complete example.
Note: When we add a new module, class, API, or any code change that affect the documents, we need to repeat Step 3 and Step 4 to update the documents.
Step 5: Build the Documents
The last step to generate the documents is to issue make html (if we want to generate HTML-based documents).
On Linux
(sphinxvenv) user@ubuntu:~/python-sample-code/docs$ make html
On Windows
(sphinxvenv) shunsvineyard@remote-ubuntu:~/python-sample-code/docs$ make html
After we run make html command, a build folder is created under docs. Also, the HTML-based documents are located at build/html. The generated document looks like:
The pre-generated documents can also be viewed at https://htmlpreview.github.io/?https://github.com/shunsvineyard/shunsvineyard/blob/master/use-sphinx-for-python-documentation/final_output/index.html (Note that the preview does not render the page properly due to the limit of the htmlpreview tool) or download the HTML files from https://github.com/shunsvineyard/shunsvineyard (located at shunsvineyard/use-sphinx-for-python-documentation/final_output/).
Conclusion
Although we still need to manually edit the generated reStructuredText files, Sphinx does provide an easier way to build a nice document. It also features configurable and extensible abilities via conf.py and extensions. To learn more about Sphinx, you can check the following online resources:
History
- 20th September, 2019: Initial version
