Introduction
In this Article, we will discuss the following topics:
- Getting the Python C/C++ API for working.
- Initialize and Destroy a Python environment.
- Running a Simple inline Python code from C/C++.
- Running a Simple Python program from file from C/C++ program.
- Call a Python method from C/C++.
- Call a C/C++ function from Python code.
- Why are we doing this???? (Points of Interest)
Background
Before we continue, the reader must be well versed in C/C++ programming. And have basics programming knowledge in Python.
Before we begin : What we need???
- Create a Console Application.
- Assuming that Python 3 is installed on the system, completely, at a location "D:\Python34".
- Now add the path of "D:\Python34\include" to Include path of the C/C++ project
- Add the path of "D:\Python34\libs" to Lib path of the project
- Try compiling the project. If it builds correctly, we are ready to begin our work!!!
Initialize and Clean the Python Environment
The simplest program to embed is:
#include <stdio.h>
#include <conio.h>
#include <Python.h>
int main()
{
PyObject* pInt;
Py_Initialize();
PyRun_SimpleString("print('Hello World from Embedded Python!!!')");
Py_Finalize();
printf("\nPress any key to exit...\n");
if(!_getch()) _getch();
return 0;
}
- The Python header file is "Python.h" which includes whatever code we need. The linker library file is Python34.lib (in general, PythonXX.lib, XX=34 here).
- Initialize Python environment by calling
Py_Initialize()
- Destroy and cleanup the environment by calling
Py_Finalize()
- That's all, to get Python Interpreter to get up and running in our C Code.
As you can see that, we have an embedded Python code in our program. This code is:
print('Hello World from Embedded Python!!!')
This code in Python will print the line "Hello World from Embedded Python!!!" on the Console screen. This program is executed by our code using PyRun_SimpleString(char* our_python_code_to_run)
. But the caller must ensure that, this function is called after Python interpreter is initialized and before it is destroyed.
Thus, we now know how to execute a simple Python code from String in C. Next we will examine how to execute a Python code from an external text file.
Execute a Python Program program from file
Consider the Simple Python program to execute, which is stored in the file "pyemb7.py
":
print('External Python program running...')
print('Hello World from Python program')
We can run the above program from C/C++ code using the following program:
#include <stdio.h>
#include <conio.h>
#include <Python.h>
int main()
{
char filename[] = "pyemb7.py";
FILE* fp;
Py_Initialize();
fp = _Py_fopen(filename, "r");
PyRun_SimpleFile(fp, filename);
Py_Finalize();
return 0;
}
- Initialize the Python Environment as discussed before.
- Declare a
FILE*
to store our program file object. - Now open the Python program file using
_Py_fopen(char* program_filename_with_py_extension, char* file_open_mode)
. This function is similar to the fopen
function of standard C/C++.Here we have opened the pyemb7.py
in r
ead mode. - Check the
FILE*
object returned. If it is NULL
, the file cannot be opened, so we cannot proceed further. Report an error and abort. - Now we have the file opened. We have to execute it using
PyRun_SimpleFile(opened_python_program_file_pointer, char* program_filename_which_becomes_argv_0)
. - Destroy the Python Environment, as previously shown.
Now we, are done executing a Python code from external file from within the native code. Next we will discuss a code, to call a specific Python function from a Python file.
C++ Helper for Python
Content of C++ Header file pyhelper.hpp
:
#ifndef PYHELPER_HPP
#define PYHELPER_HPP
#pragma once
#include <Python.h>
class CPyInstance
{
public:
CPyInstance()
{
Py_Initialize();
}
~CPyInstance()
{
Py_Finalize();
}
};
class CPyObject
{
private:
PyObject *p;
public:
CPyObject() : p(NULL)
{}
CPyObject(PyObject* _p) : p(_p)
{}
~CPyObject()
{
Release();
}
PyObject* getObject()
{
return p;
}
PyObject* setObject(PyObject* _p)
{
return (p=_p);
}
PyObject* AddRef()
{
if(p)
{
Py_INCREF(p);
}
return p;
}
void Release()
{
if(p)
{
Py_DECREF(p);
}
p= NULL;
}
PyObject* operator ->()
{
return p;
}
bool is()
{
return p ? true : false;
}
operator PyObject*()
{
return p;
}
PyObject* operator = (PyObject* pp)
{
p = pp;
return p;
}
operator bool()
{
return p ? true : false;
}
};
#endif
From now on we will use C++ project, and above file instead.
To automatically initialize Python instance we will just declare an object of CPyInstance pyInstance
. When this object goes out of scope, the Python environment is destroyed.
Now we can write the first program above as:
#include <stdio.h>
#include <conio.h>
#include <pyhelper.hpp>
int main()
{
CPyInstance pyInstance;
PyRun_SimpleString("print('Hello World from Embedded Python!!!')");
printf("\nPress any key to exit...\n");
if(!_getch()) _getch();
return 0;
}
See, how we have removed the 2 functions: Py_Initialize()
and Py_Finalize()
. Advantage of the CPyInstance
class is that, the Python is deinitialized automatically and programmer is free to focus on rest of the code.
In the next section we will discuss the use of Python object PyObject
and the corresponding helper CPyObject
class.
PyObject and the CPyObject helper
PyObject
is a Python object used to access the Python objects used or returned by Python interpreter. Python uses references to track the objects returned. When an object is no more needed, it must be deferenced, using DECREF
and XDECREF
. Consider the code sample below:
PyObject* pObj = ...;
.... the pObj is used in this part of code ....
DECREF(pObj);
In this above code, assume that pObj
is returned by a Python function, is used in the further code path. Once the object is no more required, we have called DECREF
macro and passed the pObj
pointer to it, and we are done. After this, the pObj
code will become useless, and caller should never use it further. What about XDECREF
macro? Calling DECREF
requires that the passed PyObject
pointer should never be null. But XDECREF
is safe. XDECREF
checks for NULL
pointer and then internally call DECREF
if the PyObject
pointer passed is NOT NULL
.
If we want to re-reference the PyObject
further, or pass it to another function, then we must increase its reference. Hence use INCREF
or XINCREF
, and pass the PyObject
pointer to it. XINCREF
is safe, and you can pass a NULL
pointer too. But INCREF
cannot process NULL
pointer.
To be safe, we will use the C++ class CPyObject
. Consider the code sample:
CPyObject pObj = ...;
... use the pObj in further code ...
... forget about pObj it will be deferenced when pObj goes out of scope ...
Thus, in above code CPyObject
is used instead of PyObject*
. This helper class encapsulate the PyObject*
within CPyObject
class. Once the class goes out of scope, the object is automatically dereferenced. Also, the NULL
object is automatically taken care of, and can also be automatically type casted to PyObject*
. But to increase the reference pointer, call the AddReff()
, method of the CPyObject
class.
#include <pyhelper.hpp>
int main()
{
CPyInstance pInstance;
CPyObject p;
p = PyLong_FromLong(50);
printf_s("Value = %ld\n", PyLong_AsLong(p));
return 0;
}
- Initialize a Python environment by declaring an object of
CPyInstance
. - Now we write an object of
CPyObject
called p
. - Now we create a Python object from
long
value using a call to Python_FromLong(...)
. This function has the signature PyObject* PyLong_FromLong(long v)
. The function takes in a long
value and returns a PyObject
. - Next we want to convert
PyObject*
to long
value. We know that the p
object represents a long
value. So we call PyLong_AsLong(...)
, to get the long
value. The function signature is long PyLong_AsLong(PyObject *pylong)
. This function takes in a PyObject*
and returns a long
value. In the above program, we have printed this Python object's value. - After we are done, we just forget about the object, it will automatically be dereferenced.
Next, we will call a Python function from C++.
Call a Python function (method) from C++
Consider the following Python program, stored in pyemb3.py
:
def getInteger():
print('Python function getInteger() called')
c = 100*50/30
return c
Now we want to call the function getInteger()
from the following C++ code and print the value returned this function. This is the client C++ code:
#include <stdio.h>
#include <Python.h>
#include <pyhelper.hpp>
int main()
{
CPyInstance hInstance;
CPyObject pName = PyUnicode_FromString("pyemb3");
CPyObject pModule = PyImport_Import(pName);
if(pModule)
{
CPyObject pFunc = PyObject_GetAttrString(pModule, "getInteger");
if(pFunc && PyCallable_Check(pFunc))
{
CPyObject pValue = PyObject_CallObject(pFunc, NULL);
printf_s("C: getInteger() = %ld\n", PyLong_AsLong(pValue));
}
else
{
printf("ERROR: function getInteger()\n");
}
}
else
{
printf_s("ERROR: Module not imported\n");
}
return 0;
}
So, we will use the follow the following rule to call a Python function:
- Initialize the Python environment.
- Import the Python module.
- Get the reference to Python function, to call.
- Check if the function can be called, and call it.
- Then object the returned Python object, returned by the function, after execution.
- Convert returned Python object to C++ object, and print it on screen.
Now, we explain the program code:
- First, we initialize the Python environment using
CPyInstance hInstance
. - Next, we get a Python object representing the module to call. Our module name is
pyemb3.py
, so we get Python object using from the function PyObject *PyUnicode_FromString(const char *u)
. This function returns PyObject*
in CPyObject
as CPyObject pName = PyUnicode_FromString("pyemb3");
. - Next, using
PyObject* PyImport_Import(PyObject *name)
, we will import the module and return an object handle representing the imported module. We use this function as CPyObject pModule = PyImport_Import(pName);
and if the module is loaded then a valid Python object is returned. Here, no null check is done, programmers may check for it, if they want. - Next, we get the function attribute object using the function
PyObject* PyObject_GetAttrString(PyObject *o, const char *attr_name)
. So, we use this as CPyObject pFunc = PyObject_GetAttrString(pModule, "getInteger");
. The first is the handle to the module being used, then the second argument is the name of the function being called. - Next, we check if the returned function object is
NULL
. If it is not NULL
, then check if we can call the function. For this we use int PyCallable_Check(PyObject *o)
. We pass the function object pointer to this function. If we get a 1
in return, then we can call the function. Only if we can call the function, we can proceed further. - Next, we call the Python function using
PyObject* PyObject_CallObject(PyObject *callable_object, PyObject *args)
. The first argument here is the function object, we obtained previously. The second is the arguments that will be passed to the Python method. But we do not pass any argument, so we pass NULL to this. The function after execution, will return an object that is returned by the Python method. The result of the Python function. - Since we know, that the function returns a
long
value, so we print the value using the program line printf_s("C: getInteger() = %ld\n", PyLong_AsLong(pValue));
.
Thus we see the output of the program above is:
Python function getInteger() called
C: getInteger() = 166
Next, we will discuss, how we can call a C/C++ function from an embedded Python program.
Calling a C/C++ function from Python code
Consider the following Python code pyemb6.py
:
import arnav
print('in python: ')
val = arnav.foo()
print('in python:arnav.foo() returned ', val)
arnav.show(val*20+80)
And now consider the C++ program below, that calls the code above:
#include <stdio.h>
#include <Python.h>
#include <pyhelper.hpp>
static PyObject* arnav_foo(PyObject* self, PyObject* args)
{
printf_s("... in C++...: foo() method\n");
return PyLong_FromLong(51);
}
static PyObject* arnav_show(PyObject* self, PyObject* args)
{
PyObject *a;
if(PyArg_UnpackTuple(args, "", 1, 1, &a))
{
printf_s("C++: show(%ld)\n", PyLong_AsLong(a));
}
return PyLong_FromLong(0);
}
static struct PyMethodDef methods[] = {
{ "foo", arnav_foo, METH_VARARGS, "Returns the number"},
{ "show", arnav_show, METH_VARARGS, "Show a number" },
{ NULL, NULL, 0, NULL }
};
static struct PyModuleDef modDef = {
PyModuleDef_HEAD_INIT, "arnav", NULL, -1, methods,
NULL, NULL, NULL, NULL
};
static PyObject* PyInit_arnav(void)
{
return PyModule_Create(&modDef);
}
int main()
{
PyImport_AppendInittab("arnav", &PyInit_arnav);
CPyInstance hInstance;
const char pFile[] = "pyemb6.py";
FILE* fp = _Py_fopen(pFile, "r");
PyRun_AnyFile(fp, pFile);
return 0;
}
First, concentrating on the main()
. The familiar part of the code are, initializing Python environment, then load a Python program and run the program. But now we will conventrate on how to add a new module to Python environment.
First of all, the imported module must be added before the Python instance is initialized. Now how we do it:
- The C function must have the format-
PyObject* self
as first object, representing the function itself. PyObject* args
as the second object, representing the arguments that is passed by the Python program. PyObject*
returned by the function itself. This represents the data to be returned to the Python program calling this function. - The function name following the format
moduleNamespace_functionName
.
So we have a function foo
in module namespace arnav
, which is oc C type long arnav.foo(void)
, the code is:
static PyObject* arnav_foo(PyObject* self, PyObject* args)
{
printf_s("... in C++...: foo() method\n");
return PyLong_FromLong(51);
}
Next, we declare another function show
in arnav
which is of type long arnav.show(long value)
:
static PyObject* arnav_show(PyObject* self, PyObject* args)
{
PyObject *a;
if(PyArg_UnpackTuple(args, "", 1, 1, &a))
{
printf_s("C++: show(%ld)\n", PyLong_AsLong(a));
}
return PyLong_FromLong(0);
}
Note the use of the function PyArg_UnpackTuple
, which ise used to parse the arguments. Its signature is int PyArg_UnpackTuple(PyObject *args, const char *name, Py_ssize_t min, Py_ssize_t max, ...)
. First argument is the reference to the args
pointer. The parameter name
is not used, it refers to access mode possibly. The min
represents minimum number of arguments (tuples) to be passed by Python code, and max
must be maximum number of arguments that can be passed. We need only 1 argument to our function, so we set min = max =1
. - Next, we declare a method table:
static struct PyMethodDef methods[] = {
{ "foo", arnav_foo, METH_VARARGS, "Returns the number"},
{ "show", arnav_show, METH_VARARGS, "Show a number" },
{ NULL, NULL, 0, NULL }
};
This is done using PyMethodDef
structure array. It's fields are:
- First field represents the name of the method, that the Python code will use to call our native code.
- The C/C++ function pointer, that will be invoked when Python calls in.
- Method can pass in variable numbers of arguments. This is must.
- Last field, represents a help string. Specify what our function will do. Can be thought of as a Help for our function, in Python environment.
- A newly introduced way to develop import module in Python 3, is to declare
PyModuleDef
:
static struct PyModuleDef modDef = {
PyModuleDef_HEAD_INIT, "arnav", NULL, -1, methods,
NULL, NULL, NULL, NULL
};
At the simplest we will use only few fields of the PyModuleDef
structure.
- First argument - Specify
PyModuleDef_HEAD_INIT
must be specified. - Second argument- The name of the module/namespace to be used in Python code. In our case, it is
arnav
. - Third argument - We are not using it. Specify
NULL
. - Fourth argument - Specify
-1
. - Fifth argument - We pass the pointer to our method declaration structure
PyMethodDef
. In our case we pass our static variable method
. - Last 4 parameters, we do not use. So we specify
NULL
for these.
NOTE: We are using a minimal declaration, which will solve most of our problems. So, in future article, we will use more fields of this structure and describe in details about each field and their effect. - Next, we declare the following code:
static PyObject* PyInit_arnav(void)
{
return PyModule_Create(&modDef);
}
This function will be called in, whenever we need to load our module. Here we declare a function of format PyInit_moduleName(void)
. In our case for namespace arnav
, we use function name PyInit_arnav(void)
. This function does not take anything, but returns a PyObject*
. Note the use of PyObject* PyModule_Create(PyModuleDef *module)
. Here, we pass our PyModuleDef
structure. This function will create a new module. In our case we pass the address of our modDef
variable. The function returns a PyObject*
when the module is loaded, the function PyInit_arnav(void)
returns this object created. - Now, within
main()
function, before the Python environment is initialized. This is done through the use of function PyImport_AppendInittab
. This function has the signature int PyImport_AppendInittab(const char *name, PyObject* (*initfunc)(void))
. We pass our initialization function PyInit_arnav
pointer to this function. And, if successful (function will not return -1
), the new module will be added to the Python environment. - Finally, the module will become callable from the Python module.
Now we will examine how the Python source utilizes this new namespace arnav
:
The output of the program is:
in python:
... in C++...: foo() method
in python:arnav.foo() returned 51
C++: show(1100)
Why are we doing this? (Points to remember)
Python is a very popular language with an extremely large code base. Also, the Python API is elaborate yet easy to use. Further, embedding an interpreter within a Native language will enable us to change the course of the program after it had been designed, makes dynamic programming approach to the programming.
In future, we would investigate more into the embedded Python and examine the above discussed structures in details.
- Download the attached ZIP file. It has the Python3 SDK bundled along with some demo applications, discussed in this article. You can directly use it.
- If you are installing Python3, use the x86 build, for building x86 based applications.
- Boost and other API alternatives are available for the same work, but since I have never used one; I am sticking to the Python official SDK.
- Always distribute
Python34.dll
with your C/C++ application. Your native application depends on it. Otherwise, the client system must have Python3 installed correctly.