Python与C语言混合编程：通过distutils或setuptools实现的一个简单的C扩展

待我称王封你为后i 2022-03-22 08:42 177阅读 0赞

这个介绍一个扩展Python组件的例子，组件是通过C语言开发的(用python的Ｃ-API实现)， 然后通过python的distutils组件安装，例子中内容来自《python cocobook》英文版，一本非常不错的书。

## 目录结构 ##

首先目录结构如下

## 源码内容 ##

sample.h

/* sample.h */
    
    extern int gcd(int x, int y);
    extern int in_mandel(double x0, double y0, int n);
    extern int divide(int a, int b, int *remainder);
    extern double avg(double *a, int n);
    
    typedef struct Point {
        double x,y;
    } Point;
    
    extern double distance(Point *p1, Point *p2);

sample.c

/* sample.c */
    #include <math.h>
    
    /* Compute the greatest common divisor */
    int gcd(int x, int y) {
        int g = y;
        while (x > 0) {
            g = x;
            x = y % x;
            y = g;
        }
        return g;
    }
    
    /* Test if (x0,y0) is in the Mandelbrot set or not */
    int in_mandel(double x0, double y0, int n) {
      double x=0,y=0,xtemp;
      while (n > 0) {
        xtemp = x*x - y*y + x0;
        y = 2*x*y + y0;
        x = xtemp;
        n -= 1;
        if (x*x + y*y > 4) return 0;
      }
      return 1;
    }
    
    /* Divide two numbers */
    int divide(int a, int b, int *remainder) {
      int quot = a / b;
      *remainder = a % b;
      return quot;
    }
    
    /* Average values in an array */
    double avg(double *a, int n) {
      int i;
      double total = 0.0;
      for (i = 0; i < n; i++) {
        total += a[i];
      }
      return total / n;
    }
    
    /* A C data structure */
    typedef struct Point {
        double x,y;
    } Point;
    
    /* Function involving a C data structure */
    double distance(Point *p1, Point *p2) {
       return hypot(p1->x - p2->x, p1->y - p2->y);
    }

pysample.c

#include "Python.h"
    #include "sample.h"
    
    /* int gcd(int, int) */
    static PyObject *py_gcd(PyObject *self, PyObject *args) {
      int x, y, result;
    
      if (!PyArg_ParseTuple(args,"ii", &x, &y)) {
        return NULL;
      }
      result = gcd(x,y);
      return Py_BuildValue("i", result);
    }
    
    /* int in_mandel(double, double, int) */
    static PyObject *py_in_mandel(PyObject *self, PyObject *args) {
      double x0, y0;
      int n;
      int result;
      
      if (!PyArg_ParseTuple(args, "ddi", &x0, &y0, &n)) {
        return NULL;
      }
      result = in_mandel(x0,y0,n);
      return Py_BuildValue("i", result);
    }
    
    /* int divide(int, int, int *) */
    static PyObject *py_divide(PyObject *self, PyObject *args) {
      int a, b, quotient, remainder;
      if (!PyArg_ParseTuple(args, "ii", &a, &b)) {
        return NULL;
      }
      quotient = divide(a,b, &remainder);
      return Py_BuildValue("(ii)", quotient, remainder);
    }
    
    /* Module method table */
    static PyMethodDef SampleMethods[] = {
      {"gcd",  py_gcd, METH_VARARGS, "Greatest common divisor"},
      {"in_mandel", py_in_mandel, METH_VARARGS, "Mandelbrot test"},
      {"divide", py_divide, METH_VARARGS, "Integer division"},
      { NULL, NULL, 0, NULL}
    };
    
    /* Module structure */
    static struct PyModuleDef samplemodule = {
      PyModuleDef_HEAD_INIT,
      "sample",           /* name of module */
      "A sample module",  /* Doc string (may be NULL) */
      -1,                 /* Size of per-interpreter state or -1 */
      SampleMethods       /* Method table */
    };
    
    /* Module initialization function */
    PyMODINIT_FUNC
    PyInit_sample(void) {
      return PyModule_Create(&samplemodule);
    }

setup.py

# setup.py
    from distutils.core import setup, Extension
    
    setup(name="sample", 
          ext_modules=[
            Extension("sample",
                      ["../sample.c", "pysample.c"],
                      include_dirs = ['..'],
                      )
            ]
    )

## 编译安装 ##

在目录subtest01下面，直接编译安装  
（注意一般情况下，大家直接使用python命令即可，我使用的是自己编译的版本python372，编译过程可参考：  
[https://blog.csdn.net/tanmx219/article/details/86518446][https_blog.csdn.net_tanmx219_article_details_86518446]）

$  python372 setup.py build\_ext --inplace  
此时会输出一些信息，

running build_ext
    building 'sample' extension
    creating build
    creating build/temp.linux-x86_64-3.7-pydebug
    gcc -pthread -Wno-unused-result -Wsign-compare -g -Og -Wall -fPIC -fPIC -I.. -I/usr/local/include/python3.7dm -c ../sample.c -o build/temp.linux-x86_64-3.7-pydebug/../sample.o
    gcc -pthread -Wno-unused-result -Wsign-compare -g -Og -Wall -fPIC -fPIC -I.. -I/usr/local/include/python3.7dm -c pysample.c -o build/temp.linux-x86_64-3.7-pydebug/pysample.o
    gcc -pthread -shared build/temp.linux-x86_64-3.7-pydebug/../sample.o build/temp.linux-x86_64-3.7-pydebug/pysample.o -L/usr/local/lib -lpython3.7dm -o /home/matthew/dev/tutorial/subtest01/sample.cpython-37dm-x86_64-linux-gnu.so

此时，如果你要在subtest01目录下启动python372，是可以直接使用sample这个组件的，如，

$ python372
    Python 3.7.2 (default, Jan 18 2019, 20:12:21) 
    [GCC 7.3.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sample
    >>> sample.gcd(35,42)
    7
    >>> sample.divide(42,8)
    (5, 2)
    >>> sample
    <module 'sample' from '~/dev/tutorial/subtest01/sample.cpython-37dm-x86_64-linux-gnu.so'>
    >>> exit()

但若要在任意目录下都能运行sample组件，则需要把组件拷贝到python的库目录下面，

$ sudo  cp  sample.cpython-37dm-x86_64-linux-gnu.so      /usr/local/lib/python3.7/lib-dynload/

测试完了就删除掉吧，没啥用

$ sudo rm -f  /usr/local/lib/python3.7/lib-dynload/sample.cpython-37dm-x86_64-linux-gnu.so

整个过程还是比较简单的。

如果你的工程项目比较大的话，相信都会综合应用一些相关的工具来实现，比如cmake，那么这些安装卸载的东西就能通过cmake脚本实现啦。

## 使用setuptools ##

本质上setuptools和distutils没有什么太大的不同，可以看作是distutils的增强版。所以如果你想使用setuptools，只要在setup.py中把distutils换成setuptools即可，下面我给出本演示中可以使用的完整的setup.py源码，编译安装命令仍然是上面那个：  
｀python setup.py build\_ext --inplace｀

# setup.py
    from setuptools import setup, Extension
    
    setup(name="sample", 
          ext_modules=[
            Extension("sample",
                      ["../sample.c", "pysample.c"],
                      include_dirs = ['..'],
                      )
            ]
    )

如果你想把打包发布的话，使用命令｀python setup.py sdist｀会得到一个压缩包，拷贝到你需要的地方解压缩再｀python setup.py install｀就可以了，这个过程这里就不再展开详述了。

## 其他工具 ##

其他工具如cython, swig我都没怎么用过，这里给个链接为参考：  
[https://blog.csdn.net/tanmx219/article/details/86666456][https_blog.csdn.net_tanmx219_article_details_86666456]

还有一个工具pybind11（python + c++11)，这个在pytorch源码third-party中有用到，源码量也比较少，参考：  
[https://github.com/pybind/pybind11][https_github.com_pybind_pybind11]

## 参考资料 ##

[https://packaging.python.org/][https_packaging.python.org]  
[https://media.readthedocs.org/pdf/python-packaging-user-guide/latest/python-packaging-user-guide.pdf][https_media.readthedocs.org_pdf_python-packaging-user-guide_latest_python-packaging-user-guide.pdf]  
[https://thomasnyberg.com/cpp\_extension\_modules.html][https_thomasnyberg.com_cpp_extension_modules.html]  
[https://thomasnyberg.com/what\_are\_extension\_modules.html][https_thomasnyberg.com_what_are_extension_modules.html]

![20190127142355598.png][]

[https_blog.csdn.net_tanmx219_article_details_86518446]: https://blog.csdn.net/tanmx219/article/details/86518446
[https_blog.csdn.net_tanmx219_article_details_86666456]: https://blog.csdn.net/tanmx219/article/details/86666456
[https_github.com_pybind_pybind11]: https://github.com/pybind/pybind11
[https_packaging.python.org]: https://packaging.python.org/
[https_media.readthedocs.org_pdf_python-packaging-user-guide_latest_python-packaging-user-guide.pdf]: https://media.readthedocs.org/pdf/python-packaging-user-guide/latest/python-packaging-user-guide.pdf
[https_thomasnyberg.com_cpp_extension_modules.html]: https://thomasnyberg.com/cpp_extension_modules.html
[https_thomasnyberg.com_what_are_extension_modules.html]: https://thomasnyberg.com/what_are_extension_modules.html
[20190127142355598.png]: /images/20220322/0448ac3b47b948bf83864cc4adef20ae.png