From 81c1140b7fa2b1fda0e29b0a1d7d869abf4b6dc8 Mon Sep 17 00:00:00 2001 From: "Fabien@laptop" Date: Mon, 5 Jul 2010 22:24:27 +0900 Subject: [PATCH] Updated wrapping_CPlusPlus.html for Cython v0.13 --- src/userguide/wrapping_CPlusPlus.rst | 402 +++++++++++++++------------ 1 file changed, 231 insertions(+), 171 deletions(-) diff --git a/src/userguide/wrapping_CPlusPlus.rst b/src/userguide/wrapping_CPlusPlus.rst index 5c302fa2..e09c8341 100644 --- a/src/userguide/wrapping_CPlusPlus.rst +++ b/src/userguide/wrapping_CPlusPlus.rst @@ -9,33 +9,36 @@ Wrapping C++ Classes in Cython Overview ========= -This page aims to get you quickly up to speed so you can wrap C++ interfaces -with a minimum of pain and 'surprises'. +Cython v0.13 introduces native support for most of the C++ language. This means that the previous tricks that were used to wrap C++ classes (as described in http://wiki.cython.org/WrappingCPlusPlus_ForCython012AndLower) are no longer useful. -In the past, Pyrex only supported wrapping of C APIs, and not C++. To wrap -C++, one had to write a pure-C shim, containing functions for -constructors/destructors and method invocations. Object pointers were passed -around as opaque void pointers, and cast to/from object pointers as needed. -This approach did work, but it got awfully messy and error-prone when trying -to wrap APIs with large class hierarchies and lots of inheritance. +Wrapping C++ classes with Cython is now much more straightforward. This document describe in details the new way of wrapping C++ code. -These days, though, Pyrex offers an adequate bare minimum of C++ support, -which Cython has inherited. The approach described in this document will help -you wrap a lot of C++ code with only moderate effort. There are some -limitations, which we will discuss at the end of the document. +What's new in Cython v0.13 about C++ +--------------------------------------------------- + +For users of previous Cython versions, here is a brief overview of the main new features of Cython v0.13 regarding C++ support: + +* C++ objects can now be dynamically allocated with ``new`` and ``del`` keywords. +* C++ objects can now be stack-allocated. +* C++ classes can be declared with the new keyword ``cppclass``. +* Templated classes are supported. +* Overloaded functions are supported. +* Overloading of C++ operators (such as operator+, operator[],...) is supported. Procedure Overview -==================== +------------------- +The general procedure for wrapping a C++ file can now be described as follow: * Specify C++ language in :file:`setup.py` script -* Create ``cdef extern from`` blocks and declare classes as - ``ctypedef struct`` blocks -* Create constructors and destructors -* Add class methods as function pointers -* Create Cython wrapper class +* Create ``cdef extern from`` blocks with the optional namespace (if exists) and the namespace name as string +* Declare classes as ``cdef cppclass`` blocks +* Declare public attributes (variables, methods and constructors) + +A simple Tutorial +================== An example C++ API -=================== +------------------- Here is a tiny C++ API which we will use as an example throughout this document. Let's assume it will be in a header file called @@ -43,17 +46,19 @@ document. Let's assume it will be in a header file called .. sourcecode:: c++ - class Rectangle { - public: - int x0, y0, x1, y1; - Rectangle(int x0, int y0, int x1, int y1); - ~Rectangle(); - int getLength(); - int getHeight(); - int getArea(); - void move(int dx, int dy); - }; - + namespace shapes { + class Rectangle { + public: + int x0, y0, x1, y1; + Rectangle(int x0, int y0, int x1, int y1); + ~Rectangle(); + int getLength(); + int getHeight(); + int getArea(); + void move(int dx, int dy); + }; + } + and the implementation in the file called :file:`Rectangle.cpp`: .. sourcecode:: c++ @@ -98,11 +103,11 @@ and the implementation in the file called :file:`Rectangle.cpp`: This is pretty dumb, but should suffice to demonstrate the steps involved. Specify C++ language in setup.py -================================= +--------------------------------- In Cython :file:`setup.py` scripts, one normally instantiates an Extension object. To make Cython generate and compile a C++ source, you just need -to add a keyword to your Extension construction statement, as in:: +to add the keyword ``language="c++"`` to your Extension construction statement, as in:: ext = Extension( "rectangle", # name of extension @@ -114,103 +119,85 @@ to add a keyword to your Extension construction statement, as in:: cmdclass = {'build_ext': build_ext} ) -and Cython will generate and compile the :file:`rectangle.cpp` file (from the +Cython will generate and compile the :file:`rectangle.cpp` file (from the :file:`rectangle.pyx`), then it will compile :file:`Rectangle.cpp` (implementation of the ``Rectangle`` class) and link both objects files together into :file:`rectangle.so`, which you can then import in Python using ``import rectangle`` (if you forget to link the :file:`Rectangle.o`, you will get missing symbols while importing the library in Python). -With the language="c++" keyword, Cython distutils will generate a C++ file. -Create cdef extern from block -============================== +Alternatively, one can also use the ``cython`` command-line utility to generate a C++ ``.cpp`` file, and then compile it into a python extension. C++ mode for the ``cython`` command is turned on with the ``--cplus`` option. + +Declaring a C++ class interface +-------------------------------- The procedure for wrapping a C++ class is quite similar to that for wrapping normal C structs, with a couple of additions. Let's start here by creating the basic ``cdef extern from`` block:: - cdef extern from "Rectangle.h": + cdef extern from "Rectangle.h" namespace "shapes": -This will make the C++ class def for Rectangle available. +This will make the C++ class def for Rectangle available. Note the namespace declaration. -Declare class as a ctypedef struct ------------------------------------ +Declare class with cdef cppclass +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Now, let's add the Rectangle class to this extern from block -- just copy the -class def from :file:`Rectangle.h` and adjust for Cython syntax, so now it -becomes:: +Now, let's add the Rectangle class to this extern from block - just copy the class name from Rectangle.h and adjust for Cython syntax, so now it becomes:: - cdef extern from "Rectangle.h": - # known in Cython namespace as 'c_Rectangle' but in C++ as 'Rectangle' - ctypedef struct c_Rectangle "Rectangle": - int x0, y0, x1, y1 - -We don't have any way of accessing the constructor/destructor or methods, but -we'll cover this now. + cdef extern from "Rectangle.h" namespace "shapes": + cdef cppclass Rectangle: + +Add public attributes +^^^^^^^^^^^^^^^^^^^^^^ -Add constructors and destructors ----------------------------------- +We now need to declare the attributes for use on Cython:: -We now need to expose a constructor and destructor into the Cython -namespace. Again, we'll be using C name specifications:: - - cdef extern from "Rectangle.h": - ctypedef struct c_Rectangle "Rectangle": + cdef extern from "Rectangle.h" namespace "shapes": + cdef cppclass Rectangle: + Rectangle(int, int, int, int) int x0, y0, x1, y1 - c_Rectangle *new_Rectangle "new Rectangle" (int x0, int y0, int x1, int y1) - void del_Rectangle "delete" (c_Rectangle *rect) - -Add class methods -------------------- + int getLength() + int getHeight() + int getArea() + void move(int, int) -Now, let's add the class methods. You can circumvent Cython syntax -limitations by declaring these as function pointers. Recall that in the C++ -class we have: +Declare a var with the wrapped C++ class +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. sourcecode:: c++ +Now, we use cdef to declare a var of the class with the know C++ ``new`` statement:: - int getLength(); - int getHeight(); - int getArea(); - void move(int dx, int dy); + cdef Rectangle *rec = new Rectangle(1, 2, 3, 4) + cdef int recLength = rec.getLength() + ... + del rec #delete heap allocated object -So if we convert each of these to function pointers and stick them in our -extern block, we now get:: +It's also possible to declare a stack allocated object, but it's necessary to have a "default" constructor:: - cdef extern from "Rectangle.h": - ctypedef struct c_Rectangle "Rectangle": - int x0, y0, x1, y1 - int getLength() - int getHeight() - int getArea() - void move(int dx, int dy) - c_Rectangle *new_Rectangle "new Rectangle" (int x0, int y0, int x1, int y1) - void del_Rectangle "delete" (c_Rectangle *rect) + cdef extern from "Foo.h": + cdef cppclass Foo: + Foo() -This will fool Cython into generating C++ method calls even though -Cython is mostly oblivious to C++. + cdef Foo foo -In Pyrex you must explicitly declare these as function pointers, i.e. -``(int *getArea)()``. +Note that, like C++, if the class has only one constructor and it is a default one, it's not necessary to declare it. Create Cython wrapper class -============================= +---------------------------- -At this point, we have exposed into our pyx file's namespace a struct which -gives us access to the interface of a C++ Rectangle type. Now, we need to make +At this point, we have exposed into our pyx file's namespace the interface of the C++ Rectangle type. Now, we need to make this accessible from external Python code (which is our whole point). Common programming practice is to create a Cython extension type which holds a C++ instance pointer as an attribute ``thisptr``, and create a bunch of forwarding methods. So we can implement the Python extension type as:: - cdef class Rectangle: - cdef c_Rectangle *thisptr # hold a C++ instance which we're wrapping + cdef class PyRectangle: + cdef Rectangle *thisptr # hold a C++ instance which we're wrapping def __cinit__(self, int x0, int y0, int x1, int y1): - self.thisptr = new_Rectangle(x0, y0, x1, y1) + self.thisptr = new Rectangle(x0, y0, x1, y1) def __dealloc__(self): - del_Rectangle(self.thisptr) + del self.thisptr def getLength(self): return self.thisptr.getLength() def getHeight(self): @@ -229,62 +216,141 @@ attribute access, you could just implement some properties:: def __set__(self, x0): self.thisptr.x0 = x0 ... -Caveats and Limitations -======================== -In this document, we have discussed a relatively straightforward way of -wrapping C++ classes with Cython. However, there are some limitations in -this approach, some of which could be overcome with clever workarounds (anyone -here want to share some?), but some of which will require new features in -Cython. +Advanced C++ features +====================== -The major limitations I'm most immediately aware of (and there will be many -more) include: +We describe here all the C++ features that were not discussed in the above tutorial. Overloading ------------ -Presently, it's not easy to overload methods or constructors, but there may be -a workaround if you try some creative C name specifications +Overloading is very simple. Just declare the method with different parameters and use any of them:: -Access to C-only functions ---------------------------- + cdef extern from "Foo.h": + cdef cppclass Foo: + Foo(int) + Foo(bool) + Foo(int, bool) + Foo(int, int) -Whenever generating C++ code, Cython generates declarations of and calls -to functions assuming these functions are C++ (ie, not declared as extern "C" -{...} . This is ok if the C functions have C++ entry points, but if they're C -only, you will hit a roadblock. If you have a C++ Cython module needing -to make calls to pure-C functions, you will need to write a small C++ shim -module which: +Overloading operators +---------------------- -* includes the needed C headers in an extern "C" block -* contains minimal forwarding functions in C++, each of which calls the - respective pure-C function +Cython uses C++ for overloading operators:: -Inherited C++ methods ----------------------- + cdef extern from "foo.h": + cdef cppclass Foo: + Foo() + Foo* operator+(Foo*) + Foo* operator-(Foo) + int operator*(Foo*) + int operator/(int) -If you have a class ``Foo`` with a child class ``Bar``, and ``Foo`` has a -method :meth:`fred`, then you'll have to cast to access this method from -``Bar`` objects. -For example:: + cdef Foo* foo = new Foo() + cdef int x - class MyClass: - Bar *b - ... - def myfunc(self): - ... - b.fred() # wrong, won't work - ((self.b)).fred() # should work, Cython now thinks it's a 'Foo' + cdef Foo* foo2 = foo[0] + foo + foo2 = foo[0] - foo[0] -It might take some experimenting by others (you?) to find the most elegant -ways of handling this issue. + x = foo[0] * foo2 + x = foo[0] / 1 -Advanced C++ features ----------------------- + cdef Foo f + foo = f + &f + foo2 = f - f + + del foo, foo2 + +Nested class declarations +-------------------------- +C++ allows nested class declaration. Class declarations can also be nested in Cython:: + + + cdef extern from "" namespace "std": + cdef cppclass vector[T]: + cppclass iterator: + T operator*() + iterator operator++() + bint operator==(iterator) + bint operator!=(iterator) + vector() + void push_back(T&) + T& operator[](int) + T& at(int) + iterator begin() + iterator end() + + cdef vector[int].iterator iter #iter is declared as being of type vector::iterator + +Note that the nested class is declared with a ``cppclass`` but without a ``cdef``. + +C++ operators not compatible with Python syntax +------------------------------------------------ + +Cython try to keep a syntax as close as possible to standard Python. Because of this, certain C++ operators, like the preincrement ``++foo`` or the dereferencing operator ``*foo`` cannot be used with the same syntax as C++. Cython provides functions replacing these operators in a special module ``cython.operator``. The functions provided are: + +* ``cython.operator.dereference`` for dereferencing. ``dereference(foo)`` will produce the C++ code ``*foo`` +* ``cython.operator.preincrement`` for pre-incrementation. ``preincrement(foo)`` will produce the C++ code ``++foo`` +* ... + +These functions need to be cimported. Of course, one can use a ``from ... cimport ... as`` to have shorter and more readable functions. For example: ``from cython.operator cimport dereference as deref``. + +Templates +---------- + +Cython uses a bracket syntax for templating. A simple example for wrapping C++ vector:: + + from cython.operator cimport dereference as deref, preincrement as inc #dereference and increment operators + + cdef extern from "" namespace "std": + cdef cppclass vector[T]: + cppclass iterator: + T operator*() + iterator operator++() + bint operator==(iterator) + bint operator!=(iterator) + vector() + void push_back(T&) + T& operator[](int) + T& at(int) + iterator begin() + iterator end() + + cdef vector[int] *v = new vector[int]() + cdef int i + for i in range(10): + v.push_back(i) + + cdef vector[int].iterator it = v.begin() + while it != v.end(): + print deref(it) + inc(it) + + del v + +Multiple template parameters can be defined as a list, such as [T, U, V] or [int, bool, char]. + +Standard library +----------------- + +Most of the containers of the C++ Standard Library have been declared in pxd files located in ``/Cython/Includes/libcpp``. These containers are: deque, list, map, pair, queue, set, stack, vector. + +For example:: + + from libcpp.vector cimport vector + + cdef vector[int] vect + cdef int i + for i in range(10): + vect.push_back(i) + for i in range(10): + print vect[i] + +The pxd files in ``/Cython/Includes/libcpp`` also work as good examples on how to declare C++ classes. Exceptions -^^^^^^^^^^^ +----------- Cython cannot throw C++ exceptions, or catch them with a try-except statement, but it is possible to declare a function as potentially raising an C++ @@ -310,57 +376,51 @@ called, which allows one to do custom C++ to Python error "translations." If raise_py_error does not actually raise an exception a RuntimeError will be raised. -Templates -^^^^^^^^^^ - -Cython does not natively understand C++ templates but we can put them to use -in some way. As an example consider an STL vector of C ints:: - cdef extern from "some .h file which includes ": - ctypedef struct intvec "std::vector": - void (* push_back)(int elem) - intvec intvec_factory "std::vector"(int len) - -now we can use the vector like this:: - - cdef intvec v = intvec_factory(2) - v.push_back(2) - -Overloading -^^^^^^^^^^^^ - -To support function overloading simply add a different alias to each -signature, so if you have e.g. +Caveats and Limitations +======================== -.. sourcecode:: c++ +Access to C-only functions +--------------------------- - int foo(int a); - int foo(int a, int b); +Whenever generating C++ code, Cython generates declarations of and calls +to functions assuming these functions are C++ (ie, not declared as extern "C" +{...} . This is ok if the C functions have C++ entry points, but if they're C +only, you will hit a roadblock. If you have a C++ Cython module needing +to make calls to pure-C functions, you will need to write a small C++ shim +module which: -in your C++ header then interface it like this in your :: +* includes the needed C headers in an extern "C" block +* contains minimal forwarding functions in C++, each of which calls the + respective pure-C function - int fooi "foo"(int) - int fooii "foo"(int, int) +Inherited C++ methods +---------------------- -Operators -^^^^^^^^^^ +If you have a class ``Foo`` with a child class ``Bar``, and ``Foo`` has a +method :meth:`fred`, then you'll have to cast to access this method from +``Bar`` objects. +For example:: -Some operators (e.g. +,-,...) can be accessed from Cython like this:: + cdef class MyClass: + Bar *b + ... + def myfunc(self): + ... + b.fred() # wrong, won't work + ((self.b)).fred() # should work, Cython now thinks it's a 'Foo' - ctypedef struct c_Rectangle "Rectangle": - c_Rectangle add "operator+"(c_Rectangle right) +It might take some experimenting by others (you?) to find the most elegant +ways of handling this issue. Declaring/Using References -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +--------------------------- Question: How do you declare and call a function that takes a reference as an argument? -Conclusion -============ +C++ left-values +---------------- + +C++ allows functions returning a reference to be left-values. This is currently not supported in Cython. ``cython.operator.dereference(foo)`` is also not considered a left-value. -A great many existing C++ classes can be wrapped using these techniques, in a -way much easier than writing a large messy C shim module. There's a bit of -manual work involved, and an annoying maintenance burden if the C++ library -you're wrapping is frequently changing, but this recipe should hopefully keep -the discomfort to a minimum. -- 2.26.2