From 9f0b940780d9104dab8a70905c8d5831bfd1ffd2 Mon Sep 17 00:00:00 2001 From: "ggellner@encolpuis" Date: Sat, 27 Sep 2008 11:47:43 -0400 Subject: [PATCH] Started tutorial, begun refactoring the compilation section, and updating the limitations. --- conf.py | 4 +- docs/extension_types.rst | 4 +- docs/limitations.rst | 91 +++++++++++--- docs/overview.rst | 88 -------------- docs/source_files_and_compilation.rst | 80 +++++++++--- docs/tutorial.rst | 169 ++++++++++++++++++++++++++ index.rst | 1 + 7 files changed, 307 insertions(+), 130 deletions(-) create mode 100644 docs/tutorial.rst diff --git a/conf.py b/conf.py index 064b066b..449f4e00 100644 --- a/conf.py +++ b/conf.py @@ -40,9 +40,9 @@ copyright = '2008, Stefan Behnel, Robert Bradshaw, William Stein, Gary Furnish, # other places throughout the built documents. # # The short X.Y version. -version = '0.9.6' +version = '0.9.8.1' # The full version, including alpha/beta/rc tags. -release = '0.9.6.13.1' +release = '0.9.8.1' # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: diff --git a/docs/extension_types.rst b/docs/extension_types.rst index 2be5f895..95c8cb24 100644 --- a/docs/extension_types.rst +++ b/docs/extension_types.rst @@ -107,8 +107,8 @@ The same consideration applies to local variables, for example,:: cdef Shrubbery sh2 sh2 = Shrubbery() sh2.width = sh1.width - sh2.height = sh1.height - return sh2 + sh2.height = sh1.height + return sh2 Extension types and None ------------------------ diff --git a/docs/limitations.rst b/docs/limitations.rst index 4985e25e..8e2fbb67 100644 --- a/docs/limitations.rst +++ b/docs/limitations.rst @@ -1,29 +1,80 @@ .. _cython-limitations-label: +************* Limitations -=========== +************* -Unsupported Python features -^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Unsupported Python Features +============================ -Cython is not quite a full superset of Python. The following restrictions apply: +One of our goals is to make Cython as compatible as possible with standard +Python. This page lists the things that work in Python but not in Cython. -* Function definitions (whether using ``def`` or ``cdef``) cannot be nested within - other function definitions. -* Class definitions can only appear at the top level of a module, - not inside a function. -* The ``import *`` form of import is not allowed anywhere (other forms of the - import statement are fine, though). -* Generators cannot be defined in Cython. -* The ``globals()`` and ``locals()`` functions cannot be used. +.. TODO: this limitation seems to be removed +.. :: -The above restrictions will most likely remain, since removing them would be -difficult and they're not really needed for Cython's intended applications. +.. from module import * -There are also some temporary limitations, which may eventually be lifted, including: +.. This relies on at-runtime insertion of objects into the current namespace and +.. probably will be one of the few features never implemented (as any +.. implementation would be very slow). However, there is the --pre-import option +.. with treats all un-declared names as coming from the specified module, which +.. has the same effect as putting "from module import *" at the top-level of the +.. code. Note: the one difference is that builtins cannot be overriden in this +.. way, as the 'pre-import' scope is even higher than the builtin scope. +Nested def statements +---------------------- +Function definitions (whether using ``def`` or ``cdef``) cannot be nested within +other function definitions. :: + + def make_func(): + def f(x): + return x*x + return f + +(work in progress) This relies on functional closures + +Generators +----------- + +Using the yield keywords. (work in progress) This relies on functional closures + + +.. TODO Not really a limitation, rather an enchancement proposal + +.. Support for builtin types +.. -------------------------- + +.. Support for statically declaring types such as list and dict and sequence +.. should be provided, and optimized code produced. + +.. This needs to be well thought-out, and I think Pyrex has some plans along +.. these lines as well. + +Modulo '%' operation on floats +------------------------------- +:: + + a = b%c + +where `b` and `c` are floats will raise the error "Invalid operand types for '%' (float; float)" + +This can currently be worked around by putting:: + + cdef extern from "math.h": + double fmod(double x, double y) + +somewhere is the source file and then using:: + + a = fmod(b,c) + + +Other Current Limitations +========================== + +* The :func:`globals` and :func:`locals` functions cannot be used. * Class and function definitions cannot be placed inside control structures. -* There is no support for Unicode. * Special methods of extension types cannot have functioning docstrings. * The use of string literals as comments is not recommended at present, because Cython doesn't optimize them away, and won't even accept them in places @@ -38,7 +89,7 @@ Behaviour of class scopes In Python, referring to a method of a class inside the class definition, i.e. while the class is being defined, yields a plain function object, but in Cython it yields an unbound method [#]_. A consequence of this is that the -usual idiom for using the ``classmethod`` and ``staticmethod`` functions, +usual idiom for using the :func:`classmethod` and :func:`staticmethod` functions, e.g.:: class Spam: @@ -62,8 +113,8 @@ outside the class, and then assigning the result of ``classmethod`` or .. rubric:: Footnotes .. [#] The reason for the different behaviour of class scopes is that - Cython-defined Python functions are PyCFunction objects, not PyFunction - objects, and are not recognised by the machinery that creates a bound - or unbound method when a function is extracted from a class. To get + Cython-defined Python functions are ``PyCFunction`` objects, not + ``PyFunction`` objects, and are not recognised by the machinery that creates a + bound or unbound method when a function is extracted from a class. To get around this, Cython wraps each method in an unbound method object itself before storing it in the class's dictionary. diff --git a/docs/overview.rst b/docs/overview.rst index 6e3610d9..86ad46b1 100644 --- a/docs/overview.rst +++ b/docs/overview.rst @@ -57,94 +57,6 @@ behind the scenes, just as it is in interpreted Python code. And what's more, Cython lets you define new built-in Python types just as easily as you can define new classes in Python. -Sound too good to be true? Read on and find out how it's done. - -The Basics of Cython -==================== - -The fundamental nature of Cython can be summed up as follows: Cython is Python -with C data types. - -Cython is Python: Almost any piece of Python code is also valid Cython code. -(There are a few :ref:`cython-limitations-label`, but this approximation will -serve for now.) The Cython compiler will convert it into C code which makes -equivalent calls to the Python/C API. - -But Cython is much more than that, because parameters and variables can be -declared to have C data types. Code which manipulates Python values and C -values can be freely intermixed, with conversions occurring automatically -wherever possible. Reference count maintenance and error checking of Python -operations is also automatic, and the full power of Python's exception -handling facilities, including the try-except and try-finally statements, is -available to you -- even in the midst of manipulating C data. - -Here's a small example showing some of what can be done. It's a routine for -finding prime numbers. You tell it how many primes you want, and it returns -them as a Python list. - -:file:`primes.pyx`: - -.. code-block:: none - :linenos: - - def primes(int kmax): - cdef int n, k, i - cdef int p[1000] - result = [] - if kmax > 1000: - kmax = 1000 - k = 0 - n = 2 - while k < kmax: - i = 0 - while i < k and n % p[i] != 0: - i = i + 1 - if i == k: - p[k] = n - k = k + 1 - result.append(n) - n = n + 1 - return result - -You'll see that it starts out just like a normal Python function definition, -except that the parameter ``kmax`` is declared to be of type ``int`` . This -means that the object passed will be converted to a C integer (or a -``TypeError.`` will be raised if it can't be). - -Lines 2 and 3 use the ``cdef`` statement to define some local C variables. -Line 4 creates a Python list which will be used to return the result. You'll -notice that this is done exactly the same way it would be in Python. Because -the variable result hasn't been given a type, it is assumed to hold a Python -object. - -Lines 7-9 set up for a loop which will test candidate numbers for primeness -until the required number of primes has been found. Lines 11-12, which try -dividing a candidate by all the primes found so far, are of particular -interest. Because no Python objects are referred to, the loop is translated -entirely into C code, and thus runs very fast. - -When a prime is found, lines 14-15 add it to the p array for fast access by -the testing loop, and line 16 adds it to the result list. Again, you'll notice -that line 16 looks very much like a Python statement, and in fact it is, with -the twist that the C parameter ``n`` is automatically converted to a Python -object before being passed to the append method. Finally, at line 18, a normal -Python return statement returns the result list. - -Compiling primes.pyx with the Cython compiler produces an extension module -which we can try out in the interactive interpreter as follows:: - - >>> import primes - >>> primes.primes(10) - [2, 3, 5, 7, 11, 13, 17, 19, 23, 29] - -See, it works! And if you're curious about how much work Cython has saved you, -take a look at the C code generated for this module. - -Language Details -================ - -For more about the Cython language, see :ref:`language-basics-label`. - Future Plans ============ Cython is not finished. Substantial tasks remaining. See diff --git a/docs/source_files_and_compilation.rst b/docs/source_files_and_compilation.rst index e15d9503..fff97783 100644 --- a/docs/source_files_and_compilation.rst +++ b/docs/source_files_and_compilation.rst @@ -1,4 +1,4 @@ -.. TODO: Rewrite this to be more comprehensive, with examples! +.. _compilation_label: **************************** Source Files and Compilation @@ -8,16 +8,6 @@ Cython source file names consist of the name of the module followed by a ``.pyx`` extension, for example a module called primes would have a source file named :file:`primes.pyx`. -If your module is destined to live in a package, the source file name should -include the full dotted name that the module will eventually have. For -example, a module called primes that will be installed in a package called -numbers should have a source file called numbers.primes.pyx. This will ensure -that the :attr:`__name__` properties of the module and any classes defined in -it are set correctly. If you don't do this, you may find that pickling doesn't -work, among other problems. It also ensures that the Cython compiler has the -right idea about the layout of the module namespace, which can be important -when accessing extension types defined in other modules. - Once you have written your ``.pyx`` file, there are a couple of ways of turning it into an extension module. One way is to compile it manually with the Cython compiler, e.g.:: @@ -26,12 +16,66 @@ compiler, e.g.:: This will produce a file called :file:`primes.c`, which then needs to be compiled with the C compiler using whatever options are appropriate on your -platform for generating an extension module. There's a Makefile in the Demos -directory (called Makefile.nodistutils) that shows how to do this for Linux. +platform for generating an extension module. For these options look at the +official Python documentation. The other, and probably better, way is to use the :mod:`distutils` extension -provided with Cython. See the :file:`setup.py` file in the Demos directory for an -example of how to use it. This method has the advantage of being -cross-platform -- the same setup file should work on any platform where -:mod:`distutils` can compile an -extension module. +provided with Cython. The benifit of this method is that it will give the +platform specific compilation options, acting like a stripped down autotools. + +Basic setup.py +=============== +The distutils extension provided with Cython allows you to pass ``.pyx`` files +directly to the ``Extension`` constructor in your setup file. + +If you have a single Cython file that you want to turn into a compiled +extension, say with filename :file:`example.pyx` the associated :file:`setup.py` +would be:: + + from distutils.core import setup + from distutils.extension import Extension + from Cython.Distutils import build_ext + + setup( + cmdclass = {'build_ext': build_ext}, + ext_modules = [Extension("example", ["example.pyx"])] + ) + +To understand the :file:`setup.py` more fully look at the official +``distutils`` documentation. To compile the extension for use in the +current directory use:: + + $ python setup.py build_ext --inplace + +Cython Files Depending on C Files +=================================== + +TODO + +Multiple Cython Files in a Package +=================================== + +TODO + +Distributing Pyrex modules +=========================== +It is strongly recommended that you distribute the generated ``.c`` files as well +as your Cython sources, so that users can install your module without needing +to have Cython available. + +It is also recommended that Cython compilation not be enabled by default in the +version you distribute. Even if the user has Cython installed, he probably +doesn't want to use it just to install your module. Also, the version he has +may not be the same one you used, and may not compile your sources correctly. + +This simply means that the :file:`setup.py` file that you ship with will just +be a normal distutils file on the generated `.c` files, for the basic example +we would have instead:: + + from distutils.core import setup + from distutils.extension import Extension + + setup( + ext_modules = [Extension("example", ["example.c"])] + ) + diff --git a/docs/tutorial.rst b/docs/tutorial.rst new file mode 100644 index 00000000..0f66b85c --- /dev/null +++ b/docs/tutorial.rst @@ -0,0 +1,169 @@ +.. _tutorial_label: + +********* +Tutorial +********* + +The Basics of Cython +==================== + +The fundamental nature of Cython can be summed up as follows: Cython is Python +with C data types. + +Cython is Python: Almost any piece of Python code is also valid Cython code. +(There are a few :ref:`cython-limitations-label`, but this approximation will +serve for now.) The Cython compiler will convert it into C code which makes +equivalent calls to the Python/C API. + +But Cython is much more than that, because parameters and variables can be +declared to have C data types. Code which manipulates Python values and C +values can be freely intermixed, with conversions occurring automatically +wherever possible. Reference count maintenance and error checking of Python +operations is also automatic, and the full power of Python's exception +handling facilities, including the try-except and try-finally statements, is +available to you -- even in the midst of manipulating C data. + + + +Cython Hello World +=================== + +As Cython can accept almost any valid plain python file (though see the +`exceptions`), one of the hardest things in getting started is just figuring +out how to compile your file. + +So lets start with the canonical python hello world:: + + print "Hello World" + +So the first thing to do is rename the file to :file:`helloworld.pyx`. Now we +need to make the :file:`setup.py`, which is like a python Makefile if you are +familiar.:: + + from distutils.core import setup + from distutils.extension import Extension + from Cython.Distutils import build_ext + + setup( + cmdclass = {'build_ext': build_ext}, + ext_modules = [Extension("helloworld", ["helloworld.pyx"])] + ) + +To use this to build your Cython file use the commandline options:: + + $ python setup.py build_ext --inplace + +Which will leave a file in your local directory called `helloworld.so`. Now to +use this file start the python interpreter and:: + + >>> import helloworld + "Hello World" + +Congratulations! You know know how to build a Cython extension. But So Far +this example doesn't really show us why we would even want to use Cython, so +lets do a more realistic example. + +Fibonacci Fun +============== + +From the official Python tutorial a simple fibonacci function is defined as:: + + def fib(n): + """Print the Fibonacci series up to n.""" + a, b = 0, 1 + while b < n: + print b, + a, b = b, a + b + +Now following the steps for the Hello World example we first rename the file +to have a `.pyx` extension, lets say :file:`fib.pyx`, then we create the +:file:`setup.py` file. Using the file created for the Hello World example, all +that you need to change is the name of the Cython filename, and the resulting +module name, doing this we have:: + + from distutils.core import setup + from distutils.extension import Extension + from Cython.Distutils import build_ext + + setup( + cmdclass = {'build_ext': build_ext}, + ext_modules = [Extension("fib", ["fib.pyx"])] + ) + +Build the extension with the same command used for the helloworld.pyx:: + + $ python setup.py build_ext --inplace + +And use the new extension with:: + + >>> import fib + >>> fib.fib(2000) + 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 + +Primes +======= + +Here's a small example showing some of what can be done. It's a routine for +finding prime numbers. You tell it how many primes you want, and it returns +them as a Python list. + +:file:`primes.pyx`: :: + + def primes(int kmax): + cdef int n, k, i + cdef int p[1000] + result = [] + if kmax > 1000: + kmax = 1000 + k = 0 + n = 2 + while k < kmax: + i = 0 + while i < k and n % p[i] != 0: + i = i + 1 + if i == k: + p[k] = n + k = k + 1 + result.append(n) + n = n + 1 + return result + +You'll see that it starts out just like a normal Python function definition, +except that the parameter ``kmax`` is declared to be of type ``int`` . This +means that the object passed will be converted to a C integer (or a +``TypeError.`` will be raised if it can't be). + +Lines 2 and 3 use the ``cdef`` statement to define some local C variables. +Line 4 creates a Python list which will be used to return the result. You'll +notice that this is done exactly the same way it would be in Python. Because +the variable result hasn't been given a type, it is assumed to hold a Python +object. + +Lines 7-9 set up for a loop which will test candidate numbers for primeness +until the required number of primes has been found. Lines 11-12, which try +dividing a candidate by all the primes found so far, are of particular +interest. Because no Python objects are referred to, the loop is translated +entirely into C code, and thus runs very fast. + +When a prime is found, lines 14-15 add it to the p array for fast access by +the testing loop, and line 16 adds it to the result list. Again, you'll notice +that line 16 looks very much like a Python statement, and in fact it is, with +the twist that the C parameter ``n`` is automatically converted to a Python +object before being passed to the append method. Finally, at line 18, a normal +Python return statement returns the result list. + +Compiling primes.pyx with the Cython compiler produces an extension module +which we can try out in the interactive interpreter as follows:: + + >>> import primes + >>> primes.primes(10) + [2, 3, 5, 7, 11, 13, 17, 19, 23, 29] + +See, it works! And if you're curious about how much work Cython has saved you, +take a look at the C code generated for this module. + +Language Details +================ + +For more about the Cython language, see :ref:`language-basics-label`. + diff --git a/index.rst b/index.rst index 221d928d..053204be 100644 --- a/index.rst +++ b/index.rst @@ -11,6 +11,7 @@ Contents: :maxdepth: 2 docs/overview + docs/tutorial docs/language_basics docs/extension_types docs/sharing_declarations -- 2.26.2