1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head><meta content="text/html; charset=ISO-8859-1" http-equiv="content-type"><title>Source Files and Compilation</title></head>
<body>
<h1>
<hr style="width: 100%; height: 2px;">
Source
Files and Compilation
<hr style="width: 100%; height: 2px;">
</h1><ul id="mozToc"><!--mozToc h2 1 h3 2--><li><a href="#mozTocId581403">File Names and Extensions</a></li><li><a href="#mozTocId146379">Modules in Packages</a></li><li><a href="#mozTocId431365">Building an Extension</a><ul><li><a href="#mozTocId233830">Command Line</a></li><li><a href="#mozTocId49372">Calling the Pyrex compiler from Python</a></li><li><a href="#mozTocId11953">Using the distutils extension</a></li></ul></li><li><a href="#mozTocId559484">Distributing Pyrex modules</a></li></ul><h2><a class="mozTocH2" name="mozTocId581403"></a>File Names and Extensions</h2>Pyrex
source file names consist of the name of the module followed by a <span style="font-family: monospace;">.pyx</span> extension,
for example a module called <span style="font-family: monospace;">primes</span>
would have a source file named <span style="font-family: monospace;">primes.pyx</span>.<br><h2><a class="mozTocH2" name="mozTocId146379"></a>Modules in Packages</h2>If
your module is destined to live in a package, the Pyrex compiler needs
to know the fully-qualified name that the module will eventually have.<br><br>There are currently two ways to give it this information:<br><ol><li>Name the source file with the full dotted name of the module. For
example, a module called <span style="font-family: monospace;">primes</span> to be installed in a package called <span style="font-family: monospace;">numbers</span> would
have a source file called <span style="font-family: monospace;">numbers.primes.pyx</span>.<br><br></li><li>Place the source file in a <span style="font-style: italic;">package directory</span>. To Pyrex, a package directory is one that contains a file called either <span style="font-family: monospace;">__init__.py</span> or <span style="font-family: monospace;">__init__.pyx</span>. For example, a package called <span style="font-family: monospace;">numbers</span> containing a module called <span style="font-family: monospace;">primes</span> would have the source files laid out like this:</li></ol><div style="margin-left: 80px;"><span style="font-family: monospace;">numbers</span><br style="font-family: monospace;"><div style="margin-left: 40px;"><span style="font-family: monospace;">__init__.py</span><br style="font-family: monospace;"><span style="font-family: monospace;">primes.pyx</span><br></div></div><br>This will ensure that the __name__ properties of the module and any
classes defined in it are set correctly. If you don't do this, you may
find that pickling doesn't work, among other problems. It also ensures
that the Pyrex compiler has the right idea about the layout of the
module namespace, which can be important when accessing extension types
defined in other modules.<br>
<h2><a class="mozTocH2" name="mozTocId431365"></a>Building an Extension</h2>There are two steps involved in creating an extension module from Pyres sources:<br><ol><li>Use the Pyrex compiler to translate the .pyx file into a .c file.</li><li>Compile the .c file with a C compiler and link it with whatever libraries it needs, to produce an extension module.</li></ol>There
are a variety of ways of accomplishing these steps, either separately
or together. One way is to compile the Pyrex source manually from the
command line
with the Pyrex compiler, e.g.<br>
<br><div style="margin-left: 40px;"><span style="font-family: monospace;">pyrexc -r primes.pyx</span><br>
</div><br>
This will compile <span style="font-family: monospace;">primes.pyx</span>
and any other source files that it depends on, if any of them have
changed since the last compilation, and produce a file called <span style="font-family: monospace;">primes.c</span>,
which then needs to be compiled with the C compiler using whatever
options are appropriate on your platform for generating an extension
module.<br><br>You
can perform the C compilation using distutils and a <span style="font-family: monospace;">setup.py</span> file, or with a conventional
Makefile. There's a Makefile in the Demos directory (called <span style="font-family: monospace;">Makefile.nodistutils</span>)
that shows how to do this for Linux.<br><br>Another approach is to put code at the beginning of your <span style="font-family: monospace;">setup.py</span> file to import the Pyrex compiler and call it from Python. You can then follow this with a normal call to <span style="font-family: monospace;">setup()</span> to compile the resulting .c files.<br>
<br>You can also perform both steps at once in a <span style="font-family: monospace;">setup.py</span> file using the distutils
extension provided with Pyrex. See the <span style="font-family: monospace;">Setup.py</span>
file in the <span style="font-family: monospace;">Demos</span>
directory for an example of how to use it. A disadvantage of this
method is that you won't be able to take advantage of Pyrex's own
dependency checking features to compile only the Pyrex sources which
have changed.<br><h3><a class="mozTocH3" name="mozTocId233830"></a>Command Line</h3>You can run the Pyrex compiler from the command line using either the <span style="font-family: monospace;">pyrexc</span> shell command or the Python version of it, <span style="font-family: monospace;">pyrexc.py</span>.<br><br>The following command line options exist:<br><br><table style="text-align: left; margin-left: 40px;" border="1" cellpadding="2" cellspacing="2"><tbody><tr><td style="font-weight: bold;" align="left" nowrap="nowrap" valign="top">Short</td><td style="font-weight: bold;" align="left" nowrap="nowrap" valign="top">Long</td><td style="font-weight: bold;" align="left" nowrap="nowrap" valign="top">Description</td></tr><tr><td align="left" nowrap="nowrap" valign="top">-v</td><td align="left" nowrap="nowrap" valign="top">--version</td><td align="left" nowrap="nowrap" valign="top">Display the version number of the Pyrex compiler</td></tr><tr><td align="left" nowrap="nowrap" valign="top">-l</td><td align="left" nowrap="nowrap" valign="top">--create-listing</td><td align="left" nowrap="nowrap" valign="top">Produces a .lis file for each compiled .pyx file containing error messages</td></tr><tr><td align="left" nowrap="nowrap" valign="top">-I</td><td align="left" nowrap="nowrap" valign="top">--include-dir</td><td style="vertical-align: top; text-align: left; width: 200px;">Specifies
a directory to be searched for included files and top-level package
directories. Multiple -I options may be given, each specifying one
directory.</td></tr><tr><td align="left" nowrap="nowrap" valign="top">-o</td><td align="left" nowrap="nowrap" valign="top">--output-file</td><td style="vertical-align: top; text-align: left; width: 200px;">Specifies name of generated C file. Only meaningful when a single .pyx file is being compiled.</td></tr><tr><td align="left" nowrap="nowrap" valign="top">-r</td><td align="left" nowrap="nowrap" valign="top">--recursive</td><td style="vertical-align: top; text-align: left; width: 200px;">Compile the given .pyx files, plus those of any modules it depends on directly or indirectly via <a href="sharing.html#CImportStatement">cimport</a> statements. The include path specified by -I options is used to find the .pyx files of dependent modules.</td></tr><tr><td align="left" nowrap="nowrap" valign="top">-t</td><td align="left" nowrap="nowrap" valign="top">--timestamps</td><td style="vertical-align: top; text-align: left; width: 200px;">Use
modification times of files to decide whether to compile a .pyx file.
This is the default when -r is used, unless -f is also used.</td></tr><tr><td align="left" nowrap="nowrap" valign="top">-f</td><td align="left" nowrap="nowrap" valign="top">--force</td><td style="vertical-align: top; text-align: left; width: 200px;">Compile all .pyx files regardless of modification times. This is the default when -r is not given.</td></tr><tr><td align="left" nowrap="nowrap" valign="top">-q</td><td align="left" nowrap="nowrap" valign="top">--quiet</td><td style="vertical-align: top; text-align: left; width: 200px;">When -r is given, don't display the names of source files being compiled.</td></tr></tbody></table><br><h3><a class="mozTocH3" name="mozTocId49372"></a>Calling the Pyrex compiler from Python</h3>The module <span style="font-family: monospace;">Pyrex.Compiler.Main</span> exports the following classes and functions to facilitate invoking the compiler from another Python program.<br><br><span style="font-family: monospace;">compile(</span><span style="font-style: italic;">source</span> [, <span style="font-style: italic;">options</span>] [, <span style="font-style: italic;"> option</span> = <span style="font-style: italic;">value</span> ]...<span style="font-family: monospace;">)</span><br><br><div style="margin-left: 40px;">Compiles one or more Pyrex source files, which should be <span style="font-family: monospace;">.pyx</span> files. The <span style="font-style: italic;">source</span> argument may be either a single filename or a list of filenames.<br><br>Depending on the <span style="font-family: monospace;">recursive</span> option,
it may compile just the specified source files, or the specified source
files plus those of other modules that they depend on via <span style="font-style: italic;">cimport</span> statements. The options may be given either as keyword arguments or a <span style="font-family: monospace;">CompilationOptions</span> instance. If both are used, keyword arguments take precedence.<br><br>The return value depends on whether a list of sources was specifed and whether the <span style="font-family: monospace;">recursive</span> option is in effect. If a single source file is specified and the <span style="font-family: monospace;">recursive</span> option is false, the return value is a <span style="font-family: monospace;">CompilationResult</span> instance. Otherwise, the return value is a <span style="font-family: monospace;">CompilationResultSet</span><span style="font-family: monospace;"></span> containing a <span style="font-family: monospace;">CompilationResult</span>
for each of the modules which were actually compiled (which may or may
not include ones corresponding to the specified source files).<br><br>Note:
If you have more than one source file to compile, it is more efficient
to do so with a single call to compile rather than one call for each
source file. This is because, if more than one source cimports the same
.pxd file, the .pxd file is parsed only once instead of being
parsed each time it is cimported.<br></div><br><span style="font-family: monospace;">compile_single(</span><span style="font-style: italic;">source_path</span> [, <span style="font-style: italic;">options</span>] [, <span style="font-style: italic;"> option</span> = <span style="font-style: italic;">value</span> ]...<span style="font-family: monospace;">)</span><br><br><div style="margin-left: 40px;">Compiles just a single .pyx source file, specified as a string, with no dependency or timestamp checking (the <span style="font-family: monospace;">recursive</span> and <span style="font-family: monospace;">timestamps</span> options are ignored). Always returns a <span style="font-family: monospace;">CompilationResult</span>.</div><span style="font-family: monospace;"><br></span><span style="font-family: monospace;">compile_multiple(</span><span style="font-style: italic;">source_list</span> [, <span style="font-style: italic;">options</span>] [, <span style="font-style: italic;"> option</span> = <span style="font-style: italic;">value</span> ]...<span style="font-family: monospace;">)</span><br><br><div style="margin-left: 40px;">Compiles
a list of .pyx source files, with optional dependency and timestamp
checking as for compile. Always takes a list of source pathnames, and
always returns a <span style="font-family: monospace;">CompilationResultSet</span>.</div><span style="font-family: monospace;"><br>class CompilationOptions</span><br><br><div style="margin-left: 40px;">A collection of options to be passed to the <span style="font-family: monospace;">compile()</span> function. The following options may be specified as keyword arguments to either the <span style="font-family: monospace;">CompilationOptions</span> constructor or the <span style="font-family: monospace;">compile()</span> function.<br></div><div style="margin-left: 80px;"><dl><dt style="font-family: monospace;">show_version</dt><dd>Display the version number of the Pyrex compiler.<br></dd><dt style="font-family: monospace;">use_listing_file</dt><dd>Produce a .lis file for each .pyx file containing compilation errors.<br></dd><dt style="font-family: monospace;">include_path</dt><dd>A list of directories to search for included files and top-level package directories.<br></dd><dt style="font-family: monospace;">output_file</dt><dd>Use the given name for the generated .c file (only in non-recursive mode).<br></dd><dt style="font-family: monospace;">recursive</dt><dd>Compile the .pyx files of any cimported modules in addition to the one specified.<br></dd><dt style="font-family: monospace;">timestamps</dt><dd>Only
compile modified sources as determined by file modification times. This
may be true, false or None. If None, timestamps are used if and only if
recursive mode is in effect.<br></dd><dt style="font-family: monospace;">quiet</dt><dd>Don't display names of sources being compiled in recursive mode.</dd></dl></div><span style="font-family: monospace;">class CompilationResult</span><br><br><div style="margin-left: 40px;">An object providing information about the result of compiling a .pyx file. It has the following attributes:<br></div><div style="margin-left: 80px;"><dl><dt style="font-family: monospace;">c_file</dt><dd>Pathname of the generated C source file.<br></dd><dt style="font-family: monospace;">h_file</dt><dd>Pathname of the generated C header file, if any.<br></dd><dt style="font-family: monospace;">api_file</dt><dd>Pathname of the generated C API header file, if any.<br></dd><dt style="font-family: monospace;">listing_file</dt><dd>Pathname of the generated error message file, if any.<br></dd><dt style="font-family: monospace;">num_errors</dt><dd>Number of compilation errors.</dd></dl></div><span style="font-family: monospace;">class CompilationResultSet</span><div style="margin-left: 40px;"></div><div style="margin-left: 40px;">This is a mapping object whose keys are the pathnames of .pyx files and whose values are the corresponding <span style="font-family: monospace;">CompilationResult</span> instances. It also has the following additional attributes:<br><dl style="margin-left: 40px;"><dt style="font-family: monospace;">num_errors</dt><dd>Total number of compilation errors.</dd></dl></div><h3><a class="mozTocH3" name="mozTocId11953"></a>Using the distutils extension</h3>The
distutils extension provided with Pyrex allows you to pass .pyx files
directly to the Extension constructor in your setup file.<br><br>To use it, you need to put the following at the top of your setup.py file:<br><br><div style="margin-left: 40px; font-family: monospace;">from Pyrex.Distutils.extension import Extension<br>from Pyrex.Distutils import build_ext<br></div><br>and you need to specify the Pyrex version of the build_ext command class in your setup() call:<br><br><div style="margin-left: 40px; font-family: monospace;">setup(<br> ...<br> cmdclass = {'build_ext': build_ext}<br>)<br></div><br>Using
the distutils extension is not currently recommended, because it's
unable to automatically find cimported modules or check the timestamps
of .pxd files and included sources. It also makes it harder to turn off
the use of Pyrex for people who are only installing your module and not
modifying the sources.<br><h2><a class="mozTocH2" name="mozTocId559484"></a>Distributing Pyrex modules</h2>It
is strongly recommended that you distribute the generated .c files as
well as your Pyrex sources, so that users can install your module
without needing to have Pyrex available.<br><br>It is also recommended that Pyrex compilation <span style="font-style: italic;">not</span>
be enabled by default in the version you distribute. Even if the user
has Pyrex installed, he probably doesn't want to use it just to install
your module. Also, the version he has may not be the same one you used,
and may not compile your sources correctly.<br><br>---<br></body></html>
|
