1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
|
<!DOCTYPE html>
<html lang="en" data-content_root="./">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Common basics — Gentoo Python Guide documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=d1102ebc" />
<link rel="stylesheet" type="text/css" href="_static/alabaster.css?v=12dfc556" />
<script src="_static/documentation_options.js?v=5929fcd5"></script>
<script src="_static/doctools.js?v=9a2dae69"></script>
<script src="_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="python-any-r1 — build-time dependency" href="any.html" />
<link rel="prev" title="Choosing between Python eclasses" href="eclass.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="common-basics">
<h1>Common basics<a class="headerlink" href="#common-basics" title="Link to this heading">¶</a></h1>
<p>The various eclasses in python-r1 try to follow a single design. You
will probably use more than one of them, so it is worthwhile to shortly
explain the common bits used by all of them, as well as the non-obvious
differences between them.</p>
<section id="python-compat">
<span id="index-0"></span><h2>PYTHON_COMPAT<a class="headerlink" href="#python-compat" title="Link to this heading">¶</a></h2>
<p>The <code class="docutils literal notranslate"><span class="pre">PYTHON_COMPAT</span></code> variable is used by all Python eclasses, and must
be declared in all ebuilds before they are inherited. It specifies
the list of Python implementations supported by the package.</p>
<p>The valid values are:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">pythonX_Y</span></code> for CPython X.Y</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">pypy3</span></code> for PyPy3.</p></li>
</ul>
<p>Typical use:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">PYTHON_COMPAT</span><span class="o">=(</span><span class="w"> </span>python3_<span class="o">{</span><span class="m">6</span>,7,8<span class="o">}</span><span class="w"> </span>pypy3<span class="w"> </span><span class="o">)</span>
inherit<span class="w"> </span>python-single-r1
</pre></div>
</div>
</section>
<section id="python-deps-and-python-required-use">
<span id="index-2"></span><span id="index-1"></span><h2>PYTHON_DEPS and PYTHON_REQUIRED_USE<a class="headerlink" href="#python-deps-and-python-required-use" title="Link to this heading">¶</a></h2>
<p>The <code class="docutils literal notranslate"><span class="pre">python-any-r1</span></code>, <code class="docutils literal notranslate"><span class="pre">python-single-r1</span></code> and <code class="docutils literal notranslate"><span class="pre">python-r1</span></code> eclasses
all assume that the developer is responsible for explicitly putting
the dependency strings and USE requirements in correct variables.
This is meant to consistently cover packages that use Python
unconditionally and conditionally, at build time and at runtime.</p>
<p>For <code class="docutils literal notranslate"><span class="pre">python-single-r1</span></code> and <code class="docutils literal notranslate"><span class="pre">python-r1</span></code>, the most basic form to use
Python unconditionally is to define the following:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">REQUIRED_USE</span><span class="o">=</span><span class="si">${</span><span class="nv">PYTHON_REQUIRED_USE</span><span class="si">}</span>
<span class="nv">RDEPEND</span><span class="o">=</span><span class="si">${</span><span class="nv">PYTHON_DEPS</span><span class="si">}</span>
<span class="nv">BDEPEND</span><span class="o">=</span><span class="si">${</span><span class="nv">RDEPEND</span><span class="si">}</span>
</pre></div>
</div>
<p>For <code class="docutils literal notranslate"><span class="pre">python-any-r1</span></code>, only build-time dependencies are used:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">BDEPEND</span><span class="o">=</span><span class="si">${</span><span class="nv">PYTHON_DEPS</span><span class="si">}</span>
</pre></div>
</div>
<p>This does not apply to <code class="docutils literal notranslate"><span class="pre">distutils-r1</span></code> as it does the above assignment
by default.</p>
</section>
<section id="python-environment">
<span id="index-4"></span><span id="index-3"></span><h2>Python environment<a class="headerlink" href="#python-environment" title="Link to this heading">¶</a></h2>
<p>The eclasses commonly use the concept of <em>Python environment</em>. This
means a state of environment enforcing a particular Python
implementation. Whenever the ebuild code is run inside this
environment, two variables referring to the specific Python interpreter
are being exported:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">EPYTHON</span></code> containing the interpreter’s basename (also used
as the implementation identifier), e.g. <code class="docutils literal notranslate"><span class="pre">python3.10</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">PYTHON</span></code> containing the absolute final path to the interpreter,
e.g. <code class="docutils literal notranslate"><span class="pre">/usr/bin/python3.10</span></code></p></li>
</ul>
<p>The full path should only be used to provide the value that should
be embedded in the installed programs, e.g. in the shebangs.
For spawning Python during the build, <code class="docutils literal notranslate"><span class="pre">EPYTHON</span></code> is preferable.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Using full path rather than the basename will bypass the virtualenv
created by <code class="docutils literal notranslate"><span class="pre">distutils-r1.eclass</span></code> in PEP 517 mode. This may cause
failures to import Python modules, or use of the previous installed
version rather than the just-built one. Using <code class="docutils literal notranslate"><span class="pre">${EPYTHON}</span></code>
resolves these problems.</p>
</div>
<p>Wrappers for <code class="docutils literal notranslate"><span class="pre">python</span></code>, <code class="docutils literal notranslate"><span class="pre">pythonN</span></code> and some common tools are provided
in PATH, and <code class="docutils literal notranslate"><span class="pre">/usr/bin/python</span></code> etc. also enforce the specific
implementation via python-exec (for programs that hardcode full path).</p>
<p>The environment can be either established in local scope, or globally.
The local scope generally applies to multi-impl packages, and is created
either by calls to <code class="docutils literal notranslate"><span class="pre">python_foreach_impl</span></code> from <code class="docutils literal notranslate"><span class="pre">python-r1</span></code>, or inside
sub-phase functions in <code class="docutils literal notranslate"><span class="pre">distutils-r1</span></code>. The global scope setup is done
via calling <code class="docutils literal notranslate"><span class="pre">python_setup</span></code>, either directly or via default
<code class="docutils literal notranslate"><span class="pre">pkg_setup</span></code> in <code class="docutils literal notranslate"><span class="pre">python-any-r1</span></code> and <code class="docutils literal notranslate"><span class="pre">python-single-r1</span></code>.</p>
</section>
<section id="dependencies-in-python-packages">
<h2>Dependencies in Python packages<a class="headerlink" href="#dependencies-in-python-packages" title="Link to this heading">¶</a></h2>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The following sections focus specifically on dependencies that
are Python packages. Python software often depends on external
tools, libraries written in other programming languages, etc.
For these dependencies, the usual Gentoo rules apply.</p>
</div>
<section id="the-most-common-dependency-types">
<span id="index-7"></span><span id="index-6"></span><span id="index-5"></span><h3>The most common dependency types<a class="headerlink" href="#the-most-common-dependency-types" title="Link to this heading">¶</a></h3>
<p>The dependencies found in Python packages can usually be classified
into two categories: runtime dependencies and build-time dependencies.</p>
<p><em>Runtime dependencies</em> are packages that are required to be present
in order for the installed Python modules and scripts to be usable.
In general, these are all packages whose modules are imported
in the installed Python files. Generally runtime dependencies
are not needed at build time and therefore the build systems
do not verify whether they are installed. However, modern Python
scripts based on entry points often refuse to run if their dependencies
are not satisfied. Runtime dependencies should be placed
in <code class="docutils literal notranslate"><span class="pre">RDEPEND</span></code>.</p>
<p>A special subclass of runtime dependencies are <em>optional runtime
dependencies</em> (often called ‘extra’ dependencies). The dependencies are
optional if the package can still be meaningfully functional when they
are not installed. This usually means that the package either handles
failing imports gracefully, or that they are imported only in a subset
of package’s installed modules and that the package can still be
meaningfully used without importing these modules.</p>
<p>There are multiple approaches to handling optional dependencies.
Depending on the specifics, they can:</p>
<ol class="arabic simple">
<li><p>be added unconditionally to <code class="docutils literal notranslate"><span class="pre">RDEPEND</span></code> (if they are considered
important and/or light enough);</p></li>
<li><p>be listed as an informational message in <code class="docutils literal notranslate"><span class="pre">pkg_postinst</span></code> (usually
utilizing <code class="docutils literal notranslate"><span class="pre">optfeature.eclass</span></code>);</p></li>
<li><p>be added to <code class="docutils literal notranslate"><span class="pre">RDEPEND</span></code> conditionally to USE flags (this is only
acceptable if the package is cheap to rebuild).</p></li>
</ol>
<p><em>Build-time dependencies</em> are the packages needed for the package
to be built and installed. In general, they include the packages
providing the build system. In some cases, they may also include some
runtime dependencies, e.g. when they are needed to import
the <code class="docutils literal notranslate"><span class="pre">__init__.py</span></code> of the package. As a rule of thumb, if the package
can be built correctly when the specific dependency is not installed,
it does not need to be listed as a build dependency. Most of the time,
build dependencies belong in <code class="docutils literal notranslate"><span class="pre">BDEPEND</span></code>.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">distutils-r1</span></code> class generally takes care of adding the dependency
on the build system and basic tooling. However, additional plugins
(e.g. <code class="docutils literal notranslate"><span class="pre">dev-python/setuptools_scm</span></code>) need to be listed explicitly.</p>
<p>A special class of build-time dependencies are requirements specific
to running the test suite and building documentation. Most of the time
the former include not only the test runner but also all runtime
dependencies of the package (since the test suite runs its code).
Sometimes this is also required to build documentation. These classes
of dependencies go into <code class="docutils literal notranslate"><span class="pre">BDEPEND</span></code> under <code class="docutils literal notranslate"><span class="pre">test</span></code> and <code class="docutils literal notranslate"><span class="pre">doc</span></code> USE flags
respectively.</p>
<p>Note that sometimes test dependencies can also be optional (including
optional runtime dependencies). They should generally be added
unconditionally to ensure maximum test coverage. Also note that
(as explained further in the Guide), some test dependencies
(e.g. on linters or coverage reporting tools) may actually
be undesirable.</p>
<p>Again, <code class="docutils literal notranslate"><span class="pre">distutils-r1</span></code> provides functions to conveniently add support
for common test runner and Sphinx-based documentation. The former also
takes care of copying <code class="docutils literal notranslate"><span class="pre">RDEPEND</span></code> into test dependencies.</p>
<p>Some Python packages include C extensions that depend on external
libraries. In this case, similarly to non-Python packages,
the dependency on packages providing these libraries needs to go
into <code class="docutils literal notranslate"><span class="pre">RDEPEND</span></code> and <code class="docutils literal notranslate"><span class="pre">DEPEND</span></code> (not <code class="docutils literal notranslate"><span class="pre">BDEPEND</span></code>).</p>
<p>Finally, there are Python packages providing C headers such
as <code class="docutils literal notranslate"><span class="pre">dev-python/numpy</span></code>. If the package in question uses both headers
and Python code from NumPy, the dependency may need to be included
in all three of <code class="docutils literal notranslate"><span class="pre">RDEPEND</span></code>, <code class="docutils literal notranslate"><span class="pre">DEPEND</span></code> and <code class="docutils literal notranslate"><span class="pre">BDEPEND</span></code> (unconditionally
or for tests).</p>
</section>
<section id="finding-dependency-lists-from-build-systems">
<h3>Finding dependency lists from build systems<a class="headerlink" href="#finding-dependency-lists-from-build-systems" title="Link to this heading">¶</a></h3>
<p>Most of the modern Python build systems include all the package metadata
in the <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file. Setuptools are using <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>
and/or <code class="docutils literal notranslate"><span class="pre">setup.py</span></code>. Some packages also include custom code to read
dependencies from external files; it is usually worthwhile to look
for <code class="docutils literal notranslate"><span class="pre">requirements</span></code> in the name.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Unconditional runtime dependencies and unconditional build-time
dependencies are often enforced by the script wrappers and build
systems respectively. If upstream lists spurious dependencies,
they often need to be explicitly stripped rather than just ommitted
from ebuild.</p>
</div>
<p>The keys commonly used to list specific kinds of dependencies in common
Python build systems:</p>
<ol class="arabic simple">
<li><p>Runtime dependencies (unconditional):</p>
<ul class="simple">
<li><p><a class="reference external" href="https://www.python.org/dev/peps/pep-0621">PEP 621</a> metadata: <code class="docutils literal notranslate"><span class="pre">project.dependencies</span></code></p></li>
<li><p>older flit versions: <code class="docutils literal notranslate"><span class="pre">tool.flit.metadata.requires</span></code></p></li>
<li><p>poetry: <code class="docutils literal notranslate"><span class="pre">tool.poetry.dependencies</span></code> (note: this also includes
special <code class="docutils literal notranslate"><span class="pre">python</span></code> entry to indicate compatible Python versions)</p></li>
<li><p>setuptools: <code class="docutils literal notranslate"><span class="pre">install_requires</span></code></p></li>
</ul>
</li>
<li><p>Optional runtime and/or build-time dependencies:</p>
<ul class="simple">
<li><p><a class="reference external" href="https://www.python.org/dev/peps/pep-0621">PEP 621</a> metadata: <code class="docutils literal notranslate"><span class="pre">project.optional-dependencies</span></code></p></li>
<li><p>older flit versions: <code class="docutils literal notranslate"><span class="pre">tool.flit.metadata.requires-extra</span></code></p></li>
<li><p>poetry: <code class="docutils literal notranslate"><span class="pre">tool.poetry.dependencies</span></code> with <code class="docutils literal notranslate"><span class="pre">optional</span> <span class="pre">=</span> <span class="pre">true</span></code>,
sometimes grouped using <code class="docutils literal notranslate"><span class="pre">tool.poetry.extras</span></code></p></li>
<li><p>setuptools: <code class="docutils literal notranslate"><span class="pre">extras_require</span></code></p></li>
</ul>
</li>
<li><p>Build-time dependencies (unconditional):</p>
<ul class="simple">
<li><p>all <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> build systems: <code class="docutils literal notranslate"><span class="pre">build-system.requires</span></code></p></li>
<li><p>poetry: <code class="docutils literal notranslate"><span class="pre">tool.poetry.dev-dependencies</span></code></p></li>
<li><p>setuptools: <code class="docutils literal notranslate"><span class="pre">setup_requires</span></code> (deprecated)</p></li>
</ul>
</li>
<li><p>Test dependencies (in addition to <code class="docutils literal notranslate"><span class="pre">RDEPEND</span></code>):</p>
<ul class="simple">
<li><p>often listed as <code class="docutils literal notranslate"><span class="pre">test</span></code> key in optional dependencies</p></li>
<li><p>setuptools: <code class="docutils literal notranslate"><span class="pre">tests_require</span></code> (deprecated)</p></li>
<li><p>in some cases they can also be found in <code class="docutils literal notranslate"><span class="pre">tox.ini</span></code>
or <code class="docutils literal notranslate"><span class="pre">noxfile.py</span></code></p></li>
</ul>
</li>
<li><p>Doc building dependencies:</p>
<ul class="simple">
<li><p>often listed as <code class="docutils literal notranslate"><span class="pre">doc</span></code> key in optional dependencies</p></li>
</ul>
</li>
<li><p>Python version compatibility:</p>
<ul class="simple">
<li><p><a class="reference external" href="https://www.python.org/dev/peps/pep-0621">PEP 621</a> metadata: <code class="docutils literal notranslate"><span class="pre">project.requires-python</span></code></p></li>
<li><p>older flit versions: <code class="docutils literal notranslate"><span class="pre">tool.flit.metadata.requires-python</span></code></p></li>
<li><p>poetry: <code class="docutils literal notranslate"><span class="pre">python</span></code> in <code class="docutils literal notranslate"><span class="pre">tool.poetry.dependencies</span></code></p></li>
<li><p>setuptools: <code class="docutils literal notranslate"><span class="pre">python_requires</span></code></p></li>
</ul>
</li>
</ol>
</section>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h1 class="logo"><a href="index.html">Gentoo Python Guide</a></h1>
<h3>Navigation</h3>
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="preface.html">Preface</a></li>
<li class="toctree-l1"><a class="reference internal" href="interpreter.html">Python interpreters</a></li>
<li class="toctree-l1"><a class="reference internal" href="eclass.html">Choosing between Python eclasses</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Common basics</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#python-compat">PYTHON_COMPAT</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-deps-and-python-required-use">PYTHON_DEPS and PYTHON_REQUIRED_USE</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-environment">Python environment</a></li>
<li class="toctree-l2"><a class="reference internal" href="#dependencies-in-python-packages">Dependencies in Python packages</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="any.html">python-any-r1 — build-time dependency</a></li>
<li class="toctree-l1"><a class="reference internal" href="single.html">python-single-r1 — single-impl packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="multi.html">python-r1 — multi-impl packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils.html">distutils-r1 — standard Python build systems</a></li>
<li class="toctree-l1"><a class="reference internal" href="test.html">Tests in Python packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils-legacy.html">distutils-r1 legacy concepts</a></li>
<li class="toctree-l1"><a class="reference internal" href="pypi.html">pypi — helper eclass for PyPI archives</a></li>
<li class="toctree-l1"><a class="reference internal" href="helper.html">Common helper functions</a></li>
<li class="toctree-l1"><a class="reference internal" href="depend.html">Advanced dependencies</a></li>
<li class="toctree-l1"><a class="reference internal" href="pytest.html">pytest recipes</a></li>
<li class="toctree-l1"><a class="reference internal" href="concept.html">Advanced concepts</a></li>
<li class="toctree-l1"><a class="reference internal" href="expert-multi.html">Expert python-r1 usage</a></li>
<li class="toctree-l1"><a class="reference internal" href="buildsys.html">Integration with build systems written in Python</a></li>
<li class="toctree-l1"><a class="reference internal" href="porting.html">Porting tips</a></li>
<li class="toctree-l1"><a class="reference internal" href="migration.html">Migration guides</a></li>
<li class="toctree-l1"><a class="reference internal" href="qawarn.html">QA checks and warnings</a></li>
<li class="toctree-l1"><a class="reference internal" href="package-maintenance.html">Python package maintenance</a></li>
<li class="toctree-l1"><a class="reference internal" href="interpreter-maintenance.html">Maintenance of Python implementations</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="eclass.html" title="previous chapter">Choosing between Python eclasses</a></li>
<li>Next: <a href="any.html" title="next chapter">python-any-r1 — build-time dependency</a></li>
</ul></li>
</ul>
</div>
<search id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</search>
<script>document.getElementById('searchbox').style.display = "block"</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2020, Michał Górny, license: CC BY 4.0.
|
Powered by <a href="https://www.sphinx-doc.org/">Sphinx 7.3.7</a>
& <a href="https://alabaster.readthedocs.io">Alabaster 0.7.16</a>
|
<a href="_sources/basic.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>
|
GitWeb