This is an experimental feature subject to breaking changes in future releases.
This syntax supersedes the legacy python_requires() syntax. The most important changes are:
- These new python_requires affect the consumers
package_id. So different binaries can be managed, and CI systems can re-build affected packages according to package ID modes and versioning policies.
- The syntax defines a class attribute instead of a module function call, so recipes are cleaner and more aligned with other types of requirements.
- The new python_requires will play better with lockfiles and deterministic dependency graphs.
- They are able to extend base classes more naturally without conflicts of ConanFile classes.
python_requires feature is a very convenient way to share files and code between
different recipes. A python requires is similar to any other recipe, it is the way it is
required from the consumer what makes the difference.
A very simple recipe that we want to reuse could be:
from conans import ConanFile myvar = 123 def myfunct(): return 234 class Pkg(ConanFile): pass
And then we will make it available to other packages with
conan export. Note that we are
conan create, because this recipe doesn’t have binaries. It is just the python
code that we want to reuse.
$ conan export . pyreq/0.1@user/channel
We can reuse the above recipe functionality declaring the dependency in the
attribute and we can access its members using
from conans import ConanFile class Pkg(ConanFile): python_requires = "pyreq/0.1@user/channel" def build(self): v = self.python_requires["pyreq"].module.myvar # v will be 123 f = self.python_requires["pyreq"].module.myfunct() # f will be 234 self.output.info("%s, %s" % (v, f))
$ conan create . pkg/0.1@user/channel ... pkg/0.1@user/channel: 123, 234
It is also possible to require more than one python-require, and use the package name to address the functionality:
from conans import ConanFile class Pkg(ConanFile): python_requires = "pyreq/0.1@user/channel", "other/1.2@user/channel" def build(self): v = self.python_requires["pyreq"].module.myvar # v will be 123 f = self.python_requires["other"].module.otherfunc("some-args")
Extending base classes¶
A common use case would be to declare a base class with methods we want to reuse in several recipes via inheritance. We’d write this base class in a python-requires package:
from conans import ConanFile class MyBase(object): def source(self): self.output.info("My cool source!") def build(self): self.output.info("My cool build!") def package(self): self.output.info("My cool package!") def package_info(self): self.output.info("My cool package_info!") class PyReq(ConanFile): name = "pyreq" version = "0.1"
And make it available for reuse with:
$ conan export . pyreq/0.1@user/channel
Note that there are two classes in the recipe file:
MyBaseis the one intended for inheritance and doesn’t extend
PyReqis the one that defines the current package being exported, it is the recipe for the reference
Once the package with the base class we want to reuse is available we can use it in other
recipes to inherit the functionality from that base class. We’d need to declare the
python_requires as we did before and we’d need to tell Conan the base classes to use
in the attribute
python_requires_extend. Here our recipe will inherit from the
from conans import ConanFile class Pkg(ConanFile): python_requires = "pyreq/0.1@user/channel" python_requires_extend = "pyreq.MyBase"
The resulting inheritance is equivalent to declare our
Pkg class as
class Pkg(pyreq.MyBase, ConanFile).
So creating the package we can see how the methods from the base class are reused:
$ conan create . pkg/0.1@user/channel ... pkg/0.1@user/channel: My cool source! pkg/0.1@user/channel: My cool build! pkg/0.1@user/channel: My cool package! pkg/0.1@user/channel: My cool package_info! ...
If there is extra logic needed to extend from a base class, like composing the base class settings
with the current recipe, the
init() method can be used for it:
class PkgTest(ConanFile): license = "MIT" settings = "arch", # tuple! python_requires = "base/1.1@user/testing" python_requires_extend = "base.MyConanfileBase" def init(self): base = self.python_requires["base"].module.MyConanfileBase self.settings = base.settings + self.settings # Note, adding 2 tuples = tuple self.license = base.license # License is overwritten
For more information about the
init() method visit init()
There are a few limitations that should be taken into account:
versionfields shouldn’t be inherited.
set_version()might be used.
short_pathscannot be inherited from a
python_requires. Make sure to specify it directly in the recipes that need the paths shortened in Windows.
exports_sourcesshouldn’t be inherited from a base class, but explictly defined directly in the recipes. A reusable alternative might be using the
build_policyshouldn’t be inherited from a base class, but explictly defined directly in the recipes.
- Mixing Python inheritance with
python_requires_extendshould be avoided, because the inheritance order can be different than the expected one. Multiple level
python_requires_extendmight be possible, but don’t mix both approaches (also in general try to avoid multiple inheritance and multiple level hierarchies, try to keep it simple).
It is possible to access the files exported by a recipe that is used with
We could have this recipe, together with a myfile.txt file containing the “Hello” text.
from conans import ConanFile class PyReq(ConanFile): exports = "*"
$ echo "Hello" > myfile.txt $ conan export . pyreq/0.1@user/channel
Now the recipe has been exported, we can access its path (the place where myfile.txt is) with the
import os from conans import ConanFile, load class Pkg(ConanFile): python_requires = "pyreq/0.1@user/channel" def build(self): pyreq_path = self.python_requires["pyreq"].path myfile_path = os.path.join(pyreq_path, "myfile.txt") content = load(myfile_path) # content = "Hello" self.output.info(content) # we could also copy the file, instead of reading it
Note that only
exports work for this case, but not
python-requires will affect the
package_id of the packages using those dependencies.
By default, the policy is
minor_mode, which means:
- Changes to the patch version of a python-require will not affect the package ID. So depending
"pyreq/1.2.4"will result in identical package ID (both will be mapped to
"pyreq/1.2.Z"in the hash computation). Bump the patch version if you want to change your common code, but you don’t want the consumers to be affected or to fire a re-build of the dependants.
- Changes to the minor or major version will produce a different package ID. So if you depend
"pyreq/1.2.3", and you bump the version to
"pyreq/1.3.0", then, you will need to build new binaries that are using that new python-require. Bump the minor or major version if you want to make sure that packages requiring this python-require will be built using these changes in the code.
- Both changing the minor and major requires a new package ID, and then a build from source. You could use changes in the minor to indicate that it should be source compatible, and consumers wouldn’t need to do changes, and changes in the major for source incompatible changes.
As with the regular
requires, this default can be customized. First you can customize it at attribute
global level, modifying the conan.conf
default_python_requires_id_mode, which can take the values
For example, if you want to make the package IDs never be affected by any change in the versions of python-requires, you could do:
Read more about these modes in Using package_id() for Package Dependencies.
It is also possible to customize the effect of
python_requires per package, using the
from conans import ConanFile class Pkg(ConanFile): python_requires ="pyreq/[>=1.0]" def package_id(self): self.info.python_requires.patch_mode()
Resolution of python-requires¶
There are few things that should be taken into account when using
- Python requires recipes are loaded by the interpreter just once, and they are common to
all consumers. Do not use any global state in the
- Python requires are private to the consumers. They are not transitive. Different consumers can require different versions of the same python-require.
python-requirescan use version ranges expressions.
python-requireother recipes too, but this should probably be limited to very few cases, we recommend to use the simplest possible structure.
python-requirescan conflict if they require other recipes and create conflicts in different versions.
python-requirescannot use regular
- It is possible to use
python-requireswithout user and channel.
python-requirescan use native python
importto other python files, as long as these are exported together with the recipe.
python-requiresshould not create packages, but use
python-requirescan be used as editable packages too.
python-requiresare locked in lockfiles.