Conan: A Python package manager =============================== Conan is a C and C++ package manager, and to deal with the vast variability of C and C++ build systems, compilers, configurations, etc., it was designed with a great flexibility in mind, trying to let the users do almost what they want. This is one of the reasons to use Python as the scripting language for Conan package recipes. With this flexibility, Conan is able to do very different tasks: package `Visual Studio modules `_, :ref:`package Go code `, build packages from sources or from binaries retrieved from elsewhere, etc. :ref:`Python code can be reused and packaged with Conan ` to share functionalities or tools among *conanfile.py* files. Here we can see a full example of Conan as a Python package manager. A full Python and C/C++ package manager ---------------------------------------- The real utility of this is that Conan is a C and C++ package manager. So if we want to create a python package that wraps the functionality of, lets say the Poco C++ library, it can be easily done. Poco itself has transitive (C/C++) dependencies, but they are already handled by Conan. Furthermore, a very interesting thing is that nothing has to be done in advance for that library, thanks to useful tools as **pybind11**, that allows to create python bindings easily. So let's build a package with the following files: - *conanfile.py*: The package recipe. - *__init__.py*: necessary file, blank. - *pypoco.cpp*: The C++ code with the ``pybind11`` wrapper for Poco that generates a python extension (a shared library that can be imported from python). - *CMakeLists.txt*: cmake build file that is able to compile *pypoco.cpp* into a Python extension (*pypoco.pyd* in Windows, *pypoco.so* in Linux) - *poco.py*: A Python file that makes use of the pypoco Python binary extension built with *pypoco.cpp*. - *test_package/conanfile.py*: A test consumer recipe for convenience to create and test the package. The *pypoco.cpp* file can be coded easily thanks to the elegant ``pybind11`` library: .. code-block:: cpp :caption: pypoco.cpp #include #include "Poco/Random.h" using Poco::Random; namespace py = pybind11; PYBIND11_PLUGIN(pypoco) { py::module m("pypoco", "pybind11 example plugin"); py::class_(m, "Random") .def(py::init<>()) .def("nextFloat", &Random::nextFloat); return m.ptr(); } And the *poco.py* file is straigthforward: .. code-block:: cpp :caption: *poco.py* import sys import pypoco def random_float(): r = pypoco.Random() return r.nextFloat() The *conanfile.py* has a few more lines than the above, but it is still quite easy to understand: .. code-block:: python :caption: *conanfile.py* from conans import ConanFile, tools, CMake class PocoPyReuseConan(ConanFile): name = "PocoPy" version = "0.1" requires = "Poco/1.9.0@pocoproject/stable", "pybind11/any@memsharded/stable" settings = "os", "compiler", "arch", "build_type" exports = "*" generators = "cmake" build_policy = "missing" def build(self): cmake = CMake(self) pythonpaths = "-DPYTHON_INCLUDE_DIR=C:/Python27/include -DPYTHON_LIBRARY=C:/Python27/libs/python27.lib" self.run('cmake %s %s -DEXAMPLE_PYTHON_VERSION=2.7' % (cmake.command_line, pythonpaths)) self.run("cmake --build . %s" % cmake.build_config) def package(self): self.copy('*.py*') self.copy("*.so") def package_info(self): self.env_info.PYTHONPATH.append(self.package_folder) The recipe now declares 2 ``requires``, the **Poco library** and the **pybind11 library** that we will be using to create the binary extension. As we are actually building some C++ code, we need a few important things: - Input ``settings`` that define the OS, compiler, version and architecture we are using to build our extension. This is necessary because the binary we are building must match the architecture of the python interpreter that we will be using. - The ``build()`` method is used actually to invoke CMake. See we had to hardcode the python path in the example, as the *CMakeLists.txt* call to ``find_package(PythonLibs)`` didn't find my python installed in *C:/Python27*, quite a standard path. I have added the ``cmake`` generator too to be able to easily use the declared ``requires`` build information inside my *CMakeLists.txt*. - The *CMakeLists.txt* is not posted here, but is basically the one used in the pybind11 example with just 2 lines to include the conan generated cmake file for dependencies. It can be inspected in the github repo. - Note that we are using Python 2.7 as an input option. If necessary, more options for other interpreters/architectures could be easily provided, as well as avoiding the hardcoded paths. Even the Python interpreter itself could be packaged in a conan package. The above recipe will generate a different binary for different compilers or versions. As the binary is being wrapped by python, we could avoid this and use the same binary for different setups, modifying this behavior with the ``conan_info()`` method. .. code-block:: bash $ conan export . memsharded/testing $ conan install PocoPy/0.1@memsharded/testing -s arch=x86 -g virtualenv $ activate $ python >>> import poco >>> poco.random_float() 0.697845458984375 Now the first invocation of :command:`conan install` will build retrieve the dependencies and build the package. The next invocation will use the cached binaries and be much faster. Note how we have to specify ``-s arch=x86`` to build matching the architecture of the python interpreter to be used, in our case, 32 bits. We can also read in the output of the :command:`conan install` the dependencies that are being pulled: .. code-block:: bash Requirements OpenSSL/1.0.2l@conan/stable from conan.io Poco/1.9.0@pocoproject/stable from conan.io PocoPy/0.1@memsharded/testing from local pybind11/any@memsharded/stable from conan.io zlib/1.2.11@conan/stable from conan.io This is the great thing about using Conan for this task, by depending on Poco, other C and C++ transitive dependencies are being retrieved and used in the application. If you want to have a further look to the code of these examples, you can check `this github repo `_. The above examples and code have been tested only in Win10, VS14u2, but might work with other configurations with little or no extra work.