Python Packaging & Distribution

distuils & setuptools

distuils is the standard packaging tool of Python, and setuptools is its improvement.

Example: distuils

1. setup.py

from distutils.core import setup

setup(
    name = 'hello',
    version = '1.0',
    packages = ['hello'],
)

2. shell command

python setup.py sdist
# got a tar.gz file in dist dictionary

Example: setuptools

1. setup.py

from setuptools import setup
setup(     
    name='mypackage',
    author='Giorgos Myrianthous',     
    version='0.1',     
    install_requires=[         
        'pandas',         
        'numpy',
        'matplotlib',
    ],
    # ... more options/metadata
)

2. setup.cfg
   1. just copy parameters to cfg file

[metadata]
name = mypackage
author = Giorgos Myrianthous
version = 0.1
[options]
install_requires =
    pandas
    numpy
    matplotlib

3. shell command

python setup.py bdist_egg
# got a egg file in dist dictionary

---

PyPI & egg, PIP & easy_install

PyPI · The Python Package Index

Once you upload your egg file onto PyPI, you can download and install your module throught easy_install(deprecated) or pip

---

requirements.txt

The requirements.txt is a file listing all the dependencies for a specific Python project.

Example: requirements.txt

1. requirements.txt

matplotlib>=2.2
numpy>=1.15.0, <1.21.0
pandas
pytest==4.0.1

2. shell command

pip install -r requirements.txt

Do we need both requirements.txt and setup.py/setup.cfg files?

• If your package is mostly for development purposes but you aren’t planning on redistributing it, requirements.txt should be enough (even when the package is developed on multiple machines).
• If your package is developed only by yourself (i.e. on a single machine) but you are planning to redistribute it, then setup.py/setup.cfg should be enough.
• If you package is developed on multiple machines and you also need to redistribute it, you will need both the requirements.txt and setup.py/setup.cfg files.

In case you are using both, your setup.py (and/or setup.cfg) files should include the list of abstract dependencies, whereas the requirements.txt file must contain the concrete dependencies with the specific pins for each package version (using == pin).

---

pyproject.toml

The goal of this file is to allow you to define what build tools are needed in order to build your package – no longer assuming it must be Setuptools. And it can be Poetry or Filt.

/path/to/example/project/
├── src/                            Source dir.
│   └── example/                    Python package directory.
│       ├── __init__.py             This makes the directory a package.
│       └── example_module.py       Example module.
├── pyproject.toml                  Definition of build process of the package.
├── README.md                       README with info of the project.
└── setup.cfg                       Configuration details of the python package.

Tools are standardizing on pyproject.toml, so now projects like Black, coverage.py, towncrier, and tox (in a way) allow you to specify their configurations in pyproject.toml instead of in a separate file.