Deploying Python Packages to PyPI using Flit
The traditional way of package deployment on Python is using a setup.py script located in the root of your project and then running python setup.py sdist upload to release a new version.
It works by using distutils, setuptools or distribute and there is also twine which is a command line application to manage uploads.
History
distutils is the standard way of Python package distribution included in standard library since Python 2.x then setuptools was created to overcome distutils limitations and also introduces a command line application called easy_install (which is now pip) and a very handy feature called pkg_resources and one of the characteristics of setuptools is that it uses Monkey Patching over the standard distutils.
Some forks of setuptools has been created to fix some issues and ad common developers preferences and well known forks like distribute and distutils2 and distlib has been merged back to the original setuptools
Lots of other packaging tools has been created to try to fix the distribution problems.
How it works in standard way
using one of the above you should create a file called setup.py in the root of your project, e.g:
from <my_favorite_dist_tool> import setup
# Example taken from Django's repository
setup(
name='Django',
version=version,
url='http://ift.tt/q4NHeK',
author='Django Software Foundation',
author_email='foundation@djangoproject.com',
description=('A high-level Python Web framework that encourages '
'rapid development and clean, pragmatic design.'),
license='BSD',
packages=find_packages(exclude=EXCLUDE_FROM_PACKAGES),
include_package_data=True,
scripts=['django/bin/django-admin.py'],
entry_points={'console_scripts': [
'django-admin = django.core.management:execute_from_command_line',
]},
install_requires=['pytz'],
extras_require={
"bcrypt": ["bcrypt"],
"argon2": ["argon2-cffi >= 16.1.0"],
},
zip_safe=False,
classifiers=[
'Development Status :: 2 - Pre-Alpha',
'Environment :: Web Environment',
'Framework :: Django',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Topic :: Internet :: WWW/HTTP',
'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
'Topic :: Internet :: WWW/HTTP :: WSGI',
'Topic :: Software Development :: Libraries :: Application Frameworks',
'Topic :: Software Development :: Libraries :: Python Modules',
],
)
As you can see it is very confusing to decide which of them to addopt and how the setup.py should be writen because there are different examples over the github most famous Python repositories.
Making things easier with Flit
NOTE: Forget about all that history you read above and consider Flit.
Flit is a simple way to Package and deploy Python projects on PyPI, Flit makes it easier by using a simple flit.ini file and assumes common defaults to save your time and typing.
I knew Flit when I was taking a look at Mariatta Wijaya game called Tic Tac Taco Pizza and noted that she used flit to deploy the game, so we also asked her the reason on the podcast we recorded so I decided to try porting my projects to Flit.
How it works?
Instead of a setup.py you put a flit.ini in the root of your project and it looks like:
[metadata]
module = markdocs
author = Bruno Rocha
author-email = rochacbruno@gmail.com
maintainer = rochacbruno
maintainer-email = rochacbruno@gmail.com
home-page = http://ift.tt/2w1LXQ7
requires = mistune
click
description-file = README.md
classifiers = Programming Language :: Python :: 3.6
Intended Audience :: Developers
License :: OSI Approved :: MIT License
Topic :: Documentation
Topic :: Software Development :: Documentation
Topic :: Software Development :: Quality Assurance
requires-python = >=3.6
[scripts]
markdocs = markdocs:main
Now you only need to have flit installed in your local machine pip3 install flit as it works only in Python3+ and also is recommended to have pandoc and pypandoc also installed because it can convert your README.md into the .rst format still used by PyPI.
The advantages are:
- No more complicated
setup.py - If you omit some fields it will assume common used defaults
- It is easier to read and write
- Will convert your README.md
- Will take the
__version__included in yourprogram/__init__.py - Can develop to TestPyPI
- Avoids over engineering on
setup.py
Development installation
To install your package during development use
flit install --symlink --python path/to/virtualenv/bin/python
the
--pythonis optional, by default will take the currentwhich python
Registering and deploying
It is easy and will register the new project for you if doesn't exist on PyPI
flit puslish
Flit packages a single importable module or package at a time, using the import name as the name on PyPI. All subpackages and data files within a package are included automatically.
Important!
- Flit will use the data from
~/.pypircto authenticate and to find the server deployment addresses - You can also set
FLIT_USERNAMEandFLIT_PASSWORDandFLIT_INDEX_URLas environment variables which makes flit good for CI deployment (e.g: TravisCI)
What is missing?
NOTE: Flit is open-source, so some of this things are already under consideration.
- Flit will not bump your project version automatically, you can still use tools like bumpversion but this feature would be better if builtin
- Flit will not parse a
requirements.txtfile and would be nice to have it as tools likepyup.iocan track those files but notflit.iniyet - Flit does not create a
.lockfrozen file with the version used on specific release and it is interesting just likepipenvdoes
Conclusion
Flit is the easier way to deploy packaged to PyPI following 3 steps (1. install flit, 2. describe flit.ini 3. run flit publish) your library is released to PyPI.
But Python still needs better standards because you still need separated tools to make common tasks and using a single tool to pack, install, deploy, create would be better (just like what cargo/crates does for Rust) for example, in Python you have:
- Flit to deploy packages to PyPI for distribution
- bumpversion to bump your semver number
- pip to install and update packages from PyPI (or pipenv/WIP to do the same with more powers)
- Cookiecutter to create a new Python Package (from strucured templates)
- safety to check dependencies security
- flake or pylint to static and styling checks
- venv, pyenv or virtualenvwrapper to manage isolated environments
- pytest to run tests
- pyup to watch for depdendency updates
So many tools brings a lot of confusion and makes it hard to choose, why not having a single tool, based on different plugins sharing the same API?
Just an idea
python -m manage [install packagename] # <-- calls pip or pipenv
[publish --options --bump=version] # <-- calls `flit` and `bumpversion`
[new packagename templatename] # <-- calls cookiecutter
[safecheck] # <-- calls safety
[checkupdates] # <-- checks in the same way as Pyup
[test path] # <-- calls pytest, nose or unittest
[lint path] # <-- calls flake, pylint
[venv options] # <-- calls the existing venv module
All above configurable via config file or env vars.
and each of that endpoints would be provided by many plugins sharing the same API, so you could choose between flit or twine as your publish manager etc..
So maybe I can implement that features in manage
Please share in comments if you know some other Python management tool
