Installing Packages¶

This section covers the basics of how to install Python packages .

It’s important to note that the term “package” in this context is being used to describe a bundle of software to be installed (i.e. as a synonym for a distribution ). It does not to refer to the kind of package that you import in your Python source code (i.e. a container of modules). It is common in the Python community to refer to a distribution using the term “package”. Using the term “distribution” is often not preferred, because it can easily be confused with a Linux distribution, or another larger software distribution like Python itself.

Requirements for Installing Packages¶

This section describes the steps to follow before installing other Python packages.

Ensure you can run Python from the command line¶

Before you go any further, make sure you have Python and that the expected version is available from your command line. You can check this by running:

You should get some output like Python 3.6.3 . If you do not have Python, please install the latest 3.x version from python.org or refer to the Installing Python section of the Hitchhiker’s Guide to Python.

If you’re a newcomer and you get an error like this:

It’s because this command and other suggested commands in this tutorial are intended to be run in a shell (also called a terminal or console). See the Python for Beginners getting started tutorial for an introduction to using your operating system’s shell and interacting with Python.

If you’re using an enhanced shell like IPython or the Jupyter notebook, you can run system commands like those in this tutorial by prefacing them with a ! character:

It’s recommended to write rather than plain python in order to ensure that commands are run in the Python installation matching the currently running notebook (which may not be the same Python installation that the python command refers to).

Due to the way most Linux distributions are handling the Python 3 migration, Linux users using the system Python without creating a virtual environment first should replace the python command in this tutorial with python3 and the python -m pip command with python3 -m pip —user . Do not run any of the commands in this tutorial with sudo : if you get a permissions error, come back to the section on creating virtual environments, set one up, and then continue with the tutorial as written.

Ensure you can run pip from the command line¶

Additionally, you’ll need to make sure you have pip available. You can check this by running:

If you installed Python from source, with an installer from python.org, or via Homebrew you should already have pip. If you’re on Linux and installed using your OS package manager, you may have to install pip separately, see Installing pip/setuptools/wheel with Linux Package Managers .

If pip isn’t already installed, then first try to bootstrap it from the standard library:

If that still doesn’t allow you to run python -m pip :

Run python get-pip.py . 2 This will install or upgrade pip. Additionally, it will install setuptools and wheel if they’re not installed already.

Be cautious if you’re using a Python install that’s managed by your operating system or another package manager. get-pip.py does not coordinate with those tools, and may leave your system in an inconsistent state. You can use python get-pip.py —prefix=/usr/local/ to install in /usr/local which is designed for locally-installed software.

Ensure pip, setuptools, and wheel are up to date¶

While pip alone is sufficient to install from pre-built binary archives, up to date copies of the setuptools and wheel projects are useful to ensure you can also install from source archives:

Optionally, create a virtual environment¶

See section below for details, but here’s the basic venv 3 command to use on a typical Linux system:

This will create a new virtual environment in the tutorial_env subdirectory, and configure the current shell to use it as the default python environment.

Creating Virtual Environments¶

Python “Virtual Environments” allow Python packages to be installed in an isolated location for a particular application, rather than being installed globally. If you are looking to safely install global command line tools, see Installing stand alone command line tools .

Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python3.6/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.

Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.

Also, what if you can’t install packages into the global site-packages directory? For instance, on a shared host.

In all these cases, virtual environments can help you. They have their own installation directories and they don’t share libraries with other virtual environments.

Currently, there are two common tools for creating Python virtual environments:

venv is available by default in Python 3.3 and later, and installs pip and setuptools into created virtual environments in Python 3.4 and later.

virtualenv needs to be installed separately, but supports Python 2.7+ and Python 3.3+, and pip , setuptools and wheel are always installed into created virtual environments by default (regardless of Python version).

Making a Python Package¶

Specifying how to build your python package

Python Packages¶

What is a “package” in Python ?

Packages, modules, imports, oh my!¶

Modules

A python “module” is a single namespace, with a collection of values:

• functions
• constants
• class definitions
• really any old value.

A module usually corresponds to a single file: something.py

Packages¶

A “package” is essentially a module, except it can have other modules (and indeed other packages) inside it.

A package usually corresponds to a directory with a file in it called __init__.py and any number of python files or other package directories:

The __init__.py can be totally empty – or it can have arbitrary python code in it.

The code will be run when the package is imported – just like a module,

modules inside packages are not automatically imported. So, with the above structure:

will run the code in a_package/__init__.py .

Any names defined in the __init__.py will be available in:

will not exist. To get submodules, you need to explicitly import them:

The module search path¶

The interpreter keeps a list of all the places that it looks for modules or packages when you do an import:

You can manipulate that list to add or remove paths to let python find modules on a new place.

And every module has a __file__ name that points to the path it lives in. This lets you add paths relative to where you are, etc.

NOTE: it’s usually better to use setuptools’ “develop” mode instead – see below.

Building Your Own Package¶

The very basics of what you need to know to make your own package.

Why Build a Package?¶

There are a bunch of nifty tools that help you build, install and distribute packages.

Using a well structured, standard layout for your package makes it easy to use those tools.

Even if you never want to give anyone else your code, a well structured package simplifies development.

What is a Package?¶

A collection of modules

… and the documentation

… and any top-level scripts

… and any data files required

… and a way to build and install it…

Python packaging tools:¶

distutils : included with python

Getting clunky, hard to extend, maybe destined for deprecation …

setuptools : for extra features, technically third party

• present in most modern Python installations

“The Python Packaging Authority” – PyPA

setuptools¶

setuptools is an extension to distutils that provides a number of extensions:

superset of the distutils setup

This buys you a bunch of additional functionality:

• auto-finding packages
• better script installation
• resource (non-code files) management
• develop mode
• a LOT more

Where do I go to figure this out?¶

This is a really good guide:

Python Packaging User Guide:

and a more detailed tutorial:

Follow one of them

There is a sample project here:

(this has all the complexity you might need…)

You can use this as a template for your own packages.

Here is an opinionated update – a little more fancy, but some good ideas:

Basic Package Structure:¶

CHANGES.txt : log of changes with each release

LICENSE.txt : text of the license you choose (do choose one!)

MANIFEST.in : description of what non-code files to include

README.txt : description of the package – should be written in ReST or Markdown (for PyPi):

setup.py : the script for building/installing package.

bin/ : This is where you put top-level scripts

( some folks use scripts )

docs/ : the documentation

package_name/ : The main package – this is where the code goes.

test/ : your unit tests. Options here:

Put it inside the package – supports

Or keep it at the top level.

Some notes on that:

` Where to put Tests `_

The setup.py File¶

Your setup.py file is what describes your package, and tells setuptools how to package, build and install it

It is python code, so you can add anything custom you need to it

But in the simple case, it is essentially declarative.

What Does setup.py Do?¶

• Version & package metadata
• List of packages to include
• List of other files to include
• List of dependencies
• List of extensions to be compiled (if you are not using scikit-build.

An example setup.py :¶

setup.cfg ¶

Provides a way to give the end user some ability to customize the install

It’s an ini style file:

simple to read and write.

command is one of the Distutils commands (e.g. build_py, install)

option is one of the options that command supports.

Note that an option spelled —foo-bar on the command-line is spelled foo_bar in configuration files.

Running setup.py ¶

With a setup.py script defined, setuptools can do a lot:

Builds a source distribution (a tar archive of all the files needed to build and install the package):

(you need the wheel package for this to work:)

pip install wheel

Build from source:

Develop mode¶

Install in “develop” or “editable” mode:

Under Development¶

Develop mode is really, really nice:

(the e stands for “editable” – it is the same thing)

It puts a link (actually *.pth files) into the python installation to your code, so that your package is installed, but any changes will immediately take effect.

This way all your test code, and client code, etc, can all import your package the usual way.

No sys.path hacking

Good idea to use it for anything more than a single file project.

Install	Development Install
Copies package into site-packages	Adds a .pth file to site-packages, pointed at package source root
Used when creating conda packages	Used when developing software locally
Normal priority in sys.path	End of sys.path (only found if nothing else comes first)

Aside on pip and dependencies¶

• pip does not currently have a solver: http://github.com/pypa/pip/issues/988
• pip may replace packages in your environment with incompatible versions. Things will break when that happens.
• use caution (and ideally, disposable environments) when using pip

Getting Started With a New Package¶

For anything but a single-file script (and maybe even then):

1. Create the basic package structure
2. Write a setup.py
3. pip install -e .
4. Put some tests in package/test
5. pytest in the test dir, or pytest —pyargs package_name

or use “Cookie Cutter”:

Exercise: A Small Example Package¶

• Create a small package
  • package structure
  • setup.py
  • python setup.py develop
  • at least one working test

Start with the silly code in the tutorial repo in:

or you can download a zip file here:

capitalize¶

capitalize is a useless little utility that will capitalize the words in a text file.

But it has the core structure of a python package:

• a library of “logic code”
• a command line script
• a data file
• tests

So let’s see what’s in there:

What are these files?¶

So it works, as long as you are in the directory with all the code.

Setting up a package structure¶

Create a basic package structure:

Let’s create all that for capitalize:

Make the package:

Move the code into it:

Create a dir for the tests:

Move the tests into that:

Create a dir for the script:

Move the script into that:

Create directory for data:

Move data into that:

Now we have a package!

OK, that didn’t work. Why not?

Well, we’ve moved everytihng around:

The modules don’t know how to find each other.

Let’s Write a setup.py ¶

(remember that a “package” is a folder with a __init__.py__ file)

How do I install Python packages on Windows?

I’m having a hard time setting up python packages. EasyInstall from SetupTools is supposed to help that, but they don’t have an executable for Python 2.6.

For instance to install Mechanize, I’m just supposed to put the Mechanize folder in C:\Python24\Lib\site-packages according to INSTALL.txt, but runnning the tests does not work. Can someone help shed some light on this? Thanks!

12 Answers 12

The accepted answer is outdated. So first, pip is preferred over easy_install , (Why use pip over easy_install?). Then follow these steps to install pip on Windows, it’s quite easy.

Optionally, you can add the path to your environment so that you can use pip anywhere. It’s somewhere like C:\Python33\Scripts .

Newer versions of Python for Windows come with the pip package manager. (source)

pip is already installed if you’re using Python 2 >=2.7.9 or Python 3 >=3.4

Use that to install packages:

So in your case it’d be:

This is a good tutorial on how to get easy_install on windows. The short answer: add C:\Python26\Scripts (or whatever python you have installed) to your PATH.

You don’t need the executable for setuptools. You can download the source code, unpack it, traverse to the downloaded directory and run python setup.py install in the command prompt

Starting with Python 2.7, pip is included by default. Simply download your desired package via

Packaging in Python is dire. The root cause is that the language ships without a package manager.

Fortunately, there is one package manager for Python, called Pip. Pip is inspired by Ruby’s Gem, but lacks some features. Ironically, Pip itself is complicated to install. Installation on the popular 64-bit Windows demands building and installing two packages from source. This is a big ask for anyone new to programming.

So the right thing to do is to install pip. However if you can’t be bothered, Christoph Gohlke provides binaries for popular Python packages for all Windows platforms http://www.lfd.uci.edu/

In fact, building some Python packages requires a C compiler (eg. mingw32) and library headers for the dependencies. This can be a nightmare on Windows, so remember the name Christoph Gohlke.

I had problems in installing packages on Windows. Found the solution. It works in Windows7+. Mainly anything with Windows Powershell should be able to make it work. This can help you get started with it.

• Firstly, you’ll need to add python installation to your PATH variable. This should help.
• You need to download the package in zip format that you are trying to install and unzip it. If it is some odd zip format use 7Zip and it should be extracted.
• Navigate to the directory extracted with setup.py using Windows Powershell (Use link for it if you have problems)
• Run the command python setup.py install

That worked for me when nothing else was making any sense. I use Python 2.7 but the documentation suggests that same would work for Python 3.x also.