How to locally develop a python package?

I'm trying to make changes to an existing python module, and then test it locally. What's the best way to do this?

I cloned the github module and made changes, but I'm not sure how to import the local package instead of the already installed one.

python

4 Answers
4

The easiest way to do such testing would be to create a virtual environment, and then installing the package in development mode.

Assuming you are on Linux it would look something like this.

$ virtualenv dev_env
$ source dev_env/bin/activate
$ cd ~/project_folder
$ pip install -e .


This workflow would not overwrite the already installed package on your system. Other maybe even simpler alternatives would to just use a IDE like PyCharm that handles most of this for you.

One way consists in using sys.path().

For example:

import sys
sys.path.insert(0, path/to/module)


In this way, you give priority to a specific path when looking for a module.

This means that the module you want to import will be searched first in path/to/module and after in the other directories already in sys.path.

path/to/module

sys.path

The advantage of this approach is that this new order will hold only inside your script without changing the import order of the other ones.

Note: For development purposes you should use a virtualenv as suggested by @eandersson.

__init__.py

You should probably be doing most of your development work in a virtual environment. Your workflow for this could look like:

# activate the virtual environment in ~/vpy
. $HOME/vpy/bin/activate

# install my app and its dependencies
cd $HOME/src/myapp
pip install -e .

# use my forked library instead
cd $HOME/src/forkedlib
pip install -e .
pytest # or whatever tests the forked lib has

# try it out with my application too
cd $HOME/src/myapp
pytest # or whatever tests your app has
myapp


pip install -e does some magic so that, whenever you import the module in the library, it gets routed directly to the checked-out source tree, so if you make edits in forkedlib and then re-run myapp, you'll see those changes directly.

pip install -e

import

forkedlib

myapp

When you're done, you can pip uninstall forkedlib and then re-run pip install -e . to reinstall your application's (declared) dependencies. (Or delete and recreate the virtual environment, if that's easier.)

pip uninstall forkedlib

pip install -e .

The approach of the answer by abc adding the module path to the system path is fine for local, instant testing, but it's not the full solution, for example when C code needs to be compiled or command line hooks must be set. For a full test you might want to install the package instead.

A typical Python has a setup.py and can be packaged into a distribution file (wheel, ...) which can then be installed locally from a local file.

setup.py

The workflow would be:

python setup.py bdist_wheel

pip install --no-index --find-links=..

This results in exactly the same situation every future user of the package will find itself in and is a complete test (including the installing process), but it's also a lot of effort, that's why I usually only us the system path method during development, but the full installation way only directly before a release.

Thanks for contributing an answer to Stack Overflow!

But avoid …

To learn more, see our tips on writing great answers.

Required, but never shown

Required, but never shown

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.