pip ignores dependency_links in setup.py
A couple of notes on some issues I found, in particular for installing from private repos.
Installing from pip & setuptools have some subtle differences; but this way should work for both.
from setuptools import setup
import os
# get deploy key from https://help.github.com/articles/git-automation-with-oauth-tokens/
github_token = os.environ['GITHUB_TOKEN']
setup(
# ...
install_requires='package',
dependency_links = [
'git+https://{github_token}@github.com/user/{package}.git/@{version}#egg={package}-0'
.format(github_token=github_token, package=package, version=master)
]
A couple of notes here:
- For private repos, you need to authenticate with GitHub; the simplest way I found is to create an oauth token, drop that into your environment, and then include it with the URL
- You need to include some version number (here is
0
) at the end of the link, even if there's no package on PyPI. This has to be a actual number, not a word. - You need to preface with
git+
to tell setuptools it's to clone the repo, rather than pointing at a zip / tarball version
can be a branch, a tag, or a commit hash- You need to supply
--process-dependency-links
if installing from pip
since pip version 18.1 PEP 508 URL is supported. That means you don't need the deprecated dependency_links anymore. You write the dependency directly in the install_requires list instead. The example from @Chad looks like this:
setup(
name='yourpackage',
version='1.7.5',
packages=[],
url='',
license='',
author='',
author_email='',
description='',
install_requires=[
'somepackage==1.2.0',
'repo @ https://github.com/user/archive/master.zip#egg=repo-1.0.0',
'anotherpackage==4.2.1'
],
)
To install your package you can simply write:
pip install yourpackage
(without --process-dependency-links)
This answer should help. In a nutshell, you need to specify the version (or "dev") for the #egg=python-s3
so it looks like #egg=python-s3-1.0.0
.
Updates based on @Cerin's comment:
- Pip 1.5.x has a flag to enable dependency-links processing:
--process-dependency-links
. I haven't tested it because I agree with the point below. - This discussion seems to indicate that using dependency-links for pip is a bad practice. Although this feature was enlisted for deprecation, it's not anymore. There's a valid use case for private packages.
I realize this is an old question, but, just in case you find yourself here like I did, this is what worked for me.
I've got a package on GitHub (not registered with pypi) that relies on other GitHub (non-pypi) packages. I spent an inordinate amount of time trying to figure out how to get pip to handle this correctly. I will include what I did to fix it here.
Putting dependencies in a requirements.txt file is the preferred method of listing dependencies. However, you also need to populate install_requires in setup. It was at this stage that I ran into a roadblock with pip not wanting to install dependencies from GitHub.
Most places, including answers to this question, tell you to populate the dependency_links section of setup. However, you also need to populate the install_requires field with the name of the package referenced in dependency_links.
For example, if your requirements.txt contains the following.
somepackage==1.2.0
https://github.com/user/repo/tarball/master#egg=repo-1.0.0
anotherpackage==4.2.1
Then, your setup call should look like this:
setup(
name='yourpackage',
version='1.7.5',
packages=[],
url='',
license='',
author='',
author_email='',
description='',
install_requires=[
'somepackage==1.2.0',
'repo==1.0.0',
'anotherpackage==4.2.1'
],
dependency_links=[
'https://github.com/user/repo/tarball/master#egg=repo-1.0.0'
]
)
Ok, so now we've got our package configured; installing it is the next task. This is where I spent a lot of time. I could not figure out why specifying dependency_links apparently did nothing. The trick is that in some cases, you need to set the allow-all-external (can be more specific) flag for pip. For example:
pip install git+https://github.com/user/anotherrepo.git
--process-dependency-links --allow-all-external
You're done and it works!
DISCLAIMER: dependency_links and the flags process-dependency-links and allow-all-external are deprecated, so they will be removed soon. In the time I spent, I could not locate a better, prefered method and still have pip function properly.