Make Python script combined with linux packages easy installable for end-user
I think a good starting point will be a Dockerfile, in which you start from an Ubuntu image and then install your desired dependencies(samtools, bwa) and also the python packages (placed in a requirements.txt)
pip freeze > requirements.txt
to have your desired python packages (numpy, tqdm etc)create a Dockerfile (which resides in the same directory as requirements.txt) e.g:
FROM ubuntu:16.04 RUN apt-get update && \ apt-get upgrade -y && \ apt-get install -y python && \ apt-get install -y wget \ curl \ bc \ unzip \ less \ bedtools \ samtools \ openjdk-8-jdk \ tabix \ bwa\ python-pip\ software-properties-common && \ apt-get -y clean && \ apt-get -y autoclean && \ apt-get -y autoremove RUN mkdir -p /usr/src/app WORKDIR /usr/srv/app COPY . /usr/srv/app RUN pip install -r requirements.txt CMD ["/bin/bash"]
build your docker image
docker built --tag repository/imagename .
(DOT is the current directory)- start image
docker run -dti --name test repository/imagename
- enter in the container to start working
docker attach test
- as you can see, I tested it and inside the container I have all the desired tools and packages
If you want to add new tools just add them in the Dockerfile and if you want to add python packages just expand the requirements.txt file.
If you have a python script that does something and you need to include it into the Docker container , just put it in the same directory with the Dockerfile and requirements.txt and update the Dockerfile (in order to have a container that starts your python script), more precisely in the Dockerfile the last line will be
CMD [ "python", "./my_script.py" ]
I wouldn't go as far as installing software onto some computer/server because it might break other software running on that system. Instead, I would create a deb package and set dependencies on which software has to be installed to guide the user through installation.
1. Create the deb directory structure
The directory deb
holds final scripts/binaries for the package. source
has your python code.
$ cd $HOME/path/to/some/directory
$ mkdir -p ./deb/DEBIAN ./source
$ touch ./Makefile ./source/Makefile ./deb/DEBIAN/{control,preinst} ./source/hello-world.py
$ chmod +x ./deb/DEBIAN/preinst
2. Add files
Below I've added the content of every file. You should be able to implement it to your needs.
./Makefile: This file is responsible for generating the source and the deb.
PACKAGE := hello-world
DEB_DIR := deb
DEB_FILE := $(PACKAGE).deb
SOURCE_DIR := source
SOURCE_IN := $(SOURCE_DIR)/$(PACKAGE).py
SOURCE_OUT := $(DEB_DIR)/usr/bin/
.PHONY: all clean
all:
@make -C $(SOURCE_DIR) CROSS_COMPILE=$(CROSS_COMPILE)
@mkdir -p $(SOURCE_OUT)
@cp -r $(SOURCE_IN) $(SOURCE_OUT)
@chmod -R a-s $(DEB_DIR)
@dpkg-deb --build $(DEB_DIR) $(DEB_FILE)
clean:
@rm -rf $(DEB_FILE) $(SOURCE_OUT)
@make -C $(SOURCE_DIR) clean
./deb/DEBIAN/control: The control file for Debian's package manager.
Package: hello-world
Version: 1
Section: misc
Priority: optional
Architecture: all
Depends: libc6 (>= 2.24), python (>=2.7.13), virtualenv (>=1.11.6)
Maintainer: Your name <[email protected]>
Description: This is a deb package to install an application
./deb/DEBIAN/preinst: The file in which you can install python packages. It is run before the installation of hello-world.py
. Again, make sure you don't just (re)install/update a library. It could cause incompatibility problems in other software.
#! /bin/bash
virtualenv -p /usr/bin/python3 venv/
source venv/bin/activate
pip install -r ./requirements.txt
./source/Makefile: A file I used to test this setup.
MYFILE=hello-world.py
all:
@echo "#!/usr/bin/python\n\
print('Hello world!')" > $(MYFILE)
@chmod +x $(MYFILE)
clean:
@rm $(MYFILE)
3. Installation and removal
Installing is very easy, just use dpkg
.
dpkg -i hello-world.deb # for installation
dpkg -r hello-world # for removal
A nice feature to have is you don't need to worry about versioning your software, since it's done in the control
file. Debian has a pretty good tutorial if you're interested in reading more.
EDIT:
Added virtualenv
to example. Python packages should be installed in a virtual environment instead of globally.