Make Python script combined with linux packages easy installable for end-user

I think a good starting point will be a Dockerfile, in which you start from an Ubuntu image and then install your desired dependencies(samtools, bwa) and also the python packages (placed in a requirements.txt)

  • pip freeze > requirements.txt to have your desired python packages (numpy, tqdm etc)
  • create a Dockerfile (which resides in the same directory as requirements.txt) e.g:

    FROM ubuntu:16.04
    
    RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y python && \
    apt-get install -y wget \
    curl \
    bc \
    unzip \
    less \
    bedtools \
    samtools \
    openjdk-8-jdk \
    tabix \
    bwa\
    python-pip\
    software-properties-common && \
    apt-get -y clean  && \
    apt-get -y autoclean  && \
    apt-get -y autoremove
    
    RUN mkdir -p /usr/src/app
    
    WORKDIR /usr/srv/app
    
    COPY . /usr/srv/app
    
    RUN pip install -r requirements.txt
    
    CMD ["/bin/bash"]
    
  • build your docker image docker built --tag repository/imagename . (DOT is the current directory)

  • start image docker run -dti --name test repository/imagename
  • enter in the container to start working docker attach test
  • as you can see, I tested it and inside the container I have all the desired tools and packages enter image description here

  • If you want to add new tools just add them in the Dockerfile and if you want to add python packages just expand the requirements.txt file.

  • If you have a python script that does something and you need to include it into the Docker container , just put it in the same directory with the Dockerfile and requirements.txt and update the Dockerfile (in order to have a container that starts your python script), more precisely in the Dockerfile the last line will be CMD [ "python", "./my_script.py" ]


I wouldn't go as far as installing software onto some computer/server because it might break other software running on that system. Instead, I would create a deb package and set dependencies on which software has to be installed to guide the user through installation.

1. Create the deb directory structure
The directory deb holds final scripts/binaries for the package. source has your python code.

$ cd $HOME/path/to/some/directory
$ mkdir -p ./deb/DEBIAN ./source
$ touch ./Makefile ./source/Makefile ./deb/DEBIAN/{control,preinst} ./source/hello-world.py
$ chmod +x ./deb/DEBIAN/preinst

2. Add files
Below I've added the content of every file. You should be able to implement it to your needs.

./Makefile: This file is responsible for generating the source and the deb.

PACKAGE         := hello-world

DEB_DIR         := deb
DEB_FILE        := $(PACKAGE).deb

SOURCE_DIR      := source
SOURCE_IN       := $(SOURCE_DIR)/$(PACKAGE).py
SOURCE_OUT      := $(DEB_DIR)/usr/bin/

.PHONY: all clean
all:
        @make -C $(SOURCE_DIR) CROSS_COMPILE=$(CROSS_COMPILE)
        @mkdir -p $(SOURCE_OUT)
        @cp -r $(SOURCE_IN) $(SOURCE_OUT)
        @chmod -R a-s $(DEB_DIR)
        @dpkg-deb --build $(DEB_DIR) $(DEB_FILE)

clean:
        @rm -rf $(DEB_FILE) $(SOURCE_OUT)
        @make -C $(SOURCE_DIR) clean

./deb/DEBIAN/control: The control file for Debian's package manager.

Package: hello-world
Version: 1
Section: misc
Priority: optional
Architecture: all
Depends: libc6 (>= 2.24), python (>=2.7.13), virtualenv (>=1.11.6)
Maintainer: Your name <[email protected]>
Description: This is a deb package to install an application

./deb/DEBIAN/preinst: The file in which you can install python packages. It is run before the installation of hello-world.py. Again, make sure you don't just (re)install/update a library. It could cause incompatibility problems in other software.

#! /bin/bash
virtualenv -p /usr/bin/python3 venv/
source venv/bin/activate
pip install -r ./requirements.txt

./source/Makefile: A file I used to test this setup.

MYFILE=hello-world.py
all:
        @echo "#!/usr/bin/python\n\
        print('Hello world!')" > $(MYFILE)
        @chmod +x $(MYFILE)
clean:
        @rm $(MYFILE)

3. Installation and removal Installing is very easy, just use dpkg.

dpkg -i hello-world.deb # for installation
dpkg -r hello-world     # for removal

A nice feature to have is you don't need to worry about versioning your software, since it's done in the control file. Debian has a pretty good tutorial if you're interested in reading more.

EDIT:
Added virtualenv to example. Python packages should be installed in a virtual environment instead of globally.