Software installations

In general when installing software, we prefer to install software as modules. This facilitates for users to revert to older software versions, as well as to keep track of what version was used for a specific project. However, most general libraries (eg. in many cases packages called libsomething in Debian) should be installed as Debian packages. If unsure, please discuss with Mikael and/or Les. Debian packages can only be installed by system administrators, while software packages installed and made available as software modules can also be installed by users who have been granted software install permissions (and adhere to the software install guidelines.)

Which software should be installed and made available to all users? In general, software that will have a use by more than a single project, and that can be installed in a manner that can be shared by many users can be installed. (This excludes for example excludes software that needs write permission to a central database och file set)

Software install procedure

Software install is a two step procedure. First you install the software into a given directory that is created under /opt/sw and made available to you. Once the module file is written, a system administrator can make the software available for all users. This two step procedure is implemented for security and quality reasons. Software under /opt/sw is installed in directories adhering to the naming standard /opt/sw/SOFTWARE/VERSION. Since we are now using Lmod, module files can be written either in the legacy TCL/C syntax, or in Lua. Please visit Lmod homepage for more information ​https://github.com/TACC/Lmod/

Detailed procedure (not involving software specifics on how to set installation location). See below for instructions on how to install python/perl modules.

1. Create a ticket regarding the software install, unless the user has already done so.
2. module load gridadmin to enable software installation related modules.
3. Run the command gridsw to switch primare group to mykopat-grid-software. This must be done prior to any work on software in the shell you will be working with.
4. Run grid-mkswdir SOFTWARE VERSION to create a directory /opt/sw/SOFTWARE/VERSION, where you can install the package. This directory is read and writable by you, readable by other software admins, but not by ordinary users.
5. Dowload software source or binaries archive to /opt/sources
6. Unzip archive, and if necessary configure/compile whith correctly specified installation path
7. Install package into the the folder obtained at step 4. (This should be done as your ordinary user, in a shell where you have run the gridsw command.) If the install script/makefile/install procedure complains about denied permissions, DO NOT USE SUDO. You should rather check that all install paths etc are correct.
8. Write a module file (or copy and edit previous version) to reflect the new software version you have installed. The module file should be created in /opt/modulefiles_staging/Core for standard software and named SOFTWARE/VERSION.lua. See below for special routines for installing Python packages and Perl packages.
9. Add the module file to subversion with svn add software/version.lua. If this is the first version installed of the software, you also need to add the directory to Subversion (svn add /opt/modulefiles_staging/Core/software/). Once the file (and directory are added), commit the module file with svn commit software, note both the file and folder must be committed, to do this at the same time use svn ci software_folder_name.
10. Update the wiki page for the software referenced from the software index found on Software (or create a new one). This page should, as a bare minimum, contain any local modifications or specifics, and a link to a manual for the software. If you need to create a new page, please use the software page template.
11. Document any deviations from the documented install procedure of the software in the ticket.
12. Ask a system administrator to publish the newly installed software by providing SOFTWARE and VERSION as per above, as well as ticket ID. This can be done by reassigning the ticket to Les or Mikael

Installation of Python and Perl packages as modules

The basic Python/Perl? interpreter should be installed in the same manner as any other package. Any python/perl package should be installed as modules which then set a depends_on() for the specific version of the interpreter it was installed with. The dependent version of the interpreter should be documented in the wiki page. Modules for python packages should be named -py3 if they are installed against python3.

Python Packages

In many cases, installation of python packages can be trivial. If they are available in the python package index, then the actual compile/install steps boils down to pip install --prefix=/opt/sw/SOFTWARE/VERSION SOFTWARE (maybe with an added -py3 and pip3). The python module you want to install against has to be loaded. In cases where you have to run setup.py yourself, you need to specify a proper prefix to the install command.

If the package you are about to install requires any of our currently available modules, then please load the module first, and then install the package. Don't forget to add the pre-loaded modules as requirements in the new module file. (Check e.g. the biopython-py3 module for how to achieve this.)

Perl packages

Similar to python packages, the install prefix has to be set when installing perl packages. Packages that can be installed using CPAN, if prefix paths are set first using o conf makepl_arg PREFIX=/opt/sw/SOFTWARE/VERSION and conf mbuildpl_arg "--prefix /opt/sw/bioperl/2.1.8" and finally o conf commit.¨