Whether you are a developer, a tester, a system maintainers, or just curious about the LSST code, you will eventually find yourself in need of installing all or part of the LSST Software Stack. This section describes how to install the software stack and development environment. It is mainly intended for the LSST developer; however, this document will be helpful to administrators that just wish to install a stable release.
Further detailed information on the build system can be found in the document the "LSST Build and Test Systems".
In general, the development environment is made up of three areas.
The first is a complete installation of the standard software stack,
which we refer to as LSST_HOME. This
includes both third-party packages and packages developed by the LSST
team. This stack may contain multiple versions of the various
packages, and our tools allow you to switch between them as necessary.
The second area is the developer's sandbox or
LSST_DEVEL. This is where you install
and test your own private versions of LSST software. This area looks
just like the LSST_HOME area, except that it normally will only
include versions of software that you are actually developing. The
third area is your copy of the software source tree, checked out of
the code repository.
In brief, to create a complete LSST development environment, you will..
If someone else has already installed the LSST stack on your platform,
then you skip the installation instructions and go directly to
Section 3 to learn how to using the software
stack. Either way, once you have an LSST_HOME area
installed, you will probably find it convenient to add some lines to
your shell "rc" file (Section 3.2). You will
also need to become familiar with the developers' build process:
Section 4 introduces our Subversion code
repository, and Section 5 explains how to build
LSST packages from source.
Before you get started however, you may need to arrange for some login accounts. You can get that process started now; then, while you are waiting for the accounts to be set up, you can get started on the software download process. Now consider these questions:
svn.lsstcorp.org. Contact David Gehrig (dgehrig at
ncsa.uiuc.edu) for this. In this section, we will download and install the standard software stack, or LSST_HOME. You will typically treat this as a read-only area, updating it only when new releases are announced.
It may be the case that you are developing on a shared system where the software stack has already been installed for you. If so, you can skip this section.
In this section, you will make your first encounter with our primary
product for managing LSST software: EUPS. EUPS is to select and keep
track of the different versions of our software packages and how they
depend on other packages. It includes the ability to download the
packages from the LSST
development server, lsstdev.ncsa.uiuc.edu
and install them on your machine.
To create your LSST_HOME area, follow
this recipe:
If you have already once installed the software stack, you
should be sure unset your environment variables for that
stack. To do this, you can type (after the %
prompt) one of the following:
% unsetenv LSST_HOME EUPS_PATH # for tcsh users % unset LSST_HOME EUPS_PATH # for bash users
The home of the LSST installation is your choice. (Just don't use the same location as a previous LSST installation unless you move it out of the way first.) For the convenience of typing, you can set the LSST_HOME environment variable now:
% setenv LSST_HOME /the/LSST/installation/root # for tcsh users % export LSST_HOME=/the/LSST/installation/root # for bash users
Now create and change into this directory:
% mkdir -p $LSST_HOME; cd $LSST_HOME
% curl -o newinstall.sh http://lsstdev.ncsa.uiuc.edu/dmspkgs/newinstall.sh % bash newinstall.sh LSSTPipe
Now get a beverage, go to lunch, or work on that paper as this step will take about 60 minutes.
Note:
If you do not havecurlon your machine, you can usewgetinstead, as in:% curl -o newinstall.sh http://lsstdev.ncsa.uiuc.edu/dmspkgs/newinstall.sh curl: Command not found. % wget http://lsstdev.ncsa.uiuc.edu/dmspkgs/newinstall.sh % bash newinstall.sh LSSTPipe
This script has two phases to it. First, the script will create some directories that will contain installed software and then install some core or foundation software packages. The first of these packages is the EUPS tool, which we use to keep track of the packages we install. Next it proceed to install other package that are important for downloadin, building, and installing packages. This includes our designated version of Python, our "configure-and-make" software, SCons, and some specialized build utilities.
If you had not included the
LSSTPipe argument, the
script would end once the core infrastructure packages were
installed; with arguments, the script will enter its second
stage. Each argument is the name of a package to install
(along with all its dependencies).
LSSTPipe is a "pseudo-package"
that represents all of the packages needed for development and
execution of the LSST pipelines. Note that the packages that
make up this set will change as data challenge development
proceeds. You can download the latest packages at any type
later using the lsstpkg tool as
described in section 2.3 below.
% source loadLSST.csh # for tcsh users % source loadLSST.sh # for bash users
(See Section 3 for more details.)% ls EupsBuildDir eups Linux loadLSST.csh loadLSST.sh newinstall.sh ups_db
This directory is what we refer to as the
LSST_HOME. Everything thing that was
installed can be found below this directory.
In summary, the contents include:
Linux,
Linux64, or
DarwinX86: a directory
that represents your platform flavor (more on this below). eups: the EUPS package that keeps
track of all the installed packages. ups_db: the data directory used
by EUPS to keep track of the software. loadLSST.csh and
loadLSST.sh:
the environment initialization scripts. EupsBuildDir: a scratch directory used
for building packages before they are installed. newinstall.sh: the script
you just ran. loadLSST.csh and
loadLSST.sh are scripts that when
"sourced", loads the basic LSST software environment:
% source loadLSST.csh # for tcsh users % source loadLSST.sh # for bash or zsh users
A user would usually do this at the beginning of a session, either explicitly or via a shell "rc" file (see Section 3.2).
The "flavor" directory--usually calledLinux,
Linux64, or
DarwinX86--is where all of the
LSST software packages (other than EUPS) can be found. When you load
the LSST environment, an environment variable called
LSST_PKGS is set to this directory:
% echo $LSST_PKGS /the/LSST/installation/root/Linux
Note:
If you want to include software stacks for multiple platforms under the sameLSST_HOMEdirectory, then the directory would contain one directory for each platform.
Inside the flavor directory, you will find the installed LSST packages, looking something like this:
% ls Linux afw daf_base external pex_exceptions sconsUtils ap daf_data lsst pex_harness security base daf_persistence lssteups pex_logging utils ctrl_events detection mops pex_policy
These are primarily package LSST packages--that is, packages written
by the LSST DM team. The external
sub-directory contains all of the 3rd party software (e.g. Python,
swig, scons, boost, wcslib, etc.).
Each package (be it an LSST package or 3rd party) has its own named directory which contains one or more subdirectories named after the package version:
% ls $LSST_PKGS/external/python 2.5.2
If multiple versions are installed, there will be one version subdirectory for each.
lsstpkg. For example, to install a specific version of
some package, such as version 1.35.0 of the boost package, you
might type,
% lsstpkg install boost 1.35.0 eups distrib install boost 1.35.0-1 Required product termcap 1.3.1 already installed Required product tcltk 8.5a4 already installed Required product readline 5.2 already installed Required product python 2.5.2 already installed now installing boost 1.35.0 Version: svn Setting up environment (via ./eupssetups.sh)... /usr/bin/curl http://lsstdev.ncsa.uiuc.edu/dmspkgs/external/boost/1.35.0/boost_1_35_0.tar.gz > boost_1_35_0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 27.2M 100 27.2M 0 0 7198k 0 0:00:03 0:00:03 --:--:-- 7257k boost_1_35_0.tar.gz tar zvxf boost_1_35_0.tar.gz ./configure --prefix=/home/lsst/dmspkgs/Linux64/external/boost/1.35.0 /usr/bin/make /usr/bin/make install Emptying the build directory rmdir /home/lsst/dmspkgs/EupsBuildDir/boost-1.35.0 Package boost 1.35.0 installed successfully
If you get a message complaining that the lsstpkg command
is not found, then you don't have the environment fully setup up.
Sourcing the loadLSST script should fix this.
You don't have to specify the version. If you leave it off, as in:
lsstpkg install boost
you will get the so-called "current" version--that is, the version that is currently recommended by the project.
As the first line of its output is meant to suggest, the lsstpkg
command is a thin layer around the eups distrib command.
To learn more about the eups tool, see section 5.1.
Can one use 'Finally, if you want to know what packages are available from the server, you can useeups distrib' instead of 'lsstpkg'?
Yes.lsstpkgis just a convenience command. The main reason for using lsstpkg is that it guarantees that LSST extensions to the EUPS tool have been setup; if they have not been,eups distribwill fail. So, if you useeups distribdirectly, we recommend that you be sure to first setting up thelsstpackage (type:setup lsst).
lsstpkg to list them:
% lsstpkg list boost eups distrib list boost ... % lsstpkg list boost eups distrib list ...
When a package fails to install, you should see output that ends like this:
... lssteupsbuild.sh: scons install failed; see .../EupsBuildDir/ip_diffim-3.0/ip_diffim-3.0/build.log for details Failed to build and install ip_diffim-3.0.tar.gz
Prior to these final messages, you should also see some indication of what went wrong, like the last error messages that were encountered before the install process exited.
Consider installation failures as falling into two categories of possible causes:
lsstdev.ncsa.uiuc.edu, loadLSST.(c)sh has not been
sourced). lsstdev.ncsa.uiuc.edu server
For problems in latter category, please issue a bug report via the
Trac system, or (if you are not yet familiar with Trac or don't have
permission to submit bugs), report the problem to Ray Plante (rplante
at ncsa.uiuc.edu). Include in your report the output from the
lsstpkg command.
You might also consult our Platform support page to make sure that you have the necessary prerequisite libraries install. Follow the link to the page on your architecture (Linux or Macs) and find the section for your OS distribution.
If you are inclined to investigate the problem further on your
platform (always appreciated), note that you can find the uninstalled
package under $LSST_HOME/EupsBuildDir. Usually you must
descend two directories named after the failed package
(e.g. "ip_diffim-3.0") to find the build log called
build.log.
If the problem has been corrected (either on the server or with your
OS), you restart the installation process where you left off without
having to reinstall packages which were previously installed
correctly. Before you restart though, it's a good idea to first
clean out the package that failed. This is also done with the
lsstpkg command which can remove the any remnants of a
partially built or installed package. For example, suppose you had a
failure while installing Boost. First, lsstpkg will
require that you specify the version of the package, so you will need
to determine that. This is obvious from the error message shown above
(the version was "3.0"). To clean out the package, type:
% lsstpkg clean ip_diffim 3.0
You can now restart lsstpkg:
% lsstpkg install LSSTPipe Installing 0.1 of LSSTPipe Product termcap (version 1.3.1, flavor Linux) is already declared Product tcltk (version 8.5a4, flavor Linux) is already declared Product readline (version 5.2, flavor Linux) is already declared ...
Note that you do not need to the entire LSSTPipe stack installed in order to start working with parts of the stack. If you are able to run thelsstpkg command, then you have a minimal stack
sufficient for checking out code and building new packages. If there
is a specific package that you need that is not yet installed
(how to list what's installed), you
can try using the lsstpkg to install it directly
(Section 2.3).
loadLSST.*sh set-up
script. For example, if the stack is installed in the
/lsst/stack directory and you use the bash shell, then
you can type:
% source loadLSST.sh
If you use csh or tcsh as you shell, you instead type:% source loadLSST.csh
Now you are ready to use and build LSST packages.
Sourcing this set-up script does the following:
$LSST_DEVEL to the EUPS_PATH
environment variable (discussed in Section 3.3).
LSST_HOME: the root directory of the stack
(e.g. /lsst/stack) LSST_PKGS: the directory where packages
are installed--that is, $LSST_HOME/platform
where platform indicates your development
platform (or flavor: e.g. Linux,
Linux64, or DarwinX86). LSST_SVN: the root of the Subversion code
repository. LSST_DMS: the code repository directory
where LSST software packages are found. lsst package.
Note that you are not obligated use this setup script; some users may
prefer to take greater control over the LSST environment. If you
don't use the loadLSST.*sh script, then you will at a
minimum need to set up EUPS yourself (by sourcing
eups/default/bin/setups.*sh under the
$LSST_HOME directory). Then you will need to use the
EUPS command setup to set up the individual
packages you need.
When you set up the lsst package you will get basic
environment needed to build or install an LSST package. This
includes the following packages,
python scons, used to build LSST packages from sourcelsst and lssteups, providing the
lsstpkg tool used for installing packages
downloaded from the LSST development server.
If you are going to do regular development, then you will probably
find it convenient to update your shell "rc"
file--e.g. .cshrc or .bashrc--to automate
the setting up of the LSST environment. This update is optional,
however; see the discussion below for details.
csh (or tcsh) as your default
shell:
# Load the LSST environment # # set useful environment variables. None of these are required, but they # are helpful. # setenv LSST_HOME /the/installation/root4/LSST setenv LSST_DEVEL /my/LSST/sandbox # # the setlsst command loads the environment on demand # alias setlsst source $LSST_HOME/loadLSST.csh # # uncomment this line to load the LSST environment automatically when # the shell starts # # setlsst
Alternatively, if you usebash:
# Load the LSST environment # # set useful environment variables. None of these are required, but they # are helpful. # export LSST_HOME=/the/installation/root4/LSST export LSST_DEVEL=/my/LSST/sandbox # # the setlsst command loads the environment on demand # alias setlsst=source $LSST_HOME/loadLSST.sh # # uncomment this line to load the LSST environment automatically when # the shell starts # # setlsst
Note these comments:loadLSST.csh (or loadLSST.sh) file.
Any needed environment variables (namely,
$LSST_HOME and $EUPS_PATH) are set to
defaults.
LSST_DEVEL. New packages will automatically be
installed into $LSST_HOME instead. setup commands to select certain packages by
default (see section 5.1 for details).
$LSST_HOME and
$LSST_DEVEL, then you should also
explicitly set the EUPS_PATH
variable, e.g.
% setenv EUPS_PATH ${$LSST_DEVEL}:${$LSST_HOME}:/other/eups/directory # tcsh % export EUPS_PATH=${$LSST_DEVEL}:${$LSST_HOME}:/other/eups/directory # bash
You may be working on a system in which the software stack has been
installed for you by an administrator, to be shared by a group of
users. If you do not have write permission on
$LSST_HOME, you will not be able to install new
packages into that stack. Or, if you do have permission, you may not
want new packages to get installed there (e.g. because you don't want
to make an experimental or "in-development" package installation
visible to others). In both situations, it is useful to have an
private "sandbox" where you can install new packages.
The lsst packages provides a script to create a sandbox
directory. Once created, we can integrate this private sandbox into
build system by setting the LSST_DEVEL environment
variable to the sandbox directory name. Then (after sourcing
loadLSST.*sh), this directory will not only become the
directory that packages are installed into by default, it will be the
first directory the EUPS system will look for packages to load into
the environment. If it doesn't find it in $LSST_DEVEL,
it will look in $LSST_HOME. Thus, the sandbox only needs
to contain packages one wishes to override packages in $LSST_HOME.
To create a sandbox called lsstsandbox under the current
directory, use the mksandbox command:
% mksandbox lsstsandbox % ls lsstsandbox Linux64 ups_db
Command not found?Now to use this sandbox, set
Did runningmksandboxreturnmksandbox: Command not found? This means your LSST software environment is not set (discussed in Section 3.1). This is most easily corrected by typing "source loadLSST.sh" (for bash users) or "source loadLSST.csh" (for csh/tcsh users). If you do not useloadLSST.*shsetup EUPS, then you need to make sure thelsstpackage is setup by typingsetup lsst.
LSST_DEVEL and reset your
LSST software environment. For example, if your shell is bash:
% export LSST_DEVEL=$PWD/lsstsandbox % source $LSST_HOME/loadLSST.sh
or if you are using csh/tcsh:% setenv LSST_DEVEL $PWD/lsstsandbox % source $LSST_HOME/loadLSST.csh
If you installed the software stack yourself
(Section 2), then you probably know that you
can use the lsstpkg to install new packages. If not,
review Section 2.3 to see how to use this
tool to install individual packages.
To get the latest versions of required pipeline packages, you try
re-installing the pseudo-package LSSTPipe at any
time:
% lsstpkg install LSSTPipe Installing 0.1 of LSSTPipe Product tcltk (version 8.5a4, flavor Linux) is already declared Product python (version 2.5.2, flavor Linux) is already declared. ...
This will install what are considered the "current" versions of all of
the pipeline packages. If "current" set has not changed since the
last time you ran this, then you will only see above kinds of messages
about a package already being installed. If a new package has been
added to the LSSTPipe set or if the "current" version of
a package has changed, the new package will be downloaded and
installed.
Note that new versions of packages may become available that are not yet considered "current". If you want to install a non-current version of a package, you will need to request it explicitly:
% lsstpkg install afw 3.2
If you want to just find out what versions of a package are
available from the server, use can use lsstpkg list:
% lsstpkg list afw eups distrib list afw Available products from primary server (http://lsstdev.ncsa.uiuc.edu/dmspkgs): afw generic 3.0 afw generic 3.1 afw generic 3.2
Note:
This section is under construction...
We use Subversion (svn) as our
version control system, and our repository is located at
svn.lsstcorp.org. We restrict access to
the svn tree using SSH. Thus, in order to check out code, you will need a
login to our svn server. If you haven't requested a login already, consult
these instructions for more information.
Once you have a login, you should test to make sure it works by
logging into svn.lsstcorp.org with ssh.
Next, you will probably want to set up some ssh keys so that you can
log in without typing your password. This is not require; however, if
you don't set this up, svn will repeatedly and inconveniently ask you
to type in your password which can be painful. For detailed
instructions on this, consult "Setting up SSH for
Subversion" (Document-1808) in the Docushare repository.
The client program you will use is called svn, so if you
do not have this program on your system, you will need to install it. The
Subversion source code is available from
http://subversion.tigris.org/
and builds quite readily under Linux. This site also provides
pointers to binary distributions as well.
If you are already familiar with using the Subversion client,
svn, then skip to the next section,
"Repository Organization". Note that Section 4.2 provides a
survival guide to SVN.
Access to the LSST subversion repository is through use of ssh. The general syntax to access the LSST repository is:
% svn command svn+ssh://svn.lsstcorp.org/path/to/code
The svn+ssh prefix tells subversion to use the ssh protocol to connect and once connected, run the subversion server to access the repository.
For those familiar with CVS, there is no mechanism for setting the
default root directory of the repository in subversion; however, for your
convenience two environment variables are set for you when you source
the loadLSST.*sh file:
$LSST_SVN -- svn+ssh://svn.lsstcorp.org $LSST_DMS -- svn+ssh://svn.lsstcorp.org/DMS This makes working with the SVN server a bit easier:
% svn command $LSST_SVN/path/to/code
We can now try a simple command to list the top level directories in the repository
% svn list $LSST_SVN DC2/ GilTest/ branches/ tags/ trunk/
The software being developed for the DC3 can all be found under a
top-level directory called DMS. Under that can be found
directories representing a package or a set of related packages:
% svn list $LSST_DMS # same as list $LSST_SVN/DMS afw/ analysis/ ap/ astrocal/ base/ cat/ catf/ coadd/ ctrl/ daf/ detection/ devenv/ ip/ mops/ pex/ photocal/ report/ security/ testdata/ utils/
Remember:
$LSST_DMSis an environment variable we set to besvn+ssh://svn.lsstcorp.org/DMS.
Note:
You may be prompted multiple times for your ssh password. This can be avoided by either using ssh-agent to manage your ssh session or by installing an ssh public key into the .ssh/authorized_users file in your home directory on the svn.lsstcorp.org server. See "Setting up SSH for Subversion" (Document-1808) for details.
You may check out the entire DMS tree; however, it is recommended that you really only check out the packages you plan to work on.
A package directory is one that contains four standard subdirectories:
% svn list $LSST_SVN/DC2/afw branches/ tags/ tickets/ trunk/
These sub-trees play the following roles:
trunk branches branches and checks in modifications there.
Later, if the developer wants to bring the new ideas into the
main-line, there is a merging process. (See "LSST Build and
Test Systems" for details.) tags tags named after the version--e.g.,
tags/1.0. (See section 5.4,
"Creating a Package Release" for details.)tickets
Note:
Normally, all development takes place under a tickets directory. This will ensure that the trunk always contains working, reviewed code.
If a directory does not contain these subdirectories, then the package represents a set of related packages, e.g.:
% svn list $LSST_DMS/pex exceptions/ harness/ logging/ policy/ % svn list $LSST_DMS/pex/logging branches/ tags/ tickets/ trunk/
As a start, we will establish our SVN development tree and can check
out the framework package, fw, into it.
Let's call this tree DMS:
% mkdir DMS; cd DMS
Now to check out the framework package:
% svn co $LSST_DMS/afw/trunk afw
Note:
You may be prompted multiple times for your ssh password. This can be avoided by either using ssh-agent to manage your ssh session or by installing an ssh public key into the .ssh/authorized_users file in your home directory on the svn.lsstcorp.org server. See "Setting up SSH for Subversion" (Document-1808) for details.
This checks out the main trunk of the fw
package into a new directory called fw:
% ls afw doc examples include lib python SConstruct src tests ups
You are now ready to hack! Proceed to section 5, "Working with LSST packages" to learn more about building an LSST package.
The Subversion command-line client svn interface is modeled after CVS. See also svn help for more commands and options, as well as the on-line book at http://svnbook.red-bean.com/nightly/en/svn-book.html Section 3 ("Guided Tour") which is especially useful for getting started.
If you are familiar with CVS, you should note these behavioral differences:
CVSROOT.
In the examples given below, it is assumed that the SVN environment variable has been set to svn+ssh://svn.lsstcorp.org/ as suggested above.
svn checkout URL PATH
This downloads the part of the repository pointed to by URL into the local directory given by PATH.
Example: Check out a package for normal development and put it into a local directory. Note that the trunk represents the leading edge of main development, excluding special branches or tagged sets.
% svn checkout $SVN/DC2/fw/trunk fw
Example: Check out a particular release of a package
% svn checkout $SVN/DC2/fw/tags/0.2 fw-0.2
svn add PATH svn delete PATH svn commit -m "LOG-MESSAGE" [--recursive] PATH ...
The first two commands above update your local copy of the code tree, while the third line commits the local changes to the repository.
Example: add a new source file and delete an old one.
% svn add foo.py % svn delete bar.py % svn commit -m "preferring foo.py over bar.py" foo.py bar.py
Here's another way to do the commit that catches all pending changes under a directory:
% svn commit -m "preferring foo.py over bar.py" --recursive .
An easy way to add an entire package tree to the repository is with the import command:
svn import PATH URL
This recursively imports a copy of PATH to repository.
Example: add in a new package, code and all:
% svn import mypkg $SVN/DC2/mypkg/trunk
svn update [PATH ...] ... svn commit -m "UPDATE-MESSAGE" [PATH ...]
The update command downloads any changes to the files or directories pointed to by PATH since the last time you checked out or updated them. The behavior is much like in CVS.
Example: update and commit some files
% svn update foo.c bar.c choo.c U foo.c G bar.c C choo.c
In this example, an updated version of foo.c was
downloaded. bar.c was also updated; however,
svn noticed that your local version included
uncommitted changes that you made, and these changes were
merged into the updated version. With choo.c, the
merge was not successful, and special tags were inserted that
mark where you need to resolve conflicts between your local
changes and the changes downloaded from the repository.
The difference in behavior from CVS is that in the case of a conflict, svn may create additional versions of the file to help with resolving the conflicts:
choo.c |
the updated version with conflicts tagged | |
choo.c.mine |
your pre-updated version | |
choo.c.r1 |
the committed version prior to your updates | |
choo.c.r2 |
the committed version to be merged with your version |
After resolving conflicts and making new changes, you can commit:
% svn commit -m "bug fix: memory leak" foo.c bar.c choo.c
If svn detects that someone else has checked in any changes since you did your update, the entire commit for all three files will fail (unlike with CVS). In this case, you should run the update command again on those files and then resolve any conflicts before trying another commit.
It's often useful to peruse the differences between your local files and those in the repository without actually doing an update. These command are useful for doing this.
svn status [PATH ...] svn diff [PATH ...] svn revert [--recursive] [PATH ...]
For more information on these commands see the SVN book, Section 3 (Guided Tour/Examine Your Changes) and Section 9 (Subversion Complete Reference).
EUPS is a tool that allows us to have installed multiple versions of
our packages on the same machine and provides us with the ability to
easily switch between them. It does this by manipulating key
environment variables, including PATH,
PYTHONPATH, and LD_LIBRARY_PATH, so that our
applications always finds the correct, desired version. It keeps
track of dependencies between packages as well, so that when a
particular version of a package is selected, all of the correct
versions of the dependent packages are chosen as well.
This section provides you with a quick introduction to the EUPS tools
and we use them in LSST development. For detailed information, you
can consult the
EUPS
Manual. Note also that all the EUPS commands accept a
--help or -h option:
% eups declare --help Wink lsst5 11: eups declare -h Usage: eups [commonOptions] declare [options] [product] [version] Declare a product Options: -c, --current Declare product current -Z, --database arg Use this products path. Default: $EUPS_PATH Alias: --with-eups --debug arg Permitted Values: raise -f, --flavor arg Use this flavor. Default: $EUPS_FLAVOR or `eups flavor` -F, --force Force requested behaviour -h, --help Print this help message -M arg Import the given table file directly into the database (may be "-" for stdin) -n, --noaction Don't actually do anything -r, --root arg Location of product being declared -z, --select-db arg Select the product paths which contain this directory. Default: all -m, --table arg Use table file (may be "none") Default: product.table -v, --verbose Be chattier (repeat for even more chat) -V, --version Print eups version number and exit
The most important environment variable used by EUPS is
EUPS_PATH. This variable contains a colon-separated list
of directories where it will look for information about installed packages.
In particular, each directory should contain a EUPS database,
represented by a subdirectory called ups_db. (Typically, a
EUPS_PATH directory also contains the actual packages
registered in the database, but it doesn't have to.) When you select
a package with the EUPS command, setup
(section 5.1.3), the EUPS_PATH directory
databases are searched in order and the first one matching the request
is selected.
The EUPS_PATH also plays an important role when
registering new packages with EUPS using eups declare
(section 5.1.4) and installing packages
with eups distrib (section
2.1). In both cases, only the first directory is used by default;
the others are ignored. That is, eups declare will
register any new packages into the database belonging to the first
EUPS_PATH directory, and eups distrib will
install new packages below the first EUPS_PATH
directory.
In Section 3.2, we recommended that you set the
value EUPS_PATH variable to
${LSST_DEVEL}:${LSST_HOME}.
The EUPS_FLAVOR directory contains the name of your
platform (currently, either "Linux" or "Darwin"). EUPS uses the value
of this variable to select only packages that have be registered as
supporting your platform. If it is not set, EUPS attempts to
auto-detect your platform.
Finally, the EUPS_PKGROOT is used only by eups
distrib to provide a default base URL from which packages can
be downloaded and installed. You do not have to explicitly set this
as it is done automatically when you load the LSST environment.
% echo $EUPS_PKGROOT http://lsstdev.ncsa.uiuc.edu/dmspkgs
The eups list command is useful for seeing what versions
of packages are available. For example,
% eups list activemqcpp 2.1-RC1 Current afw 3.0 Current ap 3.0 Current base 3.1 Current Setup boost 1.36.0 Current cfitsio 3006.2 Current ctrl_events 3.0 Current daf_base 3.0 Current daf_data 3.0 Current daf_persistence 3.1.1 Current detection 3.0 Current doxygen 1.5.4 Current eups LOCAL:/the/LSST/installation/root/eups/1.0.4 Setup ip_diffim 3.0.1 Current lsst 0.3 Current Setup lssteups 0.3 Current Setup minuit 1.7.9 Current mops 3.0 Current mpich2 1.0.5p4 Current mysqlclient 5.0.45+1 Current numpy 1.0.1 Current pex_exceptions 3.0 Current pex_harness 3.0.1 Current pex_logging 3.0 Current pex_policy 3.0.1 Current python 2.5.2 Current Setup scons 1.20 Current Setup sconsDistrib 0.98.5 Current Setup sconsUtils 1.20 Current Setup security 3.0 Current swig 1.3.34 Current swig 1.3.36 tcltk 8.5a4 Current Setup utils 3.0 Current vw 1.0.1+3 Current vwlapack 1.0.0 Current wcslib 4.2-lsst.2 Current xpa 2.1.7b2 Current
The "Current" label in the third column indicates that
that particular version of a package has been flagged as the current
default version. If you do not specify a version to the
setup command, then this
flagged version will be selected. If the third column also includes a
"Setup" label, then that particular version has already
been selected with the setup
command. (If you sourced the loadLSST.*sh start-up
script, then several packages will be pre-selected for you.)
You can restrict the listing output to a specific package by providing
the package's name to eups list. Add a version string,
and the listing will be restricted to that version.
% eups list afw 3.0 Current % eups list afw 3.0 0.2 Current
Add a -v (or --verbose), and the directory
where the package is installed is shown as well.
% eups list fw -v 3.0 /the/LSST/installation/root/Linux /the/LSST/installation/root/Linux/fw/3.0 Current
The first path is the EUPS_PATH directory where the
package was found, and the second path is the directory where that
particular version of the package is installed.
The setup command is used to select a particular version
of a package for use.
% setup afw # selects the version flagged as "Current" % setup afw 3.0 # selects version 3.0
When you select a package withsetup, it also implicitly
selects the other packages the requested package depends on. To see
exactly which packages those are, add a -v (or
--verbose):
% setup -v afw Setting up: afw Flavor: Linux64 Version: 3.0 Setting up: |daf_data Flavor: Linux64 Version: 3.0 Setting up: | daf_persistence Flavor: Linux64 Version: 3.1.1 Setting up: | |pex_policy Flavor: Linux64 Version: 3.0.1 Setting up: | | boost Flavor: Linux64 Version: 1.36.0 Setting up: | | |python Flavor: Linux64 Version: 2.5.2 Setting up: | | pex_exceptions Flavor: Linux64 Version: 3.0 Setting up: | | |pex_logging Flavor: Linux64 Version: 3.0 Setting up: | | | daf_base Flavor: Linux64 Version: 3.0 Setting up: | | | |utils Flavor: Linux64 Version: 3.0 Setting up: | | | | base Flavor: Linux64 Version: 3.1 Setting up: | | | | swig Flavor: Linux64 Version: 1.3.34 Setting up: | | | | doxygen Flavor: Linux64 Version: 1.5.4 Setting up: | |mysqlclient Flavor: Linux64 Version: 5.0.45+1 Setting up: | security Flavor: Linux64 Version: 3.0 Setting up: |vw Flavor: Linux64 Version: 1.0.1+3 Setting up: |cfitsio Flavor: Linux64 Version: 3006.2 Setting up: |wcslib Flavor: Linux64 Version: 4.2-lsst.2 Setting up: |xpa Flavor: Linux64 Version: 2.1.7b2 Setting up: | tcltk Flavor: Linux64 Version: 8.5a4 Setting up: |numpy Flavor: Linux64 Version: 1.0.1 Setting up: |minuit Flavor: Linux64 Version: 1.7.9
The indentation of the package names indicates the hierarchy of
dependencies. In this case, the afw package version 3.0
requires daf_data version 3.0, which in turn requires
daf_persistence 3.0, security 3.0, and,
indirectly python 2.5.2.
When you select a package with setup, environment
variables are updated. First, an environment variable of the form
PRODUCT_DIR is set to the directory where the
selected product is installed.
% echo $AFW_DIR /the/LSST/installation/root/Linux/afw/3.0
Then, depending on what the package provides, other environment variables likePATH, PYTHONPATH, and
LD_LIBRARY_PATH are altered. For example,
fw provides a python module, so setup adds
its python directory to PYTHONPATH:
% echo $PYTHONPATH | sed -e 's/:/\n/g' /the/LSST/installation/root/Linux64/afw/3.0/python /the/LSST/installation/root/Linux64/lssteups/0.3/python /the/LSST/installation/root/Linux64/sconsUtils/1.20/python /the/LSST/installation/root/eups/1.0.4/bin /the/LSST/installation/root/Linux64/base/3.1/python /the/LSST/installation/root/Linux64/utils/3.0/python /the/LSST/installation/root/Linux64/daf_base/3.0/python /the/LSST/installation/root/Linux64/pex_logging/3.0/python /the/LSST/installation/root/Linux64/pex_exceptions/3.0/python /the/LSST/installation/root/Linux64/pex_policy/3.0.1/python /the/LSST/installation/root/Linux64/daf_persistence/3.1.1/python /the/LSST/installation/root/Linux64/security/3.0/python /the/LSST/installation/root/Linux64/daf_data/3.0/python /the/LSST/installation/root/Linux64/external/numpy/1.0.1/lib/python % which python /the/LSST/installation/root/Linux/external/python/2.5/bin/python
setup also made sure we used the right version of
Python by updating PATH. Note that if a different
version of a package had been previously declared, mention of the
previous version are removed from the environment variables
before the new version is added.
unsetup command.
% unsetup afw
Obviously, you do not need to provide the version number when
un-selecting a package. Note that this also un-selects the other
packages it relies on. If you still need any of those dependent
packages, you will need to run setup again for those
specific packages.
When you installed packages with lsstpkg
(Section 2.1), the eups declare
command was run for you. This registered each downloaded package into
an EUPS database. On occasion however, you will have to run
eups declare by hand. For example, you may want to
register a private version of package that you built from the SVN code
tree. An invocation usually takes the following form:
eups declare package_name version_name -r path/to/package
For example,
eups declare fw svn -r /path/to/private/afw
Note that the version_name can be any arbitrary string; it
simply needs to be distinct from other the other registered versions
of the package. The -r tells EUPS what the root
directory for the package is (and thus what it should set, in this
case, the AFW_DIR variable to).
In order to declare a package in this way, its root directory must
contain a ups subdirectory that contains a file of the
form product.table. This table file contains an
enumeration of the other packages that this package requires as well
as how the environment variables should be updated when the package is
selected. In the absence of a ups subdirectory, you can
specify a specific table file with the --table (or
-t) option.
If you want to declare a package as "Current", regardless of whether
it was already registered or not, add the --current
option to the command line:
% eups declare --current fw svn -r /path/to/private/afw
You can also un-declare a package:
% eups undeclare afw svn
Obviously, you do not need to mention the root directory.
SCons is the tool we use to build the
LSST packages. To build a package, you first need to be sure you have
the LSST environment loaded. If you haven't already, load it now:
% source $LSST_HOME/loadLSST.csh # or, 'source $LSST_HOME/loadLSST.sh' for bash
Important:Sourcing this file will automatically setup the development environment package,lsst. If you do not as matter of preference source this file, you need to explicitly setup this package yourself:% setup lsst
One of the things this does is load SCons into your environment.
At this point, you should, if you haven't already, check out (using
svn) the LSST package you want to build;
see section 4.1 for details. Once you have
done this, change into the directory where you have checked out the
package.
We now need to load into our environment all of the other packages that this package depends on. We do this by running a special setup command:
% setup -r .
This reads the EUPS table file (e.g. ups/afw.table) to
figure out what the dependencies and the requisite environment are.
We are now ready to use scons to build the package; to do this, simply
type scons in the top directory of the
package:
% scons
So far, we have just compiled all of the code; we haven't installed
it, yet. And once we have installed, we'll want to declare it to EUPS
so that we can use it. To do this, we pass the "install" and
"declare" targets to the scons command:
% scons install declare
As you might have guessed, we could have just skipped the first
scons invocation and gone straight to
scons install declare.
By default, the package will get installed into our
$LSST_DEVEL area (or if we don't have
one, $LSST_HOME). We can change this
destination passing prefix=dir
as an argument to the scons command.
Our issue tracking system is a web-based product called Trac which can be access via http://dev.lsstcorp.org/trac. It includes two very useful features that help support collaborative development:
The software stack is automatically built and tested using the BuildBot system on dev.lsstcorp.org. To see the current status, see the BuildBot summary page.
editing in progress