wiki:Installing/Summer2013
Last modified 5 years ago Last modified on 03/11/2014 03:41:13 PM

Installing LSST Software Stack, Summer 2013 Release

Before you begin

Should you even begin?

Getting help

Installing

If your platform is supported, we strongly recommend you choose to install precompiled binaries.

Prerequisites and Supported Platforms

OS Binary Available Prerequisites Notes
RHEL 6 and derivatives yes gcc builds: atlas blas bison bzip2-devel flex freetype-devel gcc-c++ gcc-gfortran libuuid-devel libXt-devel make ncurses-devel openssl-devel perl readline-devel zlib-devel 
clang builds: clang 3.0
Officially supported platform
Mac OS X 10.7 (Lion) and 10.8 (Mountain Lion) yes If building from source: XCode 4.3, Command-line tools (use Download prefs pane in Xcode), gfortran for XCode 4.3 Frequently used platform
RHEL 5 and derivatives yes bison bzip2-devel e2fsprogs-devel flex freetype-devel gcc44-c++ gcc44-gfortran perl gcc-c++ libpng-devel libX11-devel libXt-devel libstdc++44-devel make openssl-devel readline-devel which zlib-devel  
Ubuntu 12.04 yes curl dpkg-dev gfortran libbz2-dev libfreetype6-dev libreadline6-dev libssl-dev libx11-dev libxaw7-dev texlive-latex-base zlib1g-dev 
Ubuntu 10.04 no Minimum:  bison curl flex g++ gfortran graphviz libbz2-dev libreadline-dev libncurses5-dev libxt-dev make perl uuid-dev zlib1g-dev 
Development:  autoconf automake bison flex g++ gfortran git-core libncurses5-dev libreadline5-dev libtool libx11-dev libuuid1 latex2html libxaw7-dev zlib1g-dev libbz2-dev libssl-dev m4 make patch uuid-dev 

Binary Installs

Binaries are installed using the lsst-distrib script which will autodetect your platform and download the correct binaries.

The binaries will be downloaded to /opt/lsst/$PLATFORM (e.g., /opt/lsst/rhel-6-x86_64). You can download to a different directory by setting the environment variable LSST_HOME=/directory/to/download. Note however that this feature is highly experimental.

Installing on RHEL 5, RHEL 6 and compatibles

curl -O http://lsst-web.ncsa.illinois.edu/~lsstsw/lsst-distrib
su -c "bash lsst-distrib install"

To see all available install options, run: bash lsst-distrib -h.

Installing on Ubuntu 12.04 LTS, OS X 10.7 (Lion), and OSX 10.8 (Mountain Lion)

curl -O http://lsst-web.ncsa.illinois.edu/~lsstsw/lsst-distrib
sudo bash lsst-distrib install

To see all available install options, run: bash lsst-distrib -h.

Other platforms

More platforms may be added in the future, depending demand and community contributions. In the meantime, please install from source.

Building From Source

RHEL 6 (or derivatives)

# bash users:
cd root/directory/where/lsst/stack/will/be/installed (e.g., ~/lsst)

unset LSST_HOME EUPS_PATH LSST_DEVEL

export NCORES=$((sysctl -n hw.ncpu || (test -r /proc/cpuinfo && grep processor /proc/cpuinfo | wc -l) || echo 2) 2>/dev/null)
export MAKEFLAGS="-j $NCORES"
export SCONSFLAGS="-j $NCORES"

curl -O http://sw.lsstcorp.org/pkgs/std/w12/newinstall.sh
bash newinstall.sh

source loadLSST.sh
eups distrib install --nolocks -t Summer2013 lsst_distrib

RHEL 5 (or derivatives)

# bash users:
cd root/directory/where/lsst/stack/will/be/installed (e.g., ~/lsst)

unset LSST_HOME EUPS_PATH LSST_DEVEL

export NCORES=$((sysctl -n hw.ncpu || (test -r /proc/cpuinfo && grep processor /proc/cpuinfo | wc -l) || echo 2) 2>/dev/null)
export MAKEFLAGS="-j $NCORES"
export SCONSFLAGS="-j $NCORES"

curl http://sw.lsstcorp.org/pkgs/std/w12/newinstall.sh -o newinstall.sh
bash newinstall.sh
source loadLSST.sh

eups distrib install rhel5_gcc44 4.4
setup rhel5_gcc44

eups distrib install --nolocks -t Summer2013 lsst_distrib

Ubuntu 12.04 LTS and similar

# bash users:
cd root/directory/where/lsst/stack/will/be/installed (e.g., ~/lsst)

unset LSST_HOME EUPS_PATH LSST_DEVEL

export NCORES=$((sysctl -n hw.ncpu || (test -r /proc/cpuinfo && grep processor /proc/cpuinfo | wc -l) || echo 2) 2>/dev/null)
export MAKEFLAGS="-j $NCORES"
export SCONSFLAGS="-j $NCORES"

# Additional environment variables following toolchain changes (https://wiki.ubuntu.com/NattyNarwhal/ToolchainTransition)
export LDFLAGS+=" -Wl,--no-as-needed"
SCONSFLAGS+=" LINKFLAGS='-Wl,--no-as-needed' --setenv"

curl -O http://sw.lsstcorp.org/pkgs/std/w12/newinstall.sh
bash newinstall.sh

source loadLSST.sh
eups distrib install --nolocks -t Summer2013 lsst_distrib

Mac OS X 10.7 (Lion) and 10.8 (Mountain Lion)

Verify that you have gfortran linked in /usr/bin/gfortran. Otherwise, run:

# make a symlink for gfortran:
sudo ln -s /usr/bin/gfortran-4.2 /usr/bin/gfortran

To install:

# bash users:
cd root/directory/where/lsst/stack/will/be/installed (e.g., ~/lsst)

unset LSST_HOME EUPS_PATH LSST_DEVEL

export LANG=C
export CC=clang
export CXX=clang++

export NCORES=$((sysctl -n hw.ncpu || (test -r /proc/cpuinfo && grep processor /proc/cpuinfo | wc -l) || echo 2) 2>/dev/null)
export MAKEFLAGS="-j $NCORES"
export SCONSFLAGS="-j $NCORES cc=clang"

curl -O http://sw.lsstcorp.org/pkgs/std/w12/newinstall.sh
bash newinstall.sh

source loadLSST.sh
eups distrib install --nolocks -t Summer2013 lsst_distrib

Linux using the clang compiler

# bash users:
cd root/directory/where/lsst/stack/will/be/installed (e.g., ~/lsst)

unset LSST_HOME EUPS_PATH LSST_DEVEL

export LANG=C
export CC=clang
export CXX=clang++

export NCORES=$((sysctl -n hw.ncpu || (test -r /proc/cpuinfo && grep processor /proc/cpuinfo | wc -l) || echo 2) 2>/dev/null)
export MAKEFLAGS="-j $NCORES"
export SCONSFLAGS="-j $NCORES cc=clang"

curl -O http://sw.lsstcorp.org/pkgs/std/w12/newinstall.sh
bash newinstall.sh

source loadLSST.sh
eups distrib install --nolocks -t Summer2013 lsst_distrib

Working with the Stack

To be able to execute DM stack commands and load its components, make sure to always set up the environment first:

# for bash users
source $LSST_HOME/loadLSST.sh

or

# for csh users
source $LSST_HOME/loadLSST.csh

where $LSST_HOME is the directory where you've installed the stack.

(Note: there's no need to set the $LSST_HOME environment variable yourself; loadLSST.sh will do it for you).

Running a Demo

We provide a simple demonstration of using the LSST stack to detect sources in a simulated LSST image (a single chip):

curl -O https://dev.lsstcorp.org/cgit/contrib/demos/lsst_dm_stack_demo.git/snapshot/lsst_dm_stack_demo-7.2.0.0.tar.gz

tar xzf lsst_dm_stack_demo-7.2.0.0.tar.gz
cd lsst_dm_stack_demo-7.2.0.0

setup obs_sdss -t Summer2013
./bin/demo.sh

(note: this is a ~210MB download). Look into the README file for more information.

On the NCSA lsst* machines this repository is not yet available as /lsst3/lsst_dm_stack_demo-Summer2013

Known Issues

EUPS Locking

Some users have encountered difficulties with the more aggressive locking in the new EUPS. If a stack will be used by multiple users, you should at least modify the EUPS lock directory to point to a location writable by all users. This can be accomplished by adding the following to $LSST_HOME/site/startup.py:

hooks.config.site.lockDirectoryBase = "/tmp"

If you still encounter problems, you can disable locking entirely by instead using:

hooks.config.site.lockDirectoryBase = None

Building the stack takes a long time

The current build system defaults to using a single core when building the stack. This behavior can be overridden using the MAKEFLAGS and SCONSFLAGS environment variables:

#
# THIS SNIPPET ASSUMES YOU ARE IN THE DIRECTORY WHERE YOU WISH TO INSTALL THE STACK
#

# Find out the number of CPUs to speed up builds via SCONSFLAGS and MAKEFLAGS (this works on Linux and Mac)
export NCORES=${NCORES:-$(sysctl -n hw.ncpu 2>/dev/null || (test -r /proc/cpuinfo && grep processor /proc/cpuinfo | wc -l) 2>/dev/null || echo 2)}

# Set up the basic environment to ~/lsst
curl http://sw.lsstcorp.org/pkgs/std/w12/newinstall.sh -o newinstall.sh
MAKEFLAGS="-j $NCORES" bash newinstall.sh

# Install Summer2013 stack
# Install astrometry_net and wcslib separately because of -jN bugs (e.g. #1970 and related)
source loadLSST.sh
MAKEFLAGS="-j $NCORES" SCONSFLAGS="-j $NCORES" eups distrib install --nolocks -t Summer2013 lsst_distrib

Numpy may fail to build

When you are installing "pipe_tasks" the installation will sometimes fail on numpy. The workaround is to install numpy, then resume your installation of lsst_distrib. For example:

  • eups distrib install -t Summer2013 numpy
  • eups distrib install -t Summer2013 lsst_distrib

The underlying problem is that LD_LIBRARY_PATH is set as part of installing lsst_distrib, and this can confuse the numpy installer.

Further Reading