Ticket #2506 (closed new functionality: fixed)

Opened 7 years ago

Last modified 7 years ago

lsst_distrib_tool: A temporary mechanism for distributing DM stack binaries

Reported by: mjuric Owned by: robyn
Priority: critical Milestone:
Component: TCT Keywords:
Cc: price, rhl Blocked By:
Blocking: Project: LSST
Version Number:
How to repeat:

not applicable

Description

I've written a tool that makes it easy to install a pre-compiled stack (binaries) onto one's machine. It's a single bash shell script, named lsst-distrib, available from:

http://dev.lsstcorp.org/cgit/contrib/lsst_distrib_tool.git/

lsst-distrib is a simple wrapper around rsync. It detects the OS, checks for prerequisites (RPMs). If the user is running a supported Mac or Linux flavor, and has all prerequisites installed, it rsyncs a pre-built binary distribution to the user's machine. There's really not much more to it. It is a temporary solution until something better comes along (e.g., binary distributions using EUPS).

By default, the script downloads to /opt/lsst (where the binaries have been built). There's an experimental feature to download to an arbitrary directory, but it hasn't been exhaustively tested -- it's not known if all of the stack is relocatable. A test with the Summer 2012 demo worked, though.

With this, a new install should be as easy as:

curl -O https://moya.lsst.majuric.org/lsst-distrib
bash lsst-distrib install

Installing to a non-standard directory:

curl -O https://moya.lsst.majuric.org/lsst-distrib
bash lsst-distrib install /non/standard/directory

Updating an existing installation:

setup lsst_distrib_tools
lsst-distrib update

After some testing and once it passes design review, I'll move the URLs to sw.lsstcorp.org.

Right now, I have pre-built v6_1 binaries for 64bit variants of OSX 10.7, 10.8 and RHEL6 compatibles. RHEL6 build is special in that I'm using system Tk/Tcl? (otherwise, our stack can break a local build of ds9). OSX 10.8 and 10.7 builds are special in that I'm using system numpy (note it's v1.5.1 on 10.7 -- is this version compatible with our stack)?

I'd ask for an expedited testing/comments/design review, so this could be pushed out as soon as appropriate. There is some urgency, as the Camera team has decided to use our stack for sensor I&T pipelines, and initial install has been identified as a significant obstacle to adoption.

Some additional background: RHL, KTL, GPDF and I were at SLAC last week, presenting and teaching the DM stack to the Camera team(s) who need to implement sensor integration and test pipelines. The final outcome was very positive: they've decided to adopt the DM stack as a basis for their software effort. This is extremely good news for the Project overall, both technically and sociologically. I hope it will reduce duplication of effort, increase sharing/understanding of code across subsystem boundaries, make us work closer across subsystem boundaries, etc. It's also a recognition of the excellent job the DM team has done designing and coding up the stack!

There was one significant blemish, though -- the experience of building the stack was very poor. Builds failed more often than not. We wasted a hours debugging various problems with the builds, largely tracing them to idiosyncrasies of peoples' macports/fink/homebrew setups. The experience made it clear to us that we need a better way to get the stack onto end-users' machines, and that we need it quickly to keep the momentum and show we're responsive and truly behind this joint push to use the DM stack across subsystems. Feb'13 review of Camera sensor acceptance criteria, for which they have to write some of their code, added to the urgency. That is how this package got started.

Change History

comment:1 follow-up: ↓ 2 Changed 7 years ago by ktl

Some quick comments:

  • The script mentions running as user root in some error messages. This does not seem to be strictly necessary, and user-space installations are highly desirable.
  • If the detected OS is incorrect, there seems to be no way to override it. I'd suggest that instead of "Detected $OS. Is this correct (y/n)?" the prompt be more like "Install version for which architecture? [detected: $OS]".
  • I'm confused about the "update" mode. First, it should be lsst_distrib_tool without the trailing "s". Second, the tool is not available via eups distrib, so it's a little strange to have it appear in the stack. Third, you're relying on (the older version of) the tool to update itself, rather than updating the tool and then the distribution; this could be problematic if we changed URLs, for example.
  • At 200 or so lines, this is about the largest I'd like us to get with shell scripts. But since this is a bootstrap script, a lowest-common-denominator language (rather than Python) makes sense.

comment:2 in reply to: ↑ 1 Changed 7 years ago by mjuric

Replying to ktl:

Some quick comments:

  • The script mentions running as user root in some error messages. This does not seem to be strictly necessary, and user-space installations are highly desirable.

Fixed. Now it just comments on not having sufficient permissions.

  • If the detected OS is incorrect, there seems to be no way to override it. I'd suggest that instead of "Detected $OS. Is this correct (y/n)?" the prompt be more like "Install version for which architecture? [detected: $OS]".

This one got me thinking that what we really should do is install into /opt/lsst/$OS, rather than /opt/lsst. This would allow support for multi-platform stack setups on shared network disks (e.g., like the one you have at SLAC). I implemented that, in addition to what you requested (and as a consequence, I'm now rebuilding all stacks in their new base directories).

This now allows one to, in principle, install a RHEL6 stack on a Mac, or vice versa. Since in such a case the destination system won't have the necessary prerequisites, I added an environment variable that can be set to disable prerequisites checks (see usage()). This is all considered to be advanced usage.

I also removed the "dest_dir" command line parameter, since it's not proven to work always (I don't want to draw attention to it). It can still be forced using an environment variable, $LSST_HOME.

  • I'm confused about the "update" mode. First, it should be lsst_distrib_tool without the trailing "s". Second, the tool is not available via eups distrib, so it's a little strange to have it appear in the stack. Third, you're relying on (the older version of) the tool to update itself, rather than updating the tool and then the distribution; this could be problematic if we changed URLs, for example.

Fixed the typo in the README. Re. the tool not being available via 'eups distrib', that's just a temporary inconsistency -- I didn't want to unilaterally put it up before the review. Re. self-updates, I think they should be OK. If we change URLs, we'd have to tell people where the new URL is anyway (unless we set up forwarding); I see no good way around it.

  • At 200 or so lines, this is about the largest I'd like us to get with shell scripts. But since this is a bootstrap script, a lowest-common-denominator language (rather than Python) makes sense.

Agreed.

comment:3 Changed 7 years ago by price

  • Cc price, rhl added

comment:4 Changed 7 years ago by robyn

Emailed comments on the Release of this new facility:

From: 	Mike Freemon <mfreemon@illinois.edu>
Date: 	Sat, 15 Dec 2012 07:47:47 -0600

Is this intended to be a temporary solution to address only the short-term needs, or a permanent solution not only for now, but throughout Construction and Operations?
From: 	Mario Juric <mjuric@lsst.org>
Date: 	Sat, 15 Dec 2012 09:28:43 -0700
To: 	Mike Freemon <mfreemon@illinois.edu>

On 12/15/12 6:47 , Mike Freemon wrote:
> Hi Mario,
>
> Is this intended to be a temporary solution to address only the
> short-term needs, or a permanent solution not only for now, but
> throughout Construction and Operations?
>

Definitely a temporary one, just enough to enable others who are
beginning to use our stack (Camera, wavefront sensing, and science
collaborations for now).

comment:5 Changed 7 years ago by mjuric

  • Status changed from new to closed
  • Resolution set to fixed

Following TCT approval to use this method for binary distributions, I've moved the git repository from contrib/ to LSST/DMS.

Note: See TracTickets for help on using tickets.