Last modified 6 years ago Last modified on 10/02/2012 03:54:33 PM

Proposal to TCT: Third-Party Package Upgrades in W13

All of the packages below have been tested on Ubuntu 10.04 with the stack up to pipe_tasks. I don't anticipate we'll have any trouble with other operating systems or the rest of the stack (though this doesn't include MacOS 10.8, on which I don't think we've tested any of the old third-party packages either).

Clarification for Preapproved Updates

We need to state which version number we consider "safe" to upgrade. Usually this is just the third (bugfix) number, but for some packages we may want to preapprove minor release upgrades for packages that take care to make these backward-compatible, and also don't provide bugfix updates for older minor releases.

Here's what I recommend; in parenthesis is (<current> -> <proposed>):

  • sqlite: bugfix updates: 3.7.x (3.7.8 -> 3.7.14)
  • python: bugfix updates: 2.7.x (2.7.2 -> 2.7.3)
  • doxygen: minor updates: 1.x ( -> 1.8.2)
  • eigen: bugfix updates: 3.1.x (3.0.2 -> 3.1.1)
    • I'd like to request updating from 3.0.x to 3.1.x now, but I'm not proposing to preapprove going to 3.2.x
  • numpy: minor updates: 1.x (1.6.1 -> 1.6.2)
    • No new minor update is available now (1.7 has not been released).
  • xpa: bugfix updates: 2.1.x (2.1.13 -> 2.1.14)
  • wcslib: minor updates: 4.x (4.4.4 -> 4.14)
  • cfitsio: minor updates: 3.x (3.29 -> 3.31)
    • Need to check for bugs in the latest version; some problems reported by RHL.
  • pyfits: minor updates: 3.x (2.4.0 -> 3.1)
    • I'd like to request updating from 2.4.0 to 3.x now, but I'm not proposing to preapprove going to 4.x

Additional Preapproved Updates

I think a few more packages should be considered safe to preapprove for upgrade. Here's the list:

  • tcltk: bugfix updates: 8.5.x (8.5.9 -> 8.5.12)
  • mysqlclient: bugfix updates: 5.1.x (5.1.50 -> 5.1.65)
  • matplotlib: minor updates: 1.1.x (1.1.0 -> 1.1.1)
    • No new minor update is available now.
  • fftw: bugfix updates: 3.3.x (3.3 -> 3.3.2)
  • boost: bugfix updates: 1.51.x
    • I'd like to request updating from 1.47.0 to 1.51.0 now, but I'm not proposing to preapprove going to 1.x
  • scisql: minor updates: 0.x (0.1 -> 0.3)
    • While this is a third-party package, it's developed by LSST developers, so I think these upgrades should generally be both safe and desirable for us.

Deprecate pysqlite

I now have a Python install that depends sqlite and builds the sqlite3 module that's included with Python, which makes pysqlite unnecessary (#2212). We won't be able to remove pysqlite without making some trivial changes to our code, but we can have both pysqlite and sqlite3 present at the same time until those changes are done. It should be mostly just changing the import statements. I think we should be able to remove pysqlite entirely in S13.

New Packages (?)

I'd like to make ipython and scipy available as distrib packages and install them at NCSA, but I don't think any of our packages should start depending on them, and I don't intend to support them working on every platform (though I suspect IPython will just work on most platforms). Do we need to have a "contrib" section of the distrib server for this? Or is adding third-party packages there okay, as long as I don't attach any offical server tags to them?

Other Related Notes

(for posterity, not TCT votes)

  • swig 2.0.8 does *not* work with the stack; meas_astrom fails to build. A quick google search suggests a parser regression introduced in 2.0.5 is at fault, but this was supposed to have been fixed in 2.0.7. I was not able to reproduce this with a minimal test case, so I haven't reported it upstream.
  • mysqlclient 5.5.27 is also available, but the build system has switched from autoconf to cmake, meaning our current build file doesn't work. It also still doesn't build with clang, and our patch for 5.1.x doesn't work with it.


Kian-Tat Lim Wed, 26 Sep 2012 13:13:19 -0700

Overall, I'm voting to approve Jim's requests. #2212 needs to be closed out before pysqlite is deprecated. I'd like to have "distributing unofficial third-party packages" (and incorporating them into the common stack) as a requirement for the new distrib server system but not implement it for iPython or scipy until that new system is available.

Russell Owen 2102-10-2 3:30 Pacific

I agree with K-T. Note: matplotlib 1.2 is in release candidate status but I'd rather not wait for it, so I agree with Jim's recommendation.

One suggested supplement to Jim's proposal: ask NCSA to install git, the current stable version. The lsstX machines at NCSA have git version 1.7.1. I have used 1.7.12 with no problems.

Mario Juric 2102-10-2 16:00 Pacific

I vote a "conditional yes", with the condition that these packages be verified to build on OS X (10.7 or 10.8). Though it's not an officially supported platform (and we should consider making it one), so many developers (and outside users) work on a Mac that we should not break Mac compatibility unless absolutely necessary.