Last modified 8 years ago Last modified on 07/21/2011 06:15:30 PM

Data Management User Support Plan

This page describes key elements of the LSST-DM user support plan for Data Challenge 3b. Changes to the plan for the Summer, 2011 data release (formerly known as PT1.2) are described first, followed by the baseline plan as it was developed for the PT1.1 release in Jan, 2011.

Changes for the Summer, 2011 Data Release

The following resources and services will be revised or updated by July 30, or whenever this DR production has completed and has been approved for release:

  • The new data products will replace the old data products. That is, data from PT1.1 (images and catalogs) will no longer be provided from the public access points. These products will be retained (probably in the MSS at NCSA) for internal DM team use, but users who wish to access the old data products will need to make a special request via the HelpDesk.
  • Image data will be migrated to TBD low-latency storage hardware. (Latency time-outs have been the source of a sizable fraction of User Support questions).
    • The image data will be provided via a CGI script on NCSA/lsst1, which will use REDDnet/Lstore as a virtual disk cache
      • Fallback: Disk space to be borrowed from IPAC
    • Images may begin migrating from NCSA to REDDnet/LStore after the 1000-series production run data have been reviewed for quality and blessed
    • A VO-compliant SIAP service will be developed for programmatic access to images
      • This will allow programmatic access to the image files
  • We will provide direct access to the Science DB for a few, select power users to support full SQL queries.
    • Users will need to make a special request for a MySQL account, with justification for this level of access.
    • The rule is one DB account per human: no shared accounts.
    • The baseline plan is allow direct access to the DB on hosted by NCSA on lsst10 for this purpose.
      • An alternative to host a replicant of the DB at SLAC appears not to be necessary.
  • The Data Challenge Handbook will need to be updated in several respects:
    • Fix errata that have been identified since the last publication
    • Revise Ch 1 to describe the new nomenclature for data releases, and to update the goals/status for this new data release
    • Revise Ch 2 to reflect changes in the user interfaces (if any)
    • Add to Ch 3: updates/improvements to the ImSims, and add a table of header keywords that are particularly important, with Cliff Notes on the rest of the header contents
    • Update Ch 4 with descriptions of new or revised processing algorithms (not many, I think)
    • Major update to Ch 5 with plots and summary of data quality analysis for this new data release

Baseline Plan


LSST/DM will benefit from involving members of the Science Collaborations in the scientific and technical analysis of the Data Challenge production runs. Begining with DC3b-PT1.1, DM has provided external access to the input and output data products, including raw images, input catalogs and calibration reference files, and processed images and object+source catalogs.

DM provides user help in a few ways, with emphasis on self-help mechanisms. Support from DM staff members will be on a best-effort basis. The goal is to resolve questions submitted through the Help Desk within a few business days.


The resources that are available to users include the following, in roughly the order of expected use:

All relevant web pages will include links to the DM Help Desk.

Support Processes

The primary support mechanism is the web-based access to the catalogs & data, followed by documentation (DCH, Schema page), followed by the Science User Forum & Wiki, and finally the DM Help Desk. For the latter, the process is this:

  • Help Desk Web page will include an e-mail address for user inquiries
  • Help Desk e-mail exploder (dc-help @ will forward all questions to the following list of people to avoid single-point failures. Only one of the following will be designated "primary" at a time:
    • Dick Shaw
    • Robyn Allsman
    • Jeff Kantor
    • Tim Axelrod
    • Gregory Dubois-Felsmann
  • Primary HD person will either resolve the question, forward it to the User Forum, or will forward it to any appropriate member of the DM team
  • All e-mail correspondence will be archived
  • Developers will file tickets, if appropriate to the resolution
  • DCH editors will incorporate revisions to fix errors/omissions if identified
  • Primary HD will monitor progress, will summarize resolution for FAQ if appropriate

Unresolved Questions

  • We should consider whether to use third-party HelpDesk software once user traffic increases significantly. One possibility is JIRA, but we would need to identify a machine and someone to set up the software.