wiki:DC3Management

Version 36 (modified by dgehrig, 10 years ago) (diff)

--

Data Challenge 3 Management

from: LSST Wiki Table of Contents.

Data Challenge 3 will focus primarily on the processing at the Archive Center that generates a Data Release.

This page collects documents related to the management and execution of Data Challenge 3.

DC3 Startup Tasks

DC3 Planning

Development in DC3

  1. Developing with Virtual Machines
  2. Using the Pipeline Harness
  3. Configuring Predefined Stage Classes
  4. Trac Ticket Workflow tuned for Data Challenge 3
  5. DC3 Software Tour

Pre- DC3a Integration Week

  • Developers:
    • resolve all your DC3a delivery Tickets; merge Ticket branches onto trunk; tag packages
  • Pipeline Gurus:
    • create (and svn archive) an integration test for your pipeline which
      • includes all its stages; and
      • uses the pipeline harness running single slice on a single node.
    • test that pipeline either on your local development system or the LSST cluster at NCSA.
    • TBD expanded by Ray...check in your pipeline policies in dc3pipe .......

DC3a Benchmarking

  • TBD: Setting up the Run directory (out-of-date ref SetUpRun by Gehrig)
  • Running the Pipeline Harness (current by Daues)
  • TBD: Accessing the LSST Cluster (out-of-date ref LSSTCluster by Gehrig)
  • TBD: How to Run the Complete Set of DC3a Pipelines through ORCA (out-of-date ref DC2PipeHowto)
  • TBD: DC3a Test Data (out-of-date ref DC2TestData)
    • Exposures and ancillary Calibration Artifacts
    • Catalogs
    • NightMOPS proprietary artifacts located in lsst9:/lsst/DC3/opt. See directory's README.

Meetings

Reference Documents

DC3 Software at ROE