wiki:DM/SAT/DR-planning-DC3b
Last modified 8 years ago Last modified on 01/21/2011 05:02:20 PM

DC3b PT1.2 Reviews

  • afw changes proposed as a result of Multifit design changes: Monday 1/24, 9:00-11:00 PST (notes)
    • Material: (include links to supporting material here)

DC3b Design Reviews - planning

The following areas of DC3b have been proposed for design reviews. The reviews are intended to cover both the scientific or computational intent (what the code is meant to accomplish) and its design and implementation as computer code.

Note that the focus this time around is meant to be sharply on DC3b and not on longer-term issues.

The anticipated "presenter(s)" of the design in each case is indicated with a "p:"; these names are in the process of being supplied by the team leads. The reviews will be carried out by Gregory and a yet-to-be-determined rotation of 2-3 other appropriate members of the DM team.

Applications

  • Coadds (see the long recent thread in LSST-data for the many issues that need to be covered), and, in particular, template generation, warping, and downsampling (p: Russell + ?)
  • Deep detection (p: Robert, possibly to be delegated)
  • Multifit (review already done, but needs to be documented with actions) (p: Jim and Martin)
  • Masking of moving objects in deep coadds and inputs to Multifit (unclear this is in scope of DC3b)
  • Difference imaging and PSF modeling and determination (p: Andy B)
  • Stellar photometry (p: Robert, possibly to be delegated), global photometric calibration (p: Tim)
  • Astrometric calibration (p: Dave Monet and/or Robert)
  • DayMOPS (p: Jon, assisted by Lynne)
  • meas tree design and implementation (p: Robert, Tim (will post concerns))
  • Camera classes (review already done, but needs to be documented with actions); note that some of the content of this design also needs to be recast as an ICD, at the request of the Systems Engineer (p: Simon, need update from Robert)
  • Database schema (p: Jacek) and planned access tools for DC3b (p: Daniel)
  • DC3b and simulation validation (p: Deborah and ???) -- joint review with ImSim team

Middleware and Infrastructure

  • Interslice communication (p: Greg)
  • Updates to persistence (p: KTL)
  • Pipeline framework & development environment issues (main focus: "framework lite" and related tools) (p: Ray)
  • File and image access (p: Ray)
  • Pipeline orchestration: are all the features available that are needed by the pipeline skeleton design as reviewed? In the Condor environment? (p: Steve Pietrowicz (general orca design), Ray (application to DC3b))
    • Note also that there is a TCT item here: approval of DAGMan as a third-party package.

Pending Topics - some may lead to reviews, others to design discussions; the SAT needs to decide which

  • Pixel coordinate system issues (raised by Russell)
  • Metadata (raised by Tim and several others) - need to follow up on DC3b planning meeting
  • Time representation issues: timebase (e.g., TAI), in-memory and schema formats (raised by Jacek)
    • Note that this overlaps with the remit of the "time committee" chaired by Chuck Claver
  • Units of measurement (e.g., decimal degrees vs. arcmin) (raised by Jacek)
  • Moments and covariance matrices (raised by Jacek)
  • (possibly) Tools for performance monitoring and tuning (e.g., profilers) -- assess need after PT1

Scheduling and Logistics

These reviews are anticipated to start in late January and continue through February. They need to conclude in time to have an effect on the remainder of DC3b code development.

We are currently trying to decide on possible "chunking" or refactoring of the above topics, which do contain some overlaps. We are also looking at the spectrum of choices between holding a series of 2-3 hour teleconference reviews (audio plus GoToMeeting) versus a small number of all-day in-person meetings, which could be conducted either at a central location or at a site associated with one or more review topics.

Sequencing

(The schedule is currently being organized, attempting to prioritize areas that are either a) ready for review or b) prerequisites for other work. Watch this space for updates.)

Done:

  • orca design for orchestration requirements of DC3b: *rescheduled* to Wednesday 2/24, 11:00-13:00 PST (notes)

Remaining:

  • DayMOPS: Thursday 3/11, 14:00-16:00 PST (notes)

Expectations

Designs to be reviewed may be presented as UML, Trac pages plus proposed headers and other code, or slides. The intent is that following the review the designs, with any agreed changes, will be captured in the DC3b UML model.

The presentations should cover (list taken from Jeff's lsst-data e-mail of 2009.1.15):

  • The identification of the components under review in terms of where they fit within the overall architecture (e.g. a stage or algorithm within a stage, an interface to the outside world, a common service that many things use, etc.)
  • use cases and requirements applicable to the components under review that show how they will be used and how they respond/support all usage (robustness diagrams are useful in this context)
  • an API or other description of the public interfaces to the components under review
  • a description of the internal patterns and algorithms to be used in the design, known limitations to those, and justification why the limitations are okay for this development
  • a description of the technological approach to implementation (e.g. this will be a python only or python/C++ stage implementation) and reuse of existing elements (e.g. this will be a specialization of the XYZ framework classes)
  • a description of how the function and performance of the component(s) under review will be tested