wiki:DC3/DC3bPT1IntegrationTestPlan
Last modified 9 years ago Last modified on 09/24/2010 08:16:17 PM

DRAFT Last revised 24 Sep 2010

DC3b PT1 Subsystem Integration Test (IT) Plan

1. Test Plan Identification

  • Baseline Identifier
    • DM DC3b PT1
  • Testing Level
    • DM Subsystem Integration

2. References

3. Use Case and Activity Implementations being Validated

A visual representation of the Data Release (DR) activities is available in DC3b Processing Flow; activities being tested are highlighted in green. For an overview of the pipelines, and the stages implementing them, refer to DC3b PT1: Processing Pipelines and Stages. Detailed documentation on those activities is available at http://www.lsstcorp.org/EADC3Model/DC3DRProduction/index.htm.

The Middleware "Run Pipeline" use case is being tested; documentation is available at XXXXXXXXXXXXXXXXX*.

4. Features being tested

The features being tested for DC3b PT1 are enumerated at: http://dev.lsstcorp.org/trac/wiki/DC3bPT1_1#Goals.

5. Approach

5.1 Testing Strategy

Refer to SDP: Verification and Validation, 6.1-4.3 DM Integration Test for the sequenced process by which the testing and validation will occur.

DC3b Integration and Test will iteratively integrate the component pipelines using scenarios reflecting the basic course scenario for use cases and the non-exception scenarios for activity flows. The resulting composite component will embody the validated Data Release Production.

5.2 Test Types

Subsystem Integration tests verify that the major software components work correctly with the rest of the system, and as specified in the architectural design. The tests are described in SDP: Verification and Validation, 4.3 Integration Test.

5.3 Overall Test Sequencing

5.3.1 Middleware Integration and Test Sequence

The Middleware components' integration and tests should all occur prior to the Applications components' integration and tests. Due to dependencies, the sequence of the integration testing should follow the order listed below. Test harnesses for the Middleware integration tests which create the appropriate external operating environment, will be used.

  • Persist Data from Pipeline
  • Prepare Data Access for Pipeline
  • Stage Input Data
  • Record Pipeline Provenance
  • Event Handling
  • Run Event Monitor
  • Run a Pipeline
    • An accurately deployed pipeline (ie stages, events & communications framework) but using dummy stages
    • An accurately deployed pipeline using stages implementing algorithms

5.3.2 Applications Integration and Test Sequence

Each pipeline will have a test harness which creates the appropriate external operating environment necessary for testing the pipeline's progress through its stages. This allows pipeline testing and validation in isolation.

When defining the composition in the pipeline realization of the EA package: Data Release Production, some conceptually disjoint pipelines will be aggregated in order to optimize computational and I/O throughput.

The following re-grouping of the pipelines and stages defined in the DR Production's EA package were selected:

  • realization of Image Processing Pipeline
    • stages comprising EA package: Instrument Signature Removal (ISR) Pipeline
    • stages comprising EA package: Ccd Assembly (CA) Pipeline
    • stages comprising EA package: Cosmic Ray Split (CrSplit?) Pipeline
    • stages comprising EA package: Image Characterization (ImgChar?) Pipeline
    • stages comprising EA package: Single Frame Measurement (SFM) Pipeline
  • realization of Source Association Pipeline
    • stages comprising EA Package: Source Association (SourceAssoc?) Pipeline

The realizations will first be validated individually and finally as the composite Data Release Production.

5.4 Test Coverage Metric

A minimum test coverage metric is not defined for DC3.

6. Test Criteria

Refer to SDP: VerificationAndValidation: 6.1-4.3 DM Integration Test.

7. Test Deliverables

Refer to:

8. Testing Tasks

  • identify the set of tasks necessary to prepare for and perform testing;
  • identify all inter-task dependencies.

9. Environmental Needs

See DC3b Hardware Requirements] for the hardware and operating system specifications for the systems to be used for DC3b benchmarking runs. Third party software requirements are maintained within the LSST build manifest.

10. Schedule

See DC3Schedule for the project schedule and milestones.

11. Planning Risks and Contingencies

DC3b PT1 Data Release Production is testing

  • algorithmic development for image characterization and single frame photometric measurements; and
  • enhancements to the middleware supporting distributed parallel computing.

11.1 Contingency: Simulated LSST Exposure Calibration Dataset

The DC3b PT1 benchmark' Instrument Signature Removal (ISR) pipeline requires not only the use of the DC3a simulated LSST exposures (ImSim) but also use of suitably matched and constructed calibration artifacts.

Should the ImSim dataset not include the calibrated artifacts, the CFHT-LS dataset will be used as the alternate dataset.

11.2 Contingency: Astrometric and Photometric Calibration Catalogs

DC3b PT1 requires the use of astrometric and photometric catalogs for Image Characterization pipeline for both the ImSim and the CFHT-LS dataset.

Should the astrometric and/or photometric catalogs for the simulated images not be available, the use of simulated images will be deferred until DC3b PT2.

Should the astrometric and/or photometric catalogs for the CFHT-LS images not be available, the hand constructed calibration tables used by DC3a will be used.

11.3 Contingency: Paired Exposures

CFHT-LS does not have paired exposures. LSST simulated images have been constructed with paired exposures per visit.

Should paired exposures be unavailable for the selected dataset, the Cosmic Ray Split (CrSplit?) pipeline will be bypassed and only a single exposure will be processed through the Data Release input stream.

11.4 Contigency: CCD Assembly Pipeline

Should the new CCD Assembly Pipeline be unavailable, the CCD assembly technique used for DC3a will be used instead.

11.5 Contingency: Image Characterization Pipeline

The new Image Characterization (ImgChar?) Pipeline is an enhancement of the DC3a WCS pipeline. Should the new ImgChar? pipeline be unavailable, the DC3a WCS pipeline will be used instead.

11.6 Contingency: Middleware Pipeline Orchestration

The new middleware pipeline orchestration layer is an enhancement of the DC3a orchestration model. Should the new ImgChar? pipeline be unavailable, the DC3a WCS pipeline will be used instead.

12. Test Case Specification

Test specifications will be generated using the EA LsstTool capability.

Appendix

Simulated Input Data

Refer to DC3b Simulated Input Data for specifications of the Image Simulation (ImSim) input prepared for DC3b. DC3b PT1 will use a subset of the full ImSim dataset.

CFHT Input Data

Refer to DC3b CFHT-LS Input Data for specification of the CFHT-LS data to be used in DC3b. DC3b PT1 will use only the CFHT-LS Deep fields.