wiki:DC2PipelineInterfaces
Last modified 11 years ago Last modified on 01/28/2008 10:20:24 AM

DC2 Pipeline Interfaces

from: DC2 Management -> Component Design and Implementation Documentation-> DC2PipelineInterfaces.

This page describes the expected inputs and outputs of stages in the DC2 pipeline, including both Clipboards and Events.

lsst.imageproc.pipeline.ImageSubtractStage

Note: the image subtraction stage will accept two float or two double precision exposures and will output a difference exposure of the same type.

Inputs

  • triggerVisitEvent: This event triggers pipeline execution.
    • exposureId: id of the exposure. At most 32 bits.
    • visitId: id of the visit. Same as exposureId for DC2.
    • FOVRA: right ascension of the exposure, in degrees double.
    • FOVDec: declination of the exposure, in degrees double.
    • filterName: filter used in exposure; in DC2 this is "i" or "r"
    • visitTime: time of observation; MJD double.
    • dateObs: time/date of observation; string
    • airMass: air mass; double.
    • exposureTime: duration of exposure, seconds (?); double
    • equinox: equinox; double
    • Used by the input and output stages of the image subtraction pipeline; ignored by the image subtraction stage.
  • ScienceExposure: The exposure to be processed.
    • Type/PythonType?: Exposure{D|F}/lsst.fw.Core.fwLib.Exposure{D|F}
    • For DC2 this will be retrieved from FitsStorage. The Location will be "/share/DC2root/%{runId}/ipd/input/%{exposureId}/%{ccdId}/%{exposureId}p_%{ccdId}".
  • TemplateExposure: The template corresponding to the science exposure.
    • Type/PythonType?: Exposure{D|F}/lsst.fw.Core.fwLib.Exposure{D|F}
    • For DC2 this will be retrieved from FitsStorage. The Location will be "/share/DC2root/%{runId}/input/%{exposureId}/%{ccdId}/%{exposureId}p_%{ccdId}_tmpl".
  • PsfMatchBasisKernelSet: (optional) the basis kernels to use to create the psf-matching kernel for each footprint
    • Type/PythonType?: Not currently Persistable; lsst.fw.Core.fwLib.vectorKernelDPtr

Output

  • DifferenceExposure: The difference image.
    • Type/PythonType?: Exposure{D|F}/lsst.fw.Core.fwLib.Exposure{D|F}
    • Normally persisted to FitsStorage in addition to passing to DetectionStage. The Location will be "/share/DC2root/%{runId}/output/%{exposureId}/%{ccdId}/%{exposureId}_%{ccdId}_diff".
  • PsfMatchKernel: The spatially varying psf-matching kernel used to subtract the images.
    • Type/PythonType?: Will be made Persistable; lsst.fw.Core.fwLib.LinearCombinationKernelDPtr
  • BackgroundModel: A function describing the background of the science image.

lsst.detection.pipeline.DetectionStage

Inputs

  • DifferenceExposure: The difference image.
    • Type/PythonType?: ExposureF/lsst.fw.Core.fwLib.ExposureF
    • Normally passed from ImageSubtractStage.

Outputs

  • DiaSources: The list of difference sources. Ids are provided by the OutputStage.
    • Type/PythonType?: DiaSourceVector/lsst.fw.Core.fwCatalog.DiaSourceVec
    • Normally persisted to DbTsvStorage.
    • sliceId, visitId, and exposureId must be passed in the additionalData for the OutputStage. This can be handled by adding the following to the OutputStage policy:
      AdditionalData: "exposureId=triggerVisitEvent.exposureId"  "visitId=triggerVisitEvent.visitId"
      
    • The following sub-policy should be included in the OutputStage policy to properly configure the database table schema:
      Persistence: {
           Formatter: {
               DiaSourceVector: {
                   DiaSources: {
                       templateTableName: "DIASourceTemplate"
                   }
               }
           }
      }
      

Moving Object Pipeline

Inputs

  • mops1Event: This event triggers pipeline execution.
    • visitId: As above.
    • exposureId: for DC2, same as visitId
    • FOVRA: Right ascension of center of telescope FOV.
    • FOVDec: Declination of center of telescope FOV.
    • visitTime: Expected UTC of beginning of observation.
    • filterName: As above
    • dateObs: time/date of observation; string
    • airMass: air mass; double.
    • exposureTime: duration of exposure, seconds (?); double
    • equinox: equinox; double

Outputs

  • MopsPreds: The list of moving object predictions.
    • Type/PythonType?: MovingObjectPredictionVector/lsst.fw.Core.fwCatalog.MopsPredVec
    • Normally persisted to DbTsvStorage.
    • visitId must be passed in the additionalData for the OutputStage. This can be handled by adding the following to the OutputStage policy:
      AdditionalData: "visitId=mops1Event.visitId"
      

Association Pipeline

Inputs

  • triggerAssociationEvent: This event triggers pipeline execution after the detection pipeline has executed.
    • visitId: As above.
    • visitTime: time of observation; MJD double.
    • FOVRA: As above (degrees).
    • FOVDec: As above (degrees).
    • filterName: filter name for the visit.
    • dateObs: time/date of observation; string
    • airMass: air mass; double.
    • exposureTime: duration of exposure, seconds (?); double
    • equinox: equinox; double
    • matchRadius: [optional] can be used to override the default difference source to object match radius (arc-seconds)
  • DiaSources: The list of difference sources from the detection pipeline.
    • Type/PythonType?: DiaSourceVector/lsst.fw.Core.fwCatalog.DiaSourceVec
    • Normally retrieved from DbTsvStorage.
    • visitId must be passed in the additionalData for the InputStage. This can be handled by adding the following to the InputStage policy:
      AdditionalData: "visitId=triggerAssociationEvent.visitId"
      
  • triggerMatchMopsPredsEvent: This event triggers pipeline continuation after the moving object pipeline has executed.
    • visitId: As above, must be identical to triggerAssociationEvent.visitId.
  • MopsPreds: The list of moving object predictions from the moving object pipeline.
    • Type/PythonType?: MovingObjectPredictionVector/lsst.fw.Core.fwCatalog.MopsPredVec
    • Normally retrieved from DbTsvStorage.
    • visitId must be passed in the additionalData for the InputStage. This can be handled by adding the following to the InputStage policy:
      AdditionalData: "visitId=triggerAssociationEvent.visitId"
      

Outputs

  • DiaSourceToObjectMatches: The list of matched pairs of difference sources and objects.
    • Type/PythonType?: MatchPairVector/lsst.ap.interface.MatchPairVec
    • Normally persisted to DbTsvStorage.
    • visitId must be passed in the additionalData for the OutputStage. This can be handled by adding the following to the OutputStage policy:
      AdditionalData: "visitId=triggerAssociationEvent.visitId"
      
    • The following sub-policy should be included in the OutputStage policy to properly configure the database table schema:
      Persistence: {
           Formatter: {
               MatchPairVector: {
                   DiaSourceToObjectMatches: {
                       templateTableName: "MatchPair"
                   }
               }
           }
      }
      
  • MopsPredToDiaSourceMatches: The list of matched pairs of moving object predictions and difference sources.
    • Type/PythonType?: MatchPairVector/lsst.ap.interface.MatchPairVec
    • Normally persisted to DbTsvStorage.
    • visitId must be passed in the additionalData for the OutputStage. This can be handled by adding the following to the OutputStage policy:
      AdditionalData: "visitId=triggerAssociationEvent.visitId"
      
    • The following sub-policy should be included in the OutputStage policy to properly configure the database table schema:
      Persistence: {
           Formatter: {
               MatchPairVector: {
                   MopsPredToDiaSourceMatches: {
                       templateTableName: "MatchPair"
                   }
               }
           }
      }
      
  • NewObjectIdPairs: The list of new objects, unmatched with existing objects or moving objects.
    • Type/PythonType?: IdPairVector/lsst.ap.interface.IdPairVec
    • Normally persisted to DbTsvStorage.
    • visitId must be passed in the additionalData for the OutputStage. This can be handled by adding the following to the OutputStage policy:
      AdditionalData: "visitId=triggerAssociationEvent.visitId"
      
    • The following sub-policy should be included in the OutputStage policy to properly configure the database table schema:
      Persistence: {
           Formatter: {
               IdPairVector: {
                   NewObjectIdPairs: {
                       templateTableName: "IdPair"
                   }
               }
           }
      }