Changes between Version 3 and Version 4 of dbObjectIdSync


Ignore:
Timestamp:
09/16/2008 03:16:42 PM (11 years ago)
Author:
ktl
Comment:

Added response with regard to data release and resync.

Legend:

Unmodified
Added
Removed
Modified
  • dbObjectIdSync

    v3 v4  
    3232Alerts may publish an associated objectId. It's possible that the object is "new"; that is, not present in the deep detection output used to seed the nightly pipelines for the data release cycle in progress. Associating with the previous catalog release will fail to catch such cases. Therefore it seems desirable to sync objectIds of an about to be released catalog with those of the up-to-date catalog immediately prior to release. However, since the association between ''about-to-be-released'' and ''up-to-date'' catalogs cannot be performed in zero time, it may still be possible for new objects to appear in the ''up-to-date'' catalog while the association step is performed (these objects won't participate in the association). This brings up another problem - the deep detection run producing the ''about-to-be-released'' catalog may have been running for quite a while (months?) and is therefore unlikely to include images that would cause deep-detect to pick up many "new" objects. Furthermore, new objects created by the nightly pipelines might only show up in a very small number of images - if as a result they fall below the SNR threshold used for detection on coadds, then deep detect may never create corresponding objects and we will have published alerts referring to objects that don't exist in public catalogs. Is this a problem, and if so how should it be handled? 
    3333 
     34KTL: Data release processing will include difference image analysis, so transients that are not found in deep detection will still be added to the data release object and DIASource catalogs.  Data release processing will ''not'' include the images taken during that processing, so some catch-up association will be needed to resynchronize the nightly pipeline with the data release object catalog. 
     35 
    3436One approach would be to have alerts include objectIds only for those objects which appear in a DR - for alerts on (or signaling the creation of) new objects, only tentative object attributes would be published. Another possibility is to extract "new" objects from the ''up-to-date'' catalog and append them to the ''about-to-be-released'' catalog right before release. 
    3537