wiki:dbProvInDC3
Last modified 10 years ago Last modified on 12/08/2008 05:46:42 PM

Provenance and Operational Metadata in DC3

LSST Database

Related ticket: #511

Provenance

In DC3a we will capture contents of all policy files using the prv_PolicyKey and prv_cnf_PolicyKey tables.

There no plans to capture additional provenance related information (like hardware setup).

Operational Metadata

Monitoring

It would be nice to integrate per-run operational statistics into the DC3a schema, including things like when each run/pipeline/stage/slice started/finished etc. This is currently logged by the orchestration harness into a separate database.

Maintenance

We will need one global database in addition to database-per-run to capture per-run information. IT will be used to track when each run can be deleted. This information can be kept in a table like:

CREATE TABLE Run
   runId  INTEGER NOT NULL PRIMARY KEY,
   runType ENUM ('DC1', 'DC2', 'DC3a', 'DC3b', 'DC4', 'COMMISSIONING', PRODUCTION'),
   runName VARCHAR(255),             -- run description / comments
   runDbName VARCHAR(64),            -- corresponding database name 
   runInitiator VARCHAR(255),        -- username of the person who started this run
   expirationDate DATETIME NOT NULL, -- datetime when this run can be deleted
   flags ENUM('STARTED', 'KILLED', 'FAILED', 'FINISHED'), -- flags for indicating run status
   deleted CHAR DEFAULT 'n'          -- 'y' / 'n'
) ;

We could capture more per-run statistics here as needed (eg when run started/finished)