wiki:DesDataChallenges
Last modified 10 years ago Last modified on 05/02/2009 01:21:48 PM

Learning from DES Data Challenges on LSST

All LSST DC leaders should read DES DC4 report and DC5 plan to mine ideas/lessons learned, contact Project Manager for link

DES separates development from Data Challenge schedule, alternates between them

DES has DC Readiness Review (like reviewing status after our integration week) prior to each DC start of production runs

RICE Compression (DES getting factor of 5, lossy on FP, 2 non-lossy on raw integer, 4 noise bits)

QA metrics list developed from DC metrics and vice versa

Blind test with simulated data and known truth

Science Working Groups act as tool testers, data quality analysts

Science Working Groups doing analysis codes (Photo-Z H. Lin and photo-z WG, Weak Lensing M. Jarvis) (unfunded)

Live view of production runs/statistics, images available via public portal

Using Open Science Grid for computing to create simulated data

Using Blue Arc 40 TB disk for storing simulated data

DES did not specify data quality targets DC4, they did assessment against the overall project requirements post facto, will do so starting with DC5