wiki:DC3bPT1.2-3000

Version 10 (modified by jbosch, 8 years ago) (diff)

--

LSST DC3b PT 1.2 3000 run

We expect to carry out a third ("3000") processing run in July 2011; I'd like to start the machines running by the 8th.

This page is intended to help us decide what to include. Some of this is already done on trunk or a branch, but I'd like a list of everything that we want that is not in level 2000. Please sign your contributions; this is not a commitment to implement them. The basic idea is to get a stack that reflects where we are (or nearly are) as of 2011-07-01.

As I have the enviable task of shepherding this data, we need to have a way to test each adopted feature. Ideally this would mean a pipeQA plot, but in some cases all I need is the test that shows that we really did implement the feature (e.g. the Chebyshev changes).

Desired Features

Chebyshev interpolation (RHL)
For PSF interpolation (using the RHL pcaPsf for now). Test: confirm that high-order polynomials make sense. Status: In on trunk as of 2011-06-22 - one week
A star/galaxy number (RHL)
Probably based on the instFlux (==gaussianFlux) or the modelFlux from UCDavis. Test: use the numbers in pipeQA. Status: Done if we overload "Source.apDia"
Proper flat fields (RHL)
There is evidence of problems in the corners of the array. Test: look at pipeQA's (psf - cat) plots. Status: new flats are being generated
Proper propagation of measurement masks (RHL)
From measurement to Source; no need to handle proper assigment of bits to different algorithms (i.e. a single global enum is OK for now). Test: look at the output bits. Status: thought to be done. May need cleanup.
Diffim for snap pairs (ACB)
This needs to be implemented as a pipeline of some sort; possibly using a single pipette-based stage + two I/O stages. Test: ???. Status: Code is mostly written (ip_diffim). Policy specialised for this application. KT will help with pipeline stage.
aperture corrections for model fluxes (RHL)
The modelFlux for a point source must equal the PSF flux. N.b. this requires that the same aperture correction be applied as for the psfFlux. Test: the pipeQA (psf - model) plots have a nice stellar locus Status: maybe a ways off, and may not make 3000
cleanup aperture correction application code (RHL)
The current code (in the stage) is specific to psf mags + cut-and-paste copies [mea culpa]. Test: Look at the code; also needed to support the previous item. Status: written but not committed
Look at background level bias (RHL)
I think that there's evidence for a bias at the two DN level. Test: Look at pipeQA's (psf - ap) for larger apertures. Status: the functionality to ignore detected objects exists in the Background code
Improve centroids for faint extended source (RHL)
Problems show up in centroids off the edge of the Footprint. Test:? Status: mostly written
Corrected astrometry.net index files (KSK)
An update to the index files in /lsst2/krughoff/astrometry_net_data/imsim-2011-06-20-0 (likely called /lsst2/krughoff/astrometry_net_data/imsim-2011-06-20-1) will be released as a package. Status: Package release requested.
Check flat production and flat accuracy (KSK)
There is evidence of gradients in the most extreme edge chips. This suggests an error in the application or production of the calibration flats. There is also evidence in the photometric zeropoints that the analytic model of the vignetting is not accurate enough to model the true vignetting function to the 1% desired over the full focal plane. Status: Looking into the flat production. Comparing flats produced photon by photon to the analytical model.

Things we'll keep working on until someone says "code freeze!"

Better diagnostics and experimentation with galaxy modeling (UCDavis)
Some funny things are still going on with some datasets: galaxy models fluxes are systematically offset from PSF fluxes by too much, and objects that are clearly stars are being fit with extended models. We're making rapid progress, but it's not clear how much we have left to do. See GalaxyModelingToDoList for more details.

Things that should not be included

Cleanup of Policy (RHL)
Replacement of sets of stages with pipette mega-stages (RHL)
The Angle changes from months ago (RHL)
They are ready-to-merge, but let's hold off until post 3000
Simpler galaxy models with a better optimizer (UCDavis)