wiki:DC2TestData
Last modified 11 years ago Last modified on 12/19/2007 12:29:26 PM

Data Challenge 2 Test Data

from: DC2 Management -> Test Data.

The following data is on lsst9.ncsa.uiuc.edu:

Observation Images, Unsorted

The unsorted, unrenamed D4 .fits files are in the directories

/lsst/images/D4/r  [images with r' filter]
/lsst/images/D4/i  [images with i' filter]

In the directory /lsst/images/headers are the headers containing the updated distortion information.

Observation Images, Renamed

At /lsst/precursor there is a file showing the original name of a calibrated image file and where it is located in the precursor archive.

For example, the first line of the file is

704893p.fits 52795/cal/cal-52795-i-704893.fits

This means that the there is, in the directory /lsst/precursor/CFHTLS-DEEP/52795/cal a link named cal-52795-i-704893.fits which links to the file 704893p.fits in the /lsst/images/D4/i directory.

The input data for DC2 should be named in a similar fashion -- i.e. the file lsst/precursor/CFHTLS-DEEP/52795/cal/cal-52795-i-704893_22.fits should represent the 22nd CCD of the image.

Template Images

In the directory /lsst/images/templates, three stacked images from CHFTLS-DEEP d4:

D4.2007B.G.fits
D4.2007B.I.fits
D4.2007B.R.fits

Each of these image files is approximately 1.5 GB.

D4.2007B.G.cat
D4.2007B.I.cat
D4.2007B.R.cat 

Each of these catalog files contains information on roughly 350,000 objects in the associated image files.

For each object, the catalog includes:

#   1 NUMBER          Running object number
#   2 X_IMAGE         Object position along x                         [pixel]
#   3 Y_IMAGE         Object position along y                         [pixel]
#   4 ALPHA_J2000     Right ascension of barycenter (J2000)           [deg]
#   5 DELTA_J2000     Declination of barycenter (J2000)               [deg]
#   6 MAG_AUTO        Kron-like elliptical aperture magnitude         [mag]
#   7 MAGERR_AUTO     RMS error for AUTO magnitude                    [mag]
#   8 MAG_BEST        Best of MAG_AUTO and MAG_ISOCOR                 [mag]
#   9 MAGERR_BEST     RMS error for MAG_BEST                          [mag]
#  10 MAG_APER        Fixed aperture magnitude vector                 [mag]
#  11 MAGERR_APER     RMS error vector for fixed aperture mag.        [mag]
#  12 A_WORLD         Profile RMS along major axis (world units)      [deg]
#  13 ERRA_WORLD      World RMS position error along major axis       [pixel]
#  14 B_WORLD         Profile RMS along minor axis (world units)      [deg]
#  15 ERRB_WORLD      World RMS position error along minor axis       [pixel]
#  16 THETA_J2000     Position angle (east of north) (J2000)          [deg]
#  17 ERRTHETA_J2000  J2000 error ellipse pos. angle (east of north)  [deg]
#  18 ISOAREA_IMAGE   Isophotal area above Analysis threshold         [pixel**2]
#  19 MU_MAX          Peak surface brightness above background        [mag * arcsec**(-2)]
#  20 FLUX_RADIUS     Fraction-of-light radii                         [pixel]
#  21 FLAGS           Extraction flags 

TALCS

Images to be used for TALCS are in /lsst/images/TALCS with sorted/renamed links at /lsst/precursor/TALCS.

Database Object Catalog

  • The mysql dump of the Object Catalog is on lsst9.ncsa.uiuc.edu:/lsst/images/d4_object_ri.sql
  • The complete DC2 database schema is in subversion (DC2/database/trunk/sql/lsstSchema4mysql.sql), here is the source
  • The Object Catalog was built by associating detections in R and I. Association included finding all neighbors within 3 arc sec radius, and picking the closest match. Detections that didn't match were added for both filters. Statistics: 328,017 matched out of 367,175 in r and 378,169 in i (~12% didn't match). Final Object Catalog size: 417,327 objects.
  • Mapping used:
    in D4           in Database
 -------------------------------
   1 NUMBER          objectId
   2 X_IMAGE         N/A
   3 Y_IMAGE         N/A
   4 ALPHA_J2000     ra
   5 DELTA_J2000     decl
   6 MAG_AUTO        xPetroMag
   7 MAGERR_AUTO     xPetroMagErr
   8 MAG_BEST        xMag
   9 MAGERR_BEST     xMagErr
  10 MAG_APER        xApMag
  11 MAGERR_APER     xApMagErr
  12 A_WORLD         xIxx
  13 ERRA_WORLD      xIxxErr
  14 B_WORLD         xIyy
  15 ERRB_WORLD      xIyyErr
  16 THETA_J2000     xIxy
  17 ERRTHETA_J2000  xIxyErr
  18 ISOAREA_IMAGE   xIsoAreaImage
  19 MU_MAX          xMuMax
  20 FLUX_RADIUS     xFluxRadius
  21 FLAGS           xFlags

Where x is the filter name (r or i).
  • The objectId was calculated as r.NUMBER + 1,000,000 for all cases except unmatched i detections. For unmatched i detections i.NUMBER + 2,000,000 was used
  • ra, decl was taken from r for all cases except unmatched i detections.
  • The Object Catalog contains all columns that we have in the precursor schema (over 300), all unused columns are set to NULL.
  • Detailed recipe how the catalog was built can be found here

Preparation of Precursor Data

  • D4 : Images are currently having their headers updated with Stephen Gwyn's .head files. These updated images are being used to Swarp the templates to match the science images. Finally, both the science and template images are being turned into MaskedImages?. These data are landing in
    • r-band : /lsst/images/becker/D4/r/output
    • i-band : /lsst/images/becker/D4/i/output
  • TALCS : Images are currently being Scamped at UW. They will then be Swarped at NCSA, one for each passband of data (g,r). These data will land in
    • g-band : /lsst/images/becker/TALCS/g
    • r-band : /lsst/images/becker/TALCS/r