Ticket #818 (closed defect: fixed)

Opened 10 years ago

Last modified 7 years ago

Defect removal has an assertion failure on SimDeep data

Reported by: ktl Owned by: becker
Priority: blocker Milestone:
Component: meas_algorithms Keywords:
Cc: krughoff, rhl, RayPlante Blocked By:
Blocking: Project: LSST
Version Number:
How to repeat:
cd /lsst/DC3root/ticketFiles/818
python fail.py

Description (last modified by ktl) (diff)

The ISR stage is failing on the SimDeep data, visit 000001, exposure 0 with an assertion failure in meas_algorithms classify_defects. This occurs either with defects*.paf files that contain nothing or that contain a single pixel at 0,0. This only appears to happen on amps 3 and 11 for some reason.

The assertion failure is:

python: src/Interp.cc:133:
std::vector<boost::shared_ptr<lsst::meas::algorithms::Defect>,
std::allocator<boost::shared_ptr<lsst::meas::algorithms::Defect> > >
lsst::meas::algorithms::classify_defects(
const std::vector<boost::shared_ptr<lsst::meas::algorithms::Defect>,
std::allocator<boost::shared_ptr<lsst::meas::algorithms::Defect> > >&, int, int, int):
Assertion `defect_m->getX1() < defect->getX0()' failed.

Package versions are:

ctrl_dc3pipe          3.3.2             Current Setup
ip_isr                3.3.9             Current Setup
meas_algorithms       3.0.8             Current Setup

Change History

comment:1 Changed 10 years ago by ktl

  • How to repeat modified (diff)
  • Description modified (diff)

comment:2 Changed 10 years ago by ktl

  • Description modified (diff)

comment:3 Changed 10 years ago by ktl

  • Description modified (diff)

comment:4 Changed 10 years ago by becker

It looks more like a problem with meas_algorithms than with Isr, but I will do some more checking before I reassign this to RHL. The solution for Isr might be to just make an empty defects file; I don't know why there is a bogus defect in the sim defects file, as opposed to no defects at all.

comment:5 Changed 10 years ago by ktl

There is now a bogus defect in the defects files because it was initially failing with no defects at all. Adding the bogus one doesn't seem to have changed the behavior, but of course we should set it back.

comment:6 Changed 10 years ago by becker

There is no logical reason that this should fail for amps 3 and 11 only if its the defects file. I've verified that it works for amp 1, and

diff defect-c011-a001.paf defect-c011-a003.paf 

returns nothing. Am I looking at the correct data? /lsst/images/repository/calib/SimDeep_051809? Or have things been moved to /lustre/?

comment:7 Changed 10 years ago by becker

OK, its not the defect files, its the location of a saturated pixel that is causing the problem. So it just happens that amps 3,11 have invalid saturated pixels that cannot be interpolated over. I'll next find out why. In the meantime its probably OK to revert to empty defects files.

comment:8 Changed 10 years ago by becker

I think the issue is we are growing saturated pixel footprints, and nothing to catch the case where the footprint grows off the image. We need to fix this in Isr, and Rhl needs to fix meas_algorithms to throw an exception instead of seg faulting.

comment:9 follow-up: ↓ 10 Changed 10 years ago by ktl

FYI: It looks like the defect files have been reverted to empty versions.

comment:10 in reply to: ↑ 9 Changed 10 years ago by krughoff

Replying to ktl:

FYI: It looks like the defect files have been reverted to empty versions.

I reverted the defects to empty files since this reflects the state of the sim images at the moment. To add a defect I just put in a single square defect:
Defects: {

x0: 1 # Starting column
width: 1 # number of columns
y0: 1 # Starting row
height: 1 # number of rows

}

comment:11 follow-up: ↓ 14 Changed 10 years ago by becker

AArgh, I'm getting bit by XY0 again. This really needs to be rethought...

comment:12 Changed 10 years ago by becker

OK, its not that the defect is being grown off the image. I'm not entirely sure why this is failing; but its failing at

algorithms.interpolateOverDefects(mi, psf, defectList)

KT, is there a way to persist defectList - a algorithms.DefectListT() - so I can make a ticket for RHL?

comment:13 Changed 10 years ago by rhl

  • Status changed from new to closed
  • Resolution set to fixed
  • reviewstatus changed from notReady to selfReviewed

The assertion is that the defects are properly sorted; they are not in this case (as one of them lies entirely below the line being processed) due to bug in the classification routine. Fixed in r9891, after adding an stripped down version of this routine to meas/algorithms/tests

comment:14 in reply to: ↑ 11 Changed 10 years ago by rhl

Replying to becker:

AArgh, I'm getting bit by XY0 again. This really needs to be rethought...

This amp region is only a part of the chip; the defect map refers to the entire CCD. You are going to have to think about the origin of your coordinate system, i.e. obey XY0, to handle subimages, and the code is required to handle subimages correctly. Sorry; the reason why we have non-trivial origins is that they occur in practice, and you can't just wish them away. The way that we are doing so isn't the only possibility and I'm open to rethinking our solution --- but a solution is required.

comment:15 Changed 10 years ago by rhl

  • Blocked By 828 added

comment:15 Changed 10 years ago by rhl

  • Blocked By 828 removed

The fixes are available in meas_algorithms 3.0.9 (distribution request: #828)

comment:16 Changed 7 years ago by robyn

  • Milestone DC3a Completed deleted

Milestone DC3a Completed deleted

Note: See TracTickets for help on using tickets.