Ticket #1556 (closed enhancement: fixed)

Opened 8 years ago

Last modified 7 years ago

Overhaul multifit for PT1.2

Reported by: dubcovsky Owned by: jbosch
Priority: normal Milestone:
Component: meas_multifit Keywords:
Cc: jbosch, dubcovsky Blocked By:
Blocking: Project: LSST
Version Number:
How to repeat:

not applicable

Description (last modified by jbosch) (diff)

complete overhaul of multifit to using shapelet based models. Will be replacing most of the code in meas_multifit, and making small huge changes to dependent packages such as afw.

Change History

comment:1 Changed 8 years ago by DefaultCC Plugin

  • Cc dubcovsky added

comment:2 follow-up: ↓ 4 Changed 8 years ago by jbosch

  • Description modified (diff)

The #1556 ticket branch for afw has been used as a development branch for a number of far-reaching changes to afw from other tickets that have been made while overhauling meas/multifit. These include:

  • #1240: geom::Point and geom::Extent can now be constructed directly from (x,y) pairs, and the makePoint and makeExtent functions have been removed.
  • #1238: Box2I and Box2D are now the preferred names for the geom::Box classes; BoxI and BoxD are typedef'd.
  • #1551: Upstream updates to geom::ellipses. Adds separate classes for complex ellipticity and radius definitions.
  • #1552: Added a minimal shapelet library (afw::math::shapelets) and interface-only support for shapelet extraction from arbitrary Psfs via the new LocalPsf class.

Two additional, more major changes do not have other tickets associated with them:

ImageBase and FITS IO internals have changed to support strided access to the memory owned by an Image via ndarray.

  • afw now depends on the ndarray package.
  • The memory storage order is now naturally row-major, instead of having an inverted Y-axis. This change should be invisible to most users, but it makes it possible to obtain a correctly-ordered matrix or vector view of the image. It also makes the in-memory storage order the same as the on-disk storage order in FITS, potentially eliminating a deep copy during IO.
  • A reference-counted, writeable ndarray::Array view into an image can be obtained via ImageBase::getArray(). In Python, this yields a numpy array. Be aware that numpy/ndarray indexing is [y,x], however, so the view may appear transposed when accessing individual elements - but it will appear correctly when plotted with matplotlib, for instance. The now-redundant (and slower) pure-Python routines to get a numpy copy of an image in image/testUtils.py have been removed (but note that these are the transpose of the result of getArray(), and produced deep copies rather than shallow views). MaskedImage has a getArrays() method in Python only, returning a tuple of image, mask, and variance arrays (or None where appropriate). ndarray::viewAsEigen() can be used to get an Eigen view into the ndarray view (see the ndarray documentation or ask Jim for more info).
  • ImageBase subclasses now have a constructor that takes an ndarray::Array (or in Python, a numpy array). These have the same transpose-indexing semantics as getArray(). Because an ndarray::Array can be constructed from arbitrary blocks of memory, Image now can be too (again, see the ndarray documentation for more info). Pure-python makeArrayFromImage (etc) functions have been added to the main lsst.afw.image python package (in __init__.py) to replace those previously found in image/testUtils.py.

The geometry (BBox, BCircle, PointI, PointD) in image/Utils.h have been removed and replaced throughout afw with their afw::geom equivalents.

  • As part of this change, we have introduced an additional enum, ImageOrigin {LOCAL, PARENT} to help clarify the use of xy0 in Image and friends: LOCAL indicates the coordinate system in which xy0 is ignored, and PARENT indicates the coordinate system of the parent image, in which xy0 sets the origin of the sub image in the parent image coordinates. This probably invalidates #1334, which seems to have been dormant for a while.
  • The most widespread change is in the memory-allocating constructors for Image and MaskedImage, which now take Extent2I or Box2I, instead of BBox, std::pair<int,int>, or two separate integer arguments.
  • Other objects in afw that previous returned a std::pair<int,int> or std::pair<double,double> for dimensions (such as Kernel) have been switched to use Extent.
  • The signature of all subimage constructors have changed, adding the enum argument to specify how the box is to be interpreted. This is the one known change that may cause incorrect behavior rather than a C++ compile failure or straightforward Python type error: a bool argument (usually for the "deep" or "conformMasks" arguments) in current code can be overload-matched in Python to the ImageOrigin enum value, leading to confusing results for the image origin, copy semantics, and mask-plane definitions. We have made the SWIG bool conversions more restrictive to ensure the reverse confusion does not happen.

comment:3 Changed 8 years ago by jbosch

  • Status changed from new to assigned
  • Owner changed from dubcovsky to jbosch

Additional changes left out of previous list:

  • Eigen SWIG typemaps have been moved out of afw and into ndarray. The basic procedure is still the same, but the names and locations of the header files and the names of the SWIG macro have changed. The same methods now work on ndarray::Array as well as Eigen types. See afw/math/shapeletsLib.i or the ndarray test code for examples.

comment:4 in reply to: ↑ 2 Changed 8 years ago by jbosch

Replying to jbosch:

Pure-python makeArrayFromImage (etc) functions have been added to the main lsst.afw.image > python package (in __init__.py) to replace those previously found in image/testUtils.py.

Correction: we have added makeImageFromArray, not makeArrayFromImage.

comment:5 Changed 8 years ago by rowen

Russell's notes on converting code to the new afw


  • BoxI -> Box2I
  • BoxD -> Box2D
  • makePoint -> Point2
  • makeExtent -> Point2
  • afwImage::PointI -> afwGeom::Point2I
  • Look for std::pair<int, int> used as an image extent and replace with afwGeom::Extent2I
  • afwImage::BBox -> afwGeom::Box2I but the arguments change as well, so be careful
  • afwImage::PointI -> afwGeom::Point2I

The Image/Mask/MaskedImage/Exposure? constructors that took separate width and height are gone. Specify an Extent2I for dimensions or a Box2I for a parent bbox.

In /python/...SConscript add ndarray to the CheckSwig call

The package coaddUtils.bboxFromImage is gone. Thus replace:

  • bboxFromImage(image)-> image.getBBox(afwImage::PARENT)
  • imageFromBBox(bbox, image.Factory) -> image.Factory(bbox, afwImage.PARENT)
  • makeBlankExposure(fromExposure) -> exposure = fromExposure.Factory(fromExposure.getBBox(afwImage.PARENT), fromExposure.getWcs())

afwImage.testUtils has lost all its numpy<->image functions (which were slow and will not be missed):

  • arrayFromImage, arrayFromMask, arraysFromMaskedImage are replaced by image.getArray(), mask.getArray(), maskedImage.getArrays()
  • imageFromArray, maskedImageFromArray are replaced by afwImage.makeImageFromArray, makeMaskFromArray and makeMaskedImageFromArray (these infer the type of the result from the type of the array).
  • WARNING: the indices of the new numpy arrays are reversed and the result is a view, not a deep copy, so update carefully! However, numpy arrays offer copy() and transpose() methods so you can just use those. Note that transpose() returns a transposed view of the data.

comment:6 Changed 8 years ago by rowen

coadd_utils ticket #1556 is done and ready to be merged to trunk.

comment:7 Changed 8 years ago by jbosch

  • Status changed from assigned to closed
  • Resolution set to fixed

comment:8 Changed 7 years ago by robyn

  • Milestone DC3b Completed deleted

Milestone DC3b Completed deleted

Note: See TracTickets for help on using tickets.