System Architecture Team, 2010.6.4
See the agenda previously posted.
Present: Lupton, Lim, gpdf, Axelrod, Allsman, Plante
gpdf: Turning to PT2 issues (and beyond).
Discussion of future agenda items.
Mike Freemon suggested to gpdf covering large-memory architectures; added to list. Related to an existing item ("Support of pipelines (like parts of DayMOPS) with algorithms that require more space than physical memory: should we, and if so how?").
Brief discussion of some build-system issues. Dependency tracking, requirement to use eups, use of scons, "link farms" vs. environment variables.
Decide to have build system review on July 9th when everyone is available.
DataButler
Some things are currently in Python code that KT would like to move into policy files. Planning to make that change post-PT1? In particular, the mapping from datasets to pathnames could be made much more generic.
Some discussion of accessing datasets that are stored in something other than a POSIX filesystem -- for instance, in a database or in xrootd. This can be handled by the DataButler but it would require extra configuration for the associated Mapper.
Some concern from Robert about the wisdom of allowing both numeric (ccd=35) and symbolic (ccdName="CFHT 7") lookups of CCDs in the butler. Should this be handled by the user instead, since the user can get the camera geometry object?
Rename "fileExists" to "datasetExists", to continue removing file-centrism.
RHL: Can this integrate with IRODS-type mappings of logical-to-physical? gpdf/KT: Sure, you would just end the mappers' work with the file service's logical names, and let IRODS worry about the physical names.
Dependencies Exposure
gpdf's major concern with Exposure is with regard to its dependencies. Since so many things depend on Exposure, we might want to keep it as lightweight as possible. It seems to be developing a somewhat fat interface, with references to Wcs, Detector (camera geometry), Filter, and Calib. (See highlighted lines in Exposure.h.)
KTL: Also expect it to refer to Psf eventually, and there may be more.
gpdf: How many more? TA: Probably no more than a factor of two.
gpdf: Two concerns: a) the sheer breadth of the interface, but more importantly b) the header-file complexity it introduces.
The latter point can be addressed by moving from the Foo::Ptr syntax to the PTR(Foo) syntax that we agreed on at the last SAT meeting. This will allow the corresponding header files to be removed from Exposure.h. This can be done now, with the latest release of base, but should only be done after svn is branched to allow PT2 development to proceed in parallel with the final refinements to PT1. We agreed that it should be done fairly early, as it may require some non-trivial followups to ensure that needed header files are provided elsewhere.
The migration looks very simple, except for Wcs, where there is some use of methods of Wcs in inlined methods in Exposure.h.
Brief discussion of the presence of non-trivial methods as inlines. RHL: Needed because they are templated, e.g. the templated copy constructor. gpdf: But these don't need to be inlined, they just need to be in the header file (in the current LSST scheme for template management).
(Note also that the TanWcs? header doesn't seem to be needed at all. And the getCalib methods are in the wrong place in the header file.)
Discussion of the "don't use me" public setXY0() methods. Shouldn't the design enforce the underlying idea? gpdf: Uncomfortable with the presence of setters affecting internal objects maintained via exposed shared_ptr's -- things can get changed out from underneath you.
gpdf observes that each of Wcs, Detector, Filter, and Calib are treated differently in Exposure. Why is this?
| Object | Header dependency | Polymorphism-friendly | Value maintained by | Initialized by | Has public setter | Access provided |
| Wcs | yes | yes | shared_ptr (Wcs::Ptr) | clone | yes | shared_ptr to non-const returned from const method |
| Detector | yes | yes | shared_ptr (Detector::Ptr) | copy of shared_ptr | yes | shared_ptr to non-const returned from const method |
| Filter | yes | no | value | copy constructor | yes | value, from const method |
| Calib | no | no | shared_ptr (explicit) | new copy-constructed heap object | no | shared_ptr to (non)const returned from (non)const method |
Some discussion. E.g., Filter is meant to be a tiny value class, so copy-by-value is not unreasonable. No clear reason for the difference in approaches to polymorphism. No real review of the issue of providing non-const access through const methods.
Jocular suggestion that gpdf take responsibility for migrating Exposure to use the PTR macros once the branch for PT1 has been done. gpdf: OK, if people don't mind being asked a lot of questions.
Definitely need a more serious design review of Exposure at a later date.
Unfinished business
The DC3 meeting will decide when the PT1/PT2 branching is done.
We will address KT's daf_data issues offline or at the next meeting; see: DafDataIssues.
