wiki:DC3ApplicationsMeeting
Last modified 10 years ago Last modified on 09/07/2009 09:45:56 PM

Agenda for the Applications Working Group (19th -20th Feb) at IPAC

Goals:

Discussion and brainstorming session on the current state of applications and where we need to go. Each person listed is the leader of the discussion but, apart from the initial overview, everyone should have input throughout each session.

We will discuss in detail what changes we need to make before moving on to DC3 (ie a detailed look at how applications are affected by framework classes, algorithms, software libraries, class design, programming structure, balance between python and c++, interfaces and utilities and tools for debugging, master classes in debugging).

For those people assigned to lead the discussions please send the presentations to lsst-data or put on the wiki prior to the meeting (ie no writing it on the plane :)

Tuesday 19th:

9.00-9.15am Welcome to IPAC (Suzy Dodd)

9.15-10.30 Introduction to the DC2 pipeline structure (Tim Axelrod and Ray Plante)

Overview of the current applications pipeline (Tim) Introduction to the framework of the pipelines Current limitations of running pipleines Documentation and where to find it Building and installing the software stack (Ray) How we invoke a pipeline at NCSA Where the input/output data and logs reside svn and the life of a ticket

focus will be on details that will introduce the UCD and IPAC groups to pipeline development

10:30 - 10.45 Coffee

10.45 - 3.45pm Application Pipelines

Each group will do an end-to-end breakdown of the pipeline including: description of algorithms used, examples of data pushed through including edges cases where the pipeline doesnt work (and how we catch these cases), breakdown of timings of components, discussion of where the algorithms fail, list of proposed class changes and framework changes required by the subgroup, remaining tasks for that section and priorities. All classes used by the subgroup should be displayed and described through the presentation.

10.45 - 11.00 DC2 Timing and Statistics (Ray Plante)

11.00-12.00 Image subtraction (Andy Becker)

algorithms timings convolution/deconvolution

12.00- 1.00pm Lunch

1.00-1.45 Detection (Robert Lupton)

algorithms convolution with matched kernels chi-sq images thresholds

1.45-2.45pm Association (Serge Monkewitz)

matching algorithms

2.45-3.45pm MOPS (Francesco )

algorithms timings scaling to LSST (memory/parallel application)

3.45-4.00pm Coffee Break

4.00-5.30 Framework and Class Design (Tim Axelrod, K-T Lim)

Description of current framework classes (including image classes), discussion of workarounds used for DC2, discussion of required changes to the classes (led by each of the subgroups above). General work through of the class design work on any new class design (as an example the use of maskbits in an image). Persistence/retrieval of those classes and FITS metadata in images.

Wednesday 19th

8.00-9.00am Required Software Libraries and Utilities (Russell Owen)

What libraries or python modules do we need for DC2/DC3 and for debugging. Are there new python modules we should be considering; for example plotting tools such as matplotlib and what is their required functionality (images, footprints of detected sources, histograming, scatter plots). Which packages should we use. Are their high quality stats packages that are useful to us. Evaluation of current libraries used by the pipelines (e.g vision workbench) and how are they working out (are there areas we need to change or functionality we dont have). Are the better versions available to us and should we consider them?

9-10am Python interfaces to C++ (Russell Owen and Robert Lupton)

Discussion of interfacing C++ and python focusing on swigging the code and how well shared pointers work. Discussion of the balance between python and C++ in the current code we are writing; are we writing algorithms in C++ and interfacing to python or moving too much code into python.

10-10.15 Coffee break

10.15-11.00am Management structure for Applications in DC3 (Suzy Dodds)

11.00-11.30am Policy files (Ray Plante)

13.00-12.30 Simulationss Requirements (Andy Connolly)

12.30-1.30 Lunch

1.30 - 2.30 Debugging Master Class

How to debug code using gdb when everything is wrapped with python.

2.30 - 4.30 Use Cases for DC3

Data Release Usecases (Tim Axelrod)

CoAdd? Images

Produce Template Images

Multifit (Dave Wittman)

DQA (Deborah Levine)

Astrometric Calibration (Dave Monet)

Photometric Calibration(Tim Axelrod)

New Nightly Pipeline Usecases

Linearization (Nicole Silvestri)

PSF Characterization

Detection for single images (Robert Lupton)

4.30 - 5.00 Action Items and wrap up

Attachments