wiki:MWInfReport031510
Last modified 9 years ago Last modified on 03/22/2010 10:42:34 AM

Previous month Apps Middleware/Infrastructure - Current Apps - Next month Apps Middleware/Infrastructure

Middleware/Infrastructure? Status Report for March 15, 2010

Guide on Reporting Percentage Completed:

For software development tasks,

  • use 0% if the solution is still being designed;
  • use 25% if the solution's design is completed;
  • use 50% if the solution is implemented and tested within the Ticket branch;
  • use 75% if the implementation review is complete, the Ticket branch has been merged onto the Trunk branch, and the Ticket is closed.
  • use 100% if the affected Trunk packages are tagged and released.

For other tasks, still use these increments, but apply their meaning as best as possible.

Note: Please place an asterisk (*) after the percent if you updated this value in this report.

Baker Bill

Reported 03/06/10

IDPercentTaskComments
MWI56920%Maintain automated testing structure
MWI4125%(BB) support distributable VMs for new developers
MWI7450%configure test sytem for automated V&V
MWI63725%Automated standards checking script
MWI6380%Buildbot to detect ABI changes on trunknot started
MWI6610%Advanced interslice communications supportnot started
MWI6620%Machine-readable inputs/outputs manifestnot started
MWI7070%*Automating nightly DC3a runRobyn is working on scripts for this, which I will configure to run on the cluster under buildbot

*Percent updated in this report

Comments:

Other Activities

Becla Jacek

Reported 03/17/10

IDPercentTaskComments
MWI680100%*Updates: source-related tables, Object table
MWI6810%Updates: exposure-related tablesnot started
MWI69120%*Partitioning: design partitioning of MovingObject? tablenot started
MWI69375%Tools for partitioning data
MWI69725%Task manager: design presentation
MWI70025%Benchmarking
MWI70125%Data Release tools
MWI70225%Documentation
MWI16025%*capturing full SW/HW infonot started
MWI1590%demonstrating re-creation of Calibrated Science Images not started
MWI16541%support persistence and retrieval of deep detection products and inputs
MWI46540%implement objectId synchronization between data releases (not DC3)

*Percent updated in this report

Comments:

Other Activities

  • schema design (sources, diasource, forced sources, exposure)
  • discussions about ImSim db
  • coding for qserv (lua script for mysqlproxy)
  • coordinating servers at slac (a) for DC3b database, (b) reddnet
  • arranging upgrade of lsst-related test bed at slac (faster 3 node cluster, access to a new 64 node cluster)
  • xldb
    • PI on a new qserv-related proposal for DOE/ASCR - a complete proposal coordinated/prepared, including narrative and budget [this took A LOT of my time this month]
    • preparing for xldb event
  • scidb
    • assistance with various legal issues, including some meetings

Bosch Jim

Reported xx/xx/xx

IDPercentTaskComments
MWI6855%Near neighbor query over 2 or 3 nodes
MWI6940%Basic generic query parser in placenot started
MWI6950%Demonstrate joining Object table with non-partitioned tablesnot started
MWI6960%DC3b syntax parser and aggregationnot started
MWI6980%Task manager: multiple queriesnot started
MWI6990%Task manager: interations with usernot started
MWI70125%Data Release tools
MWI70225%Documentation

*Percent updated in this report

Comments:

Other Activities

Cribbs Chris

Reported xx/xx/xx

IDPercentTaskComments

*Percent updated in this report

Comments:

Other Activities

Daues Greg

Reported xx/xx/xx

IDPercentTaskComments
MWI65650%Pipeline shutdown mechanisms
MWI6640%Freeze-drying clipboards: Harness supportnot started
MWI1060%Work to support communication of general serializable types over MPI not started
MWI1070%Develop mechanisms for identifying/accessing not started
MWI1130%Improving harness exception handlingnot started
MWI70925%Improve simple, interactive harness execution: interface

*Percent updated in this report

Comments:

Other Activities

Gehrig David

Reported xx/xx/xx

IDPercentTaskComments
MWI470%build warnings if unittest doesn't satisfy DC3 coverage requirement not started
MWI520%any failures cause automatic 'Failed Nightly Build' ticket at high priority not started
MWI540%any failure causes automatic 'Failed Production sanity check' ticket at high prioritynot started
MWI550%auxiliary automatic notification to highest authority on failure (separate from ticket notification) not started
MWI570%Standards: specify requirement to version and archive build's provenance;not started
MWI580%Configuration Management: define procedure to satisfy that requirement;not started
MWI590%Configuration Management: implement its instantiation. not started

*Percent updated in this report

Comments:

Other Activities

  • working on LSST cluster infrastructure

removing Lustre, getting lsst2 ready for iRODS

  • working on PT1 task replan
  • working with Dick S to determine which CFHT calibration data to download for PT1
  • vacation - March 5 to 12

Hanushevsky Andy

Reported xx/xx/xx

IDPercentTaskComments
MWI16541%support persistence and retrieval of deep detection products and inputs

*Percent updated in this report

Comments:

Other Activities

Lim KT

Reported xx/xx/xx

IDPercentTaskComments
MWI6910%Partitioning: design partitioning of MovingObject? tablenot started
MWI6980%Task manager: multiple queriesnot started
MWI6990%Task manager: interations with usernot started
MWI70225%Documentation
MWI1600%capturing full SW/HW infonot started
MWI1590%demonstrating re-creation of Calibrated Science Images not started
MWI16541%support persistence and retrieval of deep detection products and inputs
MWI46540%implement objectId synchronization between data releases (not DC3)
MWI1800%persistence framework refinementnot started
MWI6430%Upgrade OutputStage? to register outputsnot started
MWI6440%Implement simple, non-Policy retrievalnot started
MWI6450%App Class: remove need to declare friend classesnot started
MWI6460%App Class: remove intrusiveness of Boost persistencenot started
MWI6470%App Class: legacy file-based persistence/retrievalnot started
MWI6480%Python pers: investigate efficiency of methodsnot started
MWI6490%Python pers: define app class interfacenot started
MWI6500%Implement simple non-Policy persistencenot started
MWI6510%Upgrade InputStage?not started
MWI6520%Generic ORM: mapping attributes to columnsnot started
MWI6530%Generic ORM: collection persistence and retrievalnot started
MWI6630%Freeze-drying clipboards: designnot started
MWI6690%Determine segment images over a given sky tilenot started
MWI6700%LRU cache for raw segment imagesnot started
MWI6710%LRU cache for calibrated science exposuresnot started
MWI6720%Write/retrieve images to disk/tapenot started
MWI6730%Determine, retrieve cutout areas over given sky tilenot started
MWI6750%Write to co-add template pixel store on sky tile basisnot started
MWI6740%Determine template pixels overlaying given coordsnot started
MWI6760%Retrieve template pixelsnot started
MWI6770%Investigate local disk use for postage stamp generationnot started

*Percent updated in this report

Comments:

Other Activities

Monkewitz Serge

Reported xx/xx/xx

IDPercentTaskComments
MWI69025%Partitioning: research htm/healpix/stomp/dif
MWI6920%Implement generic query partitionernot started
MWI69375%Tools for partitioning data
MWI6960%DC3b syntax parser and aggregationnot started
MWI46540%implement objectId synchronization between data releases (not DC3)

*Percent updated in this report

Comments:

Other Activities

Pietrowicz Steve

Reported 03/22/10

IDPercentTaskComments
MWI6650% * Expose Event APIServer-side selectors tested, working on testing new rev. of ActiveMQCPP library.
MWI22825%Event system failover
MWI6580%I/O stage for interactive pipeline launchnot started
MWI65950%Refactor Orca LSST cluster orchestration done, working on Condor orchestration
MWI5380%Event Monitor integrationnot started
MWI5890%Pipeline Fault Tolerancenot started
MWI57025% *Events orchestration layer supportShutdown support has already been added to Orca
MWI6660%Scripts to monitor for node/pipeline failuresnot started
MWI6670%Python-based high-level event monitor scriptsnot started

*Percent updated in this report

Comments:

New version of Orca was tagged and released. Update coming for Condor support in orchestration. Prototyping of changes required for update to ActiveMQCPP library, for proposed Event API changes to support Blackboards, and for Event objects in the Event API.

Other Activities

Plante Ray

Reported 03/18/10

IDPercentTaskComments
MWI62475%User preferences directory
MWI62575%Support user/server tags
MWI6260%Improve packages installation/declarationnot started
MWI62725%Improve EUPS Docs
MWI6280%Meta releasesnot started
MWI6290%User publishing of packages to servernot started
MWI6310%As-built manifests into install directoriesnot started
MWI6320%Make SConsUtils dependency on EUPS optionalnot started
MWI6330%Upgrade SConsUtils for latest Sconsnot started
MWI6410%Logging tasksnot started

*Percent updated in this report

Comments:

Other Activities

This month was focused on primarily on PT1 preparation

  • worked Pietrowicz and Daues on completing ctrl_orca for PT1
    • created and initially developed the ctrl_provenance package for use by ctrl_orca (Pietrowicz completed this work)
    • presented DC3b orchestration plan for orchestration design review
    • did study of Blackboard-based approaches to pipeline orchestration; designed a proposed implementation
    • ctrl_sched: completed 50% of Blackboard implementation and unit tests (including items and queues)
  • worked on updating packages for the software stack
  • contributed to working groups: middleware, TCB, SAT
  • assisted Gehrig with "re-plan" activities

Wang Daniel

Reported 03/17/10

IDPercentTaskComments
MWI16541%support persistence and retrieval of deep detection products and inputsOngoing. Debugging execution and re-writing frontend task mgmt.

*Percent updated in this report

Comments:

  • qserv development.
  • Work with xrootd developer (Andy) on new LSST-specific features, and debugging asynchronous dispatch primitives.
  • Re-work front-end interface for mysqlproxy usage.

Other Activities

Most(?) of the time this period was spent on drafting/editing a proposal for future work on qserv.



Objectives for the Next Period

What you expect to accomplish.

NCSA

LSSTC

Database Group



Problems Encountered and Solutions Being Pursued

Budget or schedule variance, technical issues, management issues.

NCSA

LSSTC

Database Group