wiki:MWInfReport011510
Last modified 9 years ago Last modified on 01/21/2010 07:25:47 PM

Previous month Apps Middleware/Infrastructure - Current Apps - Next month Apps Middleware/Infrastructure

Middleware/Infrastructure? Status Report for January 15, 2010

Guide on Reporting Percentage Completed:

For software development tasks,

  • use 0% if the solution is still being designed;
  • use 25% if the solution's design is completed;
  • use 50% if the solution is implemented and tested within the Ticket branch;
  • use 75% if the implementation review is complete, the Ticket branch has been merged onto the Trunk branch, and the Ticket is closed.
  • use 100% if the affected Trunk packages are tagged and released.

For other tasks, still use these increments, but apply their meaning as best as possible.

Note: Please place an asterisk (*) after the percent if you updated this value in this report.

Baker Bill

Reported 1/20/2010

IDPercentTaskComments
MWI56920%*Maintain automated testing structureOngoing. This month: improved support for builds of trunk packages vs. current releases, plus trunk vs. trunk where necessary.
MWI4125%*(BB) support distributable VMs for new developersNew releases once stack is fixed. Ray wants them ready for the JWST workshop Jan 25-26.
MWI7450%configure test sytem for automated V&V
MWI63725%*Automated standards checking scriptDeveloping web interface for automated standards checking with Parasoft.
MWI6380%Buildbot to detect ABI changes on trunknot started
MWI6610%Advanced interslice communications supportnot started
MWI6620%Machine-readable inputs/outputs manifestnot started
MWI7070%Automating nightly DC3a runnot started

*Percent updated in this report

Comments:

Other Activities

Becla Jacek

Reported 01/21/10

IDPercentTaskComments
MWI68075%Updates: source-related tables, Object table
MWI6810%Updates: exposure-related tablesnot started
MWI682N/A*Synthetic sources of dataPer Tim (DataAccWG telecon Jan 15, 2010) this is not needed for DC3b. Planning to do after DC3b.|
MWI68475%Standardizing names
MWI6910%Partitioning: design partitioning of MovingObject? tablenot started
MWI69375%Tools for partitioning data
MWI69725%Task manager: design presentation
MWI70025%Benchmarking
MWI70125%*Data Release tools
MWI70225%*Documentation
MWI1600%capturing full SW/HW infonot started
MWI1590%demonstrating re-creation of Calibrated Science Images not started
MWI16540%*support persistence and retrieval of deep detection products and inputs
MWI4650%implement objectId synchronization between data releases (not DC3)not started

*Percent updated in this report

Comments:

Other Activities

  • discussing and documenting sql query API restrictions
  • rewriting queries in trac
  • db schema improvements
  • schema browser tweaks
  • attended nsf review
  • started work on planning lsst data access construction at slac
  • discussions at slac to upgrade lsst-dev testbed
  • researched InfiniDB
  • mtg about SciDB
  • discussions with ICRAR

xldb

  • discussions with and about Ingres/VectorWise?
  • published use cases on xldb wiki
  • arranging location for serving science benchmark input data
  • setup xldb doe advisory board
  • finalizing report publishing with codata Data Science Journal

scidb

  • discsussions about licenses
  • public outreach: announcements, website updates

Daues Greg

Reported 01/21/2010

IDPercentTaskComments
MWI65650%Pipeline shutdown mechanisms
MWI6640%Freeze-drying clipboards: Harness supportnot started
MWI1060%Work to support communication of general serializable types over MPI not started
MWI1070%Develop mechanisms for identifying/accessing not started
MWI1130%Improving harness exception handlingnot started
MWI660100%*Improve simple, interactive harness execution: infrastructure
MWI70925%Improve simple, interactive harness execution: interface

*Percent updated in this report

The fine print:

  • The harness has been successfully separated into two packages pex_harness and pex_mpiharness. The new pex_harness does not depend on MPI and runs pipelines on a single node with Python threads. The completion of this package gives the 100% for MWI660.
  • Shutdown mechanism for our levels 1--4 are complete; a level 5 corresponding to "there is no more data" remains.

Comments:

Other Activities

Hanushevsky Andy

Reported 01/21/10

IDPercentTaskComments
MWI16540%*support persistence and retrieval of deep detection products and inputsrewrote sections of xrootd for qserv (alpha version)

*Percent updated in this report

Comments:

Other Activities

Lim KT

Reported xx/xx/xx

IDPercentTaskComments
MWI6910%Partitioning: design partitioning of MovingObject? tablenot started
MWI6980%Task manager: multiple queriesnot started
MWI6990%Task manager: interations with usernot started
MWI7020%Documentationnot started
MWI1600%capturing full SW/HW infonot started
MWI1590%demonstrating re-creation of Calibrated Science Images not started
MWI16540%*support persistence and retrieval of deep detection products and inputs
MWI4650%implement objectId synchronization between data releases (not DC3)not started
MWI1800%persistence framework refinementnot started
MWI6430%Upgrade OutputStage? to register outputsnot started
MWI6440%Implement simple, non-Policy retrievalnot started
MWI6450%App Class: remove need to declare friend classesnot started
MWI6460%App Class: remove intrusiveness of Boost persistencenot started
MWI6470%App Class: legacy file-based persistence/retrievalnot started
MWI6480%Python pers: investigate efficiency of methodsnot started
MWI6490%Python pers: define app class interfacenot started
MWI6500%Implement simple non-Policy persistencenot started
MWI6510%Upgrade InputStage?not started
MWI6520%Generic ORM: mapping attributes to columnsnot started
MWI6530%Generic ORM: collection persistence and retrievalnot started
MWI6630%Freeze-drying clipboards: designnot started
MWI6690%Determine segment images over a given sky tilenot started
MWI6700%LRU cache for raw segment imagesnot started
MWI6710%LRU cache for calibrated science exposuresnot started
MWI6720%Write/retrieve images to disk/tapenot started
MWI6730%Determine, retrieve cutout areas over given sky tilenot started
MWI6750%Write to co-add template pixel store on sky tile basisnot started
MWI6740%Determine template pixels overlaying given coordsnot started
MWI6760%Retrieve template pixelsnot started
MWI6770%Investigate local disk use for postage stamp generationnot started

*Percent updated in this report

Comments:

Other Activities

Monkewitz Serge

Reported xx/xx/xx

IDPercentTaskComments
MWI69025%Partitioning: research htm/healpix/stomp/dif
MWI6920%Implement generic query partitionernot started
MWI69375%Tools for partitioning data
MWI6960%DC3b syntax parser and aggregationnot started
MWI4650%implement objectId synchronization between data releases (not DC3)not started

*Percent updated in this report

Comments:

Other Activities

Pietrowicz Steve

Reported 01/20/10

IDPercentTaskComments
MWI66550%Expose Event API
MWI22825%Event system failover
MWI6580%I/O stage for interactive pipeline launchnot started
MWI65950%Refactor Orca
MWI5380%Event Monitor integrationnot started
MWI5890%Pipeline Fault Tolerancenot started
MWI5700%Events orchestration layer supportnot started
MWI6660%Scripts to monitor for node/pipeline failuresnot started
MWI6670%Python-based high-level event monitor scriptsnot started

*Percent updated in this report

Comments:

Continuing work to add additional functionality, which falls under "refactor Orca". "Vanilla" Condor jobs with Dagman submissions are now working on Abe, as well as regular (ie, DC2 style) submissions to the LSST Cluster. Still need to integrate and test with the new pex_harness.

Other Activities

Plante Ray

Reported 1/21/10

IDPercentTaskComments
MWI62475%User preferences directory
MWI62575%Support user/server tags worked with Lupton on various bug fixes
MWI6260%Improve packages installation/declaration started in on this.
MWI62725%Improve EUPS Docs
MWI6280%Meta releasesnot started
MWI6290%User publishing of packages to servernot started
MWI6310%As-built manifests into install directoriesnot started
MWI6320%Make SConsUtils dependency on EUPS optionalnot started
MWI6330%Upgrade SConsUtils for latest Sconsnot started
MWI6410%Logging tasksnot started

*Percent updated in this report

Comments:

Other Activities

  • Participated in NSF site review of FDP proposal (Tucson, 12/16-17)
  • Developed test pipeline for PSFMatch
  • Co-wrote Teragrid proposal for DC3b allocation

Wang Daniel

Reported xx/xx/xx

IDPercentTaskComments
MWI16540%*support persistence and retrieval of deep detection products and inputsOngoing. Working on scalability issues at 1B objects on 22 nodes.

*Percent updated in this report

Comments:

Other Activities



Objectives for the Next Period

What you expect to accomplish.

NCSA

LSSTC

Database Group



Problems Encountered and Solutions Being Pursued

Budget or schedule variance, technical issues, management issues.

NCSA

LSSTC

Database Group