wiki:MWInfReport041510
Last modified 9 years ago Last modified on 04/24/2010 01:41:22 PM

Previous month Apps Middleware/Infrastructure - Current Apps - Next month Apps Middleware/Infrastructure

Middleware/Infrastructure? Status Report for April 15, 2010

Guide on Reporting Percentage Completed:

For software development tasks,

  • use 0% if the solution is still being designed;
  • use 25% if the solution's design is completed;
  • use 50% if the solution is implemented and tested within the Ticket branch;
  • use 75% if the implementation review is complete, the Ticket branch has been merged onto the Trunk branch, and the Ticket is closed.
  • use 100% if the affected Trunk packages are tagged and released.

For other tasks, still use these increments, but apply their meaning as best as possible.

Note: Please place an asterisk (*) after the percent if you updated this value in this report.

Allsman Robyn

Reported xx/xx/xx

IDPercentTaskComments
MWI8270%prototype buildbot script to run pipeline unittest not started

*Percent updated in this report

Comments:

Other Activities

Baker Bill

Reported 4/19/10

IDPercentTaskComments
MWI56920%Maintain automated testing structure
MWI4125%Support distributable VMs for new developers
MWI7450%configure test sytem for automated V&V
MWI63725%Automated standards checking script
MWI6380%Buildbot to detect ABI changes on trunknot started
MWI6610%Advanced interslice communications supportnot started
MWI6620%Machine-readable inputs/outputs manifestnot started
MWI70725%Automating nightly DC3b run

*Percent updated in this report

Comments:

Other Activities

  • Work on pex_policy
    • bug fixes: #1210 - included in 3.4.4
      • improve handling of sub-dictionaries
      • push more validation to construction instead of at use time
    • load Policy by URN: #1035 - future version (3.5?)
      • format is urn:eupspkg:<package>[:<repository>]:<path>
      • implementation is about 90% complete
  • Buildbot
    • Working on a trunk vs. trunk build
  • Long-term
    • developing web interface to Parasoft CppTest? for sharing among developers

Becker Andy

Reported xx/xx/xx

IDPercentTaskComments
MWI8380%Chop CFHT data by ampnot started

*Percent updated in this report

Comments:

Other Activities

Becla Jacek

Reported 04/16/10

IDPercentTaskComments
MWI68175%*Updates: exposure-related tables waiting for Apps team for input
MWI69120%Partitioning: design partitioning of MovingObject? table
MWI69375%Tools for partitioning data
MWI83160%qserv: MyProxy?
MWI83225%*DB backup and replication
MWI70025%Benchmarking
MWI16025%capturing full SW/HW info
MWI1590%demonstrating re-creation of Calibrated Science Images not started
MWI16541%support persistence and retrieval of deep detection products and inputs
MWI46540%implement objectId synchronization between data releases (not DC3)

*Percent updated in this report

Comments:

Other Activities

  • attended mysql conf, debrief sent to lsst-data
  • SAT schema review
  • schema updates (exposure tables, provenance)
  • coordinating installing reddnet machines at slac
  • qserv discussions about configuration
  • telecon about Gator
  • imSim planning
  • xldb
    • website contents
    • put together organizing committee
    • draft budget and other docs for DOE (got the approval from DOE!)
  • scidb
    • wrote/submitted ldrd proposal
  • 1 week vacation

Daues Greg

Reported xx/xx/xx

IDPercentTaskComments
MWI65650%Pipeline shutdown mechanisms
MWI6640%Freeze-drying clipboards: Harness supportnot started
MWI1060%Work to support communication of general serializable types over MPI not started
MWI1070%Develop mechanisms for identifying/accessing not started
MWI11350%*Improving harness exception handling
MWI70950%*Improve simple, interactive harness execution: interface

*Percent updated in this report

Comments:

  • For MWI113, within ticket 1187 support for a failure Stage (that is executed when an exception is caught in the Pipeline or a Slice) has been added.
  • For MWI656, within ticket 1187 shutdown mechanisms were made more robust.
  • For MWI709, Daues extended the datarel package with harness-level pipeline execution sets (adding to the original orca-level sets) for test pipelines SFM and CrSplit. This activity was undertaken during Integration week at IPAC.

Other Activities

Freemon Mike

Reported xx/xx/xx

IDPercentTaskComments
MWI8490%Prepare LSST10 for primary catalog databasenot started

*Percent updated in this report

Comments:

Other Activities

Gehrig David

Reported xx/xx/xx

IDPercentTaskComments
MWI72325%*Image file access server ready
MWI72425%*Unify account management procedures - svn mysql linux svn
MWI72290%Prepare LSST2 for iRODS
MWI84830%Prepare LSST1 as web data server / schema browser
MWI8490%Prepare LSST10 for primary catalog databasenot started
MWI83750%*Download CFHT Data

*Percent updated in this report

Comments:

Other Activities

  • Updated MWI project plan reflecting design review scoping changes

Hanushevsky Andy

Reported xx/xx/xx

IDPercentTaskComments
MWI16541%support persistence and retrieval of deep detection products and inputs

*Percent updated in this report

Comments:

Other Activities

Lim KT

Reported xx/xx/xx

IDPercentTaskComments
MWI69120%Partitioning: design partitioning of MovingObject? table
MWI16025%capturing full SW/HW info
MWI1590%demonstrating re-creation of Calibrated Science Images not started
MWI16541%support persistence and retrieval of deep detection products and inputs
MWI46540%implement objectId synchronization between data releases (not DC3)
MWI1800%persistence framework refinementnot started
MWI6430%Upgrade OutputStage? to register outputsnot started
MWI6440%Implement simple, non-Policy retrievalnot started
MWI6450%App Class: remove need to declare friend classesnot started
MWI6460%App Class: remove intrusiveness of Boost persistencenot started
MWI6470%App Class: legacy file-based persistence/retrievalnot started
MWI6480%Python pers: investigate efficiency of methodsnot started
MWI6490%Python pers: define app class interfacenot started
MWI6500%Implement simple non-Policy persistencenot started
MWI6510%Upgrade InputStage?not started
MWI6520%Generic ORM: mapping attributes to columnsnot started
MWI6530%Generic ORM: collection persistence and retrievalnot started
MWI6630%Freeze-drying clipboards: designnot started
MWI6730%Determine, retrieve cutout areas over given sky tilenot started
MWI6750%Write to co-add template pixel store on sky tile basisnot started
MWI6740%Determine template pixels overlaying given coordsnot started
MWI6760%Retrieve template pixelsnot started
MWI6770%Investigate local disk use for postage stamp generationnot started

*Percent updated in this report

Comments:

Other Activities

Monkewitz Serge

Reported xx/xx/xx

IDPercentTaskComments
MWI69025%Partitioning: research htm/healpix/stomp/dif
MWI6920%Implement generic query partitionernot started
MWI69375%Tools for partitioning data
MWI6960%DC3b syntax parser and aggregationnot started
MWI46540%implement objectId synchronization between data releases (not DC3)
MWI669100%Determine segment images over a given sky tiledone (for the PT1 sky-tiling scheme)
? 75%Spatial support for qserv and corresponding MySQL UDFsStill need to implement UDFs

*Percent updated in this report

Comments:

Other Activities

Pietrowicz Steve

Reported xx/xx/xx

IDPercentTaskComments
MWI6580%I/O stage for interactive pipeline launchnot started
MWI65950%Refactor Orca
MWI5380%Event Monitor integrationnot started
MWI5890%Pipeline Fault Tolerancenot started
MWI5700%Events orchestration layer supportnot started

*Percent updated in this report

Comments:

this activity isn't listed above: 50% Worked on implementing Event typing. This is currently in the trunk and is being tested with the Job Office.

Other Activities

Plante Ray

Reported xx/xx/xx

IDPercentTaskComments
MWI62475%User preferences directory
MWI62575%Support user/server tags
MWI6260%Improve packages installation/declarationnot started
MWI62725%Improve EUPS Docs
MWI6280%Meta releasesnot started
MWI6290%User publishing of packages to servernot started
MWI6310%As-built manifests into install directoriesnot started
MWI6320%Make SConsUtils dependency on EUPS optionalnot started
MWI6330%Upgrade SConsUtils for latest Sconsnot started
MWI6410%Logging tasksnot started
MWI84025%Update and test stack for latest product versions
MWI8413%Job scheduling via JobOffice/Blackboard?
MWI6700%LRU cache for raw segment imagesnot started
MWI6710%LRU cache for calibrated science exposuresnot started
MWI6720%Write/retrieve images to disk/tapenot started
MWI72550%Define directory structure
MWI8480%Prepare LSST1 as web data server / schema browsernot started

*Percent updated in this report

Comments:

Other Activities

Wang Daniel

Reported 04/20/10

IDPercentTaskComments
MWI16541%support persistence and retrieval of deep detection products and inputsOngoing. Finishing up parsing/front-end qserv prototype.

*Percent updated in this report

Comments:

  • qserv development.
  • Add new ANTLR grammar for query parser, which should be compliant with all (?) SQL92 syntax. Add rules and code to process column and table names. Support table remapping, depending on whether chunks or subchunks are available, and whether subchunking is needed for a query.
  • Add tool for daemon-less testing of qserv.
  • Cleaned up build configuration.
  • Added hint string handling.
  • Integrated Serge's geometry code for partitioning.
  • Add configuration file facility for qserv.
  • Test near end-to-end query execution from submit query+hintstring to c++ async executor.
  • Finish two-file/url dispatch mechanism for lower latency and resource usage.
  • Add worker-side query throttling to prevent server crash.

Other Activities



Objectives for the Next Period

What you expect to accomplish.

NCSA

LSSTC

Database Group



Problems Encountered and Solutions Being Pursued

Budget or schedule variance, technical issues, management issues.

NCSA

LSSTC

Database Group