wiki:Report091509

Version 25 (modified by daues, 10 years ago) (diff)

--

Status Report for September 15, 2009

Previous Month Next Month


Guide on Reporting Percentage Completed:

For software development tasks,

  • use 0% if the solution is still being designed;
  • use 25% if the solution's design is completed;
  • use 50% if the solution is implemented and tested within the Ticket branch;
  • use 75% if the implementation review is complete, the Ticket branch has been merged onto the Trunk branch, and the Ticket is closed.
  • use 100% if the affected Trunk packages are tagged and released.

For other tasks, still use these increments, but apply their meaning as best as possible.

Note: Please place an asterisk (*) after the percent if you updated this value in this report.

The "ID" number corresponds to the "Unique ID" column in the Middleware/Infrastructure?/DB project plan, a PDF of which is attached at wiki:DC3bManagement.

Summary of progress

Work activity; whether it reached associated milestones.

Allsman Robyn

Reported xx/xx/xx

IDPercentTaskComments

*Percent updated in this report

Comments:

Other Activities

  • Started testing process/procedure/suitability of Automatic Coding Standards Checking when Tickets are designated as 'inStandardsCheck'.

Axelrod Tim

Reported xx/xx/xx

IDPercentTaskComments
6830%Defining unitsnot started

*Percent updated in this report

Comments:

Other Activities

Baker Bill

Reported 9/18/9

IDPercentTaskComments
56910%Maintain automated testing structureongoing
4175%*(BB) support distributable VMs for new developersongoing; latest VMs are ready & waiting for verification
7450%*configure test sytem for automated V&Vsoftware installed; Robyn is configuring
6370%Automated standards checking scriptnot started
6380%Buildbot to detect ABI changes on trunknot started
64025%*Validated policy filesimplementation mostly complete
6610%Advanced interslice communications supportnot started
6620%Machine-readable inputs/outputs manifestnot started

*Percent updated in this report

Comments:

Other Activities

Becla Jacek

Reported xx/xx/xx

IDPercentTaskComments
6800%Updates: source-related tables, Object tablediscussions with App team will start Sep 18
6810%Updates: exposure-related tablesdiscussions with App team will start Sep 18
6820%Synthetic sources of datanot started
68425%*Standardizing namesImplemented in schema, need to update formatters and test
6865%*Evaluate hadoopDBDiscussed hadoopDB in details with hadoopDB developers
68725%*MySQL: test scalability and architectrun some tests, still need to try at least binary bindings
6880%Decide xrootd versus gearmannot started
6890%Run near neighbor query on lsst10not started
6910%Partitioning: design partitioning of MovingObject? tablenot started
6930%Tools for partitioning datanot started
6970%Task manager: design presentationnot started
7000%Benchmarkingnot started
7010%Data Release toolsnot started
7020%Documentationnot started
1600%capturing full SW/HW infonot started
1590%demonstrating re-creation of Calibrated Science Images not started
1650%support persistence and retrieval of deep detection products and inputsnot started
4650%implement objectId synchronization between data releases (not DC3)not started

*Percent updated in this report

Comments:

Other Activities

LSST

  • organized 1 DataAccWG telecon
  • documented spec for testbed and DC3b hardware, see dbDC3bHardware?

VLDB/XLDB

  • prepared for XLDB, coordinated all logistics pre- and post-meeting (agenda, dinner, sponsors, badges, handouts, conf room arrangements, attaching photos and slides to website, mail list announcements, many others...)
  • attended VLDB, helped with the SciDB demo
  • organized XLDB3
  • all paperwork related to the trip
  • arranged article for "SLAC Today".
  • started writing xldb report

SciDB

  • organizing face to face SciDB developers meeting at SLAC (coordinating all logistics, invitations for international collaborators)
  • discussions with slac management about funding and future proposals
  • discussions with slac lawyers and CFO about bringing external funds to the lab
  • reaching out to geoscience, including abstract for AGU conference
  • meeting with CTO of Persistent Systems
  • meeting with VCs

Daues Greg

Reported xx/xx/xx

IDPercentTaskComments
65525%*Refactor harnessStage splitting design is done and now being implemented
6560%Pipeline shutdown mechanismsnot started
6640%Freeze-drying clipboards: Harness supportnot started
10675%Work to support communication of general serializable types over MPI
1070%Develop mechanisms for identifying/accessing not started
1130%Improving harness exception handlingnot started
66025%*Improve simple, interactive harness execution: infrastructureThread based Pipeline designed and now being implemented
7090%Improve simple, interactive harness execution: interfacenot started

*Percent updated in this report

Comments:

Other Activities

  • Stage splitting changes have been implemented in Pipeline and Slice. Two pex_harness examples have been transformed and tested, including tests of the NoOp functionality for the case where a serialClass or parallelClass is not provided.
  • In the context of simple, interactive harness execution, a Pipeline launching Slices using Python threads (no MPI) is working in a primitive form with the beginnings of thread synchronization (duplicating MPI barrier functionality with base Python Events)

Gehrig David

Reported 09/21/09

IDPercentTaskComments

*Percent updated in this report

Comments:

Other Activities

  • Worked with Suzi and Jeff re DC3b project plan
  • Received training in Primavera project planning/costing software
  • Upgraded Trac installation on dc33
  • Created anonymous access to SVN per Lupton's request
  • Additional account management for NCSA cluster
  • Vacation Sept. 10-11

Hanushevsky Andy

Reported xx/xx/xx

IDPercentTaskComments
1650%support persistence and retrieval of deep detection products and inputsnot started

*Percent updated in this report

Comments:

Other Activities

Laher Russ

Reported xx/xx/xx

IDPercentTaskComments
48230%C++
4810%Pythonnot started

*Percent updated in this report

Comments:

Other Activities

Lim KT

Reported xx/xx/xx

IDPercentTaskComments
6880%Decide xrootd versus gearmannot started
6910%Partitioning: design partitioning of MovingObject? tablenot started
6980%Task manager: multiple queriesnot started
6990%Task manager: interations with usernot started
7020%Documentationnot started
469*100%DC3 provenance capture
1600%capturing full SW/HW infonot started
1590%demonstrating re-creation of Calibrated Science Images not started
166*100%support persistence and retrieval of nightly pipeline products and inputs
1650%support persistence and retrieval of deep detection products and inputs Design started
4650%implement objectId synchronization between data releases (not DC3) Not DC3 task
1800%persistence framework refinement Design started
6430%Upgrade OutputStage? to register outputs Design started
6440%Implement simple, non-Policy retrieval Design started
6450%App Class: remove need to declare friend classes DC3b stretch goal
6460%App Class: remove intrusiveness of Boost persistence DC3b stretch goal
6470%App Class: legacy file-based persistence/retrieval DC3b stretch goal
6480%Python pers: investigate efficiency of methods DC3b stretch goal
6490%Python pers: define app class interface DC3b stretch goal
6500%Implement simple non-Policy persistence DC3b stretch goal
6510%Upgrade InputStage? DC3b stretch goal
6520%Generic ORM: mapping attributes to columns DC3b stretch goal
6530%Generic ORM: collection persistence and retrieval DC3b stretch goal
6630%Freeze-drying clipboards: design Design started
6690%Determine segment images over a given sky tile Prototype started
6700%LRU cache for raw segment imagesnot started
6710%LRU cache for calibrated science exposuresnot started
6720%Write/retrieve images to disk/tapenot started
6730%Determine, retrieve cutout areas over given sky tile Prototype started
6750%Write to co-add template pixel store on sky tile basis Prototype started
6740%Determine template pixels overlaying given coords Prototype started
6760%Retrieve template pixelsnot started
6770%Investigate local disk use for postage stamp generationnot started

*Percent updated in this report

Comments:

Other Activities

Monkewitz Serge

Reported xx/xx/xx

IDPercentTaskComments
6900%Partitioning: research htm/healpix/stomp/difnot started
6920%Implement generic query partitionernot started
6930%Tools for partitioning datanot started
6960%DC3b syntax parser and aggregationnot started
4650%implement objectId synchronization between data releases (not DC3)not started

*Percent updated in this report

Comments:

Other Activities

Pietrowicz Steve

Reported 09/15/09

IDPercentTaskComments
66575%Expose Event API
2280%Event system failovernot started
6660%Scripts to monitor for node/pipeline failuresnot started
6670%Python-based high-level event monitor scriptsnot started
6580%I/O stage for interactive pipeline launchnot started
65950% *Refactor Orcain branch, in testing phase
2290%life-support servicesnot started
5380%Event Monitor integrationnot started
5700%Events orchestration layer supportnot started

*Percent updated in this report

Comments:

Other Activities

Plante Ray

Reported xx/xx/xx

IDPercentTaskComments
623 100%* Summarize design for EUPS changes Reviewed 9/2; see summary
624 25%* User preferences directory Implementation is under way, 2/3 done
625 25%* Support user/server tags Implementation is under way, 2/3 done
626 0% Improve packages installation/declaration Design has been started
627 25%* Improve EUPS documentation Mostly in the form of in-line python docs; user docs will not start until after Oct. 1
628 0% Meta releases not started
629 0% User publishing of packages to server not started
631 0% As-built manifests into install directories not started
632 0% Make SConsUtils dependency on EUPS optional not started
633 0% Upgrade SConstUtils for latest SCons not started
641 0% Logging tasks Harness refactoring/scripting tasks depend on this; not started
654 100%* Update UML + docs for STage code usage Reviewed 9/9; see summary
111 100%* Fill out design for orchestration layer Done. see attached design documents to Orca refactor task page

*Percent updated in this report

Other Activities
  • assisted Gehrig with Data Challenge planning
  • ran two Middleware telecons
  • assisted Cribbs and Freemon with infrastructure cost spreadsheets
  • participated in SAT: assisted with DC3b scoping
  • assisted with hardware requirements/resource development

Other Activities

Wang Daniel

Reported xx/xx/xx

IDPercentTaskComments
6855%*Near neighbor query over 2 or 3 nodesImplementing parser updates (aliasing, stored procedure, subquery).
6880%Decide xrootd versus gearmannot started
6940%Basic generic query parser in placenot started
6950%Demonstrate joining Object table with non-partitioned tablesnot started
6960%DC3b syntax parser and aggregationnot started
6980%Task manager: multiple queriesnot started
6990%Task manager: interactions with usernot started
7010%Data Release toolsnot started
7020%Documentationnot started

*Percent updated in this report

Comments:

Other Activities

  • SciDB
    • Added no-nesting hack to cooking code
    • Diagnosed Q5 problems
  • Attended VLDB
    • Fixed/helped-fix some last minute issues with SciDB for demo
    • Staffed demo sessions, collected questions and feedback
  • Attended XLDB
    • Took photos,
    • Moderated geoscience session
    • Presented introduction to benchmark
  • Various travel paperwork

(Vacation 8/31-9/2, 9/14-9/15)

Major Accomplishments

Significant breakthroughs, issues resolved.



Objectives for the Next Period

What you expect to accomplish.

NCSA

Ray Plante

  • finishing Logging task
  • finish EUPS implementation changes

LSSTC

Database Group



Problems Encountered and Solutions Being Pursued

Budget or schedule variance, technical issues, management issues.

NCSA

Ray Plante

I am encouraging in our group greater use of Trac in the documentation of progress on tasks. Several of us are maintaining pages like the following as a place to gather notes, design issues, and progress:

LSSTC

Database Group