wiki:MWInfReport101509

Version 12 (modified by bbaker, 10 years ago) (diff)

--

Previous Month - Next Month Apps Middleware/Infrastructure

Status Report for October 15, 2009


Guide on Reporting Percentage Completed:

For software development tasks,

  • use 0% if the solution is still being designed;
  • use 25% if the solution's design is completed;
  • use 50% if the solution is implemented and tested within the Ticket branch;
  • use 75% if the implementation review is complete, the Ticket branch has been merged onto the Trunk branch, and the Ticket is closed.
  • use 100% if the affected Trunk packages are tagged and released.

For other tasks, still use these increments, but apply their meaning as best as possible.

Note: Please place an asterisk (*) after the percent if you updated this value in this report.

The "ID" number corresponds to the "Unique ID" column in the Middleware/Infrastructure?/DB project plan, a PDF of which is attached at wiki:DC3bManagement.

Allsman Robyn

Reported 15/Oct/2009

IDPercentTaskComments
MWI7100%Coverage Checker Release for developer use: Pythonnot started
MWI370%Create table of Science Data Analystsminimal progress
MWI725 75% * Initial Release Code Standards Check 17 modules checked and forwarded to developers; 3 non-C++; rest waiting on compilable trunk version requisite for Standards Checking
MWI7260%Final DC3b Code Standards Checknot started

*Percent updated in this report

Comments:

Other Activities

  • nudging holders of ancient Tickets to complete and or close them
  • reviewing the C++ Coding Standards in light of the infraction reports generated by the Standards checker. Preparing TCT proposal for revision of some Rules due to inconsistency between Rules, developer resistance to 'illogical' rules, etc.
  • using the EA Test Case Generator on the DC3 model to generate preliminary DC3b integration test case requirements.

Axelrod Tim

Reported xx/xx/xx

IDPercentTaskComments
MWI6830%Defining unitsnot started
MWI7440%Report scalability resultsnot started
MWI7310%DC3b Reportnot started

*Percent updated in this report

Comments:

Other Activities

Baker Bill

Reported 10/13/2009

IDPercentTaskComments
MWI56910%Maintain automated testing structureongoing; okay
MWI4175%(BB) support distributable VMs for new developers
MWI7450%configure test sytem for automated V&V
MWI6370%Automated standards checking scriptnot started
MWI6380%Buildbot to detect ABI changes on trunknot started
MWI64025%Validated policy filesin review
MWI6610%Advanced interslice communications supportnot started
MWI6620%Machine-readable inputs/outputs manifestnot started
MWI7070%Automating nightly DC3a runRay making bash script

*Percent updated in this report

Comments:

Other Activities

Becla Jacek

Reported 10/15/09

IDPercentTaskComments
MWI6800%Updates: source-related tables, Object tablediscussions started (Oct 6 and Oct 9 DataAcc telecons)
MWI6810%Updates: exposure-related tablesnot started
MWI6820%Synthetic sources of datanot started
MWI68425%Standardizing names
MWI6865%Evaluate hadoopDB
MWI68750%*MySQL: test scalability and architect
MWI688100%*Decide xrootd versus gearman
MWI689100%*Run near neighbor query on lsst10 dbSpatialJoinPerf?
MWI6910%Partitioning: design partitioning of MovingObject table started, see MOPSPartitioning
MWI6930%Tools for partitioning datanot started
MWI6970%Task manager: design presentationnot started
MWI7000%Benchmarkingnot started
MWI7010%Data Release toolsnot started
MWI7020%Documentationnot started
MWI1600%capturing full SW/HW infonot started
MWI1590%demonstrating re-creation of Calibrated Science Images not started
MWI1650%support persistence and retrieval of deep detection products and inputsnot started
MWI4650%implement objectId synchronization between data releases (not DC3)not started

*Percent updated in this report

Comments:

Other Activities

  • LSST
    • mysql performance/scalability tests for 1 billion row table, primarily to understand how to deal with the lsst reference catalog
    • debugging problems with mysql authorization (for buildbot tests), see http://bugs.mysql.com/bug.php?id=47843
    • multiple discussions related to scalable architecture
    • documented sql sub-set to be supported in DC3b dbScalableArchSQL
    • organized 3 DataAccWG telecons
  • XLDB
    • wrote draft XLDB report
  • SciDB
    • coordinating input from science about benchmark
    • coordinating involvement of 3 developers from Persistent Systems (via telecons/emails)
    • discussions about scidb documentation
    • met with eBay
    • two telecons with East Coast VCs
  • Others

Bosch Jim

Reported xx/xx/xx

IDPercentTaskComments
MWI6855%Near neighbor query over 2 or 3 nodes
MWI6880%Decide xrootd versus gearmannot started
MWI6940%Basic generic query parser in placenot started
MWI6950%Demonstrate joining Object table with non-partitioned tablesnot started
MWI6960%DC3b syntax parser and aggregationnot started
MWI6980%Task manager: multiple queriesnot started
MWI6990%Task manager: interations with usernot started
MWI7010%Data Release toolsnot started
MWI7020%Documentationnot started

*Percent updated in this report

Comments:

Other Activities

Cribbs Chris

Reported 10/23/09

IDPercentTaskComments
MWI7211000%Update NCSA cluster for Lustre kernel

*Percent updated in this report

Comments:

Other Activities

Daues Greg

Reported 10/22/09

IDPercentTaskComments
MWI65550%*Refactor harness
MWI6560%Pipeline shutdown mechanismsnot started
MWI6640%Freeze-drying clipboards: Harness supportnot started
MWI1060%Work to support communication of general serializable types over MPI not started
MWI1070%Develop mechanisms for identifying/accessing not started
MWI1130%Improving harness exception handlingnot started
MWI66025%Improve simple, interactive harness execution: infrastructure
MWI70925%*Improve simple, interactive harness execution: interface
MWI7480%Configure cluster for final releasenot started

*Percent updated in this report

Comments:

  • For MWI660 a large portion of Python thread based code has been merged with the code of MWI655 (for the new stage API) and is now being tested. Updated run/launching scripts are part of this code update, and so I have moved MWI709 off of "not started" to 25%

Other Activities

Dubois-Felsmann Gregory

Reported xx/xx/xx

IDPercentTaskComments
MWI7370%Identify hardward platformnot started

*Percent updated in this report

Comments:

Other Activities

Gehrig David

Reported 10/23/09

IDPercentTaskComments
MWI635100%*New ticket workflow

*Percent updated in this report

Comments:

Other Activities

Hanushevsky Andy

Reported xx/xx/xx

IDPercentTaskComments
MWI1650%support persistence and retrieval of deep detection products and inputsnot started

*Percent updated in this report

Comments:

Other Activities

Krughoff Simon

Reported xx/xx/xx

IDPercentTaskComments
MWI66575%Expose Event API
MWI2280%Event system failovernot started
MWI6660%Scripts to monitor for node/pipeline failuresnot started
MWI6670%Python-based high-level event monitor scriptsnot started
MWI6580%I/O stage for interactive pipeline launchnot started
0%life-support servicesnot started
MWI5380%Event Monitor integrationnot started
MWI5700%Events orchestration layer supportnot started

*Percent updated in this report

Comments:

Other Activities

Laher Russ

Reported xx/xx/xx

IDPercentTaskComments
MWI4810%Pythonnot started

*Percent updated in this report

Comments:

Other Activities

Lim KT

Reported xx/xx/xx

IDPercentTaskComments
MWI6880%Decide xrootd versus gearmannot started
MWI6910%Partitioning: design partitioning of MovingObject? tablenot started
MWI6980%Task manager: multiple queriesnot started
MWI6990%Task manager: interations with usernot started
MWI7020%Documentationnot started
MWI1600%capturing full SW/HW infonot started
MWI1590%demonstrating re-creation of Calibrated Science Images not started
MWI1650%support persistence and retrieval of deep detection products and inputsnot started
MWI4650%implement objectId synchronization between data releases (not DC3)not started
MWI1800%persistence framework refinementnot started
MWI6430%Upgrade OutputStage? to register outputsnot started
MWI6440%Implement simple, non-Policy retrievalnot started
MWI6450%App Class: remove need to declare friend classesnot started
MWI6460%App Class: remove intrusiveness of Boost persistencenot started
MWI6470%App Class: legacy file-based persistence/retrievalnot started
MWI6480%Python pers: investigate efficiency of methodsnot started
MWI6490%Python pers: define app class interfacenot started
MWI6500%Implement simple non-Policy persistencenot started
MWI6510%Upgrade InputStage?not started
MWI6520%Generic ORM: mapping attributes to columnsnot started
MWI6530%Generic ORM: collection persistence and retrievalnot started
MWI6630%Freeze-drying clipboards: designnot started
MWI6690%Determine segment images over a given sky tilenot started
MWI6700%LRU cache for raw segment imagesnot started
MWI6710%LRU cache for calibrated science exposuresnot started
MWI6720%Write/retrieve images to disk/tapenot started
MWI6730%Determine, retrieve cutout areas over given sky tilenot started
MWI6750%Write to co-add template pixel store on sky tile basisnot started
MWI6740%Determine template pixels overlaying given coordsnot started
MWI6760%Retrieve template pixelsnot started
MWI6770%Investigate local disk use for postage stamp generationnot started

*Percent updated in this report

Comments:

Other Activities

Lupton Robert

Reported xx/xx/xx

IDPercentTaskComments
MWI7380%Evaluate performance of ported algorithmnot started
MWI7400%Determine scope of datanot started
MWI7440%Report scalability resultsnot started

*Percent updated in this report

Comments:

Other Activities

Monkewitz Serge

Reported xx/xx/xx

IDPercentTaskComments
MWI6900%Partitioning: research htm/healpix/stomp/difnot started
MWI6920%Implement generic query partitionernot started
MWI6930%Tools for partitioning datanot started
MWI6960%DC3b syntax parser and aggregationnot started
MWI4650%implement objectId synchronization between data releases (not DC3)not started

*Percent updated in this report

Comments:

Other Activities

Pietrowicz Steve

Reported 10/15/09

IDPercentTaskComments
MWI65950%Refactor Orca in review
MWI66550% *Expose Event APIThis was incorrectly marked last month. This is pending design review.
MWI22825% *Event system failover
MWI6660%Scripts to monitor for node/pipeline failuresnot started
MWI6670%Python-based high-level event monitor scriptsnot started
MWI6580%I/O stage for interactive pipeline launchnot started
MWI2290%life-support servicesnot started
MWI5380%Event Monitor integrationnot started
MWI5700%Events orchestration layer supportnot started

*Percent updated in this report

Comments:

Other Activities

  • Working on fixing problems detected by code analysis tool.
  • Working on clearing tickets
  • Working on solution for standalone data areas for things that need them (like ActiveMQ), so users don't step on each other using a common scratch space.
  • Ticket tested fix, and released #959
  • Consulting with Tim and Francesco about ActiveMQ.

Plante Ray

Reported 10/14/09

IDPercentTaskComments
MWI62425%User preferences directory Implementation is complete. Because I have a few more unittests to add, I cannot yet close this ticket. Completion to 50% expected this week.
MWI62525%Support user/server tags Implementation is complete; a bit more testing is needed. Completion to 50% expected this week.
MWI6260%Improve packages installation/declaration most of this work is already done as part of above work, but I won't finish this off until MWI624, MWI625 are complete
MWI62725%Improve EUPS Docs
MWI6280%Meta releasesnot started
MWI6290%User publishing of packages to servernot started
MWI6310%As-built manifests into install directoriesnot started
MWI6320%Make SConsUtils dependency on EUPS optionalnot started
MWI6330%Upgrade SConsUtils for latest Sconsnot started
MWI6410%Logging tasksnot started
MWI7320%DC3b Production Runnot started
MWI7310%DC3b Reportnot started

*Percent updated in this report

Comments:

Other Activities

  • supported Freemon and Cribbs with Infrastructure cost sheet estimates
  • worked on sub-contract renewal
  • reviewed file management demo plan
  • code reviews

Resource_Names

Reported xx/xx/xx

IDPercentTaskComments
Text2Percent_CompleteName

*Percent updated in this report

Comments:

Other Activities

Science Collaboration

Reported xx/xx/xx

IDPercentTaskComments
MWI7240%Science Algorithm Verificationnot started
MWI7300%Science Algorithm Verificationnot started

*Percent updated in this report

Comments:

Other Activities

TBD

Reported xx/xx/xx

IDPercentTaskComments
MWI7390%Perform the initial portnot started
MWI7380%Evaluate performance of ported algorithmnot started
MWI7420%Run ported algorithmnot started
MWI7430%Collect metrics, assessnot started
MWI7440%Report scalability resultsnot started

*Percent updated in this report

Comments:

Other Activities

Major Accomplishments

Significant breakthroughs, issues resolved.



Objectives for the Next Period

What you expect to accomplish.

NCSA

LSSTC

Database Group



Problems Encountered and Solutions Being Pursued

Budget or schedule variance, technical issues, management issues.

NCSA

LSSTC

Database Group