Previous month Apps Middleware/Infrastructure - Current Apps - Next month Apps Middleware/Infrastructure
Middleware/Infrastructure Status Report for January 15, 2010
Guide on Reporting Percentage Completed:
For software development tasks,
- use 0% if the solution is still being designed;
- use 25% if the solution's design is completed;
- use 50% if the solution is implemented and tested within the Ticket branch;
- use 75% if the implementation review is complete, the Ticket branch has been merged onto the Trunk branch, and the Ticket is closed.
- use 100% if the affected Trunk packages are tagged and released.
For other tasks, still use these increments, but apply their meaning as best as possible.
Note: Please place an asterisk (*) after the percent if you updated this value in this report.
Baker Bill
Reported 1/20/2010
| ID | Percent | Task | Comments |
| MWI569 | 20%* | Maintain automated testing structure | Ongoing. This month: improved support for builds of trunk packages vs. current releases, plus trunk vs. trunk where necessary. |
| MWI41 | 25%* | (BB) support distributable VMs for new developers | New releases once stack is fixed. Ray wants them ready for the JWST workshop Jan 25-26. |
| MWI74 | 50% | configure test sytem for automated V&V | |
| MWI637 | 25%* | Automated standards checking script | Developing web interface for automated standards checking with Parasoft. |
| MWI638 | 0% | Buildbot to detect ABI changes on trunk | not started |
| MWI661 | 0% | Advanced interslice communications support | not started |
| MWI662 | 0% | Machine-readable inputs/outputs manifest | not started |
| MWI707 | 0% | Automating nightly DC3a run | not started |
*Percent updated in this report
Comments:
Other Activities
Becla Jacek
Reported 01/21/10
| ID | Percent | Task | Comments |
| MWI680 | 75% | Updates: source-related tables, Object table | |
| MWI681 | 0% | Updates: exposure-related tables | not started |
| MWI682 | N/A* | Synthetic sources of data | Per Tim (DataAccWG telecon Jan 15, 2010) this is not needed for DC3b. Planning to do after DC3b.| |
| MWI684 | 75% | Standardizing names | |
| MWI691 | 0% | Partitioning: design partitioning of MovingObject? table | not started |
| MWI693 | 75% | Tools for partitioning data | |
| MWI697 | 25% | Task manager: design presentation | |
| MWI700 | 25% | Benchmarking | |
| MWI701 | 25%* | Data Release tools | |
| MWI702 | 25%* | Documentation | |
| MWI160 | 0% | capturing full SW/HW info | not started |
| MWI159 | 0% | demonstrating re-creation of Calibrated Science Images | not started |
| MWI165 | 40%* | support persistence and retrieval of deep detection products and inputs | |
| MWI465 | 0% | implement objectId synchronization between data releases (not DC3) | not started |
*Percent updated in this report
Comments:
Other Activities
- discussing and documenting sql query API restrictions
- rewriting queries in trac
- db schema improvements
- schema browser tweaks
- attended nsf review
- started work on planning lsst data access construction at slac
- discussions at slac to upgrade lsst-dev testbed
- researched InfiniDB
- mtg about SciDB
- discussions with ICRAR
xldb
- discussions with and about Ingres/VectorWise
- published use cases on xldb wiki
- arranging location for serving science benchmark input data
- setup xldb doe advisory board
- finalizing report publishing with codata Data Science Journal
scidb
- discsussions about licenses
- public outreach: announcements, website updates
Daues Greg
Reported 01/21/2010
| ID | Percent | Task | Comments |
| MWI656 | 50% | Pipeline shutdown mechanisms | |
| MWI664 | 0% | Freeze-drying clipboards: Harness support | not started |
| MWI106 | 0% | Work to support communication of general serializable types over MPI | not started |
| MWI107 | 0% | Develop mechanisms for identifying/accessing | not started |
| MWI113 | 0% | Improving harness exception handling | not started |
| MWI660 | 100%* | Improve simple, interactive harness execution: infrastructure | |
| MWI709 | 25% | Improve simple, interactive harness execution: interface |
*Percent updated in this report
The fine print:
- The harness has been successfully separated into two packages pex_harness and pex_mpiharness. The new pex_harness does not depend on MPI and runs pipelines on a single node with Python threads. The completion of this package gives the 100% for MWI660.
- Shutdown mechanism for our levels 1--4 are complete; a level 5 corresponding to "there is no more data" remains.
Comments:
Other Activities
Hanushevsky Andy
Reported 01/21/10
| ID | Percent | Task | Comments |
| MWI165 | 40%* | support persistence and retrieval of deep detection products and inputs | rewrote sections of xrootd for qserv (alpha version) |
*Percent updated in this report
Comments:
Other Activities
Lim KT
Reported xx/xx/xx
| ID | Percent | Task | Comments |
| MWI691 | 0% | Partitioning: design partitioning of MovingObject? table | not started |
| MWI698 | 0% | Task manager: multiple queries | not started |
| MWI699 | 0% | Task manager: interations with user | not started |
| MWI702 | 0% | Documentation | not started |
| MWI160 | 0% | capturing full SW/HW info | not started |
| MWI159 | 0% | demonstrating re-creation of Calibrated Science Images | not started |
| MWI165 | 40%* | support persistence and retrieval of deep detection products and inputs | |
| MWI465 | 0% | implement objectId synchronization between data releases (not DC3) | not started |
| MWI180 | 0% | persistence framework refinement | not started |
| MWI643 | 0% | Upgrade OutputStage? to register outputs | not started |
| MWI644 | 0% | Implement simple, non-Policy retrieval | not started |
| MWI645 | 0% | App Class: remove need to declare friend classes | not started |
| MWI646 | 0% | App Class: remove intrusiveness of Boost persistence | not started |
| MWI647 | 0% | App Class: legacy file-based persistence/retrieval | not started |
| MWI648 | 0% | Python pers: investigate efficiency of methods | not started |
| MWI649 | 0% | Python pers: define app class interface | not started |
| MWI650 | 0% | Implement simple non-Policy persistence | not started |
| MWI651 | 0% | Upgrade InputStage? | not started |
| MWI652 | 0% | Generic ORM: mapping attributes to columns | not started |
| MWI653 | 0% | Generic ORM: collection persistence and retrieval | not started |
| MWI663 | 0% | Freeze-drying clipboards: design | not started |
| MWI669 | 0% | Determine segment images over a given sky tile | not started |
| MWI670 | 0% | LRU cache for raw segment images | not started |
| MWI671 | 0% | LRU cache for calibrated science exposures | not started |
| MWI672 | 0% | Write/retrieve images to disk/tape | not started |
| MWI673 | 0% | Determine, retrieve cutout areas over given sky tile | not started |
| MWI675 | 0% | Write to co-add template pixel store on sky tile basis | not started |
| MWI674 | 0% | Determine template pixels overlaying given coords | not started |
| MWI676 | 0% | Retrieve template pixels | not started |
| MWI677 | 0% | Investigate local disk use for postage stamp generation | not started |
*Percent updated in this report
Comments:
Other Activities
Monkewitz Serge
Reported xx/xx/xx
| ID | Percent | Task | Comments |
| MWI690 | 25% | Partitioning: research htm/healpix/stomp/dif | |
| MWI692 | 0% | Implement generic query partitioner | not started |
| MWI693 | 75% | Tools for partitioning data | |
| MWI696 | 0% | DC3b syntax parser and aggregation | not started |
| MWI465 | 0% | implement objectId synchronization between data releases (not DC3) | not started |
*Percent updated in this report
Comments:
Other Activities
Pietrowicz Steve
Reported 01/20/10
| ID | Percent | Task | Comments |
| MWI665 | 50% | Expose Event API | |
| MWI228 | 25% | Event system failover | |
| MWI658 | 0% | I/O stage for interactive pipeline launch | not started |
| MWI659 | 50% | Refactor Orca | |
| MWI538 | 0% | Event Monitor integration | not started |
| MWI589 | 0% | Pipeline Fault Tolerance | not started |
| MWI570 | 0% | Events orchestration layer support | not started |
| MWI666 | 0% | Scripts to monitor for node/pipeline failures | not started |
| MWI667 | 0% | Python-based high-level event monitor scripts | not started |
*Percent updated in this report
Comments:
Continuing work to add additional functionality, which falls under "refactor Orca". "Vanilla" Condor jobs with Dagman submissions are now working on Abe, as well as regular (ie, DC2 style) submissions to the LSST Cluster. Still need to integrate and test with the new pex_harness.
Other Activities
Plante Ray
Reported 1/21/10
| ID | Percent | Task | Comments |
| MWI624 | 75% | User preferences directory | |
| MWI625 | 75% | Support user/server tags | worked with Lupton on various bug fixes |
| MWI626 | 0% | Improve packages installation/declaration | started in on this. |
| MWI627 | 25% | Improve EUPS Docs | |
| MWI628 | 0% | Meta releases | not started |
| MWI629 | 0% | User publishing of packages to server | not started |
| MWI631 | 0% | As-built manifests into install directories | not started |
| MWI632 | 0% | Make SConsUtils dependency on EUPS optional | not started |
| MWI633 | 0% | Upgrade SConsUtils for latest Scons | not started |
| MWI641 | 0% | Logging tasks | not started |
*Percent updated in this report
Comments:
Other Activities
- Participated in NSF site review of FDP proposal (Tucson, 12/16-17)
- Developed test pipeline for PSFMatch
- Co-wrote Teragrid proposal for DC3b allocation
Wang Daniel
Reported xx/xx/xx
| ID | Percent | Task | Comments |
| MWI165 | 40%* | support persistence and retrieval of deep detection products and inputs | Ongoing. Working on scalability issues at 1B objects on 22 nodes. |
*Percent updated in this report
Comments:
Other Activities
Objectives for the Next Period
What you expect to accomplish.
NCSA
LSSTC
Database Group
Problems Encountered and Solutions Being Pursued
Budget or schedule variance, technical issues, management issues.
NCSA
LSSTC
Database Group
