wiki:Report021509

Version 14 (modified by dgehrig, 11 years ago) (diff)

--

Status Report for February 15, 2009



Summary of progress

Work activity; whether it reached associated milestones.

NCSA

Bill (submitted on 2/16/2009)

  • Automated build system is much more mature
    • Trunk build is being tested properly (which in this case means that the build still fails for some packages)
    • Automatic incremental build of trunk on checkins, delayed by 1/2 hour after last checkin
  • Working on security document for PDR (finished any day now ...)
  • Security support -- installed one-time-password (OTP) for admin access to more LSST infrastructure machines at NCSA
  • Worked on new release of VMs with LSST stack pre-installed, for download by developers
    • Ray will actually install stack, and then I will release the VM images for download

Chris (submitted on 2/xx/2009)

David (submitted on 2/24/2009)

  • Working with Jeff and Suzy on DC3a project plan
  • Downloaded template data from CFHT Deep survey for DC3a
  • Created directory area and structure for DC3a test data
  • Worked with Steven Lo of IPAC over network latency issues for integration week
  • Maintaining trac, svn, cluster (ongoing)

Greg (submitted on 2/23/2009)

  • Transition of pex_harness to new middleware completed.
  • Changes for support of localLogMode in the harness completed; ticket 343 ready for review.
  • Updated/cleaned up examples within pex_harness, adding a "template" example that can serve as a starting point for testing a single Stage in a simple manner within a Pipeline.
  • Adding doxygen documentation for Slice/Pipeline? as part of comemnts within 343 review.
  • Organizing changes to Clipboard needed for ticket 222; nearly complete.
  • Some development completed on a "sliceleaders" topology that may be needed for support of WCS use case.

Steve (submitted on 2/13/2009)

  • Worked with Greg and Ray on Orchestration Layer design.
  • Added the preliminary implementation of Orchestration into SVN. Work continues on this.

Ray (submitted on 2/18/2009)

  • Worked with Steve on Orchestration Layer design
  • Maintained distribution server, tested/updated numerous packages related to #458
  • #485: got DC2 pipelines running successfully under the DC3 stack
    • saw a factor of two improvement in overall performance
  • Travel: Science and DM meeting 2/9-10 (UC Davis)
  • updated Infrastructure Costing model changes:
    • worked Kantor, Cribbs, Freemon to integrate internal review changes
    • working with Becla, Lin on addressing database performance requirements
  • progress on tickets:
Task %Done* Task Title Comments
28/39 90% update distribution server little bit of clean up left
114/111 75% orchestration layer design Worked more with Steve, Greg
84/535 25% integrate logging and Trace Just need to finish tests
78/532 25% PropertySet? Integration into Policy actual implementation is 80% done


LSSTC

Robyn (submitted on 2/13/2009)

Note: Only my tasks, not listed on the most recent Project Schedule as 100% completed, are included.

Status NL Not Lead
Status NW No work done
Id PM Project Management Plan
Id DC Data Challenge Plan
Id AP Applications Plan
Id MW Middleware Plan


.Status.Id...........Title.........................Comment..............
NL NW PM50 ...Update SysML/UML models and prepare Technical Documents
NL NW PM51 ......Complete LSST Software Development Plan Waiting on Review
NL NW PM51 .........LSST Software Development Plan
NL NW PM52 .........LSST Configuration Management Guidelines
NL NW PM53 .........LSST Software Quality Assurance Guidelines
NL NW PM54 .........LSST Verification and Validation Guidelines
NL NW PM60 ...Complete DM Inputs to System Level Requirements and Design
NL NW PM67 ......Create document templates, recipes, tools for Enterprise Architect output/gen
NL NW PM68 ...Complete Data Management Reference Design
NL NW PM69 ......Clean up DC3 UML Model, Update LSST Reference Design UML Model
NL NW PM70 ......Update EA and UML Modeling Conventions Document for Activity Diagrams
PM75 ...Create draft DM Integration Test Plan
0% PM77 ......Generate test cases Need TestGen? update
NL NW PM78 Complete PM Control System Updates
NL NW PM81 ...Publish cost book, do team review
NL NW PM85 ...Conduct PM Internal Cost Review and updates
NL NW PM86 Complete DM Inputs to Risk Management, System Integration, Commissioning, and Operations Plans
0% PM87 ...Update DMS Integration and Test Plan
PM90 ...Contribute DM sections to LSST Risk Management, Commissioning, I&T, and Operations Plans
DC56 Data Challenge 3 Software Development Plan
100% DC57 ...DC3 Release Project Plan
50% DC58 ...DC3 Configuration Management Plan Need feedback from Gehrig, Baker, Plante
100% DC59 ...DC3 Software Quality Assurance Plan
DC60 ...DC3 Verification and Validation Plan
75% ......DC3a Verification and Validation Plan Close but some sections missing
0% ......DC3b Verification and Validation Plan
100% MW6 Software Quality Assurance
100% MW7 ...Acquire SQA Support tools
100% MW8 ......Configurable tool to perform Lsst software standards checking
100% MW9 .........C++ LSST benchmark test suite packaged
100% MW10 .........Benchmark
MW11 superfluous - drop from plan
MW12 superfluous - drop from plan
MW13 superfluous - drop from plan
100% MW14 .........Acquire
100% MW15 ......Coverage Analysis Tool
100% MW16 .........Benchmark
100% MW17 .........Release for developer use
100% MW18 ...Refine workflow process to incorporate additional V&V...

Database Group (submitted on 2/xx/2009)

K-T (submitted on 2/xx/2009)

Andy H (submitted on 2/xx/2009)

  • running GridSQL tests on slac 3-machine testbed

Serge (submitted on 2/xx/2009)

  • helped debug SWIG issues
  • helped get stack to build on gcc 4.3
  • bug fixes to doxygen builder in sconsUtils
  • updated MOPS formatter code to work with new exceptions/PropertySet
  • began work on updating AP code to work with new exceptions/PropertySet

Daniel (submitted on 2/13/2009)

  • just joined slac (January 28)
  • reading up, setting up various accounts
  • scidb
    • working on scidb prototype (udfs)

Jacek (submitted on 2/13/2009)

  • storage estimates, mainly query IO
  • database setup scripts improved
  • discussed large scale tests
  • paperwork/logistics related to hiring (Daniel hired!)
  • discussion with slac office of tech transfer re open source licenses for lsst
  • removed unused columns for dc3a from db schema
  • discussions with database people from ASKAP / radio astronomy
  • meetings:
    • FRS meeting + db related follow ups
    • telecon with mysql cofounder, discussed mysql future, drizzle
    • several DataAccWG, DC3, infrastructure
  • scidb
    • telecon with pnnl on open provenance
    • provided various texts for scidb.org website
    • working with sciAdvisors on collecting input for scidb.org
    • met with PNNL dir for Tech Deployment and Outreach
    • connecting with Univ of Utah / geophysics / oil and gas research
    • connecting with UCSD
    • steering telecon and vldb demo planning
  • xldb
    • finalized xldb3 conf room in Lyon, France
  • other potentially relevant
    • visited pnnl (slac-pnnl seminar)


Major Accomplishments

Significant breakthroughs, issues resolved.



Objectives for the Next Period

What you expect to accomplish.

NCSA

  • Bill:
    • Finish security document (finally)
    • Support DC3a integration as necessary
    • Push out VM images for download
  • Chris:

  • David:
    • Continue downloading DC3a test data for preparation by Andy B
    • Make sure test data is in one consistent place with documented directory structure
  • Steve:
    • Finalize Orchestration layer framework for DC3a integration week

  • Greg:

  • Mike:
  • Ray:


LSSTC

  • Robyn:
    • Complete the DC3a Integration Test Plan
    • Ensure misc input needed for Integration Test are available prior to Integration Week.
    • Cajole pipeline developers to implement single slice pipeline test harness for their pipeline.


Database Group



Problems Encountered and Solutions Being Pursued

Budget or schedule variance, technical issues, management issues

  • Robyn is nudging the cross-group preparation of the CFHT and simulated LSST datasets for DC3a use.