wiki:PT1_2Report20110415

Version 31 (modified by jkantor, 8 years ago) (diff)

--

Table of Contents

    Previous month PT1.2 - Mgt/SE

    Current month Mgt/SE

    Next month PT1.2 - Mgt/SE

    Guide on Reporting Percentage Completed:

    For software development tasks:

    • use 0% if the solution is still being designed;
    • use 25% if the solution's design is completed;
    • use 50% if the solution is implemented and tested within the Ticket branch;
    • use 75% if the implementation review is complete, the Ticket branch has been merged onto the Trunk branch, and the Ticket is closed.
    • use 100% if the affected Trunk packages are tagged and released.

    For level of effort tasks, use % of effort expended rounded to nearest 5%:

    For other tasks, still use these increments, but apply their meaning as best as possible:

    • use 0% if the deliverable is still being designed;
    • use 25% if the deliverable's design is completed;
    • use 50% if the deliverable is implemented awaiting review/approval;
    • use 75% if the deliverable has been reviewed and in revision;
    • use 100% if the deliverable is approved and posted to appropriate repository.

    Note: Please place an asterisk (*) after the percent if you updated this value in this report.

    Task Start Finish Resources (lead first) % Complete Comments Last Updated
    Infrastructure Mike Freemon Recovered from a motherboard hardware failure on the database server; Received shipment of the new storage server and current conduction shakedown testing before putting into production; Two major activities are migrating DC3b data off of the SAN and physically relocating the development servers, both due to non-LSST requirements/deadlines 20110422
    Acquire and configure new storage to replace SAN 20110131 20110425 Pietrowicz Steve[5%] 75%* This machine has arrived, and is being integrated into our cluster. 20110419
    Relocate servers from ACB to NCSA building 20110425 20110627 Gehrig David[10%] 25%* Plan has been re-ordered owing to lack of SAN access in new room; we'll copy data, then move servers 20110322
    Environment and Tools Ray Plante
    TCT approval for environment/tool updates 20110309 20110309 Allsman Robyn[50%] 10% This is an on-demand task. 'cfitio' uprev was OKed 20110315
    Install, configure, publish environment/tool updates 20110201 20110315 Plante Ray[25%] 25% work continues 20110424
    Complete eups release 20110208 20110325 Plante Ray[25%] 25% work continues; hope to finish this month 20110424
    Update environment and tools documentation 20110325 20110330 Plante Ray[50%] 0% work continues 20110324
    Support trac, svn, systems administration 20110131 20110617 Gehrig David[15%] 50%* level-of-effort task 20110421
    ImSim Data Simon Krughoff
    Prepare and code to generate images and catalogs 20110131 20110405 Krughoff Simon[30%] *75% Implemented testing ongoing 20110421
    Test generation of images and catalogs 20110328 20110411 Krughoff Simon[30%] *75% Star images and reference catalogs tested. Delivering galaxies next week. 20110421
    Generate images and catalogs 20110411 20110509 Krughoff Simon[30%],Gibson Rob 0% 20110421
    Support ImSim data 20110509 20110606 Krughoff Simon[10%] 0% 20110421
    Import reference catalogs into Gator for comparison to output catalogs 20110509 20110511 Monkewitz Serge[50%] 0% 20110307
    Production-Pipeline-Stage Middleware Steve Pietrowicz
    Data Access Middleware and Database KT Lim Major butler implementation change complete 20110422
    Butler additions/fixes 20110131 20110324 Lim KT[50%] 100% 20110422
    Serialization and Persistence additions/fixes 20110214 20110325 Lim KT[50%] 25% 20110422
    Schema updates for galaxy photometry and others as needed 20110309 20110324 Becla Jacek[25%] 80%* To the best of my knowledge, the bulk of schema updates done, keeping at 80% to capture currently unforeseen future changes 20110422
    Spatial support (deploying udfs, indexes etc) 20110325 20110428 Becla Jacek[25%] 50%* Discussed with JHU, working on long term solution (merging JHU and our versions), Serge almost done with implementing HTM-bases solution that should easily handle current needs 20110422
    Improve schema documentation (udfs, stored functions/procedures) 20110429 20110526 Becla Jacek[25%] 0% This will get done once we deploy udfs on lsst10 20110422
    Redo db initialization (setup_perRunTables.sql etc) 20110527 20110607 Becla Jacek[25%] 100%* Done 20110422
    ISR and CCD Assembly Simon Krughoff
    Implement more SDQA metrics for ISR 20110309 20110322 Krughoff Simon[50%] 0% 20110421
    Clean-up/re-write ISR python code 20110323 20110412 Krughoff Simon[50%] *25% Begun as part of the #1556 merge 20110421
    Implement raw CCDAssembly method 20110413 20110419 Krughoff Simon[50%] 0% 20110421
    Point Source Photometry Robert Lupton
    Identify and fix zeropoint calculation error 20110131 20110309 Becker Andy[25%],Lupton Robert[25%] 75% 20110426
    Test aperture corrections 20110204 20110311 Lupton Robert[25%] 75% 20110307
    PSF Estimation Robert Lupton
    Finish PSF unification 20110131 20110318 Owen Russell[20%],Jarvis Mike[80%] 50%* Merged HSC changes into meas_algorithms ticket #1573 and updated meas_pipeline to match. Mike's code still needs unit tests. 20110420
    Update PSF formatter 20110325 20110330 Lim KT[50%] 0% 20110422
    Debug PCA PSF code 20110131 20110401 Lupton Robert[50%] 75% 20110307
    Galaxy Photometry Jim Bosch Merged 1556 changes to trunk, moving on with PSF and optimizer work. 20110419
    Finish porting external shapelet code to meas/multifit 20110131 20110405 Bosch Jim[50%] *100% Done. Much of it ended up in afw rather than meas/multifit 20110419
    Finish updates to afw to support meas/multifit 20110131 20110325 Dubcovsky Martin[50%] *100% Done. 20110419
    Implement LocalPsf? interface (with shapelets) for existing Psf classes 20110328 20110415 Dubcovsky Martin[50%],Bosch Jim[50%] *100% Done. More testing with "real" data is the next step 20110419
    R&D for optimizers, parameterizations, and inputs 20110415 20110506 Dubcovsky Martin[50%],Bosch Jim[50%] 5% Re-starting work after merges. 20110419
    Implement gaussian photometry 20110131 20110310 Lupton Robert[25%] 75% 20110307
    Implement Petrosian photometry 20110207 20110314 Lupton Robert[25%] 50% 20110307
    Implement Kron photometry 20110214 20110317 Lupton Robert[25%] 50% 20110307
    Fix Source class for memory bloat for Source Association 20110420 20110426 Monkewitz Serge[25%],Lupton Robert[50%] 0% 20110307
    Co-addition Owen Russell[0%] coadd_utils and coadd_chisquared are ready. coadd_psfmatched has been udpated for #1556 but not tested (I had to update ip_diffim first, now I'm busy with other things). coadd_pipeline needs updating (but is not used by pipette coaddition scripts). 20110426
    Pipette scripts 20110131 20110315 Owen Russell[50%] 50%* All scripts are written and updated for the #1556 merge but those changes need to be tested and outlier-rejected coadd is new and completely untested. 20110420
    Write outlier-rejected psf-matched co-addition 20110315 20110329 Owen Russell[50%] 25% Written as a pipette script but not tested. No pipeline stage in coadd_pipeline yet. 20110426
    Chi-squared co-addition 20110329 20110404 Owen Russell[50%] 75%* coadd_chisquared package is done. Needs a pipette script and coadd_pipeline stage updates. 20110426
    Update co-addition to use the new afw with unified geometry classes 20110404 20110408 Owen Russell[50%] 75% see "Co-addition" above 20110426
    Finish sky pixelization 20110408 20110506 Owen Russell[50%] 0% No new work; other tasks took priority. 20110426
    Templates and Difference Imaging Becker Andy[0%] 20110426
    Merge ip_diffim development ticket to trunk 20110131 20110329 Becker Andy[50%] 25% Started work on this 20110426
    Support Psf matching for template creation 20110228 20110408 Becker Andy[50%] 50% Written but not merged to trunk 20110426
    Assess quality of generated templates 20110408 20110506 Becker Andy[50%] 0% No templates have been made 20110426
    Implement image subtraction on visit pairs 20110330 20110412 Becker Andy[50%] 50% Required merging #1140 to trunk 20110426
    Quality Assessment Andy Becker 20110419
    Implement SDQA metrics for Source Association 20110309 20110310 Monkewitz Serge 0% 20110307
    Implement QA Pipeline package 20110131 20110419 Bickerton Steve[50%],Becker Andy[5%] 50% pipeQA being used to monitor weekly runs. Bickerton is now refining the code. 20110419
    Generate automatic QA metrics on pipeline runs 20110328 20110513 Becker Andy[50%] 50% With Bickerton's refinements this should enable automated QA 20110419
    Build and Test Robyn Allsman Monitoring and maintenance of buildbot runs is a continuing process.A new web page: http://dev.lsstcorp.org/trac/wiki/DM/buildbot/Daily_Status provides the daily buildbot status. The cluster's Weekly Production Run stacks are now generated by buildbot. 20110419
    Install and integrate new Parasoft release 20110131 20110311 Allsman Robyn[50%] 50% Parasoft is debugging their recent release which no longer handles LSST's C++ template usage 20110418
    Buildbot development for pipette 20110311 20110318 Allsman Robyn[50%] 0% 20110418
    Incorporate standards into buildbot trunk validation 20110318 20110603 Allsman Robyn[25%] 0% Low priority since no Parasoft to integrate. 20110418
    Maintain buildbot integration and test 20110131 20110617 Allsman Robyn[25%] 30% * Setup daily status page: http://dev.lsstcorp.org/trac/wiki/DM/buildbot/Daily_Status 20110418
    Production Runs Greg Daues/ Robyn Allsman The Weekly Production Run procedure is close to turnkey. Build generation of the code stack is initiated from a web-based form; the Weekly Run, using that stack, is initiated by a single command. A new status page details the Weekly Runs; see http://dev.lsstcorp.org/trac/wiki/DM/buildbot/Weekly_Production 20110419
    Migrate processing from abe to new platform 20110225 20110408 Plante Ray[10%],Daues Greg[10%] 50% Condor Glide-in to TACC Ranger being tested/refined: Ranger nodes join the LSST cluster, but job matchmaking is being debugged. 20110420
    Set up weekly runs 20110131 20110322 Allsman Robyn ?% Daues Greg [10%] 100%* The Cluster's Weekly Production Runs stacks are now generated by buildbot. 20110420
    Weekly runs 20110323 20110510 Allsman Robyn?%, Daues Greg[5%] 50% Maintaining/updating the weekly runs is a continuing process, e.g., changes to the run characteristics such as: new input data, new reference catalogs, and new hardware. 20110420
    Production debug/practice runs 20110420 20110510 Daues Greg[20%] 0% 20110307
    Full Production runs 20110511 20110628 Daues Greg[50%] 0% 20110307
    User Support Dick Shaw
    Update Data Challenge Handbook 20110131 20110421 Shaw Dick[50%] 10% 20110421
    Update DM User Support 20110421 20110526 Shaw Dick[50%] 5% It is not obvious that User Support processes need to be updated for the June release. 20110421
    Configure/tune catalogs 20110526 20110609 Becla Jacek[10%] 50%* All tune-ups needed so far has been done 20110422
    Data Analysis Tim Axelrod
    DM Scientist/Engineer? data analysis 20110420 20110628 Axelrod Tim[50%],Lupton Robert[50%],Bosch Jim[50%],Becker Andy[50%],Van Dyk Schuyler[25%],Shaw Dick[50%] 0% 20110307
    Science Collaboration data analysis 20110609 20110721 Science Collaboration 0% 20110307
    Data analysis report in handbook 20110721 20110804 Axelrod Tim[50%],Lupton Robert[25%],Bosch Jim[50%],Becker Andy[50%],Van Dyk Schuyler[25%],Shaw Dick[50%],Science Collaboration 0% 20110307