Ticket #741 (closed defect: fixed)

Opened 10 years ago

Last modified 7 years ago

switch use of productDir() from eups package to utils

Reported by: RayPlante Owned by: RayPlante
Priority: critical Milestone:
Component: unknown Keywords:
Cc: becker, nms Blocked By:
Blocking: Project: LSST
Version Number:
How to repeat:

not applicable

Description

eups' use of its cache is not robust to simultaneous use by multiple processes, particularly when those processes are on different machines (that share eups' filesystem). Consequently, use of eups functions in python called within a pipeline can cause at a minimum heavy lock contention, resulting in many messages like:

Waiting for /tmp/eups/lsst/DC3/stacks/gcc412/20feb/ups_db/.lock (User Raymond Plante, PID 15484)

and occasionally, at worst, pipeline failures.

The IsrStages.py file uses eups.productDir() to locate an internal data file which triggers the above-mentioned problem. The same functionality can be obtained via lsst.utils.productDir() which does not touch eups' cache but rather simply does a lookup on the environment variable. Swapping in utils' version should fix the problem.

Change History

comment:1 Changed 10 years ago by RayPlante

  • Status changed from new to assigned

comment:2 Changed 10 years ago by RayPlante

  • Status changed from assigned to closed
  • Resolution set to fixed

fix merged into trunk in r9071. I tested this in a pipeline execution; the eups error messages went away.

comment:3 Changed 7 years ago by robyn

  • Milestone DC3a Completed deleted

Milestone DC3a Completed deleted

Note: See TracTickets for help on using tickets.