Ticket #772 (closed defect: fixed)

Opened 10 years ago

Last modified 7 years ago

slice events leak across pipelines

Reported by: RayPlante Owned by: RayPlante
Priority: blocker Milestone:
Component: pex_harness Keywords:
Cc: Blocked By:
Blocking: Project: LSST
Version Number:
How to repeat:

not applicable

Description

In pipeline harness, the Pipeline object will accept events prior to any stage as configured in the pipeline policy. The Pipeline communicates the event data to the Slice instances via a second event having the same name but with "_slice" appended.

This naming convention causes a problem when there are multiple pipelines that are configured to listen for the same event--as in the alert production in which both the IPSD and nightmops pipelines are listening for "triggerImageprocEvent0" events. Each time they receive the event, both pipelines generate events intended for their own Slices with the same name, "triggerImageprocEvent0_slice", but which subsequently get received by slices in both pipelines. The effects are (1) causing the pipelines to process the same event twice, and (2) causing the triggerImageprocEvent0 and triggerImageprocEvent1 events to become out-of-sync, due to extra triggerImageprocEvent0_slice events getting inserted into the event queue.

Since the event that the Pipeline instance issues is intended only for its own slices, the event name needs a qualifier that uniquely identifies the pipeline. The Pipeline attribute, _pipelineName (recently added to differentiate log messages), would do nicely in this role.

Also, to allow for multiple production runs running simultaneously, the event name qualifier should also include the runID; however, completely supporting this possibility would require deeper changes, including to the event generator in ctrl_dc3pipe. Since this change is not critically needed for the blocked milestone, I recommend that this be handled under a separate ticket.

Change History

comment:1 Changed 10 years ago by RayPlante

  • Status changed from new to assigned

comment:2 Changed 10 years ago by RayPlante

  • Status changed from assigned to inTicketWork

comment:3 Changed 10 years ago by RayPlante

  • Status changed from inTicketWork to inStandardsReview
  • Owner changed from RayPlante to daues
  • reviewstatus changed from notReady to needsReview

source:DMS/pex/harness/tickets/772 ([r9476]) contains the fix as recommended in the description.

Greg, can you review the changes? If you don't see any issues, please pass this ticket back to me. I can merge it into trunk and cut a release (unless you want to). Fast turnaround would be helpful. thanks!

comment:4 Changed 10 years ago by RayPlante

The changeset link: [9476]

comment:5 Changed 10 years ago by daues

  • Owner changed from daues to RayPlante
  • reviewstatus changed from needsReview to reviewed

The slice events were successfully encoded with topics that include the "name" of the Pipeline, eliminating the leak across pipelines. The ticket can be resolved.

comment:6 Changed 10 years ago by RayPlante

  • Owner changed from RayPlante to daues

We may be caught in a workflow vortex.

comment:7 Changed 10 years ago by RayPlante

  • Owner changed from daues to RayPlante
  • Status changed from inStandardsReview to inTrunkMerge

comment:8 Changed 10 years ago by RayPlante

  • Status changed from inTrunkMerge to inQaReview

comment:9 Changed 10 years ago by RayPlante

  • Status changed from inQaReview to closed
  • Resolution set to fixed

comment:10 Changed 7 years ago by robyn

  • Milestone DC3a Completed deleted

Milestone DC3a Completed deleted

Note: See TracTickets for help on using tickets.