Ticket #822 (closed enhancement: wontfix)

Opened 10 years ago

Last modified 7 years ago

Please enforce the rule that only a dictionary file may describe Policy entries

Reported by: rowen Owned by: RayPlante
Priority: normal Milestone:
Component: pex_policy Keywords:
Cc: bbaker Blocked By:
Blocking: Project: LSST
Version Number:
How to repeat:

not applicable

Description (last modified by rowen) (diff)

As I understand the way a Policy object is intended to be unpersisted:

  • First a dictionary file is read. This defines the allowed entries, including name, type, default value (or absence of a default, meaning it must be set elsewhere) and related information. Typically a package will have one dictionary file per Policy that it uses. The middleware will use that Dictionary file when running pipelines.
  • Then a policy file is read. This must set any required values and may override any default values. (It must also allow comments about why values were set a particular way.) Typically a pipeline will have a set of policy files for its various stages, and typically these will be fairly small (since defaults can often be used).

This provides several benefits, including:

  • Documentation is centralized in one file.
  • Default values are centralized, keeping policy files small.

Unfortunately, it was stated at the recent DC3 meeting that policy files and dictionary files have the same format, implying that the design described above will not be enforced: that policy files can define new entries and provide data about entries, rather than simply providing values for entries.

I request that the above design be enforced; that policy files be required to only provide values for entries that have already been define in a dictionary file.

One way to do this without changing the format of dictionary and policy files is to have a flag that accepts or rejects data about entries. Allow data about entries when reading dictionary files. Reject data about entries and only accept values for pre-existing entries when reading policy files.

Change History

comment:1 Changed 10 years ago by rowen

  • Description modified (diff)

comment:2 Changed 10 years ago by RayPlante

  • Cc bbaker added
  • Status changed from new to closed
  • Resolution set to wontfix

Unfortunately, it was stated at the recent DC3 meeting that policy files and dictionary files have the same format, implying that the design described above will not be enforced: that policy files can define new entries and provide data about entries, rather than simply providing values for entries.

Not quite correct. A Dictionary uses the basic Policy format with a predefined schema (see PolicyFormats). Nevertheless, the createPolicy() function, when given a Dictionary file, will create a Policy loaded with the default values defined in the Dictionary; thus, it might appear that the formats are the same.

Another clarification: when stages are created and configured, the loading of policy data will actually have to be done in the opposite order than as stated in the description above, due to the way policy data is passed to a Stage. Nevertheless, the effect will be the same: the thin Policy file will be required to be compliant with the Dictionary; otherwise, an exception will be thrown.

Now regarding the implied points mentioned above...

  1. the thin policy file will not be allowed to include entries that are not defined in the Dictionary; this is an explicit requirement for compliance.
  2. The harness creates the Policy objects that will be passed to a Stage by calling createPolicy(). If the file happens to be in Dictionary format, the data will still load; however, all other information from the dictionary will be thrown away and therefore will not affect subsequent validation checks. (It's possible to override this default behavior with a switch passed to createPolicy(), but that is not how pex_harness calls it.) Thus, if a Dictionary is given instead of simple Policy file, the affect at runtime is equivalent to providing the "wrong" policy file: if the values it provides are compliant with the real Dictionary, then the configuration will work; if they are not, it will fail.

Given (2) above, I would recommend that we not make any changes to pex_policy towards enforcing use of policies and dictionaries other than what is already planned with validation. At worst, this might be confusing to a person running a pipeline if it is configured with a dictionary rather than a policy file. Still, I don't think it is a good idea to hard-code something that is essentially a "policy" (in the general sense) or a pattern of use; otherwise, if we need to change this pattern or make an exception, we would have to change pex_policy.

I'm going to close this with a "won't fix"; feel free to re-open if you'd like further discussion.

comment:3 Changed 7 years ago by robyn

  • Milestone DC3a MW Event Framework deleted

Milestone DC3a MW Event Framework deleted

Note: See TracTickets for help on using tickets.