这是indexloc提供的服务,不要输入任何密码
Skip to content

Loading External Reference Data into CDM - FpML Coding Schemes #879

@mgratacos

Description

@mgratacos

Problem Statement

CDM currently loads FpML coding schemes and compiles them generating a static enumerated list using the Rune DSL docReference annotations.

  • This creates extra work for maintaining and releasing the CDM model.
  • This causes issues with current CDM users since these FpML coding schemes are updated more frequently than the CDM users upgrade their CDM implemented code.
  • FpML based code lists have changed an average of monthly for over a decade (140+ changes over 11 years).
  • This implies 140+ releases of CDM for reference data updates.
  • If multiple major versions of CDM are supported at once, this may imply several times more releases of CDM.
  • Rapidly evolving reference data exacerbates the challenge (e.g. large commodity code lists).
  • Inefficiencies limit overall effectiveness, inhibit interoperability and have more impact as CDM grows in adoption.

Lists in Scope

There are currently 15 CDM enumerations automatically regenerated by CDM on each release based on the FpML coding scheme. These all should be converted to the new mechanism.

10 CDM enumerated lists look like good candidates to be moved to the new mechanism because the schemes are relatively large (10+ values) and are expected to change relatively frequently, and the values are used infrequently in the code in most cases. However, they require further analysis and discussion.

Proposed Implementation

The proposed implementation changes the Rune syntax by extending the string data type (details subject to refinement based on detailed design and implementation work and feedback from the CDM TAWG and Rune working groups):

  • The existing Rune string type would be enhanced to include a new user-defined validation mechanism.
    • The existing string type supports validated properties (constraints) for minLength, maxLength, and pattern.
    • Two new constraint properties would be added:
      • validationRule would be a string that identifies the name of a user-defined validation rule that would be executed when the string is set, similar to the existing validation constraints. For the purposes of this project, we would always set this to the same value, something like "CodeListValidation”. Different values of this property could be used for other types of validation, such as CRCs/CheckSums, or database or API lookups.
      • domain would be a string that identifies the list of values to validate against, e.g. “currency-code” or “business-center” or “floating-rate-index”, and could be used by the validation rule to refine its validation.
    • The validation generation logic that currently validates the length and pattern constraints would be enhanced to invoke a validation stub function ValidateString with the parameters validationRule, domain, and the value of the string to be validated.
    • In the Rune runtime, this validation stub function would simply return a warning to implement the validation function.
  • We would create a CDM IsValidString user-defined function written in Rune to override the default validation logic from the Rune run-time and apply the appropriate user-defined validation logic, depending on the validationRule parameter. For the CodeListValidation rule it would validate the string value against a value retrieved based on the supplied and domain. For example, this could be a request to validate the code value “XYZ” or “USD” against the “currency-code” code list domain.
  • We would bind the Rune ValidateString function to the CDM IsValidString UDF implementation, so that when the string validation logic is triggered by the generated code and Rune runtime, it will call out to the CDM validation function.
  • For each enumeration being converted to a validated code, we would create a typeAlias including the appropriate constraint properties. For example:
    • typeAlias Currency : string (validationRule: “CodeListValidation”, domain: “currency-code”)
    • typeAlias BusinessCenter : string (validationRule: “CodeListValidation”, domain: “business-center”)
  • The IsValidString function would be a single function written in Rune as part of the CDM base functionality that could be used to check against any code list.
  • Existing occurrences of the enumerations would be replaced by the new typeAlias.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Closed

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions