-
Notifications
You must be signed in to change notification settings - Fork 93
Create a simple verification attestation #470
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
7f7a9ec
to
e0eab76
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for (re)starting this conversation! I was tempted to suggest we call this a sVSA
. 😂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't seem to resolve the conversations around my comments. I like the simplicity of this!
Just wanted to make sure this comment was seen and at least considered. CC @AdamZWu |
"predicate": { | ||
"verifier": { | ||
"id": "<URI>" | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
VSAs supported additional verifier metadata with verifier versions. Was this found to not be necessary, or if we wanted to include such metadata, would we do so as a property now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When @TomHennen and I were discussing this, we were talking about how the additional information was often confusing. This can certainly still be captured in a provenance attestation for the SVA itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@haydentherapper , do you think that we should provide more information on this in the predicate specification?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thinking more, does a consumer of a SVA need to know that a verifier has changed its verification logic? These verifiers aren't expected to be auditable, so a new revision of a verifier isn't meaningful, unless we expect that verifiers will publish the list of properties that they may include in a SVA.
Another option is to ask that verifiers version their identifiers, as in if they update their verification logic, they use a new unique ID. This puts additional work on the SVA consumer however, to maintain an up to date list of verifier IDs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 on versioning the identifier.
e0eab76
to
dd3b04d
Compare
@arewm should we also run this past the SLSA spec folks as we'd want to update SLSA to point here too? |
While a VSA exists within the SLSA specification, the current tracks have grown past the the initially proposed spec. We are proposing to create a new attestation predicate to replace the VSA which is also simpler to consume. There is intentionally less information in this attestation. This was partially done to create smaller predicates to enable faster verification as well as removing informational fields. Even though policy information is not present in the attestation itself, we call out the ability to use a SLSA Provenance attestation to provide more information about how the policy was evaluated. This provenance MAY be used to qualify trust in the simple verification attestation if needed. resolves: in-toto#277 Signed-off-by: arewm <arewm@users.noreply.github.com>
dd3b04d
to
5d2e344
Compare
"id": "<URI>" | ||
}, | ||
"timeCreated": "<TIMESTAMP>", | ||
"resourceUri": "<artifact-URI>", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps I'm missing something, but isn't resourceUri
essentially the attestation subject? Also, should this field be a resource descriptor?
An SVA is an attestation that a verifier (
verifier.id
) has checked an artifact (resourceUri
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps I'm missing something, but isn't
resourceUri
essentially the attestation subject? Also, should this field be a resource descriptor?An SVA is an attestation that a verifier (
verifier.id
) has checked an artifact (resourceUri
)
IMO they are orthogonal.
The subject records info relevant for the artifact being verified; the "resourceUri", OTOH, identifies the protection domain the artifact is verified against.
For example,
- The subject may have a uri "sftp://akamai.net/foo/bar.zip" -- that is where it is retrieved from;
- The SVA resourceUri may be "https://foo.com/app_bar" -- that is the service the artifact is going to be deployed onto (hence it needs to be verified against a policy for this URI).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that this field may not necessarily be the Statement subject, but I also think that turning this into a ResourceDescriptor
type makes sense. It facilitates describing the verified artifact in more detail, if need be, in a standard format.
@arewm Can you please line wrap at 80 chars? Just makes it a bit easier to comment on specific pieces. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for sending this @arewm and @TomHennen. Apologies for the delayed review. I've got several comments about making the predicate spec a bit crisper and clearer, but in general really like where this is heading.
I did make a comment at a SLSA meeting a while back about whether there'd be potential confusion between SVA <> VSA, but it seems like this may not be a widespread concern. One potential way to address this is to call the predicate an SV predicate only or change the name to something like SVR (where R stands for Result) since I think it's implied that the predicate would be part of an attestation.
|
||
## Purpose | ||
|
||
A Simple Verification Attestation (SVA) communicates that an artifact has been evaluated against one or more policies, and records the properties that were verified, at the point-in-time of the evaluation. This enables consumers to trust a concise summary of verification results, without requiring access to all underlying attestations or policies. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can probably be a bit more precise here
A Simple Verification Attestation (SVA) communicates that an artifact has been evaluated against one or more policies, and records the properties that were verified, at the point-in-time of the evaluation. This enables consumers to trust a concise summary of verification results, without requiring access to all underlying attestations or policies. | |
A Simple Verification Attestation (SVA) communicates that an artifact has been evaluated against one or more policies, and records the properties that were verified, at the point-in-time of the evaluation. This enables consumers to establish trust in a concise summary of verification results coming from a trusted verifier, without requiring access to all underlying attestations or policies. |
"id": "<URI>" | ||
}, | ||
"timeCreated": "<TIMESTAMP>", | ||
"resourceUri": "<artifact-URI>", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that this field may not necessarily be the Statement subject, but I also think that turning this into a ResourceDescriptor
type makes sense. It facilitates describing the verified artifact in more detail, if need be, in a standard format.
"verifier": { | ||
"id": "<URI>" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm assuming this is defined as an object with a single field so that verifier information can be extended. Is there any reason the verifier
couldn't just be defined as a ResourceDescriptor
?
**`resourceUri`, required** string (ResourceURI) | ||
|
||
> URI that identifies the resource associated with the artifact being verified. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason this artifact doesn't need to be immutable? I think the spec should at least RECOMMEND describing an immutable artifact as that removes ambiguity about which artifact was verified. This may be another reason to switch to a ResourceDescriptor
field type.
I also suggest picking a less generic field name, like verifiedArtifact
, so that it's unambiguous what this field represents.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't intending for it to be mutable. I agree that it makes sense to be immutable.
**`properties`, required** array of string | ||
|
||
> Indicates the properties verified for the artifact. These SHOULD be scoped according | ||
> to the framework being verified or the verifier's policy rules. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this a list of properties that were verified
regardless of pass/fail status, or only the properties that passed validation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only properties that passed validation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, thanks. This should be clarified in the description.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, Andrew, much appreciated!
Just a few comments right now:
- Please see my comment about properties vs policies.
- Would you mind linking to our talk here as a reference?
}, | ||
"timeCreated": "<TIMESTAMP>", | ||
"resourceUri": "<artifact-URI>", | ||
"properties": ["<VERIFIED_PROPERTY>", ...] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While I understand the simplicity of stringly-typed properties, I think they're a bit too simple to be generalised.
Personally, I think it's more useful to point to a policy along with an attestation bundle as supporting evidence. A single policy can be used to succinctly summarise a bunch of properties at once. As usual, policies and attestations can be identified with ResourceDescriptors.
More importantly, I'm curious to know what the group's thinking here is about what an SVA should say with respect to policies. Should an SVA always be restricted to one policy? What about multiple policies? Furthermore, what about explicitly saying that an artifact failed to meet a certain policy at a certain time? These are things we found useful in practice to answer at Datadog, and I'd like to know what others think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Policy rules would be an input to some evaluation. The output of these evaluations would be that some rules pass while others fail. The implementation of how those evaluations are run are ultimately irrelevant -- i.e. whether one or multiple policies are consumed. If you have two different policy engines, each capable of defining their own policy rules (i.e. AMPEL and Conforma), then the output of those passed rules can be collected together with policy-engine scoped properties.
The reproducibility of these results can be achieved by leveraging the provenance of the SVA.
Furthermore, what about explicitly saying that an artifact failed to meet a certain policy at a certain time?
I posit that if you have a policy rule which can pass one day and fail another then it doesn't belong as a verified property. It is not a permanent property and therefore shouldn't be considered as part of a permanent attestation.
what about explicitly saying that an artifact failed to meet a certain policy at a certain time?
I wanted to keep it simple by only having the positive case. If a policy rule not being met needs to be used by some verifier then that might need to be its own positive policy rule.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I posit that if you have a policy rule which can pass one day and fail another then it doesn't belong as a verified property. It is not a permanent property and therefore shouldn't be considered as part of a permanent attestation.
I slightly disagree. If we consider time as an input to the verification, I think there are several layers / classes of determinism, from weakest to strongest:
- Non-deterministic: given the exact same input (incl. time), the verifier can produce different verdicts.
- I don't think such verifiers are considered acceptable. :P
- Deterministic but not monotonic: the exact same input (incl. time) will produce the same verdict, however as the time (as the sole variable input) progresses, the verdict can flip-flop.
- I think this layer / class is not good to be stored in SVA or VSA.
- Deterministic and monotonic: the exact same input (incl. time) will produce the same verdict, and the verdict can make a one-way flip along the time axis.
- I feel / wish the SVA / VSA is better support this layer / class. It probably reflects the most common usage for time-sensitive verification in the SSC (artifact freshness, vulnerability check, etc.)
- Deterministic and permanent: the verdict is insensitive to time input at all.
- That is what build provenance verification belongs to. There may be a new other verification in this class, but I suspect it may not be the majority of all SSC checks.
@arewm Friendly ping. Please let us know if our feedback doesn't make sense. |
Yes, please, @TomHennen, too. My hot take is that given exactly the same Artifact and Attestation Bundle as well as "semantically equivalent" Policies (defined in the Framework), any two Policy Verifiers MUST output essentially the same SVA. This implies that SVAs MUST be reproducible: that is, they MUST contain enough information that another Policy Verifier can look at it and try to reproduce its results. This, in turn, implies that the complete set of Policies and Attestation Bundles must be referenced. |
While a VSA exists within the SLSA specification, the current tracks have grown past the the initially proposed spec. We are proposing to create a new attestation predicate to replace the VSA which is also simpler to consume.
There is intentionally less information in this attestation. This was partially done to create smaller predicates to enable faster verification as well as removing informational fields. Even though policy information is not present in the attestation itself, we call out the ability to use a SLSA Provenance attestation to provide more information about how the policy was evaluated. This provenance MAY be used to qualify trust in the simple verification attestation if needed.
resolves: #277