Closed
Description
What happened?
Hello everyone,
I have the following image ghcr.io/freggy/annotations-not-working. It has been built by a small custom tool i've written that's using the go-containerregistry library (you can see the code here: gist). Pulling it using crictl pull and then inspecting it using crictl inspecti, I can see that there are no annotations are returned.
Viewing the manifest of the image using crane manifest ghcr.io/freggy/annotations-not-working | jq, confirms that they are present in the image.
crictl inspecti output
{
"info": {
"imageSpec": {
"architecture": "arm64",
"config": {},
"created": "2025-06-16T18:20:53.751447+02:00",
"history": [
{
"created": "2025-06-16T18:20:53.751447+02:00",
"created_by": "test"
},
{
"created": "0001-01-01T00:00:00Z"
}
],
"os": "linux",
"rootfs": {
"diff_ids": [
"sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
],
"type": "layers"
}
}
},
"status": {
"id": "d0b73eda5332d22c7a66b7aa3ebceb468b69a09fe4fcb99ec51867e77552dfad",
"pinned": false,
"repoDigests": [
"ghcr.io/freggy/annotations-not-working@sha256:132e33a2c0d4bc173b80b3a834d0d715f3848c91c18fb2750bfd9e848b8ea958"
],
"repoTags": [
"ghcr.io/freggy/annotations-not-working:latest"
],
"size": "1222",
"spec": {
"annotations": {},
"image": "",
"runtimeHandler": "",
"userSpecifiedImage": ""
},
"username": ""
}
}
crane manifest output
$ crane manifest ghcr.io/freggy/annotations-not-working | jq
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 322,
"digest": "sha256:d0b73eda5332d22c7a66b7aa3ebceb468b69a09fe4fcb99ec51867e77552dfad"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 23,
"digest": "sha256:81da0491c5af5635831f6a3febb5d9bfd66987ba3ecc42e58dc3d80938c25705"
}
],
"annotations": {
"foo": "bar"
}
}
If I build the same image using buildah (script is here: gist), the annotations show up correctly.
buildah crictl inspecti output
{
"info": {
"imageSpec": {
"architecture": "arm64",
"config": {
"Labels": {
"io.buildah.version": "1.40.1"
},
"WorkingDir": "/"
},
"created": "2025-06-16T16:28:24.471709Z",
"history": [
{
"created": "2025-06-16T16:28:24.472466Z",
"created_by": "/bin/sh"
}
],
"os": "darwin",
"rootfs": {
"diff_ids": [
"sha256:52c7a3b17a34982364804e8c0089115605067335aaca0e4ba645766acbdf1a58"
],
"type": "layers"
},
"variant": "v8"
},
"labels": {
"io.buildah.version": "1.40.1"
}
},
"status": {
"id": "eb34b85f917569157f3f91452b1bea24a501ba476a6583f063d37fc18daa5863",
"pinned": false,
"repoDigests": [
"ghcr.io/freggy/annotations-working@sha256:c008fa7b4e4aff34bb46cabed36d1822ef4ad0003603cf703682006137f81790"
],
"repoTags": [
"ghcr.io/freggy/annotations-working:latest"
],
"size": "2911",
"spec": {
"annotations": {
"foo": "bar",
"org.opencontainers.image.base.digest": "",
"org.opencontainers.image.base.name": ""
},
"image": "",
"runtimeHandler": "",
"userSpecifiedImage": ""
},
"username": ""
}
}
buildah crane ouput
$ crane manifest ghcr.io/freggy/annotations-working | jq
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:eb34b85f917569157f3f91452b1bea24a501ba476a6583f063d37fc18daa5863",
"size": 353
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:0ceea2c2a6416928ba8170163976ab7cad072fdd569f9a23856ea76048ebe475",
"size": 103
}
],
"annotations": {
"foo": "bar",
"org.opencontainers.image.base.digest": "",
"org.opencontainers.image.base.name": ""
}
}
What did you expect to happen?
I'd expect that annotations are not empty for the image that has been built using my custom tool.
How can we reproduce it (as minimally and precisely as possible)?
The images i provided are publicly available, you just need to be authenticated to ghcr.io.
Pull and inspect the not working image
- Run
crictl pull ghcr.io/freggy/annotations-not-working - Run
crictl inspecti <img-id>
Pull and inspect the working image
- Run
crictl pull ghcr.io/freggy/annotations-working - Run
crictl inspecti <img-id>
Anything else we need to know?
No response
CRI-O and Kubernetes version
$ crio --version
crio version 1.33.0
GitCommit: 9d143f94054fc5c8338673f35f7c0164490363b3
GitCommitDate: 2025-05-15T18:26:05Z
GitTreeState: dirty
BuildDate: 1970-01-01T00:00:00Z
GoVersion: go1.24.2
Compiler: gc
Platform: linux/arm64
Linkmode: static
BuildTags:
static
netgo
osusergo
exclude_graphdriver_btrfs
seccomp
apparmor
selinux
LDFlags: unknown
SeccompEnabled: true
AppArmorEnabled: falseKubernetes is not deployed.
OS version
# On Linux:
$ cat /etc/os-release
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.2 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
$ uname -a
Linux lima-xcomp 6.8.0-60-generic #63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 18:51:58 UTC 2025 aarch64 aarch64 aarch64 GNU/LinuxAdditional environment details (AWS, VirtualBox, physical, etc.)
Running on a VM created with https://github.com/lima-vm/lima.
Config:
vmType: "vz"
memory: 4GiB
disk: 30GiB
images:
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-amd64.img"
arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-arm64.img"
arch: "aarch64"
mounts:
- location: "/Users/yannic/proj/spc/explorer"
writable: true
mountType: "virtiofs"