docker v2 registry 本地仓库如何只删除指定 tag 而非 manifest 下的所有 tag

linux docker golang

字数统计: 2.6k阅读时长: 15 min

 2024/01/26 

使用 docker registry 本地仓库时候，一个镜像推送多个 tag 后，删除其中一个 tag 会造成所有 tag 都被删除，如何避免

由来

这几天整了个 TUI 应用，删除仓库上选中的 tag 镜像，查看 github 上代码的时候，折腾出来一个镜像具有多个 tag 下只删除指定 tag 的步骤。

过程

registry

先起一个 registry，偷懒命令行启动个不带授权的仓库：

docker run --rm -d \
	--net host \
	-e REGISTRY_HTTP_ADDR="127.0.0.1:5000" \
	-e REGISTRY_STORAGE_DELETE_ENABLED=true \
	-v $PWD/registry-data:/var/lib/registry  \
	--name registry registry:2.8

docker tag alpine 127.0.0.1:5000/zhangguanzhang/alpine
docker tag alpine 127.0.0.1:5000/zhangguanzhang/alpine:test1
docker push 127.0.0.1:5000/zhangguanzhang/alpine
docker push 127.0.0.1:5000/zhangguanzhang/alpine:test1

正常删除

如果有授权，下面的 curl 命令带 -u user:pass 就行。

# 可以测试授权正常否
$ curl -u 'xxx:xxxx' 127.0.0.1:5000/v2/
# 获取 repo 列表
$ curl -u 'xxx:xxxx' 127.0.0.1:5000/v2/_catalog?n=1000
{"repositories":["zhangguanzhang/alpine"]}
# 获取 tagList
$ curl -u 'xxx:xxxx' 127.0.0.1:5000/v2/zhangguanzhang/alpine/tags/list
{"name":"zhangguanzhang/alpine","tags":["latest","test1"]}
# 加 header 获取 tag 的 manifest，也就是 digest `Docker-Content-Digest`:
$ curl -u 'xxx:xxxx' 127.0.0.1:5000/v2/zhangguanzhang/alpine/manifests/test1 \
  -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' -v
...
Docker-Content-Digest: sha256:13b7e62e8df80264dbb747995705a986aa530415763a6c58f84a3ca8af9a5bcd

然后使用这个 digest 删除 manifest：

# 删除
$ curl -u 'xxx:xxxx' -X DELETE 127.0.0.1:5000/v2/zhangguanzhang/alpine/manifests/sha256:13b7e62e8df80264dbb747995705a986aa530415763a6c58f84a3ca8af9a5bcd
# 查询为 null
$ curl -u 'xxx:xxxx' 127.0.0.1:5000/v2/zhangguanzhang/alpine/tags/list
{"name":"zhangguanzhang/alpine","tags":null}

截至 2024/01/26 ，gitlab 的 container-registry 有 API DELETE /v2/<name>/tags/reference/<reference> 支持删除指定 tag 而非 manifest ，而 registry 官方也跟进了，但是 v3 还没正式发版。

需求实现

分析

今天查看 issue private registry delete manifest results in broken state 发现有 Contributor 说他写的库 regclient 可以实现这种需求，然后看了下代码：

# https://github.com/regclient/regclient/blob/v0.5.6/tag.go#L13-L18
// TagDelete deletes a tag from the registry. Since there's no API for this,
// you'd want to normally just delete the manifest. However multiple tags may
// point to the same manifest, so instead you must:
// 1. Make a manifest, for this we put a few labels and timestamps to be unique.
// 2. Push that manifest to the tag.
// 3. Delete the digest for that new manifest that is only used by that tag.

# https://github.com/regclient/regclient/blob/v0.5.6/scheme/reg/tag.go#L168-L192

大概就是上面两个代码的地方，就是三个步骤：

创建一个 manifest ，带一些不同的属性防止和远端仓库上一致
推送到这个 manifest 到要删除的 tag 上
删除这个 manifest

代码里是先创建空的 layer，再推送空的 config 再推送 manifest ，registry 日志请求过程：

time="2024-01-26T17:39:13.006086513+08:00" level=info msg="authorized request" go.version=go1.11.2 http.request.host="127.0.0.1:5000" http.request.id=18ca175a-862e-408d-8932-4b7fb90315f5 http.request.method=POST http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/blobs/uploads/?mount=sha256%3A44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a" http.request.useragent="regclient/regctl ((devel))" vars.name="zhangguanzhang/alpine" 
time="2024-01-26T17:39:13.044043468+08:00" level=info msg="response completed" go.version=go1.11.2 http.request.host="127.0.0.1:5000" http.request.id=18ca175a-862e-408d-8932-4b7fb90315f5 http.request.method=POST http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/blobs/uploads/?mount=sha256%3A44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a" http.request.useragent="regclient/regctl ((devel))" http.response.duration=42.746469ms http.response.status=202 http.response.written=0 
127.0.0.1 - - [26/Jan/2024:17:39:13 +0800] "POST /v2/zhangguanzhang/alpine/blobs/uploads/?mount=sha256%3A44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a HTTP/1.1" 202 0 "" "regclient/regctl ((devel))"
time="2024-01-26T17:39:13.062288812+08:00" level=info msg="authorized request" go.version=go1.11.2 http.request.contenttype="application/octet-stream" http.request.host="127.0.0.1:5000" http.request.id=20927091-fd3c-4582-8d8c-8b540269d758 http.request.method=PUT http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/blobs/uploads/9a5c21df-7d5f-47e0-afa9-eac0574ea8bd?_state=82vrGJDDwpp23C4CotVd3ZVHW9eduCbMPl9fcrI8cMZ7Ik5hbWUiOiJ3cHMvZmxhbm5lbCIsIlVVSUQiOiI5YTVjMjFkZi03ZDVmLTQ3ZTAtYWZhOS1lYWMwNTc0ZWE4YmQiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjQtMDEtMjZUMDk6Mzk6MTMuMDA2MjQ0MzNaIn0%3D&digest=sha256%3A44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a" http.request.useragent="regclient/regctl ((devel))" vars.name="zhangguanzhang/alpine" vars.uuid=9a5c21df-7d5f-47e0-afa9-eac0574ea8bd 
127.0.0.1 - - [26/Jan/2024:17:39:13 +0800] "PUT /v2/zhangguanzhang/alpine/blobs/uploads/9a5c21df-7d5f-47e0-afa9-eac0574ea8bd?_state=82vrGJDDwpp23C4CotVd3ZVHW9eduCbMPl9fcrI8cMZ7Ik5hbWUiOiJ3cHMvZmxhbm5lbCIsIlVVSUQiOiI5YTVjMjFkZi03ZDVmLTQ3ZTAtYWZhOS1lYWMwNTc0ZWE4YmQiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjQtMDEtMjZUMDk6Mzk6MTMuMDA2MjQ0MzNaIn0%3D&digest=sha256%3A44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a HTTP/1.1" 201 0 "" "regclient/regctl ((devel))"
time="2024-01-26T17:39:13.1071948+08:00" level=info msg="response completed" go.version=go1.11.2 http.request.contenttype="application/octet-stream" http.request.host="127.0.0.1:5000" http.request.id=20927091-fd3c-4582-8d8c-8b540269d758 http.request.method=PUT http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/blobs/uploads/9a5c21df-7d5f-47e0-afa9-eac0574ea8bd?_state=82vrGJDDwpp23C4CotVd3ZVHW9eduCbMPl9fcrI8cMZ7Ik5hbWUiOiJ3cHMvZmxhbm5lbCIsIlVVSUQiOiI5YTVjMjFkZi03ZDVmLTQ3ZTAtYWZhOS1lYWMwNTc0ZWE4YmQiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjQtMDEtMjZUMDk6Mzk6MTMuMDA2MjQ0MzNaIn0%3D&digest=sha256%3A44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a" http.request.useragent="regclient/regctl ((devel))" http.response.duration=49.705047ms http.response.status=201 http.response.written=0 


time="2024-01-26T17:39:37.715196631+08:00" level=info msg="authorized request" go.version=go1.11.2 http.request.host="127.0.0.1:5000" http.request.id=2241ece9-1d76-45f5-9808-63aa1029fc98 http.request.method=POST http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/blobs/uploads/?mount=sha256%3A413332e3ed9dc45c8be309b50ea9983ea68e673b99a39f8458c4d6441802f50a" http.request.useragent="regclient/regctl ((devel))" vars.name="zhangguanzhang/alpine" 
127.0.0.1 - - [26/Jan/2024:17:39:37 +0800] "POST /v2/zhangguanzhang/alpine/blobs/uploads/?mount=sha256%3A413332e3ed9dc45c8be309b50ea9983ea68e673b99a39f8458c4d6441802f50a HTTP/1.1" 202 0 "" "regclient/regctl ((devel))"
time="2024-01-26T17:39:37.734255241+08:00" level=info msg="response completed" go.version=go1.11.2 http.request.host="127.0.0.1:5000" http.request.id=2241ece9-1d76-45f5-9808-63aa1029fc98 http.request.method=POST http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/blobs/uploads/?mount=sha256%3A413332e3ed9dc45c8be309b50ea9983ea68e673b99a39f8458c4d6441802f50a" http.request.useragent="regclient/regctl ((devel))" http.response.duration=25.558253ms http.response.status=202 http.response.written=0 
time="2024-01-26T17:39:37.762736172+08:00" level=info msg="authorized request" go.version=go1.11.2 http.request.contenttype="application/octet-stream" http.request.host="127.0.0.1:5000" http.request.id=85ae6c53-1e82-416b-a3e6-4bd470eb1b53 http.request.method=PUT http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/blobs/uploads/3155a926-ba92-4611-8d74-04cbf82475d8?_state=hM1XV7Skiah6ud5-bNyuJwNzCjHkQfYx0SvFpx1iMDd7Ik5hbWUiOiJ3cHMvZmxhbm5lbCIsIlVVSUQiOiIzMTU1YTkyNi1iYTkyLTQ2MTEtOGQ3NC0wNGNiZjgyNDc1ZDgiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjQtMDEtMjZUMDk6Mzk6MzcuNzE1MzM1NjIzWiJ9&digest=sha256%3A413332e3ed9dc45c8be309b50ea9983ea68e673b99a39f8458c4d6441802f50a" http.request.useragent="regclient/regctl ((devel))" vars.name="zhangguanzhang/alpine" vars.uuid=3155a926-ba92-4611-8d74-04cbf82475d8 
time="2024-01-26T17:39:37.784189549+08:00" level=info msg="response completed" go.version=go1.11.2 http.request.contenttype="application/octet-stream" http.request.host="127.0.0.1:5000" http.request.id=85ae6c53-1e82-416b-a3e6-4bd470eb1b53 http.request.method=PUT http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/blobs/uploads/3155a926-ba92-4611-8d74-04cbf82475d8?_state=hM1XV7Skiah6ud5-bNyuJwNzCjHkQfYx0SvFpx1iMDd7Ik5hbWUiOiJ3cHMvZmxhbm5lbCIsIlVVSUQiOiIzMTU1YTkyNi1iYTkyLTQ2MTEtOGQ3NC0wNGNiZjgyNDc1ZDgiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjQtMDEtMjZUMDk6Mzk6MzcuNzE1MzM1NjIzWiJ9&digest=sha256%3A413332e3ed9dc45c8be309b50ea9983ea68e673b99a39f8458c4d6441802f50a" http.request.useragent="regclient/regctl ((devel))" http.response.duration=26.556218ms http.response.status=201 http.response.written=0 
127.0.0.1 - - [26/Jan/2024:17:39:37 +0800] "PUT /v2/zhangguanzhang/alpine/blobs/uploads/3155a926-ba92-4611-8d74-04cbf82475d8?_state=hM1XV7Skiah6ud5-bNyuJwNzCjHkQfYx0SvFpx1iMDd7Ik5hbWUiOiJ3cHMvZmxhbm5lbCIsIlVVSUQiOiIzMTU1YTkyNi1iYTkyLTQ2MTEtOGQ3NC0wNGNiZjgyNDc1ZDgiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjQtMDEtMjZUMDk6Mzk6MzcuNzE1MzM1NjIzWiJ9&digest=sha256%3A413332e3ed9dc45c8be309b50ea9983ea68e673b99a39f8458c4d6441802f50a HTTP/1.1" 201 0 "" "regclient/regctl ((devel))"


time="2024-01-26T17:39:42.630908535+08:00" level=info msg="authorized request" go.version=go1.11.2 http.request.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.request.host="127.0.0.1:5000" http.request.id=5d7c3eda-1ec4-41c3-9f1e-604a350e183c http.request.method=PUT http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/manifests/test1" http.request.useragent="regclient/regctl ((devel))" vars.name="zhangguanzhang/alpine" vars.reference=test1 
127.0.0.1 - - [26/Jan/2024:17:39:42 +0800] "PUT /v2/zhangguanzhang/alpine/manifests/test1 HTTP/1.1" 201 0 "" "regclient/regctl ((devel))"
time="2024-01-26T17:39:42.639925495+08:00" level=info msg="response completed" go.version=go1.11.2 http.request.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.request.host="127.0.0.1:5000" http.request.id=5d7c3eda-1ec4-41c3-9f1e-604a350e183c http.request.method=PUT http.request.remoteaddr="127.0.0.1:60484" http.request.uri="/v2/zhangguanzhang/alpine/manifests/test1" http.request.useragent="regclient/regctl ((devel))" http.response.duration=13.597646ms http.response.status=201 http.response.written=0

然后查阅官方 manifest 文档，manifest 格式为下面：

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json",
        "digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7",
        "size": 7023
    },
    "layers": [
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
            "size": 32654
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b",
            "size": 16724
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "digest": "sha256:ec4b8955958665577945c89419d1af06b5f7636b4ac3da7f12184802ad867736",
            "size": 73109
        }
    ]
}

docker 镜像上传，其实就是 layer，config + manifest，config 实际上也是通过 blob 接口上传，上传后会返回一个 digest ，我 registry 的客户端库是使用的 go-containerregistry ，找了下覆盖的方法大致为下面：

package main

import (


	"github.com/google/go-containerregistry/pkg/authn"
	"github.com/google/go-containerregistry/pkg/name"

	"github.com/google/go-containerregistry/pkg/v1/empty"
	"github.com/google/go-containerregistry/pkg/v1/remote"
)

func ....
	ref, err := name.NewTag(fmt.Sprintf("%s/%s", r.RegistryStr(), nameStr))
	if err != nil {
		return "", err
	}
	remote.Write(ref, empty.Image, remote.WithAuthFromKeychain(authn.DefaultKeychain))
}

然后抓包看了下上面请求的接口过程，发现它比 regclient 的步骤不一样：

通过 blob 接口上传空的 config
创建一个空 layer 的 manifest ，manifest.config.digest 使用上面得到的

最终步骤

然后使用 curl 测试了下正常：

1 2	docker push 127.0.0.1:5000/zhangguanzhang/alpine docker push 127.0.0.1:5000/zhangguanzhang/alpine:test1

造好数据后发送请求

config_str='{"architecture":"","created":"0001-01-01T00:00:00Z","os":"","rootfs":{"type":"layers","diff_ids":null},"config":{}}'
config_sha256sum=$(echo -n $config_str | sha256sum | cut -d ' ' -f 1)

# 获取上传链接 Location
patch_url=$(curl -s  -u 'xxx:xxxx' -X POST 127.0.0.1:5000/v2/zhangguanzhang/alpine/blobs/uploads/ -v |& grep -Po 'ocation:\s+\Khttp\S+')
put_url=$(echo -n "$config_str" |curl -u 'xxx:xxxx' -X PATCH "${patch_url}" \
	-H "Content-Type: application/octet-stream" \
	-d @- -v |& grep -Po 'ocation:\s+\Khttp\S+')

config_digest=$( curl -u 'xxx:xxxx' -X PUT "${put_url}&digest=sha256:${config_sha256sum}" -v |& grep -Po 'igest:\s+\Ksha256\S+'  )

curl -u 'xxx:xxxx' -X PUT 127.0.0.1:5000/v2/zhangguanzhang/alpine/manifests/test1 \
	-H 'Content-Type: application/vnd.docker.distribution.manifest.v2+json' \
	-d @- << EOF
{
   "schemaVersion":2,
   "mediaType":"application/vnd.docker.distribution.manifest.v2+json",
   "config":{
      "mediaType":"application/vnd.docker.container.image.v1+json",
      "size":115,
      "digest":"${config_digest}"
   },
   "layers":[
      
   ]
}
EOF

然后删除该 tag：

manifest_digest=$(curl -u 'xxx:xxxx' 127.0.0.1:5000/v2/zhangguanzhang/alpine/manifests/test1 \
  -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' -v |& \
  grep -Po 'igest:\s+\Ksha256\S+')

curl -u 'xxx:xxxx' -X DELETE 127.0.0.1:5000/v2/zhangguanzhang/alpine/manifests/${manifest_digest}

$ curl -u 'xxx:xxxx' 127.0.0.1:5000/v2/zhangguanzhang/alpine/tags/list
{"name":"zhangguanzhang/alpine","tags":["latest"]}

注意上面的 config blob 会先 head 下看看是否 404，404 才上面上传，否则就能获取到 config_digest ，最后执行 garbage-collect

docker exec registry registry garbage-collect /etc/docker/registry/config.yml --dry-run
docker exec registry registry garbage-collect /etc/docker/registry/config.yml -m
# gc 后一定要重启 registry 容器
docker restart registry