概述:
之前已经介绍Talos Linux,虽然我比较喜欢命令行,但对于管理便捷性还是非常差的,同时有很多小伙伴也关心在内网如何使用,本期我讲带大家使用Talos Linux的Web管理平台-Omni和离线环境搭建。
相关工具:
Omni DocsOmni官方文档,包含AirGap环境搭建指导和On-Prem部署指导。
Image Factory本地TalOS构建平台,支持构建裸金属、各种云平台,Omni会集成本地的Image-Factory。
Talos Linux ExtensionsTalos Linux的扩展image,例如:iSCSI支持、虚拟机Tools等。
Talos Linux Image Factory在线Image构建平台,可自定义系统扩展(vmtools-guest-agent、Nvidia container toolkit、qemu-guest-agent等)。 Talos Docs官方文档,包含AirGap环境搭建指导。
Skopeo用于上传image到本地仓库。
Cosign用于给本地仓库的image签名(因为Image-Factory必须对签名进行验证,所以会单独对本地image进行签名,以保证Image-Factory可以工作)
Kubelogin用于oidc登录,omni平台生成的kubeconfig需要通过oidc登录认证。
环境需求:
为验证整体架构和部署,整体采用生产架构,如果只是为了体验Talos可以通过Docker运行。
- 一台可以上网Linux虚拟机,安装Docker、jq环境,用于获取离线环境所需的所有images;
- 一台内网Linux虚拟机,安装Docker环境,用于运行离线镜像仓库-Registry、Omni、Keycloak、Image-Factory;
- 1个虚拟化/物理机环境,用于验证通过Omni部署Talos Linux Kubernetes(单节点);
- 一个DHCP的网络,用于Talos Linux首次启动获取IP地址,初始化完成后不再需要。当然也可以使用静态IP地址,为了简便,这里使用HDCP;
注意:如果需要多个版本的Talos Linux和Kubernetes,离线的镜像会比较大,请提前规划好磁盘空间。
在线Linux获取image脚本
先构建一个在线拉去image的脚本,其会拉去所有所需的image到2个压缩包和2个list文件,这里采用skopeo工具进行拉去,
cat pull-image-by-skopeo.sh
#!/bin/bash
# 安装skopeo工具
wget -O skopeo-linux-amd64 https://github.com/lework/skopeo-binary/releases/download/v1.16.1/skopeo-linux-amd64
chmod +x skopeo-linux-amd64
cp skopeo-linux-amd64 /usr/local/bin
export PATH=$PATH:/usr/local/bin
# 定义需要下载的 Talos 版本(可自行调整)
TALOS_VERSIONS=(
# "v1.7.0"
# "v1.7.1"
# "v1.7.2"
# "v1.7.3"
"v1.7.4"
"v1.7.5"
"v1.7.6"
"v1.7.7"
"v1.8.0"
)
# 定义 Kubernetes 大版本列表,这里会下载所有的版本,如果不需要所有版本,请调整get_minor_versions()函数中per_page的页数。
KUBERNETES_VERSIONS=(
# "v1.27"
# "v1.28"
# "v1.29"
"v1.30"
"v1.31"
)
# 定义 Kubernetes 相关组件的动态镜像列表(根据版本号动态生成)
DYNAMIC_IMAGES=(
"registry.k8s.io/kube-apiserver"
"registry.k8s.io/kube-controller-manager"
"registry.k8s.io/kube-scheduler"
"registry.k8s.io/kube-proxy"
"ghcr.io/siderolabs/kubelet"
)
# 定义额外需要拉取的静态镜像列表,这些Image用于Docker启动环境,不用上传到私有仓库中。
STATIC_IMAGES=(
"postgres:16"
"quay.io/keycloak/keycloak:25.0"
"quay.io/minio/minio:RELEASE.2024-10-02T17-50-41Z"
"ghcr.io/siderolabs/omni:latest"
"registry:2"
"ghcr.io/siderolabs/image-factory:v0.5.0"
)
# 定义文件及目录,动态Image采用tar.gz压缩(需要先解压缩,再执行上传脚本),静态Image采用tar压缩(可以直接Load到本地)
DYNAMIC_IMAGE_LIST_FILE="dynamic_images-list.txt"
STATIC_IMAGE_LIST_FILE="static_images-list.txt"
DYNAMIC_IMAGE_TAR_GZ="dynamic_images.tar.gz"
STATIC_IMAGE_TAR="static_images.tar"
# 定义 GitHub API 的 URL 来获取 Kubernetes Releases
GITHUB_API_URL="https://api.github.com/repos/kubernetes/kubernetes/releases"
# 清空旧的镜像列表文件
> $STATIC_IMAGE_LIST_FILE
> $DYNAMIC_IMAGE_LIST_FILE
# 1. 下载特定版本的 `talosctl`,判断是否存在,如果存在就跳过
download_talosctl() {
local version=$1
local filename="talosctl-${version}"
local url="https://github.com/siderolabs/talos/releases/download/${version}/talosctl-linux-amd64"
if [ -f "$filename" ]; then
echo "talosctl version ${version} already exists locally, skipping download."
else
echo "Downloading talosctl version: ${version} from ${url}..."
wget -q $url -O "$filename"
chmod +x "$filename"
fi
}
# 2. 使用特定版本的 `talosctl` 来获取默认镜像清单
fetch_talos_images() {
local version=$1
local filename="talosctl-${version}"
echo "Fetching images for Talos version: ${version} using talosctl..."
./$filename image default | grep -oP '^\S+:\S+' >> $DYNAMIC_IMAGE_LIST_FILE
}
# 3. 从 factory.talos.dev 获取扩展组件和 overlays 镜像
fetch_factory_images() {
local version=$1
local api_base="https://factory.talos.dev/version/${version}"
# 3.1 获取扩展镜像
echo "Fetching official extensions for version: ${version}..."
curl -s "${api_base}/extensions/official" | jq -r '.[].ref' >> $DYNAMIC_IMAGE_LIST_FILE
# 3.2 获取 overlays 镜像
echo "Fetching official overlays for version: ${version}..."
curl -s "${api_base}/overlays/official" | jq -r '.[].ref' >> $DYNAMIC_IMAGE_LIST_FILE
}
# 4. 使用 GitHub API 获取所有小版本号
get_minor_versions() {
local major_version=$1
local api_url="https://api.github.com/repos/kubernetes/kubernetes/releases?per_page=100"
curl -s $api_url | grep -Po "\"tag_name\":\s*\"$major_version\.\d+\"" | grep -Po "$major_version\.\d+" | sort -V
}
# 6. 根据 Talos 版本定义版本特定镜像
add_version_specific_images() {
local version=$1
VERSION_SPECIFIC_IMAGES+=(
"ghcr.io/siderolabs/extensions:${version}"
"ghcr.io/siderolabs/installer:${version}"
"ghcr.io/siderolabs/imager:${version}"
)
}
# 7. 生成镜像列表文件
echo "Generating image lists..."
# 7.1 收集所有 Talos 相关的镜像
for version in "${TALOS_VERSIONS[@]}"; do
download_talosctl $version
fetch_talos_images $version
fetch_factory_images $version
add_version_specific_images $version
done
# 7.2 收集所有 Kubernetes 组件的镜像
for major_version in "${KUBERNETES_VERSIONS[@]}"; do
MINOR_VERSIONS=$(get_minor_versions "$major_version")
echo "$MINOR_VERSIONS" | while read -r minor_version; do
for IMAGE in "${DYNAMIC_IMAGES[@]}"; do
IMAGE_WITH_TAG="${IMAGE}:${minor_version}"
echo "$IMAGE_WITH_TAG" >> $DYNAMIC_IMAGE_LIST_FILE
done
done
done
# 7.3 收集静态镜像
for IMAGE in "${STATIC_IMAGES[@]}"; do
echo "$IMAGE" >> $STATIC_IMAGE_LIST_FILE
done
# 7.4. 收集所有版本特定镜像
for IMAGE in "${VERSION_SPECIFIC_IMAGES[@]}"; do
echo "$IMAGE" >> $DYNAMIC_IMAGE_LIST_FILE
done
# 8. 去重并生成最终的镜像列表
echo "Removing duplicate images..."
sort -u $STATIC_IMAGE_LIST_FILE -o $STATIC_IMAGE_LIST_FILE
sort -u $DYNAMIC_IMAGE_LIST_FILE -o $DYNAMIC_IMAGE_LIST_FILE
# 9. 下载静态镜像
echo "Downloading static images using docker pull..."
while read -r image; do
echo "Pulling static image: $image"
#docker pull "$image"
done < $STATIC_IMAGE_LIST_FILE
# 10. 下载动态镜像
echo "Downloading dynamic images using skopeo..."
mkdir -p dynamic_images
while read -r image; do
echo "Copying dynamic image: $image"
skopeo copy --retry-times 3 --all --preserve-digests "docker://${image}" "dir:dynamic_images/$(echo "$image" | tr '/:' '_')"
done < $DYNAMIC_IMAGE_LIST_FILE
# 11. 保存所有静态镜像到 tar 文件
echo "Saving static images to ${STATIC_IMAGE_TAR}..."
docker save -o $STATIC_IMAGE_TAR $(cat $STATIC_IMAGE_LIST_FILE)
# 12. 保存所有动态镜像到 tar 文件
echo "Saving dynamic images to ${DYNAMIC_IMAGE_TAR}..."
tar -czvf $DYNAMIC_IMAGE_TAR_GZ dynamic_images/
# 最终结果提示
echo "All static images have been saved to ${STATIC_IMAGE_TAR}."
echo "All dynamic images have been saved to ${DYNAMIC_IMAGE_TAR}."
echo "Static image list has been saved to ${STATIC_IMAGE_LIST_FILE}."
echo "Dynamic image list has been saved to ${DYNAMIC_IMAGE_LIST_FILE}."
在线Linux获取管理工具脚本
#!/bin/bash
# 创建目录用于保存工具
mkdir -p tools
cd tools
# 下载工具
wget -O skopeo-linux-amd64 https://github.com/lework/skopeo-binary/releases/download/v1.16.1/skopeo-linux-amd64
wget -O cosign-linux-amd64 https://github.com/sigstore/cosign/releases/download/v2.4.0/cosign-linux-amd64
wget -O kubelogin_linux_amd64.zip https://github.com/int128/kubelogin/releases/download/v1.30.1/kubelogin_linux_amd64.zip
wget -O talosctl-linux-amd64 https://github.com/siderolabs/talos/releases/download/v1.8.0/talosctl-linux-amd64
wget -O omnictl-linux-amd64 https://github.com/siderolabs/omni/releases/download/v0.42.3/omnictl-linux-amd64
wget -O kubectl https://dl.k8s.io/release/v1.31.1/bin/linux/amd64/kubectl
wget -O jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64
CFSSL_VERSION=1.6.5
wget -q -O cfssl https://github.com/cloudflare/cfssl/releases/download/v${CFSSL_VERSION}/cfssl_${CFSSL_VERSION}_linux_amd64
wget -q -O cfssljson https://github.com/cloudflare/cfssl/releases/download/v${CFSSL_VERSION}/cfssljson_${CFSSL_VERSION}_linux_amd64
# 解压 `kubelogin`
unzip kubelogin_linux_amd64.zip
# 修改权限
chmod +x skopeo-linux-amd64 cosign-linux-amd64 kubelogin talosctl-linux-amd64 omnictl-linux-amd64 kubectl
# 删除无关的压缩包
rm -f kubelogin_linux_amd64.zip
# 打包所有工具为 tar.gz
cd ..
tar -czvf tools_package.tar.gz tools/
echo "Tools has package to: tools_package.tar.gz"
离线Linux配置—启动本地镜像仓库
先加载所有image到本地,再解压缩动态image压缩包
sudo mkdir -p /usr/local/omni
chown -R $USER /usr/local/omni
cd /usr/local/omni
#upload to /usr/local/omni
#dynamic_images-list.txt
#static_images-list.txt
#dynamic_images.tar.gz
#static_images.tar
#tools_package.tar.gz
docker load -i static_images.tar
tar -zxvf dynamic_images.tar.gz
查看是否包含Image的文件夹
ls dynamic_images
启动本地镜像仓库和打开防火墙端口。
mkdir -p /usr/local/omni/registrydata
docker run -d -p 5000:5000 \
--restart always \
--name registry-airgapped \
-v /usr/local/omni/registrydata:/var/lib/registry \
-e "REGISTRY_STORAGE_DELETE_ENABLED=true" \
registry:2
sudo firewall-cmd --permanent --add-port=5000/tcp
sudo firewall-cmd --reload
离线Linux Tools安装脚本
构建install_tools.sh脚本,用于安装相关工具。
#!/bin/bash
# Check if tools_package.tar.gz exists
if [ ! -f "tools_package.tar.gz" ]; then
echo "tools_package.tar.gz not found. Please place the package in the current directory."
exit 1
fi
# Extract the tools package
tar -xzvf tools_package.tar.gz
# Enter the tools directory
cd tools
# Check if /usr/local/bin is in $PATH
if ! echo "$PATH" | grep -q "/usr/local/bin"; then
echo "/usr/local/bin is not in the PATH. Adding it now..."
# Add to /etc/profile
echo 'export PATH=$PATH:/usr/local/bin' | sudo tee -a /etc/profile
# Update the current shell's PATH
export PATH=$PATH:/usr/local/bin
echo "Successfully added /usr/local/bin to the PATH."
else
echo "/usr/local/bin is already in the PATH."
fi
# Install tools to /usr/local/bin
sudo cp skopeo-linux-amd64 /usr/local/bin/skopeo
sudo cp cosign-linux-amd64 /usr/local/bin/cosign
sudo cp kubelogin /usr/local/bin/kubectl-oidc_login
sudo cp talosctl-linux-amd64 /usr/local/bin/talosctl
sudo cp omnictl-linux-amd64 /usr/local/bin/omnictl
sudo cp kubectl /usr/local/bin/kubectl
sudo cp jq /usr/local/bin/jq
sudo cp cfssl /usr/local/bin/cfssl
sudo cp cfssljson /usr/local/bin/cfssljson
# Set executable permissions
sudo chmod +x /usr/local/bin/skopeo
sudo chmod +x /usr/local/bin/cosign
sudo chmod +x /usr/local/bin/kubectl-oidc_login
sudo chmod +x /usr/local/bin/talosctl
sudo chmod +x /usr/local/bin/omnictl
sudo chmod +x /usr/local/bin/kubectl
sudo chmod +x /usr/local/bin/jq
sudo chmod +x /usr/local/bin/cfssl
sudo chmod +x /usr/local/bin/cfssljson
# Add command completion to /etc/profile
echo "Adding command completions to /etc/profile..."
echo 'source <(talosctl completion bash)' | sudo tee -a /etc/profile
echo 'source <(omnictl completion bash)' | sudo tee -a /etc/profile
echo 'source <(kubectl completion bash)' | sudo tee -a /etc/profile
# Apply the changes immediately
source /etc/profile
echo "All tools have been successfully installed to /usr/local/bin, and command completions have been configured."
离线Linux配置—安装工具
./install_tools.sh
离线Linux配置—生成omni和keycloak所需的SSL证书
我们通过CFSSL工具来生成证书,keycloak和omni使用相同的证书,脚本如下:
cat > generate_omni_certs.sh <<EOOF
#!/bin/bash
mkdir -p /usr/local/omni/certs
cd /usr/local/omni/certs
# 参数定义
OMNI_IP="10.40.43.112"
CA_CERT_FILE="omni-ca.pem"
CA_KEY_FILE="omni-ca-key.pem"
# 创建CA配置
cat > omni-ca-csr.json <<EOF
{
"CN": "Omni CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Changchun",
"O": "DevOps",
"OU": "OMNI",
"ST": "Jilin"
}
],
"ca": {
"expiry": "438000h"
}
}
EOF
# 检查 CA 证书和密钥是否存在
if [[ -f "$CA_CERT_FILE" && -f "$CA_KEY_FILE" ]]; then
echo "CA 证书和密钥已存在,跳过创建步骤。"
else
echo "CA 证书和密钥不存在,正在创建..."
# 创建 CA 证书和密钥的命令
cfssl gencert -initca omni-ca-csr.json | cfssljson -bare omni-ca
echo "CA 证书和密钥创建完成。"
fi
# 生成 Omni证书配置
cat > config.json <<EOF
{
"signing": {
"default": {
"usages": ["digital signature","key encipherment","signing","server auth"],
"expiry": "87600h"
}
}
}
EOF
#
cat > omni-csr.json <<EOF
{
"CN": "$OMNI_IP",
"hosts": [
"127.0.0.1",
"$OMNI_IP"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Changchun",
"O": "DevOps",
"OU": "OMNI",
"ST": "Jilin"
}
]
}
EOF
# 生成 Omni 证书
cfssl gencert -ca=omni-ca.pem -ca-key=omni-ca-key.pem -config=config.json omni-csr.json | cfssljson -bare omni
cat omni.pem omni-ca.pem > omni-chain.pem
EOOF
生成私有CA和Omni证书
chmod +x generate_omni_certs.sh
./generate_omni_certs.sh
离线Linux配置—生成omni和image-factory所需的key
准备生成相关key的脚本,包括用于ETCD加密的GPG和Cosign的私钥和公钥。
# cat generate-gpg-cert.sh
#!/bin/bash
# 定义参数
EMAIL="[email protected]"
KEY_NAME="Omni (Used for etcd data encryption) <${EMAIL}>"
OUTPUT_DIR="/usr/local/omni"
GPG_ASC_FILE="${OUTPUT_DIR}/omni.asc"
# 检查并创建输出目录
mkdir -p "$OUTPUT_DIR"
cd $OUTPUT_DIR
# 使用 gpg --with-colons --fingerprint 检查是否存在相同 uid 的 GPG 密钥,并获取指纹
FPR=$(gpg --with-colons --fingerprint "$EMAIL" | grep -E "^fpr" | head -n 1 | cut -d: -f10)
if [ -n "$FPR" ]; then
echo "Found an existing GPG key for ${EMAIL} with fingerprint: ${FPR}"
else
echo "No existing GPG key found for ${EMAIL}. Generating a new GPG key for ${KEY_NAME}..."
# 使用 --batch 模式生成 GPG 主密钥,并设置参数无交互生成
gpg --batch --gen-key <<EOF
%no-protection
Key-Type: RSA
Key-Length: 4096
Subkey-Type: RSA
Subkey-Length: 4096
Name-Real: Omni
Name-Comment: Used for etcd data encryption
Name-Email: ${EMAIL}
Expire-Date: 0
%commit
EOF
# 再次检查是否生成了相同的 GPG 密钥,并获取指纹
FPR=$(gpg --with-colons --fingerprint "$EMAIL" | grep -E "^fpr" | head -n 1 | cut -d: -f10)
if [ -z "$FPR" ]; then
echo "Error: Failed to generate the GPG key or retrieve its fingerprint."
echo "Checking the generated GPG keys for troubleshooting:"
gpg --list-secret-keys --fingerprint
exit 1
fi
fi
# 导出 GPG 密钥到 omni.asc
if [ -f "${GPG_ASC_FILE}" ]; then
echo "GPG key file ${GPG_ASC_FILE} already exists. Skipping export."
else
echo "Exporting GPG key to ${GPG_ASC_FILE}..."
gpg --export-secret-key --armor "${FPR}" > "${GPG_ASC_FILE}"
if [ -f "${GPG_ASC_FILE}" ]; then
echo "Successfully generated and exported GPG key to ${GPG_ASC_FILE}"
else
echo "Error: Failed to export GPG key."
exit 1
fi
fi
# image factory
openssl ecparam -name prime256v1 -genkey -noout -out cache-signing-key.key
echo "All keys have been successfully generated and exported!"
生成key
chmod +x generate-gpg-cert.sh
./generate-gpg-cert.sh
查看key
ls /usr/local/omni/omni.asc /usr/local/omni/cache-signing-key.key
为Cosign准备证书
这次我们采用上一步创建的omni-ca来签发证书,并在签发后使用Cosign导入成Cosign使用的key和pub。
cd /usr/local/omni/certs
# 构建profiles文件
echo "Building cosign certificate"
cat > cosign-config.json <<EOF
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"cosign": {
"usages": ["signing", "digital signing"],
"expiry": "87600h",
"copy_extensions": true
}
}
}
}
EOF
# 配置证书所需变量
#extensions参考:https://github.com/sigstore/fulcio/blob/main/docs/oid-info.md
# OIDC_ISSUER用于验证时的--certificate-oidc-issuer or --certificate-oidc-issuer-regexp参数;
OIDC_ISSUER=http://localhost
OIDC_ISSUER_BASE64=$(echo -n $OIDC_ISSUER | base64 -w0)
#CERTIFICATE_IDENTITY用于验证时的--certificate-identity or --certificate-identity-regexp参数;
[email protected]
# 构建CSR文件
cat cosign-csr.json
{
"CN": "Cosign Certificate",
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "CN",
"L": "Changchun",
"O": "DevOps",
"OU": "OMNI",
"ST": "Jilin"
}
],
"hosts": [
"${CERTIFICATE_IDENTITY}"
],
"extensions": [
{
"id": [1,3,6,1,4,1,57264,1,1],
"value": "${OIDC_ISSUER_BASE64}"
},
{
"id": [1,3,6,1,4,1,57264,1,8],
"value": "${OIDC_ISSUER_BASE64}"
}
]
}
# 签发Cosign证书
cfssl gencert \
-ca omni-ca.pem \
-ca-key omni-ca-key.pem \
-config cosign-config.json \
-profile cosign cosign-csr.json \
|cfssljson -bare cosign
# 转换成Cosign使用的key和pub
cosign import-key-pair --yes --key cosign-key.pem --output-key-prefix import-cosign
# 查看cosign的key和pub
ls import-cosign.key import-cosign.pub
Cosign测试签名
cosign sign --key certs/import-cosign.key --cert certs/cosign.pem --cert-chain certs/omni-ca.pem --tlog-upload=false 10.40.43.112:5000/siderolabs/installer:v1.8.0
Cosign测试验证签名 这里指定根证书等参数后,可以验证通过。
$ cosign verify --ca-roots=certs/omni-ca.pem --certificate-identity='[email protected]' --certificate-oidc-issuer='http://localhost' --insecure-ignore-sct --insecure-ignore-tlog 10.40.43.112:5000/siderolabs/installer:v1.8.0
WARNING: Skipping tlog verification is an insecure practice that lacks of transparency and auditability verification for the signature.
Verification for 10.40.43.112:5000/siderolabs/installer:v1.8.0 --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The code-signing certificate was verified using trusted certificate authority certificates
[{"critical":{"identity":{"docker-reference":"10.40.43.112:5000/siderolabs/installer"},"image":{"docker-manifest-digest":"sha256:8f034be82a4f4518ad7f1cb7cc730fcb4a5b160b708a3f7b1530ca168100af32"},"type":"cosign container image signature"},"optional":{"1.3.6.1.4.1.57264.1.1":"http://localhost","Issuer":"http://localhost","Subject":"[email protected]"}}]
为SecureBoot准备证书和PCR密钥
如果希望Image-Factory支持构建SecureBoot的ISO,那么我们需要生成SecureBoot所需证书和私钥,这里直接使用Talosctl工具来生成。
cd /usr/local/omni
talosctl gen secureboot uki --common-name "SecureBoot Key" -output secureboot_keys
talosctl gen secureboot pcr --output secureboot_keys
ls secureboot_keys/
# pcr-signing-key.pem uki-signing-cert.der uki-signing-cert.pem uki-signing-key.pem
创建Skopeo的配置文件
cat > /etc/containers/policy.json <<EOF
{
"default": [
{
"type": "insecureAcceptAnything"
}
]
}
EOF
离线Linux上传Image到本地镜像仓库脚本
构建 push-image-by-skopeo.sh 脚本,用于上传Image到仓库,并对Image进行签名。
#!/bin/bash
# 定义私有仓库地址
REGISTRY_URL="10.40.43.112:5000"
# Set Cosign password to empty (to skip password prompt)
export COSIGN_PASSWORD=""
# Cosign key file路径
CERTS_DIR="./certs"
COSIGN_KEY="${CERTS_DIR}/import-cosign.key"
COSIGN_CERT="${CERTS_DIR}/cosign.pem"
COSIGN_CERT_CHAIN="${CERTS_DIR}/omni-ca.pem"
# 定义文件和目录路径
DYNAMIC_IMAGE_LIST_FILE="dynamic_images-list.txt" # 只处理 image:tag 格式
IMAGE_DIR="dynamic_images"
UPLOADED_IMAGES_FILE="uploaded_images.txt"
UPLOAD_LOG="upload.log"
SIGN_LOG="sign.log"
# 清空现有文件
> "${UPLOADED_IMAGES_FILE}"
> "${UPLOAD_LOG}"
> "${SIGN_LOG}"
# Function: 去除字符串前后的空格
trim() {
echo "$1" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//'
}
# Function: 使用 skopeo 上传镜像
upload_image_with_skopeo() {
local image_tag
image_tag=$(trim "$1") # 去除空格
local local_image_path="$2" # 本地镜像目录路径
local registry_url="$3" # 目标仓库地址
# 提取镜像的 repo 名称(不包含前缀)
local repo_name="${image_tag#*/}"
local target_tag="${registry_url}/${repo_name}" # 目标 tag 格式
echo "Pushing image from ${local_image_path} to ${target_tag}..." | tee -a "${UPLOAD_LOG}"
# 使用 skopeo copy 将目录中的镜像推送到私有仓库,只上传 tag
skopeo copy --all --preserve-digests --dest-tls-verify=false \
"dir:${local_image_path}" \
"docker://${target_tag}" &>> "${UPLOAD_LOG}"
if [[ $? -eq 0 ]]; then
echo "Successfully pushed image ${target_tag}." | tee -a "${UPLOAD_LOG}"
echo "${target_tag}" >> "${UPLOADED_IMAGES_FILE}"
else
echo "Failed to push image ${target_tag}." | tee -a "${UPLOAD_LOG}"
fi
}
# 1. 上传动态镜像:读取 dynamic_images-list.txt,并上传镜像目录中的文件
echo "Uploading dynamic images..." | tee -a "${UPLOAD_LOG}"
while IFS= read -r image_tag; do
image_tag=$(trim "${image_tag}") # 提取 image:tag 格式并去除空格
# 根据 image_tag 提取 registry 前缀,并去掉 registry 前缀
echo $image_tag
# 根据 image_tag 提取本地镜像目录路径(确保目录结构匹配)
image_safe_name=$(echo "${image_tag}" | tr '/:' '_') # 替换字符以构建目录名(如 "pause_3.9")
echo $image_safe_name
# 根据 image_tag 提取本地镜像目录路径(确保目录结构匹配)
local_image_path="${IMAGE_DIR}/${image_safe_name}" # 本地镜像目录路径
echo $local_image_path
if [[ -d "${local_image_path}" ]]; then
# 上传动态镜像
upload_image_with_skopeo "$image_tag" "$local_image_path" "$REGISTRY_URL"
else
echo "Local image directory not found for ${image_tag}, skipping..." | tee -a "${UPLOAD_LOG}"
fi
done < "${DYNAMIC_IMAGE_LIST_FILE}"
# 2. 基于上传成功的镜像列表文件进行签名
echo "All images have been uploaded successfully. Starting Cosign signing..." | tee -a "${SIGN_LOG}"
while IFS= read -r image; do
image_tag=$(trim "${image}") # 提取 image:tag 格式并去除空格
# 构建签名对象:tag 格式
repo_name="${image_tag#*/}"
target_tag="${REGISTRY_URL}/${repo_name}"
# 使用 Cosign 签名 tag
echo "Signing image ${target_tag} ..." | tee -a "${SIGN_LOG}"
cosign sign --key "${COSIGN_KEY}" --cert "${COSIGN_CERT}" --cert-chain="${COSIGN_CERT_CHAIN}" --tlog-upload=false "${target_tag}" &>> "${SIGN_LOG}"
if [[ $? -eq 0 ]]; then
echo "Successfully signed tag: ${target_tag}." | tee -a "${SIGN_LOG}"
else
echo "Failed to sign tag: ${target_tag}." | tee -a "${SIGN_LOG}"
fi
done < "${UPLOADED_IMAGES_FILE}"
echo "All images have been signed successfully." | tee -a "${SIGN_LOG}"
# 检查上传和签名日志中的错误
echo "Checking logs for errors..." | tee -a "${UPLOAD_LOG}"
UPLOAD_ERRORS=$(grep -iE "failed|error|timeout" "${UPLOAD_LOG}")
SIGN_ERRORS=$(grep -iE "failed|error|timeout" "${SIGN_LOG}")
if [[ -n "$UPLOAD_ERRORS" || -n "$SIGN_ERRORS" ]]; then
echo "Some issues were detected during the process:" | tee -a "${UPLOAD_LOG}"
if [[ -n "$UPLOAD_ERRORS" ]]; then
echo "Upload errors:" | tee -a "${UPLOAD_LOG}"
echo "$UPLOAD_ERRORS" | tee -a "${UPLOAD_LOG}"
fi
if [[ -n "$SIGN_ERRORS" ]]; then
echo "Sign errors:" | tee -a "${SIGN_LOG}"
echo "$SIGN_ERRORS" | tee -a "${SIGN_LOG}"
fi
else
echo "No errors found in the log files." | tee -a "${UPLOAD_LOG}"
fi
echo "Process completed!"
离线Linux配置—上传并签名image
chmod +x push-image-by-skopeo.sh
./push-image-by-skopeo.sh
准备Docker Compose相关文件
我们采用Docker-Compose来运行KeyCloak、Omni和Image-Factroy、MinIO,其中KeyCloak用于SAML认证和用户管理,MinIO用于存储集群备份。
注意1:请修改相应的IP地址、密码和UUID 注意2:Omni容器中/etc/ssl/certs/目录是存放私有CA信任的位置,只需要把Omni的根CA映射到此目录,即可实现证书信任。
准备文件目录
chown -R ops /usr/local/omni
mkdir -p /usr/local/omni/data/{postgres,keycloak,minio}
mkdir -p /usr/local/omni/etcd
chmod 777 /usr/local/omni/data/{postgres,keycloak,minio}
# 由于omni需要使用tun,而系统默认没有打开tun,此处通过modprobe加载
sudo modprobe tun
lsmod |grep tun
准备.env文件,其用于Docker Compose启动参数
cat > /usr/local/omni/.env <<EOF
POSTGRES_VERSION=16
KC_VERSION=25.0
KC_PORT=8443
KC_LOG_LEVEL=INFO
KC_REALM_NAME=omni
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=VMware1!
PG_USER=keycloak
PG_PASSWORD=VMware1!
OMNI_IP=10.40.43.112
OMNI_VERSION=v0.42.3
#此UUID可以uuidgen生成
OMNI_ACCOUNT_UUID=b5841399-88f7-4129-98a5-49f4bff92671
MINIO_VERSION=RELEASE.2024-10-02T17-50-41Z
EOF
准备docker-compose.yml文件,用于启动环境
cat /usr/local/omni/docker-compose.yml
version: '3.8'
x-logging: &logging
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
services:
postgres:
image: postgres:${POSTGRES_VERSION}
restart: unless-stopped
healthcheck:
test: ["CMD", "pg_isready", "-U", "keycloak"]
environment:
POSTGRES_DB: keycloak
POSTGRES_USER: ${PG_USER}
POSTGRES_PASSWORD: ${PG_PASSWORD}
volumes:
- ./data/postgres:/var/lib/postgresql/data
networks:
- omni
logging: *logging
keycloak:
image: quay.io/keycloak/keycloak:${KC_VERSION}
command: ["start", "--https-port=8443"]
restart: unless-stopped
environment:
KC_DB: postgres
KC_DB_USERNAME: ${PG_USER}
KC_DB_PASSWORD: ${PG_PASSWORD}
KC_DB_URL: "jdbc:postgresql://postgres:5432/keycloak"
KC_METRICS_ENABLED: false
KC_LOG_LEVEL: ${KC_LOG_LEVEL}
KC_REALM_NAME: ${KC_REALM_NAME}
KEYCLOAK_ADMIN: ${KEYCLOAK_ADMIN}
KEYCLOAK_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD}
KC_HOSTNAME: ${OMNI_IP}
KC_TRUSTSTORE_PATHS: /opt/keycloak/certificates/truststore
KC_HTTPS_CERTIFICATE_FILE: /opt/keycloak/certificates/omni-chain.pem
KC_HTTPS_CERTIFICATE_KEY_FILE: /opt/keycloak/certificates/omni-key.pem
ports:
- ${KC_PORT}:8443
volumes:
- ./data/keycloak:/opt/keycloak/data:rw
- ./certs/omni-chain.pem:/opt/keycloak/certificates/omni-chain.pem:ro
- ./certs/omni-key.pem:/opt/keycloak/certificates/omni-key.pem:ro
- ./certs/omni-ca.pem:/opt/keycloak/certificates/truststore/omni-ca.pem:ro
networks:
- omni
depends_on:
postgres:
condition: service_healthy
restart: true
logging: *logging
omni:
image: ghcr.io/siderolabs/omni
container_name: omni
restart: unless-stopped
network_mode: host
cap_add:
- NET_ADMIN
volumes:
- ./etcd:/_out/etcd
- ./certs/omni-chain.pem:/omni-chain.pem
- ./certs/omni-key.pem:/omni-key.pem
- ./certs/omni-ca.pem:/etc/ssl/certs/omni-ca.pem
- ./omni.asc:/omni.asc
devices:
- "/dev/net/tun:/dev/net/tun"
command: >
--account-id=${OMNI_ACCOUNT_UUID}
--cert=/omni-chain.pem
--key=/omni-key.pem
--siderolink-api-cert=/omni-chain.pem
--siderolink-api-key=/omni-key.pem
--private-key-source=file:///omni.asc
--event-sink-port=8091
--bind-addr=0.0.0.0:443
--siderolink-api-bind-addr=0.0.0.0:8090
--k8s-proxy-bind-addr=0.0.0.0:8100
--advertised-api-url=https://${OMNI_IP}:443/
--siderolink-api-advertised-url=https://${OMNI_IP}:8090/
--siderolink-wireguard-advertised-addr=${OMNI_IP}:50180
--advertised-kubernetes-proxy-url=https://${OMNI_IP}:8100/
--auth-auth0-enabled=false
--auth-saml-enabled
--auth-saml-url=https://${OMNI_IP}:8443/realms/omni/protocol/saml/descriptor
--talos-installer-registry=${OMNI_IP}:5000/siderolabs/installer
--kubernetes-registry=${OMNI_IP}:5000/siderolabs/kubelet
--image-factory-address=http://${OMNI_IP}:8080
--image-factory-pxe-address=http://${OMNI_IP}:8080
--registry-mirror=docker.io=http://${OMNI_IP}:5000,gcr.io=http://${OMNI_IP}:5000,ghcr.io=http://${OMNI_IP}:5000,registry.k8s.io=http://${OMNI_IP}:5000,factory.talos.dev=http://${OMNI_IP}:8080
depends_on:
keycloak:
condition: service_started
restart: true
logging: *logging
image-factory:
image: "ghcr.io/siderolabs/image-factory:v0.5.0"
restart: unless-stopped
container_name: image-factory
privileged: true
ports:
- "8080:8080"
environment:
SIGSTORE_ROOT_FILE: /root.pem
SIGSTORE_REKOR_PUBLIC_KEY: /cosign.pub
SIGSTORE_CT_LOG_PUBLIC_KEY_FILE: /cosign.pub
volumes:
- /dev:/dev
- /sys:/sys
- ./cache-signing-key.key:/cache-signing-key.key
- ./certs/import-cosign.key:/cosign.key
- ./certs/import-cosign.pub:/cosign.pub
- ./tmp:/tmp
- ./certs/omni-ca.pem:/root.pem
- ./secureboot_keys/uki-signing-key.pem:/uki-signing-key.pem
- ./secureboot_keys/uki-signing-cert.pem:/uki-signing-cert.pem
- ./secureboot_keys/pcr-signing-key.pem:/pcr-signing-key.pem
command: >
-http-port=0.0.0.0:8080
-image-registry ${OMNI_IP}:5000
-external-url http://${OMNI_IP}:8080
-schematic-service-repository ${OMNI_IP}:5000/image-factory/schematic
-installer-internal-repository ${OMNI_IP}:5000/siderolabs
-installer-external-repository ${OMNI_IP}:5000/siderolabs
-cache-repository ${OMNI_IP}:5000/cache
-secureboot
-secureboot-pcr-key-path /pcr-signing-key.pem
-secureboot-signing-key-path /uki-signing-key.pem
-secureboot-signing-cert-path /uki-signing-cert.pem
-insecure-image-registry
-insecure-cache-repository
-insecure-schematic-service-repository
-insecure-installer-internal-repository
-cache-signing-key-path /cache-signing-key.key
-container-signature-pubkey /cosign.pub
-container-signature-issuer http://localhost
logging: *logging
minio:
image: quay.io/minio/minio:RELEASE.2024-10-02T17-50-41Z
container_name: minio
restart: unless-stopped
ports:
- "9000:9000"
- "9001:9001"
environment:
MINIO_ROOT_USER: admin
MINIO_ROOT_PASSWORD: VMware1!
volumes:
- ~/data/minio:/data
command: server /data --console-address ":9001"
networks:
omni:
离线Linux配置—打开防火墙端口
sudo firewall-cmd --permanent \
--add-port=8080/tcp \
--add-port=8090/tcp \
--add-port=8100/tcp \
--add-port=8091/tcp \
--add-port=8091/udp \
--add-port=8090/udp \
--add-port=8100/udp \
--add-port=50180/tcp \
--add-port=50180/udp \
--add-port=8092/tcp \
--add-port=8092/udp \
--add-port=10000/tcp \
--add-port=10000/udp \
--add-port=8093/tcp \
--add-port=8093/udp \
--add-port=50001/tcp \
--add-port=50001/udp \
--add-port=5000/tcp \
--add-port=8200/tcp \
--add-port=80/tcp \
--add-port=443/tcp \
--add-port=8443/tcp \
--add-port=9000/tcp \
--add-port=9001/tcp
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" forward-port port="80" protocol="tcp" to-port="443"'
sudo firewall-cmd --reload
启动环境
cd /usr/local/omni
docker compose up -d
配置Keycloak Realm
配置完成后,再次启用omni容器(因为Keycloak未配置时,Omni会启动失败)
docker compose up -d
此时可以看到Omini容器已经可以正常启动
测试Omni
您可以在Windows/Linux机器中添加omni-ca.pem的根证书信任,这样浏览器就不会再弹出警告,具备方法自行搜索,这里不做示例。
首先打开浏览器登录omn(https://10.40.43.112) ,自动跳转到keycloak认证
认证成功后,点击“Log In”登录
登录后,我们可以看到Omni的管理页面,包括API Endpoint信息等
先测试本地构建image是否可以工作,点击右侧的“Download Installation Media”,我们可以看到Omni自动识别了已在离线Registry上传的Talos Linux版本(1.7.4-1.8.0),第二个选项中可以选择Image类型(ISO、VMware、Openstack等),在添加所需的Extension,还可以添加额外标签、内核参数,也可以生成PXE启动URL。
由于我们使用了自建名证书,会导致节点启动后无法连接omni,这里需要把证书信任添加到Kernel参数中
echo "apiVersion: v1alpha1
kind: TrustedRootsConfig
name: root-ca
certificates: |
$(cat certs/omni-ca.pem | sed 's/^/ /')" > trusted-roots-config.yaml
cat trusted-roots-config.yaml |zstd --compress --ultra -22 | base64 -w 0
Kernel参数:
echo talos.config.inline=<base64>
这里我选择1.8.0和VMware平台,点击“Download”
构建完成后,浏览器会自动下载“vmware-amd64-omni-default-v1.8.0.ova”
通过vCenter部署OVA,不需要提供任何额外的参数,因为所有参数都通过kernel Arguments传递了。可以通过Console看到,虚拟机已经启动,并连接到Omni;
在Omni管理页面上,也可以看到机器;
下面,我们创建一个单节点K8S集群;我们可以选择Kubernetes的版本(Talos版本需要和运行的版本一致,此处选择1.8.0),Kubernetes的版本是从仓库中读取出来的,根据需要选择;选择可用的节点,点击“Create Cluster”
还需要提供“Config Patch”,用于信任Omni CA证书,否则Node重启后,无法连接到Omni;
echo "apiVersion: v1alpha1
kind: TrustedRootsConfig
name: root-ca
certificates: |
$(cat certs/omni-ca.pem | sed 's/^/ /')" > trusted-roots-config.yaml
echo trusted-root-config.yaml
最终的Config Path如下所示:
apiVersion: v1alpha1
kind: TrustedRootsConfig
name: omni-ca
certificates: |
-----BEGIN CERTIFICATE-----
MIIDmDCCAoCgAwIBAgIUbHcCbcWgVQytszXCKlgeP2YcR/8wDQYJKoZIhvcNAQEL
BQAwYzELMAkGA1UEBhMCQ04xDjAMBgNVBAgTBUppbGluMRIwEAYDVQQHEwlDaGFu
Z2NodW4xDzANBgNVBAoTBkRldk9wczENMAsGA1UECxMET01OSTEQMA4GA1UEAxMH
T21uaSBDQTAgFw0yNDEwMDQxMzA3MDBaGA8yMDc0MDkyMjEzMDcwMFowYzELMAkG
A1UEBhMCQ04xDjAMBgNVBAgTBUppbGluMRIwEAYDVQQHEwlDaGFuZ2NodW4xDzAN
BgNVBAoTBkRldk9wczENMAsGA1UECxMET01OSTEQMA4GA1UEAxMHT21uaSBDQTCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMJqM71fjKYGuu2rn2glElBp
lhz3I9czd3dZyhos2WMEvGKHZ59fzDJR/ft/rRwTg0ORwq3u++wLBSjdY8SPIZoQ
JCKfBoL0n/wlDT8cD38efnj4OSxkzO1oMvwsdT6pr39BpcKax8miGKxX+V7i2ums
ed8sxlqKhe9IP2nD25Z6KEiXilovZ3sj9Vrwx2OOY87eOHzHQvR+X2veoXk+qLTM
TFc1R1Sf6w+4A67W0e5EznThzMKTVGFmteKYl1ZpuhK82VqE89XW+riqpXM5XwkL
n4iJBeaSYilfQaq0+haBk30KjuTHoCpvZ9N/cn1Kmk2OUHs9sLLMVtYJOW3hA+0C
AwEAAaNCMEAwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0O
BBYEFHqh+kEAUo11lunyYbkGjRw/IcraMA0GCSqGSIb3DQEBCwUAA4IBAQAyhlH7
CaXcbZxq2FM/1rSwy0q0y1Aud901RQY4OYds/6pSvIewNneFK4+hb1n39PujiMA4
SfzmmibbrIYGfwjn60P2WqCY73xweJrmiSrxhz6Hp9DcFVrH4CwKhATClvc5NZan
vVRRSfNhQE9fQqfqY4kZ0P2C76od0GUB4U71b2Ih2gXD1ciWSBSz3WuXQGF7qNsY
hkBWzJz4zKd3JRjmNrsCKgzWFwmaSKXRIbHnx6gKav1INiteM9iH8PKbzW2wvmCr
vL96XSkMf6tLpZELpEQNO3wCgOBesS1KuLLdVZ19agTOj7PUifzyBQ4T4Jrzglze
jmu/SjbzXlVHIkZe
-----END CERTIFICATE-----
稍等片刻,Cluster创建完成。
虚拟机Console显示“Healthy”状态
通Omni查看Node的状态、控制台日志和配置
完成
由于Omni和Talos涉及大量技术依赖,离线环境部署还是比较复杂的,所以,本博客的内容也非常多,我也是尽量把有用的内容放进来,减少大家再去研究。
当然,这也不仅仅适用于Omni,如果单独运行Talos Linux的离线环境,也适用,只需要做好Image部分的离线就可以。