由来 内部有需求需要静态编译 nginx,尝试了下,搞出来了。先是按照官方 nginx Dockerfile 的逻辑走不通,后面下载 nginx 官方源码编译才行。
buildx 使用 见文章 buildx 使用
nginx Dockerfile 先说下官方的失败尝试。先 clone 项目:
1 2 git clone https://github.com/nginxinc/docker-nginx.git cd docker-nginx
分为 stable
和 mainline
。大概研究了下,发现 case "$apkArch" in x86_64|aarch64)
的情况是利用包管理直接安装的,其他的架构才是源码编译安装。改了下这个 case ,先用 arm64 的试下走源码编译。然后看下逻辑是下载源码进去 make all
:
1 2 3 4 5 curl -f -O https://hg.nginx.org/pkg-oss/archive/${NGINX_VERSION}-${PKG_RELEASE}.tar.gz tar xzvf ${NGINX_VERSION}-${PKG_RELEASE}.tar.gz && cd pkg-oss-${NGINX_VERSION}-${PKG_RELEASE} \ && cd alpine \ && make all
而源码里 Makefile
的前面内容如下:
1 2 3 4 5 6 7 8 9 10 11 12 ifeq ($(MODULE_TARGET), plus) APKBUILD_TEMPLATE= APKBUILD-plus-module.in MODULE_SUFFIX= -plus MODULE_SUMMARY_PREFIX=NGINX Plus TARGET_VERSION=$(PLUS_VERSION) MODULE_PACKAGE_PREFIX=nginx-plus-module else APKBUILD_TEMPLATE= APKBUILD-module.in MODULE_SUMMARY_PREFIX=nginx TARGET_VERSION=$(BASE_VERSION) MODULE_PACKAGE_PREFIX=nginx-module endif
因为不是构建 nginx-plus
所以我们看 cat APKBUILD-module.in
里找到了下面的:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 build() { cd "$builddir" _nproc=`getconf _NPROCESSORS_ONLN` if [ $_nproc -gt 1 ]; then _make_opts="-j$_nproc" fi %%MODULE_PREBUILD%% cd "$builddir" CFLAGS= %%MODULE_ENV%% ./configure %%BASE_CONFIGURE_ARGS%% %%MODULE_CONFIGURE_ARGS%% --with-cc-opt="$CFLAGS %%MODULE_CC_OPT_DEBUG%%" --with-ld-opt="$LDFLAGS %%MODULE_LD_OPT_DEBUG%%" --with-debug make $_make_opts modules for so in `find objs/ -maxdepth 1 -type f -name "*.so"`; do \ debugso=`echo ${so} | sed -e 's|\.so$|-debug.so|'` ; \ mv ${so} ${debugso} ; \ done CFLAGS= %%MODULE_ENV%% ./configure %%BASE_CONFIGURE_ARGS%% %%MODULE_CONFIGURE_ARGS%% --with-cc-opt="$CFLAGS %%MODULE_CC_OPT%%" --with-ld-opt="$LDFLAGS %%MODULE_LD_OPT%%" make $_make_opts modules }
CFLAGS
和 LDFLAGS
是支持环境变量传入的,所以改下 Dockerfile 加入下面的:
然后开始构建:
1 2 3 docker buildx build -t zhangguanzhang/nginx:arm64-static . \ --platform linux/arm64 \ --build-arg=CFLAGS="-static -s" --build-arg=LDFLAGS=-static
然后失败,报错 ./configure: error: the invalid value in --with-ld-opt="-static"
:
1 2 3 4 5 6 7 8 9 #6 481.1 >>> nginx: Unpacking /tmp/tmp.OciAKf/pkg-oss-1.21.5-1/alpine/abuild-base/nginx-1.21.5.tar.gz... #6 482.9 checking for OS #6 482.9 + Linux 5.4.0-91-generic aarch64 #6 482.9 checking for C compiler ... found #6 483.6 + using GNU C compiler #6 483.7 + gcc version: 10.3.1 20211027 (Alpine 10.3.1_git20211027) #6 483.7 checking for gcc -pipe switch ... found #6 484.3 checking for --with-ld-opt="-static" ... not found #6 484.3 ./configure: error: the invalid value in --with-ld-opt="-static"
然后在 Dockerfile 里 apk add 加了 glibc-static
还是一样报错。然后尝试下官方的
自己编译 手动步骤 1 docker run --rm -ti --name t1 test alpine
前置依赖
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 if [ -f /etc/apk/repositories ];then sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories; fi && \ if [ -f /etc/apt/sources.list ];then sed -ri 's/(deb|security).debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list; fi && \ if [ ! -e /etc/nsswitch.conf ];then echo 'hosts: files dns myhostname' > /etc/nsswitch.conf; fi apk add --no-cache --virtual .build-deps \ gcc \ libc-dev \ make \ openssl-dev \ pcre2-dev \ zlib-dev \ openssl-libs-static zlib-static \ linux-headers \ libxslt-dev \ gd-dev \ geoip-dev \ perl-dev \ libedit-dev \ bash \ alpine-sdk \ findutils bash
下载源码包
1 2 3 wget https://nginx.org/download/nginx-1.21.6.tar.gz tar zxf nginx-1.21.6.tar.gz cd nginx-1.21.6
找下官方的编译参数,下面是我 arm64 上:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 $ docker run --rm --entrypoint nginx nginx:alpine-perl -V nginx version: nginx/1.21.6 built by gcc 10.3.1 20211027 (Alpine 10.3.1_git20211027) built with OpenSSL 1.1.1l 24 Aug 2021 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-perl_modules_path=/usr/lib/perl5/vendor_perl --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-Os -fomit-frame-pointer -g' --with-ld-opt=-Wl,--as-needed,-O1,--sort-common $ docker run --rm --entrypoint nginx nginx:alpine-perl -V |& grep -Po -- "--[a-z_-]+(=('[^']+'|\S+))?" --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-perl_modules_path=/usr/lib/perl5/vendor_perl --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-Os -fomit-frame-pointer -g' --with-ld-opt=-Wl,--as-needed,-O1,--sort-common
with-http_mp
和 with-http_v
似乎对应 --with-http_mp4_module
和 --with-http_v2_module
,更多编译参数参考官方文档 nginx-configure 改下 --with-cc-opt
和 --with-ld-opt
后编译
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 ./configure \ --prefix=/etc/nginx \ --sbin-path=/usr/sbin/nginx \ --modules-path=/usr/lib/nginx/modules \ --conf-path=/etc/nginx/nginx.conf \ --error-log-path=/var/log/nginx/error.log \ --http-log-path=/var/log/nginx/access.log \ --pid-path=/var/run/nginx.pid \ --lock-path=/var/run/nginx.lock \ --http-client-body-temp-path=/var/cache/nginx/client_temp \ --http-proxy-temp-path=/var/cache/nginx/proxy_temp \ --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \ --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \ --http-scgi-temp-path=/var/cache/nginx/scgi_temp \ --with-perl_modules_path=/usr/lib/perl5/vendor_perl \ --user=nginx \ --group=nginx \ --with-compat \ --with-file-aio \ --with-threads \ --with-http_addition_module \ --with-http_auth_request_module \ --with-http_dav_module \ --with-http_flv_module \ --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_mp4_module \ --with-http_random_index_module \ --with-http_realip_module \ --with-http_secure_link_module \ --with-http_slice_module \ --with-http_ssl_module \ --with-http_stub_status_module \ --with-http_sub_module \ --with-http_v2_module \ --with-mail \ --with-mail_ssl_module \ --with-stream \ --with-stream_realip_module \ --with-stream_ssl_module \ --with-stream_ssl_preread_module \ --with-cc-opt='-static -s' \ --with-ld-opt=-static make make install
Dockerfile 构建 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 FROM alpine AS buildARG NGINX_VERSION=1.21 .6 WORKDIR /opt RUN if [ -f /etc/apk/repositories ];then sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories; fi && \ if [ -f /etc/apt/sources.list ];then sed -ri 's/(deb|security).debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list; fi && \ if [ ! -e /etc/nsswitch.conf ];then echo 'hosts: files dns myhostname' > /etc/nsswitch.conf; fi && \ apk add --no-cache --virtual .build-deps \ gcc \ libc-dev \ make \ openssl-dev \ pcre2-dev \ zlib-dev \ openssl-libs-static zlib-static \ linux-headers \ libxslt-dev \ gd-dev \ geoip-dev \ perl-dev \ libedit-dev \ bash \ alpine-sdk \ findutils && \ wget http://nginx.org/download/nginx-${NGINX_VERSION} .tar.gz && tar zxf nginx-${NGINX_VERSION} .tar.gz RUN cd nginx-${NGINX_VERSION} && \ ./configure \ --prefix=/etc/nginx \ --sbin-path=/usr/sbin/nginx \ --modules-path=/usr/lib/nginx/modules \ --conf-path=/etc/nginx/nginx.conf \ --error-log-path=/var/log/nginx/error.log \ --http-log-path=/var/log/nginx/access.log \ --pid-path=/var/run/nginx.pid \ --lock-path=/var/run/nginx.lock \ --http-client-body-temp-path=/var/cache/nginx/client_temp \ --http-proxy-temp-path=/var/cache/nginx/proxy_temp \ --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \ --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \ --http-scgi-temp-path=/var/cache/nginx/scgi_temp \ --with-perl_modules_path=/usr/lib/perl5/vendor_perl \ --user=nginx \ --group=nginx \ --with-compat \ --with-file-aio \ --with-threads \ --with-http_addition_module \ --with-http_auth_request_module \ --with-http_dav_module \ --with-http_flv_module \ --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_mp4_module \ --with-http_random_index_module \ --with-http_realip_module \ --with-http_secure_link_module \ --with-http_slice_module \ --with-http_ssl_module \ --with-http_stub_status_module \ --with-http_sub_module \ --with-http_v2_module \ --with-mail \ --with-mail_ssl_module \ --with-stream \ --with-stream_realip_module \ --with-stream_ssl_module \ --with-stream_ssl_preread_module \ --with-cc-opt='-static -s' \ --with-ld-opt=-static && \ make && \ make install && \ cp `which nginx` /nginx-$(cat /etc/apk/arch) FROM scratch AS binCOPY --from=build /nginx-* /
构建
1 2 3 docker buildx build . --platform linux/amd64,linux/arm64 \ --target bin --output . \ --build-arg=NGINX_VERSION=1.21.6
结果:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 # ll total 4 drwxr-xr-x 4 root root 62 Jan 26 16:21 ./ drwxr-xr-x 6 root root 98 Jan 26 15:31 ../ -rw-r--r-- 1 root root 2718 Jan 26 16:05 Dockerfile drwxr-xr-x 2 root root 26 Jan 26 16:21 linux_amd64/ drwxr-xr-x 2 root root 27 Jan 26 16:21 linux_arm64/ # ll linux_a* linux_amd64: total 20424 drwxr-xr-x 2 root root 26 Jan 26 16:21 ./ drwxr-xr-x 4 root root 62 Jan 26 16:21 ../ -rwxr-xr-x 1 root root 20910696 Jan 26 16:12 nginx-x86_64* linux_arm64: total 20444 drwxr-xr-x 2 root root 27 Jan 26 16:21 ./ drwxr-xr-x 4 root root 62 Jan 26 16:21 ../ -rwxr-xr-x 1 root root 20932656 Jan 26 16:21 nginx-aarch64* # ./linux_amd64/nginx-x86_64 -V nginx version: nginx/1.21.6 built by gcc 10.3.1 20211027 (Alpine 10.3.1_git20211027) built with OpenSSL 1.1.1l 24 Aug 2021 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-perl_modules_path=/usr/lib/perl5/vendor_perl --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-static -s' --with-ld-opt=-static # file ./linux_amd64/nginx-x86_64 ./linux_amd64/nginx-x86_64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped # file ./linux_arm64/nginx-aarch64 ./linux_arm64/nginx-aarch64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, with debug_info, not stripped
符号链接可以 strip -s $(which nginx)
去掉减少大小。
对接到真实场景 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 FROM nginx:alpine AS confFROM alpine AS buildARG NGINX_VERSION=1.21 .6 WORKDIR /opt RUN if [ -f /etc/apk/repositories ];then sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories; fi && \ if [ -f /etc/apt/sources.list ];then sed -ri 's/(deb|security).debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list; fi && \ if [ ! -e /etc/nsswitch.conf ];then echo 'hosts: files dns myhostname' > /etc/nsswitch.conf; fi && \ apk add --no-cache --virtual .build-deps \ gcc \ libc-dev \ make \ openssl-dev \ pcre2-dev \ zlib-dev \ openssl-libs-static zlib-static \ linux-headers \ libxslt-dev \ gd-dev \ geoip-dev \ perl-dev \ libedit-dev \ bash \ alpine-sdk \ findutils && \ wget http://nginx.org/download/nginx-${NGINX_VERSION} .tar.gz && tar zxf nginx-${NGINX_VERSION} .tar.gz RUN cd nginx-${NGINX_VERSION} && \ ./configure \ --prefix=/etc/nginx \ --sbin-path=/usr/sbin/nginx \ --modules-path=/usr/lib/nginx/modules \ --conf-path=/etc/nginx/nginx.conf \ --error-log-path=/var/log/nginx/error.log \ --http-log-path=/var/log/nginx/access.log \ --pid-path=/var/run/nginx.pid \ --lock-path=/var/run/nginx.lock \ --http-client-body-temp-path=/var/cache/nginx/client_temp \ --http-proxy-temp-path=/var/cache/nginx/proxy_temp \ --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \ --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \ --http-scgi-temp-path=/var/cache/nginx/scgi_temp \ --with-perl_modules_path=/usr/lib/perl5/vendor_perl \ --user=nginx \ --group=nginx \ --with-compat \ --with-file-aio \ --with-threads \ --with-http_addition_module \ --with-http_auth_request_module \ --with-http_dav_module \ --with-http_flv_module \ --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_mp4_module \ --with-http_random_index_module \ --with-http_realip_module \ --with-http_secure_link_module \ --with-http_slice_module \ --with-http_ssl_module \ --with-http_stub_status_module \ --with-http_sub_module \ --with-http_v2_module \ --with-mail \ --with-mail_ssl_module \ --with-stream \ --with-stream_realip_module \ --with-stream_ssl_module \ --with-stream_ssl_preread_module \ --with-cc-opt='-static -s' \ --with-ld-opt=-static RUN cd nginx-${NGINX_VERSION} && \ mkdir /install_root && \ make && \ make DESTDIR=/install_root install && \ rm -f /install_root/etc/nginx/*.default && \ rmdir /install_root/var/run ; true && \ mkdir -p /install_root/var/cache/nginx/ \ /install_root/usr/lib/nginx/modules \ /install_root/etc/nginx/conf.d \ /install_root/usr/share/nginx && \ mv /install_root/etc/nginx/html /install_root/usr/share/nginx/ && \ ln -sf /dev/stdout /install_root/var/log/nginx/access.log && \ ln -sf /dev/stderr /install_root/var/log/nginx/error.log COPY --from=conf /etc/nginx/nginx.conf /install_root/etc/nginx/nginx.conf COPY --from=conf /etc/nginx/conf.d /install_root/etc/nginx/conf.d RUN find /install_root FROM alpineCOPY --from=build /install_root / RUN addgroup -g 101 -S nginx && \ adduser -S -D -H -u 101 -h /var/cache/nginx -s /sbin/nologin -G nginx -g nginx nginx && \ if [ ! -e /etc/nsswitch.conf ];then echo 'hosts: files dns myhostname' >> /etc/nsswitch.conf; fi
注意 worker_processes auto;
在高核心 cpu 上会非常吃配置和浪费配置,特别是 arm64 的国产服务器上,需要改成固定的数字。
后续一些额外模块测试 vts 模块验证不影响静态
1 2 3 4 5 6 7 wget https://github.com/vozlt/nginx-module-vts/archive/refs/tags/v0.1.18.tar.gz && \ tar zxf v0.1.18.tar.gz -C /tmp/ && \ rm -rf v0.1.18.tar.gz RUN cd nginx-${NGINX_VERSION} && \ ... --add-module=/tmp/nginx-module-vts-0.1.18 \
参考