zhangguanzhang's Blog

编译mips64le架构的consul

字数统计: 1.1k阅读时长: 6 min
2021/07/16

编译

建议使用容器编译,否则建议 clone 进 GOPATH 里

clone

1
2
git clone https://github.com/hashicorp/consul.git
cd consul

线上使用的是 v1.8 版本,这里我以 v1.8.14 (2021/07/19 发布的)搞的。

1
git checkout v1.8.14

准备工作

拉取需要的镜像。貌似 golang 1.16 更好的支持 mips64 了,所以条件允许的话,这里可以下改下 golang 的版本试试

1
2
3
4
$ head -n2 build-support/docker/Build-Go.dockerfile
ARG GOLANG_VERSION=1.14.11
FROM golang:${GOLANG_VERSION}
$ docker pull golang:1.14.11

开始编译

相关变量可以查看 GNUmakefile ,实际会在容器里运行 build-support/functions/20-build.sh 里的 function build_consul

1
2
3
4
5
6
7
local container_id=$(docker create -it \
${volume_mount} \
-e CGO_ENABLED=0 \
-e GOLDFLAGS="${GOLDFLAGS}" \
-e GOTAGS="${GOTAGS}" \
${image_name} \
./build-support/scripts/build-local.sh -o "${XC_OS}" -a "${XC_ARCH}")

而在容器里执行build-support/scripts/build-local.sh -o "${XC_OS}" -a "${XC_ARCH}" 会执行 build-support/functions/20-build.sh 里的 function build_consul_local,主要编译是下面的这行:

1
debug_run env CGO_ENABLED=0 GOOS=${os} GOARCH=${arch} go install -ldflags "${GOLDFLAGS}" -tags "${GOTAGS}" && cp "${MAIN_GOPATH}/bin/${GOBIN_EXTRA}${binname}" "${outdir}/${binname}"

开始编译

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
export DOCKER_BUILD_QUIET=0
export XC_OS=linux XC_ARCH=mips64le
make consul-docker


==> Building Consul - OSes: linux, Architectures: mips64le
Building sequentially with go install
---> linux/mips64le
# github.com/boltdb/bolt
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/bolt_unix.go:62:15: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/bucket.go:135:15: undefined: brokenUnaligned
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:101:13: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:317:12: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:335:10: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:336:8: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/freelist.go:169:19: undefined: maxAllocSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/freelist.go:176:14: undefined: maxAllocSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/freelist.go:204:17: undefined: maxAllocSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/freelist.go:207:7: undefined: maxAllocSize
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/freelist.go:207:7: too many errors
ERROR: Failed to build Consul for linux/mips64le
make: *** [consul-docker] Error 1

编译报错,是因为 boltdb 的库现在 archive 了。而且相关文件用 golang 的 build tag 区分架构了,没有预置 mips64le 的,搜了几个 issue 后搞了下编译还是失败了,仔细看上面的 path,没有读取 vendor 目录,得改编译容器里的 mod 目录下的 boltdb 文件。所以得 hack 下。

hack

先产生临时文件/tmp/hack_consul.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
cat > /tmp/hack_consul.sh << 'consul_eof'
if [ $arch == "mips64le" ];then
cat > $(go env GOPATH)/pkg/mod/github.com/boltdb/bolt@v1.3.1/bolt_mips64x.go <<'EOF'
// +build mips64 mips64le

package bolt

// maxMapSize represents the largest mmap size supported by Bolt.
const maxMapSize = 0x8000000000 // 512GB

// maxAllocSize is the size used when creating array pointers.
const maxAllocSize = 0x7FFFFFFF

// Are unaligned load/stores broken on this arch?
var brokenUnaligned = false
EOF

cat > $(go env GOPATH)/pkg/mod/github.com/boltdb/bolt@v1.3.1/bolt_mips.go <<'EOF'
// +build mips mipsle

package bolt

// maxMapSize represents the largest mmap size supported by Bolt.
const maxMapSize = 0x40000000 // 1GB

// maxAllocSize is the size used when creating array pointers.
const maxAllocSize = 0xFFFFFFF

// Are unaligned load/stores broken on this arch?
var brokenUnaligned = false
EOF
fi
consul_eof

取 go install 命令的行数,在前面一行插入 hack 的脚本

1
2
3
4
5

CONSUL_HACK_NUM=$(grep -Pn 'debug_run.+?go install' build-support/functions/20-build.sh | cut -d: -f 1)

sed -i "$[CONSUL_HACK_NUM-1]r /tmp/hack_consul.sh" build-support/functions/20-build.sh

继续编译

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$ make consul-docker
Building Golang build container
Sending build context to Docker daemon 2.048kB
Step 1/5 : ARG GOLANG_VERSION=1.14.11
Step 2/5 : FROM golang:${GOLANG_VERSION}
---> f93db70cba35
Step 3/5 : ARG GOTOOLS="github.com/elazarl/go-bindata-assetfs/... github.com/hashicorp/go-bindata/... golang.org/x/tools/cmd/cover golang.org/x/tools/cmd/stringer github.com/axw/gocov/gocov gopkg.in/matm/v1/gocov-html"
---> Using cache
---> e0979c09e72f
Step 4/5 : RUN GO111MODULE=on go get -v ${GOTOOLS} && mkdir -p /consul
---> Using cache
---> 9cb7abf49b86
Step 5/5 : WORKDIR /consul
---> Using cache
---> fb1467e4623c
Successfully built fb1467e4623c
Successfully tagged consul-build-go:latest
==> Building Consul
Ensuring Go modules are up to date
Creating the Go Build Container with image: consul-build-go
Copying the source from '/root/go/src/github.com/hashicorp/consul' to /consul
Running build in container
==> Building Consul - OSes: linux, Architectures: mips64le
Building sequentially with go install
---> linux/mips64le
Copying back artifacts

$ ls -l pkg/bin/linux_mips64le/
total 112192
-rwxr-xr-x 1 root root 114881061 Jul 16 12:34 consul

测试

拷贝到龙芯机器上测试下可否运行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ lscpu
Architecture: mips64el
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 4
座: 2
NUMA 节点: 2
型号名称: Loongson-3A R4 (Loongson-3B4000)
CPU max MHz: 1800.0000
CPU min MHz: 900.0000
BogoMIPS: 3594.02
L1d 缓存: 64K
L1i 缓存: 64K
L2 缓存: 256K
L3 缓存: 2048K
NUMA 节点0 CPU: 0-3
NUMA 节点1 CPU: 4-7
$ /tmp/consul --version
Consul v1.8.14
Revision 94c1bdd3b+CHANGES
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

参考

CATALOG
  1. 1. 编译
    1. 1.1. clone
    2. 1.2. 准备工作
    3. 1.3. 开始编译
    4. 1.4. hack
    5. 1.5. 继续编译
  2. 2. 测试
  3. 3. 参考