zhangguanzhang's Blog

ansible数量限制上的幂等性

ansible consul
字数统计: 2.7k阅读时长: 15 min
2019/10/11

由来

准备写上一篇文章的consul成ansible，在证书方面，cli和ca需要执行一次生成，而server和agent证书需要多次执行产生。另外我们也要考虑，dc1-server-consul-0*.pem需要拷贝到机器1，1拷贝到机器2.
如何在生成证书的时候中断后再次运行也会达到目标的数量来实现幂等性，而不是证书生成多了
另外证书还得按照index去拷贝到指定机器，也就是类似下面的

1
2
3
scp 001.pem ip1:/etc/xxx
scp 002.pem ip2:/etc/xxx
scp 003.pem ip3:/etc/xxx

场景1 – tls的幂等性

规划

这里我是三台server

1
2
3
4
5
6
7
8
[root@consul ansible]# cat inventory/hosts 
[server]
172.19.0.3
172.19.0.4
172.19.0.5
[client]
172.19.0.8
172.19.0.9

剧本文件见 https://github.com/zhangguanzhang/consul-tls-ansible 下面仅仅列出相关文件

1
2
3
4
5
6
7
8
[root@consul ansible]# ll
total 24
-rw-r--r-- 1 root root   84 Oct 11 20:18 ansible.cfg
drwxr-xr-x 2 root root 4096 Oct 11 20:20 group_vars
drwxr-xr-x 2 root root 4096 Oct 14 09:50 inventory
drwxr-xr-x 5 root root 4096 Oct 14 10:03 roles
-rw-r--r-- 1 root root   61 Oct 14 10:40 03-server.yml
-rw-r--r-- 1 root root   63 Oct 11 20:20 02-tls.yml

对比下四个证书，我们步骤都是一样，需要的action和一些属性，假设dc名设置为dc1，我们抽象来定义成loop

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
ca:
  command: consul tls ca create -days={{ tls_days }}
  file: 
    - consul-agent-ca.pem
    - consul-agent-ca-key.pem
  count: 1
cli:
  command: consul tls cert create -cli -days={{ tls_days }}
  file: 
    - dc1-cli-consul-0.pem
    - dc1-cli-consul-0-key.pem
  count: 1
server:
  command: consul tls cert create -server -days={{ tls_days }}
  file: 
    - dc1-server-consul-*.pem
    - dc1-server-consul-*-key.pem
  count: {{ groups['server'] | length }}
client:
  command: consul tls cert create -client -days={{ tls_days }}
  file: 
    - dc1-client-consul-*.pem
    - dc1-client-consul-*-key.pem
  count: {{ groups['client'] | length }}

每个证书的do步骤都可以为:

  1. 检查文件的数量
  2. 文件数量是否达到了count，没达到就生成

第一个步骤挺简单的，搜索了下使用find模块，返回值的matched是匹配的文件数量。因为consul每次生成一个都是cert+key，可以只看cert或者key。
server和client的证书是多个，而find模块的use_regex是支持正则的，我们给每个证书的loop属性加一个use_regex，最终的属性为

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
ca:
  command: consul tls ca create -days={{ tls_days }}
  use_regex: no
  file: 
    - consul-agent-ca.pem
  count: 1
cli:
  command: consul tls cert create -cli -days={{ tls_days }}
  use_regex: no
  file: 
    - dc1-cli-consul-0.pem
  count: 1
server:
  command: consul tls cert create -server -days={{ tls_days }}
  use_regex: no
  file: 
    - dc1-server-consul-*.pem
  count: {{ groups['server'] | length }}
client:
  command: consul tls cert create -client -days={{ tls_days }}
  use_regex: no
  file: 
    - dc1-client-consul-*.pem
  count: {{ groups['client'] | length }}

第一个步骤可以写成

1
2
3
4
5
6
7
8
- name: Check if the file already exists
  find: 
    paths: .
    file_type: file
    use_regex: "{{ out_var.use_regex }}"
    patterns:
      - "{{ out_var.file_name_patterns }}"
  register: state

第二个步骤可以写成,利用with_sequence来一个task执行多次，until控制非常麻烦，虽然能忽略错误，但是运行看着不友好

1
2
3
4
- name: create tls file for {{ out_var.type }}
  command: "{{ out_var.command }}"
  with_sequence: start=0 end={{ out_var.count }}
  when: state.matched < out_var.count|int

外层套一个loop_var

1
2
3
4
5
6
7
8
- include_tasks: do.yml
  loop_control:
    loop_var: out_var
  with_items:
    - { type: "ca", file_name_patterns: 'consul-agent-ca.pem', use_regex: 'no', command: 'consul tls ca create -days={{ tls_days }}', count: 1 }
    - { type: "cli", file_name_patterns: '{{ dc }}-cli-consul-0.pem', use_regex: 'no', command: 'consul tls cert create -cli -dc={{ dc }} -days={{ tls_days }}', count: 1 }
    - { type: "server", file_name_patterns: '{{ dc }}-server-consul-[0-9]+\.pem', use_regex: 'yes', command: 'consul tls cert create -server -dc={{ dc }} -days={{ tls_days }}', count: "{{ groups['server'] | length }}" }
    - { type: "client", file_name_patterns: '{{ dc }}-client-consul-[0-9]+\.pem', use_regex: 'yes', command: 'consul tls cert create -client -dc={{ dc }} -days={{ tls_days }}', count: "{{ groups['client'] | length }}" }

核心主要是第二个步骤实现幂等性，之前的好像是上面写的，当时的现象是多次运行后server和client的数量一直产生，并没有达到幂等性。因为每次with_sequence的次数都是期望数量的action次数。后面修改了下end减去匹配的文件数量，大致下面的

1
2
3
4
- name: create tls file for {{ out_var.type }}
  command: "{{ out_var.command }}"
  with_sequence: start=0 end={{ out_var.count|int - state.matched }}
  when: state.matched < out_var.count|int

但是这样第二次的时候end等于0，会报错Ansible - condition on with_sequence loop with variable end that could be less than start

在 https://stackoverflow.com/questions/44724805/ansible-condition-on-with-sequence-loop-with-variable-end-that-could-be-less-t
搜索到了一个写法

1
with_sequence: start=1 end={{countvar.stdout|int if countvar.stdout|int > 0 else 1}}

最终的步骤2为

1
2
3
4
5
- block:
  - name: create tls file for {{ out_var.type }}
    command: "{{ out_var.command }}"
    with_sequence: start=1 end={{ out_var.count|int - state.matched if state.matched < out_var.count|int else 1 }}
  when: state.matched < out_var.count|int

测试

现在来测试下，第一次执行生成证书

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
[root@consul ansible]# ansible-playbook 02-tls.yml -k
SSH password: 

PLAY [localhost] ***************************************************************************************************************************************************************************************************************************

TASK [tls : include_tasks] *****************************************************************************************************************************************************************************************************************
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost

...
[root@consul ansible]# ll
total 80
-rw-r--r-- 1 root root   84 Oct 11 20:18 ansible.cfg
-rw-r--r-- 1 root root  227 Oct 14 16:34 consul-agent-ca-key.pem
-rw-r--r-- 1 root root 1253 Oct 14 16:34 consul-agent-ca.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-cli-consul-0-key.pem
-rw-r--r-- 1 root root 1086 Oct 14 16:34 dc1-cli-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-client-consul-0-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-client-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-client-consul-1-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-client-consul-1.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-server-consul-0-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-server-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-server-consul-1-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-server-consul-1.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-server-consul-2-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-server-consul-2.pem
drwxr-xr-x 2 root root 4096 Oct 11 20:20 group_vars
drwxr-xr-x 2 root root 4096 Oct 14 09:50 inventory
drwxr-xr-x 5 root root 4096 Oct 14 10:03 roles
-rw-r--r-- 1 root root   61 Oct 14 10:40 03-server.yml
-rw-r--r-- 1 root root   63 Oct 11 20:20 02-tls.yml

证书已经生成完，执行第二次

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
[root@consul ansible]# ansible-playbook 02-tls.yml -k
SSH password: 

PLAY [localhost] ***************************************************************************************************************************************************************************************************************************

TASK [tls : include_tasks] *****************************************************************************************************************************************************************************************************************
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost

...
[root@consul ansible]# ll
total 80
-rw-r--r-- 1 root root   84 Oct 11 20:18 ansible.cfg
-rw-r--r-- 1 root root  227 Oct 14 16:34 consul-agent-ca-key.pem
-rw-r--r-- 1 root root 1253 Oct 14 16:34 consul-agent-ca.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-cli-consul-0-key.pem
-rw-r--r-- 1 root root 1086 Oct 14 16:34 dc1-cli-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-client-consul-0-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-client-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-client-consul-1-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-client-consul-1.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-server-consul-0-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-server-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-server-consul-1-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-server-consul-1.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-server-consul-2-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-server-consul-2.pem
drwxr-xr-x 2 root root 4096 Oct 11 20:20 group_vars
drwxr-xr-x 2 root root 4096 Oct 14 09:50 inventory
drwxr-xr-x 5 root root 4096 Oct 14 10:03 roles
-rw-r--r-- 1 root root   61 Oct 14 10:40 03-server.yml
-rw-r--r-- 1 root root   63 Oct 11 20:20 02-tls.yml

可以看到证书并没有多生成，现在删除一个证书，再次执行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
[root@consul ansible]# rm -f dc1-server-consul-0*
[root@consul ansible]# ansible-playbook test.yml -k
SSH password: 

PLAY [localhost] ***************************************************************************************************************************************************************************************************************************

TASK [tls : include_tasks] *****************************************************************************************************************************************************************************************************************
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost
included: /root/ansible/roles/tls/tasks/do.yml for localhost

...
[root@consul ansible]# ll
total 80
-rw-r--r-- 1 root root   84 Oct 11 20:18 ansible.cfg
-rw-r--r-- 1 root root  227 Oct 14 16:34 consul-agent-ca-key.pem
-rw-r--r-- 1 root root 1253 Oct 14 16:34 consul-agent-ca.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-cli-consul-0-key.pem
-rw-r--r-- 1 root root 1086 Oct 14 16:34 dc1-cli-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-client-consul-0-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-client-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-client-consul-1-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-client-consul-1.pem
-rw-r--r-- 1 root root  227 Oct 14 16:35 dc1-server-consul-0-key.pem
-rw-r--r-- 1 root root 1139 Oct 14 16:35 dc1-server-consul-0.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-server-consul-1-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-server-consul-1.pem
-rw-r--r-- 1 root root  227 Oct 14 16:34 dc1-server-consul-2-key.pem
-rw-r--r-- 1 root root 1143 Oct 14 16:34 dc1-server-consul-2.pem
drwxr-xr-x 2 root root 4096 Oct 11 20:20 group_vars
drwxr-xr-x 2 root root 4096 Oct 14 09:50 inventory
drwxr-xr-x 5 root root 4096 Oct 14 10:03 roles
-rw-r--r-- 1 root root   61 Oct 14 10:40 03-server.yml
-rw-r--r-- 1 root root   63 Oct 11 20:20 02-tls.yml

证书没有多生成，幂等性保证了

场景2 – index to index

这个场景主要是如何把index转成hostvars，这里实际上有个冷门的模板方法
利用play_hosts.index(inventory_hostname)配合set_fact实现，也就是

1
2
3
4
5
- name: set_fact for filename
  set_fact: 
    server_cert_index: "{{ '{0}'.format(play_hosts.index(inventory_hostname)) }}"
- name: copy cert
  debug: msg={{ dc }}-{{ role }}-consul-{{ server_cert_index }}.pem

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
[root@consul ansible]# ansible-playbook 03-server.yml -k
SSH password: 

PLAY [server] ******************************************************************************************************************************************************************************************************************************

TASK [server : create user for consul running] *********************************************************************************************************************************************************************************************
ok: [172.19.0.3]
ok: [172.19.0.4]
ok: [172.19.0.5]


TASK [server : copy cert] ******************************************************************************************************************************************************************************************************************
ok: [172.19.0.3] => {
    "msg": "dc1-server-consul-0.pem"
}
ok: [172.19.0.4] => {
    "msg": "dc1-server-consul-1.pem"
}
ok: [172.19.0.5] => {
    "msg": "dc1-server-consul-2.pem"
}

PLAY RECAP *********************************************************************************************************************************************************************************************************************************
172.19.0.3                 : ok=4    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
172.19.0.4                 : ok=4    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
172.19.0.5                 : ok=4    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

参考:

https://docs.ansible.com/ansible/latest/modules/find_module.html

https://docs.ansible.com/ansible/latest/plugins/lookup/sequence.html

原文作者：Zhangguanzhang

原文链接：http://zhangguanzhang.github.io/2019/10/11/ansible-count-Idempotency/

发表日期：October 11th 2019, 9:41:08 pm

更新日期：October 11th 2019, 9:41:08 pm

版权声明：本文采用知识共享署名-非商业性使用 4.0 国际许可协议进行许可

CATALOG
  1. 1. 由来
    1. 1.1. 场景1 – tls的幂等性
      1. 1.1.1. 规划
      2. 1.1.2. 测试
    2. 1.2. 场景2 – index to index
      1. 1.2.1. 测试
  2. 2. 参考:
Total : 186
2024
2023
2022
2021
2020
2019
2018
2017
2016
linux 微信 diy 8266 总结 ipmi CLP k8s Mismatch Altium Designer ansible consul docker lnmp 安卓 Linux kernel prometheus exporter boot grub cad tcp golang centos6 chromedp chrony kubernetes cni-plugins mips64le kubeconfig swap dlv adguardhome dns Dockerfile e2fsck fsck keepavlied ecs xmrig etcd fio flannel ftp go mod go gobot esp8266 hello world host-gw mqtt 原创 jira confluence kube-controller-manager java springboot vip keepalived ConfigMap PodAffinity kylin arm64 Kylin audit fs nginx microsoft graph onedrive nodeport usb-net overlay kernal preseed Prometheus openstack cloud-init python containerd keepalive shell bash skopeo ssh kubelet systemd dockerhub gcr.io api ubuntu18 initrd tftp uos x86 openwrt zabbix zombie 正则 awk EmuELEC ingress N1 cobra hang cifs http lvs ipvsadm kube-proxy 63s timeout lamp nfc pm3 coredns openshift ocp squashfs timezone rpm rpmbuild segfault suse wireguard raid kubeadm operator exsi redhat8.4 vxlan veth panic supervisor pyinstaller ttf woff hostname qemu binfmt_misc loongarch64 externalIP cgo nfs ipset proxmox non-root containers vmware hung apparmor serviceAccountToken gpu nvidia cri-dockerd pprof trivy redis /etc/resolv.conf hostPort hotplug android headscale derper iptables
日志 心得 闲搞 ipmi kubernetes ansible kubelet Linux 心得 prometheus boot Auto cad boot golang kubeconfig grub linux centos6 容器 docker docker 容器 openstack boot vsftp go version panic gobot 闲聊 iptables admission kube-controller-manager keepalived k8s Prometheus Job k8s lua microsoft openwrt preseed Prometheus proxmox 总结 CI Kylin ubuntu18 pxe uos graph 运维 zombie 技巧 travis docker onedrive kickstart consul fstab http vxlan coredns node-cache golang kmem suse wireguard raid kubeadm LVM kickstart prometheus raid