Kubernetes技巧与窍门：加速大型数据库的引导

• MySQL中有一个具有1 TB数据库的应用程序，有10个开发人员开发了自己的功能。
• 开发人员需要单独的测试循环，以及一些用于测试和/或演示的更具体的循环。
• 另外，需要在生产电路的测试电路中恢复生产基地的夜间维护，以保持正常的时间-以重现客户或bug的问题。
• 最后，可以减少至少150 GB的数据库大小-不算多，但仍可以节省空间。 即 我们仍然需要以某种方式准备转储。

步骤1：从转储中准备新数据库

  nodeSelector: dedicated: non-prod-db tolerations: - key: "dedicated" operator: "Equal" value: "non-prod-db" effect: "NoExecute" 

初始化容器：get-bindump

  - name: get-bindump image: db-dumps imagePullPolicy: Always command: [ "/bin/sh", "-c", "/get_bindump.sh" ] resources: limits: memory: "5000Mi" cpu: "1" requests: memory: "5000Mi" cpu: "1" volumeMounts: - name: dump mountPath: /dump - name: mysqlbindir mountPath: /var/lib/mysql - name: id-rsa mountPath: /root/.ssh 

 #!/bin/bash date if [ -f /dump/version.txt ]; then echo "Dump file already exists." exit 0 fi rm -rf /var/lib/mysql/* borg extract --stdout user@your.server.net:somedb-mysql::${lastdump} stdin | xbstream -x -C /var/lib/mysql/ echo $lastdump > /dump/version.txt 

初始化容器：prepare-bindump

  - name: prepare-bindump image: db-dumps imagePullPolicy: Always command: [ "/bin/sh", "-c", "/prepare_bindump.sh" ] resources: limits: memory: "5000Mi" cpu: "1" requests: memory: "5000Mi" cpu: "1" volumeMounts: - name: dump mountPath: /dump - name: mysqlbindir mountPath: /var/lib/mysql - name: debian-cnf mountPath: /etc/mysql/debian.cnf subPath: debian.cnf 

 #!/bin/bash date if [ -f /dump/healthz ]; then echo "Dump file already exists." exit 0 fi innobackupex --apply-log /var/lib/mysql/ chown -R mysql:mysql /var/lib/mysql chown -R mysql:mysql /var/log/mysql echo "`date`: Starting mysql" /usr/sbin/mysqld --character-set-server=utf8 --collation-server=utf8_general_ci --innodb-data-file-path=ibdata1:200M:autoextend --user=root --skip-grant-tables & sleep 200 echo "`date`: Creating mysql root user" echo "update mysql.user set Password=PASSWORD('password') WHERE user='root';" | mysql -uroot -h 127.0.0.1 echo "delete from mysql.user where USER like '';" | mysql -uroot -h 127.0.0.1 echo "delete from mysql.user where user = 'root' and host NOT IN ('127.0.0.1', 'localhost');" | mysql -uroot -h 127.0.0.1 echo "FLUSH PRIVILEGES;" | mysql -uroot -h 127.0.0.1 echo "truncate somedb.somedb_table_one;" | mysql -uroot -h 127.0.0.1 -ppassword somedb /usr/bin/mysqladmin shutdown -uroot -ppassword cd /var/lib/mysql/ tar -czf /dump/mysql_bindump.tar.gz ./* touch /dump/healthz rm -rf /var/lib/mysql/* 

下

  - name: nginx image: nginx:alpine resources: requests: memory: "1500Mi" cpu: "400m" lifecycle: preStop: exec: command: ["/usr/sbin/nginx", "-s", "quit"] livenessProbe: httpGet: path: /healthz port: 80 scheme: HTTP timeoutSeconds: 7 failureThreshold: 5 volumeMounts: - name: dump mountPath: /usr/share/nginx/html - name: nginx-config mountPath: /etc/nginx/nginx.conf subPath: nginx.conf readOnly: false volumes: - name: dump emptyDir: {} - name: mysqlbindir emptyDir: {} 

 --- apiVersion: apps/v1beta1 kind: Deployment metadata: name: db-dumps spec: strategy: rollingUpdate: maxUnavailable: 0 revisionHistoryLimit: 2 template: metadata: labels: app: db-dumps spec: imagePullSecrets: - name: regsecret nodeSelector: dedicated: non-prod-db tolerations: - key: "dedicated" operator: "Equal" value: "non-prod-db" effect: "NoExecute" initContainers: - name: get-bindump image: db-dumps imagePullPolicy: Always command: [ "/bin/sh", "-c", "/get_bindump.sh" ] resources: limits: memory: "5000Mi" cpu: "1" requests: memory: "5000Mi" cpu: "1" volumeMounts: - name: dump mountPath: /dump - name: mysqlbindir mountPath: /var/lib/mysql - name: id-rsa mountPath: /root/.ssh - name: prepare-bindump image: db-dumps imagePullPolicy: Always command: [ "/bin/sh", "-c", "/prepare_bindump.sh" ] resources: limits: memory: "5000Mi" cpu: "1" requests: memory: "5000Mi" cpu: "1" volumeMounts: - name: dump mountPath: /dump - name: mysqlbindir mountPath: /var/lib/mysql - name: log mountPath: /var/log/mysql - name: debian-cnf mountPath: /etc/mysql/debian.cnf subPath: debian.cnf containers: - name: nginx image: nginx:alpine resources: requests: memory: "1500Mi" cpu: "400m" lifecycle: preStop: exec: command: ["/usr/sbin/nginx", "-s", "quit"] livenessProbe: httpGet: path: /healthz port: 80 scheme: HTTP timeoutSeconds: 7 failureThreshold: 5 volumeMounts: - name: dump mountPath: /usr/share/nginx/html - name: nginx-config mountPath: /etc/nginx/nginx.conf subPath: nginx.conf readOnly: false volumes: - name: dump emptyDir: {} - name: mysqlbindir emptyDir: {} - name: log emptyDir: {} - name: id-rsa secret: defaultMode: 0600 secretName: somedb-id-rsa - name: nginx-config configMap: name: somedb-nginx-config - name: debian-cnf configMap: name: somedb-debian-cnf --- apiVersion: v1 kind: Service metadata: name: somedb-db-dump spec: clusterIP: None selector: app: db-dumps ports: - name: http port: 80 

1. 就我们而言，我们每晚使用GitLab中的预定作业准备一个新的转储。 即 每天晚上，此部署都会自动推出，从而产生一个新的转储，并准备将其分发给所有测试开发人员环境。
2. 为什么还要将volume /dump扔到init容器中（并且在脚本中检查/dump/version.txt的存在）？ 如果在其下运行的服务器重新启动，则可以这样做。 容器将重新开始，并且没有进行此检查，转储将再次开始下载。 如果我们已经准备好一次转储，那么在下一次启动时（如果服务器重新启动），/ /dump/version.txt标志/dump/version.txt将提供有关此信息。
3. 什么是db-dumps ？ 我们使用dapp收集它，其Dappfile如下所示：
   
   

    dimg: "db-dumps" from: "ubuntu:16.04" docker: ENV: TERM: xterm ansible: beforeInstall: - name: "Install percona repositories" apt: deb: https://repo.percona.com/apt/percona-release_0.1-4.xenial_all.deb - name: "Add repository for borgbackup" apt_repository: repo="ppa:costamagnagianfranco/borgbackup" codename="xenial" update_cache=yes - name: "Add repository for mysql 5.6" apt_repository: repo: deb http://archive.ubuntu.com/ubuntu trusty universe state: present update_cache: yes - name: "Install packages" apt: name: "{{`{{ item }}`}}" state: present with_items: - openssh-client - mysql-server-5.6 - mysql-client-5.6 - borgbackup - percona-xtrabackup-24 setup: - name: "Add get_bindump.sh" copy: content: | {{ .Files.Get ".dappfiles/get_bindump.sh" | indent 8 }} dest: /get_bindump.sh mode: 0755 - name: "Add prepare_bindump.sh" copy: content: | {{ .Files.Get ".dappfiles/prepare_bindump.sh" | indent 8 }} dest: /prepare_bindump.sh mode: 0755 

步骤2：在开发人员环境中启动数据库

 spec: strategy: rollingUpdate: maxUnavailable: 0 

 curl "$DUMP_URL" | tar -C /var/lib/mysql/ -xvz 

 apiVersion: apps/v1beta1 kind: Deployment metadata: name: mysql spec: strategy: rollingUpdate: maxUnavailable: 0 template: metadata: labels: service: mysql spec: imagePullSecrets: - name: regsecret nodeSelector: dedicated: non-prod-db tolerations: - key: "dedicated" operator: "Equal" value: "non-prod-db" effect: "NoExecute" initContainers: - name: getdump image: mysql-with-getdump command: ["/usr/local/bin/getdump.sh"] resources: limits: memory: "6000Mi" cpu: "1.5" requests: memory: "6000Mi" cpu: "1.5" volumeMounts: - mountPath: /var/lib/mysql name: datadir - mountPath: /etc/mysql/debian.cnf name: debian-cnf subPath: debian.cnf env: - name: DUMP_URL value: "http://somedb-db-dump.infra-db.svc.cluster.local/mysql_bindump.tar.gz" containers: - name: mysql image: mysql:5.6 resources: limits: memory: "1024Mi" cpu: "1" requests: memory: "1024Mi" cpu: "1" lifecycle: preStop: exec: command: ["/etc/init.d/mysql", "stop"] ports: - containerPort: 3306 name: mysql protocol: TCP volumeMounts: - mountPath: /var/lib/mysql name: datadir - mountPath: /etc/mysql/debian.cnf name: debian-cnf subPath: debian.cnf env: - name: MYSQL_ROOT_PASSWORD value: "password" volumes: - name: datadir emptyDir: {} - name: debian-cnf configMap: name: somedb-debian-cnf --- apiVersion: v1 kind: Service metadata: name: mysql spec: clusterIP: None selector: service: mysql ports: - name: mysql port: 3306 protocol: TCP --- apiVersion: v1 kind: ConfigMap metadata: name: somedb-debian-cnf data: debian.cnf: | [client] host = localhost user = debian-sys-maint password = password socket = /var/run/mysqld/mysqld.sock [mysql_upgrade] host = localhost user = debian-sys-maint password = password socket = /var/run/mysqld/mysqld.sock 

总结

• 获取新的数据库转储
• 以某种方式为测试环境中的正确操作做好准备（例如，对某些表进行trankeytit，替换实际用户数据，创建必要的用户等）；
• 通过按CI中的按钮，每个开发人员都有机会将这样一个准备好的数据库推出到Deployment中的名称空间-由于其中提供了Service ，因此数据库将在mysql上可用（例如，它可能是名称空间中服务的名称）。

PPS

• “ NGINX Ingress中的个性化错误页面 ”；
• “ 将集群中工作的资源转移给Helm 2管理人员 ”；
• “ 关于节点的分配和Web应用程序上的负载 ”；
• “ 访问开发人员站点 。”

• “ 使用dapp和GitLab CI在Kubernetes中构建和安装应用程序 ”；
• “ 练习dapp。 第1部分：构建简单的应用程序 ”；
• “ 练习dapp。 第2部分。使用Helm在Kubernetes中部署Docker映像 ”；
• “ Kubernetes中的CockroachDB DBMS编排 ”；
• “ 我们在小型项目中使用Kubernetes的经验 ” （视频报告，其中包括Kubernetes的技术设备介绍）；
• “ 与Kubernetes一起使用时有用的实用程序 。”