使用Telegram MTProxy的5.94米docker映像

众所周知,Telegram于5月底推出了官方的MTProto代理(又名MTProxy)服务器,该服务器使用以下语言编写。 在2018年,没有Docker的地方不多,因为它伴随着零配置格式的相同“正式” 方式 。 一切都会好起来的,但是三个“ buts”却有点破坏了发行版的印象:图像重于> 130 Mb(有相当丰满的Debian,而不是通常的Alpine),由于“ zero-config”,它并不总是很方便地配置(仅通过环境设置)你们忘了竞选,布置了Dockerfile。



TL; DR我们将制作一个大小为5.94MB的,实际上是基于阿尔卑斯山的官方1:1官方docker镜像,并将其放在此处 (和dockerfile放在此处 ); 在此过程中,我们将弄清楚有时您如何可以使用Nippers和文件通过Alpine软件结交朋友,并且我们会玩一些游戏,只是为了娱乐。

图片内容


再一次,因为什么大惊小怪的? 让我们看看使用history命令表示的官方图像是什么:

$ docker history --no-trunc --format "{{.Size}}\t{{.CreatedBy}}" telegrammessenger/proxy 

从下至上分别读取层:



最厚的是Debian Jessie,从中继承了原始图像,我们必须首先删除它(高山:3.6,相比之下,它的重量为3.97MB); 其次是卷曲和新鲜证书。 要了解其他两个文件和目录的含义,我们将使用run命令查看内部,用bash替换CMD(这将使您可以在启动的图像周围走动,更紧密地了解彼此,运行某些片段,复制有用的东西):

 $ docker run -it --rm telegrammessenger/proxy /bin/bash 

现在,我们可以轻松地恢复事件的图片-就像丢失的官方Dockerfile一样:

 FROM debian:jessie-20180312 RUN set -eux \ && apt-get update \ && apt-get install -y --no-install-recommends curl ca-certificates \ && rm -rf /var/lib/apt/lists/* COPY ./mtproto-proxy /usr/local/bin RUN mkdir /data COPY ./secret/ /etc/telegram/ COPY ./run.sh /run.sh CMD ["/bin/sh", "-c", "/bin/bash /run.sh"] 

如果mtproto-proxy是已编译的服务器,则secret文件夹仅包含带有AES加密密钥的hello-explorers-您正在做的文件(请参阅服务器命令,顺便说一句,官方建议通过API来获取密钥,但要这样写)可能是为了避免API也被阻止的情况),然后run.sh会为启动代理做所有准备工作。

原始run.sh的内容
 #!/bin/bash if [ ! -z "$DEBUG" ]; then set -x; fi mkdir /data 2>/dev/null >/dev/null RANDOM=$(printf "%d" "0x$(head -c4 /dev/urandom | od -t x1 -An | tr -d ' ')") if [ -z "$WORKERS" ]; then WORKERS=2 fi echo "####" echo "#### Telegram Proxy" echo "####" echo SECRET_CMD="" if [ ! -z "$SECRET" ]; then echo "[+] Using the explicitly passed secret: '$SECRET'." elif [ -f /data/secret ]; then SECRET="$(cat /data/secret)" echo "[+] Using the secret in /data/secret: '$SECRET'." else if [[ ! -z "$SECRET_COUNT" ]]; then if [[ ! ( "$SECRET_COUNT" -ge 1 && "$SECRET_COUNT" -le 16 ) ]]; then echo "[F] Can generate between 1 and 16 secrets." exit 5 fi else SECRET_COUNT="1" fi echo "[+] No secret passed. Will generate $SECRET_COUNT random ones." SECRET="$(dd if=/dev/urandom bs=16 count=1 2>&1 | od -tx1 | head -n1 | tail -c +9 | tr -d ' ')" for pass in $(seq 2 $SECRET_COUNT); do SECRET="$SECRET,$(dd if=/dev/urandom bs=16 count=1 2>&1 | od -tx1 | head -n1 | tail -c +9 | tr -d ' ')" done fi if echo "$SECRET" | grep -qE '^[0-9a-fA-F]{32}(,[0-9a-fA-F]{32}){,15}$'; then SECRET="$(echo "$SECRET" | tr '[:upper:]' '[:lower:]')" SECRET_CMD="-S $(echo "$SECRET" | sed 's/,/ -S /g')" echo -- "$SECRET_CMD" > /data/secret_cmd echo "$SECRET" > /data/secret else echo '[F] Bad secret format: should be 32 hex chars (for 16 bytes) for every secret; secrets should be comma-separated.' exit 1 fi if [ ! -z "$TAG" ]; then echo "[+] Using the explicitly passed tag: '$TAG'." fi TAG_CMD="" if [[ ! -z "$TAG" ]]; then if echo "$TAG" | grep -qE '^[0-9a-fA-F]{32}$'; then TAG="$(echo "$TAG" | tr '[:upper:]' '[:lower:]')" TAG_CMD="-P $TAG" else echo '[!] Bad tag format: should be 32 hex chars (for 16 bytes).' echo '[!] Continuing.' fi fi curl -s https://core.telegram.org/getProxyConfig -o /etc/telegram/backend.conf || { echo '[F] Cannot download proxy configuration from Telegram servers.' exit 2 } CONFIG=/etc/telegram/backend.conf IP="$(curl -s -4 "https://digitalresistance.dog/myIp")" INTERNAL_IP="$(ip -4 route get 8.8.8.8 | grep '^8\.8\.8\.8\s' | grep -Po 'src\s+\d+\.\d+\.\d+\.\d+' | awk '{print $2}')" if [[ -z "$IP" ]]; then echo "[F] Cannot determine external IP address." exit 3 fi if [[ -z "$INTERNAL_IP" ]]; then echo "[F] Cannot determine internal IP address." exit 4 fi echo "[*] Final configuration:" I=1 echo "$SECRET" | tr ',' '\n' | while read S; do echo "[*] Secret $I: $S" echo "[*] tg:// link for secret $I auto configuration: tg://proxy?server=${IP}&port=443&secret=${S}" echo "[*] t.me link for secret $I: https://t.me/proxy?server=${IP}&port=443&secret=${S}" I=$(($I+1)) done [ ! -z "$TAG" ] && echo "[*] Tag: $TAG" || echo "[*] Tag: no tag" echo "[*] External IP: $IP" echo "[*] Make sure to fix the links in case you run the proxy on a different port." echo echo '[+] Starting proxy...' sleep 1 exec /usr/local/bin/mtproto-proxy -p 2398 -H 443 -M "$WORKERS" -C 60000 --aes-pwd /etc/telegram/hello-explorers-how-are-you-doing -u root $CONFIG --allow-skip-dh --nat-info "$INTERNAL_IP:$IP" $SECRET_CMD $TAG_CMD 

组装方式


在Habré上已收集的 CentOS 7 MTProxy下,我们将尝试在Alpine下收集映像,并在生成的docker映像中保存130兆字节的广告。

Alpine Linux的一个独特功能是使用musl而不是glibc。 两者都是标准的C库。 Musl很小(它没有“标准”的五分之一),但是体积和性能(至少是有希望的)决定了何时使用Docker。 而且将glibc放在Alpine上在种族上是不正确的,例如,Jakub Jirutka叔叔不会理解

我们还将构建docker以隔离依赖关系并获得实验的自由,因此创建一个新的Dockerfile:

 FROM alpine:3.6 RUN apk add --no-cache git make gcc musl-dev linux-headers openssl-dev RUN git clone --single-branch --depth 1 https://github.com/TelegramMessenger/MTProxy.git /mtproxy/sources RUN cd /mtproxy/sources \ && make -j$(getconf _NPROCESSORS_ONLN) 

在依赖项中,git将派上用场(不仅用于克隆官方存储库,make文件还将sha提交附加到版本),make,gcc和头文件(通过经验获得最小集)。 我们只会克隆1次提交深度的master分支(我们绝对不需要历史记录)。 好吧,让我们尝试在使用-j开关进行编译时利用所有主机资源。 我故意将其分为多个层,以便在重建期间获得方便的缓存(通常有很多层)。

我们将运行build命令(位于Dockerfile目录中):

 $ docker build -t mtproxy:test . 

这是第一个问题:

 In file included from ./net/net-connections.h:34:0, from mtproto/mtproto-config.c:44: ./jobs/jobs.h:234:23: error: field 'rand_data' has incomplete type struct drand48_data rand_data; ^~~~~~~~~ 

实际上,所有后续的都将与之连接。 首先,对于那些不熟悉自己的人,编译器实际上发誓缺少drand48_data结构的声明。 其次,musl开发人员在线程安全的随机函数(带有_r后缀)和与其相关的所有事物(包括结构)上得分。 反过来,Telegram的开发人员也不必为未实现random_r及其对应物的系统进行编译(在许多OS库中,您可以看到HAVE_RANDOM_R标志或其任意+存在或不存在通常在自动配置器中存在的这组功能)。

好吧,现在我们一定要安装glibc吗? 不行 我们将从glibc复制所需的内容,并为MTProxy源制作补丁。

除了random_r的问题之外,我们还遇到了backtrace函数(execinfo.h)的问题,该函数用于在发生异常的情况下输出堆栈backtrace:您可以尝试使用libunwind中的实现替换它,但这是不值得的,因为调用是通过检查__GLIBC__。

Random_compat.patch补丁内容
 Index: jobs/jobs.h IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- jobs/jobs.h (revision cdd348294d86e74442bb29bd6767e48321259bec) +++ jobs/jobs.h (date 1527996954000) @@ -28,6 +28,8 @@ #include "net/net-msg.h" #include "net/net-timers.h" +#include "common/randr_compat.h" + #define __joblocked #define __jobref Index: common/server-functions.c IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- common/server-functions.c (revision cdd348294d86e74442bb29bd6767e48321259bec) +++ common/server-functions.c (date 1527998325000) @@ -35,7 +35,9 @@ #include <arpa/inet.h> #include <assert.h> #include <errno.h> +#ifdef __GLIBC__ #include <execinfo.h> +#endif #include <fcntl.h> #include <getopt.h> #include <grp.h> @@ -168,6 +170,7 @@ } void print_backtrace (void) { +#ifdef __GLIBC__ void *buffer[64]; int nptrs = backtrace (buffer, 64); kwrite (2, "\n------- Stack Backtrace -------\n", 33); @@ -178,6 +181,7 @@ kwrite (2, s, strlen (s)); kwrite (2, "\n", 1); } +#endif } pthread_t debug_main_pthread_id; Index: common/randr_compat.h IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- common/randr_compat.h (date 1527998264000) +++ common/randr_compat.h (date 1527998264000) @@ -0,0 +1,72 @@ +/* + The GNU C Library is free software. See the file COPYING.LIB for copying + conditions, and LICENSES for notices about a few contributions that require + these additional notices to be distributed. License copyright years may be + listed using range notation, eg, 2000-2011, indicating that every year in + the range, inclusive, is a copyrightable year that would otherwise be listed + individually. +*/ + +#pragma once + +#include <endian.h> +#include <pthread.h> + +struct drand48_data { + unsigned short int __x[3]; /* Current state. */ + unsigned short int __old_x[3]; /* Old state. */ + unsigned short int __c; /* Additive const. in congruential formula. */ + unsigned short int __init; /* Flag for initializing. */ + unsigned long long int __a; /* Factor in congruential formula. */ +}; + +union ieee754_double +{ + double d; + + /* This is the IEEE 754 double-precision format. */ + struct + { +#if __BYTE_ORDER == __BIG_ENDIAN + unsigned int negative:1; + unsigned int exponent:11; + /* Together these comprise the mantissa. */ + unsigned int mantissa0:20; + unsigned int mantissa1:32; +#endif /* Big endian. */ +#if __BYTE_ORDER == __LITTLE_ENDIAN + /* Together these comprise the mantissa. */ + unsigned int mantissa1:32; + unsigned int mantissa0:20; + unsigned int exponent:11; + unsigned int negative:1; +#endif /* Little endian. */ + } ieee; + + /* This format makes it easier to see if a NaN is a signalling NaN. */ + struct + { +#if __BYTE_ORDER == __BIG_ENDIAN + unsigned int negative:1; + unsigned int exponent:11; + unsigned int quiet_nan:1; + /* Together these comprise the mantissa. */ + unsigned int mantissa0:19; + unsigned int mantissa1:32; +#else + /* Together these comprise the mantissa. */ + unsigned int mantissa1:32; + unsigned int mantissa0:19; + unsigned int quiet_nan:1; + unsigned int exponent:11; + unsigned int negative:1; +#endif + } ieee_nan; +}; + +#define IEEE754_DOUBLE_BIAS 0x3ff /* Added to exponent. */ + +int drand48_r (struct drand48_data *buffer, double *result); +int lrand48_r (struct drand48_data *buffer, long int *result); +int mrand48_r (struct drand48_data *buffer, long int *result); +int srand48_r (long int seedval, struct drand48_data *buffer); \ No newline at end of file Index: Makefile IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- Makefile (revision cdd348294d86e74442bb29bd6767e48321259bec) +++ Makefile (date 1527998107000) @@ -40,6 +40,7 @@ DEPENDENCE_NORM := $(subst ${OBJ}/,${DEP}/,$(patsubst %.o,%.d,${OBJECTS})) LIB_OBJS_NORMAL := \ + ${OBJ}/common/randr_compat.o \ ${OBJ}/common/crc32c.o \ ${OBJ}/common/pid.o \ ${OBJ}/common/sha1.o \ Index: common/randr_compat.c IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- common/randr_compat.c (date 1527998213000) +++ common/randr_compat.c (date 1527998213000) @@ -0,0 +1,120 @@ +/* + The GNU C Library is free software. See the file COPYING.LIB for copying + conditions, and LICENSES for notices about a few contributions that require + these additional notices to be distributed. License copyright years may be + listed using range notation, eg, 2000-2011, indicating that every year in + the range, inclusive, is a copyrightable year that would otherwise be listed + individually. +*/ + +#include <stddef.h> +#include "common/randr_compat.h" + +int __drand48_iterate (unsigned short int xsubi[3], struct drand48_data *buffer) { + uint64_t X; + uint64_t result; + + /* Initialize buffer, if not yet done. */ + if (!buffer->__init == 0) + { + buffer->__a = 0x5deece66dull; + buffer->__c = 0xb; + buffer->__init = 1; + } + + /* Do the real work. We choose a data type which contains at least + 48 bits. Because we compute the modulus it does not care how + many bits really are computed. */ + + X = (uint64_t) xsubi[2] << 32 | (uint32_t) xsubi[1] << 16 | xsubi[0]; + + result = X * buffer->__a + buffer->__c; + + xsubi[0] = result & 0xffff; + xsubi[1] = (result >> 16) & 0xffff; + xsubi[2] = (result >> 32) & 0xffff; + + return 0; +} + +int __erand48_r (unsigned short int xsubi[3], struct drand48_data *buffer, double *result) { + union ieee754_double temp; + + /* Compute next state. */ + if (__drand48_iterate (xsubi, buffer) < 0) + return -1; + + /* Construct a positive double with the 48 random bits distributed over + its fractional part so the resulting FP number is [0.0,1.0). */ + + temp.ieee.negative = 0; + temp.ieee.exponent = IEEE754_DOUBLE_BIAS; + temp.ieee.mantissa0 = (xsubi[2] << 4) | (xsubi[1] >> 12); + temp.ieee.mantissa1 = ((xsubi[1] & 0xfff) << 20) | (xsubi[0] << 4); + + /* Please note the lower 4 bits of mantissa1 are always 0. */ + *result = temp.d - 1.0; + + return 0; +} + +int __nrand48_r (unsigned short int xsubi[3], struct drand48_data *buffer, long int *result) { + /* Compute next state. */ + if (__drand48_iterate (xsubi, buffer) < 0) + return -1; + + /* Store the result. */ + if (sizeof (unsigned short int) == 2) + *result = xsubi[2] << 15 | xsubi[1] >> 1; + else + *result = xsubi[2] >> 1; + + return 0; +} + +int __jrand48_r (unsigned short int xsubi[3], struct drand48_data *buffer, long int *result) { + /* Compute next state. */ + if (__drand48_iterate (xsubi, buffer) < 0) + return -1; + + /* Store the result. */ + *result = (int32_t) ((xsubi[2] << 16) | xsubi[1]); + + return 0; +} + +int drand48_r (struct drand48_data *buffer, double *result) { + return __erand48_r (buffer->__x, buffer, result); +} + +int lrand48_r (struct drand48_data *buffer, long int *result) { + /* Be generous for the arguments, detect some errors. */ + if (buffer == NULL) + return -1; + + return __nrand48_r (buffer->__x, buffer, result); +} + +int mrand48_r (struct drand48_data *buffer, long int *result) { + /* Be generous for the arguments, detect some errors. */ + if (buffer == NULL) + return -1; + + return __jrand48_r (buffer->__x, buffer, result); +} + +int srand48_r (long int seedval, struct drand48_data *buffer) { + /* The standards say we only have 32 bits. */ + if (sizeof (long int) > 4) + seedval &= 0xffffffffl; + + buffer->__x[2] = seedval >> 16; + buffer->__x[1] = seedval & 0xffffl; + buffer->__x[0] = 0x330e; + + buffer->__a = 0x5deece66dull; + buffer->__c = 0xb; + buffer->__init = 1; + + return 0; +} \ No newline at end of file 

将其放在./patches文件夹中,并稍微修改我们的Dockerfile以便即时应用补丁:

 FROM alpine:3.6 COPY ./patches /mtproxy/patches RUN apk add --no-cache --virtual .build-deps \ git make gcc musl-dev linux-headers openssl-dev \ && git clone --single-branch --depth 1 https://github.com/TelegramMessenger/MTProxy.git /mtproxy/sources \ && cd /mtproxy/sources \ && patch -p0 -i /mtproxy/patches/randr_compat.patch \ && make -j$(getconf _NPROCESSORS_ONLN) \ && cp /mtproxy/sources/objs/bin/mtproto-proxy /mtproxy/ \ && rm -rf /mtproxy/{sources,patches} \ && apk add --no-cache --virtual .rundeps libcrypto1.0 \ && apk del .build-deps 

现在,至少启动了组装好的mtproto-proxy二进制文件,我们可以继续进行。

清仓


现在是时候将原始run.sh转换为docker-entrypoint.sh了。 我认为,当“强制性绑定”进入ENTRYPOINT时(这总是可以从外部重载),并且启动dockerized应用程序的参数适合CMD中的最大值(+环境变量作为研究不足),这是合乎逻辑的。

我们可以在高山图像中安装bash和full grep(我将在后面解释),以避免头痛并按原样使用原始代码,但是它将使我们的微型图像膨胀到可耻的地步,因此我们将种植一个真正的他的母亲盆景。

让我们从shebang开始,将#!/bin/bash替换为#!/bin/sh 。 高山灰的默认设置能够消化几乎所有bash的“原样”语法,但是我们仍然遇到一个问题-由于未知原因,他拒绝接受其中一种条件的括号,因此我们将通过反转比较逻辑来扩展它:



现在,我们正在等待grep的摊牌,在忙碌箱交付中,这与通常的摊派略有不同(顺便说一句,要慢得多,请记住您的项目)。 首先,他不理解表达式{,15} ,他将必须明确指定{0,15} 。 其次,它不支持-P标志(perl样式),但是在启用扩展(-E)时安静地摘要了表达式。

在我们的依赖项中,仅保留curl(没有用busybox中的wget替换它)和libcrypto(足够了,在此程序集中根本不需要直接使用openssl)。

几年前,一个漂亮的多阶段构建出现在Docker中,例如,它非常适合Go应用程序或组装很复杂并且比最终清理更容易在图像之间传输工件的情况。 我们将用它来种植盆景,这将节省一些。

 FROM alpine:3.6 #         ( ) RUN apk add --no-cache --virtual .build-deps \ # ... ,    && make -j$(getconf _NPROCESSORS_ONLN) FROM alpine:3.6 #   ,  ,   WORKDIR /mtproxy COPY --from=0 /mtproxy/sources/objs/bin/mtproto-proxy . #          #          

盆景应该是盆景-摆脱libcrypto安装。 构建时,我们需要openssl-dev包中的头文件,该文件在依赖项中将拉起libcrypto,而我们的可执行文件将使用libcrypto.so.1.0.0。 但这是唯一的依赖项,此外,它已预安装在Alpine中(在版本3.6中,它是libcrypto.so.41,3.7-libcrypto.so.42,它在/ lib /中)。 他们现在骂我,这不是最可靠的方法,但是值得,我们仍然在现有版本中添加符号链接(如果您有更好的方法,我很乐意接受PR)。

最后的修饰和结果:

Docker中心
Github

如有任何建议和贡献,我将不胜感激。

Source: https://habr.com/ru/post/zh-CN413881/


All Articles