Non-Baidu Cloud Host Obtains Client's Real Source IP

If your backend server with a ADAS IP configuration is not Baidu Cloud Compute (BCC), the source station needs to load TTM module to obtain the client's real source IP. You can use the method described in this article to download and install the TTM module.

TTM Module Principle

After the client's data package is forwarded by the ADAS IP, the source address and port number of the data package will be changed to the address and port number of the ADAS IP back-to-source. In order to send the client's real source IP and port number to the server, the ADAS IP will add the client's real source IP and port number to the tcp option field of the TCP message when forwarding the message. After the source station loads the TTM module, the TTM module resolutes the client's real source IP and port number from the tcp option field of the TCP message by hooking the functions of the TCP protocol stack of the Linux kernel, as described below:

1. The Linux TCP protocol stack calls the tcp_v4_syn_recv_sock function after receiving the three-way handshake ACK message from the client during the connection establishment phase. The TTM module hooks the tcp_v4_syn_recv_sock function. The hooked tcp_v4_syn_recv_sock_ttm function will first call the original tcp_v4_syn_recv_sock function, and then call the get_ttm_data_from_ack functionm, extract the client's real source IP and port number from the tcp option field of the TCP message and store them in the sk_user_data variable of sock, where each stream corresponds to a sock.
2. When the client application calls getpeername or accept interface in user mode, it will eventually call the inet_getname function. The TTM module hooks the inet_getname function, the hooked inet_getname_ttm function will first call the original inet_getname function, and then determine whether the sk_user_data variable of sock is empty. If the sk_user_data variable is not empty, extract the client's real source IP and port number from this variable, and replace the back-to-source address and port number of the ADAS IP returned by the original inet_getname. This way, the client application calls getpeername or the accept interface to obtain the client's real source IP and port number.

Operating System Supported by TTM Module

• Linux

Notes

1. It is recommended to test in a test environment first, and then deploy to the official environment after confirming that the function is normal and the operation is stable.
2. The TTM module currently only supports IPv4, and only supports 64-bit operating system.
3. The non-TCP protocol does not support obtaining the real source IP and port number of the client.
4. If the source station has already loaded similar module and hooked the tcp_v4_syn_recv_sock and inet_getname functions of the Linux protocol stack, loading the TTM module will cause the functionality of the original module ineffective.
5. The seven-layer service (HTTP/HTTPS protocol) can directly obtain the client's real source IP through the X_forwarded_for field in http header.

TTM Module Installation Steps

1. Download the TTM module of Linux version and load it

wget https://sdk.bce.baidu.com/console-sdk/3.10.xxx.zip   
unzip 3.10.xxx.zip  
cd 3.10.xxx  
mv bce_ttm.ko /lib/modules/$(uname -r)/kernel/net/ipv4/  
insmod /lib/modules/$(uname -r)/kernel/net/ipv4/bce_ttm.ko  

Note: If there is no / lib/modules/$(uname -r)/kernel/net/ipv4 / directory, you can also put bce_ttm.ko in any other directory. The path of the following steps also needs to be replaced with the directory where bce_ttm.ko is located

3. View TTM module loading

lsmod |grep bce_ttm  

4. If you need to automatically load the TTM module after the machine restarts, you can execute the following command.

echo 'insmod /lib/modules/$(uname -r)/kernel/net/ipv4/bce_ttm.ko' >> /etc/rc.local  

5. If you no longer use the TTM module, you can execute the following command to unload it

rmmod bce_ttm  

Make TTM Module

If there is no TTM module of Linux version in the TTM download list, you can also manually make the TTM module and load it according to the following steps.

1. Install compiling environment

yum -y install gcc kernel-headers kernel-devel 

2. Download the bce_ttm module source file and decompress it.

wget -c "https://codeload.github.com/baidu/ttm/zip/master" -O bce_ttm.zip  
unzip bce_ttm.zip  

3. Compile the TTM module. The bce_ttm.ko file will be generated in the current directory after compilation.

cd ttm-master/  
make  

4. Load the TTM module, the loading method is the same as the TTM module installation steps.