Network Protocols Dynamic Reverse Engineering - Part 1 (GDB)

In this document I will show some GDB debugging tips to help with network protocols dynamic reverse engineering.

Why did I decided to write this document?
After reading Attacking Network Protocols (by James Forshaw) I came up with the idea of documenting these debugging tips to help others and myself have a reference/guideline for the future.
As mentioned in Chapter 6 (Application Reverse Engineering) of the book, the location of each breakpoint at the time of reversing network protocols is a key part of the process. Reverse engineering network protocols from complex applications can be really hard and the starting point should always be the network protocol entry/exit point, the function that receives the frame or the one that sends it. This document is going to use the Linux platform, so we are going to work with the following functions:

• recv
• send
• recvfrom
• sendto

In my opinion practical experience helps in the learning process, so I built a series of executables to be used as examples by the reader at the time of practicing. These “challenges” can be solved doing some reverse engineering and using the debugging tips described in this document. The challenges source code is also available but if you don’t want the spoiler read them after finding the solution.

Download Challenges

GDB Requirements & Notes

Below is a brief description of the GDB commands we use in this document.

b Set a breakpoint.
bt Print the backtrace of the entire stack.
tb Set a temporary breakpoint.
commands Give any breakpoint a series of commands to execute when your program stops due to that breakpoint.
set Set the value of a variable.
c Continue execution.

Define the function below on your .gdbinit file, type it in the gdb session or download the helper.gdb file.

To execute the helper.gdb when running gdb do the following:

Calling convention

This document uses executables built for the X86-64 architecture. We just need to know the following two concepts related to the calling convention:

1. The first six integer or pointer arguments are passed in registers RDI, RSI, RDX, RCX, R8, R9.
2. Integral return values up to 64 bits in size are stored in RAX while values up to 128 bits are stored in RAX and RDX.

TCP

In most cases developers use send and recv functions for TCP communications, but it’s also possible to use write and read functions with socket file descriptors.

Below we have the send function signature and the registers involved in the call.

ssize_t send(int sockfd, const void *buf, size_t len, int flags);

sockfd -> RDI
buf -> RSI
len -> RDX
flags -> RCX

Creating a breakpoint in GDB to get the buff argument of the send function is quite straight forward.

Below we have the recv function signature, registers for the recv function are the same as for the send function.

ssize_t recv(int sockfd,void *buf, size_t len, int flags);

The recv brakepoint it is a little bit more complicated because we need to read the buf argument pointer after the recv function finishes. So we have a couple of options in this case:

1. Find every recv call and create breakpoints on the addresses following the call. We can script the process of getting all the cross references to this call using tools such as IDA Pro, Binary Ninja among others. But in some cases when dealing with function pointers or OOP programs we could miss some references if we do this from a static analysis point of view. Also we could end with too many breakpoints.
2. Create a breakpoint command that creates breakpoints dynamically. This option can save us time and shouldn’t miss any calls to recv function.

The challenges executables have only one call to recv, yet below I show a way to dynamically create temporary breakpoints.

The line below is the one that creates a temporary breakpoint on the return address stored in the stack.

> tb *(*(void**)($rsp))

NOTE: When commands doesn’t have the breakpoint id argument it will set the commands to execute for the last created breakpoint. In our case the temporary breakpoint we dynamically created.

Now using these breakpoints and some reversing try to get the solution for the tcp executable.

UDP

For UDP communications sendto and recvfrom functions are used. Below we have the function signatures and the registers involved in both calls.

ssize_t sendto(int sockfd, const void *buf, size_t len, int flags, const struct sockaddr *dest_addr, socklen_t* addrlen);

ssize_t recvfrom(int sockfd,void *buf,size_t len,int flags, struct sockaddr *src_addr, socklen_t *addrlen);

In both cases and as we already used on send and recv before the interesting registers are the following:

buf -> RSI
len -> RDX

So the breakpoints for these functions are almost the same as we used with TCP sockets.

Now using these breakpoints and some reversing try to get the solution for the udp executable.

SSL

In most cases when we are working with secure communications we usually want to analyze the protocol that is contained inside the encrypted frame. If we are working with a custom implementation we are going to need some analysis of how things are working.

So, how could we analyze the network protocol inside the secure channel? We need to find the function that adds the secure layer to the network protocol. If we don’t have the information on the library the program is using, we could take for granted that one of the four functions we mentioned above(recv, send, recvfrom, sendto) is going to be use by this library to send the encrypted frames. As it was mention before developers can use send or write to interact with socket descriptors. So taking this into account, we create a breakpoint in write and send functions and use the bt command to get a backtrace to analyze the call path and find these functions.

NOTE: Another way to do this, is to break on the socket function and get the socket descriptor. Once we know where the socket descriptor is stored in memory we can create a watchpoint(stop execution whenever the value of an expression changes). In both cases we need to perform some analysis to get the target function.

As we saw in the backtrace above the target function appears to be named BIO_puts.

int BIO_puts(BIO *b,const char *buf);

int BIO_read(BIO *b, void *buf, int len);

Now using these breakpoints and some reversing try to get the solution for the ssl executable.

References

1. Debugging with GDB - https://sourceware.org/gdb/current/onlinedocs/gdb/

2. X86 Calling Conventions - https://en.wikipedia.org/wiki/X86_calling_conventions#System_V_AMD64_ABI