Introduction

Several Intrinsec consultants participated in Nuit du Hack 2017 Qualifying CTF, which took place on the night of March 31st. We finished 36th and here is a write-up about one of the events. reverse that we did not resolve before the end of the regulation time.

Here is the statement of the test:

Identification

The "step3" binary of the test is accessible here.

(SHA256: f7c926721b03de5f2afefaeeb552441f848ea280031c95b1c6ffe1a7103762b2)

The binary retrieved once step 2 is validated is a 64-bit Linux executable stripper, as shown by the output of the "file" command:During the first execution, two interesting elements are observed:

• The password is passed as a parameter.
• The program output, "Try again :(", is displayed after the program terminates, indicating the use of subroutines created via functions such as "fork()" or "system()".

The final comparison validating the challenge is easily identifiable: a hard-coded string is compared with a variable whose value is derived from the entered password:

The goal here is therefore to find the password that will generate this string.

Bypassing anti-debug protection

After a quick analysis of the program's "main()" function using the IDAPro tool, several calls to the "fork()" function are confirmed:

If we look closely at this snippet of assembly code, which is reused several times, we can see a call to the "rand()" function in the first basic block. This function returns a pseudo-random number which, depending on its parity, will determine which branch will be executed.

The left branch will therefore execute a "fork()". The two processes will then perform the following comparison:

eax test, eax

Two scenarios will then occur:

• Execution by the parent process: the eax register contains the PID of the child process and is therefore different from 0. The comparison will then be validated, the jump made, and the system call "syscall(0x3c)" bypassed.
• Execution by child process: the eax register is 0, the system call "syscall(0x3c)" is executed, and the child process terminates.

Note As indicated in this painting In system call matching, the value "0x3c" corresponds to the function "exit()".

Now, if we look at the right branch, the condition is reversed: the parent process ends and the child continues its execution.

To bypass this protection in GDB, several options are available:

• Remove calls to "fork()" with instructions padding («nop», «mov eax, eax», etc.)
• Write a GDBInit configuration file

It should be noted, however, that modifying the binary can cause problems later if the developer has implemented code integrity checking. Therefore, the second option seemed to us to be the most sensible.

The idea is simple: place a breakpoint on all calls to the "fork()" function, which will define the behavior of the debugger ("follow the father" or "follow the son").

Here is an overview of the "GDB" configuration file:

The two functions "fc()" and "fp()" define the tracking mode of "gdb": "fc()" tracks the execution of the child while "fp()" takes care of tracking the execution of the parent.

The instruction "commands 1" will execute the "fp()" routine when the breakpoint 1, which in our case is at address 0x400940.

Calls to the "fork()" function are no longer a problem for continuing binary analysis; GDB automatically switches between processes:

In-depth analysis

Once this protection is bypassed, code analysis can begin.

Two XOR encryption routines are applied to the password, the first having the key "pony":

The second routine encrypts the binary blob generated by the first routine using the key "platypus":

To illustrate this behavior, here is a quick implementation of these two operations in Python:

For example, the encrypted version of a password "abcdefghi" will therefore be "aalily|bi".

An encoding routine at address 0x400cdc is then applied to the encrypted password "aalily|bi":

A dynamic analysis shows that the password value before and after encoding is contained in the RBX and RAX registers respectively:

The encoding closely resembles "base64", but a comparison with this algorithm shows us that the return string is different:

Static analysis of the encoding routine allows us to determine the list of characters used, referenced at address 0x3613c0.

By placing a new breakpoint and by displaying the value of the previous string, we observe that it undergoes a permutation before the call to the encoding function.  

For clarity, we will subsequently use the term "reference string" to refer to this variable.

Implementation of a solution

We then implemented this encoding in Python: 

The `fake64_e()` function takes the string to be encoded ("data") and the reference string for encoding ("key") as parameters. Essentially, this function splits the input string into 3-byte substrings. For each substring, bitwise operations are performed, generating four numbers. These numbers correspond to index numbers of the reference string ("key"). When combined, these numbers generate the encoded result.

During binary execution, this encoding algorithm is applied 16 times to the input password, and at each iteration, a permutation on the reference string is performed.

We therefore iterated on the encoding algorithm and retrieved the 16 reference strings to integrate them into our Python script. The final string, mentioned at the beginning of this article, was also inserted in order to apply the decoding process that will allow us to recover the exam password:

A decoding function was therefore developed in Python:

All that remains is to perform the 16 decoding iterations on the final binary string and apply the two XOR encryptions mentioned in the previous section:

Validation

After these 16 iterations and decryptions, the password appears: 

A huge congratulations to the NDH team for providing us with a CTF with high-quality events!