Anti Disassembly Techniques

2022-04-30

Definitions

Anti-Debugging

Techniques to compromise debuggers and/or the debugging process

Anti-Disassembly

Techniques to compromise disassemblers and/or the disassembling process

Obfuscation

Techniques to make the signatures creation more difficult and the disassembled code harder to be analyzed by a professional

Anti-VM

Techniques to detect and/or compromise virtual maschnies

Anti-disassembly techniques

API obfuscation

API obfuscation changes the names of identifies (class names, method names, field names) to random names so that the reader of the code doesn’t know what the code is doing.

Opcode/assembly code obfuscation

Opcode/assembly code obfuscation makes disassembly of malware difficult by using tactics like executables with decrypted sections and code instructions that are hard to read or nonsensical.

Junk/spaghetti code

used to confuse the reverse engineer and hide what’s the code is trying to accomplish

Control flow graph flattening

Control flow graph flattening, or simply CFG flattening, flattens the control flow of each function by first breaking up the nesting of loops and if-statements and then hiding each of them in a case of a large switch statement wrapped inside the body of a loop.

Jump instruction with same target

Jump instruction with the same target is produced using a combination of jz with jnz. This is an unconditional jump that the disassembler doesn’t recognize because it only disassembles one instruction at a time.

Detecting debuggers

Windows API

The most common technique malware uses is Windows API, as it provides several functions that can be used by malware to detect debuggers. The most used one is IsDebuggerPresent. It checks for a specific flag in the Process Environment Block (PEB) for the field BeingDebugged, which will return zero if the process is not running into a debugger or nonzero if a debugger is attached.

Another similar function is CheckRemoteDebuggerPresent, which checks if a remote process is debugging the current one

CloseHandle/NtClose

CloseHandle/NtClose is another anti-debugging malware uses. Calling the dispatcher with an invalid handle throws an invalid handle exception, STATUS_INVALID_HANDLE.

FindWindow

FindWindow is also used to find the debugger by providing window class (e.g., OLLYDBG).

NtGlobalFlag

Malware can also take advantage of PEB’s NtGlobalFlag flag at offset 0x68 (which is called to check if it’s being debugged) by verifying if its value is equal to 0x70.

Advanced anti-debugging techniques

Timing defense

The Read Time-Stamp Counter (RDTSC) is a common example of a timing defense. Debugging tools take time and effort to process each instruction. This process creates slowness and fluctuation that is different from the computer’s fixed processing speed threshold. Malware avoids detection by reading the computer’s time stamp multiple times; if it finds time irregularities, it shows an error because it knows a debugger is searching the system.

Breakpoint detection

There are various types of breakpoints, but the following are most commonly used by reverse engineers.

Software breakpoints are set by replacing the instruction at the target address with the byte 0xCC (INT3) so the malware can count the number of 0xCC bytes (for example, using the instruction repne scasb) and to determine whether the program is being debugged or not. Another way that malware uses is by performing a CRC or MD5 checksum of the opcodes
Hardware breakpoints are detected by using GetThreadContext/SetThreadContext APIs and checking if DRs are set. If yes, then hardware breakpoint is set

Interrupts

An interrupt is a condition that halts the microprocessor temporarily or permanently to work on a different task called an interrupt routine (also called Interrupt Service Routine or simple ISR) whose address is stored in the Interrupt Descriptor Table (IDT). It then returns to its previous task. Malware can use some interrupt instructions to detect a debugger. The following are some of these instructions:

Two-byte INT3: (0xCC, 0xCD+0x03) Can be used as false breakpoint
INT 0x2C: Raises a debug assertion exception
INT 0x2D: Issues an EXCEPTION_BREAKPOINT if no debugger is attached
ICEBP (0xF1): Generates a single-step exception
Trap flag: Generates a single-step exception (int 0x01h) after executing an instruction
Stack segment: Tracing over SS (e.g., mov ss, pop ss), the debugger will not break on those, effectively stopping on the following instruction. In other words, an unset of the trap flag won’t be possible after that, and if check is done here, a debugger will be detected

TLS callbacks

Almost all debuggers start the program’s entry point, as defined by the PE header. Thread Local Storage (TLS) callback is a technique that can be used to execute a code before or after the main application code execution. This means that when loaded in a debugger, the program does all the checking before reaching the entry point and the analyst wouldn’t know what is happening.

Anti-VM techniques

CPUID

This is the most popular among these techniques. This instruction is executed with EAX=0x1 as input, and the return value describes the processors features. The 31st bit of ECX on a physical machine will be equal to 0. On a guest VM, it will equal to 1.

Another method using CPUID executes it with EAX=0x40000000, which is called “hypervisor brand.” The return result will be the virtualization vendor (“VMwareVMware” for VMWare and “Microsoft HV” for Microsoft). The result is stored in ECX and EDX.

Red Pill and No Pill

Red Pill is an anti-VM technique that executes the SIDT instruction to grab the value of the IDTR register. The VM monitor must relocate the guest’s IDTR to avoid conflict with the host’s IDTR. Since the VM monitor is not notified when the VM runs the SIDT instruction, the IDTR for the VM is returned.

No Pill relies on the fact that LDT structure is assigned to a processor, not an OS. I’s location on a host machine will be zero and on a VM will be non-zero.

sha256: 69a42cf3f962fc39bbe2ade5a51d7eac12e9c144fa6c76d763e0bebd01cd931d