How buffer overflow exploits occur

Attackers use buffer overflow exploits to run executable code by overflowing the fixed-size memory buffers reserved for an input process. This code lets the attacker take over the target computer or compromise its data.

There are two types of buffer overflow exploits:
  • Heap based attacks — They flood the memory space reserved for a program, but they are difficult to perform and rare.
  • Stack based attacks — They use the stack memory objects to store user input and are the most common.
The following process describes stack-based buffer overflow attacks:
  1. Normal stack memory process — The fixed-size stack memory object is usually empty and waiting for user input. When a program receives input from the user, such as their name, the data is stored on top of the stack and assigned a return memory address. When the stack is processed, the user’s input is sent to the return address specified by the program.
  2. Overflowing the stack — When the program is written, a specific amount of memory space is reserved for the data. The stack overflows if the data written is larger than the space reserved for it within the memory stack. This is only a problem when combined with malicious input.
  3. Exploiting the overflow — If the program is waiting for a user to enter their name, but the attacker enters an executable command that exceeds the stack size, that command is saved outside of the reserved space.
  4. Running the malicious code — The command is not automatically run just because it exceeds the stack buffer space. But it could be if a return address that points to the malicious command is provided by the attacker. Initially the program starts to crash because of the buffer overflow, but the program tries to recover by using the return address provided by the attacker. If the return address is a valid address, the malicious command is executed.
  5. Exploiting the permissions — Since programs usually run either in kernel mode or with permissions inherited from a service account, the malicious code is now running with the same permissions as the application that was compromised. This could mean the attacker can gain full control of the operating system.

How buffer overflow exploits occur