Buffer overflow protection based on segment limitation

A non-executable stack approach is proposed and evaluated to defense against stack-based buffer overflow attacks under Windows and Intel 32-bit CPUs. A kernel device driver is designed to relocate the application's user-mode stack to the higher address and to modify the effective limit in the code segment descriptor, so the relocated stack is excluded from the code segment. Once any malicious code that attempts to execute in the stack, a general-protection exception is triggered, then the malicious code will be terminated. It is highly effective in preventing both known and yet unknown stack smashing attacks, and its performance overhead is lower than the page-based non-executable stack approach.