What is the EASIEST way to bypass a sorting function? Figure this out and it will make the task much simpler. Reminder: You are NOT allowed to change any code.

computer science


CS 6035 Introduction to Information Security Project #1 Buffer Overflows

Fall 2019

      Join the slack channel (You will get an invite) – TA’s will be active in here daily to answer your questions.

      Read Piazza – Lots of questions are answered there daily. Be sure to check there before asking a question.

      Plagiarism will not be tolerated! Anything more than 10% of work that is not yours, will be considered plagiarism, and will be given a 0% and not graded. Everything that is not yours BE SURE TO CITE.

          We will be using anti-cheating software, so if you copy you will be caught and reported to OSI.

          You must include a Works Cited/Bibliography page in MLA format.

          You can use easybib or citationmachine or anything else to help cite.


The goals of this project:

          Understanding how stack and heap memory are used

          Understanding the concepts of buffer overflow

          Exploiting a stack buffer overflow vulnerability

          Understanding code reuse attacks (advanced buffer overflow attacks)

          With the knowledge about buffer overflow, students are expected to launch an attack that exploits a stack buffer overflow vulnerability in a provided sort.c program.

          Students are asked to read up on and write about code reuse attacks.

          Should be able to explain what a buffer overflow is and why are they dangerous?

          Should be able to explain how an actual buffer overflow works on both the stack and the heap.

A Helpful Suggestion:

It is natural for everyone to want to go complete the exploit first. However, if you follow the project flow and answer the first two questions before moving on to the exploit, it will make the process a whole lot easier to understand.

Helpful Hint/Question:

               What is the EASIEST way to bypass a sorting function? Figure this out and it will make the task much simpler. Reminder: You are NOT allowed to change any code.

               If you are struggling, use the #define to comment out the sort, recompile, and try and get the shell to execute. Once satisfied, uncomment the #define, recompile, and ensure the shell still executes.

               Do not wait until the last minute - this exploit does take significant time, especially if you are new to C and debugging.

               No C or assembly programming is required in this project.


Some information you could probably use throughout this project will be listed here. Understand these are only helpful topics and will not give you the answers to solve the problems but could help guide you to finding better information.


      When debugging using GDB, compile with: gcc -g sort.c -o sort -fno-stack-protector

      Useful tips for GDB debugging: Useful Commands for GDB or Tricky Useful Commands for GDB

      More useful commands for GDB: Most Tricky Useful Commands for GDB Debugger

      What is a stack or the heap? Information about Stacks / Information about Heaps

      What is a buffer overflow? Buffer Overflow

      Setting up the VM? Try this one or check piazza for the thread about setting up your VM.

      Add a shared folder via the VM: Getting Files off the VM

      A good video about understanding ROP: Understanding ROP -- YouTube

      Do not worry about endianness!

      Addresses in data.txt should be 12345A, NOT 0x12345A or /x12/x34/x5A. The file is read as hex data.

Understanding Buffer Overflow (40 points)


Note: For this task, you may use online resources to learn about a program with these vulnerabilities, but please cite these online sources. The diagrams you use can be copied from these online resources, but if they are, explain the diagram thoroughly in your own words. Review “Suggestions/Warnings above about how to cite and the percentage allowed to be copied.


1.       Stack Buffer Overflow (25 points)

a.               (15 points) Memory Architecture. (Diagram(s) would be helpful, but are not required)

i.              Describe the stack in the address space of the VM (in generalities).

ii.                Addresses where in memory the stack would be located (specifically).

          Which direction, relative to overall memory, does a stack consume memory when it grows?

iii.                 Explain how program control flow is implemented using the stack.

iv.               How does the stack structure get affected when a buffer of size ‘non-binary’ is allocated by a function (ie – buffer size which causes misalignment within the stack)? [Also known as ‘non-binary’]

v.              Create a diagram that includes the following

          What does the stack structure looks like when data is pushed onto the stack and popped off the stack?

          Show what register values are placed onto and used with the stack.

          Where would arguments be placed on the stack?

          Where are local user variables placed on the stack?

b.             (10 points) Testing Program – Stack Buffer Overflow

i.              (4 Points) Write a testing program (in C) that contains a stack buffer overflow vulnerability. (You cannot use sort.c from task 2) . You are not required to exploit it.

          Provide this program in your PDF writeup. (copy/paste is fine. No Screenshots)

ii.                (6 Points) Show what the stack layout looks like and explain how to exploit it. (Include a diagram)

          Include the following items:

a.               The order of parameters (if applicable), return address,  saved registers (if applicable), and local variable(s).

b.              The sizes in bytes.

c.               The overflow direction in the stack.

d.             Size of the overflowing buffer to reach and overwrite the return address.

e.               Overflow data that is meaningful for an exploit (this can be general).

Related Questions in computer science category