Lab 9: Testing and Debugging
Contents
Introduction
It is quite often the case in programming that you will get a program to compile, but when it runs, it either crashes or something about the program's behavior or output is not what you expect. When this happens, we say that your program has a bug. (See this page about where the term comes from.) Specifically, we say that your program has either a logic error or run-time error (there are also compile-time errors, which prevent code from compiling in the first place, but that's not today's topic). It is often not an easy task to track down the causes of such bugs, which then makes fixing them that much harder. Today's lab will give you some tools for combating these kinds of bugs. First, lets define the two types of bugs.
Run-time error
A run-time error results in the program crashing. In C++, especially when dynamic memory is being used, the most common run-time error is a segmentation fault (or segfault for short). Segmentation faults are a result of you trying to use or delete memory that isn't allocated. Other run-time errors are caused because unexpected input chokes your code.
Logic error
Logic errors, also called semantic errors, are errors that result in bad output or unexpected program behavior. The program doesn't crash, but it doesn't do what you expected it to do. For instance, say you write a program that will prompt a user for input over and over again, until the user enters 'q', but when you run it, the user is only prompted once, event though 'q' was never entered. Logic errors are generally the most difficult to track down.
Description
The purpose of this lab is to get you familiar with how to test and debug programs. By the end of this lab, you should be able to:
- write unit tests for functions
- insert cout statements to keep track of program state
- use a debugger (GNU Debugger or LLDB) to track program state
Tasks
Complete each of the following parts in a separate file and have the professor check your work before the end of lab.
Task 1: Unit testing
In this task, you'll learn about a simplified version of unit testing, which can be used as a preventative measure against bugs. Unit testing is a way of ensuring that individual components of a program function as expected. Rather than running your whole program, unit testing test smaller chunks independently of each other. For instance, if your program has four functions, then we can write one unit test per function to ensure that those function behave as expected. For right now, we will only concentrate on functions that return a value or modify a parameter via pass-by-reference. Said differently, we will ignore functions that only print out information to the console.
Lets suppose that we have the following (buggy) program, which computes the amount of sales tax that should be paid on a bill.
If we run this, we get something like:
$ ./lab9-task1 Enter your bill total and tax amount: 100 0.045 Sales tax owed: 2222.22
That's a little weird, because we should have gotten 4.5. In this program, the problem either stems from the startSalesTaxInterface function or the computeSalesTax function. Because we're not going to unit test functions that don't return anything, we must ignore the first function (that's not to say it doesn't contain the bug, just that we won't unit test it to check). We can unit test the second function, though. First, note that we have add the line #include <assert.h> at the top of the file. This allows us to use a function called assert(expression). This function will test the expression. If the expression evaluates to false, then the assertion fails and it prints out some useful information. Lets see how it works.
Add this function to the above code (make sure you declare the function above main with the other function declarations):
This function checks that the value returned after invoking computeSalesTax with the bill total 100 and the tax rate of 0.045 is 4.5. It does the same for a bill total of 100 with a tax rate of 0 (which should be 0).
Now add the line testComputeSalesTax(); to the top of main. Now if we run the program, we get:
$ ./lab9-task1
Assertion failed: (computeSalesTax(100, 0.045) == 4.5), function testComputeSalesTax, file lab9-task1.cpp, line 56.
Abort trap: 6
Your output may look slightly different, but it should include the assertion that failed, the function that contained the failed assertion, and the file. So, we know that something is wrong with the computeSalesTax function itself (and it's not something with how we're reading data in from the user). If we can prevent the assertions from failing, than we can be confident that our function computeSalesTax behaves as expected.
Based on this, look closely at the computeSalesTax function and see if you can spot the error. Fix it and see if it runs. Now write two more asserts and add them to the testComputeSalesTax() function. Make sure the program passes them all!
Once testing is complete, you can remove the invocations of the testing functions from main (you don't need to run the tests every time the program is run).
Task 2: cout statements
Narrowing down logical and run-time bugs can be made substantially easier by adding in some well placed cout statement. If your program segfaults when it runs, cout statements can help you figure out where the segfault happens. All you do is add lines like: cout << "Entering functionX" << endl; at the top of a function or cout << "Deleting pointer" << endl; right before a call to delete. For example, consider this C++ program:
The expected behavior is that we should be able to delete the pointer because we've allocated space for it. However, when I run this program, it crashes. But why? Here's how we can update this program with couts:
Two things about this. First, these couts "fixed" the error I was getting (segfaults are often not reproducible in C++ due to the way that memory is used by a program). But it didn't really fix the problem. When I run this program, I get the following output:
$ ./lab9-task2
Top of main; creating pointer
Deleting pointer
Hmm, that's strange because this is suggesting that the function makePointer is never invoked. Oh, wait, that's correct! I forgot to invoke it in main. We need to change line 21 in the code above to be: int *pointer = makePointer(5);. Now if we run that, we get:
$ ./lab9-task2
Top of main; creating pointer
Entering makePointer(5)
Storing 5 to newly allocated space
Returning the new pointer
Deleting pointer
Now, try adding cout statements to the code from Task 1. Try to see why we aren't getting the value we expect. Try adding a line like: cout << "total * tax: " << (total*tax) << endl; before the function returns. Does that yield the correct value? If so, what could be causing the issue?
Debuggers
There is a way to step through a program as it is executed by the computer and see the state of the program. That is, the current value of variables, what line is being executed, etc. These external programs that show this information to you are called debuggers. A common one is the GNU Debugger, or gdb for short. On more recent versions of Mac, there is also LLDB. The work roughly the same. You first compile your program, then you run gdb or lldb, passing your executable as a command line argument.
First things first, we need to compile our program with a special debugging flag:
$ g++ -g lab9-task2.cpp -o lab9-task2
This will allow the debugger to refer to the code in our .cpp file and not just the executable. Here's what happens when you take the original program from Task 2 and run it through the debugger (this is with lldb on a newer Mac; on Windows, use gdb instead):
$ g++ -g lab9-task2.cpp -o lab9-task2 $ lldb lab9-task2 Current executable set to 'lab9-task2' (x86_64). (lldb) run Process 44843 launched: 'lab9-task2' (x86_64) lab9-task2(44843,0x7fff73aca310) malloc: *** error for object 0x7fff5fbff5a0: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Process 44843 stopped * thread #1: tid = 0x44f263, 0x00007fff8e28c866 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT frame #0: 0x00007fff8e28c866 libsystem_kernel.dylib`__pthread_kill + 10 libsystem_kernel.dylib`__pthread_kill + 10: -> 0x7fff8e28c866: jae 0x7fff8e28c870 ; __pthread_kill + 20 0x7fff8e28c868: movq %rax, %rdi 0x7fff8e28c86b: jmpq 0x7fff8e289175 ; cerror_nocancel 0x7fff8e28c870: ret (lldb) bt * thread #1: tid = 0x44f263, 0x00007fff8e28c866 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT * frame #0: 0x00007fff8e28c866 libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x00007fff8925f35c libsystem_pthread.dylib`pthread_kill + 92 frame #2: 0x00007fff8ec31b1a libsystem_c.dylib`abort + 125 frame #3: 0x00007fff8a20c07f libsystem_malloc.dylib`free + 411 frame #4: 0x0000000100000f1f lab9-task2`main + 47 at lab9-task2.cpp:23 (lldb)
We start off by typing run, then the program dies. Then we can do a backtrace, which tells us what lines things were on when the error occurred. In lldb, I typed bt. In gdb, you would type backtrace.