Code coverage using Eclipse, GNU ARM toolchain, and Gcov for embedded systems

Overview

The core difficulty of getting code coverage with gcov for embedded systems is the following: a program instrumented to collect coverage data (compiled with --coverage compiler and linker flags) once run produces .gcda files for each source file. On a desktop/server this works fine. On an embedded device, this means you need storage and a file system to allow these files to be created which you can transfer to the host later to see coverage information.

Providing for storage and a file system just to get code coverage may be not so desirable, so a “hacky” approach is chosen, which works as follows:

• Run a gdb debugging session of your instrumented program
• Try to trap into the gcov functions that produce .gcda file at the point of time where the coverage information to be written is available in memory
• Use gdb command dump binary memory to generate the .gcda files on your desktop

Existing Approaches

When I searched for existing tutorials, I found this. It presents the concept and the approach very nicely. However; when I used it, I was stuck at a hang inside the gcc function __gcov_flush which is the function that eventually creates .gcda files.

What I did is to port the whole gcc code that is required to implement gcov_exit (this and __gcov_flush are almost identical for our purpose) and plugged the whole source into my sample project to see what is causing the hang. It turned out to be an undocumented bug in the gcc code which I fixed and then I was able to get the coverage collection going.

I later also found this, which further echoes the approach I took.

Using My Ported Code

As you would figure out with the first reference tutorial I have provided above and the fact that an undocumented blocking bug exists, it is necessary to use a ported version of gcov code that I had to achieve to get the coverage going and at this point of time this nice tutorial (first reference) is not sufficient to achieve coverage.

Since it can be a bit overwhelming to use the port directly, I made a sample project that uses the port and serves as a mechanism to easily explain the process of getting the coverage data. However; if you want to directly work with the port, you can get it here. There is a README that explains how to integrate it into your project and generate the .gcda files.

NOTE: there is a significant difference between the reference tutorial and the approach you need to take while using my ported code. The reference tutorial introduces stubs for various C library functions, which are not needed when using my ported code. Please do not mix stubs from the reference while using the approach provided here.

In this post, I first present how to use the sample project to generate .gcda files, post which it’s easier to start integrating my port of gcov into your project.

Learn with a Sample Project

Get the sample project from github and follow the cloning and setup instructions in the README of this project. Once you are done, proceed here.

Import the Sample Project

Build the Project

Build the Project

Success in Console

Errors to Ignore

Start Debugging Session

As you would see while setting up, you can run this project either with a STM32F4-Discovery board using OpenOCD, or with a QEMU emulator for this board.

With QEMU

Click the Down Arrowhead

Select QEMU then click Debug

With STM32F4-Discovery using OpenOCD

First connect your STM32F4-Discovery device to your computer on a USB port and then do the following:

Click the Down Arrowhead

Select OpenOCD then click Debug

Get Familiar with Eclipse Debugging View

Once you choose any of the above debugging methods, Eclipse would prompt you (in case not done before) saying it would switch you to a different “perspective” for debugging. Say OK and it would take you to a view that looks like as follows:

Here is a break-up of the above view for your help:

Thread & Stack Trace View

Variables & Watches View

Files Area for Line-by-line execution View

Console & Logging Area

Setup Required Breakpoints

One of the required step is to setup two breakpoints. Set them up as follows:

Open Breakpoints Tab

Open gcov-io.c to setup breakpoints

Path of gcov-io.c

File opened in Files Area

Add breakpoint at line# 251

Breakpoint appears in Breakpoints Tab

Add breakpoint at line# 276

Breakpoint appears in Breakpoints Tab

Setup Required Watch Expression

We need a watch expression for the variable __gcov_var__ported. Set it up as follows:

Open Expressions View

Blank Expressions View on Opening

Add Required Watch Expression

On Successful Watch Creation

Open Details and Keep for Using Later

Run Program

Click the button to run the program and wait for hitting the breakpoints. It would run its main logic of turning on LEDs on the stm32f4-discovery board/emulator and then it would hit the line#251 breakpoint where it would stop.

View on Hitting the Breakpoint

This is what you would see in the stack trace and file area:

Stack Trace

Line in Green

The most important part of this view is the Expressions area. Here, you would see the complete path of the .gcda file that should be generated.

The filename field has the full path of a .gcda file

Generating the .gcda files

We are almost there! This is the time to generate the .gcda files.

Open the GDB Console

When the program stops on a breakpoint, you may not have the gdb console in the view at the bottom. Eclipse has various consoles, and there is way to switch among them. Do the following to get the gdb console:

Open the gdb Console

How gdb Console looks

Use dump binary command to create .gcda file

Type the dump binary memory command in the gdb console. Its syntax is as follows:

dump binary memory

The following snapshot shows you what values to provide for filename, start-addr and end-addr variables in the syntax above. We get the value from fields of the __gcov_var__ported variable as follows:

• The filename variable comes from the filename field.
• The start-addr variable comes from the gcda_buf field.
• The end-addr variable comes from the gcda_buf_pos field.

Once you press enter or return after entering the command, the .gcda file gets created; however, there is no message output to confirm that.

This is just one .gcda file!

That’s true! Now, it’s time to hit the button again and wait for the next hit of the breakpoint on line #251. This time it is going to be a different source file for which the .gcda file name would be shown. Do the same steps as above to create it.

Repeat the process till the program ends. You would be able to generate the .gcda files for all source files.

Coverage Data Visualization

Using Eclipse (Not Working Currently)

Eclipse has a nice integration with coverage data files. On clicking a .gcda file you can see a summary of code coverage data of the corresponding source file. However, because of this eclipse bug, which was not fixed at the time of writing this (present in even Eclipse Neon.1 – current version), viewing the coverage using colored lines per source file showing which lines got executed and how many times, and which did not, is not working currently.

Using lcov

lcov is a tool that generates HTML representation (looks very similar to how Eclipse view per source file should look).

Since lcov is not part of gnu arm toolchain, there are some compatibility issues making it work with the .gcda files we created here. However, I found the way to get this to work. I have presented the steps that work with the sample project below. The steps work for Linux/Mac. Similar steps would work even on Windows once you have installed lcov. On Mac, you can install lcov using Mac Ports.

Go to the Debug folder of the sample project.

$ cd /Users/reeteshranjan/Projects/OpenSource/libgcov-embedded-example/Debug


Execute the lcov command. We fix the compatibility by specifying the gnu arm gcov as the gcov tool to use to parse the .gcda files:

$ lcov --capture --output-file main.info --directory . --gcov-tool /usr/local/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gcov

NOTE: The path of the gnu arm gcov may be different in your environment.

Now runt he genhtml command to create the HTML files.

$ genhtml -o html main.info

You would get a html directory with many files. Open the index.html file to see the coverage data visualization.

HTML files enclosed in Sample Project

The reports directory in the sample project contains the HTML visualization files I have made using my own run of the process in this post.

Example HTML Visualization

Here is a snapshot of how the HTML visualization looks:

Coverage of Your Own Project

After having seen the process with the sample project, it should be easy to use the ported gcov code in your own project to do the coverage data collection. The summary of what needs to be done is as follows:

• Include the ported gcov code into your project (see below on how)
• In a gdb debugging session, set breakpoints and watches as explained above. These steps are the same for your project whether you use a command line gdb or through Eclipse or through any other IDE.
• When the break point at line#251 in gcov-io.c hits, run the gdb command dump binary memory. The syntax and the values to provide have been explained above.

Get the ported gcov code from here. The README of the project explains the process to include it in your project.