Coverage analysis

To ensure a high standard of code quality, the RTEMS project has set the goal of achieving 100% code coverage in its testing. As part of my contribution to the project, I needed to verify that the code I added was thoroughly tested. This required me to learn how to perform a coverage analysis, which was a new skill I had acquire.

I reached out to one of my mentors, who provided me with a helpful example of how to run a coverage analysis. The example was slightly different from my situation, so i had to adapt it to my necessity. The work i had to do was mainly to change the arhitecture and paths referenced in the commands. Among the files i received there was also a python script, witch also had to be adapted by changhing the referenced architercture at the bottom of the file.

After some different attempts, this is the workflow i ended up with:

- Step 0 : Build the project

Before generating the coverage report it’s necessary to build the project. To be able to generate the coveragereport it’s necessary to enable it, by adding RTEMS_GCOV_COVERAGE = True in the config.ini file. If the flag is not present, no gcov info will be generated when an excecutable is run.

- Step 1 : Run the Tests

The first step is to run the test, the command i used is this:

rtems-run --rtems-bsp=erc32-sis --log rtems_run.log build/sparc/erc32/testsuites/

In this command --log rtems_run.log specifies that the output of the execution wil be saved in a file named rtems_run.log

- Step 2 : Execute the Python Script

After running the tests, the next step is to process the results using a Python script. I executed the following command:

python run_gcov_tool.py rtems_run.log

This command runs the run_gcov_tool.py script, using the rtems_run.log file generated in the previous step as input. The script processes the log file and prepares the data for coverage analysis. The script is downloadable here.

- Step 3 : Perform the Coverage Analysis

Finally, to generate a detailed coverage report, I used the following command:

gcovr --gcov-executable sparc-rtems6-gcov --html-details ../coverage/aio-coverage.html --html-title "AIO Coverage" --gcov-ignore-parse-errors=negative_hits.warn --filter cpukit/posix/src/aio

This command runs gcovr to produce an HTML coverage report, here’s a description of the flags used:

• --html-details ../coverage/aio-coverage.html generates a detailed HTML report and saves it as aio-coverage.html in the ../coverage/ directory.
• --html-title "AIO Coverage" gives the report a specific title, making it easier to identify.
• --gcov-ignore-parse-errors=negative_hits.warn tells gcovr to ignore certain parsing errors that may occur.
• --filter cpukit/posix/src/aio restricts the coverage analysis to the files related to the Asynchronous I/O (AIO) implementation.

This way I managed to generate a coverage report limited to the code i was interested in. Here’s a screenshot of the output i was able to see.