TravisCoverals – Boost C++ Libraries

Context Navigation

Version 9 (modified by Antony Polukhin, 6 years ago) ( diff )
Updated the TravisCI to use sanitizers

Travis and Coveralls integration

Travis and Coveralls Setup

We'll be adding TravisCI and Coveralls integration to a Boost library. Let's take a Boost.LexicalCast as an example. At the end we'll get the following.

Travis requires some access permissions to the repository that are usually provided by Boostorg Admins. However bothering Admins is actually not required:

go to the Boost project page (in our example it would be https://github.com/boostorg/lexical_cast)
press the "Fork" button at the top right corner. Now you have a copy of the boost project in your account.
clone the original boostorg repo or use an existing clone. Change directory to the clone folder (cd boost_maintain/boost/libs/lexical_cast/)
add our forked repo as another remote host for default push: git remote set-url --add origin git@github.com:<your_login>/lexical_cast.git. If everything is right, you'll see something like this if you execute git remote -v:
```
origin	git@github.com:boostorg/lexical_cast.git (fetch)
origin	git@github.com:boostorg/lexical_cast.git (push)
origin	git@github.com:<your_login>/lexical_cast.git (push)
```

log in into Travis
go to the https://travis-ci.org/profile/ and enable builds for the forked project.
log into Coverals.
go to the https://coveralls.io/repos/new and add a forked repo.

now add the .travis.yml file, tune the README.md and commit changes:

git checkout develop                                         # We'll be applying all the changes to the develop branch
cp ../variant/.travis.yml ./                                 # Copying .travis.yml from Boost.Variant
sed -i 's/IGNORE_COVERAGE=.*/IGNORE_COVERAGE=/g' .travis.yml # Reseting files to ignore in coverage tests report
git add .travis.yml                                          # Adding .travis.yml for next commit

cp ../variant/README.md ./                                   # Copying README.md from Boost.Variant
sed -i 's/variant/lexical_cast/g' README.md                  # Replacing `variant` with `lexical_cast`
sed -i 's/apolukhin/<your_login>/g' README.md                # Replace `apolukhin` login name wuth your own login name
gedit README.md                                              # ... editing README.md in editor
git add README.md                                            # Adding README.md for next commit

git commit -m "Travis and Coveralls integration"             # Committing changes to GIT
git push                                                     # push changes to boostorg and forked repo using one command

That's it! If everything is done right, you'll see the build process going on TravisCI and after ~20 minutes results will be visible in README.md view on github.

Coveralls tuning

Investigating the results

After successful run of TravisCI and Coveralls we'll see tests coverage results displayed in README.md view on github. Time to investigate the results!

click on the results https://coveralls.io/r/apolukhin/lexical_cast?branch=develop
click on the build number to see detailed coverage description
click on the source file you wish to investigate

Setting sources location

Chances are high, that Coveralls will fail to automatically detect source location. In that case you need to

set "Git repo root directory:" to /home/travis/boost-local/
set "Git repo sub directory: :" to include/ (or src/ in case of a source file in src/ folder)

If everything is done right you'll see the source file with highlighted lines. Green lines are covered by tests, red lines are in binary file and are not covered by test, gray lines are not in a test's binary file (there's no code in tests that uses/compile that line).

Ignoring specified files coverage

Sometimes files from other projects could appear in coverage reports. To disable those files edit .travis.yml file and set IGNORE_COVERAGE= to files that must be ignored. IGNORE_COVERAGE variable understands wildcards, so you could disable files just like this:

    - IGNORE_COVERAGE='*/boost/progress.hpp */filesystem/src/path.cpp */numeric/conversion/converter_policies.hpp'

Profits =

Travis is a continuous integration system that builds and run tests on each push to github repository. It helps you to find and report errors faster.

Coveralls shows tests coverage. This is a useful feature, that allows you to make sure that newly created code is covered by tests. Even 100% coverage does not mean that there's no errors. However it motivates people to write tests, helps to find trivial errors and typos and sometimes even helps to find hard detectable errors (bug that have been in Variant for years and was highlighted by Coveralls)

`.travis.yml` content (05.01.2016)

# Use, modification, and distribution are
# subject to the Boost Software License, Version 1.0. (See accompanying
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
#
# Copyright Antony Polukhin 2014-2016.

#
# See https://svn.boost.org/trac/boost/wiki/TravisCoverals for description of this file
# and how it can be used with Boost libraries.
#
# File revision #6

sudo: false
language: cpp
compiler:
    - gcc
#    - clang

os:
    - linux

env:
    global:
        # Autodetect Boost branch by using the following code: - BRANCH_TO_TEST=$TRAVIS_BRANCH
        # or just directly specify it
        - BRANCH_TO_TEST=$TRAVIS_BRANCH

        # Files, which coverage results must be ignored (files from other projects).
        # Example: - IGNORE_COVERAGE='*/boost/progress.hpp */filesystem/src/*'
        - IGNORE_COVERAGE=''

        # Explicitly remove the following library from Boost. This may be usefull, if you're for example running Travis
        # from `Boost.DLL` repo, while Boost already has `dll`.
        #
        # By default is eaual to - BOOST_REMOVE=$(basename $TRAVIS_BUILD_DIR)
        # This will force to use local repo content, instead of the Boost's default.
        - BOOST_REMOVE=$(basename $TRAVIS_BUILD_DIR)

    matrix:
        # Note that "--coverage -fsanitize=address,leak,undefined -DBOOST_TRAVISCI_BUILD" are added automatically lower in code
        - CXX_FLAGS="-std=c++98" LINK_FLAGS="" TOOLSET=gcc-6
        - CXX_FLAGS="-std=c++11" LINK_FLAGS="" TOOLSET=gcc-6
        - CXX_FLAGS="-std=c++1y" LINK_FLAGS="" TOOLSET=gcc-6
        #- CXX_FLAGS="-std=c++11 -stdlib=libc++" LINK_FLAGS="-stdlib=libc++" TOOLSET=clang
        #- CXX_FLAGS="-std=c++1y -stdlib=libc++" LINK_FLAGS="-stdlib=libc++" TOOLSET=clang

###############################################################################################################
# From this point and below code is same for all the Boost libs
###############################################################################################################


# Installing additional tools
addons:
  apt:
    sources:
    - ubuntu-toolchain-r-test
    - git-core
    packages:
    - git
    - python-yaml
    - gcc-6
    - g++-6
    - lcov
    - clang
    - libc++-dev

before_install:
    # Set this to the name of the library
    - PROJECT_TO_TEST=`basename $TRAVIS_BUILD_DIR`
    # Cloning Boost libraries (fast nondeep cloning)
    - BOOST=$HOME/boost-local
    - echo "Testing $PROJECT_TO_TEST, to remove $BOOST/libs/$BOOST_REMOVE, testing branch $BRANCH_TO_TEST"
    - git init $BOOST
    - cd $BOOST
    - git remote add --no-tags -t $BRANCH_TO_TEST origin https://github.com/boostorg/boost.git
    - git fetch --depth=1
    - git checkout $BRANCH_TO_TEST
    - git submodule update --jobs=3 --init --merge
    - git remote set-branches --add origin $BRANCH_TO_TEST
    - git pull --recurse-submodules
    - git status
    - rm -rf $BOOST/libs/$BOOST_REMOVE
    - mv $TRAVIS_BUILD_DIR $BOOST/libs/$PROJECT_TO_TEST
    - TRAVIS_BUILD_DIR=$BOOST/libs/$PROJECT_TO_TEST
    - ./bootstrap.sh
    - ./b2 headers
    - cd $BOOST/libs/$PROJECT_TO_TEST/test/

script:
    # `--coverage` flags required to generate coverage info for Coveralls
    - ../../../b2 "testing.launcher=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libasan.so.3 " address-model=64 architecture=x86 toolset=$TOOLSET cxxflags="--coverage -fsanitize=address,leak,undefined -DBOOST_TRAVISCI_BUILD $CXX_FLAGS" linkflags="$LINK_FLAGS --coverage -lasan -lubsan"
    
after_success:
    # Copying Coveralls data to a separate folder
    - mkdir -p $TRAVIS_BUILD_DIR/coverals
    - find ../../../bin.v2/ -name "*.gcda" -exec cp "{}" $TRAVIS_BUILD_DIR/coverals/ \;
    - find ../../../bin.v2/ -name "*.gcno" -exec cp "{}" $TRAVIS_BUILD_DIR/coverals/ \;

    # Preparing Coveralls data by changind data format to a readable one
    - lcov --directory $TRAVIS_BUILD_DIR/coverals --base-directory ./ --capture --output-file $TRAVIS_BUILD_DIR/coverals/coverage.info

    # ... erasing /test/ /example/ folder data
    - cd $BOOST
    - lcov --remove $TRAVIS_BUILD_DIR/coverals/coverage.info "/usr*" "*/$PROJECT_TO_TEST/test/*" $IGNORE_COVERAGE "*/$PROJECT_TO_TEST/tests/*" "*/$PROJECT_TO_TEST/examples/*" "*/$PROJECT_TO_TEST/example/*" -o $TRAVIS_BUILD_DIR/coverals/coverage.info

    # ... erasing data that is not related to this project directly
    - OTHER_LIBS=`grep "submodule .*" .gitmodules | sed 's/\[submodule\ "\(.*\)"\]/"\*\/boost\/\1\.hpp" "\*\/boost\/\1\/\*"/g'| sed "/\"\*\/boost\/$PROJECT_TO_TEST\/\*\"/d" | sed ':a;N;$!ba;s/\n/ /g'`
    - echo $OTHER_LIBS
    - eval "lcov --remove $TRAVIS_BUILD_DIR/coverals/coverage.info $OTHER_LIBS -o $TRAVIS_BUILD_DIR/coverals/coverage.info"

    # Sending data to Coveralls
    - cd $TRAVIS_BUILD_DIR
    - gem install coveralls-lcov
    - coveralls-lcov coverals/coverage.info

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text