TravisCoverals – Boost C++ Libraries

Context Navigation

Travis, Appveyor and Coveralls integration

Travis, Appveyor and Coveralls Setup

We'll be adding TravisCI, Appveyor and Coveralls integration to a Boost library. Let's take a Boost.LexicalCast as an example. At the end we'll get the following.

Travis requires some access permissions to the repository that are usually provided by Boostorg Admins. However bothering Admins is actually not required:

go to the Boost project page (in our example it would be https://github.com/boostorg/lexical_cast)
press the "Fork" button at the top right corner. Now you have a copy of the boost project in your account.
clone the original boostorg repo or use an existing clone. Change directory to the clone folder (cd boost_maintain/boost/libs/lexical_cast/)
add our forked repo as another remote host for default push: git remote set-url --add origin git@github.com:<your_login>/lexical_cast.git. If everything is right, you'll see something like this if you execute git remote -v:
```
origin	git@github.com:boostorg/lexical_cast.git (fetch)
origin	git@github.com:boostorg/lexical_cast.git (push)
origin	git@github.com:<your_login>/lexical_cast.git (push)
```

log in into Travis
go to the https://travis-ci.org/profile/ and enable builds for the forked project.
log into Coverals.
go to the https://coveralls.io/repos/new and add a forked repo.
go to Appveyor
go to https://ci.appveyor.com/projects/new and enable builds for the forked project.
if you're going to add the appveyor.yml file not in the root directory, set the "Custom configuration .yml file name" in project settings https://ci.appveyor.com/project/<your_login>/lexical_cast/settings to the future file location (for example test/appveyor.yml)

now add the .travis.yml and appveyor.yml files, tune the README.md and commit changes:

git checkout develop                                         # We'll be applying all the changes to the develop branch
cp ../variant/.travis.yml ./                                 # Copying .travis.yml from Boost.Variant
cp ../variant/test/appveyor.yml ./test/                      # Copying appveyor.yml from Boost.Variant
sed -i 's/IGNORE_COVERAGE=.*/IGNORE_COVERAGE=/g' .travis.yml # Reseting files to ignore in coverage tests report
sed -i 's/variant/lexical_cast/g' test/appveyor.yml          # Resetting library to ignore
git add .travis.yml test/appveyor.yml                        # Adding .travis.yml and test/appveyor.yml for next commit

cp ../variant/README.md ./                                   # Copying README.md from Boost.Variant
sed -i 's/variant/lexical_cast/g' README.md                  # Replacing `variant` with `lexical_cast`
sed -i 's/apolukhin/<your_login>/g' README.md                # Replace `apolukhin` login name wuth your own login name
gedit README.md                                              # ... editing README.md in editor
git add README.md                                            # Adding README.md for next commit

git commit -m "Travis and Coveralls integration"             # Committing changes to GIT
git push                                                     # push changes to boostorg and forked repo using one command

That's it! If everything is done right, you'll see the build process going on TravisCI and after ~20 minutes results will be visible in README.md view on github.

Coveralls tuning

Investigating the results

After successful run of TravisCI and Coveralls we'll see tests coverage results displayed in README.md view on github. Time to investigate the results!

click on the results https://coveralls.io/r/apolukhin/lexical_cast?branch=develop
click on the build number to see detailed coverage description
click on the source file you wish to investigate

Setting sources location

Chances are high, that Coveralls will fail to automatically detect source location. In that case you need to

set "Git repo root directory:" to /home/travis/boost-local/
set "Git repo sub directory: :" to include/ (or src/ in case of a source file in src/ folder)

If everything is done right you'll see the source file with highlighted lines. Green lines are covered by tests, red lines are in binary file and are not covered by test, gray lines are not in a test's binary file (there's no code in tests that uses/compile that line).

Ignoring specified files coverage

Sometimes files from other projects could appear in coverage reports. To disable those files edit .travis.yml file and set IGNORE_COVERAGE= to files that must be ignored. IGNORE_COVERAGE variable understands wildcards, so you could disable files just like this:

    - IGNORE_COVERAGE='*/boost/progress.hpp */filesystem/src/path.cpp */numeric/conversion/converter_policies.hpp'

Profits

TravisCI and Appveyor are continuous integration systems that build and run tests on each push to github repository. TravisCI runs tests on Linux, Appveyor runs tests on Windows. They help you to find errors faster.

Coveralls shows tests coverage. This is a useful feature, that allows you to make sure that newly created code is covered by tests. Even 100% coverage does not mean that there's no errors. However it motivates people to write tests, helps to find trivial errors and typos and sometimes even helps to find hard detectable errors (bug that have been in Variant for years and was highlighted by Coveralls)

`.travis.yml` content

# Use, modification, and distribution are
# subject to the Boost Software License, Version 1.0. (See accompanying
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
#
# Copyright Antony Polukhin 2014-2016.

#
# See https://svn.boost.org/trac/boost/wiki/TravisCoverals for description of this file
# and how it can be used with Boost libraries.
#
# File revision #7

sudo: false
language: cpp
compiler:
    - gcc
#    - clang

os:
    - linux

env:
    global:
        # Autodetect Boost branch by using the following code: - BRANCH_TO_TEST=$TRAVIS_BRANCH
        # or just directly specify it
        - BRANCH_TO_TEST=$TRAVIS_BRANCH

        # Files, which coverage results must be ignored (files from other projects).
        # Example: - IGNORE_COVERAGE='*/boost/progress.hpp */filesystem/src/*'
        - IGNORE_COVERAGE=''

        # Explicitly remove the following library from Boost. This may be usefull, if you're for example running Travis
        # from `Boost.DLL` repo, while Boost already has `dll`.
        #
        # By default is eaual to - BOOST_REMOVE=$(basename $TRAVIS_BUILD_DIR)
        # This will force to use local repo content, instead of the Boost's default.
        - BOOST_REMOVE=$(basename $TRAVIS_BUILD_DIR)

    matrix:
        # Note that "--coverage -fsanitize=address,leak,undefined -DBOOST_TRAVISCI_BUILD" are added automatically lower in code
        - CXX_FLAGS="-std=c++98" LINK_FLAGS="" TOOLSET=gcc-6
        - CXX_FLAGS="-std=c++11" LINK_FLAGS="" TOOLSET=gcc-6
        - CXX_FLAGS="-std=c++1y" LINK_FLAGS="" TOOLSET=gcc-6
        #- CXX_FLAGS="-std=c++11 -stdlib=libc++" LINK_FLAGS="-stdlib=libc++" TOOLSET=clang
        #- CXX_FLAGS="-std=c++1y -stdlib=libc++" LINK_FLAGS="-stdlib=libc++" TOOLSET=clang

###############################################################################################################
# From this point and below code is same for all the Boost libs
###############################################################################################################


# Installing additional tools
addons:
  apt:
    sources:
    - ubuntu-toolchain-r-test
    - git-core
    packages:
    - git
    - python-yaml
    - gcc-6
    - g++-6
    - clang
    - libc++-dev

before_install:
    # Set this to the name of the library
    - PROJECT_TO_TEST=`basename $TRAVIS_BUILD_DIR`
    # Cloning Boost libraries (fast nondeep cloning)
    - BOOST=$HOME/boost-local
    - echo "Testing $PROJECT_TO_TEST, to remove $BOOST/libs/$BOOST_REMOVE, testing branch $BRANCH_TO_TEST"
    - git init $BOOST
    - cd $BOOST
    - git remote add --no-tags -t $BRANCH_TO_TEST origin https://github.com/boostorg/boost.git
    - git fetch --depth=1
    - git checkout $BRANCH_TO_TEST
    - git submodule update --jobs=3 --init --merge
    - git remote set-branches --add origin $BRANCH_TO_TEST
    - git pull --recurse-submodules
    - git status
    - rm -rf $BOOST/libs/$BOOST_REMOVE
    - mv $TRAVIS_BUILD_DIR $BOOST/libs/$PROJECT_TO_TEST
    - TRAVIS_BUILD_DIR=$BOOST/libs/$PROJECT_TO_TEST
    - ./bootstrap.sh
    - ./b2 headers
    - cd $BOOST/libs/$PROJECT_TO_TEST/test/

script:
    # `--coverage` flags required to generate coverage info for Coveralls
    - ../../../b2 "testing.launcher=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libasan.so.3 " address-model=64 architecture=x86 toolset=$TOOLSET cxxflags="--coverage -fsanitize=address,leak,undefined -DBOOST_TRAVISCI_BUILD $CXX_FLAGS" linkflags="$LINK_FLAGS --coverage -lasan -lubsan"

after_success:
    # Copying Coveralls data to a separate folder
    - mkdir -p $TRAVIS_BUILD_DIR/coverals
    - find ../../../bin.v2/ -name "*.gcda" -exec cp "{}" $TRAVIS_BUILD_DIR/coverals/ \;
    - find ../../../bin.v2/ -name "*.gcno" -exec cp "{}" $TRAVIS_BUILD_DIR/coverals/ \;
    - find ../../../bin.v2/ -name "*.da" -exec cp "{}" $TRAVIS_BUILD_DIR/coverals/ \;
    - find ../../../bin.v2/ -name "*.no" -exec cp "{}" $TRAVIS_BUILD_DIR/coverals/ \;
    - wget https://github.com/linux-test-project/lcov/archive/v1.12.zip
    - unzip v1.12.zip
    - LCOV="`pwd`/lcov-1.12/bin/lcov --gcov-tool gcov-6"

    # Preparing Coveralls data by changind data format to a readable one
    - echo "$LCOV --directory $TRAVIS_BUILD_DIR/coverals --base-directory `pwd` --capture --output-file $TRAVIS_BUILD_DIR/coverals/coverage.info"
    - $LCOV --directory $TRAVIS_BUILD_DIR/coverals --base-directory `pwd` --capture --output-file $TRAVIS_BUILD_DIR/coverals/coverage.info

    # ... erasing /test/ /example/ folder data
    - cd $BOOST
    - $LCOV --remove $TRAVIS_BUILD_DIR/coverals/coverage.info "/usr*" "*/$PROJECT_TO_TEST/test/*" $IGNORE_COVERAGE "*/$PROJECT_TO_TEST/tests/*" "*/$PROJECT_TO_TEST/examples/*" "*/$PROJECT_TO_TEST/example/*" -o $TRAVIS_BUILD_DIR/coverals/coverage.info

    # ... erasing data that is not related to this project directly
    - OTHER_LIBS=`grep "submodule .*" .gitmodules | sed 's/\[submodule\ "\(.*\)"\]/"\*\/boost\/\1\.hpp" "\*\/boost\/\1\/\*"/g'| sed "/\"\*\/boost\/$PROJECT_TO_TEST\/\*\"/d" | sed ':a;N;$!ba;s/\n/ /g'`
    - echo $OTHER_LIBS
    - eval "$LCOV --remove $TRAVIS_BUILD_DIR/coverals/coverage.info $OTHER_LIBS -o $TRAVIS_BUILD_DIR/coverals/coverage.info"

    # Sending data to Coveralls
    - cd $TRAVIS_BUILD_DIR
    - gem install coveralls-lcov
    - coveralls-lcov coverals/coverage.info

`appveyor.yml` content

# Use, modification, and distribution are
# subject to the Boost Software License, Version 1.0. (See accompanying
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
#
# Copyright Antony Polukhin 2016-2017.

#
# See https://svn.boost.org/trac/boost/wiki/TravisCoverals for description of this file
# and how it can be used with Boost libraries.
#
# File revision #5

init:
    - set BRANCH_TO_TEST=%APPVEYOR_REPO_BRANCH%  # Change to branch you wish to test. Use %APPVEYOR_REPO_BRANCH% for current branch.
    - set BOOST_REMOVE=variant                   # Remove this folder from lib from full clone of Boost. If you are testing `any` repo, write here `any`.

###############################################################################################################
# From this point and below code is same for all the Boost libs
###############################################################################################################

version: 1.64.{build}-{branch}
 
# branches to build
branches:
  except:
    - gh-pages

skip_tags: true

before_build:
    - set PATH=%PATH%;C:\\MinGW\\bin
    - echo "Testing %APPVEYOR_PROJECT_NAME%"
    # Cloning Boost libraries (fast nondeep cloning)
    - set BOOST=C:/boost-local
    - git init %BOOST%
    - cd %BOOST%
    - git remote add --no-tags -t %BRANCH_TO_TEST% origin https://github.com/boostorg/boost.git
    - git fetch --depth=1
    - git checkout %BRANCH_TO_TEST%
    - git submodule update --init --merge --jobs 16
    - git remote set-branches --add origin %BRANCH_TO_TEST%
    #- git pull --recurse-submodules        # Updaes submodules to most recent version. Not required
    - rm -rf %BOOST%/libs/%BOOST_REMOVE%
    - mv %APPVEYOR_BUILD_FOLDER% %BOOST%/libs/%APPVEYOR_PROJECT_NAME%

build_script:
    - call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x64
    - bootstrap.bat
    - b2.exe headers
    - cd %BOOST%/libs/%APPVEYOR_PROJECT_NAME%/test

after_build:
before_test:
test_script:
    - ..\..\..\b2.exe address-model=32 architecture=x86 toolset=msvc,gcc cxxflags="-DBOOST_TRAVISCI_BUILD" -sBOOST_BUILD_PATH=.

after_test:
on_success:
on_failure:
on_finish:

Last modified 5 years ago Last modified on May 20, 2017, 7:50:42 AM

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text