Using function as data source in BOOST_DATA_TEST_CASE initiate the conflict with static variables initialization

[mikhail@…]

I got the crash like ​here(ticket 12987) when switched my tests from boost v1.63.0 to v1.64.0. I'm using BOOST_DATA_TEST_CASE with a function as the second parameter. My function uses boost::filesystem::path to walk through directory and collect data files:

static std::vector<boost::filesystem::path> get_files()
{
    std::vector<boost::filesystem::path> result;
    boost::system::error_code error;
    boost::filesystem::directory_iterator it("C:\\", error);
    for (boost::filesystem::directory_iterator const eit; it != eit; ++it)
        if (is_regular_file(it->status()))
            result.push_back(it->path());
    return result;
}

BOOST_DATA_TEST_CASE(test, get_files(), file_name)
{
    std::cout << file_name << std::endl;
}

I was very surprised when found that my get_files() called during static variables initialization (from _initterm()):

Exception thrown at 0x0000000000000000 in BoostTests.exe: 0xC0000005: Access violation executing location 0x0000000000000000. occurred

0000000000000000()	Unknown
BoostTests.exe!`anonymous namespace'::make_permissions(const boost::filesystem::path & p, unsigned long attr) Line 629	C++
BoostTests.exe!`anonymous namespace'::dir_itr_first(void * & handle, const boost::filesystem::path & dir, std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > & target, boost::filesystem::file_status & sf, boost::filesystem::file_status & symlink_sf) Line 2275	C++
BoostTests.exe!boost::filesystem::detail::directory_iterator_construct(boost::filesystem::directory_iterator & it, const boost::filesystem::path & p, boost::system::error_code * ec) Line 2374	C++
BoostTests.exe!boost::filesystem::directory_iterator::directory_iterator(const boost::filesystem::path & p, boost::system::error_code & ec) Line 907	C++
BoostTests.exe!get_files() Line 15	C++
BoostTests.exe!test::`dynamic initializer for 'testcase_registrar2222''() Line 22	C++
BoostTests.exe!_initterm(void(*)() * first, void(*)() * last) Line 22	C++
[External Code]	

On my opinion the data collection has to be run immediately before test starting. I see some bonuses in this approach:

• no unnecessary data collecting when user run only selected set of tests;
• correct and suitable error handling for function, now any exception means crash;

After some investigation I found that test_case_gen runs test creating in constructor to fill m_test_cases variable which is used only during next() call. However, next() also called from static context from auto_test_unit_registrar. And this is the problem!

Could you please change the boost test behavior to grant that test collecting will be made in test main method, but not during static variable initialization?

[Raffi Enficiaud]

This is quite a big change in the way the initialization of the boost.test framework works, but I agree that this is a nice feature to have. I am leaving this open for the moment.

[Ilia <ki.stfu@…>]

I have faced with similar issue when upgrading to 1.65.1 and I can only say that at the moment BOOST_DATA_TEST_CASE cannot be used with something like get_files() for gettign different input files.

1. Could you update the manual to state that dataset is being initialized during initialization of statics?
2. Can you provide the estimated time when this issue be resolved? I don't want to migrate to another test framework, but now it blocks me.

[Raffi Enficiaud]

To unblock you, you can always mimic the dataset behaviour by manually registering test cases in the init of the test framework:

• http://www.boost.org/doc/libs/release/libs/test/doc/html/boost_test/adv_scenarios/test_module_init_overview.html for test initialization function
• http://www.boost.org/doc/libs/release/libs/test/doc/html/boost_test/tests_organization/test_cases/test_organization_nullary.html#ref_BOOST_TEST_CASE for manual registration

You may for instance collect the files and then manually register a test for each of them.

[Raffi Enficiaud]

I am not sure the crashes you are experiencing is due to a change in Boost.Test between 1.63 and a later version. The get_files function is always evaluated at static initialization time as the BOOST_DATA_TEST_CASE creates a static global object.

I tend to think that there is something in filesystem that prevents from calling functions during this global static initialization phase. So I suggest that you ask or raise a ticket on boost.filesystem side to make it clear.

However, there is the ticket #12953 that should help as it postpones the evaluation of the datasets, it would be good if you can give a try. To support this further, I suggest getting the files later in the init phase by providing a proper dataset interface to the function retrieving the files, as shown here (for instance getting the files when calling begin or size).

Please keep me posted.

[Raffi Enficiaud]

Any news?

[Raffi Enficiaud]

Kind reminder

[Raffi Enficiaud]

Closing, worst case related to #12953.

[Raffi Enficiaud]

Fixed via #12953