id,summary,reporter,owner,description,type,status,milestone,component,version,severity,resolution,keywords,cc
13185,serialization of null pointer fails in optimized builds,tyler@…,Robert Ramey,"invoke in oserializer.hpp dereferences a pointer (UB for nullptrs), and then checks if it is null on the next line. The optimizer detects this UB and assumes the pointer cannot be null, and optimizes the whole if block out. This affected our code in production (serializing nullptrs was a corner case for us), so I assume it's affecting other folks.

Here's a demo of clang trunk performing this optimization (earlier versions didn't perform this optimization): https://godbolt.org/g/y6sbZP

https://github.com/boostorg/serialization/blob/6b33d1cd4e11daaf97612561ecd9d4848843897c/include/boost/archive/detail/oserializer.hpp#L468
{{{
    template<class TPtr>
    static void invoke(Archive &ar, const TPtr t){
        register_type(ar, * t);
        if(NULL == t){
            basic_oarchive & boa 
                = boost::serialization::smart_cast_reference<basic_oarchive &>(ar);
            boa.save_null_pointer();
            save_access::end_preamble(ar);
            return;
        }
        save(ar, * t);
    }
}}}

Modifying the code to this, works as expected (although there's still a dereference of a null):
{{{
    template<class TPtr>
    static void invoke(Archive &ar, const TPtr t){
        if(NULL == t){
            register_type(ar, * t);
            basic_oarchive & boa
                = boost::serialization::smart_cast_reference<basic_oarchive &>(ar);
            boa.save_null_pointer();
            save_access::end_preamble(ar);
            return;
        } else {
            register_type(ar, * t);
        }
        save(ar, * t);
    }
}}}

",Bugs,new,To Be Determined,serialization,Boost Development Trunk,Problem,,,