Opened 5 years ago

Last modified 5 years ago

#13185 new Bugs

serialization of null pointer fails in optimized builds

Reported by: tyler@… Owned by: Robert Ramey
Milestone: To Be Determined Component: serialization
Version: Boost Development Trunk Severity: Problem
Keywords: Cc:

Description

invoke in oserializer.hpp dereferences a pointer (UB for nullptrs), and then checks if it is null on the next line. The optimizer detects this UB and assumes the pointer cannot be null, and optimizes the whole if block out. This affected our code in production (serializing nullptrs was a corner case for us), so I assume it's affecting other folks.

Here's a demo of clang trunk performing this optimization (earlier versions didn't perform this optimization): https://godbolt.org/g/y6sbZP

https://github.com/boostorg/serialization/blob/6b33d1cd4e11daaf97612561ecd9d4848843897c/include/boost/archive/detail/oserializer.hpp#L468

    template<class TPtr>
    static void invoke(Archive &ar, const TPtr t){
        register_type(ar, * t);
        if(NULL == t){
            basic_oarchive & boa 
                = boost::serialization::smart_cast_reference<basic_oarchive &>(ar);
            boa.save_null_pointer();
            save_access::end_preamble(ar);
            return;
        }
        save(ar, * t);
    }

Modifying the code to this, works as expected (although there's still a dereference of a null):

    template<class TPtr>
    static void invoke(Archive &ar, const TPtr t){
        if(NULL == t){
            register_type(ar, * t);
            basic_oarchive & boa
                = boost::serialization::smart_cast_reference<basic_oarchive &>(ar);
            boa.save_null_pointer();
            save_access::end_preamble(ar);
            return;
        } else {
            register_type(ar, * t);
        }
        save(ar, * t);
    }

Change History (1)

comment:1 by anonymous, 5 years ago

Component: Noneserialization
Owner: set to Robert Ramey
Version: Boost 1.63.0Boost Development Trunk
Note: See TracTickets for help on using tickets.