Changes between Version 2 and Version 3 of Guidelines/Separate


Ignore:
Timestamp:
Aug 18, 2010, 10:57:03 PM (12 years ago)
Author:
Daniel James
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Guidelines/Separate

    v2 v3  
    1 = Guidelines for Authors of Boost Libraries Containing Separate Source = #intro
     1= Guidelines for Authors of Boost Libraries Containing Separate Source =
    22
    3 These guidelines are designed for the authors of Boost
    4 libraries which have separate source that need compiling in
    5 order to use the library. Throughout, this guide refers to a
    6 fictitious "whatever" library, so replace all occurrences of
    7 "whatever" or "WHATEVER" with your own library's name when
    8 copying the examples.
    9 
    10 == Changes Affecting Source Code == #source_changes
    11 
    12 === Preventing Compiler ABI Clashes === #abi
    13 
    14 There are some compilers (mostly Microsoft Windows compilers
    15 again!), which feature a range of compiler switches that alter
    16 the ABI of C++ classes and functions. By way of example,
    17 consider Borland's compiler which has the following
    18 options:
    19 {{{
    20 -b    (on or off - effects enum sizes).
    21 -Vx   (on or off - empty members).
    22 -Ve   (on or off - empty base classes).
    23 -aX   (alignment - 5 options).
    24 -pX   (Calling convention - 4 options).
    25 -VmX  (member pointer size and layout - 5 options).
    26 -VC   (on or off, changes name mangling).
    27 -Vl   (on or off, changes struct layout).
    28 }}}
    29 
    30 These options are provided in addition to those affecting
    31 which runtime library is used (more on which later); the total
    32 number of combinations of options can be obtained by
    33 multiplying together the individual options above, so that
    34 gives 2*2*2*5*4*5*2*2 = 3200 combinations!
    35 
    36 The problem is that users often expect to be able to build
    37 the Boost libraries and then just link to them and have
    38 everything just plain work, no matter what their project
    39 settings are. Irrespective of whether this is a reasonable
    40 expectation or not, without some means of managing this issue,
    41 the user may well find that their program will experience
    42 strange and hard to track down crashes at runtime unless the
    43 library they link to was built with the same options as their
    44 project (changes to the default alignment setting are a prime
    45 culprit). One way to manage this is with "prefix and suffix"
    46 headers: these headers invoke compiler specific #pragma
    47 directives to instruct the compiler that whatever code follows
    48 was built (or is to be built) with a specific set of compiler
    49 ABI settings.
    50 
    51 Boost.config provides the macro `BOOST_HAS_ABI_HEADERS` which
    52 is set whenever there are prefix and suffix headers available
    53 for the compiler in use, typical usage in a header like
    54 this:
    55 {{{
    56 #ifndef BOOST_WHATEVER_HPP
    57 #define BOOST_WHATEVER_HPP
    58 
    59 #include <boost/config.hpp>
    60 
    61 // this must occur after all of the includes and before any code appears:
    62 #ifdef BOOST_HAS_ABI_HEADERS
    63 #  include BOOST_ABI_PREFIX
    64 #endif
    65 //
    66 // this header declares one class, and one function by way of examples:
    67 //
    68 class whatever
    69 {
    70    // details.
    71 };
    72 
    73 whatever get_whatever();
    74 
    75 // the suffix header occurs after all of our code:
    76 #ifdef BOOST_HAS_ABI_HEADERS
    77 #  include BOOST_ABI_SUFFIX
    78 #endif
    79 
    80 #endif
    81 }}}
    82 
    83 You can include this code in your library source files as
    84 well if you want, although you probably shouldn't need to:
    85 
    86  * If you ''don't'' use these in the library source
    87    files (but do in your library's headers) and the user
    88    attempts to compile the library source with a non-default ABI
    89    setting, then they will get compiler errors if there are any
    90    conflicts.
    91  * If you ''do'' include them in both the library's
    92    headers and the library source files, then the code should
    93    always compile no matter what the compiler settings used,
    94    although the result might not match what the user was
    95    expecting: since we've forced the ABI back into default
    96    mode.
    97 
    98 ==== Rationale: ==== #abi.rationale
    99 
    100 Without some means of managing this issue, users often
    101 report bugs along the line of "Your silly library always
    102 crashes when I try and call it" and so on. These issues can be
    103 extremely difficult and time consuming to track down, only to
    104 discover in the end that it's a compiler setting that's changed
    105 the ABI of the class and/or function types of the program
    106 compared to those in the pre-compiled library. The use of
    107 prefix/suffix headers can minimize this problem, although
    108 probably not remove it completely.
    109 
    110 ===== Counter Argument #1: ===== #abi.counter1
    111 
    112 Trust the user, if they want 13-byte alignment (!) let them
    113 have it.
    114 
    115 ===== Counter Argument #2: ===== #abi.counter2
    116 
    117 Prefix/suffix headers have a tendency to "spread" to other
    118 boost libraries - for example if boost::shared_ptr<>
    119 forms part of your class's ABI, then including prefix/suffix
    120 headers in your code will be of no use unless shared_ptr.hpp
    121 also uses them. Authors of header-only boost libraries may not
    122 be so keen on this solution - with some justification - since
    123 they don't face the same problem.
    124 
    125 === Static or Dynamic Libraries === #static_or_dynamic
    126 
    127 When the users runtime is dynamically linked the Boost
    128 libraries can be built either as dynamic libraries (.so's on
    129 Unix platforms, .dll's on Windows) or as static libraries (.a's
    130 on Unix, .lib's on Windows). So we have a choice as to which is
    131 supported by default:
    132 
    133  * On Unix platforms it typically makes no difference to the
    134    code: the user just selects in their makesfile which library
    135    they prefer to link to.
    136  * On Windows platforms, the code has to be specially
    137    annotated to support DLL's, so we need to pick one option as
    138    the default and one as an alternative.
    139  * On Windows platforms, we can inject special code to
    140    automatically select which library variant to link against:
    141    so again we need to decide which is to be the default (see
    142    the section on auto-linking below).
    143 
    144 The recomendation is to pick static linking by default.
    145 
    146 ==== Rationale: ==== #static-or-dynamic.rationale
    147 
    148 There is no one policy that fits all here.
    149 
    150 The rationale for the current behaviour was inherited from
    151 Boost.Regex (and it's ancestor regex++): this library
    152 originally used dynamic linking by default whenever the runtime
    153 was dynamic. It's actually safer that way should you be using
    154 regex from a dll for example. However, this behavior brought a
    155 persistent stream of user complaints: mainly about deployment,
    156 all asking if static linking could be the default. After regex
    157 changed behavior the complaints stopped, and the author hasn't
    158 had one complaint about static linking by default being the
    159 wrong choice.
    160 
    161 Note that other libraries might need to make other choices:
    162 for example libraries that are intended to be used to implement
    163 dll pluggin's would like need to use dynamic linking in almost
    164 all cases.
    165 
    166 === Supporting Windows Dll's === #windows-dlls
    167 
    168 On most Unix-like platforms no special annotations of source
    169 code are required in order for that source to be compiled as a
    170 shared library because all external symbols are exposed.
    171 However the majority of Windows compilers require that symbols
    172 that are to be imported or exported from a dll, be prefixed
    173 with `__declspec(dllimport)` or `__declspec(dllexport)`. Without
    174 this mangling of source code, it is not possible to correctly
    175 build shared libraries on Windows (historical note - originally
    176 these declaration modifiers were required on 16-bit Windows
    177 where the memory layout for exported classes was different from
    178 that of "local" classes - although this is no longer an issue,
    179 there is still no way to instruct the linker to "export
    180 everything", it also remains to be seen whether 64-bit Windows
    181 will resurrect the segmented architecture that led to this
    182 problem in the first place. Note also that the mangled names of
    183 exported symbols are different from non-exported ones, so
    184 `__declspec(dllimport)` is required in order to link to code
    185 within a dll).
    186 
    187 In order to support the building of shared libraries on MS
    188 Windows your code will have to prefix all the symbols that your
    189 library exports with a macro (lets call it `BOOST_WHATEVER_DECL`)
    190 that your library will define to expand to either
    191 `__declspec(dllexport)` or `__declspec(dllimport)` or nothing,
    192 depending upon how your library is being built or used. Typical
    193 usage would look like this:
    194 {{{
    195 #ifndef BOOST_WHATEVER_HPP
    196 #define BOOST_WHATEVER_HPP
    197 
    198 #include <boost/config.hpp>
    199 
    200 #ifdef BOOST_HAS_DECLSPEC // defined in config system
    201 // we need to import/export our code only if the user has specifically
    202 // asked for it by defining either BOOST_ALL_DYN_LINK if they want all boost
    203 // libraries to be dynamically linked, or BOOST_WHATEVER_DYN_LINK
    204 // if they want just this one to be dynamically liked:
    205 #if defined(BOOST_ALL_DYN_LINK) || defined(BOOST_WHATEVER_DYN_LINK)
    206 // export if this is our own source, otherwise import:
    207 #ifdef BOOST_WHATEVER_SOURCE
    208 # define BOOST_WHATEVER_DECL __declspec(dllexport)
    209 #else
    210 # define BOOST_WHATEVER_DECL __declspec(dllimport)
    211 #endif  // BOOST_WHATEVER_SOURCE
    212 #endif  // DYN_LINK
    213 #endif  // BOOST_HAS_DECLSPEC
    214 //
    215 // if BOOST_WHATEVER_DECL isn't defined yet define it now:
    216 #ifndef BOOST_WHATEVER_DECL
    217 #define BOOST_WHATEVER_DECL
    218 #endif
    219 
    220 //
    221 // this header declares one class, and one function by way of examples:
    222 //
    223 class BOOST_WHATEVER_DECL whatever
    224 {
    225    // details.
    226 };
    227 
    228 BOOST_WHATEVER_DECL whatever get_whatever();
    229 
    230 #endif
    231 }}}
    232 
    233 And then in the source code for this library one would use:
    234 
    235 {{{
    236  
    237 //
    238 // define BOOST_WHATEVER SOURCE so that our library's
    239 // setup code knows that we are building the library (possibly exporting code),
    240 // rather than using it (possibly importing code):
    241 //
    242 #define BOOST_WHATEVER_SOURCE
    243 #include <boost/whatever.hpp>
    244 
    245 // class members don't need any further annotation:
    246 whatever::whatever() { }
    247 // but functions do:
    248 BOOST_WHATEVER_DECL whatever get_whatever()
    249 {
    250    return whatever();
    251 }
    252 }}}
    253 
    254 ==== Importing/exporting dependencies ==== #importing-exporting
    255 
    256 As well as exporting your main classes and functions (those
    257 that are actually documented), Microsoft Visual C++ will warn
    258 loudly and often if you try to import/export a class whose
    259 dependencies are not also exported. Dependencies include: any
    260 base classes, any user defined types used as data members, plus
    261 all of the dependencies of your dependencies and so on. This
    262 causes particular problems when a dependency is a template
    263 class, because although it is technically possible to export
    264 these, it is not at all easy, especially if the template itself
    265 has dependencies which are implementation-specific details. In
    266 most cases it's probably better to simply suppress the warnings
    267 using:
    268 
    269 {{{
    270 #ifdef BOOST_MSVC
    271 #  pragma warning(push)
    272 #  pragma warning(disable : 4251 4231 4660)
    273 #endif
    274 
    275 // code here
    276 
    277 #ifdef BOOST_MSVC
    278 #pragma warning(pop)
    279 #endif
    280 }}}
    281 
    282 This is safe provided that there are no dependencies that
    283 are (template) classes with non-constant static data members,
    284 these really do need exporting, otherwise there will be
    285 multiple copies of the static data members in the program, and
    286 that's really really bad.
    287 
    288 Historical note: on 16-bit Windows you really did have to
    289 export all dependencies or the code wouldn't work, however
    290 since the latest Visual Studio .NET supports the import/export
    291 of individual member functions, it's a reasonably safe bet that
    292 Windows compilers won't do anything nasty - like changing the
    293 class's ABI - when importing/exporting a class.
    294 
    295 ==== Rationale: ==== #importing-exporting.rationale
    296 
    297 ''Why bother - doesn't the import/export mechanism take up
    298 more code that the classes themselves?''
    299 
    300 A good point, and probably true, however there are some
    301 circumstances where library code must be placed in a shared
    302 library - for example when the application consists of multiple
    303 dll's as well as the executable, and more than one those dll's
    304 link to the same Boost library - in this case if the library
    305 isn't dynamically linked and it contains any global data (even
    306 if that data is private to the internals of the library) then
    307 really bad things can happen - even without global data, we
    308 will still get a code bloating effect. Incidentally, for larger
    309 applications, splitting the application into multiple dll's can
    310 be highly advantageous - by using Microsoft's "delay load"
    311 feature the application will load only those parts it really
    312 needs at any one time, giving the impression of a much more
    313 responsive and faster-loading application.
    314 
    315 ''Why static linking by default?''
    316 
    317 In the worked example above, the code assumes that the
    318 library will be statically linked unless the user asks
    319 otherwise. Most users seem to prefer this (there are no
    320 separate dll's to distribute, and the overall distribution size
    321 is often significantly smaller this way as well: i.e. you pay
    322 for what you use and no more), but this is a subjective call,
    323 and some libraries may even only be available in dynamic
    324 versions (Boost.threads for example).
    325 
    326 === Automatic Library Selection and Linking with [http://www.boost.org/doc/libs/release/boost/config/auto_link.hpp auto_link.hpp] === #auto-link
    327 
    328 Many Windows compilers ship with multiple runtime libraries
    329 - for example Microsoft Visual Studio .NET comes with 6
    330 versions of the C and C++ runtime. It is essential that the
    331 Boost library that the user links to is built against the same
    332 C runtime as the program is built against. If that is not the
    333 case, then the user will experience linker errors at best, and
    334 runtime crashes at worst. The Boost build system manages this
    335 by providing different build variants, each of which is build
    336 against a different runtime, and gets a slightly different
    337 mangled name depending upon which runtime it is built against.
    338 For example the regex libraries get named as follows when built
    339 with Visual Studio .NET 2003:
    340 
    341 {{{
    342 boost_regex-vc71-mt-1_31.lib
    343 boost_regex-vc71-mt-gd-1_31.lib
    344 libboost_regex-vc71-mt-1_31.lib
    345 libboost_regex-vc71-mt-gd-1_31.lib
    346 libboost_regex-vc71-mt-s-1_31.lib
    347 libboost_regex-vc71-mt-sgd-1_31.lib
    348 libboost_regex-vc71-s-1_31.lib
    349 libboost_regex-vc71-sgd-1_31.lib
    350 }}}
    351 
    352 The difficulty now is selecting which of these the user
    353 should link his or her code to.
    354 
    355 In contrast, most Unix compilers typically only have one
    356 runtime (or sometimes two if there is a separate thread safe
    357 option). For these systems the only choice in selecting the
    358 right library variant is whether they want debugging info, and
    359 possibly thread safety.
    360 
    361 Historically Microsoft Windows compilers have managed this
    362 issue by providing a #pragma option that allows the header for
    363 a library to automatically select the library to link to. This
    364 makes everything automatic and extremely easy for the end user:
    365 as soon as they include a header file that has separate source
    366 code, the name of the right library build variant gets embedded
    367 in the object file, and as long as that library is in the
    368 linker search path, it will get pulled in by the linker without
    369 any user intervention.
    370 
    371 Automatic library selection and linking can be enabled for a
    372 Boost library by including the header
    373 <boost/config/auto_link.hpp>, after first defining
    374 BOOST_LIB_NAME and, if applicable, BOOST_DYN_LINK.
    375 
    376 {{{
    377 //
    378 // Automatically link to the correct build variant where possible.
    379 //
    380 #if !defined(BOOST_ALL_NO_LIB) && !defined(BOOST_WHATEVER_NO_LIB) && !defined(BOOST_WHATEVER_SOURCE)
    381 //
    382 // Set the name of our library, this will get undef'ed by auto_link.hpp
    383 // once it's done with it:
    384 //
    385 #define BOOST_LIB_NAME boost_whatever
    386 //
    387 // If we're importing code from a dll, then tell auto_link.hpp about it:
    388 //
    389 #if defined(BOOST_ALL_DYN_LINK) || defined(BOOST_WHATEVER_DYN_LINK)
    390 #  define BOOST_DYN_LINK
    391 #endif
    392 //
    393 // And include the header that does the work:
    394 //
    395 #include <boost/config/auto_link.hpp>
    396 #endif  // auto-linking disabled
    397 }}}
    398 
    399 The library's user documentation should note that the
    400 feature can be disabled by defining either BOOST_ALL_NO_LIB or
    401 BOOST_WHATEVER_NO_LIB:
    402 
    403 If for any reason you need to debug this feature, the header
    404 <boost/config/auto_link.hpp> will output some helpful
    405 diagnostic messages if you first define
    406 BOOST_LIB_DIAGNOSTIC.
    407 
    408 == Changes Affecting the Build System == #build_changes
    409 
    410 === Creating the library Jamfile === #jamfile
    411 
    412 The Jamfile for building library "whatever" typically lives
    413 in boost-root/libs/whatever/build, the only extra step required
    414 is to add a <define> requirement to the library target so
    415 that your code knows whether it's building a dll or static
    416 library, a typical Jamfile would like like this:
    417 
    418 {{{
    419 lib boost_regex : ../src/whatever.cpp :
    420   <link>shared:<define>BOOST_WHATEVER_DYN_LINK=1 ;
    421 }}}
    422 
    423 === Testing Auto-linking === #testing
    424 
    425 Testing the auto-link feature is somewhat convoluted, and
    426 requires access to a compiler that supports the feature: refer
    427 to
    428 [http://www.boost.org/doc/libs/release/libs/config/test/link/test/Jamfile.v2 libs/config/test/link/test/Jamfile.v2]
    429 for an example.
    430 
    431 Copyright John Maddock 1998.
     3This has been move back to the [http://www.boost.org/development/separate_compilation.html main site]