| 1521 | | If the last section was somewhat speculative due to the present uncertainties about the future developments in C++, these remaining two sections are almost entirely discussion pieces as they have no known good answers. Consider them therefore more as food for thought rather than recommendations. I'll make the following discussion entirely Boost centric, but lots of corporations out there (Google is the most famous example) have truly enormous monolithic C++ codebases where this discussion equally applies if not far more so, so please consider find and replacing all mentions of Boost with ''<insert your large C++ code base here>''. |
| 1522 | | |
| 1523 | | A very frequently heard request on boost-dev and boost-users is for Boost to become "more modular", where modularity can mean quite a few different things depending on the claimant. Here are some of the more common meanings of "modularity": |
| 1524 | | |
| 1525 | | 1. I would like Boost (as a whole) to have fewer system requirements (minimum compiler versions, minimum OS support etc). This is usually really an argument in favour of better support for either legacy compilers OR embedded or games systems, but do note the substantial distinction for later. |
| 1526 | | |
| 1527 | | 2. I would like my favourite Boost libraries to have fewer requirements on their dependencies including other Boost libraries, especially exact version requirements (i.e. I want my favourite Boost library to detect and work with multiple versions of Boost). This is usually an argument from those who experience problems with things mildly breaking in different places in each Boost release, and they end up having to mash up their own Boost distro made up of newer and older individual Boost libraries to get the stability they need. |
| 1528 | | |
| 1529 | | 3. I would like my favourite Boost libraries to optionally not have the word "Boost" in their names as my employer's legal department have explicitly banned all Boost libraries now and forever in perpetuity. |
| 1530 | | |
| 1531 | | 4. I would like to download my favourite Boost libraries and only their strict dependencies within Boost without having to download or even consider during build or configuration any unnecessary other Boost libraries (i.e. the package manager argument). |
| 1532 | | |
| 1533 | | 5. I would like to use my favourite Boost library using the Standard C++ Library facilities that come with C++ 11 instead of being forced into using Boost near equivalents (i.e. I get annoyed dealing with mixes of `std::future` and `boost::future`). |
| 1534 | | |
| 1535 | | 6. I would like to drop my favourite Boost library/libraries into my project as a single giant include file with no need to worry about Boost.Build or any build system or even dealing with a Boost source control system. |
| | 1521 | If the last section was somewhat speculative due to the present uncertainties about the future developments in C++, these remaining two sections are almost entirely discussion pieces as they have no known good answers. Consider them therefore more as food for thought rather than recommendations, and they are obviously my (Niall Douglas) personal perspective and interpretation on things. I'll make the following discussion entirely Boost centric, but lots of corporations out there (Google is the most famous example) have truly enormous monolithic C++ codebases where this discussion equally applies if not far more so, so please consider find and replacing all mentions of Boost with ''<insert your large C++ code base here>''. |
| | 1522 | |
| | 1523 | === A brief and cynical history of Boost and its relationship to C++ === |
| | 1524 | |
| | 1525 | Boost has been experiencing in recent years a number of problems surrounding its scalability as it grows and especially as it matures (specifically, maintenance and cultural fear of any substantial change). Historically speaking, [https://web.archive.org/web/19991011120524/http://www.boost.org/ 1999 Boost] was first [http://www.boost.org/users/proposal.pdf a proving ground for new C++ standard library ideas] making better use of the 1998 C++ ISO standard, but from [https://web.archive.org/web/20000816132250/http://www.boost.org/libs/compiler_status.htm August 2000 onwards] it also began to turn into a broken compiler workarounds layer which whilst initially was great for those with broken compilers, it incurred an enormous technical debt into the codebase which still weighs heavily upon anyone trying to change anything substantial affecting more than one library, including cultural beliefs in the importance of support of broken toolsets. Due to its roots as a proving ground for standard C++ libraries, Boost is unusually similar to the C++ standard library which is both good and bad: good in terms of the quality of design, implementation and testing - bad in terms of monolithicity, poor coupling management, and over-reliance on a single person in charge of each library (great for the library if that maintainer is active, not so great across libraries when maintainers must work together, terrible if a maintainer vanishes or departs). |
| | 1526 | |
| | 1527 | Boost's "high point" was probably between 2001 [https://web.archive.org/web/20010614021213/http://www.boost.org/index.htm when the modern peer review process was formalised] (and has remained remarkably, perhaps unfortunately, unchanged since) when it had thirty three libraries and 2005 [https://web.archive.org/web/20050815003310/http://www.boost.org/ when the Boost Software Licence was completed, most of the famous Boost libraries had reached their final designs, and ten Boost libraries were accepted into the C++ TR1 standard] when it had sixty-nine libraries. |
| | 1528 | |
| | 1529 | Some may be surprised at the choice of 2005 given that BoostCon began in 2006, but if you examine the Wayback Machine for boost.org from 2005 onwards you will see a plateau was reached from 2005 onwards, and I'm going to claim that this is because Boost had achieved its original mission for the first time and had therefore taken its first step towards obsoleting itself. The push was now on for the C++ 0x ISO standard, and I think the period 2007-2008 was crucial because in my opinion it was during this period that many of the Boost old timers began to become bitter with the C++ library process as their items began to be cut for the first initial draft of what would become C++ 11 for purely political reasons. The big problem was that if you proposed some library X for standardisation, someone on ISO would say "library X would look completely different once expected new language feature Y is in the language, therefore library X shouldn't enter the standard right now". The most damaging expected new language feature Y was undoubtedly original C++ concepts as that killed off entire tracts of exciting C++ library standardisation, much to the often bitterness of those who had invested months to years of their spare time developing those libraries. After all, most new language features are developed by highly paid employees where it is their day jobs, whereas of the twenty or so Boost libraries in C++ 11 were generally developed in the family time of enthusiasts who earned a fraction of the pay of those killing off their work, and to keep a chipper attitude whilst those who have not sacrificed shoot down often years of your work is not easy. |
| | 1530 | |
| | 1531 | 2008 was the last time the Boost web site was in any way improved, and even before that stale content was no longer being removed or repaired as it once used to be, thus making the Boost web presence increasingly less authoritative and less a "one stop shop" for answers. We then began to see a slow exodus of Boost old timers, some leaving C++ completely for good for employment in large corporations which frown on employees having any interests outside the corporation, some disavowing Boost forever and having anything to do with Boost, and some no longer trying to get their libraries into Boost (i.e. past any form of review process) and preferring to house their C++ libraries elsewhere and specifically away from the Boost community. A too frequent common factor this author has noticed is the bitterness and anger regarding Boost and its community in some of the big names in Boost who were the minds behind some of its most successful libraries. The peer review process ground to a halt from the end of 2012 onwards, and [https://www.youtube.com/watch?v=8TkCwizP14Y by early 2014 a noticeable increase was obvious in the rate of decline of Boost] since the 2011 ISO C++ standard began to be implemented in available toolchains. |
| | 1532 | |
| | 1533 | Since 2014 measures have been taken to reinvigorate Boost by a number of actors including myself, but those measures are not germane to this essay. What I will say is that by Summer 2015, by the strictest measure of these things, perhaps as many as sixty of the one hundred and thirty or so Boost libraries could be considered undermaintained or not maintained. That suggests that Boost, under its current set of procedures, culture, management and infrastructure, is struggling to scale past about seventy libraries -- or put another way, the malaise and staleness within Boost started around the same time as the number of libraries in Boost passed about seventy in 2005. The scaling limit being around seventy is of particular significance because that is exactly around where the human cognitive limit for physically separated group sizes lands, and indeed military tactical units have been organised around groups of sixty to eighty soldiers since the Romans simply because trial and error found that number worked best. ''I am therefore going to assert, rather than claim, that under a volunteer based system where one individual is the maintainer of each library that Boost cannot healthily exceed about seventy libraries (i.e. seventy maintainers) as a single collection'', and you will find all my efforts to reinvigorate Boost - including this Handbook - revolve around that assertion of there being a hard scaling limit to each collection. |
| | 1534 | |
| | 1535 | |
| | 1536 | === What does this history have to do with defaulting to standalone capable modular C++ libraries? === |
| | 1537 | |
| | 1538 | The obvious answer is that if a library is capable of coexisting inside multiple collections of libraries including the collection of just itself alone, that enormously increases the scope and opportunity for working around the size limit of seventy. You could, as I have often proposed, have a v1.x collection of legacy C++ 03 Boost libraries and a v2.x collection of C++ 14 Boost libraries where there is some overlap between the collections in that some C++ 03 Boost libraries are available in both collections. One therefore raises the scaling limit from about seventy to potentially one hundred and twenty or so for two collections without suffering from undermaintenance, malaise or staleness as an organisation. If your library defaults to being standalone capable and modular, that ''enormously'' improves your ability to have your library coexist in multiple collections of libraries. [https://github.com/boostcon/cppnow_presentations_2015/raw/master/files/Large-Projects-and-CMake-and-git-oh-my.pdf You therefore are writing ''social code'', not ''a-social code'']. |
| | 1539 | |
| | 1540 | However, let's assume you don't buy the hard scaling limit to a collection size and instead have some more practical use scalability problems such as: |
| | 1541 | |
| | 1542 | * I would like Boost (as a whole) to have fewer system requirements (minimum compiler versions, minimum OS support etc). This is usually really an argument in favour of better support for either legacy compilers OR embedded or games systems, but do note the substantial distinction for later. |
| | 1543 | |
| | 1544 | * I would like my favourite Boost libraries to have fewer requirements on their dependencies including other Boost libraries, especially exact version requirements (i.e. I want my favourite Boost library to detect and work with multiple versions of Boost). This is usually an argument from those who experience problems with things mildly breaking in different places in each Boost release, and they end up having to mash up their own Boost distro made up of newer and older individual Boost libraries to get the stability they need. |
| | 1545 | |
| | 1546 | * I would like my favourite Boost libraries to optionally not have the word "Boost" in their names as my [https://stackoverflow.com/questions/755439/boost-is-just-great-and-free-is-there-a-catch employer has explicitly banned all libraries with Boost in their name now and forever in perpetuity, requiring me to duplicate Boost code which passes the same Boost unit test suite as the original]. |
| | 1547 | |
| | 1548 | * I would like to download my favourite Boost libraries and only their strict dependencies within Boost without having to download or even consider during build or configuration any unnecessary other Boost libraries (i.e. the package manager argument). |
| | 1549 | |
| | 1550 | * I would like to use my favourite Boost library using the Standard C++ Library facilities that come with C++ 11 instead of being forced into using Boost near equivalents (i.e. I get annoyed dealing with mixes of `std::future` and `boost::future`). |
| | 1551 | |
| | 1552 | * I would like to drop my favourite Boost library/libraries into my project as a single giant include file with no need to worry about Boost.Build or any build system or even dealing with a Boost source control system. |
| | 1553 | |
| | 1554 | As much as these often cited user problems might seem unrelated, they are in fact all due to the lack of: |
| | 1555 | |
| | 1556 | 1. '''Modularity''' |
| | 1557 | |
| | 1558 | Your users can download your library as a self contained distribution, and get immediately to work. The ideal of this form is a single standalone include file which contains everything the library user needs. |
| | 1559 | |
| | 1560 | 2. '''Encapsulation''' |
| | 1561 | |
| | 1562 | Thinking about and specifying your dependencies properly instead of just firing in some reference to `boost::lib::foo` and dragging in a whole library just for a single routine. [http://arxiv.org/abs/1405.3323 C++ has also spectacularly failed to improve on Microsoft COM as the best available technology for fully encapsulating C++ libraries unfortunately], and as much as such work is "unsexy" the fact a proper C++ component and modularisation system doesn't have [https://isocpp.org/std/the-committee a study group at ISO WG21] is an appalling tragedy given the [https://en.wikipedia.org/wiki/Service-orientation enormous and very well understood productivity gains] you get from such a technology. |
| | 1563 | |
| | 1564 | 3. '''Low cost trial''' |
| | 1565 | |
| | 1566 | From a library user perspective, the biggest overwhelming incentive is usually to adopt "[https://en.wikipedia.org/wiki/Not_invented_here Not Invented Here]" as that best ensures continued employment at least in the medium term. So as a library author, if you want anyone to use your library you need to eliminate as many reasons for a library user to find an excuse not to use your library as possible. One of the biggest excuses users will have is simply that of the cost of trying out your library for fit to solving a problem: is installation ready for development a single action? Does including this library quadruple my build times? Does code using this library ever crash, including the compiler? Do I have to click the mouse more than three times to find something in the documentation? If the answer is yes to any of those questions, your library does not have a low cost trial curve, and could even be ''a-social coding''. |
| | 1567 | |
| | 1568 | 4. '''Forcing choices onto the library user unnecessarily''' |
| | 1569 | |
| | 1570 | When your library insists on only ever using `boost::future` and being incapable of using `std::future`, you force your library users to write boilerplate to convert between types of future because another library dependency insists on only ever using `std::future`. There is no good reason whatsoever why your library forces that choice on its users, except your laziness in not upgrading your library to make better use of the C++ 11 standard library. |
| | 1571 | |
| | 1572 | Most of the earlier problems go away if you address these four issues, but you'll note that dependency management is essentially being statically done by hand by the library maintainer due to that lack of a modern equivalent to Microsoft COM I mentioned. The wisdom of dependency package managers in C++ 11/14 is exactly what I'll write about next. |
| | 1573 | |
| | 1574 | == 19. COUPLING/SOAPBOX: Essay about wisdom of dependency package managers in C++ 11/14 == |
| 1539 | | I'm going to argue in favour of defaulting your C++ 11/14 libraries to use no external dependencies i.e. any C++ headers not in your git repository and not in the STL, which includes Boost. External dependencies do ''''not'''' include any git submodules you may have in your git repository, so if your user types: |
| 1540 | | |
| 1541 | | {{{ |
| 1542 | | git clone http://yourlib |
| 1543 | | cd yourlib |
| 1544 | | git submodule update --init --recursive |
| 1545 | | cmake |
| 1546 | | }}} |
| 1547 | | |
| 1548 | | ... then they have a complete, ready to build source code tree with no additional configuration required. |
| 1549 | | |
| 1550 | | The advantages of defaulting to no external dependencies are obvious: |
| 1551 | | |
| 1552 | | 1. Modularity: your users can download your library as a self contained distribution, and get immediately to work. [https://github.com/boostcon/cppnow_presentations_2015/raw/master/files/Large-Projects-and-CMake-and-git-oh-my.pdf As David Sankel would put it, this is being not anti-social in your coding]. |
| 1553 | | 2. Encapsulation: it forces you to think about your dependencies properly instead of just firing in some `boost::lib::foo` and dragging in a whole library just for a single routine. We did far too much of that historically in the Boost libraries, indeed even putting everything into the `boost` namespace rather than its own sub-namespace. |
| 1554 | | 3. Build times: it lets you assemble all the header files in your library into a giant single include file which can then be distributed as a self standing single file drop in for your users, and which can be precompiled into a precompiled header such that your users experience greatly reduced build times. |
| 1555 | | 4. Defaulting to using STL primitives such as `std::future` and `std::function` instead of alternatives greatly eases the lives of your users when they are trying to combine your library with another library in their application. One of the more tedious things in life is constantly having to invoke `std::async` to get `boost::future` and `std::future` to work together :(. |
| 1556 | | |
| 1557 | | TODO |
| 1558 | | |
| 1559 | | monolithic vs modular |
| 1560 | | git submodules vs biicode |
| 1561 | | |
| 1562 | | == 19. COUPLING/SOAPBOX: Essay about wisdom of dependency package managers in C++ 11/14 == |
| | 1578 | git submodules |
| | 1579 | biicode |
| | 1580 | etc |
