| 1563 | | The younger programmer tends to see a career in the tech industry for what it is: a lousy industry where employees are disposable resources to be mined for all their value before disposal, pensions are probably worthless assuming you'll ever retire at a reasonable age, real estate pricing is far beyond your means and you're still saddled with enormous debts from gaining all those degrees, and the only advantage of being in tech as compared to other industries is that it's far worse in most other industries available to your age cohort -- assuming you can get a job at all despite holding multiple Masters degrees and being vastly more qualified than the people interviewing you. For many a younger programmer, contributing to open source means something far different to the older generation: ''an opportunity to escape'' the meaningless existence of writing pointless code for other people who only care about you insofar as to make you drink their Koolaid and place their company before everything else in your life, including your family. And by escape, I don't just mean merely psychologically, I mean that most have a vision that one day their open source efforts could turn into a sufficient means for living away from those who exploit you, whether self employed via a startup or remote expert consulting for some crazy hourly rate at which point those same earlier employers tend to suddenly value your time and inconvenience oddly enough. |
| 1564 | | |
| 1565 | | This is obviously a very different experience and understanding of open source software. Not coincidentally most of the big open source software projects invest heavily in the business side of acquiring and dispersing funding for vital work on the software, and this trend of seeing the open source software organisation as primarily one of actualising its business side instead of being just some central source code repo and web presence is the most prevalent in the newest open source projects simply through them being founded more recently, and therefore adopting what they felt was the state of the art at the time. |
| 1566 | | |
| 1567 | | In case you are one of those engineers of a certain age and don't know what I'm talking about here, let me compare two open source projects: [http://www.boost.org/ the Boost C++ Libraries] and [https://www.djangoproject.com/ the Django web framework]. Boost was [http://www.boost.org/users/proposal.pdf originally conceived as a proving ground for new C++ standard library ideas] making better use of the 1998 C++ ISO standard. Due to its roots as a playpen and proving ground for standard C++ libraries, Boost is unusually similar to the C++ standard library which is both good and bad: good in terms of the quality of design, algorithms, implementation and testing -- bad in terms of monolithicity, poor coupling management, and over-reliance on a single person in charge of each library (great for the library if that maintainer is active, not so great across libraries when maintainers must work together, terrible if a maintainer vanishes or departs). In particular, like many other open source projects founded in the 1990s, Boost does not believe in there being a business side to itself apart from its annual conferences which began in 2006, and [https://sites.google.com/a/boost.org/steering/ its steering committee website] has this excellent paragraph which was written after I caused a fuss about how little steering the "steering committee" does: |
| | 1563 | The younger programmer tends to see a career in the tech industry for what it is: a lousy industry where employees are disposable resources to be mined for all their value before disposal, pensions are probably worthless assuming you'll ever retire at a reasonable age, real estate pricing is far beyond your means and you're still saddled with enormous debts from gaining all those degrees, and the only advantage of being in tech as compared to other industries is that it's far worse in most other industries available to your age cohort -- assuming you can get a job at all related to your qualifications despite holding multiple Masters degrees and being vastly more qualified, and sometimes more able, than the people interviewing you. This is obviously leads to a '''very''' different experience and understanding of open source software. Part of it is due to the inherant cultural differences and world view of [https://en.wikipedia.org/wiki/Millennials the Millennials] versus [https://en.wikipedia.org/wiki/Baby_boomers the Baby Boomers], but it is also due the widespread understanding that today's younger programmers will likely not retire until far beyond the present retirement age, and with very substantially fewer assets and income. That profoundly adjusts the cost benefit of serving your time and waiting your turn over the baby boomer generation, never mind working sixty plus hour weeks with two week's vacation until then. |
| | 1564 | |
| | 1565 | It's different for each of course, but for many a younger programmer including myself contributing to open source means something far different to the older generation: ''an opportunity to escape'' the meaningless existence of writing pointless code for other people who only care about you insofar as to make you drink their Koolaid and place their company's vision before everything else in your life, including your family, and will dispose of you as a cost to be reduced as soon as they are able. And by escape, I don't just mean merely psychologically, I mean that most have a vision that one day their open source efforts could turn into a sufficient means for living away from those who would exploit you, whether self employed via a startup or remote expert consulting for some solid hourly rate at which point those same earlier employers tend to suddenly value your time and inconvenience oddly enough, and don't keep creating the situations where you must constantly work overtime for no extra income. |
| | 1566 | |
| | 1567 | I suspect I am not alone in this world view. Not coincidentally most of the big and especially recent open source software projects invest heavily in the business side of acquiring and dispersing funding for vital work on the software, and this trend of seeing the open source software organisation as primarily one of actualising its business side instead of being just some central source code repo and web presence is the most prevalent in the newest open source projects simply through them being founded more recently, and therefore adopting what they felt was the state of the art at the time. |
| | 1568 | |
| | 1569 | In case you are one of those engineers of a certain age and think I'm talking out my arse here, let me compare two open source projects: [http://www.boost.org/ the Boost C++ Libraries] and [https://www.djangoproject.com/ the Django web framework]. Boost was [http://www.boost.org/users/proposal.pdf originally conceived as a proving ground for new C++ standard library ideas] making better use of the 1998 C++ ISO standard. Due to its roots as a playpen and proving ground for standard C++ libraries, Boost is unusually similar to the C++ standard library which is both good and bad: good in terms of the quality of design, algorithms, implementation and testing -- bad in terms of monolithicity, poor coupling management, and over-reliance on a single person in charge of each library (great for the library if that maintainer is active, not so great across libraries when maintainers must work together, terrible if a maintainer vanishes or departs). In particular, like many other open source projects founded in the 1990s, Boost does not believe in there being a business side to itself apart from its annual conferences which began in 2006, and [https://sites.google.com/a/boost.org/steering/ its steering committee website] has this excellent paragraph which was written after I caused a fuss about how little steering the "steering committee" does: |
| 1573 | | So what is the problem one may now ask? Well, you've got to understand what it is like trying to achieve anything in Boost, and indeed those BoostCon 2011 slides summed up the problem nicely. Firstly you must achieve consensus in the community, which involves persuading a majority on the Boost Developers list -- or rather, persuading a majority of those who ''respond'' on boost-dev that what you proposed is not a terrible idea, often by you investing dozens of hours of your free unpaid time into some working prototype you can show people. This part also usually involves you fending off trolls, those with chips on their shoulder, those with vested interests, those threatened by any form of change, those who simply don't like you and are trying to take you down a peg etc which most unfortunately can include members of the steering committee itself. To achieve any significant change usually involves ''years'' of campaigning, because to get a dispersed heterogeneous community with few common interests to reach consensus on some breaking change even with a proven working prototype takes a minimum of a year, and usually many years, and again you must have a skin thick enough to repel all those trolls and naysayers I mentioned. If this begins to sound as disspiriting and as exhausting as attending WG21 meetings -- except this effort is entirely unpaid -- you would be right. Anyway, if after years of unpaid effort and toil you finally reach consensus, ''then'' you can apply to the steering committee for funding to implement and deliver your change, by which stage -- to be blunt -- the funding is but a tiny fraction of the time, blood and sweat you have personally already invested. |
| 1574 | | |
| 1575 | | Note that if you have a job with a high status role and excellent job security and believe open source to be a noble non-commercial hobby, then taking years to change a community consensus is far less important to you than if you have no job security, your job is menial and you just want -- and I hate to be so blunt about this -- the open source project leadership to proactively help you rather than aloofly stand apart until you've invested years of free unpaid effort for potentially no real gain. Those unpaid hours of effort could go on actually writing new code, in an environment more welcoming, perhaps even in a community you yourself create. Most with this opinion do not of course voice it, and simply silently leave the community for somewhere more contemporary and less hostile to evolution. |
| | 1575 | So what is the problem one may now ask? Well, you've got to understand what it is like trying to achieve anything in Boost, and indeed those BoostCon 2011 slides summed up the problem nicely. Firstly you must achieve consensus in the community, which involves persuading a majority on the Boost Developers list -- or rather, persuading a majority of those who ''respond'' on boost-dev that what you proposed is not a terrible idea, often by you investing dozens of hours of your free unpaid time into some working prototype you can show people. This part also usually involves you fending off trolls, those with chips on their shoulder, those with vested interests, those threatened by any form of change, those who simply don't like you and are trying to take you down a peg etc which most unfortunately can include some members of the steering committee itself. To achieve any significant change usually involves ''years'' of campaigning, because to get a dispersed heterogeneous community with few common interests to reach consensus on some breaking change even with a proven working prototype takes a minimum of a year, and usually many years, and again you must have a skin thick enough to repel all those trolls and naysayers I mentioned before. If this begins to sound as disspiriting and as exhausting as attending WG21 meetings -- except this effort is entirely unpaid -- you would be right. Anyway, if after years of unpaid effort and toil you finally reach consensus, ''then'' you can apply to the steering committee for funding to implement and deliver your change, by which stage -- to be blunt -- the funding is but a tiny fraction of the time, blood and sweat you have personally already invested. |
| | 1576 | |
| | 1577 | Note that if you have a job with a high status role, excellent pay and excellent job security and believe open source to be a noble non-commercial hobby, then taking years to change a community consensus is far less important to you than if you have no job security, your job is menial and you just want -- and I hate to be so blunt about this -- the open source project leadership to proactively help you rather than aloofly stand apart until you've invested years of free unpaid effort for potentially no real gain. Those unpaid hours of effort could go on actually writing new code, in an environment more welcoming, perhaps even in a community you yourself create. Most with this reaction do not of course voice it, and simply silently leave the community for somewhere more contemporary and less hostile to evolution. |
| 1584 | | In practice that means a constant cycle of acquiring and ''maintaining'' regular donations from the firms and users who use your free of cost open source software, whether from fee paying training courses and conferences, or simply through a donate button on the website, but mostly through investing effort to build networks and relationships with your biggest commercial users and ''leveraging'' those networks and relationships into a regular donations stream. Concomitant with that leveraging is a ''reverse leveraging'' by those sponsors on the future strategic direction of the open source project, so when this funding process is working well you the open source org gets funds to dispose upon a future vision of the open source project previously agreed (or at least discussed) with your major sponsors. This is why leveraging your open source project for funding is '''not''' blackmail but rather funded coevolution, despite what many in the C++ leadership might think. |
| 1585 | | |
| 1586 | | In Django, you pitch your idea for some change, whether radical or not, to a small, contained authoritative leadership where the authority to decide and fund any initiative is clearly demarcated. They more often or not are acting really as a filter for appropriately repackaging and presenting ideas to the sponsors for funding, though they usually have some slush money around for the really radical experimental stuff (if it is cheap). The process for enacting change is therefore extremely well specified, and moreover because a central authority will approve or disapprove an idea quickly, you don't waste time pushing ideas without a chance. If they do approve an idea, you get actual true and genuine real support from the leadership |
| 1587 | | |
| 1588 | | Compare that to Boost. And Django is hardly the only business orientated open source project. [https://plone.org/foundation Plone has a well established funding pipeline for regular sprints]. [https://assoc.drupal.org/ Drupal is particularly aggressive, and indeed effectively runs a Kickstarter once per release to fund the sprint needed to make the release happen]. Some might argue that these are all products rather than an umbrella of heterogeneous libraries and are therefore fundamentally different, [https://www.apache.org/foundation/sponsorship.html if so consider the funding pipeline the Apache Software Foundation runs]. |
| | 1586 | In practice that means a constant cycle of acquiring and ''maintaining'' regular donations from the firms and users who use your free of cost open source software, whether from fee paying training courses and conferences, or simply through a donate button on the website, but mostly through investing effort to build networks and relationships with your biggest commercial users and ''leveraging'' those networks and relationships into a regular donations stream. Concomitant with that leveraging is a ''reverse leveraging'' by those sponsors on the future strategic direction of the open source project, so when this funding process is working well you the open source org gets funds to dispose upon a future vision of the open source project previously agreed (or at least discussed) with your major sponsors. This is why leveraging your open source project for funding is '''not''' blackmail but rather funded coevolution, despite what some of the older generation might think. |
| | 1587 | |
| | 1588 | In Django, you pitch your idea for some change, whether radical or not, to a small, contained authoritative leadership where the authority to decide and fund any initiative is clearly demarcated. They more often or not are acting really as a filter for appropriately repackaging and presenting ideas to the sponsors for funding, though they usually have some slush money around for the really radical experimental stuff (if it is cheap). The process for enacting change is therefore extremely well specified, and moreover because a central authority will approve or disapprove an idea quickly, you don't waste time pushing ideas without a chance. If they do approve an idea, you get actual true and genuine real support from the leadership with funding, marketing and informing the community what is happening next and why instead of being cast alone into the wilderness to argue with a mailing list to build "consensus" for some idea. |
| | 1589 | |
| | 1590 | Compare that to Boost. And Django is hardly the only business orientated open source project. [https://plone.org/foundation Plone has a well established funding pipeline for regular sprints]. [https://assoc.drupal.org/ Drupal is particularly aggressive, and indeed effectively runs a Kickstarter once per release to fund the sprint needed to make each release happen]. Some might argue that these are all products rather than an umbrella of heterogeneous libraries and are therefore fundamentally different, [https://www.apache.org/foundation/sponsorship.html if so consider the funding pipeline the Apache Software Foundation runs] and [https://www.apache.org/foundation/governance/ the full time employees they have maintaining their crucial infrastructure]. |
| 1592 | | But open source, and the world, has moved on, and the current gold standard of open source practice is to run your open source project as a '''business'''. C++ has recently [https://isocpp.org/about set up a Standard C++ Foundation] which could prove less weak willed and ineffective than Boost's current leadership, but to date the only big risk that I know they've taken was [https://isocpp.org/about/financial-assistance-policy to fund Eric Niebler's work on Ranges for C++], so I think the jury is still out on whether they will rise to the standards set by the Python Foundation, the Plone Foundation, the Django Foundation or even the Linux Foundation. Which brings me to the final major cause of why I think C++ is failing to become the best general purpose systems language available ... |
| | 1594 | But open source, and the world, has moved on, and the current gold standard of open source practice is to run your open source project as a '''business'''. C++ has recently [https://isocpp.org/about set up a Standard C++ Foundation] which could prove more promising, but to date the only big risk that I know they've taken was [https://isocpp.org/about/financial-assistance-policy to fund Eric Niebler's work on Ranges for C++], so I think the jury is still out on whether they will rise to the standards set by the Python Foundation, the Plone Foundation, the Django Foundation or even the Linux Foundation. Which brings me to the final major cause of why I think C++ is failing to become the best general purpose systems language available ... |
| 1629 | | * Show the way forwards for the design of new language features instead of trying to design top down and ending up with a cancer like the original C++ concepts. I suppose I had better explain what I mean by a cancer in this context, so here goes. During the 2007-2008 push by WG21 to reach C++ 0x, items began to be cut for the first initial draft of what would become C++ 11 for purely political reasons. The big problem was that if you proposed some library X for standardisation, someone on ISO would say "library X would look completely different once expected new language feature Y is in the language, therefore library X shouldn't enter the standard right now". The most damaging expected new language feature Y was undoubtedly original C++ concepts as that killed off entire tracts of exciting C++ library standardisation, much to the often bitterness of those who had invested months to years of their spare time developing those libraries. After all, most new language features are developed by highly paid employees where it is their day jobs, whereas of the twenty or so Boost libraries now in C++ 11 were generally developed in the family time of enthusiasts who earned a fraction of the pay of those people on WG21 killing off their work, and to keep a chipper attitude whilst those who have not sacrificed shoot down often years of your work is not easy. No wonder we have seen a slow exodus of Boost old timers since the 2011 C++ standard, some leaving C++ completely for good for employment in large corporations which frown on employees having any interests outside the corporation, some disavowing Boost forever and having anything to do with Boost, and some no longer trying to get their libraries into Boost (i.e. past any form of review process) and preferring to house their C++ libraries elsewhere and specifically away from potential standardisation. |
| | 1631 | * Show the way forwards for the design of new language features instead of trying to design top down and ending up with a cancer like the original C++ concepts. I suppose I had better explain what I mean by a cancer in this context, so here goes. During the 2007-2008 push by WG21 to reach C++ 0x, items began to be cut for the first initial draft of what would become C++ 11 for purely political reasons. The big problem was that if you proposed some library X for standardisation, someone on ISO would say "library X would look completely different once expected new language feature Y is in the language, therefore library X shouldn't enter the standard right now". The most damaging expected new language feature Y was undoubtedly original C++ concepts as that killed off entire tracts of exciting C++ library standardisation, much to the often bitterness of those who had invested months to years of their spare time developing those libraries. After all, most new language features are developed by highly paid employees where it is their day jobs, whereas of the twenty or so Boost libraries now in C++ 11 were generally developed in the family time of enthusiasts who earned a fraction of the pay of those people on WG21 killing off their work, and to keep a chipper attitude whilst those who have not sacrificed shoot down often years of your work is not easy. No wonder we have seen a slow exodus of Boost old timers since the experience which led to the 2011 C++ standard. |
| 1635 | | * Once that is achieved, a proper replacement for Microsoft COM is needed (hopefully reusing the ASIO/Networking TS event loop) |
| 1636 | | * After that, my personal preference would be for an as-if everything inlined build system that eliminates the need for all other C++ build systems. I'll speak more on that idea below. |
| 1637 | | |
| 1638 | | * Boost's leadership or a replacement for Boost takes on the role of proactively deciding what new C++ libraries are needed to solve the strategic goals set and finding contractors to design, peer review and implement such solutions. |
| | 1637 | * Once that is achieved, a proper replacement for Microsoft COM is needed (hopefully reusing the ASIO/Networking TS event loop) as a true demonstration of all modularity and reusability can be in C++. |
| | 1638 | * After that, my personal preference would be for an "as-if everything inlined" native build system that eliminates the need for all other C++ build systems. I'll speak more on that idea below. |
| | 1639 | |
| | 1640 | * Boost's leadership or a replacement for Boost takes on the role of proactively determing what new C++ libraries are needed to solve the strategic goals set by the Foundation and by commercial sponsors, and finding contractors to design, peer review and implement such solutions. |
| 1645 | | === A brief and cynical history of Boost and its relationship to C++ === |
| 1646 | | |
| 1647 | | Boost has been experiencing in recent years a number of problems surrounding its scalability as it grows and especially as it matures (specifically, maintenance and cultural fear of any substantial change). Historically speaking, [https://web.archive.org/web/19991011120524/http://www.boost.org/ 1999 Boost] was first [http://www.boost.org/users/proposal.pdf a proving ground for new C++ standard library ideas] making better use of the 1998 C++ ISO standard, but from [https://web.archive.org/web/20000816132250/http://www.boost.org/libs/compiler_status.htm August 2000 onwards] it also began to turn into a broken compiler workarounds layer which whilst initially was great for those with broken compilers, it incurred an enormous technical debt into the codebase which still weighs heavily upon anyone trying to change anything substantial affecting more than one library, including cultural beliefs in the importance of support of broken toolsets. Due to its roots as a proving ground for standard C++ libraries, Boost is unusually similar to the C++ standard library which is both good and bad: good in terms of the quality of design, implementation and testing - bad in terms of monolithicity, poor coupling management, and over-reliance on a single person in charge of each library (great for the library if that maintainer is active, not so great across libraries when maintainers must work together, terrible if a maintainer vanishes or departs). |
| 1648 | | |
| 1649 | | Boost's "high point" was probably between 2001 [https://web.archive.org/web/20010614021213/http://www.boost.org/index.htm when the modern peer review process was formalised] (and has remained remarkably, perhaps unfortunately, unchanged since) when it had thirty three libraries and 2005 [https://web.archive.org/web/20050815003310/http://www.boost.org/ when the Boost Software Licence was completed, most of the famous Boost libraries had reached their final designs, and ten Boost libraries were accepted into the C++ TR1 standard] when it had sixty-nine libraries. |
| 1650 | | |
| 1651 | | Some may be surprised at the choice of 2005 given that BoostCon began in 2006, but if you examine the Wayback Machine for boost.org from 2005 onwards you will see a plateau was reached from 2005 onwards, and I'm going to claim that this is because Boost had achieved its original mission for the first time and had therefore taken its first step towards obsoleting itself. The push was now on for the C++ 0x ISO standard, and I think the period 2007-2008 was crucial because in my opinion it was during this period that many of the Boost old timers began to become bitter with the C++ library process as their items began to be cut for the first initial draft of what would become C++ 11 for purely political reasons. The big problem was that if you proposed some library X for standardisation, someone on ISO would say "library X would look completely different once expected new language feature Y is in the language, therefore library X shouldn't enter the standard right now". The most damaging expected new language feature Y was undoubtedly original C++ concepts as that killed off entire tracts of exciting C++ library standardisation, much to the often bitterness of those who had invested months to years of their spare time developing those libraries. After all, most new language features are developed by highly paid employees where it is their day jobs, whereas of the twenty or so Boost libraries in C++ 11 were generally developed in the family time of enthusiasts who earned a fraction of the pay of those killing off their work, and to keep a chipper attitude whilst those who have not sacrificed shoot down often years of your work is not easy. |
| 1652 | | |
| 1653 | | 2008 was the last time the Boost web site was in any way improved, and even before that stale content was no longer being removed or repaired as it once used to be, thus making the Boost web presence increasingly less authoritative and less a "one stop shop" for answers. We then began to see a slow exodus of Boost old timers, some leaving C++ completely for good for employment in large corporations which frown on employees having any interests outside the corporation, some disavowing Boost forever and having anything to do with Boost, and some no longer trying to get their libraries into Boost (i.e. past any form of review process) and preferring to house their C++ libraries elsewhere and specifically away from the Boost community. A too frequent common factor this author has noticed is the bitterness and anger regarding Boost and its community in some of the big names in Boost who were the minds behind some of its most successful libraries. The peer review process ground to a halt from the end of 2012 onwards, and [https://www.youtube.com/watch?v=8TkCwizP14Y by early 2014 a noticeable increase was obvious in the rate of decline of Boost] since the 2011 ISO C++ standard began to be implemented in available toolchains. |
| 1654 | | |
| 1655 | | Since 2014 measures have been taken to reinvigorate Boost by a number of actors including myself, but those measures are not germane to this essay. What I will say is that by Summer 2015, by the strictest measure of these things, perhaps as many as sixty of the one hundred and thirty or so Boost libraries could be considered undermaintained or not maintained. That suggests that Boost, under its current set of procedures, culture, management and infrastructure, is struggling to scale past about seventy libraries -- or put another way, the malaise and staleness within Boost started around the same time as the number of libraries in Boost passed about seventy in 2005. The scaling limit being around seventy is of particular significance because that is exactly around where the human cognitive limit for physically separated group sizes lands, and indeed military tactical units have been organised around groups of sixty to eighty soldiers since the Romans simply because trial and error found that number worked best. ''I am therefore going to assert, rather than claim, that under a volunteer based system where one individual is the maintainer of each library that Boost cannot healthily exceed about seventy libraries (i.e. seventy maintainers) as a single collection'', and you will find all my efforts to reinvigorate Boost - including this Handbook - revolve around that assertion of there being a hard scaling limit to each collection. |
| 1656 | | |
| 1657 | | |
| 1658 | | === What does this history have to do with defaulting to standalone capable modular C++ libraries? === |
| | 1647 | |
| | 1648 | |
| | 1649 | === TODO Essay about wisdom of defaulting to standalone capable modular (Boost) C++ 11/14 libraries with no external dependencies VS Essay about wisdom of dependency package managers in C++ 11/14 === |
| | 1650 | |
| | 1651 | TODO |
