Prototype implementation

If you're interested, you may want to examine the ​documentation for the prototype implementation.

News

(2007/08/19)

• implemented the ProxyProducer concept, and converted Dataflow.Signals components to use it.

(2007/08/06)

• generic dataflow concepts pulled into the Dataflow library
• Signal Network refactored to into a module of Dataflow
• added an experimental module based on dataflow through pointers and Phoenix actors.

(2007/07/05)

• everything is now in boost::signals namespace
• directory structure is now more modular
• fused/unfused is now a template parameter
• connection mechanism is more easily extensible and modular
• all components can now accept both fused and unfused signals
• storage has been reimplemented through a generic component
• storage send signal is now separate from operator().
• test_disconnect now fails?

(2007/06/14)

• added branching tests as separate files
• finished updating docs for Boost community update

(2007/06/12)

• Implemented generic applicator, modifier, conditional, and instantiator components which do various generic things to signals.
• Some of the old components have been converted to derive the generic components.
• Documentation moved to qb+doxyref format and expanded.
• Tests have been split up into separate files.

(2007/06/03)

• I have reimplemented all of the prototype components (except for selector, which needs to be redisigned) using Boost.Fusion. Unfused versions of the components are made possible by two classes which I have adapted from fusion's unfused_typed class to unfuse through inheritance. Both of these classes unfuse operator(), and one of them unfuses the constructor with the same parameter types as for operator(), and the other replicates an (already unfused) constructor.
• Adapted tuple serialization code by Troy D. Straszheim to take care of fusion vector serialization (for socket_sender and socket_receiver).
• added Xcode project files, tests compile and pass on OS X/darwin

Proposed changes

• Support for default consumer categories - depending on the producer type, the consumer category could be inferred if not specified.

Introduction

The Signal Network library aims to facilitate the implementation and interconnection of objects into signal networks using Boost.Signals. To understand the concept of signals (data generators / senders) and slots (data consumers / receivers), please see the Boost.Signals documentation and the associated tutorial.

This is a proposal for development of the Signal Network library as a part of Google Summer of Code and evential inclusion into Boost. Depending on what is preferred, the Signal Network library can be included into Boost.Signals or considered as a separate library.

The Signal Network library is to provide

• a concise operator based syntax that facilitates construction and readiblity of signal networks
• mechanisms that facilitate construction of signal network components
• implemented building-block components that are applicable to a wide range of signal networks
• detailed documentation (tutorial, examples and reference) of all of the above

The development platforms are

• OS X / Xcode with Apple's branch of GCC
• Windows / MSVC8.0

Motivation

Boost.Signals provides an excellent mechanism which can be used to dynamically configure the flow of data within a program. This is helpful for registering callbacks, for event passing, and for signal processing applications.

However, when dealing with more complicated networks, there are several improvements to Boost.Signals that could facilitate the construction, configuration, and execution of said networks. Several categories which could use some improvement are listed below.

For simplicity, the examples in this documentation deal with signals which carry numbers, and components which manipulate them. The same principles are applicable to many kinds of signal oriented applications (e.g., ones where the signals carry event information, video frames, audio buffers, etc.).

Conciseness and readability of code

In complex signal-based applications, many components are signal filters or some kind (i.e., they receive some input, process it, and send some output). For processing of any substantial complexity, each type of filter is most likely implemented as a class, with one or more member boost::signal objects serving as signal outputs, and one or more member functions serving as input slots.

Suppose we would like to connect a network consisting of a signal generator, a couple of filters, and a couple of signal consumers (say, display components). The components might be declared as follows:

signal_generator generator; // contains a boost::signal accessible via signal_generator::default_signal()
signal_filter1 filter1; // contains a boost::signal accessible via signal_filter1::default_signal(),
                                                // and a slot function operator()
signal_filter2 filter2; // contains a boost::signal accessible via signal_filter2::default_signal(),
                                                // and a slot function operator()
signal_display display1, display2; // contains a slot function operator()

To connect the network, the following would be necessary using Boost.Signals:

Boost.Signals syntax:

// connect the generator to filter1
generator.default_signal().connect(boost::bind(&signal_filter1::operator(), boost::ref(filter1)));
// connect filter1 to filter2
filter1.default_signal().connect(boost::bind(&signal_filter2::operator(), boost::ref(filter2)));
// connect the output of each filter to a display
filter1.default_signal().connect(boost::bind(&signal_display::operator(), boost::ref(display1)));
filter2.default_signal().connect(boost::bind(&signal_display::operator(), boost::ref(display2)));       

The code above is rather long to write, and might not be particularly readable. What we would like to do instead is something more concise and readable:

Proposed/desired syntax:

// connect (>>=) generator to filter1,
// branch (|) from filter1 to both display1 and filter2,
// and connect (>>=) filter2 to display2.
generator
        >>= filter1
                | display1
                | filter2 >>= display2;

The above code is much more concise, and after getting familiar with the syntax, understanding the structure of the signal network is easy and quick.

Flow control components

In many cases, we would like to modify the flow of signals after the network has been constructed. Boost.Signals allows us to change the network by disconnecting old or creating new connections. So, for example, if we wanted to introduce a second signal generator into the system (e.g., generator2), we could use signal::disconnect (or signal::disconnect_all_slots) and signal::connect to toggle between the two sources:

Boost.Signals flow control:

// switch to the other signal generator:
generator.default_signal().disconnect_all_slots();
generator2.default_signal().connect(boost::bind(&signal_filter1::operator(), boost::ref(filter1)));

There are a few potential problems with this approach. First, the syntax is again perhaps a little too involved. And second, altering the network while signals are being sent may require extra care (especially in multi-threaded settings). Hence, depending on the situation, it might be difficult to use this approach to alternate between the two sources (perhaps a custom object would need to be written to choose the appropriate source).

Instead, it would be desirable to have ready-made components which can be used to alter the flow of signals without changing the architecture of the network:

Proposed/desired flow control:

// introduce a selector into the network
signal_selector selector;

// the generators are now connected to the selector
generator >>= selector.slot1;
generator2 >>= selector.slot2;

// and the rest of the network is now constructed as follows:
selector
        >>= filter1
                | display1
                | filter2 >>= display2;
                

// Now we could choose the source as follows:
selector.select(1); // select input coming into slot1
...
selector.select(2); // select input coming into slot2
...
selector.select(0); // select no input

The latter approach offers more robust control, and can be implemented in a thread-safe way (safe even when the generated signals are in threads separate from the reconfiguring thread). Similar arguments can be made for other flow control components such as junctions and gates.

Other building-block signal network components

There are many other components that are usable in various signal network settings. The Signal Network library would provide a set of such commonly needed components, such as templates for signal generators, collectors, threading components, etc.

Interest

The interest for a library such as the Signal Network library was queried in the boost developers mailing list message ​interest in a "signal flow" library?

The community reported interest ranging from very interested through conditionally interested (this would be interesting if...) to the opinion that using boost::signal directly would be more dependable and readable. Some of the suggestions are summarized in the ​prototype documentation.

Also, there were several mentions of similar "home-brewed" libraries that were in ways similar to the proposed Signal Network library, indicating that including a well designed library of this type in Boost would be useful to the community.

Relationships with other libraries

The Signal Network library is based directly on Boost.Signals, and throughout the development I plan to maintain compatilibility with both the original Boost.Signals library by Doug Gregor and the thread_safe_signals version under implementation by Frank Mori Hess (which is geared to replace Boost.Signals). Most of the tests written for the current prototype version of the library work without problems with both base signal implementations.

The library also makes use of Boost.Preprocessor, Boost.TypeTraits, Boost.Utility (call traits and enable_if). Some parts of the library also use Boost.Threads (for components that are related to threading), and Boost.Asio combined with Boost.Serialization (for components that can transmit / receive serializable signals through Boost.Asio).

A possible relationship / overlap with ​Boost.Iostreams was indicated by Jeff Garland. According to its website, Boost.Iostreams serves two main purposes:

• To allow the easy creation of standard C++ stream and stream buffer classes for new data sources and sinks.
• To provide a convenient interface for defining i/o filters and attaching them to standard streams and stream buffers.

Hence, Boost.Iostreams has similar goals for streams as the Signal Network library has for signals. The difference is that signal networks offer more permanent channels of communication (connecting a signal to a slot makes a lasting connection - everything coming out of the signal will be received by the slot until the connection is destroyed), while streams offer more of a momentary transfer of information (stream << data makes no permanent connection). Boost.Iostreams does allow permanent insertion of filters into a stream, but this does not seem to be intended for more complicated network topologies.

Also, ​Channel by Yigong Liu covers a lot of concepts that I hope to address in the Signal Network library. In particular, this library inspired designing components that would allow signal networks to be constructed accross multiple computers (using Boost.Asio and Boost.Serialization).

Google Summer of Code Scope

Even though a signal network library such as the one proposed above can grow in many directions, there is a core functionality requested by the community that can be completed as a part of Google Summer of Code.

In particular, the operator-based connection syntax, the development of flow control components, the development of signal storage and threading components, and documentation of each are feasible for me to complete as a part of Google Summer of Code.

Download

Please see the ​prototype documentation.

Acknowledgements

Thanks to the ​Arts, Media and Engineering Program at Arizona State Unversity for their support in developing and releasing the prototype Signal Network library code.

Thanks to the Boost community for their valuable suggestions and feedback:

• Tobias Schwinger indicated that the library could be used for pulling rather than pushing data.
• James Jones suggested that a ||-like operator could be used for branching.
• Paolo Coletta suggested a "video_generator >>= ( effect1 && effect2 ) >>= image_sum" - like syntax that would allow parallel processing of signals (identified as the "join" pattern by Yigong Liu).
• Yigong Liu suggested enhanced support for common network topologies, such as mesh.
• Braddock Gaskill pointed out the relationship with the "pipes and filters" pattern, and suggested the possibility of using functions as filters. He also suggested the library would be more useful if different functions executed in parallel threads, or were queued to specific worker threads, if the library would provide functionality to control and schedule the execution of the invoked functions, or traverse the call graph.

Copyright

The Signal Network library is copyright Stjepan Rajko 2007. Use, modification and distribution is subject to the Boost Software License, Version 1.0. (See copy at http://www.boost.org/LICENSE_1_0.txt)