Asio sync IO functions need timeout parameters

[Stephan Menzel <stephan.menzel@…>]

Hi Chris,

knowing this is an issue that has been discussed again and again I think it's better to handle it in a ticket.

Long story: ​http://article.gmane.org/gmane.comp.lib.boost.user/45360

Short story: I think asio's sync routines either need a timeout parameter or need to be removed.

Sync IO without a timeout is something I can hardly imagine any sensible use case for. In most cases one would definately need such functionality. Workarounds have been suggested but most would create too much performance or boilerplate overhead within the application. This should be handled within the lib and would actually render the sync routines a lot more useful.

Thanks a bunch!

Stephan

[anonymous]

Hi Chris,

any chance, this may get some attention soon?

Cheers,

Stephan

[chris_kohlhoff]

Just because you can't imagine a use case doesn't mean there aren't any :)

I don't agree with either of the suggestions:

1) Remove the synchronous functions. They may be useful to applications that are signal aware, that use non-blocking i/o, or just plain don't care. They're in POSIX this way so they're staying in asio this way.

2) Add timeout parameters. It's the wrong place in the API to add a timeout as it does not scale to higher levels of abstraction i.e. composed operations. Timers are kept separate so that they may be equally used with composed operations such as async_read or async_read_until as with the socket's async_read_some operation.

I'm interested in exploring and improving these use cases, but in my opinion these are not viable solutions.

[stephan.menzel@…]

Hi Chris,

I guess I have to work through this disappointment then.... And yet, if there are no use cases and the situation is good as it is, how come the topic pops up in both asio and boost mailing list with such a regularity?

My code has now several unportable spots in one lib caused entirely by this. Whenever things have to go synchronous, I have to workaround that way:

Example snippet UDP request/answer:

asio::ip::udp::endpoint sender_endpoint;

asio::ip::udp::socket socket(m_iosrv, asio::ip::udp::v4()); socket.send_to(asio::buffer(requeststring), m_pwh_master_ep);

asio::ip::udp::socket::native_type native_sock = socket.native(); fd_set rfds; FD_ZERO(&rfds); FD_SET(native_sock, &rfds);

struct timeval tv; tv.tv_sec = 42; tv.tv_usec = 0; if (!select(native_sock + 1, &rfds, NULL, NULL, &tv)) {

THROW(net_error, "timeout reading from socket");

}

size_t reply_length =

socket.receive_from(asio::buffer(reply, udp_max_length), sender_endpoint);

...and so forth and suchlike. Is that really what you want asio code to look like?

Realizing this with a timer would be OK for me and I tried but this doesn't work because the timers cannot cancel syncronous operations. Behold your very own docs: http://www.boost.org/doc/libs/1_41_0/doc/html/boost_asio/reference/basic_datagram_socket/cancel.html

It clearly says "cancel all asynchronous operations" so I can't timeout a sync_read with it.

Once I go in these synchronous functions asio appears not to offer any kind of defined way out of it. Please correct me if I'm wrong but all the answers to all the questions on the lists always boiled down to this.

If the given solutions are not viable, well that's alright with me. Just tried to help. But in any case the topic needs to be addressed.

By the way, If you say "They're in POSIX this way so they're staying in asio this way.", can you clearify that? In my example I have to go down to select, which has a timeout and solves the problem. Besides, I doubt very much this is a valid point. The whole boost is about offering stuff that should be cross platform standard but is not yet. Does POSIX offer regexes, time localisation, string formatting or parsing? Boost offers lots of such things and if plain POSIX is enough and everything on top of it is unnecessary, what's the point of having asio or indeed boost anyway?

Sorry for being so stubborn and verbose but I urge you to reconsider this. I'm sure there must be a way.

Cheers, Stephan

BTW: Milestone 1.39, so take your time, no rush ;-)

[anonymous]

I suspect you misinterpreted my response. To clarify:

• I did not suggest there are no use cases for a timed read in a synchronous context.

• I did say there are use cases for synchronous reads and writes, and so I will not consider removing them (as the trac item suggests).

• My reference to POSIX is that it also provides *non-timed* synchronous operations, such as read() and write(). They are the equivalent of asio's synchronous functions. They are used in similar circumstances.

If you need cancellability, then my answer will always be that you should use asynchronous operations. I am interested in improving techniques to make it easier to use asynchronous operations in synchronous call contexts. But as this trac item is predicated on changing the behaviour of the synchronous functions, I'm closing it.

P.S. your select()-based code is broken :)

[anonymous]

Well, obviously there's no point in discussing this any further.

Nevertheless it would have been a good way of discussing this to not simply poopoo all of my arguments despite the obvious truth in them but simply state a way to solve this very real and existing dilemma. Thus giving proof of it's nonexistence. Again, if the situation is good as it is, why am I having such problems with it. If you consider blocking synch operations without a timeout such a good thing to have there's nothing I can add.

Greetings...

Stephan

[stephan.menzel@…]

Ah, and by the way, nothing could prove my point any better than your post scriptum so thanks for this statement.

[stephanecharette@…]

We're recently begun investigating re-writing the networking parts of our app using ASIO, but this is a blocker for us. We can get most of the select+read calls replaced very easily with ASIO synchronous calls, but we're stuck trying to figure out how to get the select timeouts implemented. Using asynchronous calls is more work than it would be for us to port our own networking code to the necessary platforms we need. After discussing in StackOverflow and getting pointed to this bug ticket, we decided to back out of the ASIO port we began last week. Synchronous network calls are great and simple to use especially since they map relatively well to legacy socket code, but the missing timeouts from legacy select() calls make ASIO unusable for porting existing apps with ease.

At the very least, careful explanation of this in the docs or a better explanation (+ examples) of whatever we missed would probably be greatly appreciated. Otherwise, with the exception of this one tiny feature, we were impressed by how "clean" our C++ socket code felt once we'd converted parts of it to ASIO.

[anonymous]

Tried the async timer approach, but could never get that to work on our application which has to read from one thread and reply on the same socket on another thread. Yeah, tried strands, but it wouldn't work.

So synchronous would be nice, simply because synchronous UDP operations are supposed to be thread safe.

[klein]

@stephan.menzel Awesome code! The only one i found in the freaking internet, was looking for that timeout code like madness, thanks! Helped really a lot! And yeah it would be much better if there was a timeout option directly in boost syncronous udp socks, then it would be much more better than using the native approach.

[anonymous]

Why don't you just try to add or explain why you can't do that, instead of poopooing the arguments.

Many people need such a feature and it's not about if having synch. functions with no timeouts have a use. You can basically make this an optional parameter.

[fredrik]

I'd like to second klein's comment and than Stephan for the code snippet. I need to be able to have timeouts in my legacy synchronous code without the trouble of rewriting the whole thing to async I/O.

It's too bad that the asio maintainers are so disinterested.

[anonymous]

I'd like to also throw in my 2 cents in favor of having this added in for synchronous sockets. There is at least one use case where it's far easier to simple have a synchronous socket with a timeout than do the async method. For example, I'm writing a sort of local DNS server right now where I need to forward out requests to an external nameserver in certain cases. So essentially I have a simple setup where I have a single service running and a async handle send/onSent method.

So in each of these threads is where I must decode and read the incoming request and choose to forward it or not. In order to use the async way I need to basically have a socket running on a different service so when I call run() on that service, it's blocking until the async receive with timeout completes. If I set that socket up (for external resolution) to use my existing io service, guess what, I can use the run() call to be blocking within the current thread... so easy solution is to have a synchronous call for external resolution... but then since I can't put a timeout on it, if one call fails my whole program locks. Bad. We could argue for days about how to change the structure of my program to be "better".... or we could simply have a synchronous timeout and then everything works beautifully and logically. Now I've got to come up with platform specific hacks to make this work....

Don't get me wrong, open source is great, the work is much appreciated. As someone who has open sourced tens of thousands of lines of code in my career, I understand the frustration when someone whines about my hard work that I've given away for free.. but emotions aside, if they're right, they're right.

[ascensionsystems@…]

Sigh, in my previous comment as "anonymous" I made a mistake, should be:

"guess what, I CAN'T use the run() call to be blocking within the current thread..."

Thanks.

[anonymous]

Timeout is needed

[Stephan Menzel]

Well, things have been quiet around here and I would like to add a few words since so many of you have commented here.

I have reconsidered my original opinion. I believe now it was part wishful thinking and part misconception of what asio is. I have been using boost for a long time now and I remember, in the first of those years I often thought: All this is great but what's seriously missing here is networking. No matter how much boost principles, types and libs were weaved into every part of my software I was always forced to drop out and become platform dependent or DIY when it came to networking. And I happened to do most of my work with networks. So when I stumbled across the first versions of asio (not part of boost then) I was excited and adopted early on. Later, it would become part of boost and I thought: Finally! Boost has gotten its networking! And this is how I thought about boost.asio, leading me to above statements. And yet, this was wrong. It has always been and still is "ASIO" as in "asynchronous IO" and not boost.network or whatever. So Chris suggesting to be asynchronous whenever this topic came up was right from his perspective. When you wanna do async io, you may choose this but when you wanna do generic networking sync and async, asio probably ain't what you are looking for. Sadly, this would imply that boost still lacks generic networking but I wouldn't go this far. There have been efforts with cpp-netlib but I don't really see that happening. As a result of this ticket here I have reduced the extent of asio's usage in my projects. Away from "using asio and its types whenever it comes to networking" to "using ASIO behind the hood when it comes to asychronous operations". In this regard, ASIO is still an awesome lib. Nowadays, I mostly use the io_service class directly for thread multiplexing and de-multiplexing. It never let me down and is well documented. A little networking now and then but not as much as I used to. I now often go native for that. I still believe that in the brutish realities of commercial software development it is not option to simply expect to go asynchronous everywhere. Large scale legacy products need refurbishing and maintenance, blocking third party calls will force constraints on your architecture as well as decisions made in the past that may be gradually changed towards an asynch architecture but rarely in one big haul. So exitsing synchronous parts and components will still need to have their place and pure async is certainly not the answer to everything.

So after all I'm OK with asio not implementing those timeouts. I still believe the sync operations are useless without it but I respect the "a" in "asio" and don't have to use it. Let's see if cpp-netlib or anything like that make it into boost eventually. Would still be good to have sync and async networking in one go. As for asio, it became such an important part of all I do with threads and actions and such... Wouldn't miss it.

Stephan

[y]

well, I admit I'm a complete boost::noob.

I found myself in this situation: had to modify some ASIO code writted by someone else. But timeout is absolutely needed.

I have to tell you, sadly the solution that required less study/dev time was: drop asio, use my own socket class.

please, reconsider. you could use some #ifdef directive if you don't want dependences asio<->timers or whatever else, but in 2k13 a socket class without timeout is not exactly "useful"

[anonymous]

This is probably the dumbest ticket discussion I ever read.

Everybody who is using blocking socket functions wants a timeout for them. But no, there is a better solution, the "official" one: simulate a blocking function by using asynchronous functions and waiting for them to finish.

Wow. Talk about scratching your left ear with your right foot. I don't know about you guys, but Computer Science 101 taught me that when encountering a problem a program should stop and handle it, not wait and wait and wait some more for it to solve itself.

Connect or read functions which take minutes to return are just plain stupid. They should timeout and do it fast, not wait or retry forever.

PS: If you like the asynchronous approach so much, why don't you create some overloads for the synchronous functions which do what this tries to explain.

PPS: Synchronous and asynchronous are very very different. Stop suggesting asynchronous solutions for synchronous problems or designs.

[stephan.menzel@…]

Replying to anonymous:

This is probably the dumbest ticket discussion I ever read.

Anonymous, if you sit it out way past computer science 101 eventually they also tell you that a professional and polite tone in a discussion generally yields a higher chance of success.

With this being said, a discussion with no involvement of the only person who can actually do something about it, Chris for example, is indeed kinda pointless.

[anonymous]

Maybe instead of adding timeouts to sync calls a new synchronous function "select" could be added that would take a list of SyncReadStreams and SyncWriteStreams and a timeout value.

[Magallo]

Timeout is OBVIOUSLY needed!

[Magallo]

Could some possibly post a sample code, using select() on native sokcet? Someone similar to the code posted by stephan.menzel@… some comments above (see comment 4) but working?

[SlowRiot]

There are definitely places where either simulating a sync timeout using async functions, or resorting to platform-specific socket timeouts, is absolutely necessary. This is true of both new and legacy code. Unfortunately, being the sole network library in Boost, asio has too great a level of exposure to get away with ignoring this under the excuse that it's a specifically async library; asio provides those sync functions, but intentionally hobbled versions, and this is irresponsible.

Since it's possible to currently implement a native socket timeout with only a few lines of extra platform-specific code on top of asio, and many people are using this already, i can't really see any valid justification for not including that functionality in asio itself, when it would clearly serve the purpose that all boost is intended for - avoiding platform-specifics in common and advanced programming tasks, just like this one.

[stephan.menzel@…]

Very well spoken, @SlowRiot. With this bug now entering it's 4th year, without Chris responding in any way to the discussion, I have to wonder if boost asio is actually maintained. Maybe it would be better to do the ol' DIY and patch the thing. A patch could be posted on the mailing list and if it's rejected there at least it's a little more out in the open than this ticket. The question might get some attention. After all, there have been very few comments here agreeing with the absence of said timeouts. Is there anyone reading this who could assist in that? Maybe one could team up and do it? If there's reaction to this I will set up a temp email address and we can get in touch.

[anonymous]

I, too, would like to see timeouts on sync calls - I made the mistake of writing using sync. Now it's causing a problem and I have to change it all to async. I don't see many practical "real world uses" for sync without timeouts.

I just started working with Boost, this missing feature is by far the most disappointing part of the library.

[helge@…]

Although Asio has "asynchronous" in its name it is extremely useful for synchronous use, too. Writing solid networking code is not trivial and Asio simplifies the process a great deal - reducing both time and probability of error.

For that reason I second the request for timeout parameters for the synchronous functions.

[hagondes]

I know at least two projects who have their code written with sync I/O at start. They take long time to realize the potential timeout issue. They take long time to realize there are no easy fix. They take long time to rewrite everything from sync to async I/O.

I'd suggest to at least warn people in documentation to avoid this trap. And yes, a timeout for synchronous functions is definitely needed.

[hagondes]

I know at least two projects who have their code written with sync I/O at start. They take long time to realize the potential timeout issue. They take long time to realize there are no easy fix. They take long time to rewrite everything from sync to async I/O.

I'd suggest to at least warn people in documentation to avoid this trap. And yes, a timeout for synchronous functions is definitely needed.

[anonymous]

At the very least, a section in the ASIO documentation - an example, perhaps - of writing a synchronous timeout using select?

[anonymous]

At the very least, a section in the ASIO documentation - an example, perhaps - of writing a synchronous timeout using select?

[gast128@…]

This is really pity. Reading synchronously with a timeout is a common use case (e.g. serial port does not respond; website down. You don't want to block your application indefinitely) and not very well supported by Boost.Asio. I managed to get something working with deadline timers and canceling the other but it was awful. Using the Win32 API with the serial communications protocol directly which supports timeout is much more easier.

[tom@…]

Definitely, timeout is needed. What's interesting though is that it IS there, in one form. So if you use basic_socket_streambuf, so if you wrap a socket in an iostream, you DO get timeouts for connect, read and write. There's an "expiry" field you can set on the stream. They are in the associated draft standard as well. So clearly someone "high up enough" thinks it's useful enough at some level to add at least partial support for a read with timeout, it just doesn't seem to have been pushed far enough.

You can replicate the logic manually if necessary, as long as you aren't wanting to use a higher level construct such as the SSL socket stream.

Personally I think it's a huge shame that this isn't implemented fully. It seems to be all that's missing from making asio a complete and general networking library for C++. As it is, I have to decide up front whether to use it or not, depending on what I want to do. Maybe I use it in the server, but in the client I have to code differently, and use a different library. It's so nearly the go to toolbox for doing networking.

It's not like it should be that complex to implement. The basic_socket_streambuf code seems to prove what the logic should be. You want exactly that, but in the tcp::socket class instead.

Please, please please.

[tom@…]

Can I add my plea for this to be reopened, in some form? Fine. The original suggestion for the form in which this functionality should be made available wasn't suitable. No problem. But there's clearly a large weight of opinion that would dearly like to have a timeout facility on a read/write. FWIW I would like to suggest adding "expiry" to a socket like it is on basic_socket_streambuf.

What's the correct approach? Create a new issue with a different suggestion for solution? Create a new issue with a general request for some kind of functionality?

Thanks.

[stephan.menzel@…]

Hey Tom, quite frankly, I don't think this is being read by anyone except those in the Cc list. This has been going on for almost a decade and there was no word whatoever from Chris or any other maintainer. For what it's worth, Beast suffers from the same condition.

I don't think this is going to happen. As I wrote somewhere here a few years ago, asio ist not the general networking library for C++ but still a reliable asynchronous networking library for C++.

Better not hold your breath!