Boost::asio, set tcp_nodelay, bad file descriptor?

[Alex Black]

I'm experimenting with turning off the Nagle algorithm to possibly
improve the performance of my socket server, I'm processing lots of
small requests and I'd like as little latency as possible.

Here is a code snippet:

void CBaseWebServer::StartAccept()
{
try
{
shared_ptr<tcp::socket> pSocket(new
tcp::socket(m_IoService));

boost::asio::ip::tcp::no_delay option(true);
pSocket->set_option(option);

m_pAcceptor->async_accept(*pSocket,
bind(&CBaseWebServer::HandleAccept, this, pSocket,
boost::asio::placeholders::error));
}
catch ( ... )
{
cout << "Unexpected exception caught in " <<
BOOST_CURRENT_FUNCTION << endl <<
boost::current_exception_diagnostic_information();
}
}

I'm getting an exception on the set_option line:

Unexpected exception caught in void CBaseWebServer::StartAccept()
Throw in function (unknown)
Dynamic exception type:
N5boost16exception_detail10clone_implINS0_19error_info_injectorINS_6syst
em12system_errorEEEEE
std::exception::what: Bad file descriptor

What am I doing wrong here? Thanks!

- Alex

[Igor R]

               m_pAcceptor->async_accept(*pSocket,
bind(&CBaseWebServer::HandleAccept, this, pSocket,
boost::asio::placeholders::error));

Did you open() the acceptor (or created it with the "opening" constructor)?

[Alex Black]

I created the acceptor like this:

m_pAcceptor = shared_ptr<tcp::acceptor>( new tcp::acceptor(m_IoService, tcp::endpoint(tcp::v4(), port)) );

-----Original Message-----
Sent: Wednesday, August 05, 2009 3:10 PM
Subject: Re: [Boost-users] Boost::asio, set tcp_nodelay, bad
file descriptor?

               m_pAcceptor->async_accept(*pSocket,
bind(&CBaseWebServer::HandleAccept, this, pSocket,
boost::asio::placeholders::error));

Did you open() the acceptor (or created it with the "opening"
constructor)?
_______________________________________________
Boost-users mailing list
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Igor R]

       m_pAcceptor = shared_ptr<tcp::acceptor>( new tcp::acceptor(m_IoService, tcp::endpoint(tcp::v4(), port)) );

Sorry, I didn't pay attention that the exception was at set_option()
line. You try to set_option() on a new "unopened" socket. Try and open
it using one of these overloads:
http://www.boost.org/doc/libs/1_39_0/doc/html/boost_asio/reference/basic_stream_socket/open.html

[Alex Black]

Thanks Igor. I'm trying out calling set_option in my HandleAccept function (because the socket is open by that point), does that make sense?

- Alex

-----Original Message-----
Sent: Wednesday, August 05, 2009 6:38 PM
Subject: Re: [Boost-users] Boost::asio, set tcp_nodelay, bad
file descriptor?

       m_pAcceptor = shared_ptr<tcp::acceptor>( new
tcp::acceptor(m_IoService, tcp::endpoint(tcp::v4(), port)) );

Sorry, I didn't pay attention that the exception was at
set_option() line. You try to set_option() on a new
http://www.boost.org/doc/libs/1_39_0/doc/html/boost_asio/refer
ence/basic_stream_socket/open.html
_______________________________________________
Boost-users mailing list
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Igor R]

Thanks Igor.  I'm trying out calling set_option in my HandleAccept function (because the socket is open by that point), does that make sense?

Yes, it should be ok.
"Bad file descriptor" error usually means that some lowlevel API was
called with an invalid (eg., uninitialized or already closed) socket
handle.

[J.W.F. Thirion]

Dear Boost community,

In ASIO, when I execute a cancel() on a tcp::socket, does it remove all
of the handlers associated with the socket from the handler queue?

In other words, let's say I've started an async read and write
operation. Assuming the read completed and a handleRead is placed on the
internal handler queue but have not yet been called by the IO service
thread. The write operation is still in progress. Let's assume now that
the object destructs before the handleRead is called, but in the
destructor a cancel operation is done on the socket to cancel any
outstanding asynchronous requests (e.g. in the destructor of the class).
The way I've got it is that the handleWrite will be called with an
aborted/cancelled error, and the handleRead will be removed from the
handler queue. Is this correct?

Also, as reading and writing operations are done in the IO service
thread, would it be safe to call the tcp::socket cancel from another
thread (e.g. in the destructor of a class)?

Another problem then occurs: The destructor could be entered, just as
the handler function is entered (e.g. with success as error code), and
the object destroyed. Accessing of the member variables will then cause
a crash of the program. Any ideas on how to get around this problem of
the object destructing but the handler function already called by the IO
service thread? All I can think is that one posts the deletion of the
object onto the same handler queue, do nothing in the destructor of the
object, override the constructor and destructor of the object to be
private so that a getObject and destroyObject function news and posts a
delete for the object.

Any ideas would be highly appreciated.

Thanks!
Derik

[Igor R]

Post by J.W.F. Thirion
In ASIO, when I execute a cancel() on a tcp::socket, does it remove all
of the handlers associated with the socket from the handler queue?

No. Every async_XXX request must end with its handler invocation.

<...>

Post by J.W.F. Thirion
Let's assume now that
the object destructs before the handleRead is called

This is very bad and will cause crash/segfault. Parts of your handler
(incl. the bound object) must not cease before the async.operation is
complete.

Post by J.W.F. Thirion
Also, as reading and writing operations are done in the IO service thread, would it be safe to call the tcp::socket cancel from another

thread (e.g. in the destructor of a class)?

No. You should post() the cancel operation to io_service thread.

[J.W.F. Thirion]

Thank you so much. It makes sense from what I've observed. Three final
questions on this:

1) Let's say I've posted a request for a function, called handleClose,
onto the handler Queue from a function called close() which can be
called from a thread different from the IO service thread. The
handleClose function will call the cancel on the socket. Would it be
possible while I'm inside this handler, for ASIO to post more requests
onto the queue. E.g. let's say I've just entered my handleClose handler.
Before issuing the cancel, is it possible that ASIO could schedule a
handleRead for a successful read of a packet, or is the asynchronous
socket operations blocked by the fact that I'm inside a handler, meaning
that I can assume that if I'm inside my handleClose function, I won't
get any handlers posted by ASIO in the mean time, and thus the only
handleRead and handleWrite, etc. that I'll get will be ones with a
cancelled/aborted error (i.e. not success as error code when the
handleRead is called after my handleClose has executed) due to the
cancel called inside handleClose?

2) Does the above logic apply to timers also? In other words, I won't
get a handleTimer function posted with a success as error code for a
deadline timer while I'm inside the handleClose function? I'll need to
cancel the timer inside my handleClose function, but I'm afraid that
I'll get a handleTimer with a success instead of cancelled/aborted after
my handleClose function is done, if, while I'm inside my handleClose
function, just before the timer is cancelled, the timer expires, meaning
that even though I've cancelled the timer, I'll still get the
handleTimer called with success as error code indicating that the timer
has expired?

3) Are the events for the handlers placed on the queue always in
sequence. In other words, if I cancel a read operation and then post a
function handleDestroy, then I'll always get handleRead with
aborted/cancelled called before I get a handleDestroy called? In this
case the cancel interrupts the async read, which will post a handleRead
at the point of the cancel being called and not after the handleClose
function exits, which means that handleRead will occur before the
handleDestroy?

Many thanks for the advice!

Kind regards,
Derik

Post by Igor R

Post by J.W.F. Thirion
In ASIO, when I execute a cancel() on a tcp::socket, does it remove all
of the handlers associated with the socket from the handler queue?

No. Every async_XXX request must end with its handler invocation.
<...>

Post by J.W.F. Thirion
Let's assume now that
the object destructs before the handleRead is called

This is very bad and will cause crash/segfault. Parts of your handler
(incl. the bound object) must not cease before the async.operation is
complete.

Post by J.W.F. Thirion
Also, as reading and writing operations are done in the IO service thread, would it be safe to call the tcp::socket cancel from another

thread (e.g. in the destructor of a class)?
No. You should post() the cancel operation to io_service thread.
_______________________________________________
Boost-users mailing list
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Roman Shmelev]

Hi, Derik.

In my opinion, you must not place together async operations and their
sequence in the queue. The word "async" means "somewhen" and does not
means some order. So I suggest not to rely on "the handlers placed on
the queue always in sequence". I try to design my async apps with such
thinking. When you need some sequence, just schedule 1 op, and when
its completion handler fires - schedule 2nd op... and so on.

As for "2", I should say that at least in one-thread implementation,
the timer will fire with error code, not success code.
As for "1", I assume the same - error codes will be not success codes.
The most important thing - call "cancel" inside the io_service thread
(use "post" some function, as Igor already said)

[Igor R]

1, 2) IIRC, the answer is no - i.e. even if your io_service is running
in a single thread, you can't be sure that an attempt to cancel i/o
operation or timer will really have any effect, because the operation
can already be complete so that cancellation won't affect it, and even
more than that -- its handler can already be in the queue. I remember
there is a discussion on this topic somewhere, but I can't find it at
the moment.

3) It seems that in the current implementation your assumption is
correct, i.e. the handlers are processed as "fifo", but I agree with
Roman that it's not a good idea to rely on such an undocumented and
implementation-dependant behaviour. It's better to make your own queue
or to organize the things as "request-handler-request..."

[Roman Shmelev]

Post by Igor R
1, 2) IIRC, the answer is no - i.e. even if your io_service is running
in a single thread, you can't be sure that an attempt to cancel i/o
operation or timer will really have any effect, because the operation
can already be complete so that cancellation won't affect it, and even
more than that -- its handler can already be in the queue. I remember
there is a discussion on this topic somewhere, but I can't find it at
the moment.

Thank you for correcting me, Igor. I believe you are right :)
My logic was really simple:
1) async op completed. (And even) Handler put into queue.
2) cancel() called.
3) it's time to get our handler from queue and fire it
[4) why not check socket state and if it was cancelled then set error
code to corresponding value?]
5) fire the handler

[Igor R]

Post by Roman Shmelev
[4) why not check socket state and if it was cancelled then set error
code to corresponding value?]
From the point of view of the service, a handler is just a functor

that can be invoked as "f()":
http://www.boost.org/doc/libs/1_37_0/doc/html/boost_asio/reference/Handler.html

But even if we put aside the design/technical aspects and concentrate
on the semantic ones: why should the error indicate cancellation if we
were too late to cancel the operation and it actually managed to
succeed?

[Roman Shmelev]

Post by Igor R
But even if we put aside the design/technical aspects and concentrate
on the semantic ones: why should the error indicate cancellation if we
were too late to cancel the operation and it actually managed to
succeed?

Heh, from user's(or mine) point of view it hasn't succeed in time :)

Post by Igor R
From user's point of view - op succeeds only when handler was called.

mm, I see holes in my logic, it's too.. "high-level"

[J.W.F. Thirion]

I just wanted to say thanks to all who responded.

My solution is to make use of shared_ptr and enable_shared_from_this
rather than trying to manage the lifetime of the object myself. I can't
see a way of doing that at the moment due to the unpredictable nature of
the invocation process of ASIO, which isn't a bug in ASIO, but rather
the way it was designed (fully asynchronous).

Post by Roman Shmelev

Post by Igor R
But even if we put aside the design/technical aspects and concentrate
on the semantic ones: why should the error indicate cancellation if we
were too late to cancel the operation and it actually managed to
succeed?

Heh, from user's(or mine) point of view it hasn't succeed in time :)

Post by Igor R
From user's point of view - op succeeds only when handler was called.

mm, I see holes in my logic, it's too.. "high-level"
_______________________________________________
Boost-users mailing list
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Igor R]

Accessing of the member variables will then cause a crash of the program. Any ideas on how to get around this problem of

the object destructing but the handler function already called by the
IO service thread?

You can find the solution the ASIO examples and tutorial: usually it's
worth binding the handers to the shared_ptr of the object, so that the
object's life time would be managed automatically, and no destruction
would occur before all the handlers are done.