After speaking with Ben Gray and the discussion on the mailing list, this is an attempt to create a sockopt to allow connecting pipes to not immediately be available for traffic. The problem is in a PUSH to many PULL situation, where there is a connect to a PULL which is not there. This connect will immediately create a pipe (unlike bind), and traffic will be load balanced to that pipe. This means if there is a persistently unavailable end point then the traffic will queue until HWM is hit, and older messages will be lost.

This patch adds a sockopt ZMQ_DELAY_ATTACH_ON_CONNECT, which if set to 1 will attempt to preempt this behavior. It does this by extending the use of the session_base to include in the outbound as well as the inbound pipe, and only associates the pipe with the socket once it receives the connected callback via a process_attach message. This works, and a test has been added to show so, but may introduce unexpected complications. The shutdown logic in this class has become marginally more awkward because of this, requiring the session to serve as the sink for both pipes if shutdown occurs with a still-connecting pipe in place. It is also possible there could be issues around flushing the messages, but as I could not directly think how to create such an issue I have not written any code with regards to that.

The documentation has been updated to reflect the change, but please do check over the code and test and review.

src/session_base.hpp

@@ -52,6 +52,9 @@ namespace zmq
 
         //  To be used once only, when creating the session.
         void attach_pipe (zmq::pipe_t *pipe_);
+        
+        // To be used once only, for delayed connection
+        void onconnect_attach_pipe (pipe_t *pipe_);
 
         //  Following functions are the interface exposed towards the engine.
         virtual int read (msg_t *msg_);
@@ -104,6 +107,9 @@ namespace zmq
         //  Pipe connecting the session to its socket.
         zmq::pipe_t *pipe;
 
+        //  Pipe connecting the socket to the client
+        zmq::pipe_t *outpipe;
+
         //  This flag is true if the remainder of the message being processed
         //  is still in the in pipe.
         bool incomplete_in;