* Background thread scheduling
- add ZMQ_THREAD_AFFINITY ctx option; set all thread scheduling options
from the context of the secondary thread instead of using the main
process thread context!
- change ZMQ_THREAD_PRIORITY to support setting NICE of the background
thread when using SCHED_OTHER
Solution: add ZMQ_ZAP_ENFORCE_DOMAIN to hide backward incompatible
change and make it disabled by default.
In a future release that breaks API compatibility we can then switch
the default to enabled in order to achieve full RFC compatibility.
Fixes#2762
On FreeBSD the sysmbol __FreeBSD_kernel__ is only defines if a
specific param.h file is included, unlike Debian/kFreeBSD where this
symbol is always defined. So also compile the FreeBSD specific code
if __FreeBSD__ is defined for FreeBSD 11 & 12 compatibility.
Signed-off-by: Christopher Hall <hsw@ms2.hinet.net>
Solution: when a connection breaks and ZMQ_RECONNECT_IVL is set to -1,
which means a reconnection will not be attempted, send a message from
the I/O thread to the application thread to make the socket call
term_endpoint, which is the equivalent of manually calling
zmq_disconnect.
This way subsequent zmq_connect call to the same endpoint will attempt
again to do a connection.
Otherwise, for some socket types like SUBs, those new connects will
fail as the endpoint is recorded, despite the connection having been
permanently closed.
Add test cases to exercise this corner case with TCP and IPC.
zmq::socket_poller_t::wait() had an important set of common lines
between POLL and SELECT variant.
Solution: refactor zmq::socket_poller_t::wait() and add the
following methods:
zmq::socket_poller_t::zero_trail_events()
zmq::socket_poller_t::check_events()
zmq::socket_poller_t::adjust_timeout()
Signed-off-by: Marc Sune <mardevel@gmail.com>
Solution: do not assert() the values returned; if we failed to set the thread name - we just have harder debugging, not flawed production conditions.
Closes github issue #2679
Signed-off-by: Jim Klimov <EvgenyKlimov@eaton.com>
Solution: added tests for CURVE, add emitting of client-side event in curve_client_t; add ZAP code 300/500 tests for all mechanisms; suppress sending an error message for ZAP code 300
Solution: use check_basic_command_structure in curve_client_t::decode, also prepare other client mechanisms to use that method by rearranging inheritance hierarchy
* Problem: no test case with CURVE encryption and large identity
Solution: added test case (currently crashing)
* Problem: possible buffer overflow in mechanism_t::add_property
Solution: add target buffer length parameter and check the buffer is sufficiently large
* Problem: test cases accidentally excluded from build
Solution: remove #if/#endif
* Problem: possible buffer overruns related to metadata at various locations
Solution: allocate buffer large enough for actual metadata, reduce code duplication
* Problem: syntax error related to pointer type conversion
Solution: change argument type of make_command_with_basic_properties to const char *
* Problem: large metadata may cause an assertion in produce_initiate
Solution: Allow metadata of arbitrary size in produce_initiate
* Extracted connect_vanilla_socket function
* Problem: no tests for ZMTP-CURVE protocol errors
Solution: added two test cases with erroneous HELLO commands
* Problem: insufficient tests for ZMTP-CURVE protocol errors
Solution: added two test cases with erroneous HELLO command version
* Problem: test HELLO message is invalid apart from deliberate errors
Solution: create cryptographically correct HELLO message
add tweetnacl.c to test_security_curve
* Problem: nonce is incorrect, build fails with GCC
Solution: use correct non prefix
* Problem: make builds are failing
Solution: transfer CMake changes to (auto)make files
* Problem: nonce is incorrect, build fails with GCC
Solution: use correct non prefix
* Problem: make builds are failing
Solution: transfer CMake changes to (auto)make files
* Problem: no test with INITIATE command with invalid length
Solution: added test case
* Problem: code duplication between test_security_curve.cpp and curve_client.cpp
Solution: extracted parts of zmq::curve_client_t::produce_hello into reusable function
* Problem: code duplication between test_security_curve.cpp and curve_client.cpp
Solution: extracted further parts of zmq::curve_client_t into reusable functions
added missing file
* Problem: mechanism_t::add_property can be declared static
Solution: declare mechanism_t::add_property static
* Problem: intermediate crypto data needs to be passed between static function calls to curve_client_tools_t
Solution: add non-static member functions
* Problem: msg_t instance may be closed twice
Solution: remove offending close
* Problem: prepare_hello uses static curve_client_tools_t::produce_hello
Solution: Use non-static curve_client_tools_t::produce_hello
* Problem: no test with invalid command name where INITIATE command is expected
Solution: added test case
* Problem: make builds are failing due to curve_client_tools.hpp not being found
Solution: add curve_client_tools.hpp to list of source files
* Problem: wrong initializer order in zmq::curve_client_t
Solution: reorder
* Problem: under non-Windows systems, test fails because random_open was not called
Solution: call random_open/random_close within test
* Problem: conflict between custom function htonll and macro definition on Darwin
Solution: define htonll function only if not defined as a macro
* Problem: nullptr not defined on all platforms
Solution: replace nullptr by NULL
* Problem: libsodium builds not working
Solution: adapt compile and link file sets for libsodium builds
* Problem: Makefile.am broken
Solution: Fix syntax
* Problem: no tests for garbage encrypted cookie or content in INITIATE
Solution: added test cases
* Problem: test cases accidentally excluded from build
Solution: remove #if/#endif
* Solution: some error cases are unreachable
Problem: for the time being, added some comments without changing the code
* Added comments on hard-to-test cases
Solution: force the compiler to make the atomic_counter_t alignment
friendly.
This will ensure that the pointers inside the buffers allocated by
shared_message_memory are aligned, at the cost of growing the memory
size of atomic_counter_t from 4 to 8 bytes on 64 bit (when not using
mutexes).
Note that although content_t contains an atomic_counter_t, the
compiler already padded the struct so there is no change in the
buffer sizes used by the engines, save for the extra 4 bytes for the
buffer's own single atomic counter.
Fixes#2588
Solution: remove requirement to manually define macro and just check
for the C++ supported version.
Note that compiler intrinsics still have priority if available, to
avoid changes unless necessary.
* Problem: missing test for status code 300, inadequate assertion for status code 500
Solution: add test, change assertion (currently test fails)
* Problem: gcc compiler error deprecated conversion from string constant
Solution: declare variable as const
* Problem: in case of ZAP handler returning a status code other than 200, no appropriate event is emitted
Solution: immediately emit event after receiving reply from ZAP handler
* Problem: endpoint address is not included in zap-reply monitor event
Solution: added functions to retrieve endpoint address in zmq::i_engine and zmq::session_base_t
removed unused code block in zmq::stream_engine_t::next_handshake_command
* Problem: wrong formatting
Solution: fix formatting
* Problem: test fails because of EPIPE
Solution: add EPIPE/ECONNRESET/ECONNAGAIN handling for more test cases
* Fixing #2002 one way of doing it
* Mechanisms can implement a new method `error_detail()`
* This error detail have three values for the moment: no_detail
(default), protocol, encryption.
+ generic enough to make sense for all mechanisms.
- low granularity level on information.
* Fixing #2002: implementation of the error details
The ZMQ_EVENT_HANDSHAKE_FAILED event carries the error details
as value.
* Removed Microsoft extenstion for enum member access
This was leading to compilation error under linux.
* Adaptation of CURVE test cases
* Monitoring event: changed API for detailed events
Removed ZMQ_EVENT_HANDSHAKE_FAILED and replaced it by:
- ZMQ_EVENT_HANDSHAKE_FAILED_NO_DETAIL,
- ZMQ_EVENT_HANDSHAKE_FAILED_PROTOCOL,
- ZMQ_EVENT_HANDSHAKE_FAILED_ENCRYPTION
Adaptation of text case `security_curve`
* Removed event value comparison
This was introduced for the previous API model adaptation
* Removed the prints in std output and added missing details
`current_error_detail` was not set in every protocol error cases
* Fixed initialization of current_error_detail
* Fixed error in greeting test case
The handshake failure due to mechanism mismatch in greeting is actually
a protocol error. The error handling method consider it like so and
send a protocol handshake failure monitoring event instead of no_detail.
Fixed the test_security_curve expectation as well.
* Upgraded tests of monitoring events
The tests check the number of monitoring events received
* Problem: does not build under Linux or without ZMQ_DRAFT_API
Solution:
- properly use ZMQ_DRAFT_API conditional compilation
- use receive timeouts instead of Sleep
* Problem: duplicate definition of variable 'timeout'
Solution: merged definitions
* Problem: inconsistent timing dependencies
Solution: reduce timing dependency by using timeouts at more places
* Problem: assertion failure under Linux due to unexpected monitor event
Solution: output event type to aid debugging
* Problem: erroneous assertion code
* Problem: assertion failure with a garbage server key due to an extra third event
Solution: changed assertion to expect three events (needs to be checked)
* Problem: extra include directive to non-existent file
Solution: removed include directive
* Problem: assertion failure on appveyor for unknown reason
Solution: improve debug output
* Problem: no build with libsodium and draft api
Solution: add build configurations with libsodium and draft api
* Problem: assertion failure on CI
Solution: change assertion to reflect actual behaviour on CI (at least temporarily)
* Problem: error in condition in assertion code
* Problem: assertion failure on CI
Solution: generalize assertion to match behavior on CI
* Problem: assertion failures on CI
Solution: removed inconsistent assertion on no monitor events before flushing
improved debuggability by converting function into macro
* Problem: diverging test code for three analogous test cases with garbage key
Solution: extract common code into function
* Problem: does not build without ZMQ_BUILD_DRAFT_API
Solution: introduce dummy variable
* Attempt to remove workaround regarding ZMQ_EVENT_HANDSHAKE_FAILED_NO_DETAIL again
* Problem: EAGAIN error after handshake complete if there is no more data in inbuffer
Solution: Skip tcp_read attempt in that case
* Problem: handshaking event emitted after handshaking failed
Solution: use stream_engine_t::handshaking instead of mechanism_t::status() to determine whether still handshaking
* Include error code in debug output
* Improve debugging output: output flushed events
* Split up ZMQ_EVENT_HANDSHAKE_FAILED_PROTOCOL into ZMQ_EVENT_HANDSHAKE_FAILED_ZMTP and ZMQ_EVENT_HANDSHAKE_FAILED_ZAP
* Fixed compilation without ZMQ_BUILD_DRAFT_API
* Renamed ZMQ_EVENT_HANDSHAKE_SUCCEED to ZMQ_EVENT_HANDSHAKE_SUCCEEDED for language consistency
* Renamed ZMQ_EVENT_HANDSHAKE_SUCCEED to ZMQ_EVENT_HANDSHAKE_SUCCEEDED for language consistency
* Renamed ZMQ_EVENT_HANDSHAKE_SUCCEED to ZMQ_EVENT_HANDSHAKE_SUCCEEDED for language consistency
* Fixed assert_monitor_event (require event instead of allowing no event)
Reverted erroneous change to handshaking condition
Renamed test_wrong_key to test_garbage_key
Generalized assumption in test_garbage_key to allow for ZMQ_EVENT_HANDSHAKE_FAILED_NO_DETAIL with error == EPIPE
* Better isolate test cases from each other by providing a fresh context & server for each
* Added diagnostic output
* Changed assertion to reflect actual behavior on CI
* Fixed formatting, observe maximum line length
* Fixed formatting, observe maximum line length
* Increase timeout to check if this fixes valgrind run
* Close server with close_zero_linger
* Increase timeout to check if this fixes valgrind run
* Increase timeout to check if this fixes valgrind run
* Generalize assertion to also work with valgrind
* Fixed formatting
* Add more diagnostic output
* Generalize assertion to also work with valgrind
Linux now supports Virtual Routing and Forwarding (VRF) as per:
https://www.kernel.org/doc/Documentation/networking/vrf.txt
In order for an application to bind or connect to a socket with an
address in a VRF, they need to first bind the socket to the VRF device:
setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
Note "dev" is the VRF device, eg. VRF "blue", rather than an interface
enslaved to the VRF.
Add a new socket option, ZMQ_BINDTODEVICE, to bind a socket to a device.
In general, if a socket is bound to a device, eg. an interface, only
packets received from that particular device are processed by the socket.
If device is a VRF device, then subsequent binds/connects to that socket
use addresses in the VRF routing table.
Solution: add a crypto [de-]initialiser, refcounted and serialised
through critical sections.
This is necessary as utility APIs such as zmq_curve_keypair also
call into the sodium/tweetnacl libraries and need the initialisation
outside of the zmq context.
Also the libsodium documentation explicitly says that sodium_init
must not be called concurrently from multiple threads, which could
have happened until now. Also the randombytes_close function does
not appear to be thread safe either.
This change guarantees that the library is initialised only once at
any given time across the whole program.
Fixes#2632
Before this commit, xhas_out() was returning true regardless. This
was correct before the ZMQ_ROUTER_MANDATORY flag as introduced.
However, ZMQ_POLLOUT.
With this commit, _if_ ZMQ_ROUTER_MANDATORY is set, xhas_out() will
return false if ALL peer's outgoing pipes are full.
There is an outstanding high-level design question:
If ZMQ_ROUTER_MANDATORY is set, and zmq_poll() waits for ZMQ_POLLOUT
events, zmq_poll() will immediately wake up if only 1 pipe has
room to send, regardless of the peer, creating a busy loop of
zmq_poll() wake-up, zmq_send() (EAGAIN). There is no way for
the application to selectively wait for ZMQ_POLLOUT for specific
peer(s), which seems somehow necessary in ZMQ_ROUTER_MANDATORY.
This discussion will be addressed in a separate issue.
Signed-off-by: Marc Sune <marc@voltanet.io>
Signed-off-by: Fredi Raspall <fredi@voltanet.io>
Solution: if a CURVE server is using zmq_connect, the same session
will be used for any client "reconnect" (actual binds). This is
acceptable, so do not assert if zap_pipe already exists during the
handshake, but simply reuse it.
Fixes#2608
Solution: remove the pipe from the real trie when a peer disconnects.
Also add a unit test that exercises the behaviour by reconnecting
a different socket and sending a message that matches.
Fixes#2601 and introduced by #2042
Solution: revert change that made ZAP mandatory.
The "Stonehouse" pattern, where CURVE is used only for encryption and
without authentication, is a valid use case so we should still
support it.
Also restore CURVE testing in the test_heartbeat.
Fixes#2594
Solution: if inproc://zeromq.zap.01 exists, which means ZAP is
enabled, abort immediately if it cannot be used (eg: out of memory)
or it is configured incorrectly (eg: wrong socket type).
Otherwise authentication failures will simply be ignored and
unauthorised peers will be allowed to slip in.
problem: for zmq radio/dish pattern, if the radio process restarts, the dish will not resend subscriptions to radio. And the result is that the dish will never receive any more messages.
solution: in session_base_t::reconnect (), take ZMQ_DISH into consideration when invoking hiccup method.
In function session_base_t::reconnect, if we set immediate to 1 and set linger, we will get into first block of reconnect function, and set pipe to NULL, but we forget to cancel timer of linger. Once timer tiggered, we will get coredump. Solution: cancel timer in the end of set pipe to NULL
Solution: don't set thread name on Android
Setting a thread name on Android may fail with "permission
denied" error and abort the process due to failed assertion.
Tested on Android 5 and 6 (two phones).
Strangely enough, it only happens on signed APKs and is fine
in debug. Using JeroMQ is not an option as we need TCP keepalive
settings and authentication which JeroMQ doesn't support.
Solution: roll back the pipe if writing messages other than the
first fails in router::xsend. Roll it back also when the pipe is
terminating.
Also add test case that reproduces the memory leak when ran with
valgrind.
Fixes#2567
Solution: roll back the pipe if writing messages other than the
first fails in router::xsend.
Also add test case that reproduces the memory leak when ran with
valgrind.
Fixes#2567
Solution: do not define ZMQ_HAVE_POLLER in src/zmq_drafts.h otherwise
src/zmq.cpp will implement zmq_poll using the new poller classes.
Same for ZMQ_HAVE_TIMERS, even though it has no internal effect, but
to be safe against future development.
Problem: GSSAPI DRAFT code was made conditional on
ZMQ_BUILD_DRAFT_API, but zmq_draft.h duplicates the DRAFT
symbols definitions from zmq.h so this is unnecessary.
Solution: drop the extra ifdefs
Problem: GSSAPI NAMETYPE socket option numbers were modified
+1000 when moved to DRAFT section, but should use the definitive
values while in DRAFT to minimize disruption later.
Solution: renumber the socket options
Problem: The new GSSAPI NAMESPACE options should have been
added to the DRAFT section of the API so they can be changed
until stabilized.
Solution:
- Move defines to the DRAFT section of zmq.h
- Duplicate them in zmq_draft.h, as is the local custom
- Compile only if defined (ZMQ_BUILD_DRAFT_API)
- Refactor internals slightly to avoid #ifdef hell
Problem: principals are looked up unconditionally
with the GSS_C_NT_HOSTBASED_SERVICE name type.
Solution: Add two new socket options to set the name type
for ZMQ_GSSAPI_PRINCIPAL and ZMQ_GSSAPI_SERVICE_PRINCIPAL:
ZMQ_GSSAPI_PRINCIPAL_NAMETYPE
ZMQ_GSSAPI_SERVICE_PRINCIPAL_NAMETYPE
They take an integer argument which must be one of
ZMQ_GSSAPI_NT_HOSTBASED (0) - default
ZMQ_GSSAPI_NT_USER_NAME (1)
ZMQ_GSSAPI_NT_KRB5_PRINCIPAL (2)
These correspond to GSSAPI name types of:
GSS_C_NT_HOSTBASED_SERVICE
GSS_C_NT_USER_NAME
GSS_KRB5_NT_PRINCIPAL_NAME
Fixes#2542
Problem: one call to gss_import_name() includes the terminating
NULL in a gss_buffer_desc.length, and one doesn't.
According to the examples at:
http://docs.oracle.com/cd/E19253-01/816-4863/overview-22/index.html
the NULL should be included in the length.
Solution: Fix one case to include the terminating NULL in the length.
Problem: if client sets ZMQ_GSSAPI_PRINCIPAL to a name
for which credentials cannot be obtained, authentication
proceeds with default credentials.
Solution: Before initializing the security context, check
whether there was a failed attempt to acquire credentials
for a specific principal and bail out if so.
Fixes#2531
Problem: if client sets the ZMQ_GSSAPI_PRINCIPAL to a valid
principal, authentication fails.
When an application sets ZMQ_GSSAPI_PRINCIPAL, whether as a
client or a server, libzmq internally calls gss_acquire_cred()
with cred_usage=GSS_C_ACCEPT. This cred_usage setting is for
acceptors (servers) only, thus it doesn't work for initiators
(clients).
Solution: Change the cred_usage parameter to GSS_C_BOTH to allow
initiators to set ZMQ_GSSAPI_PRINCIPAL.
* change macOS < 10.12 clock to SYSTEM_CLOCK, fixes#2537
* remove clock_id option from alt_clock_gettime since we always want a monotonic clock.
* update header definition for alt_clock_gettime
* pass clock definition down to host_get_clock_service for macOS < 10.12
* change to monotonic clocks
Treat ECONNRESET, ENETRESET and EINTR as non-fatal
failures in TCP Tuning on Windows as they are in the
Unixes. Also adds ENETRESET to the Unix list that was missing.
Detect AI_V4MAPPED is not supported in getaddrinfo() on the fly
and retry query. This issue was seen on ancient FreeBSD releases
and current implementation does support this flag correctly.
* Prevent DOS by asserts in TCP tuning
-Propagates socket option errors from the
tuning functions to the callers.
-Asserts a subset of error conditions during tuning,
excluding external network causes.
-Checks tuning results in 3 call sites and treats
them like failures to connect, accept, etc.
* Fix variable name
* Remove lambda requiring C++11
* add define for windows/UWP
* prevent issue with COM references
* gettickcount not available on uwp
* add compiler definitions
* add convenitnece cmake file
* brute force uwp compilation
* fix compiler version
* cosmetics
Solution: use pthread API to set the name. For now call every thread
"ZMQ b/g thread". Would be nice to number the I/O threads and name
explicitly the reaper thread, but in reality a bit of internal API
churn would be necessary, so perhaps it's not worth it.
This is useful when debugging a process with many threads.
Solution:
Revert "Problem with ZMQ_ROUTER_MANDATORY option when peer lost"
This partially reverts commit 06740b5d8c.
The change in the mentioned commit was fixed to be backward
compatible so test changes are no longer necessary.
Partially revert the commit, leaving only the bug fix and taking out
the test change and the superfluous change in socket_base.
indention
Revert "indention"
This reverts commit a6e7e192ac2d089ac9f7dc0d31d4b1fd10de982e.
indention
indention
Fix Failure in tests
Check both pipe full and pipe close
* Problem: zmq_z85_decode does not validate its input
Solution: added checks for invalid characters and overflows
* Added tests, added further check for range overflow, removed (multiple) calls to strlen
* Problem: gcc fails to build
Solution: added missing include directive
* Added VS2015 test_utils_z85 project
* Fixed indentation and copyright notice
* Resolved garbage from merge
* Revert "Added VS2015 test_utils_z85 project"
This reverts commit c58b3c664c.
* Problem: test calls zmq_z85_decode with a NULL dest
Solution: call zmq_z85_decode with a properly sized buffer
* Problem: tests for zmq_z85_* scattered over two files
Solution: merged files
* Removed reference to removed test file from CMakeLists.txt
* Problem: Missing include directive to stdint.h
Solution: Added include directive
* Define __STDC_LIMIT_MACROS before including stdint.h
* Problem: Wrong variable is checked for invalid character marker
Solution: Use correct variable
Solution: initialise *hwmboost to -1 instead of 1, and use it only if
it's >= 0. The socket option code checks anyway that the
user-provided value is >= 0 so there is no risk of clashing. The
documentation also specifies that it has to be >= 0.
Solution: setsockopt returns EINVAL if the connection was closed by
the peer after the accept returned a valid socket. This is a valid
network error and should not cause an assert.
To handle this we have to extract the setsockopt from the stream
engine, as there's no clean way to return an error from the
constructor. Instead, try to set this option before creating the
engine in the callers, and return immediately as if the accept
had failed to avoid churn. Do the same for the connect calls by
setting the option in open_socket, so that the option for that
case is set even before connecting, so there's no possible race
condition.
Since this has to be done in 4 places (tcp/ipc listener, socks
connecter and open_socket) add an utility function in ip.cpp.
Fixes#1442
The mechanism is instanciated during the handshake itself, when and
error happen before this, the error method shall work anyway.
An error handling with a NULL mechanism means the handshake fail, so the
handshake failure event is also raised in this case.
- removed the previously added encryption_error, less changes less bug
- handshake fail is now signaled when an error happen while the
mechanism is still hanshaking
Solution: check if IPV6_TCLASS is defined so that when Hurd adds
support it will just work. Also it will avoid tripping over this on
other similar legacy systems.
Solution: wrap the event triggering in the DRAFT ifdef as well as the
defines. This ensures that the event are returned only if the
library was built with DRAFTs enabled.
Also update the test case to expect the new events since it uses
the catch-all mask. Although the sequence of event is different and
this might be considered as an API breakage, using the catch-all
ZMQ_EVENT_ALL mask implies that, well, all events are monitored so
it's normal that new ones will be added.
Users that don't want this kind of behaviour to change can simply
monitor only the event that they care about.
- Moved new events in draft section + added to zmq_draft.h
- Removed the remainning tabs
- Reverted the hard error (back to soft error) in curve_server.cpp
=> The feature doesn't works anymore
Added two new monitoring events:
- ZMQ_EVENT_HANDSHAKE_SUCCEED is raised once the encryption handshake succeed
- ZMQ_EVENT_HANDSHAKE_FAILED is raised when it failed
Both events are raised on server and client side.
* Fixed handling of WSAEWOULDBLOCK to be generic
I don't know what was the intention of this early if statement but
now this is properly evaluated in wsa_error_no function if this is
performance issue I suggest moving evaluating this error code to the
beginning of wsa_error_no.
* Fixed handling of WSAEWOULDBLOCK to be generic
Introduced default pointer to const char * and overrides this as NULL
if function is called by zmq::wsa_error()
* Fixed handling of WSAEWOULDBLOCK to be generic
Introduced default pointer to const char * and overrides this as NULL
if function is called by zmq::wsa_error()
Solution: fix the check for the socket.
This regression happens when using zloop with zmq_pollitem_it with
only file descriptors registerted through zloop_poller.
On native Windows, the file system is case-insensitive, so this this
doesn't matter there.
However, when compiling on a case-sensitive filesystem, such as when
cross-compiling from a Unixoid using mingw, we have to use the case
the file is supplied with, which is all lowercase.
Solution: restore inclusion of poll.h if using poll before zmq.h as
it was originally, as AIX redefines the POSIX structures and provides
compatibility macros.
Also add alternative aliases for 32 bit AIX's pollitem struct:
events -> reqevents
revents -> rtnevents
1. stl container is not thread safety
2. rm_fd() and loop() end to clear the retired event source will in multi-thread operation
3. may be crashed in reaper thread to delete the items in the std::vector as the source is nullptr
1. stl container is not thread safety
2. rm_fd() and loop() end to clear the retired event source will in multi-thread operation
3. may be crashed in reaper thread to delete the items in the std::vector as the source is nullptr
* bug fix: AIX pollset error
buf fix: AIX only, pollset 'rm_fd' set fd_table to null first then set pe->fd to retired_fd
* remove extra spaces
remove extra spaces
Summary:
To set `Type Of Service` for IP layer packets ZMQ provides `ZMQ_TOS` socket
option. However this only works for v4 sockets. Considering things are moving to
IPv6 heavily (especially within enterprise networks), ZMQ should support setting
`traffic class` for v6 based on `ZMQ_TOS`.
There is a subtle difference between v4 and v6 in terms of the positioning of
field but TOS has same meaning in both v4 and v6. Linux provides following APIs
for v4/v6 to set TOS field value.
```
// For v4
setsockopt(fd, IPPROTO_IP, IP_TOS, tos, sizeof(tos));
// For v6
setsockopt(fd, IPPROTO_IPV6, IPV6_TCLASS, tos, sizeof(tos));
```
Test Plan:
Make sure Cmake works fine and all tests passes.
Imported this change to `OpenR` project and tested on our local testbed.
Captured some packets exchanged between PUB/SUB and ROUTER/ROUTER sockets
pairs. Verify that `TCLASS` value is set as per expectation.
Tasks: #2208
#define ZMQ_HAVE_WINDOWS_TARGET_XP disable uncompatible WinAPI
1. Disable call if_indextoname()
2. Emulate windows Condition Variable API in class condition_variable_t with std::condition_variable
This code can be compiled in MSVC 2015 with option "Platform toolset: Visual Studio 2015 - Windows XP (v140_xp)"
Solution: Rename the custom implementation of clock_gettime for macOS to
alt_clock_gettime and wrap all usage in preprocessor macros to only enable the
alternative version when using macOS <= 10.11.
This issue occurs when targeting macOS 10.11 or earlier but using the 10.12
or newer SDK.
Solution: during a connect with a TCP endpoint if a source address is
passed set the SO_REUSEADDR flag on the socket before the bind system
call.
Add unit test to cover this case for both IPv4 and IPv6.
* - Fixed windows build errors
- Extended monitor lock scope to prevent race-condition between
process_stop and monitor
* - Fixed windows build errors
- Extended monitor lock scope to prevent race-condition between
process_stop and monitor
This should restore full compatibility with earlier zmq_poll behavior.
It complicates things a little bit, as collisions must be detected, and when collisions are found:
- event masks must be merged
- pollitems, events arrays are no longer co-ordered
Reverts the recent zmq_proxy patch to workaround the lack of repeat-item support in zmq_poll that is now fixed.
Return value is the number of events found. This also propagates to the return value of zmq_poller_wait_all.
zmq_poller_wait was only returning events on the first-registered socket.
Solution: use only Libs.private to avoid breaking application builds.
Even though Requires.private are supposed to be parsed only if
pkg-config is called with --static, the --cflags parameter is enough
to trigger the parsing, causing build failures for applications that
do not (and should not) depend on libzmq's dependencies.
Solution: add dependencies, if necessary, to the .private Libs and
Requires field of the pkgconfig file at build time.
This way pkg-config --static --libs libzmq will correctly print
dependencies if they were used to build the static libzmq.a library.
instead of allocating a new, identical array and copying the data.
This is only safe while zmq_poller_event_t and zmq::socket_poller_t::event_t are the same struct,
which they presumably will remain.
Solution: zmq_poller_wait_all signals all events
allows signaling multiple events with one call to zmq_poller_wait_all
rather than emitting only one event.
this prepares for zmq_poll being based on zmq_poller,
which requires events for all sockets rather than just one.
* fix bugs of the pollset
1. extend 'fd_table' when fd_ is greater or equal than the size of 'fd_table';
2. delete specific fd from pollset before reset pollin or pollout according the description of AIX document
* fix bugs of the pollset
edit error. remove extra spaces and paste fault
* fix bugs of pollset
remove character '-' at the end line.
Solution: add a zmq_assert to check if the ephemeral sockets created
to drain the queue of pending inproc connecting sockets was allocated
successfully.
Solution: check if the connecting inproc socket has been closed
before trying to send the identity.
Otherwise the pipe will be in waiting_for_delimiter state causing
writes to fail and the connect to assert when the context is being
torn down and the pending inproc connects are resolved.
Add test case that covers this behaviour.
Solution: allow for '[' character when doing the basic sanity check
on the TCP endpoint.
Also add unit tests for both IPv4 and IPv6 source;dest format.
Solution: Use only lower case for header file name.
We can find "wincrypt.h" by "WinCrypt.h" on Windows because Windows uses
case insensitive file system. But we can't find "wincrypt.h" by
"WinCrypt.h" on Linux Because Linux uses case sensitive file system.
The gssapi has some helper functions gssalloc_malloc()/gssalloc_free()
which on windows doesn't call malloc()/free(). Instead these are
wrappers around HeapAlloc() and HeapFree(). To complicate matters
gssapi doesn't export these helper functions, so you're left using
the allocation method of your choice.
See Here:
89683d1f13/src/lib/gssapi/generic/gssapi_alloc.h
The zmq gssapi implementation is calling malloc and then calling
gss_release_buffer() to free the memory. gss_release_buffer uses
gssalloc_free() to free this buffer which on windows calls HeapFree()
instead of free(). This causes an access violation on windows.
Linux provides accept4(2) which will return a socket with FD_CLOEXEC set
when called with the SOCK_CLOEXEC flag. So call this when available and
fall back to fcntl(..., FD_CLOEXEC) if not.
getifaddrs() can fail transiently with ECONNREFUSED on Linux.
This has been observed with Linux 3.10 when multiple processes
call zmq::tcp_address_t::resolve_nic_name() simultaneously.
Before asserting in this case, make 10 attempts, with exponential
backoff, given by (1 msec * 2^i), where i is the attempt number.
Fixes#2051
On Windows, the written message does not seem to be guaranteed to be
written to stderr, in particular when stderr is redirected to a file. I
suppose this is because RaiseException terminates the process in a way
that does not give the CRT a chance to flush stdio buffers (or if it
does, there might be a problem when more than one CRT instance is linked
into the program and they overwrite each other's exception handler). Either
way, just make sure the assertion message ends up written to stderr to
ease diagnostics.
Solution: Provide poll() for Windows as well. This is a build option that
defaults to off as the resulting binary will only run on Windows Vista or
newer.
This is not tested with alternative Winsock service providers like VMCI,
but the documentation for WSAPoll does not mention limitations.
On my local machine, throughput improves by ~10 % (20 simultaneous
remote_thr workes to one local_thr, 10 byte messages), while latency
improves by ~30 % (measured with remote/local_lat).
Solution:
- Add checks for **poller_p_ to ensure that we do not segfault when either it
or the value within it are NULL
- Add tests for the above and increase error state coverage
Solution:
Mark them with LIBZMQ_UNUSED macro as per convention; although in future the
appropriate pthread code should be updated to support thread scheduling
priorities (for Mac OS X, et. al.)
The TIPC protocol bindings in ZeroMQ defaults to a lookup domain
of 1.0.0 to prevent 'closest first' search, and instead always
do round robin if several sockets in the network or node have
the same name published. In retrospect, this might have been a
bad idea because it won't work on standalone configurations.
We solve this by allowing an optional domain suffix to be provided
in the address, and 0.0.0 should be used in that case, or if the
TIPC address range in the cluster configuration is defined to some
other value. Domain suffixes are only relevant for connecting
addresses.
Signed-off-by: Erik Hugne <erik.hugne@gmail.com>
Solution:
- Add check for the [count] parameter in zmq_sendiov() and zmq_recviov()
- Use and add test for zmq_sendiov() in tests/test_iov.cpp
- Add error state tests for zmq_sendiov() in tests/test_iov.cpp
- Add error state tests for zmq_recviov() in tests/test_iov.cpp
- Cleanup tests/test_iov.cpp for style, consistency and clarity
- Generally improve test coverage for both API methods
Hat-tip:
@somdoron, @bluca
Solution: try to resolve the TCP endpoint passed by the user in the
zmq_unbind call before giving up, if it doesn't match.
This fixes a breakage in the API, where after a call to
zmq_bind(s, "tcp://127.0.0.1:9999") with IPv6 enabled on s would
result in the call to zmq_unbind(s, "tcp://127.0.0.1:9999") failing.
Add more test cases to increase coverage on all combinations of TCP
endpoints.
Problem:
Conditional logic in check_protocol() that checks if a protocol is supported,
is duplicated twice. Moreover, the first set of checks to ascertain if a
protocol is supported is done regardless of whether the particular protocol
will be built into the library or not.
Solution:
* Simplify/collapse all supported protocol checks into one in check_protocol()
* Enclose pgm/epgm/norm socket+protocol match checks with requisite macros
Solution: return -1 (no event) instead of 0 (event)
For some reason, this just returns 0 if there are no sockets registered
on the poller. Usually this would mean there has been an event. So the
caller would have to check the return value AND the event, or write code
that takes the number of registered sockets into consideration.
By returning -1 and setting errno = ETIMEDOUT like in the usual timeout
cases, it's more consistent and convenient.
Test case included.
Solution: if options.use_fd do not create temporary random
directory for ipc://*, since the socket is already created and
passed to the library by the user.
Solution: use the less nice but correct int constant 1000000000
instead of the shorter 1E9 to avoid a compiler warning when assigning
to timespec.tv_nsec, which is a long int.
Solution: in the Windows-specific ifdef in tcp_listener set_address,
check for error and set errno only after the IPv4 fallback has failed
too, to avoid setting errno when the socket creation succeeds through
the fallback.
Solution: if opening an IPv6 TCP socket fails because IPv6 is not
available, try to open an IPv4 socket instead when creating and
connecting a TCP endpoint.
Solution: if opening an IPv6 TCP socket fails because IPv6 is not
available, try to open an IPv4 socket instead when creating and
binding a TCP endpoint.
Problem: Since pull request #1730 was merged, protocol for REQ socket is
checked at the session level and this check does not take into account
the possibility of a request_id being part of the message. Thus the option
ZMQ_REQ_CORRELATE would no longer work.
This is now fixed: the possiblity of a 4 bytes integer being present
before the delimiter frame is taken into account (whether or not this
breaks the REQ/REP RFC is another issue).
A Visual Studio build from master (commit id: dac5b45dfb) using the v140_xp toolset yields a binary that is not XP compatible.
Two libraries contain exports that cannot be found:
- IPHLPAPI.DLL : if_nametoindex
- KERNEL32.DLL : InitializeConditionVariable
The latter export is already dealt with in the file './src/condition_variable.hpp'; however this requires setting the _WIN32_WINNT pre-processor definition.
I am not experienced enough to figure a work around for the 'if_nametoindex' method, so I have created a new pre-processor definition 'ZMQ_HAVE_WINDOWS_TARGET_XP' and removed the calling of the function with the limitation that these builds cannot handle a IPv6 address with an adapter name.
To make it easier for people targeting XP with an MSVC build I have modified the MSBuild property file to add/modify the pre-processor definitions if they are building using a XP targeting tool set; such as v140_xp.
libsodium calls abort() when /dev/urandom can't be found
even if one creates ZeroMQ context before calling chroot()[1].
This happens because crypto gets initialized on handshake,
and at that moment the process is already chroot'ed.
Solution: initialize cryptographic libraries in ctx
randombytes_close() is already there in the destructor.
[1] https://download.libsodium.org/doc/usage/index.html
Problem: when using ZMQ_REQ_RELAXED + ZMQ_REQ_CORRELATE and two 'send' are
executed in a row and no server is available at the time of the sends,
then the internal request_id used to identify messages gets corrupted and
the two messages end up with the same request_id. The correlation no
longer works in that case and you may end up with the wrong message.
Solution: make a copy of the request_id instance member before sending it
down the pipe.
Solution: socket_base_t::in_event cannot do anything useful with
return status of process_commands. Asserting is the wrong solution,
as it is entirely valid to be interrupted or for the context to be
terminated, so discard the value.
Solution: remove statc initialization to NULL of thread.hpp pthread_t
descriptor. There is no portable way to statically initialize a
pthread_t variable.
Solution: The Coverity Static Code Analyzer was used on libzmq code and found
many issues with uninitialized member variables, some redefinition of variables
hidding previous instances of same variable name and a couple of functions
where return values were not checked, even though all other occurrences were
checked (e.g. init_size() return).
Solution: Corrected Toolset setting where needed and inprove compilation speed
by adding defintion of WIN32_LEAN_AND_MEAN prior to any Windows specific
include files, which skips non-essential definitions during compilation.
Solution: disable the warnings on this file only
We use pragmas wrapped in compiler conditionals. This will need
extending to non-gcc/msvc compilers. We could also fix the warnings
in the code, though I suspect it's not really possible.
libzmq used to switch off pedantic checks when using tweetnacl. As
this is now the default, that means pedantic checks are always off.
This is not good.
Solution: in tweetnacl.c alone, use a GCC pragma to disable sign
comparison warnings. We could also clean the code up yet this is
simpler. In other code, we still want those warnings, hence I've
used a pragma rather than global compile option.
Second, use -Wno-long-long all the time, as this warning does not
work with a pragma.
I removed code that set -wno-long-long, for MinGW and Solaris.
Related problem 2: --with-relaxed is badly named
This option switches off pedantic checks, so should be called
--disable-pedantic. 'with' is for optional packages.
Solution: always initialised zmq::options_t class variables arrays to
avoid reading uninitialised data when CURVE is not yet configured and
a getsockopt ZMQ_CURVE_{SERVER | PUBLIC | SECRET]KEY is issued.
It's all over the place.
Solution: remove duplicates and try to move main includes to start
of source. Also, include net/if.h always, so that the code will
compile if ZMQ_HAVE_IFADDRS isn't defined.
- they have no copyright / license statement
- they are in some randomish directory structure
- they are a mix of postable and non-portable files
- they do not conform to conditional compile environment
Overall, it makes it rather more work than needed, in build scripts.
Solution: clean up tweetnacl sauce.
- merged code into single tweetnacl.c and .h
- standard copyright header, DJB to AUTHORS
- moved into src/ along with all other source files
- all system and conditional compilation hidden in these files
- thus, they can be compiled and packaged in all cases
- ZMQ_USE_TWEETNACL is set when we're using built-in tweetnacl
- HAVE_LIBSODIUM is set when we're using external libsodium
It's unclear which we need and in the source code, conditional code
treats tweetnacl as a subclass of libsodium, which is inaccurate.
Solution: redesign the configure/cmake API for this:
* tweetnacl is present by default and cannot be enabled
* libsodium can be enabled using --with-libsodium, which replaces
the built-in tweetnacl
* CURVE encryption can be disabled entirely using --enable-curve=no
The macros we define in platform.hpp are:
ZMQ_HAVE_CURVE 1 // When CURVE is enabled
HAVE_LIBSODIUM 1 // When we are using libsodium
HAVE_TWEETNACL 1 // When we're using tweetnacl (default)
As of this patch, the default build of libzmq always has CURVE
security, and always uses tweetnacl.
And I'm on a reasonably sized laptop. I think allocating INT_MAX
memory is dangerous in a test case.
Solution: expose this as a context option. I've used ZMQ_MAX_MSGSZ
and documented it and implemented the API. However I don't know how
to get the parent context for a socket, so the code in zmq.cpp is
still unfinished.
These options are confusing and redundant. Their names suggest
they apply to the tcp:// transport, yet they are used for all
stream protocols. The methods zmq::set_tcp_receive_buffer and
zmq::set_tcp_send_buffer don't use these values at all, they use
ZMQ_SNDBUF and ZMQ_RCVBUF.
Solution: merge these new options into ZMQ_SNDBUF and ZMQ_RCVBUF.
This means defaulting these two options to 8192, and removing the
new options. We now have ZMQ_SNDBUF and ZMQ_RCVBUF being used both
for TCP socket control, and for input/output buffering.
Note: the default for SNDBUF and RCVBUF are otherwise 4096.
This option has a few issues. The name is long and clumsy. The
functonality is not smooth: one must set both this and
ZMQ_XPUB_VERBOSE at the same time, or things will break mysteriously.
Solution: rename to ZMQ_XPUB_VERBOSER and make an atomic option.
That is, implicitly does ZMQ_XPUB_VERBOSE.
Solution: be more explicit in the code, and in the zmq_recv man
page (which is the most unobvious case). Assert if length is not
zero and buffer is nonetheless null.
There's no value in this as the same pattern is repeated in several
places and it's fair to expect people to understand it.
Solution: revert to the old, one-liner style.
used static_cast<signed int> around WSA_WAIT_FAILED as it is an unsigned implicitly defined as (0xFFFFFFFF ion winbase.h) and causes a comparison error.
removed use of c++11 style initialiser list for 'sockaddr addr { 0 }' and changed it to 'sockaddr addr = { 0 }'
Solution: when iterating over a map and conditionally deleting
elements, an erased iterator gets invalidated. Call erase using postfix
increment on iterator to avoid using an invalid element in the next
iteration.
Solution: parse the value set by the ZMQ_PRE_ALLOCATED_FD sockopt
when creating a new TCP socket and use it if valid.
Add new tests/test_pre_allocated_fd_tcp.cpp unit test.
Solution: parse the value set by the ZMQ_PRE_ALLOCATED_FD sockopt
when creating a new IPC socket and use it if valid.
Add new tests/test_pre_allocated_fd_ipc.cpp unit test.
Solution: add new [set|get]sockopt ZMQ_PRE_ALLOCATED_FD to allow
users to let ZMQ use a pre-allocated file descriptor instead of
allocating a new one. Update [set|get]sockopt documentation and
test accordingly.
The main use case for this feature is a socket-activated systemd
service. For more information about this feature see:
http://0pointer.de/blog/projects/socket-activation.html
