Advise that WinSSL in versions <= XP will not be able to connect to
servers that no longer support the legacy handshakes and algorithms used
by those versions, and to use an alternate backend like OpenSSL instead.
Bug: https://github.com/bagder/curl/issues/253
Reported-by: zenden2k <zenden2k@gmail.com>
In 3013bb6 I had changed cookie export to ignore any-domain cookies,
however the logic I used to do so was incorrect, and would lead to a
busy loop in the case of exporting a cookie list that contained
any-domain cookies. The result of that is worse though, because in that
case the other cookies would not be written resulting in an empty file
once the application is terminated to stop the busy loop.
Make sure that the error buffer is always initialized and simplify the
use of it to make the logic easier.
Bug: https://github.com/bagder/curl/issues/318
Reported-by: sneis
The symbol SSL3_MT_NEWSESSION_TICKET appears to have been introduced at
around openssl 0.9.8f, and the use of it in lib/vtls/openssl.c breaks
builds with older openssls (certainly with 0.9.8b, which is the latest
older version I have to try with).
** WORK-AROUND **
The introduced non-blocking general behaviour for Curl_proxyCONNECT()
didn't work for the data connection establishment unless it was very
fast. The newly introduced function argument makes it operate in a more
blocking manner, more like it used to work in the past. This blocking
approach is only used when the FTP data connecting through HTTP proxy.
Blocking like this is bad. A better fix would make it work more
asynchronously.
Bug: https://github.com/bagder/curl/issues/278
This commit is several drafts squashed together. The changes from each
draft are noted below. If any changes are similar and possibly
contradictory the change in the latest draft takes precedence.
Bug: https://github.com/bagder/curl/issues/244
Reported-by: Chris Araman
%%
%% Draft 1
%%
- return 0 if len == 0. that will have to be documented.
- continue on and process the caches regardless of raw recv
- if decrypted data will be returned then set the error code to CURLE_OK
and return its count
- if decrypted data will not be returned and the connection has closed
(eg nread == 0) then return 0 and CURLE_OK
- if decrypted data will not be returned and the connection *hasn't*
closed then set the error code to CURLE_AGAIN --only if an error code
isn't already set-- and return -1
- narrow the Win2k workaround to only Win2k
%%
%% Draft 2
%%
- Trying out a change in flow to handle corner cases.
%%
%% Draft 3
%%
- Back out the lazier decryption change made in draft2.
%%
%% Draft 4
%%
- Some formatting and branching changes
- Decrypt all encrypted cached data when len == 0
- Save connection closed state
- Change special Win2k check to use connection closed state
%%
%% Draft 5
%%
- Default to CURLE_AGAIN in cleanup if an error code wasn't set and the
connection isn't closed.
%%
%% Draft 6
%%
- Save the last error only if it is an unrecoverable error.
Prior to this I saved the last error state in all cases; unfortunately
the logic to cover that in all cases would lead to some muddle and I'm
concerned that could then lead to a bug in the future so I've replaced
it by only recording an unrecoverable error and that state will persist.
- Do not recurse on renegotiation.
Instead we'll continue on to process any trailing encrypted data
received during the renegotiation only.
- Move the err checks in cleanup after the check for decrypted data.
In either case decrypted data is always returned but I think it's easier
to understand when those err checks come after the decrypted data check.
%%
%% Draft 7
%%
- Regardless of len value go directly to cleanup if there is an
unrecoverable error or a close_notify was already received. Prior to
this change we only acknowledged those two states if len != 0.
- Fix a bug in connection closed behavior: Set the error state in the
cleanup, because we don't know for sure it's an error until that time.
- (Related to above) In the case the connection is closed go "greedy"
with the decryption to make sure all remaining encrypted data has been
decrypted even if it is not needed at that time by the caller. This is
necessary because we can only tell if the connection closed gracefully
(close_notify) once all encrypted data has been decrypted.
- Do not renegotiate when an unrecoverable error is pending.
%%
%% Draft 8
%%
- Don't show 'server closed the connection' info message twice.
- Show an info message if server closed abruptly (missing close_notify).
Some servers will request a client certificate, but not require one.
This change allows libcurl to connect to such servers when using
schannel as its ssl/tls backend. When a server requests a client
certificate, libcurl will now continue the handshake without one,
rather than terminating the handshake. The server can then decide
if that is acceptable or not. Prior to this change, libcurl would
terminate the handshake, reporting a SEC_I_INCOMPLETE_CREDENTIALS
error.
and a conversion to markdown. Removed the lib/README.* files. The idea
being to move toward having INTERNALS as the one and only "book" of
internals documentation.
Added a TOC to top of the document.
When CURL_SOCKET_BAD is returned in the callback, it should be treated
as an error (CURLE_COULDNT_CONNECT) if no other socket is subsequently
created when trying to connect to a server.
Bug: http://curl.haxx.se/mail/lib-2015-06/0047.html
- Try building a chain using issuers in the trusted store first to avoid
problems with server-sent legacy intermediates.
Prior to this change server-sent legacy intermediates with missing
legacy issuers would cause verification to fail even if the client's CA
bundle contained a valid replacement for the intermediate and an
alternate chain could be constructed that would verify successfully.
https://rt.openssl.org/Ticket/Display.html?id=3621&user=guest&pass=guest
Since we do prefix match using given header by application code
against header name pair in format "NAME:VALUE", and VALUE part can
contain ":", we have to careful about existence of ":" in header
parameter. ":" should be allowed to match HTTP/2 pseudo-header field,
and other use of ":" in header must be treated as error, and
curl_pushheader_byname should return NULL. This commit implements
this behaviour.
Prior to this change any-domain cookies (cookies without a domain that
are sent to any domain) were exported with domain name "unknown".
Bug: https://github.com/bagder/curl/issues/292
Follow-up to e8423f9ce1 with discussionis in
https://github.com/bagder/curl/pull/258
This check scans for fopen() with a mode string without 'b' present, as
it may indicate that an FOPEN_* define should rather be used.
- Change fopen calls to use FOPEN_READTEXT instead of "r" or "rt"
- Change fopen calls to use FOPEN_WRITETEXT instead of "w" or "wt"
This change is to explicitly specify when we need to read/write text.
Unfortunately 't' is not part of POSIX fopen so we can't specify it
directly. Instead we now have FOPEN_READTEXT, FOPEN_WRITETEXT.
Prior to this change we had an issue on Windows if an application that
uses libcurl overrides the default file mode to binary. The default file
mode in Windows is normally text mode (translation mode) and that's what
libcurl expects.
Bug: https://github.com/bagder/curl/pull/258#issuecomment-107093055
Reported-by: Orgad Shaneh
Document that if Set-Cookie is used without a domain then the cookie is
sent for any domain and will not be modified.
Bug: http://curl.haxx.se/mail/lib-2015-05/0137.html
Reported-by: Alexander Dyagilev
Previously, after seeing upgrade to HTTP/2, we feed data followed by
upgrade response headers directly to nghttp2_session_mem_recv() in
Curl_http2_switched(). But it turns out that passed buffer, mem, is
part of stream->mem, and callbacks called by
nghttp2_session_mem_recv() will write stream specific data into
stream->mem, overwriting input data. This will corrupt input, and
most likely frame length error is detected by nghttp2 library. The
fix is first copy the passed data to HTTP/2 connection buffer,
httpc->inbuf, and call nghttp2_session_mem_recv().
The CURLOPT_COOKIE doc says it "sets the cookie header explicitly in the
outgoing request(s)." However there seems to be some user confusion
about cookie modification. Document that the cookies set by this option
are not modified by the cookie engine.
Bug: http://curl.haxx.se/mail/lib-2015-05/0115.html
Reported-by: Alexander Dyagilev
This function makes a platform-specific absolute path which uses
backslashes on Windows. This form works when passing it on the
command-line, as well as if the source is on another drive.
Coverity CID 1299424 identified dead code because of checks that could
never equal true (if the mechanism's name was NULL).
Simplified the function by removing a level of pointers and removing the
loop and array that weren't used.
Replace use of assert with code that properly catches bad input at
run-time even in non-debug builds.
This flaw was sort of detected by Coverity CID 1299425 which claimed the
"case RTSPREQ_NONE" was dead code.
Coverity CID 1299426 warned about possible NULL dereference otherwise,
but that would only ever happen if we get invalid HTTP/2 data with
frames for stream 0. Avoid this risk by returning early when stream 0 is
used.
Prior to this change the description for SEC_E_ILLEGAL_MESSAGE was OS
and language specific, and invariably translated to something not very
helpful like: "The message received was unexpected or badly formatted."
Bug: https://github.com/bagder/curl/issues/267
Reported-by: Michael Osipov
With many easy handles using the same connection for multiplexing, it is
important we store and keep the transfer-oriented stuff in the
SessionHandle so that callbacks and callback data work fine even when
many easy handles share the same physical connection.
Previously, when we send all given buffer in data_source_callback, we
return NGHTTP2_ERR_DEFERRED, and nghttp2 library removes this stream
temporarily for writing. This itself is good. If this is the sole
stream in the session, nghttp2_session_want_write() returns zero,
which means that libcurl does not check writeability of the underlying
socket. This leads to very slow upload, because it seems curl only
upload 16k something per 1 second. To fix this, if we still have data
to send, call nghttp2_session_resume_data after nghttp2_session_send.
This makes nghttp2_session_want_write() returns nonzero (if connection
window still opens), and as a result, socket writeability is checked,
and upload speed becomes normal.
Stop curl from failing when non-fatal alert is received during
handshake. This e.g. fixes lots of problems when working with https
sites through proxies.
BoringSSL removed support for direct callers of SSL_CTX_callback_ctrl
and SSL_CTX_ctrl, so move to a way that should work on BoringSSL and
OpenSSL.
re #275
Error: CLANG_WARNING:
lib/http.c:173:16: warning: Value stored to 'http' during its initialization is never read
Error: COMPILER_WARNING:
lib/http.c: scope_hint: In function ‘http_disconnect’
lib/http.c:173:16: warning: unused variable ‘http’ [-Wunused-variable]
.. also make __func__ replacement in multi.
Prior to this change debug builds would fail to build if the compiler
was building pre-c99 and didn't support __func__.
We could get stream ID not in the hash in on_stream_close. For
example, if we decided to reject stream (e.g., PUSH_PROMISE), then we
don't create stream and store it in hash with its stream ID.
This commit requires nghttp2 v1.0.0 to compile, and migrate to v1.0.0,
and utilize recent version of nghttp2 to simplify the code,
First we use nghttp2_option_set_no_recv_client_magic function to
detect nghttp2 v1.0.0. That function only exists since v1.0.0.
Since nghttp2 v0.7.5, nghttp2 ensures header field ordering, and
validates received header field. If it found error, RST_STREAM with
PROTOCOL_ERROR is issued. Since we require v1.0.0, we can utilize
this feature to simplify libcurl code. This commit does this.
Migration from 0.7 series are done based on nghttp2 migration
document. For libcurl, we removed the code sending first 24 bytes
client magic. It is now done by nghttp2 library.
on_invalid_frame_recv callback signature changed, and is updated
accordingly.
to allow code to act differently on the situation.
Also added some more info message for the connection re-use function to
make it clearer when connections are not re-used.
Previously when we do pause because of out of buffer, we just throw
away unread data in connection buffer. This just broke protocol
framing, and I saw occasional FRAME_SIZE_ERROR. This commit fix this
issue by remembering how much data read, and in the next iteration, we
process remaining data.
This commit fixes the bug that streams get stuck if stream gets some
DATA, and stream->closed becomes true at the same time. Previously,
in this condition, after we processed DATA, we are going to try to
read data from underlying transport, but there is no data, and gets
EAGAIN. There was no code path to evaludate stream->closed.
... from the connection struct. The stream one being the 'struct HTTP'
which is kept in the SessionHandle struct (easy handle).
lookup streams for incoming frames in the stream hash, hashing is based
on the stream id and we get the SessionHandle for the incoming stream
that way.
Previously we counted all connections to a specific host name and that
would be used for the CURLMOPT_MAX_HOST_CONNECTIONS check for example,
while servers on different port numbers are normally considered
different "origins" on the web and should thus be considered different
hosts.
All the existing Curl_bundle* functions were only ever used from within
the conncache.c file, so I moved them over and made them static (and
removed the Curl_ prefix).
This avoids unnecessary dynamic allocs and as this also removed the last
users of *hash_alloc() and *hash_destroy(), those two functions are now
removed.
The OpenSSL trace callback is wonderfully undocumented but given a
journey in the source code, it seems the cases were ssl_ver is zero
doesn't follow the same pattern and thus turned out confusing and
misleading. For now, we skip doing any CURLINFO_TEXT logging on those
but keep sending them as CURLINFO_SSL_DATA_OUT/IN.
Also, I added direction to the text info and I edited some functions
slightly.
Bug: https://github.com/bagder/curl/issues/219
Reported-by: Jay Satiro, Ashish Shukla
The missing file has not been autogenerated because a temporary fix was
employed in acinclude.m4 which blocked update. Removed that fix and a recent
version of missing is copied to build root.
That option is not mentioned in the man page of libtoolize 2.4.4.19-fda4.
Moveover, a comment in line 2623 says "--automake is for 1.5 compatibility".
This option is redundant now.
- update default versions of dependencies (except for rare/old platforms)
- update urls
- sync examples makefiles with main ones
- remove line ending space
Make the HTTP headers separated by default for improved security and
reduced risk for information leakage.
Bug: http://curl.haxx.se/docs/adv_20150429.html
Reported-by: Yehezkel Horowitz, Oren Souroujon
Add new option --data-raw which is almost the same as --data but does
not have a special interpretation of the @ character.
Prior to this change there was no (easy) way to pass the @ character as
the first character in POST data without it being interpreted as a
special character.
Bug: https://github.com/bagder/curl/issues/198
Reported-by: Jens Rantil
Elaborated on several of the remaining HTTP/2 parts and made document
use a format that ends up nicer on the web page:
http://curl.haxx.se/dev/roadmap.html
This commit fixes a regression introduced in curl-7_41_0-186-g261a0fe.
It also introduces a regression test 1424 based on tests 78 and 1423.
Reported-by: Viktor Szakats
Bug: https://github.com/bagder/curl/issues/237
When doing HTTP requests Negotiate authenticated, the entire connnection
may become authenticated and not just the specific HTTP request which is
otherwise how HTTP works, as Negotiate can basically use NTLM under the
hood. curl was not adhering to this fact but would assume that such
requests would also be authenticated per request.
CVE-2015-3148
Bug: http://curl.haxx.se/docs/adv_20150422B.html
Reported-by: Isaac Boukris
If a URL is given with a zero-length host name, like in "http://:80" or
just ":80", `fix_hostname()` will index the host name pointer with a -1
offset (as it blindly assumes a non-zero length) and both read and
assign that address.
CVE-2015-3144
Bug: http://curl.haxx.se/docs/adv_20150422D.html
Reported-by: Hanno Böck
The internal libcurl function called sanitize_cookie_path() that cleans
up the path element as given to it from a remote site or when read from
a file, did not properly validate the input. If given a path that
consisted of a single double-quote, libcurl would index a newly
allocated memory area with index -1 and assign a zero to it, thus
destroying heap memory it wasn't supposed to.
CVE-2015-3145
Bug: http://curl.haxx.se/docs/adv_20150422C.html
Reported-by: Hanno Böck
At some point, Firefox has changed and generates different directory
names for the default profile that made this script fail to find them.
Bug: https://github.com/bagder/curl/issues/207
Reported-by: sneakyimp
Add 'gdi32' and 'crypt32' Windows implibs to avoid failure
while building libcurl.dll using the mingw compiler.
The same logic is used in 'src/makefile.m32' when
building curl.exe.
The factor of 8 is a bytes-to-bits conversion factor, but pkt_size and
rate_bps are both in bytes. When using the rate limiting option, curl
waits 8 times too long, and then transfers very quickly until the
average rate reaches the limit. The average rate follows the limit over
time, but the actual traffic is bursty.
Thanks-to: Benjamin Gilbert
The key length in bits will always fit in an unsigned long so the
loss-of-data warning assigning the result of x64 pointer arithmetic to
an unsigned long is unnecessary.
Prior to this change libcurl could show multiple 'CyaSSL: Connecting to'
messages since cyassl_connect_step2 is called multiple times, typically.
The message is superfluous even once since libcurl already informs the
user elsewhere in code that it is connecting.
- cache entries must be also refreshed when they are in use
- have the cache count as inuse reference too, freeing timestamp == 0 special
value
- use timestamp == 0 for CURLOPT_RESOLVE entries which don't get refreshed
- remove CURLOPT_RESOLVE special inuse reference (timestamp == 0 will prevent refresh)
- fix Curl_hostcache_clean - CURLOPT_RESOLVE entries don't have a special
reference anymore, and it would also release non CURLOPT_RESOLVE references
- fix locking in Curl_hostcache_clean
- fix unit1305.c: hash now keeps a reference, need to set inuse = 1
This change is to allow the user's CTX callback to change the minimum
protocol version in the CTX without us later overriding it, as we did
prior to this change.
SSL_CTX_load_verify_locations can return negative values on fail,
therefore to check for failure we check if load is != 1 (success)
instead of if load is == 0 (failure), the latter being incorrect given
that behavior.
Previously in Curl_http2_switched, we called nghttp2_session_mem_recv to
parse incoming data which were already received while curl was handling
upgrade. But we didn't call nghttp2_session_send, and it led to make
curl not send any response to the received frames. Most likely, we
received SETTINGS from server at this point, so we missed opportunity to
send SETTINGS + ACK. This commit adds missing nghttp2_session_send call
in Curl_http2_switched to fix this issue.
Bug: https://github.com/bagder/curl/issues/192
Reported-by: Stefan Eissing
"name =value" is fine and the space should just be skipped.
Updated test 31 to also test for this.
Bug: https://github.com/bagder/curl/issues/195
Reported-by: cromestant
Help-by: Frank Gevaerts
(Curl_cyassl_init)
- Return 1 on success, 0 in failure.
Prior to this change the fail path returned an incorrect value and the
evaluation to determine whether CyaSSL_Init had succeeded was incorrect.
Ironically that combined with the way curl_global_init tests SSL library
initialization (!Curl_ssl_init()) meant that CyaSSL having been
successfully initialized would be seen as that even though the code path
and return value in Curl_cyassl_init were wrong.
Icecast versions 1.3.0 through 1.3.12 would reply with "ICY 200"
under certain conditions:
client_wants_icy_headers (connection_t *con)
{
const char *val;
if (!con)
return 1;
val = get_user_agent (con);
if (!val || !val[0] || strcmp (val, "(null)") == 0)
return 1;
if (con->food.client->use_icy)
return 1;
if (strncasecmp (val, "winamp", 6) == 0)
return 1;
if (strncasecmp (val, "Shoutcast", 9) == 0)
return 1;
return 0;
}
So mainly if there is no 'user agent' or it is '(null)' or contains
'winamp' or 'Shoutcast'.
No mainstream distribution carries Icecast 1.3.x anymore, after all
it was released in 2002 and superseded by Icecast 2.x.
If the handle removed from the multi handle happens to be the one
"owning" the pipeline other transfers will be waiting indefinitely. Now
we move such handles back to connect to have them race (again) for
getting the connection and thus avoid hanging.
Bug: http://curl.haxx.se/bug/view.cgi?id=1465
Reported-by: Jiri Dvorak
This fixes a build failure where openssl and libmetalink are used
together and the system linker does not do implicit linking (e.g.
Fedora 13 and later releases). The MD5 functions required for
metalink support must be pulled in from the openssl crypto library.
This is similar to commit c6e7cbb94e,
which fixes the same sort of problem for NSS builds.
... even if they don't have an associated connection anymore. It could
leave the waiting transfers pending with no active one on the
connection.
Bug: http://curl.haxx.se/bug/view.cgi?id=1465
Reported-by: Jiri Dvorak
The glob_range function used wrong offset (3 instead of 4) for parsing
integer step inside character range specification, which led to 'bad
range' error when using character ranges with explicitly specified step
(such as '[a-z:2]')
(cyassl_connect_step1)
- Use TLS 1.0-1.2 by default when available.
CyaSSL/wolfSSL >= v3.3.0 supports setting a minimum protocol downgrade
version.
cyassl/cyassl@322f79f
This header file must be included after all header files except
memdebug.h, as it does similar memory function redefinitions and can be
similarly affected by conflicting definitions in system or dependent
library headers.
It seems that some systems (e.g. fairly consistently in some recent
Solaris autobuilds) would manage to get to the connect phase before the
progress callback was called, resulting in a CURLE_COULDNT_CONNECT
error. Reworked the test to point at a test server that never returns a
full result so the progress callback always gets a chance to be called
before the transfer can complete in some other way.
The certificates were missing the digitalSignature and keyAgreement
usage types, of which at least digitalSignature was checked by CyaSSL.
This caused the test server in test 310 (among others) to fail the
startup verification and therefore run (see
http://curl.haxx.se/mail/lib-2014-07/0303.html).
CID 1202732 warns on the previous use, although I cannot fine any
problems with it. I'm doing this change only to make the code use a more
familiar approach to accomplish the same thing.
We prematurely changed protocol handler to HTTP/2 which made things very
slow (and wrong).
Reported-by: Stefan Eissing
Bug: https://github.com/bagder/curl/issues/169
- Get rid of this flood of warnings in Windows mingw build:
warning: missing terminating " character
The warning is due to the carriage return. When msysgit checks out files
from the repo by default it converts the line endings to CRLF. Prior to
this change when mkhelp.pl processed the MANUAL and curl.1 in CRLF
format the trailing carriage returns caused unnecessary CR in the
output.
Since we just started make use of free(NULL) in order to simplify code,
this change takes it a step further and:
- converts lots of Curl_safefree() calls to good old free()
- makes Curl_safefree() not check the pointer before free()
The (new) rule of thumb is: if you really want a function call that
frees a pointer and then assigns it to NULL, then use Curl_safefree().
But we will prefer just using free() from now on.
The following functions return immediately if a null pointer was passed.
* Curl_cookie_cleanup
* curl_formfree
It is therefore not needed that a function caller repeats a corresponding check.
This issue was fixed by using the software Coccinelle 1.0.0-rc24.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Bug: https://github.com/bagder/curl/pull/168
(trynextip)
- Don't try the "other" protocol family unless IPv6 is available. In an
IPv4-only build the other family can only be IPv6 which is unavailable.
This change essentially stops IPv4-only builds from attempting the
"happy eyeballs" secondary parallel connection that is supposed to be
used by the "other" address family.
Prior to this change in IPv4-only builds that secondary parallel
connection attempt could be erroneously used by the same family (IPv4)
which caused a bug where every address after the first for a host could
be tried twice, often in parallel. This change fixes that bug. An
example of the bug is shown below.
Assume MTEST resolves to 3 addresses 127.0.0.2, 127.0.0.3 and 127.0.0.4:
* STATE: INIT => CONNECT handle 0x64f4b0; line 1046 (connection #-5000)
* Rebuilt URL to: http://MTEST/
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x64f4b0; line 1083
(connection #0)
* Trying 127.0.0.2...
* STATE: WAITRESOLVE => WAITCONNECT handle 0x64f4b0; line 1163
(connection #0)
* Trying 127.0.0.3...
* connect to 127.0.0.2 port 80 failed: Connection refused
* Trying 127.0.0.3...
* connect to 127.0.0.3 port 80 failed: Connection refused
* Trying 127.0.0.4...
* connect to 127.0.0.3 port 80 failed: Connection refused
* Trying 127.0.0.4...
* connect to 127.0.0.4 port 80 failed: Connection refused
* connect to 127.0.0.4 port 80 failed: Connection refused
* Failed to connect to MTEST port 80: Connection refused
* Closing connection 0
* The cache now contains 0 members
* Expire cleared
curl: (7) Failed to connect to MTEST port 80: Connection refused
The bug was born in commit bagder/curl@2d435c7.
In function Curl_closesocket() in connect.c the call to
Curl_multi_closed() was wrongly omitted if a socket close function
(CURLOPT_CLOSESOCKETFUNCTION) is registered.
That would lead to not removing the socket from the internal hash table
and not calling the multi socket callback appropriately.
Bug: http://curl.haxx.se/bug/view.cgi?id=1493
A signal handler for SIGALRM is installed in Curl_resolv_timeout. It is
configured to interrupt system calls and uses siglongjmp to return into
the function if alarm() goes off.
The signal handler is installed before curl_jmpenv is initialized.
This means that an already installed alarm timer could trigger the
newly installed signal handler, leading to undefined behavior when it
accesses the uninitialized curl_jmpenv.
Even if there is no previously installed alarm available, the code in
Curl_resolv_timeout itself installs an alarm before the environment is
fully set up. If the process is sent into suspend right after that, the
signal handler could be called too early as in previous scenario.
To fix this, the signal handler should only be installed and the alarm
timer only be set after sigsetjmp has been called.
... by using the regular Curl_http_done() method which checks for
that. This makes test 1801 fail consistently with error 56 (which seems
fine) to that test is also updated here.
Reported-by: Ben Darnell
Bug: https://github.com/bagder/curl/issues/166
This makes curl pick better (stronger) ciphers by default. The strongest
available ciphers are fine according to the HTTP/2 spec so an OpenSSL
built curl is no longer rejected by string HTTP/2 servers.
Bug: http://curl.haxx.se/bug/view.cgi?id=1487
...after the method line:
"Since the Host field-value is critical information for handling a
request, a user agent SHOULD generate Host as the first header field
following the request-line." / RFC 7230 section 5.4
Additionally, this will also make libcurl ignore multiple specified
custom Host: headers and only use the first one. Test 1121 has been
updated accordingly
Bug: http://curl.haxx.se/bug/view.cgi?id=1491
Reported-by: Rainer Canavan
When checking for a connection to re-use, a proxy-using request must
check for and use a proxy connection and not one based on the host
name!
Added test 1421 to verify
Bug: http://curl.haxx.se/bug/view.cgi?id=1492
- Change the continuous integration script to use 'make test-full'
instead of just 'make test' so that the diagnostic log output is
printed to stdout when a test fails.
- Change the continuous integration script to use
'./configure --enable-debug' instead of just './configure' so that the
memory analyzer will work during testing.
Prior to this change Travis used its default C test script:
./configure && make && make test
Instead of priting cipher and MAC algorithms names separately, print the
whole cipher suite string which also includes the key exchange algorithm,
along with the negotiated TLS version.
The code used some happy eyeballs logic even _after_ CONNECT has been
sent to a proxy, while the happy eyeball phase is already (should be)
over by then.
This is solved by splitting the multi state into two separate states
introducing the new SENDPROTOCONNECT state.
Bug: http://curl.haxx.se/mail/lib-2015-01/0170.html
Reported-by: Peter Laser
Since 1342a96ecf, a timeout detected in the multi state machine didn't
necesarily clear everything up, like formpost data.
Bug: https://github.com/bagder/curl/issues/147
Reported-by: Michel Promonet
Patched-by: Michel Promonet
SSLeay was the name of the library that was subsequently turned into
OpenSSL many moons ago (1999). curl does not work with the old SSLeay
library since years. This is now reflected by only using USE_OPENSSL in
code that depends on OpenSSL.
Previously, we just ignored error code passed to
on_stream_close_callback and just return 0 (success) after stream
closure even if stream was reset with error. This patch records error
code in on_stream_close_callback, and return -1 and use CURLE_HTTP2
error code on abnormal stream closure.
As the 'error' and 'mute' options are now part of the GlobalConfig,
rather than per Operation, updated the warnf() function to use this
structure rather than the OperationConfig.
Removed the DataExecutionPrevention directive from the project files for
Visual Studio 2008 and above. The XML value in the VC9 project files was
set to "0" (Default) whilst the VC10+ project files contained an empty
XML element.
Visual Studio 2008 introduced support for the address space layout
randomization (ASLR) feature of Windows Vista. However, upgrading the
VC8 project files to VC9 and above disabled this feature.
Removed the RandomizedBaseAddress directive to enabled the default
setting (/DYNAMICBASE). Note: This doesn't appear to have any negative
impact when compiled and ran on Windows XP.
Otherwise it expands to:
echo ""/etc/pki/tls/certs/ca-bundle.crt""
Detected by ShellCheck:
curl-config:74:16: warning: The double quotes around this do
nothing. Remove or escape them. [SC2140]
The vtls layer now checks the return value, so it is no longer necessary
to abort if a random number cannot be provided by NSS. This also fixes
the following Coverity report:
Error: FORWARD_NULL (CWE-476):
lib/vtls/nss.c:1918: var_compare_op: Comparing "data" to null implies that "data" might be null.
lib/vtls/nss.c:1923: var_deref_model: Passing null pointer "data" to "Curl_failf", which dereferences it.
lib/sendf.c:154:3: deref_parm: Directly dereferencing parameter "data".
obj_count can be 1 if the custom read function is set or the stdin
handle is a reference to a pipe. Since the pipe should be handled
using the PeekNamedPipe-check below, the custom read function should
only be used if it is actually enabled.
According to [1]: "Returning 0 will signal end-of-file to the library
and cause it to stop the current transfer."
This change makes the Windows telnet code handle this case accordingly.
[1] http://curl.haxx.se/libcurl/c/CURLOPT_READFUNCTION.html
SSL_CTX_load_verify_locations by default (and if given non-Null
parameters) searches the CAfile first and falls back to CApath. This
allows for CAfile to be a basis (e.g. installed by the package manager)
and CApath to be a user configured directory.
This wasn't reflected by the previous configure constraint which this
patch fixes.
Bug: https://github.com/bagder/curl/pull/139
Correctly check for memcmp() return value (it returns 0 if the strings match).
This is not really important, since curl is going to use http/1.1 anyway, but
it's still a bug I guess.
Set CMAKE_REQUIRED_DEFINITIONS to include definitions needed to get
the winsock2 API from windows.h. Simplify the order of checks to
avoid extra conditions.
Use check_include_file instead of check_include_file_concat to look
for OpenSSL headers. They do not need to participate in a sequence
of dependent system headers. Also they may cause winsock.h to be
included before ws2tcpip.h, causing the latter to not be detected
in the sequence.
Reviewed-by: Brad King <brad.king@kitware.com>
Visual Studio 2005 and above defaults to disabling the startup banner
for the Compiler, Linker and MIDL tools (with /NOLOGO). As such there
is no need to explicitly set the SuppressStartupBanner directive, as
this is a leftover from the VC7 and VC7.1 projects being upgraded to
VC8 and above.
For consistency with other conditionally compiled code in openssl.c,
use OPENSSL_IS_BORINGSSL rather than HAVE_BORINGSSL and try to use
HAVE_BORINGSSL outside of openssl.c when the OpenSSL header files are
not included.
Previously we don't ignore PUSH_PROMISE header fields in on_header
callback. It makes header values mixed with following HEADERS,
resulting protocol error.
Prior to this change the options for exclusive SSL protocol versions did
not actually set the protocol exclusive.
http://curl.haxx.se/mail/lib-2015-01/0002.html
Reported-by: Dan Fandrich
The struct went private in 1.0.2 so we cannot read the version number
from there anymore. Use SSL_version() instead!
Reported-by: Gisle Vanem
Bug: http://curl.haxx.se/mail/lib-2015-02/0034.html
The file number used was wrong. This bug was introduced over 10 years
ago, proving this function isn't used much...
Bug: http://curl.haxx.se/bug/view.cgi?id=1476
Reported-by: Tamir
... by removing generated files from the *_DIST variable [*] and instead
generate them with a .dist suffix, since that is then handled and put
into the release archive by our generic dist-hook.
[*] = 'make distcheck' fails with non-existing files listed there
In preparation for adding the test suite and examples projects renamed
the top level "all" solution files to better describe what they are.
This will also enable us to use "curl" rather than "curlsrc" for the
command line tool solution and project files, which will simplify some
of the configuration.
Defined the DEBUGBUILD pre-processor variable to allow extra logging,
which is particularly useful in debug builds, as we use this and Visual
Studio typically uses _DEBUG.
We could define DEBUBBUILD, in curl_setup.h, when _MSC_VER and _DEBUG is
defined but that would also affect the makefile based builds which we
probably don't want to do.
Modified the Curl_ossl_cert_status_request() function to return FALSE
when built with BoringSSL or when OpenSSL is missing the necessary TLS
extensions.
Commit 7a8b2885e2 made some functions static and removed the public
Curl_ prefix. Unfortunately, it also removed the sasl_ prefix, which
is the naming convention we use in this source file.
curl_sasl.c:1221: error C2065: 'mechtable' : undeclared identifier
This error could also happen for non-SSPI builds when cryptography is
disabled (CURL_DISABLE_CRYPTO_AUTH is defined).
There is an issue with conflicting "struct timeval" definitions with
certain AmigaOS releases and C libraries, depending on what gets
included when. It's a minor difference - the OS one is unsigned,
whereas the common structure has signed elements. If the OS one ends up
getting defined, this causes a timing calculation error in curl.
It's easy enough to resolve this at the curl end, by casting the
potentially errorneous calculation to a signed long.
... of the other cert verification checks so that you can set verifyhost
and verifypeer to FALSE and still check the public key.
Bug: http://curl.haxx.se/bug/view.cgi?id=1471
Reported-by: Kyle J. McKay
Use a dynamicly allocated buffer for the temporary SPN variable similar
to how the SASL GSS-API code does, rather than using a fixed buffer of
2048 characters.
Carrying on from commit 037cd0d991, removed the following unimplemented
instances of curlssl_close_all():
Curl_axtls_close_all()
Curl_darwinssl_close_all()
Curl_cyassl_close_all()
Curl_gskit_close_all()
Curl_gtls_close_all()
Curl_nss_close_all()
Curl_polarssl_close_all()
Fixed the following warning and error from commit 3af90a6e19 when SSL
is not being used:
url.c:2004: warning C4013: 'Curl_ssl_cert_status_request' undefined;
assuming extern returning int
error LNK2019: unresolved external symbol Curl_ssl_cert_status_request
referenced in function Curl_setopt
Use the SECURITY_STATUS typedef rather than a unsigned long for the
QuerySecurityPackageInfo() return and rename the variable as per other
areas of SSPI code.
Also known as "status_request" or OCSP stapling, defined in RFC6066 section 8.
This requires GnuTLS 3.1.3 or higher to build, however it's recommended to use
at least GnuTLS 3.3.11 since previous versions had a bug that caused the OCSP
response verfication to fail even on valid responses.
This option can be used to enable/disable certificate status verification using
the "Certificate Status Request" TLS extension defined in RFC6066 section 8.
This also adds the CURLE_SSL_INVALIDCERTSTATUS error, to be used when the
certificate status verification fails, and the Curl_ssl_cert_status_request()
function, used to check whether the SSL backend supports the status_request
extension.
If the session is still used by active SSL/TLS connections, it
cannot be closed yet. Thus we mark the session as not being cached
any longer so that the reference counting mechanism in
Curl_schannel_shutdown is used to close and free the session.
Reported-by: Jean-Francois Durand
... and make sure we can connect the data connection to a host name that
is longer than 48 bytes.
Also simplifies the code somewhat by re-using the original host name
more, as it is likely still in the DNS cache.
Original-Patch-by: Vojtěch Král
Bug: http://curl.haxx.se/bug/view.cgi?id=1468
Added support for a WITH_CARES option to be used when invoking nmake
via Makefile.vc. This option enables linking against both the DLL and
static versions of the c-ares libraries, as well as the debug and
release varients, depending on the value of DEBUG. The USE_ARES
preprocessor symbol is also defined.
...to avoid a session ID getting cached without certificate checking and
then after a subsequent _enabling_ of the check libcurl could still
re-use the session done without cert checks.
Bug: http://curl.haxx.se/docs/adv_20150108A.html
Reported-by: Marc Hesse
As we get the length for the DN and attribute variables, and we know
the length for the line terminator, pass the length values rather than
zero as this will save Curl_client_write() from having to perform an
additional strlen() call.
curl_ntlm_core.c:146: warning: passing 'DES_cblock' (aka 'unsigned char
[8]') to parameter of type 'char *' converts
between pointers to integer types with different
sign
Rather than duplicate the code in setup_des_key() for OpenSSL and in
extend_key_56_to_64() for non-OpenSSL based crypto engines, as it is
the same, use extend_key_56_to_64() for all engines.
smb.c:780: warning: passing 'char *' to parameter of type 'unsigned
char *' converts between pointers to integer types with
different sign
smb.c:781: warning: passing 'char *' to parameter of type 'unsigned
char *' converts between pointers to integer types with
different sign
smb.c:804: warning: passing 'char *' to parameter of type 'unsigned
char *' converts between pointers to integer types with
different sign
Prefer void for unused parameters, rather than assigning an argument to
itself as a) unintelligent compilers won't optimize it out, b) it can't
be used for const parameters, c) it will cause compilation warnings for
clang with -Wself-assign and d) is inconsistent with other areas of the
curl source code.
Moved our Initialize Security Context return attribute definitions to
the SSPI module, as a) these can be used by other SSPI based providers
and b) the ISC required attributes are defined there.
sockfilt.c:288: warning: conversion to 'DWORD' from 'size_t' may alter
its value
sockfilt.c:291: warning: conversion to 'DWORD' from 'size_t' may alter
its value
sockfilt.c:323: warning: conversion to 'DWORD' from 'size_t' may alter
its value
sockfilt.c:326: warning: conversion to 'DWORD' from 'size_t' may alter
its value
curl_schannel.h:123: warning: right-hand operand of comma expression
has no effect
Some instances of the curlssl_close_all() function were declared with a
void return type whilst others as int. The schannel version returned
CURLE_NOT_BUILT_IN and others simply returned zero, but in all cases the
return code was ignored by the calling function Curl_ssl_close_all().
For the time being and to keep the internal API consistent, changed all
declarations to use a void return type.
To reduce code we might want to consider removing the unimplemented
versions and use a void #define like schannel does.
* Missing initialisation of upload status caused a seg fault
* Missing data termination caused corrupt data to be uploaded
* Data verification should be performed in <upload> element
* Added missing recipient list cleanup
For consistency, as we seem to have a bit of a mixed bag, changed all
instances of ipv4 and ipv6 in comments and documentations to use the
correct case.
Otherwise Curl_ssl_init_certinfo() can fail and set the num_of_certs
member variable to the requested count, which could then be used
incorrectly as libcurl closes down.
The return type for this function was 0 on success and 1 on error. This
was then examined by the calling functions and, in most cases, used to
return CURLE_OUT_OF_MEMORY.
Instead use CURLcode for the return type and return the out of memory
error directly, propagating it up the call stack.
Merge multiple internal arrays into one, even if some variables
will not not be used. They are all created with the number of
file descriptors as their size.
Also fix possible thread handle leak in CloseHandle-loop.
Improves performance of test cases 574 and 575 by 50%.
A value of zero causes the thread to relinquish the remainder
of its time slice to any other thread of equal priority that is
ready to run. If there are no other threads of equal priority
ready to run, the function returns immediately, and the thread
continues execution.
http://msdn.microsoft.com/library/windows/desktop/ms686307.aspx
The return type of this function is a boolean value, and even uses a
bool internally, so use bool in the function declaration as well as
the variables that store the return value, to avoid any confusion.
curl_ntlm_core.c:301: warning: pointer targets in passing argument 2 of
'CryptImportKey' differ in signedness
curl_ntlm_core.c:310: warning: passing argument 6 of 'CryptEncrypt' from
incompatible pointer type
curl_ntlm_core.c:540: warning: passing argument 4 of 'CryptGetHashParam'
from incompatible pointer type
Removed the WIN64 pre-processor definition from the libcurl project
files as:
* WIN64 is not used in our source code
* The curl projects files don't define it
* It isn't required by or used in the platform SDK
* For backwards compatability curl_setup.h defines WIN32
* The compiler automatically defines _WIN64 for x64 builds
Historically Visual Studio projects have defined WIN32, in addition to
the compiler defined _WIN32 definition, and I had incorrectly changed
that to WIN64 for the x64 libcurl builds but not in the curl projects.
As such, it is questionable whether this should be defined or not. For
more information see the following cache of a discussion that took
place on the microsoft.public.vc.mfc newsgroup:
http://www.tech-archive.net/Archive/VC/microsoft.public.vc.mfc/2008-06/msg00074.html
... as it never copies the trailing zero anyway and always just the four
bytes so let's not mislead anyone into thinking it is actually treated
as a string.
Coverity CID: 1260214
lib/setup-vms.h : VAX HP OpenSSL port is ancient, needs help.
More defines to set symbols to uppercase.
src/tool_main.c : Fix parameter to vms_special_exit() call.
packages/vms/ :
backup_gnv_curl_src.com : Fix the error message to have the correct package.
build_curl-config_script.com : Rewrite to be more accurate.
build_libcurl_pc.com : Use tool_version.h now.
build_vms.com : Fix to handle lib/vtls directory.
curl_gnv_build_steps.txt : Updated build procedure documentation.
generate_config_vms_h_curl.com :
* VAX does not support 64 bit ints, so no NTLM support for now.
* VAX HP SSL port is ancient, needs some help.
* Disable NGHTTP2 for now, not ported to VMS.
* Disable UNIX_SOCKETS, not available on VMS yet.
* HP GSSAPI port does not have gss_nt_service_name.
gnv_link_curl.com : Update for new curl structure.
pcsi_product_gnv_curl.com : Set up to optionally do a complete build.
Removed 'next' variable in Curl_convert_form(). Rather than setting it
from 'form->next' and using that to set 'form' after the conversion
just use 'form = form->next' instead.
There was a confusion between these: this commit tries to disambiguate them.
- Scope can be computed from the address itself.
- Scope id is scope dependent: it is currently defined as 1-based local
interface index for link-local scoped addresses, and as a site index(?) for
(obsolete) site-local addresses. Linux only supports it for link-local
addresses.
The URL parser properly parses a scope id as an interface index, but stores it
in a field named "scope": confusion. The field has been renamed into "scope_id".
Curl_if2ip() used the scope id as it was a scope. This caused failures
to bind to an interface.
Scope is now computed from the addresses and Curl_if2ip() matches them.
If redundantly specified in the URL, scope id is check for mismatch with
the interface index.
This commit should fix SF bug #1451.
There was a mix of GlobCode, CURLcode and ints and they were mostly
passing around CURLcode errors. This change makes the functions use only
CURLcode and removes the GlobCode type completely.
Change CURLOPT_TIMEOUT doc to warn that if CURLOPT_TIMEOUT and
CURLOPT_TIMEOUT_MS are both set whichever one is set last is the one
that will be used.
Prior to this change that behavior was only noted in the
CURLOPT_TIMEOUT_MS doc.
By counting from 0 and up instead of backwards like before, we remove
the need for the "funny" check of the unsigned variable when decreased
passed zero. Easier to read and less risk for compiler warnings.
The >= 0 is actually not required, since i underflows and
the for-loop is stopped using the < condition, but this
makes the VS2012 compiler and code analysis happy.
- do not grow memory by doubling its size
- do not leak previously allocated memory if reallocation fails
- replace while-loop with a single check to make sure
that the requested amount of data fits into the buffer
Bug: http://curl.haxx.se/bug/view.cgi?id=1450
Reported-by: Warren Menzer
There is no need to set the 'state' and 'result' member variables to
SMB_REQUESTING (0) and CURLE_OK (0) after the allocation via calloc()
as calloc() initialises the contents to zero.
I don't think both of my fix ups from yesterday were needed to fix the
compilation warning, so remove the one that I think is unnecessary and
let the next Android autobuild prove/disprove it.
For getting the date header its not necessary to access special
pages or even CGI scripts - all pages including the main index
reply with the date header, therefore shortened URLs to domain.
Removed worldtime.com; added pool.ntp.org.
smtp.c:2357 warning: adding 'size_t' (aka 'unsigned long') to a string
does not append to the string
smtp.c:2375 warning: adding 'size_t' (aka 'unsigned long') to a string
does not append to the string
smtp.c:2386 warning: adding 'size_t' (aka 'unsigned long') to a string
does not append to the string
Used array index notation instead.
This fixes compilation issues with compilers that don't support 64-bit
integers through long long or __int64 which was introduced in commit
07b66cbfa4.
Previously USE_NTLM2SESSION would only be defined automatically when
USE_NTRESPONSES wasn't already defined. Separated the two definitions
so that the user can manually set USE_NTRESPONSES themselves but
USE_NTLM2SESSION is defined automatically if they don't define it.
As the OpenSSL and NSS Crypto engines are prefered by the core NTLM
routines, to the Windows Crypt API, don't define USE_WIN32_CRYPT
automatically when either OpenSSL or NSS are in use - doing so would
disable NTLM2Session responses in NTLM type-3 messages.
If the scratch buffer was allocated in a previous call to
Curl_smtp_escape_eob(), a new buffer not allocated in the subsequent
call and no action taken by that call, then an attempt would be made to
try and free the buffer which, by now, would be part of the data->state
structure.
This bug was introduced in commit 4bd860a001.
Fixed a problem with the CRLF. detection when multiple buffers were
used to upload an email to libcurl and the line ending character(s)
appeared at the end of each buffer. This meant any lines which started
with . would not be escaped into .. and could be interpreted as the end
of transmission string instead.
This only affected libcurl based applications that used a read function
and wasn't reproducible with the curl command-line tool.
Bug: http://curl.haxx.se/bug/view.cgi?id=1456
Assisted-by: Patrick Monnerat
parsedate.c:548: warning: 'parsed' may be used uninitialized in this
function
As curl_getdate() returns -1 when parsedate() fails we can initialise
parsed to -1.
This fixes the test 506 torture test. The internal cookie API really
ought to be improved to separate cookie parsing errors (which may be
ignored) with OOM errors (which should be fatal).
Some websites meanwhile refuse to reply to requests from ancient
browsers like IE6, therefore I've comment out this setting, but
also fixed the string to now fake IE8 if someone enables it.
As Windows based autoconf builds don't yet define USE_WIN32_CRYPTO
either explicitly through --enable-win32-cypto or automatically on
_WIN32 based platforms, subsequent builds broke with the following
error message:
"Can't compile NTLM support without a crypto library."
Fixed an issue with the message size calculation where the raw bytes
from the buffer were interpreted as signed values rather than unsigned
values.
Reported-by: Gisle Vanem
Assisted-by: Bill Nagel
As it is often difficult to choose the best description for a single
feature when it spans many commits, updated the descriptions for the
recent SMB/CIFS protocol and GSS-API additions.
Don't use a hard coded size of 4 for the security layer and buffer size
in Curl_sasl_create_gssapi_security_message(), instead, use sizeof() as
we have done in the sasl_gssapi module.
Reduced the amount of free's required for the decoded challenge message
in Curl_sasl_create_gssapi_security_message() as a result of coding it
differently in the sasl_gssapi module.
Sending NTLM/Negotiate header again after successful authentication
breaks the connection with certain Proxies and request types (POST to MS
Forefront).
Prior to this change the 10-at-a-time example showed CURLE_RECV_ERROR
for the sony website because it ends the connection when the request is
missing a user agent.
Mark CURLOPT_UNIX_SOCKET_PATH as string to ensure that it ends up as
option in the file generated by --libcurl.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Add .nf and .fi such that the code gets wrapped in a pre on the web.
Fixed grammar, fixed formatting of the "See also" items.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
The ability to do HTTP requests over a UNIX domain socket has been
requested before, in Apr 2008 [0][1] and Sep 2010 [2]. While a
discussion happened, no patch seems to get through. I decided to give it
a go since I need to test a nginx HTTP server which listens on a UNIX
domain socket.
One patch [3] seems to make it possible to use the
CURLOPT_OPENSOCKETFUNCTION function to gain a UNIX domain socket.
Another person wrote a Go program which can do HTTP over a UNIX socket
for Docker[4] which uses a special URL scheme (though the name contains
cURL, it has no relation to the cURL library).
This patch considers support for UNIX domain sockets at the same level
as HTTP proxies / IPv6, it acts as an intermediate socket provider and
not as a separate protocol. Since this feature affects network
operations, a new feature flag was added ("unix-sockets") with a
corresponding CURL_VERSION_UNIX_SOCKETS macro.
A new CURLOPT_UNIX_SOCKET_PATH option is added and documented. This
option enables UNIX domain sockets support for all requests on the
handle (replacing IP sockets and skipping proxies).
A new configure option (--enable-unix-sockets) and CMake option
(ENABLE_UNIX_SOCKETS) can disable this optional feature. Note that I
deliberately did not mark this feature as advanced, this is a
feature/component that should easily be available.
[0]: http://curl.haxx.se/mail/lib-2008-04/0279.html
[1]: http://daniel.haxx.se/blog/2008/04/14/http-over-unix-domain-sockets/
[2]: http://sourceforge.net/p/curl/feature-requests/53/
[3]: http://curl.haxx.se/mail/lib-2008-04/0361.html
[4]: https://github.com/Soulou/curl-unix-socket
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
test1435: a simple test that checks whether a HTTP request can be
performed over the UNIX socket. The hostname/port are interpreted
by sws and should be ignored by cURL.
test1436: test for the ability to do two requests to the same host,
interleaved with one to a different hostname.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
The variable `$ipvnum` can now contain "unix" besides the integers 4
and 6 since the variable. Functions which receive this parameter
have their `$port` parameter renamed to `$port_or_path` to support a
path to the UNIX domain socket (as a "port" is only meaningful for TCP).
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
If sws is killed it might leave a stale socket file on the filesystem
which would cause an EADDRINUSE error. After this patch, it is checked
whether the socket is really stale and if so, the socket file gets
removed and another bind is executed.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
This extends sws with a --unix-socket option which causes the port to
be ignored (as the server now listens on the path specified by
--unix-socket). This feature will be available in the following patch
that enables checking for UNIX domain socket support.
Proxy support (CONNECT) is not considered nor tested. It does not make
sense anyway, first connecting through a TCP proxy, then let that TCP
proxy connect to a UNIX socket.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
On some platforms curl would crash if no credentials were used. As such
added detection of such a use case to prevent this from happening.
Reported-by: Gisle Vanem
Instead of depending the socket domain type on use_ipv6, specify the
domain type (AF_INET / AF_INET6) as variable. An enum is used here with
switch to avoid compiler warnings in connect_to, complaining that rc
is possibly undefined (which is not possible as socket_domain is
always set).
Besides abstracting the socket type, make the debugging messages be
independent on IP (introduce location_str which points to "port XXXXX").
Rename "ipv_inuse" to "socket_type" and tighten the scope (main).
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
This patch prepares for adding UNIX domain sockets support.
TCP_NODELAY and TCP_KEEPALIVE are specific to TCP/IP sockets, so do not
apply these to other socket types. bindlocal only works for IP sockets
(independent of TCP/UDP), so filter that out too for other types.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
smb.c:398: warning: comparison of integers of different signs:
'ssize_t' (aka 'long') and 'unsigned long'
smb.c:443: warning: comparison of integers of different signs:
'ssize_t' (aka 'long') and 'unsigned long'
smb.c:322: warning: conversion to 'short unsigned int' from 'unsigned
int' may alter its value
smb.c:323: warning: conversion to 'short unsigned int' from 'unsigned
int' may alter its value
smb.c:482: warning: conversion to 'short unsigned int' from 'int' may
alter its value
smb.c:521: warning: conversion to 'unsigned int' from 'curl_off_t' may
alter its value
smb.c:549: warning: conversion to 'unsigned int' from 'curl_off_t' may
alter its value
smb.c:550: warning: conversion to 'short unsigned int' from 'int' may
alter its value
smb.c:489: warning: declaration of 'close' shadows a global declaration
smb.c:511: warning: declaration of 'read' shadows a global declaration
smb.c:528: warning: declaration of 'write' shadows a global declaration
smb.c:212: warning: unused parameter 'done'
smb.c:380: warning: ISO C does not allow extra ';' outside of a function
smb.c:812: warning: unused parameter 'premature'
smb.c:822: warning: unused parameter 'dead'
smb.c:311: warning: conversion from 'unsigned __int64' to 'u_short',
possible loss of data
smb.c:425: warning: conversion from '__int64' to 'unsigned short',
possible loss of data
smb.c:452: warning: conversion from '__int64' to 'unsigned short',
possible loss of data
smb.c:162: error: comma at end of enumerator list
smb.c:469: warning: conversion from 'size_t' to 'unsigned short',
possible loss of data
smb.c:517: warning: conversion from 'curl_off_t' to 'unsigned int',
possible loss of data
smb.c:545: warning: conversion from 'curl_off_t' to 'unsigned int',
possible loss of data
If the scratch buffer already existed when the CRLF conversion was
performed then the buffer pointer would be checked twice for NULL. This
second check is only necessary if the call to malloc() was performed by
the first check.
Whilst I had moved the dot stuffing code from being performed before
CRLF conversion takes place to after it, in commit 4bd860a001, I had
moved it outside the 'when something read' block of code when meant
it could perform the dot stuffing twice on partial send if nread
happened to contain the right values. It also meant the function could
potentially read past the end of buffer. This was highlighted by the
following warning:
warning: `nread' might be used uninitialized in this function
Commit curl-7_23_1-143-g8218064 changed the parameter of
responsive_http_server to accept types other than IPv6 (converting
from a boolean to a string), but only considered the lower-case "ipv6"
and not the "IPv6" variant. This caused all servers to start in IPv4
mode instead.
This patch converts the remaining cases to "ipv6". While not strictly
necessary for the run*server variants, these got also converted for
consistency and to prevent future errors.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
After commit 48d19acb7c the HTTP code would call Curl_nss_force_init()
twice when decoding a NTLM type-2 message, once directly and the other
through the call to Curl_sasl_decode_ntlm_type2_message().
This commit disables pipelining for HTTP/2 or upgraded connections. For
HTTP/2, we do not support multiplexing. In general, requests cannot be
pipelined in an upgraded connection, since it is now different protocol.
Prior to this change when no file descriptors were ready on platforms
other than Windows the multi examples would sleep whatever was in
timeout, which may or may not have been less than the minimum
recommended value [1] of 100ms.
[1]: http://curl.haxx.se/libcurl/c/curl_multi_fdset.html
When the connection code decides to close a socket it informs the multi
system via the Curl_multi_closed function. The multi system may, in
turn, invoke the CURLMOPT_SOCKETFUNCTION function with
CURL_POLL_REMOVE. This happens after the socket has already been
closed. Reorder the code so that CURL_POLL_REMOVE is called before the
socket is closed.
Remove use of an old hack that takes advantage of the auto-dereference
behavior of the if() command to detect if a variable is defined. The
hack has the form:
if("${VAR} MATCHES "^${VAR}$")
where "${VAR}" is a macro argument reference. Use if(DEFINED) instead.
This also avoids warnings for CMake Policy CMP0054 in CMake 3.1.
Debug output 'typo' fix.
Don't print an extra "0x" in
* Pipe broke: handle 0x0x2546d88, url = /
Add debug output.
Print the number of connections in the connection cache when
adding one, and not only when one is removed.
Fix typos in comments.
Updated the usage of some legacy APIs, that are preventing curl from
compiling for Windows Store and Windows Phone build targets.
Suggested-by: Stefan Neis
Feature: http://sourceforge.net/p/curl/feature-requests/82/
Visual Studio 2012 introduced support for Windows Store apps as well as
supporting Windows Phone 8. Introduced build targets that allow more
modern APIs to be used as certain legacy ones are not available on these
new platforms.
Windows does not support using select() for sleeping without a dummy
socket. Instead use Windows' Sleep() and sleep for 100ms which is the
minimum suggested value in the curl_multi_fdset() doc.
Prior to this change the multi examples would exit prematurely since
select() would error instead of sleeping when called without an fd.
Reported-by: Johan Lantz
Bug: http://curl.haxx.se/mail/lib-2014-11/0221.html
Although libcurl would never return CURL_VERSION_KERBEROS4 after 7.33,
so would not be output with --version, removed krb4 from the supported
features output.
Rather than define the function as extern in the source files that use
it, moved the function declaration into the SASL header file just like
the Digest and NTLM clean-up functions.
Additionally, added a function description comment block.
Revert commit 2257deb502 (Cmake: Avoid cycle directory dependencies,
2014-08-22) and add a comment explaining the purpose of the original
code.
The check_library_exists_concat macro is intended to be called multiple
times on a sequence of possibly dependent libraries. Later libraries
may depend on earlier libraries when they are static. They cannot be
safely linked in reverse order on some platforms.
Signed-off-by: Brad King <brad.king@kitware.com>
Revert commit 1269df2e3b (Cmake: Don't check for all headers each
time, 2014-08-15) and add a comment explaining the purpose of the
original code.
The check_include_file_concat macro is intended to be called multiple
times on a sequence of possibly dependent headers. Later headers
may depend on earlier headers to provide declarations. They cannot
be safely included independently on some platforms.
For example, many POSIX APIs document including sys/types.h before some
other headers. Also on some OS X versions sys/socket.h must be included
before net/if.h or the check for the latter will fail.
Signed-off-by: Brad King <brad.king@kitware.com>
This is the only user of the backtick operator in the command. As the
commands will soon not be executed by a shell anymore (but by perl),
replace the command with its output.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Previously if HTTP/2 traffic is appended to HTTP Upgrade response header
(thus they are in the same buffer), the trailing HTTP/2 traffic is not
processed and lost. The appended data is most likely SETTINGS frame.
If it is lost, nghttp2 library complains server does not obey the HTTP/2
protocol and issues GOAWAY frame and curl eventually drops connection.
This commit fixes this problem and now trailing data is processed.
Commit fe0f8967bf fixed a problem with krb5 not being defined as a
supported feature when HAVE_GSSAPI is defined, however, it should
only be included if CURL_DISABLE_CRYPTO_AUTH is not set, like when
SPNEGO is listed as a feature.
Makes test1119 pass when building with cmake.
configurehelp.pm is generated by configure (autotools). As cmake does
not provide a separate variable for the C preprocessor, default to cpp.
Before commit ef24ecde68 ("symbol-scan:
use configure script knowledge about how to run the C preprocessor"),
this tool would also use 'cpp'.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Fix detection of the AsynchDNS feature which not just depends on
pthreads support, but also on whether USE_POSIX_THREADS is set or not.
Caught by test 1014.
This patch adds a new ENABLE_THREADED_RESOLVER option (corresponding to
--enable-threaded-resolver of autotools) which also needs a check for
HAVE_PTHREAD_H.
For symmetry with autotools, CURL_USE_ARES is renamed to ENABLE_ARES
(--enable-ares). Checks that test for the availability actually use
USE_ARES instead as that is the result of whether a-res is available or
not (in practice this does not matter as CARES is marked as required
package, but nevertheless it is better to write the intent).
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Otherwise Curl_gethostname always fails. Windows has gethostname
since Vista according to
http://msdn.microsoft.com/en-us/library/ms738527%28VS.85%29.aspx, but
accordings to byte_bucket's VC 2005 documentation, it is available even
in Windows 95. (possibly after installing a Platform SDK, the
Windows Server 2003 SP1 Platform SDK should be sufficient).
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
I noticed that a patched cmake build would pass tests with a fake local
hostname, but the autotools build skips them:
got unexpected host name back, LD_PRELOAD failed
It turns out that -fvisibility=hidden hides the symbol, and since the
tests are not part of libcurl, it fails too. Just remove the LIBCURL
guard.
Broken since cURL 7.30 (commit 83a42ee20e,
"curl.h: stricter CURL_EXTERN linkage decorations logic").
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
In preparation for moving the NTLM message code into the SASL module,
and separating the native code from the SSPI code, added functions that
simply call the functions in curl_ntlm_msg.c.
Reworked the two sections that discuss modifying the Visual Studio pre-
processor settings, and vc6libcurl.dsw/vc6libcurl.dsp, to remove the
project files references as they have been superseded by a more thorough
set of project files for VC6 through VC12, but to also give the correct
reference to this setting in later versions of Visual Studio.
USE_NTLM would only be defined if: HTTP support was enabled, NTLM and
cryptography weren't disabled, and either a supporting cryptography
library or Windows SSPI was being compiled against.
This means it was not possible to build libcurl without HTTP support
and use NTLM for other protocols such as IMAP, POP3 and SMTP. Rather
than introduce a new SASL pre-processor definition, removed the HTTP
prerequisite just like USE_SPNEGO and USE_KRB5.
Note: Winbind support still needs to be dependent on CURL_DISABLE_HTTP
as it is only available to HTTP at present.
This bug dates back to August 2011 when I started to add support for
NTLM to SMTP.
As the list has gotten a little messy and hard to read, especially with
the introduction of deprecated items, aligned the values and comments
into clean columns and reworked some of the comments in the process.
Added !SSPI to the features list of the HTTP digest tests, as SSPI
based builds now use the Windows SSPI messaging API rather than the
internal functions, and we can't control the random numbers that get
used as part of the digest.
Reworked the input token (challenge message) storage as what is passed
to the buf and desc in the response generation are typically blobs of
data rather than strings, so this is more in keeping with other areas
of the SSPI code, such as the NTLM message functions.
This temporarily breaks HTTP digest authentication in SSPI based builds,
causing CURLE_NOT_BUILT_IN to be returned. A follow up commit will
resume normal operation.
This is very handy when updating the RELEASE-NOTES as then we sometimes
have names added manually in the existing list and we use this script to
update the set.
As the curl command-line tool now includes it's own version of strdup(),
for platforms that don't have it, fixed up the git respository Visual
Studio project file generator to not include the version from lib in the
tool project files, rather than having both lib\strdup.[c|h] and
src\tool_strdup.[c|h] present.
Added forward declaration of digestdata to overcome the following
compilation warning:
warning: 'struct digestdata' declared inside parameter list
Additionally made the ntlmdata forward declaration dependent on
USE_NTLM similar to how digestdata and kerberosdata are.
To provide consistent behaviour between the various HTTP authentication
functions use CURLcode based error codes for Curl_input_digest()
especially as the calling code doesn't use the specific error code just
that it failed.
docs/THANKS-filter is a new filter file for converting contributor names
we get or have recorded in alternative formats to the one we already use
in THANKS. To help us show individual contributors using a single
presentation of their names.
The removed names also appear as:
Andrés García, François Charlier, Gökhan Şengün, Michał Górny, Sébastien
Willemijns, Christopher Conroy, John E. Malmberg, Luca Altea, Peter Su,
S. Moonesamy, Samuel Listopad, Yasuharu Yamada, Karl Moerder
These were previously hard coded, and whilst defined in security.h,
they may or may not be present in old header files given that these
defines were never used in the original code.
Not only that, but there appears to be some ambiguity between the ANSI
and UNICODE NTLM definition name in security.h.
When duplicating a handle, the data to post was duplicated using
strdup() when it could be binary and contain zeroes and it was not even
zero terminated! This caused read out of bounds crashes/segfaults.
Since the lib/strdup.c file no longer is easily shared with the curl
tool with this change, it now uses its own version instead.
Bug: http://curl.haxx.se/docs/adv_20141105.html
CVE: CVE-2014-3707
Reported-By: Symeon Paraschoudis
As implementations are refereed to GSS-API libraries as per the RFC and
GSSAPI typically refers to the SASL authentication mechanism.
...and minor rewording on the same paragraph.
- Prior to this change no SSL minimum version was set by default at
runtime for PolarSSL. Therefore in most cases PolarSSL would probably
have defaulted to a minimum version of SSLv3 which is no longer secure.
The previous condition that checked if the socket was marked as readable
when also adding a writable one, was incorrect and didn't take the pause
bits properly into account.
There were several -Wunused warnings and one duplicate macro definition.
The EXTRA_DEFINES variable of the CurlCheckCSources macro was being
abused ("__unused1\n#undef inline\n#define __unused2", seriously?) to
insert extra C code. Avoid this broken abstraction and use cmake's
check_c_source_compiles directly (works fine with CMake 2.8, maybe
even cmake 2.6).
After cleaning up all related variables (EXTRA_DEFINES,
HEADER_INCLUDES, auxiliary headers_hack), also remove a duplicate
add_headers_include macro and remove duplicate header additions before
the struct timeval check.
Oh, and now the code is converted to use CheckCSourceRuns and
CheckCSourceCompiles, the two curl-specific helpers can be removed.
Unfortunately, the cmake output is now slightly more verbose. Before:
Performing Test int send(int, const void *, size_t, int) (curl_cv_func_send_test)
Performing Test int send(int, const void *, size_t, int) (curl_cv_func_send_test) - Failed
Since check_c_source_compiles prints the varname, now you see:
Performing Test curl_cv_func_send_test
Performing Test curl_cv_func_send_test - Failed
Tested: int send(int, const void *, size_t, int)
Compared cmake output with each other using vimdiff, no functional
differences were found. Tested with GCC 4.9.1 and Clang 3.5.0.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
This patch cleans up the automatically-generated (?) code and fixes one
case that will always fail due to syntax error.
HAVE_GETHOSTBYADDR_R_5_REENTRANT always failed because of a trailing
character ("int length;q"). Several parameter type and unused variable
warnings popped up. This causes a detection failure with -Werror.
Observe that the REENTRANT cases are exactly the same as their
non-REENTRANT cases except for a `_REENTRANT` macro definition.
Merge all these pieces and build one big main function with different
cases, but reusing variables where logical.
For the cases where the parameters where NULL, I looked at
lib/hostip4.c to get an idea of the parameters types.
void-cast variables such as 'rc' to avoid -Wuninitialized errors.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
autotools does not use features.h nor _BSD_SOURCE. As this macro
triggers warnings since glibc 2.20, remove it. It should not have
functional differences.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Additionally, updated "GSSAPI" to "GSS-API" for a Cmake related change
as GSSAPI can be confused with the authentication mechanism rather than
a GSS-API implementation library such as MIT or Heimdal.
A left over from the VC6 project files, so mainly cosmetic in Visual
Studio .NET as it can handle both comma and semi-colon characters for
separating multiple pre-processor definitions.
However, the IDE uses semi-colons if the value is edited, and as such,
this may cause problems in future for anyone updating the files or
merging patches.
Used the Visual Studio IDE to correct the separator character.
..when working from the git repository. This is particularly useful
for single development environments where the project files for all
supported versions of Visual Studio may not be required.
Typically the USE_WINDOWS_SSPI definition would not be used when the
CURL_DISABLE_CRYPTO_AUTH define is, however, it is still a valid build
configuration and, as such, the SASL Kerberos V5 (GSSAPI) authentication
data structures and functions would incorrectly be used when they
shouldn't be.
Introduced a new USE_KRB5 definition that takes into account the use of
CURL_DISABLE_CRYPTO_AUTH like USE_SPNEGO and USE_NTLM do.
Basically since servers often then don't respond well to this and
instead send the full contents and then libcurl would instead error out
with the assumption that the server doesn't support resume. As the data
is then already transfered, this is now considered fine.
Test case 1434 added to verify this. Test case 1042 slightly modified.
Reported-by: hugo
Bug: http://curl.haxx.se/bug/view.cgi?id=1443
Return a more appropriate error, rather than CURLE_OUT_OF_MEMORY when
acquiring the credentials handle fails. This is then consistent with
the code prior to commit f7e24683c4 when log-in credentials were empty.
Fixed the ability to use the current log-in credentials with DIGEST-MD5.
I had previously disabled this functionality in commit 607883f13c as I
couldn't get this to work under Windows 8, however, from testing HTTP
Digest authentication through Windows SSPI and then further testing of
this code I have found it works in Windows 7.
Some further investigation is required to see what the differences are
between Windows 7 and 8, but for now enable this functionality as the
code will return an error when AcquireCredentialsHandle() fails.
HTTP 1.1 is clearly specified to only allow three digit response codes,
and libcurl used sscanf("%3d") for that purpose. This made libcurl
support smaller numbers but not larger. It does now, but we will not
make any specific promises nor document this further since it is going
outside of what HTTP is.
Bug: http://curl.haxx.se/bug/view.cgi?id=1441
Reported-by: Balaji
Don't call CompleteAuthToken() after InitializeSecurityContext() has
returned SEC_I_CONTINUE_NEEDED as this return code only indicates the
function should be called again after receiving a response back from
the server.
This only affected the Digest and NTLM authentication code.
For consistency with other areas of the NTLM code propagate all errors
from Curl_ntlm_core_mk_nt_hash() up the call stack rather than just
CURLE_OUT_OF_MEMORY.
- Remove SSLv3 from SSL default in darwinssl, schannel, cyassl, nss,
openssl effectively making the default TLS 1.x. axTLS is not affected
since it supports only TLS, and gnutls is not affected since it already
defaults to TLS 1.x.
- Update CURLOPT_SSLVERSION doc
... for the local variable name in functions holding the return
code. Using the same name universally makes code easier to read and
follow.
Also, unify code for checking for CURLcode errors with:
if(result) or if(!result)
instead of
if(result == CURLE_OK), if(CURLE_OK == result) or if(result != CURLE_OK)
Prefer usage of Perl modules for sha1 calculation since there
might be systems where openssl is not installed or not in path.
If openssl is used for sha1 calculation then dont rely on cut
since it is usually not available on other systems than Linux.
Initial work to generate a pkg-config and curl-config script. Static
linking (`curl-config --static-libs` and `pkg-config --shared --libs
libcurl`) is broken and therefore disabled.
CONFIGURE_OPTIONS does not make sense for CMake, use an empty string
for now.
At least `curl-config --features` and `curl-config --protocols` work
which is needed by runtests.pl.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
This matches the behavior from autotools. The auxiliary major, minor
and patch components are not needed anymore and therefore removed.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
For compatibility with autoconf, it will be used later for curl-config
and pkg-config. Not all features and or protocols can be enabled as
these are missing additional checks (see new TODOs).
SUPPORT_PROTOCOLS is partially scripted (grep for SUPPORT_PROTOCOLS=)
and manually verified/modified. SUPPORT_FEATURES is manually added.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
CURLOPT_COPYPOSTFIELDS with a given CURLOPT_POSTFIELDSIZE does not
require a trailing zero of the data and by making sure this test doesn't
use one we know it works (combined with valgrind).
This change allows runtests.pl to be run from the CMake builddir:
export srcdir=/tmp/curl/tests;
perl -I$srcdir $srcdir/runtests.pl -l
In order to make this possible, all test cases have been moved from
Makefile.am to Makefile.inc.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
ENABLE_IPV6 depends on HAVE_GETADDRINFO or you will get a
Curl_getaddrinfo_ex error. Enable IPv6 by default, disabling it if
struct sockaddr_in6 is not found in netinet/in.h.
Note that HAVE_GETADDRINFO_THREADSAFE is still not set as it needs more
platform checks even though POSIX requires a thread-safe getaddrinfo.
Verified on Arch Linux x86_64 with glibc 2.20-2 and Linux 3.16-rc7.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Rather than always outputting an empty manual page for the '-M' option,
generate a full manual page as done by autotools. For simplicity in
CMake, always generate the gzipped page as it will not be used anyway
when zlib is not available.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
The 2to3 tool converted socketserver (which I manually fixed up with an
import fallback) and the print(e) line. The xrange option was converted
to range, but it seems better to use the '*' operator here for
simplicity.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
It turned out some features were not enabled in the build since for
example url.c #ifdefs on features that are defined on a per-backend
basis but vtls.h didn't include the backend headers.
CURLOPT_CERTINFO was one such feature that was accidentally disabled.
It tries hard to recognise SDK's on different platforms. On windows MIT
Kerberos installs SDK with other things and puts path into registry.
Heimdal have separate zip archive. On linux pkg-config is tried, then
krb5-config script and finally old-style libs and headers detection.
Command line args:
* CMAKE_USE_GSSAPI - enables GSSAPI detection
* GSS_ROOT_DIR - if set, should point to the root of GSSAPI installation
(the one with include and lib directories)
There is no need for such function. Include_directories propagate by
themselves and having a function with one simple link statement makes
little sense.
Because we prepended libraries to list, CMake had troubles resolving
link directory order as it detected some cycles. Appending to list ensures
that dependencies will preceed dependees.
OpenLDAP might have been build with OpenSSL. Checking for OpenLDAP first
may result in undefined symbols. Of course, the found OpenSSL libraries
must also be linked whenever OpenLDAP is.
Coverity CID 252518. This function is in general far too complicated for
its own good and really should be broken down into several smaller
funcitons instead - but I'm adding this protection here now since it
seems there's a risk the code flow can end up here and dereference a
NULL pointer.
Coverity CID 1241957. Removed the unused argument. As this struct and
pointer now are used only for krb5, there's no need to keep unused
function arguments around.
Coverity CID 1243583. get_url_file_name() cannot fail and return a NULL
file name pointer so skip the check for that - it tricks coverity into
believing it can happen and it then warns later on when we use 'outfile'
without checking for NULL.
Option --pinnedpubkey takes a path to a public key in DER format and
only connect if it matches (currently only implemented with OpenSSL).
Provides CURLOPT_PINNEDPUBLICKEY for curl_easy_setopt().
Extract a public RSA key from a website like so:
openssl s_client -connect google.com:443 2>&1 < /dev/null | \
sed -n '/-----BEGIN/,/-----END/p' | openssl x509 -noout -pubkey \
| openssl rsa -pubin -outform DER > google.com.der
Coverity CID 1154198. This NULL check implies that the pointer _can_ be
NULL at this point, which it can't. Thus it is dead code. It tricks
static analyzers to warn about dereferencing the pointer since the code
seems to imply it can be NULL.
Coverify CID 1157776. Removed a superfluous if() that always evaluated
true (and an else clause that never ran), and then re-indented the
function accordingly.
Coverity CID 1215284. The server name is extracted with
Curl_copy_header_value() and passed in to this function, and
copy_header_value can actually can fail and return NULL.
For private keys, use the first match from: user-specified key file
(if provided), ~/.ssh/id_rsa, ~/.ssh/id_dsa, ./id_rsa, ./id_dsa
Note that the previous code only looked for id_dsa files. id_rsa is
now generally preferred, as it supports larger key sizes.
For public keys, use the user-specified key file, if provided.
Otherwise, try to extract the public key from the private key file.
This means that passing --pubkey is typically no longer required,
and makes the key-handling behavior more like OpenSSH.
Coverity CID 1202836. If the proxy environment variable returned an empty
string, it would be leaked. While an empty string is not really a proxy, other
logic in this function already allows a blank string to be returned so allow
that here to avoid the leak.
Coverity CID 1215287. There's a potential risk for a memory leak in
here, and moving the free call to be unconditional seems like a cheap
price to remove the risk.
Coverity CID 1215296. There's a potential risk for a memory leak in
here, and moving the free call to be unconditional seems like a cheap
price to remove the risk.
Coverity detected this. CID 1241954. When Curl_poll() returns a negative value
'mcode' was uninitialized. Pretty harmless since this is debug code only and
would at worst cause an error to _not_ be returned...
Mostly because we use C strings and they end at a binary zero so we know
we can't open a file name using an embedded binary zero.
Reported-by: research@g0blin.co.uk
The switch to using Curl_expire_latest() in commit cacdc27f52 was a
mistake and was against the advice even mentioned in that commit. The
comparison in asyn-thread.c:Curl_resolver_is_resolved() makes
Curl_expire() the suitable function to use.
Bug: http://curl.haxx.se/bug/view.cgi?id=1426
Reported-By: graysky
As we weren't using the correct phonetic description and doing it correctly
involves funny letters that I'm sure will cause problems for people in a text
document so I instead rephrased it and link to a WAV file with a person
actually saying 'curl'.
Reported-By: Dimitar Boevski
Previously we did not handle EOF from underlying transport socket and
wrongly just returned error code CURL_AGAIN from http2_recv, which
caused busy loop since socket has been closed. This patch adds the
code to handle EOF situation and tells the upper layer that we got
EOF.
Removed ISC_REQ_* flags from calls to InitializeSecurityContext to fix
bug in NTLM handshake for HTTP proxy authentication.
NTLM handshake for HTTP proxy authentication failed with error
SEC_E_INVALID_TOKEN from InitializeSecurityContext for certain proxy
servers on generating the NTLM Type-3 message.
The flag ISC_REQ_CONFIDENTIALITY seems to cause the problem according
to the observations and suggestions made in a bug report for the
QT project (https://bugreports.qt-project.org/browse/QTBUG-17322).
Removing all the flags solved the problem.
Bug: http://curl.haxx.se/mail/lib-2014-08/0273.html
Reported-by: Ulrich Telle
Assisted-by: Steve Holme, Daniel Stenberg
I use the curl repo mainly on Windows with the typical Windows git
checkout which converts the LF line endings in the curl repo to CRLF
automatically on checkout. The automatic conversion is not done on files
in the repo with mixed line endings. I recently noticed some weird
output with projects/build-openssl.bat that I traced back to mixed line
endings, so I scanned the repo and there are files (excluding the
test data) that have mixed line endings.
I used this command below to do the scan. Unfortunately it's not as easy
as git grep, at least not on Windows. This gets the names of all the
files in the repo's HEAD, gets each of those files raw from HEAD, checks
for mixed line endings of both LF and CRLF, and prints the name if
mixed. I excluded path tests/data/test* because those can have mixed
line endings if I understand correctly.
for f in `git ls-tree --name-only --full-tree -r HEAD`;
do if [ -n "${f##tests/data/test*}" ];
then git show "HEAD:$f" | \
perl -0777 -ne 'exit 1 if /([^\r]\n.*\r\n)|(\r\n.*[^\r]\n)/';
if [ $? -ne 0 ];
then echo "$f";
fi;
fi;
done
Added support for Kerberos 5 to the email protocols following the recent
additions in 7.38.0.
Removed Kerberos 4 as this has been gone for a while now.
As a sort of step forward, this script will now first try to get the
data from the HTTPS URL using curl, and only if that fails it will
switch back to the HTTP transfer using perl's native LWP functionality.
To reduce the risk of this script being tricked.
Using HTTPS to get a cert bundle introduces a chicken-and-egg problem so
we can't really ever completely disable HTTP, but chances are that most
users already have a ca cert bundle that trusts the mozilla.org site
that this script downloads from.
A future version of this script will probably switch to require a
dedicated "insecure" command line option to allow downloading over HTTP
(or unverified HTTPS).
By not detecting and rejecting domain names for partial literal IP
addresses properly when parsing received HTTP cookies, libcurl can be
fooled to both send cookies to wrong sites and to allow arbitrary sites
to set cookies for others.
CVE-2014-3613
Bug: http://curl.haxx.se/docs/adv_20140910A.html
Only minor edits to make it generate nice HTML output using markdown, as
this document serves both in source release tarballs as on the web site.
URL: http://curl.haxx.se/docs/sslcerts.html
Historically the default "unknown" value for progress.size_dl and
progress.size_ul has been zero, since these values are initialized
implicitly by the calloc that allocates the curl handle that these
variables are a part of. Users of curl that install progress
callbacks may expect these values to always be >= 0.
Currently it is possible for progress.size_dl and progress.size_ul
to by set to a value of -1, if Curl_pgrsSetDownloadSize() or
Curl_pgrsSetUploadSize() are passed a "size" of -1 (which a few
places currently do, and a following patch will add more). So
lets update Curl_pgrsSetDownloadSize() and Curl_pgrsSetUploadSize()
so they make sure that these variables always contain a value that
is >= 0.
Updates test579 and test599.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
As the current element in the list is free()d by Curl_llist_remove(),
when the associated connection is pending, reworked the loop to avoid
accessing the next element through e->next afterward.
SecCertificateCopyPublicKey() is not available on iPhone. Use
CopyCertSubject() instead to see if the certificate returned by
SecCertificateCreateWithData() is valid.
Reported-by: Toby Peterson
... as the struct is free()d in the end anyway. It was first pointed out
to me that one of the ->msglist assignments were supposed to have been
->pending but was a copy and paste mistake when I realized none of the
clearing of pointers had to be there.
... instead of scanning through all handles, stash only the actual
handles that are in that state in the new ->pending list and scan that
list only. It should be mostly empty or very short. And only used for
pipelining.
This avoids a rather hefty slow-down especially notable if you add many
handles to the same multi handle. Regression introduced in commit
0f147887 (version 7.30.0).
Bug: http://curl.haxx.se/mail/lib-2014-07/0206.html
Reported-by: David Meyer
Forwards the setting as minimum ssl version (if set) to polarssl. If
the server does not support the requested version the SSL Handshake will
fail.
Bug: http://curl.haxx.se/bug/view.cgi?id=1419
SecCertificateCreateWithData() returns a non-NULL SecCertificateRef even
if the buffer holds an invalid or corrupt certificate. Call
SecCertificateCopyPublicKey() to make sure cacert is a valid
certificate.
Introducing Curl_expire_latest(). To be used when we the code flow only
wants to get called at a later time that is "no later than X" so that
something can be checked (and another timeout be added).
The low-speed logic for example could easily be made to set very many
expire timeouts if it would be called faster or sooner than what it had
set its own timer and this goes for a few other timers too that aren't
explictiy checked for timer expiration in the code.
If there's no condition the code that says if(time-passed >= TIME), then
Curl_expire_latest() is preferred to Curl_expire().
If there exists such a condition, it is on the other hand important that
Curl_expire() is used and not the other.
Bug: http://curl.haxx.se/mail/lib-2014-06/0235.html
Reported-by: Florian Weimer
While waiting for a host resolve, check if the host cache may have
gotten the name already (by someone else), for when the same name is
resolved by several simultanoues requests.
The resolver thread occasionally gets stuck in getaddrinfo() when the
DNS or anything else is crappy or slow, so when a host is found in the
DNS cache, leave the thread alone and let itself cleanup the mess.
If the --cacert option is used with a CA certificate bundle that
contains multiple CA certificates, iterate through it, adding each
certificate as a trusted root CA.
When enabling CURLOPT_NOBODY, libcurl effectively switches off upload
mode and will do a download (without a body). This is now better
explained in this man page.
Bug: http://curl.haxx.se/mail/lib-2014-08/0236.html
Reported-by: John Coffey
I re-ran contributors.sh on all changes since 7.10 and I found these
contributors who are mentioned in the commits but never were added to
THANKS before!
I also removed a couple of duplicates (mostly due to different
spellings).
By default, configure script assumes that libcurl will use the
HP-supplied GSS-API implementation which does not have krb5-config.
If a dev needs a more recent version which has that config script,
the change will allow to pass an appropriate GSSAPI_ROOT.
This is usually due to failed auth. There's no point in us keeping such
a connection alive since it shouldn't be re-used anyway.
Bug: http://curl.haxx.se/bug/view.cgi?id=1381
Reported-by: Marcel Raad
One header at a time is the right way. Apart from that the output on
windows goes from:
...
-- Looking for include files I:/src/libssh2-1.4.3/include/libssh2.h, ws2tcpip.h
-- Looking for include files I:/src/libssh2-1.4.3/include/libssh2.h, ws2tcpip.h
- found
-- Looking for 3 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., wins
ock2.h
-- Looking for 3 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., wins
ock2.h - found
-- Looking for 4 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., stdi
o.h
-- Looking for 4 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., stdi
o.h - found
-- Looking for 5 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., wind
ows.h
-- Looking for 5 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., wind
ows.h - found
-- Looking for 6 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., wins
ock.h
-- Looking for 6 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., wins
ock.h - found
-- Looking for 7 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., sys/
filio.h
-- Looking for 7 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., sys/
filio.h - not found
-- Looking for 7 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., sys/
ioctl.h
-- Looking for 7 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., sys/
ioctl.h - not found
-- Looking for 7 include files I:/src/libssh2-1.4.3/include/libssh2.h, ..., sys/
resource.h
...
To much nicer:
...
-- Looking for ws2tcpip.h
-- Looking for ws2tcpip.h - found
-- Looking for winsock2.h
-- Looking for winsock2.h - found
-- Looking for stdio.h
-- Looking for stdio.h - found
-- Looking for windows.h
-- Looking for windows.h - found
-- Looking for winsock.h
-- Looking for winsock.h - found
-- Looking for sys/filio.h
-- Looking for sys/filio.h - not found
-- Looking for sys/ioctl.h
-- Looking for sys/ioctl.h - not found
-- Looking for sys/resource.h
At this point I can build libcurl with OpenSSL, OpenLDAP and LibSSH2.
Supported protocols are at least:
HTTP, HTTPS, FTP, SFTP, TFTP, LDAP, LDAPS, POP3, SMTP
(those are the ones we have regression tests for
in our product's testsuite)
CMake 2.6 is already a bit old. Many bugs have been fixed since
its release. We use 2.8 in our company and we have no intention
of polluting our environment with old software, so 2.6 would
not be tested. This shouldn't be a problem since all one need
to build CMake from source is C and C++ compiler.
This was done to make sure NTLM state that is bound to a connection
doesn't survive and gets used for the subsequent request - but
disconnects can also be done to for example make room in the connection
cache and thus that connection is not strictly related to the easy
handle's current operation.
The http authentication state is still kept in the easy handle since all
http auth _except_ NTLM is connection independent and thus survive over
multiple connections.
Bug: http://curl.haxx.se/mail/lib-2014-08/0148.html
Reported-by: Paras S
Problem: if CURLOPT_FORBID_REUSE is set, requests using NTLM failed
since NTLM requires multiple requests that re-use the same connection
for the authentication to work
Solution: Ignore the forbid reuse flag in case the NTLM authentication
handshake is in progress, according to the NTLM state flag.
Fixed known bug #77.
A conditionally compiled block in connect.c references WinSock 2
symbols, but used `#ifdef HAVE_WINSOCK_H` instead of `#ifdef
HAVE_WINSOCK2_H`.
Bug: http://curl.haxx.se/mail/lib-2014-08/0155.html
The URL is not a property of the connection so it should not be freed in
the connection disconnect but in the Curl_close() that frees the easy
handle.
Bug: http://curl.haxx.se/mail/lib-2014-08/0148.html
Reported-by: Paras S
Corrected a number of the error codes that can be returned from the
Curl_sasl_create_gssapi_security_message() function when things go
wrong.
It makes more sense to return CURLE_BAD_CONTENT_ENCODING when the
inbound security challenge can't be decoded correctly or doesn't
contain the KERB_WRAP_NO_ENCRYPT flag and CURLE_OUT_OF_MEMORY when
EncryptMessage() fails. Unfortunately the previous error code of
CURLE_RECV_ERROR was a copy and paste mistakes on my part and should
have been correct in commit 4b491c675f :(
Updated "FTP 4.6 GSSAPI via Windows SSPI" and "SASL 14.1 Other
authentication mechanisms" following recent additions.
Added SASL 14.2 GSSAPI via GSS-API libraries.
This repeats what has already been documented in both the curl manpage
and CURLOPT_USERPWD documentation but is provided here for completeness
as someone may not especially read the latter when using libcurl.
Added information about Kerberos V5 requiring the domain part in the
user name.
Mentioned that the user name can be specified in UPN format, and not
just in Down-Level Logon Name format, following the information
added in commit 7679cb3fa8 reworking the exisitng information in the
process.
... to handle "*/[total]". Also, removed the strange hack that made
CURLOPT_FAILONERROR on a 416 response after a *RESUME_FROM return
CURLE_OK.
Reported-by: Dimitrios Siganos
Bug: http://curl.haxx.se/mail/lib-2014-06/0221.html
In preparation for the upcoming SSPI implementation of GSSAPI
authentication, moved the definition of KERB_WRAP_NO_ENCRYPT from
socks_sspi.c to curl_sspi.h allowing it to be shared amongst other
SSPI based code.
... as mxr.mozilla.org is due to be retired.
The new host doesn't support If-Modified-Since nor ETags, meaning that
the script will now defer to download and do a post-transfer checksum
check to see if a new output is to be generated. The new output format
will hold the SHA1 checksum of the source file for that purpose.
We call this version 1.22
Reported-by: Ed Morley
Bug: http://curl.haxx.se/bug/view.cgi?id=1409
Bringing back the old functionality that was mistakenly removed when the
connection cache was remade. When creating a new connection, all the
existing ones are checked and those that are known to be dead get
disconnected for real and removed from the connection cache. It helps
the cache from holding on to very many stale connections and aids in
keeping down the number of system sockets in wait states.
Help-by: Jonatan Vela <jonatan.vela@ergon.ch>
Bug: http://curl.haxx.se/mail/lib-2014-06/0189.html
Curl_poll and Curl_wait_ms require the fix applied to Curl_socket_check
in commits b61e8b8 and c771968:
When poll or select are interrupted and coincides with the timeout
elapsing, the functions return -1 indicating an error instead of 0 for
the timeout.
Given the SSPI package info query indicates a token size of 4096 bytes,
updated to use a dynamic buffer for the response message generation
rather than a fixed buffer of 1024 bytes.
Updated to use a dynamic buffer for the SPN generation via the recently
introduced Curl_sasl_build_spn() function rather than a fixed buffer of
1024 characters, which should have been more than enough, but by using
the new function removes the need for another variable sname to do the
wide character conversion in Unicode builds.
Updated Curl_sasl_create_digest_md5_message() to use a dynamic buffer
for the SPN generation via the recently introduced Curl_sasl_build_spn()
function rather than a fixed buffer of 128 characters.
Curl_sasl_create_digest_md5_message() would simply cast the SPN variable
to a TCHAR when calling InitializeSecurityContext(). This meant that,
under Unicode builds, it would not be valid wide character string.
Updated to use the recently introduced Curl_sasl_build_spn() function
which performs the correct conversion for us.
Various parts of the libcurl source code build a SPN for inclusion in
authentication data. This information is either used by our own native
generation routines or passed to authentication functions in third-party
libraries such as SSPI. However, some of these instances use fixed
buffers rather than dynamically allocated ones and not all of those that
should, convert to wide character strings in Unicode builds.
Implemented a common function that generates a SPN and performs the
wide character conversion where necessary.
Following the recent changes and in attempt to align the SSPI based
authentication code performed the following:
* Use NULL and SECBUFFVERSION rather than hard coded constants.
* Avoid comparison of zero in if statements.
* Standardised the buf and desc setup code.
Given the SSPI package info query indicates a token size of 2888 bytes,
and as with the Winbind code and commit 9008f3d56, use a dynamic buffer
for the Type-1 and Type-3 message generation rather than a fixed buffer
of 1024 bytes.
Just as with the SSPI implementations of Digest and Negotiate added a
package info query so that libcurl can a) return a more appropriate
error code when the NTLM package is not supported and b) it can be of
use later to allocate a dynamic buffer for the Type-1 and Type-3
output tokens rather than use a fixed buffer of 1024 bytes.
OPENSSL_config() is "strongly recommended" to use but unfortunately that
function makes an exit() call on wrongly formatted config files which
makes it hard to use in some situations. OPENSSL_config() itself calls
CONF_modules_load_file() and we use that instead and we ignore its
return code!
Reported-by: Jan Ehrhardt
Bug: http://curl.haxx.se/bug/view.cgi?id=1401
If the server rejects our authentication attempt and curl hasn't
called CompleteAuthToken() then the status variable will be
SEC_I_CONTINUE_NEEDED and not SEC_E_OK.
As such the existing detection mechanism for determining whether or not
the authentication process has finished is not sufficient.
However, the WWW-Authenticate: Negotiate header line will not contain
any data when the server has exhausted the negotiation, so we can use
that coupled with the already allocated context pointer.
To prevent infinite loop in readwrite_data() function when stream is
reset before any response body comes, reset closed flag to false once
it is evaluated to true.
"Expect: 100-continue", which was once deprecated in HTTP/2, is now
resurrected in HTTP/2 draft 14. This change adds its support to
HTTP/2 code. This change also includes stricter header field
checking.
Previously it only returned a CURLcode for errors, which is when it
returns a different size than what was passed in to it.
The http2 code only checked the curlcode and thus failed.
Under these circumstances, the connection hasn't been fully established
and smtp_connect hasn't been called, yet smtp_done still calls the state
machine which dereferences the NULL conn pointer in struct pingpong.
This now provides a weak random function since PolarSSL doesn't have a
quick and easy way to provide a good one. It does however provide the
framework to make one so it _can_ and _should_ be done...
To force each backend implementation to really attempt to provide proper
random. If a proper random function is missing, then we can explicitly
make use of the default one we use when TLS support is missing.
This commit makes sure it works for darwinssl, gnutls, nss and openssl.
Removing defines, even obsolete ones that haven't been used for a very
long time, still break a lot of applications.
Bug: https://github.com/bagder/curl/pull/106
If a non-standard $TESTDIR is used the file may not be necessary.
Previously a "missing" file resulted in the warning:
readline() on closed filehandle D at ./runtests.pl line 4940.
warning C4267: '=' : conversion from 'size_t' to 'long', possible loss
of data
The member connection_id of struct connectdata is a long (always a
32-bit signed integer on Visual C++) and the member next_connection_id
of struct conncache is a size_t, so one of them should be changed to
match the other.
This patch the size_t in struct conncache to long (the less invasive
change as that variable is only ever used in a single code line).
Bug: http://curl.haxx.se/bug/view.cgi?id=1399
1 - fixes the warnings when built without http2 support
2 - adds CURLE_HTTP2, a new error code for errors detected by nghttp2
basically when they are about http2 specific things.
CyaSSL 3.0.0 returns a unique error code if no CA cert is available,
so translate that into CURLE_SSL_CACERT_BADFILE when peer verification
is requested.
- Replace CURLAUTH_GSSNEGOTIATE with CURLAUTH_NEGOTIATE
- CURL_VERSION_GSSNEGOTIATE is deprecated which
is served by CURL_VERSION_SSPI, CURL_VERSION_GSSAPI and
CURUL_VERSION_SPNEGO now.
- Remove display of feature 'GSS-Negotiate'
This reverts commit cb3e6dfa35 and instead fixes the problem
differently.
The reverted commit addressed a test failure in test 1021 by simplifying
and generalizing the code flow in a way that damaged the
performance. Now we modify the flow so that Curl_proxyCONNECT() again
does as much as possible in one go, yet still do test 1021 with and
without valgrind. It failed due to mistakes in the multi state machine.
Bug: http://curl.haxx.se/bug/view.cgi?id=1397
Reported-by: Paul Saab
The curl tool project files for VC8 to VC12 would set this setting to
$(IntDir) which is the Visual Studio default value. To avoid confusion
when viewing settings from within Visual Studio and for consistency
with the libcurl project files removed this setting.
Conflicts:
projects/Windows/VC10/src/curlsrc.tmpl
projects/Windows/VC11/src/curlsrc.tmpl
projects/Windows/VC12/src/curlsrc.tmpl
projects/Windows/VC8/src/curlsrc.tmpl
projects/Windows/VC9/src/curlsrc.tmpl
The curl tool project files for VC7 to VC12 would set this settings to
$(IntDir)$(TargetName).pch which is the Visual Studio default value. To
avoid confusion when viewing settings from within Visual Studio and for
consistency with the libcurl project files removed this setting.
Conflicts:
projects/Windows/VC10/src/curlsrc.tmpl
projects/Windows/VC11/src/curlsrc.tmpl
projects/Windows/VC12/src/curlsrc.tmpl
projects/Windows/VC8/src/curlsrc.tmpl
projects/Windows/VC9/src/curlsrc.tmpl
The curl tool project files for VC7 to VC12 would set these settings to
$(IntDir) which is the Visual Studio default value. To avoid confusion
when viewing settings from within Visual Studio and for consistency
with the libcurl project files removed these two settings.
This prevents targets like tool_hugehelp.c from leaving around
half-constructed files if the rule fails with GNU make.
Reported-by: Rafaël Carré <funman@videolan.org>
It's wrong to assume that we can send a single SPNEGO packet which will
complete the authentication. It's a *negotiation* — the clue is in the
name. So make sure we handle responses from the server.
Curl_input_negotiate() will already handle bailing out if it thinks the
state is GSS_S_COMPLETE (or SEC_E_OK on Windows) and the server keeps
talking to us, so we should avoid endless loops that way.
This is the correct way to do SPNEGO. Just ask for it
Now I correctly see it trying NTLMSSP authentication when a Kerberos ticket
isn't available. Of course, we bail out when the server responds with the
challenge packet, since we don't expect that. But I'll fix that bug next...
This is just fundamentally broken. SPNEGO (RFC4178) is a protocol which
allows client and server to negotiate the underlying mechanism which will
actually be used to authenticate. This is *often* Kerberos, and can also
be NTLM and other things. And to complicate matters, there are various
different OIDs which can be used to specify the Kerberos mechanism too.
A SPNEGO exchange will identify *which* GSSAPI mechanism is being used,
and will exchange GSSAPI tokens which are appropriate for that mechanism.
But this SPNEGO implementation just strips the incoming SPNEGO packet
and extracts the token, if any. And completely discards the information
about *which* mechanism is being used. Then we *assume* it was Kerberos,
and feed the token into gss_init_sec_context() with the default
mechanism (GSS_S_NO_OID for the mech_type argument).
Furthermore... broken as this code is, it was never even *used* for input
tokens anyway, because higher layers of curl would just bail out if the
server actually said anything *back* to us in the negotiation. We assume
that we send a single token to the server, and it accepts it. If the server
wants to continue the exchange (as is required for NTLM and for SPNEGO
to do anything useful), then curl was broken anyway.
So the only bit which actually did anything was the bit in
Curl_output_negotiate(), which always generates an *initial* SPNEGO
token saying "Hey, I support only the Kerberos mechanism and this is its
token".
You could have done that by manually just prefixing the Kerberos token
with the appropriate bytes, if you weren't going to do any proper SPNEGO
handling. There's no need for the FBOpenSSL library at all.
The sane way to do SPNEGO is just to *ask* the GSSAPI library to do
SPNEGO. That's what the 'mech_type' argument to gss_init_sec_context()
is for. And then it should all Just Work™.
That 'sane way' will be added in a subsequent patch, as will bug fixes
for our failure to handle any exchange other than a single outbound
token to the server which results in immediate success.
Before GnuTLS 3.3.6, the gnutls_x509_crt_check_hostname() function
didn't actually check IP addresses in SubjectAltName, even though it was
explicitly documented as doing so. So do it ourselves...
The old way using getpwuid could cause problems in programs that enable
reading from netrc files simultaneously in multiple threads.
Reported-by: David Woodhouse
The curl tool project files for VC7 to VC12 would override the default
setting with the output filename being the same as the linker PDB file.
As such the compiler file would be overwritten with the linker file
for all debug builds.
To avoid this overwrite and for consistency with the libcurl project
files, removed the setting to force the default filename to be used.
The AES-GCM ciphers were added to GnuTLS as late as ver. 3.0.1 but
the code path in which they're referenced here is only ever used for
somewhat older GnuTLS versions. This caused undeclared identifier errors
when compiling against those.
This seems to have become necessary for SRP support to work starting
with GnuTLS ver. 2.99.0. Since support for SRP was added to GnuTLS
before the function that takes this priority string, there should be no
issue with backward compatibility.
When an error has been detected, skip the final forced call to the
progress callback by making sure to pass the current return code
variable in the Curl_done() call in the CURLM_STATE_DONE state.
This avoids the "extra" callback that could occur even if you returned
error from the progress callback.
Bug: http://curl.haxx.se/mail/lib-2014-06/0062.html
Reported by: Jonathan Cardoso Machado
This prevents valgrind from reporting possibly lost memory that NSPR
uses for file descriptor cache and other globally allocated internal
data structures.
With all the new individual option man pages created, this now refers to
each separate one instead of duplicaing the info. Also makes this page
easier to overview.
We now provide the "real" names in the CINIT() macro setup for CURLOPT_*
symbols, and we provide backwards compatibility defines for the old
symbols as defines instead of vice versa.
This allows us to better use the CINIT() list to check for existing and
current option names.
The static connection counter caused a race condition. Moving the
connection id counter into conncache solves it, as well as simplifying
the related logic.
They were added because of an older code path that used allocations and
should not have been left in the code. With this change the logic goes
back to how it was.
Curl_rand() will return a dummy and repatable random value for this
case. Makes it possible to write test cases that verify output.
Also, fake timestamp with CURL_FORCETIME set.
Only when built debug enabled of course.
Curl_ssl_random() was not used anymore so it has been
removed. Curl_rand() is enough.
create_digest_md5_message: generate base64 instead of hex string
curl_sasl: also fix memory leaks in some OOM situations
Added required "debug" feature, missed in commit 1c9aaa0bac, as NTLMv2
calls Curl_rand() which can only be fixed to a specific entropy in
debug builds.
httpproxycode is not reset in Curl_initinfo, so a 407 is not reset even
if curl_easy_reset is called between transfers.
Bug: http://curl.haxx.se/bug/view.cgi?id=1380
The method change is forbidden by the obsolete RFC2616, but libcurl did
it anyway for compatibility reasons. The new RFC7231 allows this
behaviour so there's no need for the scary "Violate RFC 2616/10.3.x"
notice. Also update the comments accordingly.
The SASL/Digest previously used the current time's seconds +
microseconds to add randomness but it is much better to instead get more
data from Curl_rand().
It will also allow us to easier "fake" that for debug builds on demand
in a future.
Rather than use a short 8-byte hex string, extended the cnonce to be
32-bytes long, like Windows SSPI does.
Used a combination of random data as well as the current date and
time for the generation.
gcc spit out warning: variable 'x' might be clobbered by 'longjmp' or
'vfork' messages for a few variables. These automatic variables were
expected to be changed between a setjmp/longjmp and hold their values,
so are now marked volatile.
As with commit 11397eb6dd, use $(TargetDir) and $(TargetName) for the
Import Library output rather than $(OutDir)\$(ProjectName)d.lib and
$(OutDir)\$(ProjectName).lib.
"Any two of the parameters, readfds, writefds, or exceptfds, can be
given as null. At least one must be non-null, and any non-null
descriptor set must contain at least one handle to a socket."
http://msdn.microsoft.com/en-ca/library/windows/desktop/ms740141(v=vs.85).aspx
When using select(), cURL doesn't adhere to this (WinSock-specific)
rule, and can ask to monitor empty fd_sets, which leads to select()
returning WSAEINVAL (i.e. EINVAL) and connections failing in mysterious
ways as a result (at least when using the curl_multi_socket_action()
interface).
Bug: http://curl.haxx.se/mail/lib-2014-05/0278.html
OpenSSL passes out and outlen variable uninitialized to
select_next_proto_cb callback function. If the callback function
returns SSL_TLSEXT_ERR_OK, the caller assumes the callback filled
values in out and outlen and processes as such. Previously, if there
is no overlap in protocol lists, curl code does not fill any values in
these variables and returns SSL_TLSEXT_ERR_OK, which means we are
triggering undefined behavior. valgrind warns this.
This patch fixes this issue by fallback to HTTP/1.1 if there is no
overlap.
Make all code use connclose() and connkeep() when changing the "close
state" for a connection. These two macros take a string argument with an
explanation, and debug builds of curl will include that in the debug
output. Helps tracking connection re-use/close issues.
Renamed the CURLX_ONES file list definition in order to a) try and be
consistent with other file lists and b) to allow for the addition of
the curlx header files, which will assist with Visual Studio project
files generation rather than hard coding those files.
Added a default source directory so the user doesn't have to specify
one - the same as that, which the Visual Studio project files expect
the OpenSSL dependencies to be in.
Follow-up to commit 121bcfee5d. curl-config --features now lists
GSS-API but it is not a listed feature in curl -V. This should probably
be synchronized.
Verifies that the change in 68f0166a92 works as intended and that
different HTTP auth credentials to the same host still re-uses the
connection properly.
Commit 517b06d657 (in 7.36.0) that brought the CREDSPERREQUEST flag
only set it for HTTPS, making HTTP less good at doing connection re-use
than it should be. Now set it for HTTP as well.
Simple test case
"curl -v -u foo:bar localhost --next -u bar:foo localhos"
Bug: http://curl.haxx.se/mail/lib-2014-05/0127.html
Reported-by: Kamil Dudka
The variable wasn't assigned at all until step3 which would lead to a
failed connect never assigning the variable and thus returning a bad
value.
Reported-by: Larry Lin
Bug: http://curl.haxx.se/mail/lib-2014-04/0203.html
In commit 0b3750b5c2 (released in 7.36.0) we fixed a timeout issue
but instead broke the timings.
To fix this, I introduce a new timestamp to use for the timeouts and
restored the previous timestamp and timestamp position so that the old
timer functionality is restored.
In addition to that, that change also broke connection timeouts for when
more than one connect was used (as it would then count the total time
from the first connect and not for the most recent one). Now
Curl_timeleft() has been modified so that it checks against different
start times depending on which timeout it checks.
Test 1303 is updated accordingly.
Bug: http://curl.haxx.se/mail/lib-2014-05/0147.html
Reported-by: Ryan Braud
Whilst the qop directive isn't required to be present in a client's
response, as servers should assume a qop of "auth" if it isn't
specified, some may return authentication failure if it is missing.
To cater for the automatic generation of the new Visual Studio project
files, moved the lib file list into a separated variable so that lib
and lib/vtls can be referenced independently.
-p takes a list of Mozilla trust purposes and levels for certificates to
include in output. Takes the form of a comma separated list of
purposes, a colon, and a comma separated list of levels.
Now nghttp2_submit_request returns assigned stream ID, we don't have
to check stream ID using before_stream_send_callback. The
adjust_priority_callback was removed.
Depending on compiler line 3505 could generate the following warning or
error:
* warning: ISO C90 forbids mixed declarations and code
* A declaration cannot appear after an executable statement in a block
* error C2275: 'size_t' : illegal use of this type as an expression
Unlike previous versions of Visual Studio the VC12 solution file format
does not increment the format version number, but instead, only changes
the version comment text.
This incorrectly set version number would cause problems for any third
party piece of software that would read the solution file expecting the
version number to be 12.00 and found it to be 13.00, such as some build
accelerators.
Verified against a freshly created solution file which was generated
with VC12.
Angled brackets were used in the help output to indicate that the
compiler and platform arguments are mandatory. Unfortunately this
caused a "< was unexpected at this time" error as the characters are
interpreted as re-direction characters when not escaped.
As there's a default connection timeout and this wrongly used the
connection timeout during a transfer after the connection is completed,
this function would trigger timeouts during transfers erroneously.
Bug: http://curl.haxx.se/bug/view.cgi?id=1352
Figured-out-by: Radu Simionescu
If the precision is indeed shorter than the string, don't strlen() to
find the end because that's not how the precision operator works.
I also added a unit test for curl_msnprintf to make sure this works and
that the fix doesn't a few other basic use cases. I found a POSIX
compliance problem that I marked TODO in the unit test, and I figure we
need to add more tests in the future.
Reported-by: Török Edwin
Commit 07b66cbfa4 unfortunately broke native NTLM message support in
compilers, such as VC6, VC7 and others, that don't support long long
type declarations. This commit fixes VC6 and VC7 as they support the
__int64 extension, however, we should consider an additional fix for
other compilers that don't support this.
Carrying on from commit 11025613b9 added VC7.1 project files which are
capable of supporting side-by-side compilation, as well as support for
some of the third-party libraries curl uses.
This shouldn't happen with the source files in the repository, but
fixed the output when there are spurious files lying around that
contain spaces. For example "pop3 - Copy.c"
By including the offending source file in the project files the user
can then see the file and remove it if necessary.
Carrying on from commit 11025613b9 added VC7 project files which are
capable of supporting side-by-side compilation, as well as support for
some of the third-party libraries curl uses.
Carrying on from commit 11025613b9 added a more thorough version of
the VC6 project files which are capable of supporting side-by-side
compilation, as well as support for some of the third-party libraries
curl uses.
set.infilesize in this case was modified in several places, which could
lead to repeated requests using the same handle to get unintendent/wrong
consequences based on what the previous request did!
This makes the findprotocol() function work as intended so that libcurl
can properly be restricted to not support HTTP while still supporting
HTTPS - since the HTTPS handler previously set both the HTTP and HTTPS
bits in the protocol field.
This fixes --proto and --proto-redir for most SSL protocols.
This is done by adding a few new convenience defines that groups HTTP
and HTTPS, FTP and FTPS etc that should then be used when the code wants
to check for both protocols at once. PROTO_FAMILY_[protocol] style.
Bug: https://github.com/bagder/curl/pull/97
Reported-by: drizzt
Carrying on from commit 11025613b9 added VC12 project files which are
capable of supporting side-by-side compilation, 32-bit and 64-bit
builds as well as support for some of the third-party libraries curl
uses.
As this makes curl_global_init_mem() behave the same way as
curl_global_init() already does in that aspect - the same number of
curl_global_cleanup() calls is then required to again decrease the
counter and then eventually do the cleanup.
Bug: http://curl.haxx.se/bug/view.cgi?id=1362
Reported-by: Tristan
When --with-nghttp2 was used (without a given path), the
PKG_CONFIG_LIBDIR varialbe could get clobbered and ruin a proper
detection of the library.
Reported-by: Dilyan Palauzov
Bug: http://curl.haxx.se/mail/lib-2014-04/0159.html
ufds might not be allocated in case nfds overflows to zero while
extra_nfds is still non-zero. udfs is then accessed within the
extra_nfds-based for loop.
Should a command return untagged responses that contained no data then
the imap_matchresp() function would not detect them as valid responses,
as it wasn't taking the CRLF characters into account at the end of each
line.
Carrying on from commit 11025613b9 added VC11 project files which are
capable of supporting side-by-side compilation, 32-bit and 64-bit
builds as well as support for some of the third-party libraries curl
uses.
Carrying on from commit 11025613b9 added VC10 project files which are
capable of supporting side-by-side compilation, 32-bit and 64-bit
builds as well as support for some of the third-party libraries curl
uses.
Carrying on from commit 11025613b9, added VC9 project files which are
capable of supporting side-by-side compilation, 32-bit and 64-bit
builds as well as support for some of the third-party libraries curl
uses.
Given that we presently support "auth" and not "auth-int" or "auth-conf"
for native challenge-response messages, added client side validation of
the quality-of-protection options from the server's challenge message.
To avoid urldata.h being included from the header file or that the
source file has the correct include order as highlighted by one of
the auto builds recently.
When CURL_DISABLE_CRYPTO_AUTH is defined the DIGEST-MD5 code should not
be included, regardless of whether USE__WINDOWS_SSPI is defined or not.
This is indicated by the definition of USE_HTTP_NEGOTIATE and USE_NTLM
in curl_setup.h.
Added a more thorough version of the VC8 project files that exist in
the "vs" folder with the intention to add support for other versions of
Visual Studio. These files support side-by-side compilation, 32-bit and
64-bit builds as well as support for some of the third-party libraries
curl uses.
And clarify for curl that --proxy-header now must be used for headers
that are meant for a proxy, and they will not be included if the request
is not for a proxy.
Updated the docs to clarify and the code accordingly, with test 1528 to
verify:
When CURLHEADER_SEPARATE is set and libcurl is asked to send a request
to a proxy but it isn't CONNECT, then _both_ header lists
(CURLOPT_HTTPHEADER and CURLOPT_PROXYHEADER) will be used since the
single request is then made for both the proxy and the server.
When doing passive FTP, the multi state function needs to extract and
use the happy eyeballs sockets to wait for to check for completion!
Bug: http://curl.haxx.se/mail/lib-2014-02/0135.html (ruined)
Reported-by: Alan
Since all present tests now have <keywords> listed, this script will now
refuse to run a given test case if no such section is provided.
Hopefully this will help us make sure new test cases get keywords added
at start.
In addition to commit fe260b75e7 fixed the same issue for RFC-821 based
SMTP servers and allow the credientials to be given to curl even though
they are not used with the server.
This makes it possible to fetch from an IPv6 literal without specifying
the -g option. Globbing remains available elsehwere in the URL.
For example:
curl http://[::1]/file[1-3].txt
This creates no ambiguity, because there is no overlap between the
syntax of valid globs and valid IPv6 literals. Globs contain hyphens
and at most 1 colon, while IPv6 literals have no hyphens, and at least 2
colons.
The peek_ipv6() parser simply whitelists a set of characters and counts
colons, because the real validation happens later on. The character set
includes A-Z, in case someone decides to implement support for scopes
like [fe80::1%25eth0] in the future.
Signed-off-by: Paul Marks <pmarks@google.com>
When the protocol part fails, the data usually does too but the protocol
part is often more fundamental and often provide the clues you need to
fix the test case.
As the email protocols implement SASL authentication rather than IMAP,
POP3 and SMTP specific authentication, updated the authentication
keywords to reflect this.
Specifying user credentials when the SMTP server doesn't support
authentication would cause curl to display "No known authentication
mechanisms supported!" and return CURLE_LOGIN_DENIED.
Reported-by: Tom Sparrow
Bug: http://curl.haxx.se/mail/lib-2014-03/0173.html
The improved connection reuse logic would otherwise create a new
connection for each one, which isn't supported by the test
server, nor expected by the test.
This allows configure --disable-manual to run and build without having
to regenerate the src/tool_hugehelp.c file which otherwise is necessary
since we ship tarballs with that file present.
Reported-by: Remi Gacogne
Bug: http://curl.haxx.se/bug/view.cgi?id=1350
There are server certificates used with IP address in the CN field, but
we MUST not allow wild cart certs for hostnames given as IP addresses
only. Therefore we must make Curl_cert_hostcheck() fail such attempts.
Bug: http://curl.haxx.se/docs/adv_20140326B.html
Reported-by: Richard Moore
In addition to FTP, other connection based protocols such as IMAP, POP3,
SMTP, SCP, SFTP and LDAP require a new connection when different log-in
credentials are specified. Fixed the detection logic to include these
other protocols.
Bug: http://curl.haxx.se/docs/adv_20140326A.html
The debug messages printed inside PolarSSL always seems to end with a
newline. So 'infof()' should not add one. Besides the trace 'line'
should be 'const'.
To better allow arguments like "1 to 9999" without flooding the terminal
with error messages, the given test cases range is now checked and only
test numbers with existing files are actually run.
Because of the socket is unblocking, PolarSSL does need call to getsock to
get the action to perform in multi environment.
In some cases, it might happen we have not received yet all data to perform
the handshake. ssh_handshake returns POLARSSL_ERR_NET_WANT_READ, the state
is updated but because of the getsock has not the proper #define macro to,
the library never prevents to select socket for input thus the socket will
never be awaken when last data is available. Thus it leads to timeout.
API has changed since version 1.3. A compatibility header has been created
to ensure forward compatibility for code using old API:
* x509 certificate structure has been renamed to from x509_cert to
x509_crt
* new dedicated setter for RSA certificates ssl_set_own_cert_rsa,
ssl_set_own_cert is for generic keys
* ssl_default_ciphersuites has been replaced by function
ssl_list_ciphersuites()
This patch drops the use of the compatibly header.
Rename x509_cert to x509_crt and add "compat-1.2.h"
include.
This would still need some more thorough conversion
in order to drop "compat-1.2.h" include.
Port number zero is perfectly allowed to connect to. I moved to storing
the remote port number in an int so that -1 means undefined and 0-65535
can be used for legitimate port numbers.
Setting the TIMER_STARTSINGLE timestamp first in CONNECT has the
drawback that for actions that go back to the CONNECT state, the time
stamp is reset and for the multi_socket API there's no corresponding
Curl_expire() then so the timeout logic gets wrong!
Reported-by: Brad Spencer
Bug: http://curl.haxx.se/mail/lib-2014-02/0036.html
Remove slash/backslash problem, now only slashes are used,
Wmake automaticaly translate slash/backslash to proper version or tools are not sensitive for it.
Enable spaces in path.
Use internal rm command for all host platforms
Add error message if old Open Watcom version is used. Some old versions exhibit build problems for Curl latest version. Now only versions 1.8, 1.9 and 2.O beta are supported
Ensure a source file isn't generated for the following informational
command line parameters when --libcurl is specified:
--help, --manual, --version and --engine list
As the output would only include a fairly empty looking main() function
and a call to curl_easy_init() and curl_easy_cleanup() when performed
with --engine list.
Correctly output libcurl source code that includes multiply operations
as specified by --next. Note that each operation evaluates to a single
curl_easy_perform() in source code form.
Also note that the output could be optimised a little so global config
options are only output once rather than per operation as is presently
the case.
For HTTP/2, we may read up everything including responde body with
header fields in Curl_http_readwrite_headers. If no content-length is
provided, curl waits for the connection close, which we emulate it
using conn->proto.httpc.closed = TRUE. The thing is if we read
everything, then http2_recv won't be called and we cannot signal the
HTTP/2 stream has closed. As a workaround, we return nonzero from
data_pending to call http2_recv.
Added initial support for --next/-: which will be used to replace the
rather confusing : command line operation what was used for the URL
specific options prototype.
Original commit message was:
Don't omit CN verification in SChannel when an IP address is used.
Side-effect of this change:
SChannel and CryptoAPI do not support the iPAddress subjectAltName
according to RFC 2818. If present, SChannel will first compare the
IP address to the dNSName subjectAltNames and then fallback to the
most specific Common Name in the Subject field of the certificate.
This means that after this change curl will not connect to SSL/TLS
hosts as long as the IP address is not specified in the SAN or CN
of the server certificate or the verifyhost option is disabled.
In preparation for separating the global config options from the per
operation config options, reworked the list engines code to not use a
member variable in the Configurable structure.
To help assist with the detection of incorrect return codes, as per
commits ee23d13a79, 33b8960dc8 and aba98991a5, updated the operate
based functions to return CURLcode error codes.
During initialisation SetHTTPrequest() may fail and cURL would return
PARAM_BAD_USE, which is equivalent to CURLE_NOT_BUILT_IN in cURL error
terms.
Instead, return CURLE_FAILED_INIT as we do for other functions that may
fail during initialisation.
The previous test certificate contained a MD5 hash which is not
supported using TLSv1.2 with Schannel on Windows 7 or newer.
See the update to this blog post on IEInternals / MSDN:
http://blogs.msdn.com/b/ieinternals/archive/2011/03/25/
misbehaving-https-servers-impair-tls-1.1-and-tls-1.2.aspx
"Update: If the server negotiates a TLS1.2 connection with a
Windows 7 or 8 schannel.dll-using client application, and it
provides a certificate chain which uses the (weak) MD5 hash
algorithm, the client will abort the connection (TCP/IP FIN)
upon receipt of the certificate."
This patch enables HTTP POST/PUT in HTTP2.
We disabled Expect header field and chunked transfer encoding
since HTTP2 forbids them.
In HTTP1, Curl sends small upload data with request headers, but
HTTP2 requires upload data must be in DATA frame separately.
So we added some conditionals to achieve this.
Perform more work in between sleeps. This is work around the
fact that axtls does not expose any knowledge about when work needs
to be performed. Depending on connection and how often perform is
being called this can save ~25% of time on SSL handshakes (measured
on 20ms latency connection calling perform roughly every 10ms).
When allowing NTLM, the re-use connection logic was too focused on
finding an existing NTLM connection to use and didn't properly allow
re-use of other ones. This made the logic not re-use perfectly re-usable
connections.
Added test case 1418 and 1419 to verify.
Regression brought in 8ae35102c (curl 7.35.0)
Reported-by: Jeff King
Bug: http://thread.gmane.org/gmane.comp.version-control.git/242213
Rather than check for required arguments, and prompt for any host and
proxy passwords, as each operation is performed, changed the code so
all configurations are checked before any operations are performed.
This allows the user to input all the required passwords, for example,
upfront rather than wait for each operation.
This one is needed with the gcc options -fstack-protector-all -O2
That brings the number of suppressions for test 165 to four, and I
suspect I could find another two missing without trying very hard. I'm
beginning to think suppressions isn't the best way to handle these
kinds of cases.
Since protocol headers contain explicit line-endings there should
be no automatic conversion to ASCII text or CRLF line-endings.
This might break third party tools that already depend on this
behaviour. We might need to introduce an option to make this optional.
Do not try to convert line-endings to CRLF on Windows by setting stdout
to binary mode, just like the curl tool does if --ascii is not specified.
This should prevent corrupted stdout line-ending output like CRCRLF.
In order to make the previously naive text-aware tests work with
binary mode on Windows, text-mode is disabled for them if it is not
actually part of the test case and line-endings are corrected.
According to RFC 2616 and RFC 2326 individual protocol elements, like
headers and except the actual content, are terminated by using CRLF.
Therefore the test data files for these protocols need to contain
mixed line-endings if the actual protocol elements use CRLF while
the file uses LF.
gcc 4.7.2 with -O2 will optimize Curl_connect by inlining some
functions two levels deep, which makes the valgrind suppression
fail to match. The underlying reason for these idna suppressions is
a gcc strlen optimization when compiling libidn; compiling it with
-fno-builtin-strlen makes this suppression unnecessary.
For a function that returns a decoded version of a string, it seems
really strange to allow a NULL pointer to get passed in which then
prevents the decoded data from being returned!
This functionality was not documented anywhere either.
If anyone would use it that way, that memory would've been leaked.
Bug: https://github.com/bagder/curl/pull/90
Reported-by: Arvid Norberg
Make sure that the special NTLM magic we do is for HTTP+NTLM only since
that's where the authenticated connection is a weird non-standard
paradigm.
Regression brought in 8ae35102c (curl 7.35.0)
Bug: http://curl.haxx.se/mail/lib-2014-02/0100.html
Reported-by: Dan Fandrich
The code didn't properly check the return codes to detect overflows so
it could trigger incorrectly. Like on mingw32.
Regression introduced in 345891edba (curl 7.35.0)
Bug: http://curl.haxx.se/mail/lib-2014-02/0097.html
Reported-by: LM
Commmit c5f8e2f5f4 removed the easy handle clean-up from tool_operate,
letting the code that was already present in free_config_fields()
perform the task. Unfortunately, this wasn't the correct place to do
this as it broke protocols, that would perform a logout, as the main
clean-up in tool_main had already been called.
It seems the fips config option causes an error if FIPS mode was
not enabled at stunnel compile-time. FIPS support was disabled
by default in stunnel 5.00, so this is probably really only needed
on versions between 4.32 and 5.00.
when using --http2 one can now selectively disable NPN or ALPN with
--no-alpn and --no-npn. for now honored with NSS only.
TODO: honor this option with GnuTLS and OpenSSL
SSL_ENABLE_ALPN can be used for preprocessor ALPN feature detection,
but not SSL_NEXT_PROTO_SELECTED, since it is an enum value and not a
preprocessor macro.
This was already mostly being done, except that analysis after the
test still assumed that the valgrind log files would be available. An
alternative way to handle the valgrind + gdb combination could be to
enable one of the valgrind debugger hooks.
lib1515.c:38:26 warning: unused parameter 'curl'
lib1515.c:38:81 warning: unused parameter 'ptr'
lib1515.c:38:5 warning: no previous prototype for 'debug_callback'
lib1515.c:46:5 warning: no previous prototype for 'do_one_request'
lib1515.c:120:3 warning: ISO C90 forbids mixed declarations and code
As well as some code policing such as white space and braces.
Not comma, which is an inconsistency and a mistake probably inherited
from the examples section of RFC1867.
This bug has been present since the day curl started to support
multipart formposts, back in the 90s.
Reported-by: Rob Davies
Bug: http://curl.haxx.se/bug/view.cgi?id=1333
When using the multi socket interface, libcurl calls the
curl_multi_timer_callback asking to be woken up after
CURL_TIMEOUT_EXPECT_100 milliseconds.
After the timeout has expired, calling curl_multi_socket_action with
CURL_SOCKET_TIMEOUT as sockfd leads libcurl to check expired
timeouts. When handling the 100-continue one, the following check in
Curl_readwrite() fails if exactly CURL_TIMEOUT_EXPECT_100 milliseconds
passed since the timeout has been set!
It seems logical to consider that having waited for exactly
CURL_TIMEOUT_EXPECT_100 ms is enough.
Bug: http://curl.haxx.se/bug/view.cgi?id=1334
Fix for bug #1303 (030a2b8cb) was not complete.
libcurl still pruned DNS entries added manually
after detecting a dead connection. This test
checks such behavior.
Test-case 1515 reproduces bug #1303, where libcurl
would incorrectly prune DNS entries added via
CURLOPT_RESOLVE after the DNS_CACHE_TIMEOUT had
expired.
A server might respond with a content-encoding header and a response
that was encoded accordingly in HTTP-draft-09/2.0 mode, even if the
client did not send an accept-encoding header earlier. The server might
not send a content-encoding header if the identity encoding was used to
encode the response.
See:
http://tools.ietf.org/html/draft-ietf-httpbis-http2-09#section-9.3
This patch chooses different approach to integrate HTTP2 into HTTP curl
stack. The idea is that we insert HTTP2 layer between HTTP code and
socket(TLS) layer. When HTTP2 is initialized (either in NPN or Upgrade),
we replace the Curl_recv/Curl_send callbacks with HTTP2's, but keep the
original callbacks in http_conn struct. When sending serialized data by
nghttp2, we use original Curl_send callback. Likewise, when reading data
from network, we use original Curl_recv callback. In this way we can
treat both TLS and non-TLS connections.
With this patch, one can transfer contents from https://twitter.com and
from nghttp2 test server in plain HTTP as well.
The code still has rough edges. The notable one is I could not figure
out how to call nghttp2_session_send() when underlying socket is
writable.
The test contains a cookie jar file where one of the cookies has an
expiry date of 1391252187 -- Sat, 1 Feb 2014 10:56:27 GMT which has
now expired. Updated to Wed, 14 Oct 2037 16:36:33 GMT as per test
179.
Reported-by: Adam Sampson
Bug: http://curl.haxx.se/bug/view.cgi?id=1330
Since the timer resolution is lower, there are actually cases that
the compared values are equal. Therefore we check for previous
timestamps being greater than the current one instead.
According to section 2.2 of RFC959 the End-of-Line is defined as:
The end-of-line sequence defines the separation of printing
lines. The sequence is Carriage Return, followed by Line Feed.
Verified by sniffing traffic between a Windows FTP client (FileZilla)
and Unix-hosted FTP server (ProFTPD).
It makes more sense to convert the expected output to [CR][LF] on
Windows than to force the actual, probably correct, output to [LF].
This way it is actually possible to see if curl outputs the correct
line-ending excepted by a text-aware test case.
Check the NPN result before preparing an HTTP request and switch into
HTTP/2.0 mode if necessary. This is a work in progress, the actual code
to prepare and send the request using nghttp2 is still missing from
Curl_http2_send_request().
the number of elements in the 'nghttp2_session_callbacks' structure is
now reduced by 2 in version 0.3.0 (I'm not sure when the change
happened, but checking for ver 0.3.0 work for me).
Something is wrong in 'userp' for the HTTP2 recv_callback(). The
session is created using bogus user-data; '&conn' and not 'conn'.
I noticed this since the socket-value in Curl_read_plain() was set to a
impossible high value.
hostcache_timestamp_remove() should remove old *unused* entries from the
host cache, but it never checked whether the entry was actually in
use. This complements commit 030a2b8cb.
Bug: http://curl.haxx.se/bug/view.cgi?id=1327
tftp_done() can get called with its TFTP state pointer still being NULL
on an early time-out, which caused a segfault when dereferenced.
Reported-by: Glenn Sheridan
Bug: http://curl.haxx.se/mail/lib-2014-01/0246.html
Since the previous complex select function with initial support for
non-socket file descriptors, did not actually work correctly for
Console handles, this change simplifies the whole procedure by using
an internal waiting thread for the stdin console handle.
The previous implementation made it continuously trigger for the stdin
handle if it was being redirected to a parent process instead of
an actual Console input window.
This approach supports actual Console input handles as well as
anonymous Pipe handles which are used during input redirection.
It depends on the fact that ReadFile supports trying to read zero bytes
which makes it wait for the handle to become ready for reading.
Removed Unix-specific functionality in order to support Windows:
- select.epoll replaced with select.select
- SocketServer.ForkingMixIn replaced with SocketServer.ForkingMixIn
- socket.MSG_DONTWAIT replaced with socket.setblocking(False)
Even though epoll has a better performance and improved socket handling
than select, this change should not affect the actual test case.
The list was out of date and the paragraph already refers to the
CURLOPT_HTTPAUTH explanation. All the auth bits are explained properly
there.
It also removes the ambiguity for what the "added" phrase refers to.
This change based on pull request #85 on github
URL: https://github.com/bagder/curl/pull/85
Reported-by: gnawhleinad
Make it possible to call
curl_easy_getinfo(curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &filesize)
and related functions on remote sftp:// files, without downloading them.
Reported-by: Yingwei Liu
Bug: http://curl.haxx.se/mail/lib-2014-01/0139.html
This prevents sending a `Content-Length: -1` header, e.g this ocurred
with the following combination:
* standard HTTP POST (no chunked encoding),
* user-defined read function set,
* `CURLOPT_POSTFIELDSIZE(_LARGE)` NOT set.
With this fix it now behaves like HTTP PUT.
Make GnuTLS old and new consistent, specify the desired protocol, cipher
and certificate type in always in both modes. Disable insecure ciphers
as reported by howsmyssl.com. Honor not only --sslv3, but also the
--tlsv1[.N] switches.
Related Bug: http://curl.haxx.se/bug/view.cgi?id=1323
Due to the changes in commit 3c929ff9f6 and lack of subsequent
updates, curl could return a CURLE_FTP_ACCEPT_FAILED error if
checkpasswd() ran out of memory in versions 7.33.0 and 7.34.0.
Updated the function declaration and return code to return
CURLE_OUT_OF_MEMORY and CURLE_OK where appropriate.
conversion from 'curl_off_t' to 'size_t', possible loss of data
Where curl_off_t is a 64-bit word and size_t is 32-bit - for example
with 32-bit Windows builds.
In the rare instance where getparameter() may return PARAM_NO_MEM whilst
parsing a URL, cURL would return this error code, which is equivalent to
CURLE_FTP_ACCEPT_FAILED in cURL error codes terms.
Instead, return CURLE_FAILED_INIT and output the failure reason as per
the other usage of getparameter().
Increasing the update frequency of the progress bar to 10Hz greatly
improves the visual appearance of the progress bar (at least in my
impression).
Signed-off-by: Tobias Markus <tobias@markus-regensburg.de>
Currently, the progress bar is updated at 5Hz. Because it is often not
updated to 100% when the download is finished and curl exits, the bar
is often "stuck" at 90-something, thus irritating the user.
This patch fixes this by always updating the progress bar (instead of
waiting for 200ms to have elapsed) while the download is finished but
curl has not yet exited. This should not greatly affect performance
because that moment is rather short.
Signed-off-by: Tobias Markus <tobias@markus-regensburg.de>
1 - allow >31 bit max-age values
2 - don't overflow on extremely large max-age values when we add the
value to the current time
3 - make sure max-age takes precedence over expires as dictated by
RFC6265
Bug: http://curl.haxx.se/mail/lib-2014-01/0130.html
Reported-by: Chen Prog
Also, make the ftp server return a canned response that doesn't
cause XML verification problems. Although the test file format
isn't technically XML, it's still handy to be able to use XML
tools to verify and manipulate them.
The issue is with HP-UX that is comes with HP flavor of MIT
Kerberos. This means that there is no krb5-config and the lib is called
libgss.so
Bug: http://curl.haxx.se/bug/view.cgi?id=1321
Starting with Visual Studio 2013 (VC12) and Windows 8.1 the
GetVersionInfoEx() function has been marked as deprecated and it's
return value atered. Updated connect.c and curl_sspi.c to use
VerifyVersionInfo() where possible, which has been available since
Windows 2000.
Since /dev/stdout is not always emulated on Windows,
just skip the output option on Windows.
MinGW/msys support /dev/stdout only from a new login shell.
A transfer timeout could result in an error message such as "Operation
timed out after 3000 milliseconds with 19 bytes of -1 received". This
patch removes the non-sensical "of -1" when the size of the transfer
is unknown, mirroring the logic in lib/transfer.c
tstunnel on Windows does not support the pid option and is unable
to write to an output log that is already being used as a redirection
target for stdout. Therefore it does now output all log data to stdout
by default and secureserver.pl creates a fake pidfile on Windows.
Simplified the SMTP multi example as this example should demonstrate
the differences the easy and multi interfaces rather than introduce new
concepts such as user authentication and TLS which are shown in the TLS
and SSL examples.
By default even recent versions of OpenSSL support and accept both
"export strength" ciphers, small-bitsize ciphers as well as downright
deprecated ones.
This change sets a default cipher set that avoids the worst ciphers, and
subsequently makes https://www.howsmyssl.com/a/check no longer grade
curl/OpenSSL connects as 'Bad'.
Bug: http://curl.haxx.se/bug/view.cgi?id=1323
Reported-by: Jeff Hodges
With the recently added timeout "reminder" functionality, there's no
reason left for us to execute timeout code before the time is
ripe. Simplifies the handling too.
This will make the *TIMEOUT and *CONNECTTIMEOUT options more accurate
again, which probably is most important when the *_MS versions are used.
In multi_socket, make sure to update 'now' after having handled activity
on a socket.
Currently VC7 and VC7.1 builds have to be ran with the VC variable set
to vc6 which is not only inconsistent with the nmake winbuild system
but also with newer versions of Visual Studio supported by this file.
Note: This doesn't break the build for anyone still running with the
VC variable set to vc6 or not set (which defaults to vc6).
BACKGROUND:
We have learned that on some systems timeout timers are inaccurate and
might occasionally fire off too early. To make the multi_socket API work
with this, we made libcurl execute timeout actions a bit early too if
they are within our MULTI_TIMEOUT_INACCURACY. (added in commit
2c72732ebf, present since 7.21.0)
Switching everything to the multi API made this inaccuracy problem
slightly more notable as now everyone can be affected.
Recently (commit 21091549c0) we tweaked that inaccuracy value to make
timeouts more accurate and made it platform specific. We also figured
out that we have code at places that check for fixed timeout values so
they MUST NOT run too early as then they will not trigger at all (see
commit be28223f35 and a691e04470) - so there are definitately problems
with running timeouts before they're supposed to run. (We've handled
that so far by adding the inaccuracy margin to those specific timeouts.)
The libcurl multi_socket API tells the application with a callback that
a timeout expires in N milliseconds (and it explicitly will not tell it
again for the same timeout), and the application is then supposed to
call libcurl when that timeout expires. When libcurl subsequently gets
called with curl_multi_socket_action(...CURL_SOCKET_TIMEOUT...), it
knows that the application thinks the timeout expired - and alas, if it
is within the inaccuracy level libcurl will run code handling that
handle.
If the application says CURL_SOCKET_TIMEOUT to libcurl and _isn't_
within the inaccuracy level, libcurl will not consider the timeout
expired and it will not tell the application again since the timeout
value is still the same.
NOW:
This change introduces a modified behavior here. If the application says
CURL_SOCKET_TIMEOUT and libcurl finds no timeout code to run, it will
inform the application about the timeout value - *again* even if it is
the same timeout that it already told about before (although libcurl
will of course tell it the updated time so that it'll still get the
correct remaining time). This way, we will not risk that the application
believes it has done its job and libcurl thinks the time hasn't come yet
to run any code and both just sit waiting. This also allows us to
decrease the MULTI_TIMEOUT_INACCURACY margin, but that will be handled
in a separate commit.
A repeated timeout update to the application risk that the timeout will
then fire again immediately and we have what basically is a busy-loop
until the time is fine even for libcurl. If that becomes a problem, we
need to address it.
The net effect of this bug as it appeared to users, would be that
libcurl would timeout in the connect phase.
When disabling IPv6 use but still using getaddrinfo, libcurl would
wrongly not init the "hints" struct field in init_thread_sync() which
would subsequently lead to a getaddrinfo() invoke with a zeroed hints
with ai_socktype set to 0 instead of SOCK_STREAM. This would lead to
different behaviors on different platforms but basically incorrect
output.
This code was introduced in 483ff1ca75, released in curl 7.20.0.
This bug became a problem now due to the happy eyeballs code and how
libcurl now traverses the getaddrinfo() results differently.
Bug: http://curl.haxx.se/mail/lib-2014-01/0061.html
Reported-by: Fabian Frank
Debugged-by: Fabian Frank
Removed some of the infof() calls that were added with the recent
pipeline improvements but they're not useful to the vast majority of
readers and the pipelining seems to fundamentaly work - the debugging
outputs can easily be added there if debugging these functions is needed
again.
The built-in memory debug system doesn't work with multi-threaded use so
instead of causing annoying false positives, disable the memory tracking
if the threaded resolver is used.
When the requested authentication bitmask includes NTLM, we cannot
re-use a connection for another username/password as we then risk
re-using NTLM (connection-based auth).
This has the unfortunate downside that if you include NTLM as a possible
auth, you cannot re-use connections for other usernames/passwords even
if NTLM doesn't end up the auth type used.
Reported-by: Paras S
Patched-by: Paras S
Bug: http://curl.haxx.se/mail/lib-2014-01/0046.html
The Windows console version of stunnel is called "tstunnel", while
running "stunnel" on Windows spawns a new console window which
cannot be handled by the testsuite.
Previously LIST always returned a fixed hardcoded list that the ftp
server code knew about, mostly since the server didn't get any test case
number in the LIST scenario. Starting now, doing a CWD to a directory
named test-[number] will make the test server remember that number and
consider it a test case so that a subsequent LIST command will send the
<data> section of that test case back.
It allows LIST tests to be made more similar to how all other tests
work.
Test 100 was updated to provide its own directory listing.
Replaced the use of CURLOPT_USERPWD for the preferred CURLOPT_USERNAME
and CURLOPT_PASSWORD options and used the same username and password for
all email examples which is the same as that used in the test suite.
Verify the change brought in commit 8e11731653061. It makes sure that
returning a failure from the progress callback even very early results
in the correct return code.
When the progress callback returned 1 at a very early state, the code
would not make CURLE_ABORTED_BY_CALLBACK get returned but the process
would still be interrupted. In the HTTP case, this would then cause a
CURLE_GOT_NOTHING to erroneously get returned instead.
Reported-by: Petr Novak
Bug: http://curl.haxx.se/bug/view.cgi?id=1318
memdebug.h already contains all required definitions and including
curl_memory.h causes errors like the following:
tests/unit/unit1394.c:119: undefined reference to `Curl_cfree'
tests/unit/unit1394.c:120: undefined reference to `Curl_cfree'
This is a debug function only and serves no purpose in production code,
it only slows things down. I left the code #ifdef'ed for possible future
pipeline debugging.
Also, this was a global function without proper namespace usage.
Reported-by: He Qin
Bug: http://curl.haxx.se/bug/view.cgi?id=1320
If OpenSSL is built to support SSLv2 this brings back the ability to
explicitly select that as a protocol level.
Reported-by: Steve Holme
Bug: http://curl.haxx.se/mail/lib-2014-01/0013.html
This fixes two markup typos I noticed in curl_easy_setopt.3. (The use
of bold vs. italics seems a bit inconsistent in that page, but it should
at least be valid man syntax.)
Some feedback provided by byte_bucket on IRC pointed out that commit
db11750cfa wasn’t really correct because it allows for “upgrading” to a
newer protocol when it should be only allowing for SSLv3.
This change fixes that.
When SSLv3 connection is forced, don't allow SSL negotiations for newer
versions. Feedback provided by byte_bucket in #curl. This behavior is
also consistent with the other force flags like --tlsv1.1 which doesn't
allow for TLSv1.2 negotiation, etc
Feedback-by: byte_bucket
Bug: http://curl.haxx.se/bug/view.cgi?id=1319
Since ad34a2d5c8 (present in 7.34.0 release) forcing
SSLv3 will always return the error "curl: (35) Unsupported SSL protocol
version" Can be replicated with `curl -I -3 https://www.google.com/`.
This fix simply allows for v3 to be forced.
Following commit 0aafd77fa4, replaced the internal usage of
FORMAT_OFF_T and FORMAT_OFF_TU with the external versions that we
expect API programmers to use.
This negates the need for separate definitions which were subtly
different under different platforms/compilers.
Added support to the built-in printf() replacement functions, for these
non-ANSI extensions when compiling under Visual Studio, Borland, Watcom
and MinGW.
This fixes problems when generating libcurl source code that contains
curl_off_t variables.
Fixes a bug when all addresses in the first family fail immediately, due
to "Network unreachable" for example, curl would hang and never try the
next address family.
Iterate through all address families when to trying establish the first
connection attempt.
Bug: http://curl.haxx.se/bug/view.cgi?id=1315
Reported-by: Michal Górny and Anthony G. Basile
Following the addition of informational commands to the SMTP protocol,
the test server is no longer required to return the verified server
information in responses that curl only outputs in verbose mode.
Instead, a similar detection mechanism to that used by FTP, IMAP and
POP3 can now be used.
Introduced in commit 2a4ee0d221 sending of data via the FILE
protocol would always return CURLE_WRITE_ERROR regardless of whether
CURL_WRITEFUNC_PAUSE was returned from the callback function or not.
Make sure that we detect such attempts and return a proper error code
instead of silently handling this in problematic ways.
Updated the documentation to mention this limitation.
Bug: http://curl.haxx.se/bug/view.cgi?id=1286
Previously this memdebug free() replacement didn't properly work with a
NULL argument which has made us write code that avoids calling
free(NULL) - which causes some extra nuisance and unnecessary code.
Starting now, we should allow free(NULL) even when built with the
memdebug system enabled.
free(NULL) is permitted by POSIX
free() itself allows a NULL input but our memory debug system requires
Curl_safefree() to be used instead when a "legitimate" NULL may be freed. Like
in the code here.
Pointed-out-by: Steve Holme
If a user indicated they preferred to authenticate using a SASL
mechanism, but SASL authentication wasn't supported by the server, curl
would always fall back to clear text when CAPABILITY wasn't supported,
even though the user didn't want to use this.
If a user indicated they preferred to authenticate using APOP or a SASL
mechanism, but neither were supported by the server, curl would always
fall back to clear text when CAPA wasn't supported, even though the
user didn't want to use this.
This also fixes the auto build failure caused by commit 6f2d5f0562.
This commit replaces that of 9f260b5d66 because according to RFC-2449,
section 6, there is no APOP capability "...even though APOP is an
optional command in [POP3]. Clients discover server support of APOP by
the presence in the greeting banner of an initial challenge enclosed in
angle brackets."
SASL downgrade tests: 833, 835, 879, 881, 935 and 937 would fail as
they contained a minus sign in their authentication mechanism and this
would be missed by the custom reply parser.
The FILE:// code doesn't support this option - and it doesn't make sense
to support it as long as it works as it does since then it'd only block
even longer.
But: setting CURLOPT_MAX_RECV_SPEED_LARGE would make the transfer first
get done and then libcurl would wait until the average speed would get
low enough. This happened because the transfer happens completely in the
DO state for FILE:// but then it would still unconditionally continue in
to the PERFORM state where the speed check is made.
Starting now, the code will skip from DO_DONE to DONE immediately if no
socket is set to be recv()ed or send()ed to.
Bug: http://curl.haxx.se/bug/view.cgi?id=1312
Reported-by: Mohammad AlSaleh
The comment in the code mentions the zero terminating after having
copied data, but it mistakingly zero terminated the source data and not
the destination! This caused the test 864 problem discussed on the list:
http://curl.haxx.se/mail/lib-2013-12/0113.html
Signed-off-by: Daniel Stenberg <daniel@haxx.se>
Although highlighted by a bug in commit 1cfb436a2f, APOP
authentication could be chosen if the server was to reply with an empty
or missing timestamp in the server greeting and APOP was given in the
capability list by the server.
Added a loop to pop3_statemach_act() in which Curl_pp_readresp() is
called until the cache is drained. Without this multiple responses
received in a single packet could result in a hang or delay.
Similar to the processing of untagged CAPABILITY responses in IMAP and
multi-line EHLO responses in SMTP, moved the processing of multi-line
CAPA responses to pop3_state_capa_resp().
In an effort to reduce what pop3_endofresp() does and bring the POP3
source back inline with the IMAP and SMTP protocols, moved the APOP
detection into pop3_state_servergreet_resp().
Added support for downgrading the SASL authentication mechanism when the
decoding of CRAM-MD5, DIGEST-MD5 and NTLM messages fails. This enhances
the previously added support for graceful cancellation by allowing the
client to retry a lesser SASL mechanism such as LOGIN or PLAIN, or even
APOP / clear text (in the case of POP3 and IMAP) when supported by the
server.
In preparation for the upcoming SASL downgrade feature renamed the
imap__perform_authenticate(), pop3__perform_authenticate() and
smtp__perform_authenticate() functions.
Security flaw CVE-2013-6422
This is conceptually the same problem and fix that 3c3622b6 brought to the
OpenSSL backend and that resulted in CVE-2013-4545.
This version of the problem was independently introduced to the GnuTLS
backend with commit 59cf93cc, present in the code since the libcurl
7.21.4 release.
Advisory: http://curl.haxx.se/docs/adv_20131217.html
Bug: http://curl.haxx.se/mail/lib-2013-11/0214.html
Reported-by: Marc Deslauriers
Since all systems have inaccuracy in the timeout handling it is
imperative that we add an inaccuracy margin to the general timeout and
connecttimeout handling with the multi interface. This way, when the
timeout fires we should be fairly sure that it has passed the timeout
value and will be suitably detected.
For cases where the timeout fire before the actual timeout, we would
otherwise consume the timeout action and still not run the timeout code
since the condition wasn't met.
Reported-by: He Qin
Bug: http://curl.haxx.se/bug/view.cgi?id=1298
To avoid the regression when users pass in passwords containing semi-
colons, we now drop the ability to set the login options with the same
options. Support for login options in CURLOPT_USERPWD was added in
7.31.0.
Test case 83 was modified to verify that colons and semi-colons can be
used as part of the password when using -u (CURLOPT_USERPWD).
Bug: http://curl.haxx.se/bug/view.cgi?id=1311
Reported-by: Petr Bahula
Assisted-by: Steve Holme
Signed-off-by: Daniel Stenberg <daniel@haxx.se>
It is not 100% clear whether * should include clear text LOGIN or not
from RFC-5092, however, including it is then consistent with current
POP3 behaviour where clear text, APOP or SASL may be chosen.
If a specific SASL authentication mechanism was requested by the user
as part of the login options but wasn't supported by the server then
curl would fallback to clear text, when it shouldn't, rather than
reporting "No known authentication mechanisms supported" as the POP3
and SMTP protocols do.
In C, signed integer overflow is undefined behavior. Thus, the compiler
is allowed to assume that it will not occur. In the check for an
overflow, the developer assumes that the signed integer of type time_t
will wrap around if it overflows. However, this behavior is undefined in
the C standard. Thus, when the compiler sees this, it simplifies t +
delta < t to delta < 0. Since delta > 0 and delta < 0 can't both be
true, the entire if statement is optimized out under certain
optimization levels. Thus, the parsedate function would return
PARSEDATE_OK with an undefined value in the time, instead of return -1 =
PARSEDATE_FAIL.
Commit 0db811b6 made some existing config files pass on unexpected
values to libcurl that made it somewhat hard to track down what was
really going on.
This code detects unquoted white spaces in the parameter when parsing a
config file as that would be one symptom and it is generally a bad
syntax anyway.
The comment here says that SecKeychainSearch causes a deprecation
warning when used with a minimum Mac OS X SDK version of 10.7.0, which
is correct. However, the #if guard did not match. It was intended to
only use the code if 10.6.0 support was enabled, but it had 10.7.0
instead. This caused a warning if the minimum was exactly 10.7.0.
curl.h should also include <sys/select.h> on OpenBSD to reliably
pull in select(). Typically, including <sys/time.h> will be enough,
but not if strict standards-compliance is requested (e.g. by defining
_XOPEN_SOURCE).
The URI that is passed in as part of the Authorization: header needs to
be cut off at '?' if CURLAUTH_DIGEST_IE is set. Previously the code only
did when calculating the MD5sum.
Bug: http://curl.haxx.se/bug/view.cgi?id=1308
Patched-by: Sergey Tatarincev
POP3_TYPE_ANY, or ~0, is written to pop3c->preftype in lib/pop3c.c, an
unsigned int variable. The result of ~0 is -1, which caused a warning
due to writing a negative number to an unsigned variable. To fix this,
make the expression ~0U so that its value is considered the unsigned
number UINT_MAX which is what SASL_AUTH_ANY does in curl_sasl.h.
1) Renamed curl_tlsinfo to curl_tlssessioninfo as discussed on the
mailing list.
2) Renamed curl_ssl_backend to curl_sslbackend so it doesn't follow our
function naming convention.
3) Updated sessioninfo.c example accordingly.
The "fixed string" function wrongly bumped the "urlnum" counter which
made curl output the total number of URLs wrong when using
{one,two,three} lists in globs.
Reported-by: Michael-O
Bug: http://curl.haxx.se/bug/view.cgi?id=1305
This fixes a NULL dereference in the case where the client asks for
CURLINFO_TLS_SESSION data after the (TLS) session has already been
destroyed (i.e. curl_easy_perform has already completed for this
handle). Instead of crashing, we now return a CURLSSLBACKEND_NONE
error.
This is an extension to the fix in 7d80ed64e4. We may
call Curl_disconnect() while cleaning up the multi handle,
which could lead to openssl sending packets, which could get
a SIGPIPE.
Signed-off-by: Jeff King <peff@peff.net>
Commit 7d80ed64e4 introduced some helpers to handle
sigpipe in easy.c. However, that fix was incomplete, and we
need to add more callers in other files. The first step is
making the helpers globally accessible.
Since the functions are small and should generally end up
inlined anyway, we simply define them in the header as
static functions.
Signed-off-by: Jeff King <peff@peff.net>
This fixes a rare Happy Eyeballs bug where if the first IP family runs
out of addresses before the second-family-timer fires, and the second
IP family's first connect fails immediately, no further IPs of the
second family are attempted.
When adding entries to the DNS cache with CURLOPT_RESOLVE, they are
marked 'inuse' forever to prevent them from ever being removed in normal
operations. Still, the code that pruned out-of-date DNS entries didn't
care for the 'inuse' struct field and pruned it anyway!
Reported-by: Romulo A. Ceccon
Bug: http://curl.haxx.se/bug/view.cgi?id=1303
A failure during authentication, which is performed as part of the
CONNECT phrase (for IMAP, POP3 and SMTP) is considered by the multi-
interface as being closed prematurely (aka a dead connection). As such
these protocols cannot issue the relevant QUIT or LOGOUT command.
Temporarily fixed the test cases until we can fix this properly.
Added new API for returning a SSL backend type and pointer, in order to
allow access to the TLS internals, that may then be used to obtain X509
certificate information for example.
Otherwise a NOOP operation would be performed which a) only returns a
single line response and not a multiline response where -I needs to be
used, and b) provides an inconsistent user experience compared to that
of the POP3 and IMAP protocols.
singleipconnect() could return the file descriptor of an open socket
even though the function returned a CURLE_COULDNT_CONNECT error code
from commit ed1662c374 and 02fbc26d59.
This could cause tests 19, 704 and 1233 to fail on FreeBSD, AIX and
Solaris.
The error code should not be sent as data as it isn't passed onto the
client as body data, so cannot be compared in the test suite against
expected data.
* Added information about the verify and expand commands to
CURLOPT_MAIL_RCPT.
* Reworked CURLOPT_CUSTOMREQUEST section, adding information about IMAP
and SMTP custom commands.
Although this option should have already been set, the SMTP module can
now download information from and send instructional commands to, an
SMTP server, requiring the option to be set in order to perform a mail
transfer.
Incorrectly processed multiline server greeting responses as "wanted"
continue responses in smtp_endofresp(), from commit f16c0de4e9,
which in turn broke the SMTP server detection in the test suite,
because the EHLO command would not be sent.
Added a loop to smtp_statemach_act() in which Curl_pp_readresp() is
called until the cache is drained. Without this multiple responses
received in a single packet could result in a hang or delay.
Similar to the processing of untagged CAPABILITY responses in IMAP moved
the processing of multiline EHLO responses to smtp_state_ehlo_resp() and
introduced an internal response code of one to differentiate a multiline
continuation from the end of command. This also allows for the separate
processing of multiline responses from commands such as VRFY and EXPN.
singleipconnect() did not return the open socket descriptor on some
errors, thereby sometimes causing a socket leak. This patch ensures
the socket is always returned.
The ipv6 auto-detect test in configure returns a false negative when
CFLAGS contains -Werror=implicit-function-declaration. (I have been
using this flag to detect code issues that would result in SEGVs on
x86_64-cygwin.)
Patch-by: Yaakov Selkowitz
Bug: http://curl.haxx.se/bug/view.cgi?id=1304
Even though this is only a formality (since not many people build on
Mavericks while targeting Leopard), since we still support Leopard
at the earliest, we might as well be pedantic.
It turns out that some of the constants necessary to make this feature
work are missing from Snow Leopard's Security framework even though
they are defined in the headers.
Bug: http://curl.haxx.se/mail/lib-2013-11/0076.html
Reported by: myriachan
Rather than set the authentication options as part of the login details
specified in the URL, or via the older CURLOPT_USERPWD option, added a
new libcurl option to allow the login options to be set separately.
This patch fixes and issue introduced in commit 7d7df83198, if the
tunnel state was TUNNEL_CONNECT, waitconnect_getsock() would return a
bitmask indicating a readable socket but never stored the socket in the
return array.
As the IMAP regex could fail and $1 would not contain the command id
updated the unrecognised command response to be more generic and
realistic (like those used in the command handlers).
Additionally updated the POP3, SMTP and FTP responses.
Our own printf() replacement clearly can't properly handle %.*s with a
string that isn't zero terminated. Instead of fixing the printf code or
even figuring out what the proper posix behavior is, I reverted this
piece of the code back to the previous version where it does malloc +
memcpy instead.
Regression added in e839446c2a, released in curl 7.32.0.
Reported-by: Felix Yan
Bug: http://curl.haxx.se/bug/view.cgi?id=1295
This patch adds a 200ms delay between the first and second address
family socket connection attempts.
It also iterates over IP addresses in the order returned by the
system, meaning most dual-stack systems will try IPv6 first.
Additionally, it refactors the connect code, removing most code that
handled synchronous connects. Since all sockets are now non-blocking,
the logic can be made simpler.
nss.c:702: warning: pointer targets in passing argument 3 of
'Curl_extract_certinfo' differ in signedness
nss.c:702: warning: pointer targets in passing argument 4 of
'Curl_extract_certinfo' differ in signedness
Made sure the cast was correctly "unsigned char *" to "char *" and not
"unsigned char *" to "unsigned char *".
nss.c:700: warning: pointer targets in passing argument 3 of
'Curl_extract_certinfo' differ in signedness
nss.c:700: warning: pointer targets in passing argument 4 of
'Curl_extract_certinfo' differ in signedness
Introduced in commit 7d7df83198 curl would loop displaying "Whut?"
if it was trying to connect to an address and port that didn't have
anything listening on it.
Renamed copy_header_value() to Curl_copy_header_value() as this
function is now non static.
Simplified proxy flag in Curl_http_input_auth() when calling
sub-functions.
Removed unnecessary white space removal when using negotiate as it had
been missed in commit cdccb42267.
...following recent changes to Curl_base64_decode() rather than trying
to parse a header line for the authentication mechanisms which is CRLF
terminated and inline zero terminate it.
...following recent changes to Curl_base64_decode() rather than trying
to parse a header line for the authentication mechanisms which is CRLF
terminated and inline zero terminate it.
The code rejected 0 as a valid timeout while in fact the function could
indeed legitimately return that and it should be respected.
Reported-by: Bjorn Stenberg
A base64 string should be a multiple of 4 characters in length, not
contain any more than 2 padding characters and only contain padding
characters at the end of string. For example: Y3VybA==
Strings such as the following are considered invalid:
Y= - Invalid length
Y== - Invalid length
Y=== - More than two padding characters
Y=x= - Padding character contained within string
This patch fixes a bug in Happy Eyeballs where curl would wait for a
connect response from socket1 before checking socket2.
Also, it updates error messages for failed connections, showing the ip
addresses that failed rather than just the host name repeatedly.
Bug: http://curl.haxx.se/mail/lib-2013-10/0236.html
Reported-by: Paul Marks
"Dan Fandrich" <dan@coneharvesters.com> wrote:
>> But I'm not sure <unistd.h> is needed at all.
>
> It's needed for close(2). But the only reason that's needed is because fstat
> is used instead of stat(2); if you fix that, then you could remove that
> include altogether.
Okay. I've tested the following with MSVC and MingW. htttput.c now
simply uses stat():
This patch invokes two socket connect()s nearly simultaneously, and
the socket that is first connected "wins" and is subsequently used for
the connection. The other is terminated.
There is a very slight IPv4 preference, in that if both sockets connect
simultaneously IPv4 is checked first and thus will win.
Should a client application fail to decode an authentication message
received from a server, or not support any of the parameters given by
the server in the message, then the authentication phrase should be
cancelled gracefully by the client rather than simply terminating the
connection.
The authentication phrase should be cancelled by simply sending a '*'
to the server, in response to erroneous data being received, as per
RFC-3501, RFC-4954 and RFC-5034.
This patch adds the necessary state machine constants and appropriate
response handlers in order to add this functionality for the CRAM-MD5,
DIGEST-MD5 and NTLM authentication mechanisms.
This is a regression since the switch to always-multi internally
c43127414d.
Test 1316 was modified since we now clearly call the Curl_client_write()
function when doing the LIST transfer part and then the
handler->protocol says FTP and ftpc.transfertype is 'A' which implies
text converting even though that the response is initially a HTTP
CONNECT response in this case.
This workaround had been previously been implemented for IMAP and POP3
but not SMTP. Some of the recent test case additions implemented this
behaviour to emulate a bad server and the SMTP code didn't cope with it.
As the URI, which is contained within the DIGEST-MD5 response, is
constructed from the service and realm, the encoded message differs
from that generated under POP3.
From wikipedia:
Travis CI is a hosted, distributed continuous integration service used
to build and test projects hosted at GitHub.
Travis CI is configured by adding a file named .travis.yml, which is a
YAML format text file, to the root directory of the GitHub repository.
Travis CI automatically detects when a commit has been made and pushed
to a GitHub repository that is using Travis CI, and each time this
happens, it will try to build the project and run tests. This includes
commits to all branches, not just to the master branch. When that
process has completed, it will notify a developer in the way it has been
configured to do so — for example, by sending an email containing the
test results (showing success or failure), or by posting a message on an
IRC channel. It can be configured to run the tests on a range of
different machines, with different software installed (such as older
versions of a programming language, to test for compatibility).
... if not already initialized. This fixes a regression introduced by
commit 4ad8e142da, which caused test619
to intermittently fail on certain machines (namely Fedora build hosts).
Changed the failure code when TLS v1.1 and v1.2 is requested but not
supported by older OpenSSL versions, following review from libcurl
peers, and reduced the number of required preprocessor if statements.
...with the use of CURL_SSLVERSION_TLSv1_1 and CURL_SSLVERSION_TLSv1_2
being conditional on OpenSSL v1.0.1 as the appropriate flags are not
supported under earlier versions.
Commit ad34a2d5c8 relies on definitions that are only present in
OpenSSL v1.0.1 and up. This quick fix allows the builds that use
older versions of OpenSSL to continue building.
According to the documentation for libssh2_userauth_list(), a NULL
return value is not necessarily an error. You must call
libssh2_userauth_authenticated() to determine if the SSH_USERAUTH_NONE
request was successful.
This fixes a segv when using sftp on a server that allows logins with an
empty password. When NULL was interpreted as an error, it would
free the session but not flag an error since the libssh2 errno would be
clear. This resulted in dereferencing a NULL session pointer.
Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
CURL_SSLVERSION_TLSv1_0, CURL_SSLVERSION_TLSv1_1,
CURL_SSLVERSION_TLSv1_2 enum values are added to force exact TLS version
(CURL_SSLVERSION_TLSv1 means TLS 1.x).
axTLS:
axTLS only supports TLS 1.0 and 1.1 but it cannot be set that only one
of these should be used, so we don't allow the new enum values.
darwinssl:
Added support for the new enum values.
SChannel:
Added support for the new enum values.
CyaSSL:
Added support for the new enum values.
Bug: The original CURL_SSLVERSION_TLSv1 value enables only TLS 1.0 (it
did the same before this commit), because CyaSSL cannot be configured to
use TLS 1.0-1.2.
GSKit:
GSKit doesn't seem to support TLS 1.1 and TLS 1.2, so we do not allow
those values.
Bugfix: There was a typo that caused wrong SSL versions to be passed to
GSKit.
NSS:
TLS minor version cannot be set, so we don't allow the new enum values.
QsoSSL:
TLS minor version cannot be set, so we don't allow the new enum values.
OpenSSL:
Added support for the new enum values.
Bugfix: The original CURL_SSLVERSION_TLSv1 value enabled only TLS 1.0,
now it enables 1.0-1.2.
Command-line tool:
Added command line options for the new values.
The option '--bearer' might be slightly ambiguous in name. It doesn't
create any conflict that I am aware of at the moment, however, OAUTH v2
is not the only authentication mechanism which uses "bearer" tokens.
Reported-by: Kyle L. Huff
URL: http://curl.haxx.se/mail/lib-2013-10/0064.html
Setting only CURLOPT_SSL_VERIFYHOST without CURLOPT_SSL_VERIFYPEER set
should still verify that the host name fields in the server certificate
is fine or return failure.
Bug: http://curl.haxx.se/mail/lib-2013-10/0002.html
Reported-by: Ishan SinghLevett
...as it is no longer required following capability and authentication
changes and is now causing problems following commit 49341628b5 as
the test number is obtained from the client address in the EHLO.
...to the client address as this frees the RCPT strings to contain
just an email address and by passing the test number into curl as the
client address remains consistent with POP3 and IMAP tests as they are
specified in the URL.
If no WINVER and/or _WIN32_IWNNT define was set, the Windows platform
SDK often defaults to high value, e.g. 0x601 (whoch may probably depend
on the Windows version being used, in my case Windows 7).
If WINVER >= 0x600 then winsock2.h includes some defines for WSAPoll(),
e.g. POLLIN, POLLPRI, POLLOUT etc. These defines clash with cURL's
lib/select.h.
Make sure HAVE_STRUCT_POLLFD is defined then.
Bug: http://curl.haxx.se/bug/view.cgi?id=1282
Reported-by: "kdekker"
Patch-by: Marcel Raad
The latest version(s) of the icc compiler no longer accept the extra
space in the -we (warning enable), -wd (warning disable), etc.
Reported-by: Elmira A Semenova
Bug: http://curl.haxx.se/mail/lib-2013-09/0182.html
Moved the standard SASL mechanism strings into curl_sasl.h rather than
hard coding the same values over and over again in the protocols that
use SASL authentication.
For more information about the mechanism strings see:
http://www.iana.org/assignments/sasl-mechanisms
As someone reported on the mailing list a while back, the hard-coded
arbitrary timeout of 7s in test 1112 is not sufficient in some build
environments. At Arista Networks we build and test curl as part of our
automated build system, and we've run into this timeout 170 times so
far. Our build servers are typically quite busy building and testing a
lot of code in parallel, so despite being beefy machines with 32 cores
and 128GB of RAM we still hit this 7s timeout regularly.
URL: http://curl.haxx.se/mail/lib-2010-02/0200.html
As curl will send a HELO command after an negative EHLO response, added
the same detection from commit b07709f741 to the HELO handler to
ensure the test server is identified correctly and an upload isn't
performed.
Rather than detecting the TO address as missing in the DATA handler,
moved the detection to the RCPT command handler where an error response
can be generated.
Added missing information, from curl 7.31.0, regarding the use of the
optional login options that may be specified as part of --user.
For example:
--user 'user:password;auth=NTLM' in IMAP, POP3 and SMTP protocols.
Moved the special SMTP server detection code from the DATA command
handler, which happens further down the operation chain after EHLO,
MAIL and RCPT commands, to the EHLO command as it is the first command
to be generated by a SMTP operation as well as containing the special
"verifiedserver" string from the URL.
This not only makes it easier and quicker to detect but also means that
cURL doesn't need to specify "verifiedserver" as --mail-from and
--mail-rcpt arguments.
More importantly, this also makes the upcoming verification changes to
the RCPT handler easier to implement.
In ossl_connect_step2() when the "Unknown SSL protocol error" occurs, it
would output the local port number instead of the remote one which
showed when doing SSL over a proxy (but with the correct remote host
name). As libcurl only speaks SSL to the remote we know it is the remote
port.
Bug: http://curl.haxx.se/bug/view.cgi?id=1281
Reported-by: Gordon Marler
The libcurl date parser returns INT_MAX for all dates > 2037 so this
test is now made to use 2037 instead of 2038 to work the same for both
32bit and 64bit time_t systems.
Added the ability to use an XOAUTH2 bearer token [RFC6750] with POP3 for
authentication using RFC6749 "OAuth 2.0 Authorization Framework".
The bearer token is expected to be valid for the user specified in
conn->user. If CURLOPT_XOAUTH2_BEARER is defined and the connection has
an advertised auth mechanism of "XOAUTH2", the user and access token are
formatted as a base64 encoded string and sent to the server as
"AUTH XOAUTH2 <bearer token>".
iOS 7 finally added the option to enable 1/n-1 when using TLS 1.0
and a CBC cipher, so we now always turn that on unless the user
manually turns it off using CURLSSLOPT_ALLOW_BEAST.
It appears Apple also added some new PSK ciphers, but no interface to
use them yet, so we at least support printing them if we find them.
Following changes to ftpserver.pl fixed the mail from address to be a
correctly formatted address otherwise the server response will be 501
Invalid address.
MAIl_smtp() will now check for a correctly formatted FROM address as
well as the optional SIZE parameter comparing it against the server
capability when specified.
Implement: Expired Cookies These following situation, curl removes
cookie(s) from struct CookieInfo if the cookie expired.
- Curl_cookie_add()
- Curl_cookie_getlist()
- cookie_output()
Commit 32352ed6ad introduced various DNS options, however, these
would cause curl to exit with CURLE_NOT_BUILT_IN when c-ares wasn't
being used as the backend resolver even if the options weren't set
by the user.
Additionally corrected some minor coding style errors from the same
commit.
Solaris with the SunStudio Compiler is reportedly missing this define,
but as we're using it without any good reason on all the places it was
used I've now instead switched to just use sensible buffer sizes that
fit a 32 bit decimal number. Which also happens to be smaller than the
common NI_MAXSERV value which is 32 on most machines.
Bug: http://curl.haxx.se/bug/view.cgi?id=1277
Reported-by: D.Flinkmann
Otherwise, the FTP protocol would unnecessarily hang 60 seconds if
aborted in the CURLOPT_HEADERFUNCTION callback.
Reported by: Tomas Mlcoch
Bug: https://bugzilla.redhat.com/1005686
Make sure that the custom struct fields are only used by code that
doesn't use a struct defintion from the outside.
Attempts to fix the problem introduced in 3dc6fc42bf
Otherwise the connection can get stuck during various phases, waiting
for new data on the socket using select() etc., but it will never be
received as the data has already been read into SSL library.
The transfer size would be calculated incorrectly if the email contained
within the FETCH response, had been partially received by the pingpong
layer. As such the following, example output, would be seen if the
amount remaining was smaller than the amount received:
* Excess found in a non pipelined read: excess = 1394, size = 262,
maxdownload = 262, bytecount = 1374
* transfer closed with -1112 bytes remaining to read
Bug: http://curl.haxx.se/mail/lib-2013-08/0170.html
Reported-by: John Dunn
Renamed SUPPORTAUTH to AUTH and added support for specifying a list of
supported SASL mechanisms to return to the client.
Additionally added the directive to the FILEFORMAT document.
Renamed SUPPORTCAPA to CAPA and added support for specifying a list of
supported capabilities to return to the client.
Additionally added the directive to the FILEFORMAT document.
The message numbers given in the LIST response are an index into the
list, which are only valid for the current session, rather than being a
unique message identifier. An index would only be missing from the LIST
response if a DELE command had been issued within the same session and
had not been committed by the end of session QUIT command. Once
committed the POP3 server will regenerate the message numbers in the
next session to be contiguous again. As such our LIST response should
list message numbers contiguously until we support a DELE command in the
same session.
Should a POP3 user require the unique message ID for any or all
messages then they should use the extended UIDL command. This command
will be supported by the test ftpserver in an upcoming commit.
When building the code using LLVM Clang without NGHTTP2, I was getting
this warning:
../lib/http.h:155:1: warning: empty struct is a GNU extension [-Wgnu]
Placing a dummy variable into the data structure silenced the warning.
The compiler test used a variable before it was assigned when it tried
to see how it acts on a mismatching prototype, which could cause a false
positive.
Recent OpenSSL uses user interface abstraction to negotiate access to
private keys in the cryprographical engines. An OpenSSL application is
expected to implement the user interface. Otherwise a default one
provided by OpenSSL (interactive standard I/O) will be used and the
aplication will have no way how to pass a password to the engine.
Longer-desc: http://curl.haxx.se/mail/lib-2013-08/0265.html
Using the first little merge of nghttp2 into libcurl, I stumbeled on the
missing 'snprintf' in MSVCRT. Isn't this how we do it for other libcurl
files? I.e. use 'curl_msnprintf' and not 'snprintf' directly:
When an error occurs parsing an LDAP URL, The ludp->lud_attrs[i] entries
could be freed even though they sometimes point to data within an
allocated area.
This change introduces a lud_attrs_dup[] array for the duplicated string
pointers, and it removes the unused lud_exts array.
Bug: http://curl.haxx.se/mail/lib-2013-08/0209.html
XOAUTH2 would be selected in preference to LOGIN and PLAIN if the IMAP
or SMTP server advertised support for it even though a user's password
was supplied but bearer token wasn't.
Modified the selection logic so that XOAUTH2 will only be selected if
the server supports it and A) The curl user/libcurl programmer has
specifically asked for XOAUTH via the ;AUTH=XOAUTH login option or 2)
The bearer token is specified. Obviously if XOAUTH is asked for via
the login option but no token is specified the user will receive a
authentication failure which makes more sense than no known
authentication mechanisms supported!
The commit 7b074a460b to CURL_CHECK_CA_BUNDLE in 7.31 (don't check
for paths when cross-compiling) causes --without-ca-path to no longer
works when cross-compiling, since ca and capath only ever get set to
"no" when not cross-compiling, I attach a patch that works for me. Also
in the cross-compilation case, no ca-path seems to be a better default
(IMVHO) than empty ca-path.
Bug: http://curl.haxx.se/bug/view.cgi?id=1273
Patch-by: Stefan Neis
'struct monitor', introduced in 6cf8413e, already exists in an IRIX
header file (sys/mon.h) which gets included via various standard headers
by lib/easy.c
cc-1101 cc: ERROR File = ../../curl/lib/easy.c, Line = 458
"monitor" has already been declared in the current scope.
Reported-by: Tor Arntsen
Corrected the call to logmsg() in the IMAP SEARCH handler from commit
4ae7b7ea69 as it should have been outputting the what argument and
not the test number.
When waiting for a 100-continue response from the server, the
Curl_readwrite() will refuse to run if called until the timeout has been
reached.
We timeout code in multi_socket() allows code to run slightly before the
actual timeout time, so for test 154 it could lead to the function being
executed but refused in Curl_readwrite() and then the application would
just sit idling forever.
This was detected with runtests.pl -e on test 154.
Moved the calls to checkpasswd() out of the getparameter() function
which allows for any related arguments to be specified on the command
line before or after --user (and --proxy-user).
For example: --bearer doesn't need to be specified before --user to
prevent curl from asking for an unnecessary password as is the case
with commit e7dcc454c6.
Added the ability to specify an XOAUTH2 bearer token [RFC6750] via the
--bearer option.
Example usage:
curl --url "imaps://imap.gmail.com:993/INBOX/;UID=1" --ssl-reqd
--bearer ya29.AHES6Z...OMfsHYI --user username@example.com
I brought back security.h in commit bb55293313. As we actually
already found out back in 2005 in commit 62970da675, the file name
security.h causes problems so I renamed it curl_sec.h instead.
The specified curl binary will then be used to verify the running
server(s) instead of the development version. This is very useful in
some cases when the development version fails to verify correctly as
then the test case may not run at all.
The actual test will still be run with the "normal" curl executable
(unless the test case specifies something differently).
Added the ability to use an XOAUTH2 bearer token [RFC6750] with SMTP for
authentication using RFC6749 "OAuth 2.0 Authorization Framework".
The bearer token is expected to be valid for the user specified in
conn->user. If CURLOPT_XOAUTH2_BEARER is defined and the connection has
an advertised auth mechanism of "XOAUTH2", the user and access token are
formatted as a base64 encoded string and sent to the server as
"AUTH XOAUTH2 <bearer token>".
Added the ability to use an XOAUTH2 bearer token [RFC6750] with IMAP for
authentication using RFC6749 "OAuth 2.0 Authorization Framework".
The bearer token is expected to be valid for the user specified in
conn->user. If CURLOPT_XOAUTH2_BEARER is defined and the connection has
an advertised auth mechanism of "XOAUTH2", the user and access token are
formatted as a base64 encoded string and sent to the server as
"A001 AUTHENTICATE XOAUTH2 <bearer token>".
Added the ability to specify an XOAUTH2 bearer token [RFC6750] via the
option CURLOPT_XOAUTH2_BEARER for authentication using RFC6749 "OAuth
2.0 Authorization Framework".
Added the ability to generated a base64 encoded XOAUTH2 token
containing: "user=<username>^Aauth=Bearer <bearer token>^A^A"
as per RFC6749 "OAuth 2.0 Authorization Framework".
We've announced this pending removal for a long time and we've
repeatedly asked if anyone would care or if anyone objects. Nobody has
objected. It has probably not even been working for a good while since
nobody has tested/used this code recently.
The stuff in krb4.h that was generic enough to be used by other sources
is now present in security.h
Make sure we always return CURLM_CALL_MULTI_PERFORM when we reach
CURLM_STATE_DONE since the state is transient and it can very well
continue executing as there is nothing to wait for.
Bug: http://curl.haxx.se/mail/lib-2013-08/0211.html
Reported-by: Yi Huang
Renamed to "enum curl_khtype" now. Will break compilation for programs
that rely on the enum name.
Bug: https://github.com/bagder/curl/pull/76
Reported-by: Shawn Landden
... this also makes sure that the progess callback gets called more
often during TFTP transfers.
Added test 1238 to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=1269
Reported-by: Jo3
I build curl.exe (using MingW) with '-DCURLDEBUG' and by importing from
libcurl.dll. Which means the new curl_easy_perform_ev() must be
exported from libcurl.dll.
Doing curl_multi_add_handle() on an easy handle that is already added to
a multi handle now returns this error code. It previously returned
CURLM_BAD_EASY_HANDLE for this condition.
The closure_handle is "owned" by the multi handle and it is
unconditional so the setting up of it should be in the Curl_multi_handle
function rather than curl_multi_add_handle.
This function is meant to work *exactly* as curl_easy_perform() but will
use the event-based libcurl API internally instead of
curl_multi_perform(). To avoid relying on an actual event-based library
and to not use non-portable functions (like epoll or similar), there's a
rather inefficient emulation layer implemented on top of Curl_poll()
instead.
There's currently some convenience logging done in curl_easy_perform_ev
which helps when tracking down problems. They may be suitable to remove
or change once things seem to be fine enough.
curl has a new --test-event option when built with debug enabled that
then uses curl_easy_perform_ev() instead of curl_easy_perform(). If
built without debug, using --test-event will only output a warning
message.
NOTE: curl_easy_perform_ev() is not part if the public API on purpose.
It is only present in debug builds of libcurl and MUST NOT be considered
stable even then. Use it for libcurl-testing purposes only.
runtests.pl now features an -e command line option that makes it use
--test-event for all curl command line tests. The man page is updated.
libcurl quietly truncates usernames, passwords, and options from
before an '@' sign in a URL to 255 (= MAX_CURL_PASSWORD_LENGTH - 1)
characters to fit in fixed-size buffers on the stack. Allocate a
buffer large enough to fit the parsed fields on the fly instead to
support longer passwords.
After this change, there are no more uses of MAX_CURL_OPTIONS_LENGTH
left, so stop defining that constant while at it. The hardcoded max
username and password length constants, on the other hand, are still
used in HTTP proxy credential handling (which this patch doesn't
touch).
Reported-by: Colby Ranger
Instead of nesting "if(success)" blocks and leaving the reader in
suspense about what happens in the !success case, deal with failure
cases early, usually with a simple goto to clean up and return from
the function.
No functional change intended. The main effect is to decrease the
indentation of this function slightly.
libcurl truncates usernames, passwords, and options set with
curl_easy_setopt to 255 (= MAX_CURL_PASSWORD_LENGTH - 1) characters.
This doesn't affect the return value from curl_easy_setopt(), so from
the caller's point of view, there is no sign anything strange has
happened, except that authentication fails.
For example:
# Prepare a long (300-char) password.
s=0123456789; s=$s$s$s$s$s$s$s$s$s$s; s=$s$s$s;
# Start a server.
nc -l -p 8888 | tee out & pid=$!
# Tell curl to pass the password to the server.
curl --user me:$s http://localhost:8888 & sleep 1; kill $pid
# Extract the password.
userpass=$(
awk '/Authorization: Basic/ {print $3}' <out |
tr -d '\r' |
base64 -d
)
password=${userpass#me:}
echo ${#password}
Expected result: 300
Actual result: 255
The fix is simple: allocate appropriately sized buffers on the heap
instead of trying to squeeze the provided values into fixed-size
on-stack buffers.
Bug: http://bugs.debian.org/719856
Reported-by: Colby Ranger
libcurl truncates usernames and passwords it reads from .netrc to
LOGINSIZE and PASSWORDSIZE (64) characters without any indication to
the user, to ensure the values returned from Curl_parsenetrc fit in a
caller-provided buffer.
Fix the interface by passing back dynamically allocated buffers
allocated to fit the user's input. The parser still relies on a
256-character buffer to read each line, though.
So now you can include an ~246-character password in your .netrc,
instead of the previous limit of 63 characters.
Reported-by: Colby Ranger
Instead of remembering before each "return" statement which temporary
allocations, if any, need to be freed, take care to set pointers to
NULL when no longer needed and use a goto to a common block to exit
the function and free all temporaries.
No functional change intended. Currently the only temporary buffer in
this function is "proxy" which is already correctly freed when
appropriate, but there will be more soon.
Use appropriately sized buffers on the heap instead of fixed-size
buffers on the stack, to allow for longer usernames and passwords.
Callers never pass anything longer than MAX_CURL_USER_LENGTH (resp.
MAX_CURL_PASSWORD_LENGTH), so no functional change inteded yet.
The new multiply() function detects range value overflows. 32bit
machines will overflow on a 32bit boundary while 64bit hosts support
ranges up to the full 64 bit range.
Added test 1236 to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=1267
Reported-by: Will Dietz
A rather big overhaul and cleanup.
1 - curl wouldn't properly detect and reject globbing that ended with an
open brace if there were brackets or braces before it. Like "{}{" or
"[0-1]{"
2 - curl wouldn't properly reject empty lists so that "{}{}" would
result in curl getting (nil) strings in the output.
3 - By using strtoul() instead of sscanf() the code will now detected
over and underflows. It now also better parses the step argument to only
accept positive numbers and only step counters that is smaller than the
delta between the maximum and minimum numbers.
4 - By switching to unsigned longs instead of signed ints for the
counters, the max values for []-ranges are now very large (on 64bit
machines).
5 - Bumped the maximum number of globs in a single URL to 100 (from 10)
6 - Simplified the code somewhat and now it stores fixed strings as
single- entry lists. That's also one of the reasons why I did (5) as now
all strings between "globs" will take a slot in the array.
Added test 1234 and 1235 to verify. Updated test 87.
This commit fixes three separate bug reports.
Bug: http://curl.haxx.se/bug/view.cgi?id=1264
Bug: http://curl.haxx.se/bug/view.cgi?id=1265
Bug: http://curl.haxx.se/bug/view.cgi?id=1266
Reported-by: Will Dietz
Add the curl release notes to the release note document generated for
VMS packages.
Add the different filenames generated by a daily build to the
cleanup procedures.
This makes the socket callback get called with the proper bitmask as
otherwise the application could be left hanging waiting for reading on
an upload connection!
Bug: http://curl.haxx.se/mail/lib-2013-08/0043.html
Reported-by: Bill Doyle
With everything being struct SessionHandle pointers now, this rename
makes multi.c use the library-wide practise of calling that pointer
'data' instead of the previously used 'easy'.
Moved Curl_easy_addmulti() from easy.c to multi.c, renamed it to
easy_addmulti and made it static.
Removed Curl_easy_initHandleData() and uses of it since it was emptied
in commit cdda92ab67b47d74a.
All protocol handler structs are now opaque (void *) in the
SessionHandle struct and moved in the request-specific sub-struct
'SingleRequest'. The intension is to keep the protocol specific
knowledge in their own dedicated source files [protocol].c etc.
There's some "leakage" where this policy is violated, to be addressed at
a later point in time.
1 - always allocate the struct in protocol->setup_connection. Some
protocol handlers had to get this function added.
2 - always free at the end of a request. This is also an attempt to keep
less memory in the handle after it is completed.
CURLOPT_DNS_USE_GLOBAL_CACHE broke in commit c43127414d (been
broken since the libcurl 7.29.0 release). While this option has been
documented as deprecated for almost a decade and nobody even reported
this bug, it should remain functional.
Added test case 1512 to verify
The previous naming scheme ftp_state_post_XXXX() wasn't really helpful
as it wasn't always immediately after 'xxxx' and it wasn't easy to
understand what it does based on such a name.
This new one is instead ftp_state_yyyy() where yyyy describes what it
does or sends.
Since the mk-ca-bundle tool itself isn't installed with make install,
there's no point in installing its documentation.
Bug: http://curl.haxx.se/mail/lib-2013-08/0057.html
Reported-by: Guenter Knauf
These are the files needed to build VMS distribution packages known as
PCSI kits.
Also minor update to the existing files, mainly to the documentation and
file clean up code.
This is the first version of this new document, detailing the seven
perhaps most important internal structs in libcurl source code:
1.1 SessionHandle
1.2 connectdata
1.3 Curl_multi
1.4 Curl_handler
1.5 conncache
1.6 Curl_share
1.7 CookieInfo
This is a regression as this logic used to work. It isn't clear when it
broke, but I'm assuming in 7.28.0 when we went all-multi internally.
This likely never worked with the multi interface. As the failed
connection is detected once the multi state has reached DO_MORE, the
Curl_do_more() function was now expanded somewhat so that the
ftp_do_more() function can request to go "back" to the previous state
when it makes another attempt - using PASV.
Added test case 1233 to verify this fix. It has the little issue that it
assumes no service is listening/accepting connections on port 1...
Reported-by: byte_bucket in the #curl IRC channel
For some reason, OS X 10.5's GCC suddenly stopped working correctly with
macros that change MD5_Init etc. in the code to CC_MD5_Init etc., so I
worked around this by removing use of the macros and inserting static
functions that just call CommonCrypto's implementations of the functions
instead.
This changes the previous check for untrusted certs to a check for
certs explicitely marked as trusted.
The change is backward-compatible (tested with certdata.txt v1.80).
The internal function that's used to detect known file extensions for
the default Content-Type got the the wrong pointer passed in when
CURLFORM_BUFFER + CURLFORM_BUFFERPTR were used. This had the effect that
strlen() would be used which could lead to an out-of-bounds read (and
thus segfault). In most cases it would only lead to it not finding or
using the correct default content-type.
It also showed that test 554 and test 587 were testing for the
previous/wrong behavior and now they're updated as well.
Bug: http://curl.haxx.se/bug/view.cgi?id=1262
Reported-by: Konstantin Isakov
Christian Heimes brought to our attention that the certdata.txt
format has recently changed [1], causing ca-bundle.crt created
with mk-ca-bundle.[pl|vbs] to include untrusted certs.
[1] http://lists.debian.org/debian-release/2012/11/msg00411.html
Instead of claiming it is an error, we call it a "note" to reduce the
severity level. But the following text now says the [variable] "*should*
only be used to specify"... instead of previously having said "may".
The motivation for having a separate struct that keep track of an easy
handle when using the multi handle was removed when we switched to
always using the multi interface internally. Now they were just two
separate struct that was always allocated for each easy handle.
This first step just moves the Curl_one_easy struct members into the
SessionHandle struct and hides this somehow (== keeps the source code
changes to a minimum) by defining Curl_one_easy to SessionHandle
The biggest changes in this commit are:
1 - the linked list of easy handles had to be changed somewhat due
to the new struct layout. This made the main linked list pointer
get renamed to 'easyp' and there's also a new pointer to the last
node, called easylp. It is no longer circular but ends with ->next
pointing to NULL. New nodes are still added last.
2 - easy->state is now called easy->mstate to avoid name collision
Update the packages/vms/readme file to be current.
Also some files for the GNV based build were either missing or needed an
update.
curl_crtl_init.c is a special file that is run before main() to
set up the proper C runtime behavior.
generate_vax_transfer.com generates the VAX transfer vector modules from
the gnv_libcurl_symbols.opt file.
gnv_conftest.c_first is a helper file needed for configure scripts to
come up with the expected answers on VMS.
gnv_libcurl_symbols.opt is the public symbols for the libcurl shared
image.
gnv_link_curl.com builds the shared libcurl image and rebuilds other
programs to use it.
macro32_exactcase.patch is a hack to make a local copy of the VMS Macro32
assembler case sensitive, which is needed to build the VAX transfer modules.
report_openssl_version.c is a tool for help verify that the libcurl
shared image is being built for a minium version of openssl.
Also, use memset() instead of a lame loop.
The previous logic that tried to avoid too many updates were very
ineffective for really fast transfers, as then it could easily end up
doing hundreds of updates per second that would make a significant
impact in transfer performance!
Bug: http://curl.haxx.se/mail/archive-2013-07/0031.html
Reported-by: Marc Doughty
(This doesn't need to appear in the release notes.) I noticed a few places
where infof() was called, and there should've been an LF at the end of the
string, but there wasn't.
It turns out Snow Leopard not only has SecItemCopyMatching() defined in
a header not included by the omnibus header, but it won't work for our
purposes, because searching for SecIdentityRef objects wasn't added
to that API until Lion. So we now use the old SecKeychainSearch API
instead if the user is building under, or running under, Snow Leopard.
Bug: http://sourceforge.net/p/curl/bugs/1255/
Reported by: Edward Rudd
Previously we used __MAC_10_X and __IPHONE_X to mark digest-generating
code that was specific to OS X and iOS. Now we use
__MAC_OS_X_VERSION_MAX_ALLOWED and __IPHONE_OS_VERSION_MAX_ALLOWED
instead of those macros.
Bug: http://sourceforge.net/p/curl/bugs/1255/
Reported by: Edward Rudd
For the standard VMS text file formats, VMS needs to read the file to
get the actual file size.
For the standard VMS binary file formats, VMS needs a special format of
fopen() call so that it stops reading at the logical end of file instead
of at the end of the blocks allocated to the file.
I structured the patch this way as I was not sure about changing the
structures or parameters to the routines, but would prefer to only call
the stat() function once and pass the information to where the fopen()
call is made.
Bug: https://sourceforge.net/p/curl/bugs/758/
The code for CURLFORM_FILECONTENT had its check for duplicate options
wrong so that it would reject CURLFORM_PTRNAME if used in combination
with it (but not CURLFORM_COPYNAME)! The flags field used for this
purpose cannot be interpreted that broadly.
Bug: http://curl.haxx.se/mail/lib-2013-07/0258.html
Reported-by: Byrial Jensen
Linking on Solaris 10 x86 with Sun Studio 12 failed when we upgraded
automake for the release builds.
Bug: http://curl.haxx.se/bug/view.cgi?id=1217
Reported-by: Dagobert Michelsen
Two fixes:
1. Force output file format to be stream-lf so that partial downloads
can be continued.
This should have minor impact as if the file does not exist, it was
created with stream-lf format. The only time this was an issue is if
there was already an existing file with a different format.
2. Fix file uploads are now fixed.
a. VMS binary files such as ZIP archives are now uploaded
correctly.
b. VMS text files are read once to get the correct size
and then converted to line-feed terminated records as
they are read into curl.
The default VMS text formats do not contain either line-feed or
carriage-return terminated records. Those delimiters are added by the
operating system file read calls if the application requests them.
Bug: http://curl.haxx.se/bug/view.cgi?id=496
We no longer pass our 'bool' data type variables nor constants as
an argument to my_setopt(), instead we use proper 1L or 0L values.
This also fixes macro used to pass string argument for CURLOPT_SSLCERT,
CURLOPT_SSLKEY and CURLOPT_EGDSOCKET using my_setopt_str() instead of
my_setopt().
This also casts enum or int argument data types to long when passed to
my_setopt_enum().
Commit 6d30f8ebed didn't work properly. First, it used the wrong
array index, but this fix also:
1 - only does the copying if indeed there was any activity
2 - makes sure to properly translate between internal and external
bitfields, which are not guaranteed to match
Reported-by: Evgeny Turnaev
Instead of going 50,100,150 etc millisecond delay time when nothing has
been found to do or wait for, we now start lower and double each loop as
in 4,8,16,32 etc.
This lowers the minimum wait without sacrifizing the longer wait too
much with unnecessary CPU cycles burnt.
Bug: http://curl.haxx.se/mail/lib-2013-07/0103.html
Reported-by: Andreas Malzahn
In the case of an active connection when ftp_do_more() detects that the
server has connected back, it must make sure to mark it as complete so
that the multi_runsingle() function will detect this and move on to the
next state.
Bug: http://curl.haxx.se/mail/lib-2013-07/0115.html
Reported-by: Clemens Gruber
CURLOPT_XFERINFOFUNCTION is now the preferred progress callback function
and CURLOPT_PROGRESSFUNCTION is considered deprecated.
This new callback uses pure 'curl_off_t' arguments to pass on full
resolution sizes. It otherwise retains the same characteristics: the
same call rate, the same meanings for the arguments and the return code
is used the same way.
The progressfunc.c example is updated to show how to use the new
callback for newer libcurls while supporting the older one if built with
an older libcurl or even built with a newer libcurl while running with
an older.
This reverts commit 7ed25cc, reinstating commit 8ec2cb5.
As of 18-jul-2013 we still do have code in libcurl that makes use of these
memory functions. Commit 8ec2cb5 comment still applies and is yet valid.
These memory functions are solely used in Windows builds, so all related
code is protected with '#ifdef WIN32' preprocessor conditional compilation
directives.
Specifically, wcsdup() _wcsdup() are used when building a Windows target with
UNICODE and USE_WINDOWS_SSPI preprocessor symbols defined. This is the case
when building a Windows UNICODE target with Windows native SSL/TLS support
enabled.
Realizing that wcsdup() _wcsdup() are used is a bit tricky given that usage
of these is hidden behind _tcsdup() which is MS way of dealing with code
that must tolerate UNICODE and non-UNICODE compilation. Additionally, MS
header files and those compatible from other compilers use this preprocessor
conditional compilation directive in order to select at compilation time
whether 'wide' or 'ansi' MS API functions are used.
Without this code, Windows build targets with Windows native SSL/TLS support
enabled and MemoryTracking support enabled misbehave in tracking memory usage,
regardless of being a UNICODE enabled build or not.
Fixed issue with static build for MSVC2010.
After some investigation I've discovered known issue
http://public.kitware.com/Bug/view.php?id=11240 When .rc file is linked
to static lib it fails with following linker error
LINK : warning LNK4068: /MACHINE not specified; defaulting to X86
file.obj : fatal error LNK1112: module machine type 'x64' conflicts with
target machine type 'X86'
Fix add target property /MACHINE: for MSVC generation.
Also removed old workarounds - it caused errors during msvc build.
Bug: http://curl.haxx.se/mail/lib-2013-07/0046.html
Implement wrappers around strtod to convert the user argument to a
double with sane error checking. Use this to allow --max-time and
--connect-timeout to accept decimal values instead of strictly integers.
The manpage is updated to make mention of this feature and,
additionally, forewarn that the actual timeout of the operation can
vary in its precision (particularly as the value increases in its
decimal precision).
strto* functions happily chomp off leading whitespace, so simply
checking for str[0] can lead to false negatives. Do the full parse and
check the out value instead.
Update the build_vms.com to detect and use zlib shared image installed
by the ZLIB kit produced by Jean-Francois Pieronne, and the also the
future ZLIB 1.2.8 kit in addition to the older ZLIB kits.
Also fix the indentation to match one of the common standards used for
VMS DCL command files and removed the hard tab characters.
Tested on OpenVMS 8.4 Alpha and IA64, and OpenVMS 7.3 VAX.
I just noticed that OS X no longer supports SSLv2. Other TLS engines return
an error if the requested protocol isn't supported by the underlying
engine, so we do that now for SSLv2 if the framework returns an error
when trying to turn on SSLv2 support. (Note: As always, SSLv2 support is
only enabled in curl when starting the app with the -2 argument; it's off
by default. SSLv2 is really old and insecure.)
When doing multi-part formposts, libcurl used a pseudo-random value that
was seeded with time(). This turns out to be bad for users who formpost
data that is provided with users who then can guess how the boundary
string will look like and then they can forge a different formpost part
and trick the receiver.
My advice to such implementors is (still even after this change) to not
rely on the boundary strings being cryptographically strong. Fix your
code and logic to not depend on them that much!
I moved the Curl_rand() function into the sslgen.c source file now to be
able to take advantage of the SSL library's random function if it
provides one. If not, try to use the RANDOM_FILE for seeding and as a
last resort keep the old logic, just modified to also add microseconds
which makes it harder to properly guess the exact seed.
The formboundary() function in formdata.c is now using 64 bit entropy
for the boundary and therefore the string of dashes was reduced by 4
letters and there are 16 hex digits following it. The total length is
thus still the same.
Bug: http://curl.haxx.se/bug/view.cgi?id=1251
Reported-by: "Floris"
When using %x, the number must be treated as unsigned as otherwise it
would get sign-extended on for example 64bit machines and do wrong
output. This problem showed when doing printf("%08x", 0xffeeddcc) on a
64bit host.
Follow-up fix from 7d80ed64e4.
The SessionHandle may not be around to use when we restore the sigpipe
sighandler so we store the no_signal boolean in the local struct to know
if/how to restore.
When the c-ares based resolver backend failed to resolve a name, it
tried to show the name that failed from existing structs. This caused
the wrong output and shown hostname when for example --interface
[hostname] was used and that name resolving failed.
Now we use the hostname used in the actual resolve attempt in the error
message as well.
Bug: http://curl.haxx.se/bug/view.cgi?id=1191
Reported-by: Kim Vandry
When we recently started to treat a zero return code from SSL_read() as
an error we also got false positives - which primarily looks to be
because the OpenSSL documentation is wrong and a zero return code is not
at all an error case in many situations.
Now ossl_recv() will check with ERR_get_error() to see if there is a
stored error and only then consider it to be a true error if SSL_read()
returned zero.
Bug: http://curl.haxx.se/bug/view.cgi?id=1249
Reported-by: Nach M. S.
Patch-by: Nach M. S.
Something (a recent security update maybe?) changed in Lion, and now it
has changed SSLCopyPeerTrust such that it may return noErr but also give
us a null trust, which caught us off guard and caused an eventual crash.
... and restore the ordinary handling again when it returns. This is
done for curl_easy_perform() and curl_easy_cleanup() only for now - and
only when built to use OpenSSL as backend as this is the known culprit
for the spurious SIGPIPEs people have received.
Bug: http://curl.haxx.se/bug/view.cgi?id=1180
Reported by: Lluís Batlle i Rossell
This doesn't need to be in the release notes. I cleaned up a lot of the #if
lines in the code to use MAC_OS_X_VERSION_MIN_REQUIRED and
MAC_OS_X_VERSION_MAX_ALLOWED instead of checking for whether things like
__MAC_10_6 or whatever were defined, because for some SDKs Apple has released
they were defined out of place.
RFC3986 details how a path part passed in as part of a URI should be
"cleaned" from dot sequences before getting used. The described
algorithm is now implemented in lib/dotdot.c with the accompanied test
case in test 1395.
Bug: http://curl.haxx.se/bug/view.cgi?id=1200
Reported-by: Alex Vinnik
Security problem: CVE-2013-2174
If a program would give a string like "%FF" to curl_easy_unescape() but
ask for it to decode only the first byte, it would still parse and
decode the full hex sequence. The function then not only read beyond the
allowed buffer but it would also deduct the *unsigned* counter variable
for how many more bytes there's left to read in the buffer by two,
making the counter wrap. Continuing this, the function would go on
reading beyond the buffer and soon writing beyond the allocated target
buffer...
Bug: http://curl.haxx.se/docs/adv_20130622.html
Reported-by: Timo Sirainen
As a remedy to the problem when a socket gets closed and a new one is
opened with the same file descriptor number and as a result
multi.c:singlesocket() doesn't detect the difference, the new function
Curl_multi_closed() gets told when a socket is closed so that it can be
removed from the socket hash. When the old one has been removed, a new
socket should be detected fine by the singlesocket() on next invoke.
Bug: http://curl.haxx.se/bug/view.cgi?id=1248
Reported-by: Erik Johansson
When performing COOKIELIST operations the cookie lock needs to be taken
for the cases where the cookies are shared among multiple handles!
Verified by Benjamin Gilbert's updated test 506
Bug: http://curl.haxx.se/bug/view.cgi?id=1215
Reported-by: Benjamin Gilbert
When curl_multi_wait() finds no file descriptor to wait for, it returns
instantly and this must be handled gracefully within curl_easy_perform()
or cause a busy-loop. Starting now, repeated fast returns without any
file descriptors is detected and a gradually increasing sleep will be
used (up to a max of 1000 milliseconds) before continuing the loop.
Bug: http://curl.haxx.se/bug/view.cgi?id=1238
Reported-by: Miguel Angel
The initial fix to only compare full path names were done in commit
04f52e9b4d but found out to be incomplete. This takes should make the
change more complete and there's now two additional tests to verify
(test 31 and 62).
By always returning the md5 for an empty body when auth-int is asked
for, libcurl now at least sometimes does the right thing.
Bug: http://curl.haxx.se/bug/view.cgi?id=1235
Patched-by: Nach M. S.
Allow less room for "triggered too early" mistakes by applications /
timers on non-windows platforms. Starting now, we assume that a timeout
call is never made earlier than 3 milliseconds before the actual
timeout. This greatly improves timeout accuracy on Linux.
Bug: http://curl.haxx.se/bug/view.cgi?id=1228
Reported-by: Hang Su
In the pkcs12 code, we get a list of x509 records returned from
PKCS12_parse but when iterating over the list and passing each to
SSL_CTX_add_extra_chain_cert() we didn't also properly remove them from
the "stack", which made them get freed twice (both in sk_X509_pop_free()
and then later in SSL_CTX_free).
This isn't really documented anywhere...
Bug: http://curl.haxx.se/bug/view.cgi?id=1236
Reported-by: Nikaiw
After curl_multi_wait() returns, this test checked that we got exactly
one file descriptor told to read from, but we cannot be sure that is
true. curl_multi_wait() will sometimes return earlier without any file
descriptor to handle, just just because it is a suitable time to call
*perform().
This problem showed up with commit 29bf0598.
Bug: http://curl.haxx.se/mail/lib-2013-06/0029.html
Reported-by: Fabian Keil
If an ipv6-address is provided to CONNECT, the last hexadecimal group in
the address will be used as the test number! For example the address
"[1234::ff]" would be treated as test case 255.
commit 29bf0598aa introduced a problem when the "internal" timeout is
prefered to the given if shorter, as it didn't consider the case where
-1 was returned. Now the internal timeout is only considered if not -1.
Reported-by: Tor Arntsen
Bug: http://curl.haxx.se/mail/lib-2013-06/0015.html
Also added a (correctly-escaped) backslash to the autoexec.bat
example file and a new Windows character device name with
a colon as examples of other characters that are special
and potentially dangerous (this reverts and reworks commit
7d8d2a54).
If the multi handle's pending timeout is less than what is passed into
this function, it will now opt to use the shorter time anyway since it
is a very good hint that the handle wants to process something in a
shorter time than what otherwise would happen.
curl_multi_wait.3 was updated accordingly to clarify
This is the reason for bug #1224
Bug: http://curl.haxx.se/bug/view.cgi?id=1224
Reported-by: Andrii Moiseiev
When sending the HTTP Authorization: header for digest, the user name
needs to be escaped if it contains a double-quote or backslash.
Test 1229 was added to verify
Reported and fixed by: Nach M. S
Bug: http://curl.haxx.se/bug/view.cgi?id=1230
We found that in specific cases if the connection is abruptly closed,
the underlying socket is listed in a close_wait state. We continue to
call the curl_multi_perform, curl_mutli_fdset etc. None of these APIs
report the socket closed / connection finished. Since we have cases
where the multi connection is only used once, this can pose a problem
for us. I've read that if another connection was to come in, curl would
see the socket as bad and attempt to close it at that time -
unfortunately, this does not work for us.
I found that in specific situations, if SSL_write returns 0, curl did
not recognize the socket as closed (or errored out) and did not report
it to the application. I believe we need to change the code slightly, to
check if ssl_write returns 0. If so, treat it as an error - the same as
a negative return code.
For OpenSSL - the ssl_write documentation is here:
http://www.openssl.org/docs/ssl/SSL_write.html
1 - don't skip host names with a colon in them in an attempt to bail out
on HTTP headers in the cookie file parser. It was only a shortcut anyway
and trying to parse a file with HTTP headers will still be handled, only
slightly slower.
2 - don't skip domain names based on number of dots. The original
netscape cookie spec had this oddity mentioned and while our code
decreased the check to only check for two, the existing cookie spec has
no such dot counting required.
Bug: http://curl.haxx.se/bug/view.cgi?id=1221
Reported-by: Stefan Neis
I found a bug which cURL sends cookies to the path not to aim at.
For example:
- cURL sends a request to http://example.fake/hoge/
- server returns cookie which with path=/hoge;
the point is there is NOT the '/' end of path string.
- cURL sends a request to http://example.fake/hogege/ with the cookie.
The reason for this old "feature" is because that behavior is what is
described in the original netscape cookie spec:
http://curl.haxx.se/rfc/cookie_spec.html
The current cookie spec (RFC6265) clarifies the situation:
http://tools.ietf.org/html/rfc6265#section-5.2.4
This reverts commit 8ec2cb5544.
We don't have any code anywhere in libcurl (or the curl tool) that use
wcsdup so there's no such memory use to track. It seems to cause mild
problems with the Borland compiler though that we may avoid by reverting
this change again.
Bug: http://curl.haxx.se/mail/lib-2013-05/0070.html
If the mail sent during the transfer contains a terminating <CRLF> then
we should not send the first <CRLF> of the EOB as specified in RFC-5321.
Additionally don't send the <CRLF> if there is "no mail data" as the
DATA command already includes it.
The code within #ifdef HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID wrongly had two
closing braces when it should only have one, so builds without that
define would fail.
Bug: http://curl.haxx.se/mail/lib-2013-05/0000.html
The curl command line utility would display the the completed progress
bar with a percentage of zero as the progress routines didn't know the
size of the transfer.
The emails that are sent to the server during these tests were
incorrectly formatted as they contained one or more LF terminated lines
rather than being CRLF terminated as per Section 2.3.7 of RFC-2821.
This wasn't a problem for the test suite as the <stdin> data matched the
<upload> data but anyone using these tests as reference would be sending
incorrect data to a server.
Removed the hard returns from imap and pop3 by using the same style for
sending the authentication string as smtp. Moved the "Other mechanisms
not supported" check in smtp to match that of imap and pop3 to provide
consistency between the three email protocols.
Users using the Secure Transport (darwinssl) back-end can now use a
certificate and private key to authenticate with a site using TLS. Because
Apple's security system is based around the keychain and does not have any
non-public function to create a SecIdentityRef data structure from data
loaded outside of the Keychain, the certificate and private key have to be
loaded into the Keychain first (using the certtool command line tool or
the Security framework's C API) before we can find it and use it.
Updated test903 and test904 following the addition of CURLOPT_SASL_IR
as the default behaviour of SMTP AUTH responses is now to not include
the initial response. New tests with --sasl-ir support to follow.
In addition to checking for the SASL-IR capability the user can override
the sending of the client's initial response in the AUTHENTICATION
command with the use of CURLOPT_SASL_IR should the server erroneously
not report SASL-IR when it does support it.
Updated the default behaviour of sending the client's initial response in the AUTH
command to not send it and added support for CURLOPT_SASL_IR to allow the user to
specify including the response.
Related Bug: http://curl.haxx.se/mail/lib-2012-03/0114.html
Reported-by: Gokhan Sengun
By introducing an internal alternative to curl_multi_init() that accepts
parameters to set the hash sizes, easy handles will now use tiny socket
and connection hash tables since it will only ever add a single easy
handle to that multi handle.
This decreased the number mallocs in test 40 (which is a rather simple
and typical easy interface use case) from 1142 to 138. The maximum
amount of memory allocated used went down from 118969 to 78805.
When connecting back to an FTP server after having sent PASV/EPSV,
libcurl sometimes didn't use the proxy properly even though the proxy
was used for the initial connect.
The function wrongly checked for the CURLOPT_PROXY variable to be set,
which made it act wrongly if the proxy information was set with an
environment variable.
Added test case 711 to verify (based on 707 which uses --socks5). Also
added test712 to verify another variation of setting the proxy: with
--proxy socks5://
Bug: http://curl.haxx.se/bug/view.cgi?id=1218
Reported-by: Zekun Ni
... in order to prevent an artificial timeout event based on stale
speed-check data from a previous network transfer. This commit fixes
a regression caused by 9dd85bced5.
Bug: https://bugzilla.redhat.com/906031
I couldn't figure out why the host key logic isn't working, but having
it set to yes prevents my SSH-based test cases to run. I also don't see
a strong need to use strict host key checking on this test server.
So I disabled it.
...when mentioning login options. Additional minor clarification of
"Windows builds" to be "Windows builds with SSPI"as a way of enabling
NTLM as Windows builds may be built with OpenSSL to enable NTLM or
without NTLM support altogether.
Fixed an issue in parse_proxy(), introduced in commit 11332577b3,
where an empty username or password (For example: http://:@example.com)
would cause a crash.
There is no need to perform separate clearing of data if a NULL option
pointer is passed in. Instead this operation can be performed by simply
not calling parse_login_details() and letting the rest of the code do
the work.
setstropt_userpwd() was calling setstropt() in commit fddb7b44a7 to
set each of the login details which would duplicate the strings and
subsequently cause a memory leak.
Fix to prevent the options from being displayed when curl requests the
user's password if the following command line is specified:
--user username;options
In addition to parsing the optional login options from the URL, added
support for parsing them from CURLOPT_USERPWD, to allow the following
supported command line:
--user username:password;options
Added bounds checking when searching for the separator characters within
the login string as this string may not be NULL terminated (For example
it is the login part of a URL). We do this in preference to allocating a
new string to copy the login details into which could then be passed to
parse_login_details() for performance reasons.
Apparently the previous usage didn't work with that implementation,
while this updated version works with at least both Parallel BZIP2
v1.1.8 and regular bzip "Version 1.0.6, 6-Sept-2010".
As well as parsing the username and password from the URL, added support
for parsing the optional options part from the login details, to allow
the following supported URL format:
schema://username:password;options@example.com/path?q=foobar
This will only be used by IMAP, POP3 and SMTP at present but any
protocol that may be given login options in the URL will be able to
add support for them.
...instead of the 220 we otherwise expect.
Made the ftpserver.pl support sending a custom "welcome" and then
created test 1219 to verify this fix with such a 230 welcome.
Bug: http://curl.haxx.se/mail/lib-2013-02/0102.html
Reported by: Anders Havn
For libc variants without a spearate pthread lib (like bionic), try
using pthreads without the pthreads lib first and only if that fails try
the -lpthread linker flag.
Bug: http://curl.haxx.se/bug/view.cgi?id=1216
Reported by: Duncan
Accessing a file with an absolute path in the root dir but with no
directory specified was not handled correctly. This fix comes with four
new test cases that verify it.
Bug: http://curl.haxx.se/mail/lib-2013-04/0142.html
Reported by: Sam Deane
... and make 1216 also verify it with a file input
These tests verify commit 3604fde3d3c9b0d, the fix for the "cookie
domain tailmatch" vulnerability. See
http://curl.haxx.se/docs/adv_20130412.html
Cookies set for 'example.com' could accidentaly also be sent by libcurl
to the 'bexample.com' (ie with a prefix to the first domain name).
This is a security vulnerabilty, CVE-2013-1944.
Bug: http://curl.haxx.se/docs/adv_20130412.html
The previously applied patch didnt work on Windows; we cant rely
on shell commands like 'echo' since they act diffently on each
platform and each shell.
In order to keep this script platform-independent the code must
only use pure Perl.
When doing PWD, there's a 257 response which apparently some servers
prefix with a comment before the path instead of after it as is
otherwise the norm.
Failing to parse this, several otherwise legitimate use cases break.
Bug: http://curl.haxx.se/mail/lib-2013-04/0113.html
The OpenSSL pipe wrote to the final CA bundle file, but the encoded PEM
output wrote to a temporary file. Consequently, the OpenSSL output was
lost when the temp file was renamed to the final file at script finish
(overwriting the final file written earlier by openssl).
Patch posted to the list by Richard Michael (rmichael edgeofthenet org).
I noticed that aria2's SecureTransport code disables insecure ciphers such
as NULL, anonymous, IDEA, and weak-key ciphers used by SSLv3 and later.
That's a good idea, and now we do the same thing in order to prevent curl
from accessing a "secure" site that only negotiates insecure ciphersuites.
1 - We don't use the results from the test and we never did. recvfrom()
is only used by the TFTP code and it has not caused any problems.
2 - the CURL_CHECK_FUNC_RECVFROM function is extremely slow
Previously it only compared credentials if the requested needle
connection wasn't using a proxy. This caused NTLM authentication
failures when using proxies as the authentication code wasn't send on
the connection where the challenge arrived.
Added test 1215 to verify: NTLM server authentication through a proxy
(This is a modified copy of test 67)
Since qsort implementations vary with regards to handling the order
of similiar elements, this change makes the internal sort function
more deterministic by comparing path length first, then domain length
and finally the cookie name. Spotted with testcase 62 on Windows.
WINSOCK only:
Since FD_CLOSE is only signaled once, it may trigger at the same
time as FD_READ. Data actually being available makes it impossible
to detect that the connection was closed by checking that recv returns
zero. Another recv attempt could block the connection if it was
not closed. This workaround abuses exceptfds in conjunction with
readfds to signal that the connection has actually closed.
Since commit 57aeabcc1a, it handles errors on the control connection
while waiting for the data connection better.
Test 591 and 592 are updated accordingly.
When doing PORT and upload (STOR), this function needs to extract the
file descriptor for both connections so that it will respond immediately
when the server eventually connects back.
This flaw caused active connections to become unnecessary slow but they
would still often work due to the normal polling on a timeout. The bug
also would not occur if the server connected back very fast, like when
testing on local networks.
Bug: http://curl.haxx.se/bug/view.cgi?id=1183
Reported by: Daniel Theron
I am using curl_easy_setopt(CURLOPT_INTERFACE, "if!something") to force
transfers to use a particular interface but the transfer fails with
CURLE_INTERFACE_FAILED, "Failed binding local connection end" if the
interface I specify has no IPv6 address. The cause is as follows:
The remote hostname resolves successfully and has an IPv6 address and an
IPv4 address.
cURL attempts to connect to the IPv6 address first.
bindlocal (in lib/connect.c) fails because Curl_if2ip cannot find an
IPv6 address on the interface.
This is a fatal error in singleipconnect()
This change will make cURL try the next IP address in the list.
Also included are two changes related to IPv6 address scope:
- Filter the choice of address in Curl_if2ip to only consider addresses
with the same scope ID as the connection address (mismatched scope for
local and remote address does not result in a working connection).
- bindlocal was ignoring the scope ID of addresses returned by
Curl_if2ip . Now it uses them.
Bug: http://curl.haxx.se/bug/view.cgi?id=1189
This workaround fixes an issue on MinGW/Msys regarding the Perl
testsuite scripts not being able to signal or control the server
processes. The MinGW Perl runtime only sees the Msys processes and
their corresponding PIDs, but sockfilt (and other servers) wrote the
Windows PID into their PID-files. Since this PID is useless to the
testsuite, the write_pidfile function was changed to search for the
Msys PID and write that into the PID-file.
At some point recently we lost the default value for the easy handle's
connection cache, and this change puts it back to 5 - which is the
former default value and it is documented in the curl_easy_setopt.3 man
page.
The new read and write wrapper functions support reading from stdin
and writing to stdout/stderr on Windows by using the appropriate
Windows API functions and data types.
The Microsoft knowledge-base article
http://support.microsoft.com/kb/823764 describes how to use SNDBUF to
overcome a performance shortcoming in winsock, but it doesn't apply to
Windows Vista and later versions. If the described SNDBUF magic is
applied when running on those more recent Windows versions, it seems to
instead have the reversed effect in many cases and thus make libcurl
perform less good on those systems.
This fix thus adds a run-time version-check that does the SNDBUF magic
conditionally depending if it is deemed necessary or not.
Bug: http://curl.haxx.se/bug/view.cgi?id=1188
Reported by: Andrew Kurushin
Tested by: Christian Hägele
The last remaining code piece that still used FTPSENDF now uses PPSENDF.
In the problematic case, a PREQUOTE series was done on a re-used
connection when Curl_pp_init() hadn't been called so it had messed up
pointers. The init call is done properly from Curl_pp_sendf() so this
change fixes this particular crash.
Bug: http://curl.haxx.se/mail/lib-2013-03/0319.html
Reported by: Sam Deane
Modified sws to support and use custom CONNECT responses instead of the
previously naive hard-coded version. Made the HTTP test server able to
extract test case number from the host name in a CONNECT request by
finding the number after the last dot. It makes 'machine.moo.123' use
test case 123.
Adapted a larger amount of tests to the new <connect> style.
Bug: http://curl.haxx.se/bug/view.cgi?id=1204
Reported by: Martin Jansen
As of 25-mar-2013 wcsdup() _wcsdup() and _tcsdup() are only used in
WIN32 specific code, so tracking of these has not been extended for
other build targets. Without this fix, memory tracking system on
WIN32 builds, when using these functions, would provide misleading
results.
In order to properly extend this support for all targets curl.h
would have to define curl_wcsdup_callback prototype and consequently
wchar_t should be visible before that in curl.h. IOW curl_wchar_t
defined in curlbuild.h and this pulling whatever system header is
required to get wchar_t definition.
Additionally a new curl_global_init_mem() function that also receives
user defined wcsdup() callback would be required.
Proxy servers tend to add their own headers at the beginning of
responses. The size of these headers was not taken into account by
CURLINFO_HEADER_SIZE before this change.
Bug: http://curl.haxx.se/bug/view.cgi?id=1204
This patch removes building curlbuild.h from the build_vms.com procedure
and uses the one in the daily or release tarball instead.
packages/vms/build_curlbuild_h.com is obsolete with this change.
Accessing the library module name "tool_main" needs different handling
when the optional extended parsing is enabled.
Tested on IA64/VMS 8.4 and VAX/VMS 7.3
After having done a POST over a CONNECT request, the 'rewindaftersend'
boolean could be holding the previous value which could lead to badness.
This should be tested for in a new test case!
Bug: https://groups.google.com/d/msg/msysgit/B31LNftR4BI/KhRTz0iuGmUJ
Moved SMTP, POP3, IMAP and New Protocol sections to be listed after the
other protocols (FTP, HTTP and TELNET) and SASL to be after SSL and
GnuTLS as these are all security related.
Additionally fixed numbering of the SSL and GnuTLS sections as they
weren't consecutive.
Fixed incorrect initial response generation for the NTLM and LOGIN SASL
authentication mechanisms when the SASL-IR was detected.
Introduced in commit: 6da7dc026c.
These verfy that the 'memory tracking' subsystem is actually doing its
job when using curl tool (#96), a test in libtest (#558) and also a unit
test (#1330), in order to prevent regressions in this functionallity.
curl has been accepting URLs using slightly wrong syntax for a long
time, such as when completely missing as slash "http://example.org" or
missing a slash when a query part is given
"http://example.org?q=foobar".
curl would translate these into a legitimate HTTP request to servers,
although as was shown in bug #1206 it was not adjusted properly in the
cases where a HTTP proxy was used.
Test 1213 and 1214 were added to the test suite to verify this fix.
The test HTTP server was adjusted to allow us to specify test number in
the host name only without using any slashes in a given URL.
Bug: http://curl.haxx.se/bug/view.cgi?id=1206
Reported by: ScottJi
Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.
Two new options for limiting the number of connections:
CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.
CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.
Several new options for pipelining:
CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).
CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.
CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.
CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.
CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.
See the curl_multi_setopt() man page for details.
Following commit e450f66a02 and the changes in the multi interface
being used internally, from 7.29.0, the transfer cancellation in
pop3_dophase_done() is no longer required.
Added missing slash in cc_full_list.
Removed unwanted extra quotes inside symbol tool_main
for non-VAX architectures that triggered link failure.
Replaced curl_sys_inc with sys_inc.
An extern submits a psect and a global reference to the linker to point
to it. Using "extern int vms_show = 0" also creates a globaldef.
The use of the extern by itself does declare a psect but does not declare
a globalsymbol. It does declare a globalref. But the linker needs one and
only one globaldef or there is an error.
Any IMAP parameter can come in escaped and in double quotes. Added a
simple function to unquote the command parameters and applied it to
the IMAP command handlers.
The majority of lines, that specify a test file for inclusion, end with
a tab character before the slash whilst some end with a space. Corrected
those that end with a space to end with a tab character as well.
Finished segregating the email protocol tests, into their own protocol
based ranges, in preparation of adding more e-mail related tests to the
test suite.
Continued segregating the email protocol tests, into their own protocol
based ranges, in preparation of adding more e-mail related tests to the
test suite.
Continued segregating the email protocol tests, into their own protocol
based ranges, in preparation of adding more e-mail related tests to the
test suite.
Started segregating the email protocol tests, into their own protocol
based ranges, in preparation of adding more e-mail related tests to the
test suite.
When Curl_do() returns failure, the connection pointer could be NULL so
the code path following needs to that that into account.
Bug: http://curl.haxx.se/mail/lib-2013-03/0062.html
Reported by: Eric Hu
No more use exit(3) but instead tell libcurl that no byte has been
written to let it return a `CURLE_WRITE_ERROR`. In addition, check
curl easy handle return code.
Moved the blocking state machine to the disconnect functions so that the
logout / quit functions are only responsible for sending the actual
command needed to logout or quit.
Additionally removed the hard return on failure.
Added an exception, for the STORE command, to the untagged response
processor in imap_endofresp() as servers will back respones containing
the FETCH keyword instead.
GCC specific adjustments:
- check __ILP32__ before 32 and 64bit processor architectures in
order to detect ILP32 programming model on 64 bit processors
which, of course, also support LP64 programming model, when using
gcc 4.7 or newer.
- keep 32bit processor architecture checks in order to support gcc
versions older than 4.7 which don't define __ILP32__
- check __LP64__ for gcc 3.3 and newer, while keeping 64bit processor
architecture checks for older versions which don't define __LP64__
The list of unsafe functions currently consists of sprintf, vsprintf,
strcat, strncat and gets.
Subsequently, some existing code needed updating to avoid warnings on
this.
Enriched IMAP capabilities of ftpserver.pl in order to be able to
add tests for the new IMAP features.
* Added support for APPEND - Saves uploaded data to log/upload.$testno
* Added support for LIST - Returns the contents of <reply/> section in
the current test, like e.g FETCH.
* Added support for STORE - Returns hardcoded updated flags
* Changed handling of SELECT - Returns much more information in the
usual set of untagged responses; uses hardcoded data from an example
in the IMAP RFC
* Changed handling of FETCH - Fixed response format
As the UID has to be specified by the user for the FETCH command to work
correctly, added a check to imap_fetch(), although strictly speaking it
is protected by the call from imap_perform().
The option needs to be set on the SSL socket. Setting it on the model
takes no effect. Note that the non-blocking mode is still not enabled
for the handshake because the code is not yet ready for that.
Commit 26eaa83830 introduces the use of S_ISDIR() yet some compilers,
such as MSVC don't support it, so we must define a substitute using
file flags and mask.
Commit f4cc54cb47 (shipped as part of the 7.29.0 release) was a
bug fix that introduced a regression in that while trying to avoid
allowing directory names, it also forbade "special" files like character
devices and more. like "/dev/null" as was used by Oliver who reported
this regression.
Reported by: Oliver Gondža
Bug: http://curl.haxx.se/mail/archive-2013-02/0040.html
If the server hung up the connection without sending a closure alert,
then we'd keep probing the socket for data even though it's dead. Now
we're ready for this situation.
Bug: http://curl.haxx.se/mail/lib-2013-03/0014.html
Reported by: Aki Koskinen
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Added imap_custom(), which initiates the custom command processing,
and an associated response handler imap_state_custom_resp(), which
handles any responses by sending them to the client as body data.
All untagged responses with the same name as the first word of the
custom request string are accepted, with the exception of SELECT and
EXAMINE which have responses that cannot be easily identified. An
extra check has been provided for them so that any untagged responses
are accepted for them.
Added imap_parse_custom_request() for parsing the CURLOPT_CUSTOMREQUEST
parameter which URL decodes the value and separates the request from
any parameters - This makes it easier to filter untagged responses
by the request command.
For consistency changed the logic of the imap_state_append_resp()
function to test for an unsucessful continuation response rather than a
succesful one.
The APPEND operation needs to be performed in several steps:
1) We send "<tag> APPEND <mailbox> <flags> {<size>}\r\n"
2) Server responds with continuation respose "+ ...\r\n"
3) We start the transfer and send <size> bytes of data
4) Only now we end the request command line by sending "\r\n"
5) Server responds with "<tag> OK ...\r\n"
This commit performs steps 4 and 5, in the DONE phase, as more
processing is required after the transfer.
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Not processing the final FETCH responses was not optimal, not only
because the response code would be ignored but it would also leave data
unread on the socket which would prohibit connection reuse.
A typical FETCH response can be broken down into four parts:
1) "* <uid> FETCH (<what> {<size>}\r\n", using continuation syntax
2) <size> bytes of the actual message
3) ")\r\n", finishing the untagged response
4) "<tag> OK ...", finishing the command
Part 1 is read in imap_fetch_resp(), part 2 is consumed in the PERFORM
phase by the transfer subsystem, parts 3 and 4 are currently ignored.
Added a loop to imap_statemach_act() in which Curl_pp_readresp() is
called until the cache is drained. Without this multiple responses
received in a single packet could result in a hang or delay.
RFC 3501 states that "the client MUST be prepared to accept any response
at all times" yet we assume anything received with "* " at the beginning
is the untagged response we want.
Introduced a helper function that checks whether the input looks like a
response to specified command, so that we may filter the ones we are
interested in according to the current state.
Introduced similar handling to the FETCH responses, where even the
untagged data responses are handled by the response handler of the
individual state.
Removed this pointer to a downloaded bytes counter because it was set in
smtp_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
Removed this pointer to a downloaded bytes counter because it was set in
pop3_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
Removed this pointer to a downloaded bytes counter because it was set in
imap_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
From a maintenance point of view the code reads better to view tagged
responses, then untagged followed by continuation responses.
Additionally, this matches the order of responses in POP3.
Updated the POP3 sub-section to refer to message ID rather than mailbox.
Added an IMAP sub-section with example URLs depicting the specification
of mailbox, uid and section.
Updated the mailbox variable to correctly reflect it's purpose. The
name mailbox was a leftover from when IMAP and POP3 support was
initially added to curl.
Updated the FETCH command to send the UID and SECTION parsed from the
URL. By default the BODY specifier doesn't include a section, BODY[] is
now sent whereas BODY[TEXT] was previously sent. In my opinion
retrieving just the message text is rarely useful when dealing with
emails, as the headers are required for example, so that functionality
is not retained. In can however be simulated by adding SECTION=TEXT to
the URL.
Also updated test801 and test1321 due to the BODY change.
Removed user and passwd from the SMTP struct as these cannot be set on
a per-request basis and are leftover from legacy FTP code.
Changed some comments still using FTP terminology.
Removed user and passwd from the POP3 struct as these cannot be set on
a per-request basis and are leftover from legacy FTP code.
Changed some comments still using FTP terminology.
Moved the mailbox and custom request variables from the per-connection
struct pop3_conn to the new per-request struct and fixed references
accordingly.
Created a new IMAP structure and changed the type of the imap proto
variable in connectdata from FTP* to the new IMAP*.
Moved the mailbox variable from the per-connection struct imap_conn to
the new per-request struct and fixed references accordingly.
Moved the clean-up of the mailbox variable from imap_disconnect() to
imap_done() as this variable is allocated in the do phase, yet would
have only been freed only once if multiple selects where preformed
on a single connection.
* Elaborates on default values of some curl_easy_setopt() options.
* Reminds the user to cast variadic arguments to curl_easy_setopt() to
'void *' where curl internally interprets them as such.
* Clarifies the working of the CURLOPT_SEEKFUNCTION option for
curl_easy_setopt().
* Fixes typo 'forth' → 'fourth'.
* Elaborates on CURL_SOCKET_TIMEOUT.
* Adds some missing periods.
* Notes that the return value of curl_version() must not be passed to
free().
Always interprets the pointer passed with the CURLOPT_WRITEDATA or
CURLOPT_READDATA options of curl_easy_setopt() as a void pointer in
order to avoid problems in environments where FILE and void pointers
have non-trivial conversion.
* Adds several links to documentation of library functions which were
missing.
* Marks documentation of deprecated library functions "(deprecated)".
* Removes spurious .html suffixes.
Use Curl_pp_moredata() in Curl_pp_multi_statemach() to check if there is
more data to be received, rather than the socket state, as a task could
hang waiting for more data from the socket itself.
A simple function to test whether the PP is not sending and there are
still more data in its receiver cache. This will be later utilized to:
1) Change Curl_pp_multi_statemach() and Curl_pp_easy_statemach() to
not test socket state and just call user's statemach_act() function
when there are more data to process, because otherwise the task would
just hang, waiting for more data from the socket.
2) Allow PP users to read multiple responses by looping as long as there
are more data available and current phase is not finished.
(Currently needed for correct processing of IMAP SELECT responses.)
The this_url pointer wasn't being initialized, so if strdup() would return
null when copying the filename in a metalink file, then hilarity would
ensue during the cleanup phase. This change was brought to you by clang,
which noticed this and raised a warning.
Schannel and darwinssl use the certificates built into the
OS to do vert verification instead of bundles. darwinssl
is thread-safe. Corrected typos in the NSS docs.
The attempt to use gai_strerror() or alternative function didn't work as
the 'sock_error' field didn't contain the proper error code. But since
this hasn't been reported and thus isn't really a big deal I decided to
just scrap the whole attempt to output the detailed resolver error and
instead remain with just stating that the resolving of the name failed.
It seems older gcc installations (at least) will cause warnings if we
name a variable 'wait'. Now changed to 'block' instead.
Reported by: Jiří Hruška
Bug: http://curl.haxx.se/mail/lib-2013-02/0247.html
Apple made a number of changes to Xcode 4. The SDKs were moved, the entire
Developer folder was moved, and PowerPC support was removed. The script
will now adapt to those changes and should be future-proofed against
additional changes in case Apple moves the Developer folder ever again.
Also, the minimum OS X version compiler option was removed, so that the
framework can be built against the latest SDK but still run in older cats.
... since they're not used by the easy interface really, I wanted to
remove the association. Also, I unified the pingpong statemachine driver
into a single function with a 'wait' argument: Curl_pp_statemach.
A call to Curl_ssl_connect() was accidentally left in when the SSL/TLS
connection layer was reworked in 7.29. Not only would this cause the
connection to block but had the additional overhead of calling the
non-blocking connect a little bit later.
This function was only used twice, both in places where performance
isn't crucial (socks + if2ip). Removing the use of this function removes
the need to have our private version for systems without it == reduced
amount of code.
Also, in the SOCKS case it is clearly better to fail gracefully rather
than to truncate the results.
This work was triggered by a bug report on the strcal prototype in
strequal.h.
strlcat was added in commit db70cd28 in February 2001!
Bug: http://curl.haxx.se/bug/view.cgi?id=1192
Reported by: Jeremy Huddleston
Version 7.29.0 uses Makefiles generated with a newer version of the
autotools than the previous 7.28.1. These Makefiles try to install
e.g. header files by calling install-sh with multiple source files as
arguments. The bundled install-sh is to old and does not support this.
The problem only occurs, if install-sh is actually being used, ie. the
platform install executable is to old or not usable. Example: Solaris
10.
The files install-sh and mkinstalldirs are now updated with the automake
1.11.3 versions. A better fix might be to completely remove them from
git and force the files to be added/created during buildconf.
Bug: http://curl.haxx.se/bug/view.cgi?id=1195
Reported by: Rainer Jung
Fixed a null pointer reference when an empty challenge is passed to the
Curl_sasl_create_digest_md5_message() function.
Bug: http://sourceforge.net/p/curl/bugs/1193/
Reported by: Saran Neti
An ambiguity in the SSLWrite() documentation lead to a bad inference in the
code where we assumed SSLWrite() returned the amount of bytes written to
the socket, when that is not actually true; it returns the amount of data
that is buffered for writing to the socket if it returns errSSLWouldBlock.
Now darwinssl_send() returns CURLE_AGAIN if data is buffered but not written.
Reference URL: http://curl.haxx.se/mail/lib-2013-02/0145.html
Reworked the pp->endofresp() function so that the conndata, line and
line length are passed down to it just as with Curl_client_write()
rather than each implementation of the function having to query
these values.
Additionally changed the int return type to bool as this is more
representative of the function's usage.
Corrected the order of the upgrade_tls() functions and moved the handler
upgrade and getsock() functions out from the middle of the state related
functions.
Corrected the order of the pop3_state_capa() / imap_state_capability()
and the pop3_state_capa_resp() / imap_state_capability_resp() functions
to match the execution order.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added sending of initial CAPABILITY command before STARTTLS is sent.
This allows for the detection of the capability before trying to
upgrade the connection.
Follow up fix to commit 62bd217464 to cater for servers that don't
respond with a 250 in their EHLO responses. Additionally updated the
SMTP tests to respond with a 250 response code as per RFC5321.
Fixed the SASL capability detection to include the space character
before the authentication mechanism list. Otherwise a capability such
as SASLSOMETHING would be interpreted as enabling SASL and potentially
trying to identify SOMETHING as a mechanism.
Previously if a problem was found with one of the server's certificates,
we'd log an OSStatus for the end user to look up. Now we explain what
was wrong with the site's certificate chain. Also un-did part of the
previous commit where the code wouldn't catch errSSLServerAuthCompleted
if built under Leopard.
Fixed a small issue where smtp_endofresp() would look for capabilities
in the description part of a failure response. In theory a server
shouldn't respond with SIZE or AUTH in an EHLO command's failure
response but if it did then capabilities would be unnecessarily set
before eventually failing.
Renamed the authstate1 and authstate2 variables in imap_authenticate()
as the old name was a left over from when there was only one state
variable which was named due to a clash with the state() function.
Additionally this provides consistency with the smtp module.
Running tests\libtest\libntlmconnect.exe reveals a 1 byte (!) leak in
./lib/curl_ntlm_msgs.c:
perl ..\memanalyze.pl c:memdebug.curl
Leak detected: memory still allocated: 1 bytes
At 9771e8, there's 1 bytes.
allocated by curl_ntlm_msgs.c:399
Snippet from curl_ntlm_msgs.c:
/* setup ntlm identity's domain and length */
dup_domain.tchar_ptr = malloc(sizeof(TCHAR) * (domlen + 1));
(my domlen == 0).
'dup_domain.tbyte_ptr' looks to be freed in Curl_ntlm_sspi_cleanup() via
'ntlm->identity.Domain'. But I see no freeing of 'dup_domain.tchar_ptr'.
This bug report properly identified that when doing SMTP and aborting
the transfer with a callback, it must be considered aborted prematurely
by the code to avoid QUIT etc to be attempted as that would cause a
hang.
The new test case 1507 verifies this behavior.
Reported by: Patricia Muscalu
Bug: http://curl.haxx.se/bug/view.cgi?id=1184
It turns out that Leopard (OS X 10.5) doesn't have constants for the ECDH
ciphers in its headers, so the cases for them have been taken out of the
build when building under Leopard. Also added a standard function for
getting a string description of a SecCertificateRef.
Changed the SMTP_AUTH_PASSWD state constant to SMTP_AUTH_LOGIN_PASSWD to
better describe the state as the second part of an AUTH LOGIN command,
as well as for consistency with the imap and pop3 modules.
Introduced detection of the SASL-IR capability, in order to add support
for sending the initial response with the AUTHENTICATE command, as per
RFC4959.
Updated the automatic response tag generation to follow the examples
given in RC3501, which list a 4 character string such as A001, A002,
etc.
As a unique identifier should be generated for each command the string
generation is based on the connection id and the incrementing command
id.
This is untested, but ought to be enough to still allow it
to work automatically when the entire curl source tree is
dropped into a full Android source tree.
VC6 is _very_ old and we provide working makefiles even for that
compiler. Users who build with the IDE never use that method and project
file anyway and it was just lingering in the root dir.
Added IDN and HTTP data compression as they were left out of the
document until now.
Added notes for qssl, schannel and Secure Transport supporting SSLv2,
Secure Transport supports NTLM, and axTLS does not support SSLv3.
There was also a typo; "AUTH TSL" should be "AUTH TLS".
When negotiating SASL DIGEST-MD5 authentication, the function
Curl_sasl_create_digest_md5_message() uses the data provided from the
server without doing the proper length checks and that data is then
appended to a local fixed-size buffer on the stack.
This vulnerability can be exploited by someone who is in control of a
server that a libcurl based program is accessing with POP3, SMTP or
IMAP. For applications that accept user provided URLs, it is also
thinkable that a malicious user would feed an application with a URL to
a server hosting code targetting this flaw.
Bug: http://curl.haxx.se/docs/adv_20130206.html
config_h.com is a new file that generates a config.h file based on the
curl_config.h.in file and a quick scan of the configure script. This is
actually a generic procedure that is shared with other VMS packages.
The existing pre-built config-vms.h had over 100 entries that were not
correct and in some cases conflicted with the build options available in
the build_vms.com.
generate_config_vms_h_curl.com is a helper procedure to the
config_h.com. It covers the cases that the generic config_h.com is not
able to figure out, and accepts input from the build_vms.com procedure.
build_curlbuild_h.com is a new file to generate the curlbuild.h file
that Curl is now using when it is using a curl_config.h file.
post-config-vms.h is a new file that is needed to provide VMS specific
definitions, and most of them need to be set before the system header
files are included.
The VMS build procedure is fixed:
1. Fixed to link in the correct HP ssl library.
2. Fixed to detect if HP Kerberos is installed.
3. Fixed to detect if HP LDAP is installed.
4. Fixed to detect if gnv$libzshr is installed.
5. Simplified the input parameter parsing to not use a loop.
6. Warn that 64 bit pointer option support is not complete
in comments.
7. Default to IEEE floating if platform supports it so
resulting libcurl will be compatible with other
open source projects on VMS.
8. Default to LARGEFILE if platform supports it.
9. Default to enable SSL, LDAP, Kerberos, libz
if the libraries are present.
10. Build with exact case global symbols for libcurl.
11. Generate linker option file needed.
12. Compiler list option only commonly needed items.
13. fulllist option for those who really want it.
14. Create debug symbol file on Alpha, IA64.
By doing this unconditionally, we infer a simpler and more defined
behavior. This also has the upside that test 1021 no longer fails for me
even if I run with valgrind.
Also fixed some wrong comments.
According KB975858 this flag may cause problems on Windows 7 and
Windows Server 2008 R2 systems. Extended error information is not
currently used by libcurl and therefore not a requirement.
The flag may improve the SSL-connection shutdown in case of an
error. This means it might be a good improvement in the future.
Fixes bug/issue #1187 - thanks for the report
When a connection is no longer used, it is kept in the cache. If the
cache is full, the oldest idle connection is closed. If no connection is
idle, the current one is closed instead.
Added recent additions and fixes following the changes to imap, pop3
and smtp. Additionally added another contributor that helped to test
the imap sasl changes.
Tidied up code from commit 6b6bdc83bdUpdated where a few instances of
the pop3c struct variable used the longer conndata struct rather than
matching what other code in pop3_authenticate() used.
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to Clear Text authentication.
Note: In order to fallback to Clear Text authentication properly this
fix adds support for the LOGINDISABLED server capability.
imap: Fixed no known authentication mechanism when fallback is required
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to Clear Text authentication.
Note: In order to fallback to Clear Text authentication properly this
fix adds support for the LOGINDISABLED server capability.
Related bug: http://curl.haxx.se/mail/lib-2013-02/0004.html
Reported by: Stanislav Ivochkin
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to APOP or Clear Text
authentication.
Bug: http://curl.haxx.se/mail/lib-2013-02/0004.html
Reported by: Stanislav Ivochkin
Remove timeout argument that's never used.
Make the actual connection get detected on a single spot to reduce code
duplication.
Store the IPv6 state already when the connection is attempted.
- Take into account that 'wc' may return leading spaces.
- Set internationalization behavior variables.
Tor Arntsen analyzed and reported the issue.
URL: http://curl.haxx.se/mail/lib-2013-01/0351.html
There was a bug where, if SSLWrite() returned errSSLWouldBlock but did
succeed in transmitting at least something, then we'd incorrectly
resend the packet. Now we never take errSSLWouldBlock as a sign that
nothing was transferred to/from the server.
Bug: http://curl.haxx.se/mail/lib-2013-01/0295.html
Reported by: Bruno de Carvalho
Minor code tidy up to add comments similar to those used in the pop3
and imap end of resp functions, in order to assist anyone reading the
code and highlight the similarities between each of these protocols.
smtp_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to smtps and start the EHLO command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
pop3_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to pop3s and start the CAPA command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
imap_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to imaps and start the CAPABILITY command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
Some basic checks we make were placed early enough in generated
configure script when using autoconf 2.5X versions. Newer autoconf
versions expand these checks much further into the configure script,
rendering them useless. Using XC_CONFIGURE_PREAMBLE fixes placement
of early intended checks across all our autoconf supported versions.
- document the double-quote and backslash need be escaped if quoting.
- libcurl formdata escape double-quote in filename by backslash.
- curl formparse can parse filename both contains '"' and ',' or ';'.
- curl now can uploading file with ',' or ';' in filename.
Bug: http://curl.haxx.se/bug/view.cgi?id=1171
Notes:
When running a configure script that has nested packages (for example
libcurl's configure with --enable-ares and c-ares sources embedded in
curl tree) and AC_CONFIG_SUBDIRS([nested-subdir]) machinery is used to
automatically run the nested configure script from within the parent
configure script, it happens that the nested _shell_ script will
inherit shell variables exported from the parent _shell_ script.
If for example parent configure script sets and exports LDFLAGS and LIBS
variables with proper values in order to link either a parent library or
program with a library which will be configured and built by a nested
package; It will happen that when the nested configure script runs, the
nested library does not exist yet and _any_ link-test done in the nested
configure will fail, such as those that autoconf macros perform in order
to detect existing compiler and its characteristics, the result is that
the nested configure script will fail with errors such as:
configure: error: C compiler cannot create executables
For now, we no longer export variables previously exported here.
On the other hand, AC_SUBST'ing them is appropriate and even with nested
packages each package's config.status gets its own package values.
So we reinstate AC_SUBST'ing previously AC_SUBST'ed variables.
Fixed an issue where Curl_ssl_connect_nonblocking() wouldn't complete
correctly and the ssldone flag wouldn't be set to true for pop3s based
connections.
Bug introduced in commit: 4ffb8a6398.
Remove internal separated behavior of the easy vs multi intercace.
curl_easy_perform() is now using the multi interface itself.
Several minor multi interface quirks and bugs have been fixed in the
process.
Much help with debugging this has been provided by: Yang Tse
Fixes initial proxy response being processed by the tunneled protocol
handler instead of the HTTP wrapper handler. This issue would trigger
upon delayed CONNECT response from the proxy.
Additionally fixes a multi interface code-path in which connections
would not time out properly.
This does not fix known bug #39.
URL: http://curl.haxx.se/mail/lib-2013-01/0191.html
If the default value for an option taking a long as its value is non
zero, and it is set by zero by a command line option, then that command
line option is not reflected in --libcurl's output. This is because line
520-521 of tool_setopt.c look like:
if(!lval)
skip = TRUE;
An example of a command-line option doing so is the -k option that sets
CURLOPT_SLL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST to 0L, when the
defaults are non-zero.
Automake documents that doing this will make it choose a different name
for intermediate object files even when sharing source files across
targets of same Makefile.am.
Up to automake 1.13.1 target's intermediate object files were placed
in the build subdirectory of the target. We depended on this, probably
undocumented behavior, to achieve same behavior as if a per-target flag
had been specified when building targets that actually belong to
different Makefile.am files.
It seems automake 1.13.2 is going to break behavior mentioned above.
So, lets use a documented behavior in order to achieve same purpose,
across automake versions, no matter where automake wishes to place
intermediate object files.
Our build targets that already were using a per-target '_CFLAGS' or
'_CPPFLAGS' need no 'fixing', these were already 'fixed'. The only
Makefile.am or Makefile.in files in libcurl's source tree touched by
this 'fix' are tests/libtest/Makefile.inc and tests/unit/Makefile.inc.
Tested with:
buildconf: autoconf version 2.69
buildconf: autom4te version 2.69
buildconf: autoheader version 2.69
buildconf: automake version 1.13.1
buildconf: aclocal version 1.13.1
buildconf: libtool version 2.4
buildconf: GNU m4 version 1.4.16
This commit fixes a regression introduced in 052a08ff.
NSS caches certs/keys returned by the SSL_GetClientAuthDataHook callback
and if we connect second time to the same server, the cached cert/key
pair is used. If we use multiple client certificates for different
paths on the same server, we need to clear the session cache to force
NSS to call the hook again. The commit 052a08ff prevented the session
cache from being cleared if a client certificate from file was used.
The condition is now fixed to cover both cases: consssl->client_nickname
is not NULL if a client certificate from the NSS database is used and
connssl->obj_clicert is not NULL if a client certificate from file is
used.
Review by: Kai Engert
gcc on DOS hasn't really supported COFF-debug (-gcoff) on djgpp for a
long time.
"Sounds like the COFF debug info generation has bit-rotted in GCC.
Nothing new here, no other platform uses COFF AFAIK."
So lets drop it too.
URL: http://curl.haxx.se/mail/lib-2013-01/0130.html
This commit renames lib/setup.h to lib/curl_setup.h and
renames lib/setup_once.h to lib/curl_setup_once.h.
Removes the need and usage of a header inclusion guard foreign
to libcurl. [1]
Removes the need and presence of an alarming notice we carried
in old setup_once.h [2]
----------------------------------------
1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard
up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H,
this single inclusion guard is enough to ensure that inclusion of
lib/setup_once.h done from lib/setup.h is only done once.
Additionally lib/setup.h has always used __SETUP_ONCE_H macro to
protect inclusion of setup_once.h even after commit ec691ca3, this
was to avoid a circular header inclusion triggered when building a
c-ares enabled version with c-ares sources available which also has
a setup_once.h header. Commit ec691ca3 exposes the real nature of
__SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard
foreign to libcurl belonging to c-ares's setup_once.h
The renaming this commit does, fixes the circular header inclusion,
and as such removes the need and usage of a header inclusion guard
foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl.
2 - Due to the circular interdependency of old lib/setup_once.h and the
c-ares setup_once.h header, old file lib/setup_once.h has carried
back from 2006 up to now days an alarming and prominent notice about
the need of keeping libcurl's and c-ares's setup_once.h in sync.
Given that this commit fixes the circular interdependency, the need
and presence of mentioned notice is removed.
All mentioned interdependencies come back from now old days when
the c-ares project lived inside a curl subdirectory. This commit
removes last traces of such fact.
This is a work-around for bug #1180 which is really libcurl's inability
to ignore SIGPIPE in a few cases. With this work-around at least curl
won't suffer from it!
Bug: http://curl.haxx.se/bug/view.cgi?id=1180
Reported by: Lluís Batlle i Rossell
I ran the 2.59 version of autoupdate that updates obsoleted configure.ac
constructs to the 2.59 standard. With a little hands-on fiddling I
prevented it from ruining the quoting in AS_HELP_STRING() uses.
I subsequently also bumped the required autoconf version to 2.59
(released in December 2003) as I don't have an older autoconf version
around to test with and I can't be bothered to install one either...
Inspired by: Björn Stenberg
Related blog post: http://cazfi.livejournal.com/195108.html
This commit fixes support for sockets that are ready to accept
a new connection and have previously been put into listening mode.
It also includes changes which are the result of investigation
regarding Windows STDIN. These changes are the preparation for further
improvements regarding support for reading data from STDIN on Windows.
Open issue: WaitForMultipleObjectsEx does not support PIPE handles
which are returned by GetStdHandle while running without a GUI.
This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:
f871de0... build: make use of 76 lib/*.h renamed files
ffd8e12... build: rename 76 lib/*.h files
This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:
c087374... curl_setup.h: remove redundant include guard
This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:
13606bb... build: make use of 93 lib/*.c renamed files
5b6e792... build: rename 93 lib/*.c files
7d83dff... build: commit 13606bbfde follow-up 1
Start of related discussion thread:
http://curl.haxx.se/mail/lib-2013-01/0012.html
Asking for confirmation on pushing this revertion commit:
http://curl.haxx.se/mail/lib-2013-01/0048.html
Confirmation summary:
http://curl.haxx.se/mail/lib-2013-01/0079.html
NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.
lib/curl_imap.h
lib/curl_smtp.h
1. When the downloaded data file from Mozilla is current, but the output
bundle does not exist: continue processing to create the bundle. The
goal is to have the output file - not just download the latest input.
2. added -f option to force re-processing the file. Useful for
debugging/testing the process.
3. added support for output to '-' (stdout), allowing the output to be
piped.
4. All progress and error messages go to STDERR rather than STDOUT (3)
5. The script opened and closed the output file many times
unnecessarily. It now opens it once, does the output and closes it.
6. Backup of the input files happens after successful processing, not
before.
7. The output is written to a temporary file, and renamed to the
requested name after backup - this greatly reduces the window where the
file can be seen partially written.
8. all die calls have a \n at the end to suppress perl's traceback - the
traceback isn't useful to end users.
Patch: http://curl.haxx.se/mail/lib-2013-01/0045.html
lib/objnames.inc provides definition of curl_10char_object_name() shell
function. The intended purpose of this function is to transliterate a
(*.c) source file name that may be longer than 10 characters, or not,
into a string with at most 10 characters which may be used as an OS/400
object name.
Test case 1221 does unit testng of this function and also verifies
that it is possible to generate distinct short object names for all
curl and libcurl *.c source file names.
lib/objnames-test.sh is the shell script used for test case 1221.
tests/runtests.pl modified to accept shell script test cases.
More details inside lib/objnames.inc and lib/objnames-test.sh
automake 1.13 errors if AM_CONFIG_HEADER is used in configure script.
automake 1.13 no longer autoupdates AM_CONFIG_HEADER to
AC_CONFIG_HEADERS, thing which automake has been doing since automake
version 1.7
Given that our first automake supported version is automake 1.7,
simply replacing AM_CONFIG_HEADER usage with AC_CONFIG_HEADERS seems
enough to yet support same automake versions.
Dave Reisner reported issue with 1.13 and provided patch.
http://curl.haxx.se/mail/lib-2012-12/0246.html
Provide a 'traceable' AC_CONFIG_MACRO_DIR definition only when using
an autoconf version that does not provide it, instead of what we were
doing up to now of providing and overriding AC_CONFIG_MACRO_DIR for
all autoconf versions.
* Changing the order of the state machine to represent the order in
which commands are sent to the server.
* Reworking the imap_endofresp() function as the FETCH response doesn't
include the command id and shouldn't be part of the length comparison
that takes into account the id string.
Fixed a problem with the state machine when attempting to log in with
invalid credentials. The server would report login failure but libcurl
would not read the response due to inappropriate IMAP_STOP states being
set after the login was sent.
Applied some of the comment and layout changes that had already been
applied to the pop3 and smtp code over the last 6 to 9 months.
This is in preparation of adding SASL based authentication.
... on Snow Leopard and Lion
Snow Leopard introduced the SSLSetSessionOption() function, but it
doesn't disable peer verification as expected on Snow Leopard or
Lion (it works as expected in Mountain Lion). So we now use sysctl()
to detect whether or not the user is using Snow Leopard or Lion,
and if that's the case, then we now use the deprecated
SSLSetEnableCertVerify() function instead to disable peer verification.
WinSock select() does not support standard file descriptors,
it can only check SOCKETs. The following function is an attempt
to create a select() function with support for other handles.
... it also clobbered the 'result' return value so that it wouldn't
return the error back to the parent function properly, which broke test
809 when run with 'multi-always'.
When prefixing a path with /~/ it is supposed to be used relative to the
user's home directory but it didn't work. Now we cut off the entire
three byte sequenct "/~/" which seems to be how OpenSSH does it.
Bug: http://curl.haxx.se/bug/view.cgi?id=1173
Reported by: Balaji Parasuram
Issue: When building a 32bit target with large file support HP-UX
<sys/socket.h> header file may simultaneously provide two different
sets of declarations for sendfile and sendpath functions, one with
static and another with external linkage. Given that we do not use
mentioned functions we really don't care which linkage is the
appropriate one, but on the other hand, the double declaration emmits
warnings when using the HP-UX compiler and errors when using modern
gcc versions resulting in fatal compilation errors.
Mentioned issue is now fixed as long as we don't use sendfile nor
sendpath functions.
When cross-compiling, CURL_CHECK_PKGCONFIG was checking for the cross
pkg-config using ${host}-pkg-config.
The gold standard for doing this correctly is pkg-config's own macro,
PKG_PROG_PKG_CONFIG. However, on the assumption that you have a good
reason not to use that directly (reduced dependencies for maintainer
builds?), the behaviour of cURL's version should at least match.
PKG_PROG_PKG_CONFIG uses AC_PATH_TOOL, which ultimately ends up trying
${host_alias}-pkg-config; this is not quite the same as what cURL does,
and may differ because ${host} has been run through config.sub. For
instance, when cross-building to the armhf architecture on Ubuntu,
${host_alias} is arm-linux-gnueabihf while ${host} is
arm-unknown-linux-gnueabihf. This may also have been the cause of the
problem reported at http://curl.haxx.se/mail/lib-2012-04/0224.html.
AC_PATH_TOOL is significantly simpler than cURL's current code, and
dates back to well before the current minimum of Autoconf 2.57, so let's
use it instead.
A bundle is a list of all persistent connections to the same host.
The connection cache consists of a hash of bundles, with the
hostname as the key.
The benefits may not be obvious, but they are two:
1) Faster search for connections to reuse, since the hash
lookup only finds connections to the host in question.
2) It lays out the groundworks for an upcoming patch,
which will introduce multiple HTTP pipelines.
This patch also removes the awkward list of "closure handles",
which were needed to send QUIT commands to the FTP server
when closing a connection.
Now we allocate a separate closure handle and use that
one to close all connections.
This has been tested in a live system for a few weeks, and of
course passes the test suite.
BLANK_AT_MAKETIME may be used in our Makefile.am files to blank
LIBS variable used in generated makefile at makefile processing
time. Doing this functionally prevents LIBS from being used for
all link targets in given makefile.
This handling already works with the easy-interface code. When a request
is sent on a re-used connection that gets closed by the server at the
same time as the request is sent, the situation may occur so that we can
send the request and we discover the broken connection as a RECV_ERROR
in the PERFORM state and then the request needs to be retried on a fresh
connection. Test 64 broke with 'multi-always-internally'.
Although it is not explicitly stated in the documentation, NSS uses
*pRetCert and *pRetKey even if the client authentication hook returns
a failure. Namely, if we destroy *pRetCert without clearing *pRetCert
afterwards, NSS destroys the certificate once again, which causes a
double free.
Reported by: Bob Relyea
.. that are sent when auth-negotiating before a chunked
upload or when setting the 'Transfer-Encoding: chunked'
header and intentionally sending no content.
Adjust test565 and test1333 accordingly.
Blocking connect on the socket has been removed from opensocket
callback. opensocket just opens a new socket and gives it back to
libcurl and libcurl will take care of the connect. sockopt_callback has
also been removed, as it is no longer required.
AIX sys/poll.h header file defines 'events' and 'revents' as C
preprocessor macros. Usage of these literals in libcurl's external
API was introduced in commit de24d7bd4c causing AIX build failures.
Appropriate inclusion of sys/poll.h by libcurl's external interface
fixes AIX build and usage issues while avoiding a SONAME bump.
runtests.pl -am now uses the "PASS/FAIL: [desc]" output for each
executed test. You can run 'make test-am' in the root build directory to
invoke that. The reason for this output style is to better allow generic
test suite parsers to also grok our test output.
The test Makefile now also tests that perl was indeed found and that the
PERL variable points to an executable before it tries to run the main
test perl script runtests.pl,
It makes no difference from curl's point of view but
makes it more convenient to use the tests with a
lws-normalizing proxy between curl and the test server.
Consistently use CRLF instead. The mixed endings weren't
documented so I assume they were unintentional.
This change doesn't matter for curl itself but makes using
the tests with a proxy between curl and the test server
more convenient.
Tests that consistently use no carriage returns were
left unmodified as one can easily work around this.
DNS cache entries populated with CURLOPT_RESOLVE were not properly freed
again when done using the multi interface.
Test case 1502 added to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=3575448
Reported by: Alex Gruz
If we use memory functions (malloc, free, strdup etc) in C sources in
libcurl and we fail to include curl_memory.h or memdebug.h we either
fail to properly support user-provided memory callbacks or the memory
leak system of the test suite fails.
After Ajit's report of a failure in the first category in http_proxy.c,
I spotted a few in the second category as well. These problems are now
tested for by test 1132 which runs a perl program that scans for and
attempts to check that we use the correct include files if a memory
related function is used in the source code.
Reported by: Ajit Dhumale
Bug: http://curl.haxx.se/mail/lib-2012-11/0125.html
When nothing is told to configure, we should not enforce switching off
debug options with -g0 (or similar). We instead don't use -g at all in
that situaion and therefore allow the user's CFLAGS settings possibly
dictate what to do.
The {MD5,SHA1,SHA256}_Init functions from OpenSSL are called directly
without any wrappers and they return 1 for success, 0 otherwise. Hence,
we have to use the same approach in all the wrapper functions that are
used for the other crypto libraries.
This commit fixes a regression introduced in commit dca8ae5f.
When using only 1 second precision, curl doesn't create new cnonce
values quickly enough for all uses.
For example, issuing the following command multiple times to a recent
Tomcat causes authentication failures:
curl --digest -utest:test http://tomcat.test.com:8080/manager/list
This is because curl uses the same cnonce for several seconds, but
doesn't increment the nonce counter. Tomcat correctly interprets
this as a replay attack and rejects the request.
When microsecond-precision is available, this commit causes curl to
change cnonce values much more frequently.
With microsecond resolution, increasing the nounce length used in the
headers to 32 was made to further reduce the risk of duplication.
axTLS:
This will make the axTLS backend perform the RFC2818 checks, honoring
the VERIFYHOST setting similar to the OpenSSL backend.
Generic for OpenSSL and axTLS:
Move the hostcheck and cert_hostcheck functions from the lib/ssluse.c
files to make them genericly available for both the OpenSSL, axTLS and
other SSL backends. They are now in the new lib/hostcheck.c file.
CyaSSL:
CyaSSL now also has the RFC2818 checks enabled by default. There is a
limitation that the verifyhost can not be enabled exclusively on the
Subject CN field comparison. This SSL backend will thus behave like the
NSS and the GnuTLS (meaning: RFC2818 ok, or bust). In other words:
setting verifyhost to 0 or 1 will disable the Subject Alt Names checks
too.
Schannel:
Updated the schannel information messages: Split the IP address usage
message from the verifyhost setting and changed the message about
disabling SNI (Server Name Indication, used in HTTP virtual hosting)
into a message stating that the Subject Alternative Names checks are
being disabled when verifyhost is set to 0 or 1. As a side effect of
switching off the RFC2818 related servername checks with
SCH_CRED_NO_SERVERNAME_CHECK
(http://msdn.microsoft.com/en-us/library/aa923430.aspx) the SNI feature
is being disabled. This effect is not documented in MSDN, but Wireshark
output clearly shows the effect (details on the libcurl maillist).
PolarSSL:
Fix the prototype change in PolarSSL of ssl_set_session() and the move
of the peer_cert from the ssl_context to the ssl_session. Found this
change in the PolarSSL SVN between r1316 and r1317 where the
POLARSSL_VERSION_NUMBER was at 0x01010100. But to accommodate the Ubuntu
PolarSSL version 1.1.4 the check is to discriminate between lower then
PolarSSL version 1.2.0 and 1.2.0 and higher. Note: The PolarSSL SVN
trunk jumped from version 1.1.1 to 1.2.0.
Generic:
All the SSL backends are fixed and checked to work with the
ssl.verifyhost as a boolean, which is an internal API change.
The text "additional stuff not fine" text was added for debug purposes a
while ago, but it isn't really helping anyone and for some reason some
Linux distributions provide their libcurls built with debug info still
present and thus (far too many) users get to read this info.
The logic previously checked for a started NTLM negotiation only for
host and not also with proxy, leading to problems doing POSTs over a
proxy NTLM that are larger than 2000 bytes. Now it includes proxy in the
check.
Bug: http://curl.haxx.se/bug/view.cgi?id=3582321
Reported by: John Suprock
The existing logic only cut off the fragment from the separate 'path'
buffer which is used when sending HTTP to hosts. The buffer that held
the full URL used for proxies were not dealt with. It is now.
Test case 5 was updated to use a fragment on a URL over a proxy.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579813
As a handle can be re-used after having done HTTP auth in a previous
request, it must make sure to clear out the HTTP types that aren't
wanted in this new request.
This reverts commit ce8311c7e4.
The commit made test 2024 work but caused a regression with repeated
Digest authentication. We need to fix this differently.
After a research team wrote a document[1] that found several live source
codes out there in the wild that misused the CURLOPT_SSL_VERIFYHOST
option thinking it was a boolean, this change now bans 1 as a value and
will make libcurl return error for it.
1 was never a sensible value to use in production but was introduced
back in the days to help debugging. It was always documented clearly
this way.
1 was never supported by all SSL backends in libcurl, so this cleanup
makes the treatment of it unified.
The report's list of mistakes for this option were all PHP code and
while there's a binding layer between libcurl and PHP, the PHP team has
decided that they have an as thin layer as possible on top of libcurl so
they will not alter or specifically filter a 'TRUE' value for this
particular option. I sympathize with that position.
[1] = http://daniel.haxx.se/blog/2012/10/25/libcurl-claimed-to-be-dangerous/
Since automake 1.12.4, the warnings are issued on running automake:
warning: 'INCLUDES' is the old name for 'AM_CPPFLAGS' (or '*_CPPFLAGS')
Avoid INCLUDES and roll these flags into AM_CPPFLAGS.
Compile tested on:
Ubuntu 10.04 (automake 1:1.11.1-1)
Ubuntu 12.04 (automake 1:1.11.3-1ubuntu2)
Arch Linux (automake 1.12.4)
As pointed out in Bug report #3579064, curl_multi_perform() would
wrongly use a blocking mechanism internally for some commands which
could lead to for example a very long block if the LIST response never
showed.
The solution was to make sure to properly continue to use the multi
interface non-blocking state machine.
The new test 1501 verifies the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579064
Reported by: Guido Berhoerster
The URL we used before is the one that goes directly to 'add' a bug
report, but since you can only do that after first having logged in to
sourceforge, the link often doesn't work for visitors.
Bug: http://curl.haxx.se/bug/view.cgi?id=3582408
Reported by: Oscar Norlander
When given a string as 'srp' it didn't work, but required 'SRP'.
Starting now, the check disregards casing.
Bug: http://curl.haxx.se/bug/view.cgi?id=3578418
Reported by: Jeff Connelly
Previously the Metalink code used Apple's CommonCrypto library only if
curl was built using the --with-darwinssl option. Now we use CommonCrypto
on all Apple operating systems including Tiger or later, or iOS 5 or
later, so you don't need to build --with-darwinssl anymore. Also rolled
out this change to libcurl's md5 code.
The iOS build was broken by a reference to a function that only existed
under OS X; fixed. Also fixed a hard-to-reproduce problem where, if the
server disconnected before libcurl got the chance to hang up first and
SecureTransport was in use, then we'd raise an error instead of failing
gracefully.
The makefile is designed to build against a libmetalink devel package;
therefore is does not matter what will change inside libmetalink.
Add OpenSSL includes and defines for libmetalink-aware OpenSSL builds.
This is a minor change in behavior after having been pointed out by Mark
Tully and discussed on the list. Initially this case would internally
call poll() with no sockets and a timeout which would equal a sleep for
that specified time.
Bug: http://curl.haxx.se/mail/lib-2012-10/0076.html
Reported by: Mark Tully
Since there are servers that seem to return very big encrypted
data packages, we need to be able to handle those without having
an internal size limit. To avoid the buffer growing to fast to
early the initial size was decreased and the minimum free space
in the buffer was decreased as well.
During the periods of rate limitation, the speedcheck function wasn't
called and thus the values weren't updated accordingly and it would then
easily trigger wrongly once data got transferred again.
Also, the progress callback's return code was not acknowledged in this
state so it could make an "abort" return code to get ignored and not
have the documented effect of aborting an ongoing transfer.
Bug: http://curl.haxx.se/mail/lib-2012-09/0081.html
Reported by: Jie He
In Metalink v3, the type attribute of url element indicates the
type of the resource the URL points to. It can include URL to the
meta data, such as BitTorrent metainfo file. In Curl, we are not
interested in these meta data URLs. Instead, we are only
interested in the HTTP and FTP URLs. This change filters out
non-HTTP and FTP URLs. If we don't filter out them, it will be
downloaded by curl and hash check will fail if hash is provided
and next URL will be tried. This change will cut this useless
network transfer.
The Curl_reconnect_request() function could end up returning a pointer
to a free()d struct when Curl_done() failed inside. Clearing the pointer
unconditionally after Curl_done() avoids this risk.
Reported by: Ho-chi Chen
Bug: http://curl.haxx.se/mail/lib-2012-09/0188.html
Selected socks proxy in Google's Chrome browser. Resulting in the
following environment variables:
NO_PROXY=localhost,127.0.0.0/8
ALL_PROXY=socks://localhost:1080/
all_proxy=socks://localhost:1080/
no_proxy=localhost,127.0.0.0/8
... and libcurl didn't treat 'socks://' as socks but instead picked HTTP
proxy.
Reported by: Scott Bailey
Bug: http://curl.haxx.se/bug/view.cgi?id=3566860
Since Metalink support requires a crypto library for hash functions
and Windows comes with the builtin CryptoAPI, this patch adds that
API as a fallback to the supported crypto libraries.
It is automatically used on Windows if no other library is provided.
Fixed tests/libtest/libntlmconnect.c:52: warning: call to
'_curl_easy_getinfo_err_long' declared with attribute warning:
curl_easy_getinfo expects a pointer to long for this info
Since Windows/MinGW threat 0x1A as the EOF character, reading binary
files which contain that byte does not work using text mode.
The read function will only read until the first 0x1A byte. This
means that the hash is not computed from the whole file and the
final validation check using hash comparision fails.
Since Simple and Protected GSSAPI Negotiation Mechanism
is already implemented in curl and supported by the MinGW
builds, this change adds build support to winbuild makefiles.
Cleaned up order of handled build options by ordering them
nearly alphabetically by using the order of the generated
config name. Preparation for future/more build options.
While validating a new Clang diagnostic (-Wnon-literal-null-conversion -
yes, the name isn't quite correct in this case, but it suffices) I found
a few violations of it in Curl.
Each certificate section of the input certdata.txt file has a trust
section following it with details.
This script failed to detect the start of the trust for at least one
cert[*], which made the script continue pass that section into the next
one where it found an 'untrusted' marker and as a result that certficate
was not included in the output.
[*] = "Hellenic Academic and Research Institutions RootCA 2011"
Bug: http://curl.haxx.se/mail/lib-2012-09/0019.html
SMTP client will send SIZE parameter in MAIL FROM command only if server
supports it. Without this patch server might say "504 Command parameter
not implemented" and reject the message.
Bug: http://curl.haxx.se/bug/view.cgi?id=3564114
/*
* Name: curl_multi_wait()
*
* Desc: Poll on all fds within a CURLM set as well as any
* additional fds passed to the function.
*
* Returns: CURLMcode type, general multi error code.
*/
CURL_EXTERN CURLMcode curl_multi_wait(CURLM *multi_handle,
struct curl_waitfd extra_fds[],
unsigned int extra_nfds,
int timeout_ms);
The test would hang and get aborted with a "ABORTING TEST, since it
seems that it would have run forever." until I prevented that from
happening.
I also fixed the data file which got broken CRLF line endings when I
sucked down the path from Joe's repo == my fault.
Removed #37 from KNOWN_BUGS as this fix and test case verifies exactly
this.
"Currently you can only share DNS and/or COOKIE data" is incorrect since
also SSL sessions can be shared.
Bug: http://curl.haxx.se/bug/view.cgi?id=3562261
Reported by: Joe Mason
It's conceivable that after the first time curl_multi_perform returns,
the outvalue still_running will be 0, but work will have been done. This
is shown by a workload of small, purely file:// based URLs. Ensure that
we always read pending messages off the multi handle by forcing the
while loop to run at least once.
In Mountain Lion, Apple added TLS 1.1 and 1.2, and deprecated a number
of SecureTransport functions, some of which we were using. We now check
to see if the replacement functions are present, and if so, we use them
instead. The old functions are still present for users of older
cats. Also fixed a build warning that started to appear under Mountain
Lion
Commit b91d29a28e170c16d65d956db79f2cd3a82372d2 introduces a bug and breaks Curl_closesocket function. sock_accepted flag for the second socket should be tagged as TRUE before the sockopt callback is called because in case the callback returns an error, Curl_closesocket function is going to call the - fclosesocket - callback for the accept()ed socket
For active FTP connections, applications may need setting the sockopt after accept() call returns successful. This fix gives a call to the callback registered with CURL_SOCKOPTFUNCTION option. Also a new sock type - CURLSOCKTYPE_ACCEPT - is added. This type is to be passed to application callbacks with - purpose - parameter. Applications may use this parameter to distinguish between socket types.
Removing this option as it currently only functions to lure people into
wrongly using it and falsely believing that libcurl will work fine
without using nonblocking sockets internally - which leads to hard to
track or understand errors.
Commit e351972bc8 brought in the ssh agent support but some uses of
the libssh2 agent API was done unconditionally which wasn't good enough
since that API hasn't always been present.
By reading the ->head pointer and using that instead of the ->size
number to figure out if there's a list remaining we avoid the (false
positive) clang-analyzer warning that we might dereference of a null
pointer.
I suspect this is a regression introduced in commit 207cf150, included
since 7.24.0.
Avoid showing '(nil)' as hostname in verbose output by making sure the
hostname fixup function is called early enough to set the pointers that
are used for this. The name data is set again for each request even for
re-used connections to handle multiple hostnames over the same
connection (like with proxy) or that the casing etc of the host name is
changed between requests (which has proven to be important at least once
in the past).
Test1011 was modified to use a redirect with a re-used a connection
since it then showed the bug and now lo longer does. There's currently
no easy way to have the test suite detect 'nil' texts in verbose ouputs
so no tests will detect if this problem gets reintroduced.
Bug: http://curl.haxx.se/mail/lib-2012-07/0111.html
Reported by: Gisle Vanem
We found a problem with ftp transfer using libcurl (7.23 and 7.25)
inside an application which is receiving unix signals (SIGUSR1,
SIGUSR2...) almost continuously. (Linux 2.4, PowerPC, HAVE_POLL_FINE
defined).
Curl_socket_check() uses poll() to wait for the socket, and retries it
when a signal is received (EINTR). However, if a signal is received and
it also happens that the timeout has been reached, Curl_socket_check()
returns -1 instead of 0 (indicating an error instead of a timeout).
In our case, the result is an aborted connection even before the ftp
banner is received from the server, and a return value of
CURLE_OUT_OF_MEMORY from curl_easy_perform() (Curl_pp_multi_statemach(),
in pingpong.c, actually returns OOM if Curl_socket_check() fails :-)
Funny to debug on a system on which OOM is a possible cause).
Bug: http://curl.haxx.se/mail/lib-2012-07/0122.html
Due to WSAPoll bugs, libcurl does not work as intended. When the cURL
library is used to setup a connection to an incorrect port, normally the
result is CURLE_COULDNT_CONNECT, /* 7 */, but due to the bug in WSAPoll,
the result now is CURLE_OPERATION_TIMEDOUT, /* 28 - the timeout time was
reached */.
On August 1, Jan Koen Annot opened a case for this to Microsoft Premier
Online (https://premier.microsoft.com/). The support engineer handling
the case wrote that the case description is quite clear. He will try to
reproduce the issue and then proceed with troubleshooting it.
Reported by: Jan Koen Annot
Bug: http://curl.haxx.se/mail/lib-2012-07/0310.html
When figuring out if the data stream needs to be rewound when the
request is to be resent, we must not access the HTTP struct unless the
protocol used is indeed HTTP...
Bug: http://curl.haxx.se/bug/view.cgi?id=3544688
Fix a bug where closed sockets (fd -1) were left in the all_sockets
list, because of missing parens in a pointer arithmetic expression
Reenable the tests that were locking up due to this bug.
SO_KEEPALIVE flag to all sockets. Note that several loops which used to continue on a return value
of 0 (theoretical since 0 would never be returned without O_NONBLOCK) now break on 0 so that they
won't continue reading until after poll is called again.
for non-blocking sockets: now -1 means error or connection finished, 1 means data was read, and 0
means there is no data available now so need to wait for poll (new return value)
half-finished.
Note the the req struct used to be re-initialized AFTER reading pipeline data, so now that we
initialize it from the caller we must be careful not to overwrite the pipeline data.
Also we now need to handle the case where the buffer is already full when get_request is called -
previously this never happened as it was always called with an empty buffer and looped until done.
Now get_request is called in a loop, so the next step is to run the loop on a socket only when poll
signals it is readable.
Removed APOP and SASL authentication from the POP3 section and metalink
support from the client section as these features were implemented in
this release.
Moved adding gssapi to SASL into it's own section rather than repeat it
for each protocol.
This needs another look from the configure experts. I tested that
it works so far with MinGW64 cross-compiler; libcurl builds and
links fine, but curl not yet ...
Previously the curl_multi interface would freeze if darwinssl was
enabled and at least one of the handles tried to connect to a Web site
using HTTPS. Removed the "wouldblock" state darwinssl was using because
I figured out a solution for our "would block but in which direction?"
dilemma.
Replaced the Windows real path from mount hack with a more
reliable and simpler hack: the MSYS shell has a builtin pwd
which understands a -W option which does convertion to Windows
paths. Tested and confirmed that this works on all MSYS versions
I have back to a 3 year old one.
In many states the easy_conn pointer is referenced and just assumed to
be working. This is an added extra check since analyzing indicates
there's a risk we can end up in these states with a NULL pointer there.
I made "connmon" not get initialized properly before use, and I use the
big hammer and make sure we always clear the entire struct to avoid any
problem like this in the future.
Two commits ago, we fixed a bug where the connction would be closed
prematurely after a HEAD. Now I added connection-monitor to test 48 and
added a second HEAD and make sure that both are sent over the same
connection.
This triggered a failure before the bug fix and now works. Will help us
avoid a future regression of this kind.
This makes verifying easier and makes us more sure curl closes the
connection only at the correct point in time. Adjusted test 206 and 1008
accordingly and updated the docs for it.
A HEAD response has no body length and gets the headers like the
corresponding GET would so it should not get closed after the response
based on the same rules. This mistake caused connections that did HEAD
to get closed too often without a valid reason.
Bug: http://curl.haxx.se/bug/view.cgi?id=3542731
Reported by: Eelco Dolstra
1 - str2offset() no longer accepts negative numbers since offsets are by
nature positive.
2 - introduced str2unum() for the command line parser that accepts
numericals which are not supposed to be negative, so that it will
properly complain on apparent bad uses and mistakes.
Bug: http://curl.haxx.se/mail/archive-2012-07/0013.html
Since the order of the cookies is sorted by the length of the paths,
having them on the same path length will make the test depend on what
order the qsort() implementation will put them. As seen in the
windows/msys output posted by Guenter in this posting:
http://curl.haxx.se/mail/lib-2012-07/0105.html
The function https_getsock was only implemented properly when USE_SSLEAY
or USE_GNUTLS is defined, but it is also necessary for USE_SCHANNEL.
The problem occurs when Curl_read_plain or Curl_write_plain returns
CURLE_AGAIN. In that case CURL_OK is returned to the multi-interface an
the used socket is set to state CURL_POLL_REMOVE and the easy-state is
set to CURLM_STATE_PROTOCONNECT. This is fine, because later the socket
should be set to CURL_POLL_IN or CURL_POLL_OUT via multi_getsock. That's
where https_getsock is called and doesn't return any sockets.
Since WinSSL cannot be build without SSPI being enabled,
USE_WINSSL now defaults to the value of USE_SSPI.
The makefile does now raise an error if WinSSL is enabled
while SSPI is disabled.
Renamed external parameter USE_SSPI = yes/no to ENABLE_SSPI = yes/no.
Backwards compatible change: USE_SSPI can still be passed as external
parameter with yes/no value as long as ENABLE_SSPI is not given.
USE_x defines are passed around with true/false values internally,
USE_SSPI is now aligned to this approach, but still accepts external
values yes/no being passed, just like the other defines.
- Changed space usage to line up with the whole file
- Renamed CFLAGS_SSPI/IPV6 to SSPI/IPV6_CFLAGS to be
consistent with the other CFLAGS_x variables
- Make use of existing CFLAGS_IPV6 (previously IPV6_CFLAGS)
instead of appending directly to CFLAGS
The code was printing a warning when SNI was set up successfully. Oops.
Printing the cipher number in verbose mode was something only TLS/SSL
programmers might understand, so I had it print the name of the cipher,
just like in the OpenSSL code. That'll be at least a little bit easier
to understand. The SecureTransport API doesn't have a method of getting
a string from a cipher like OpenSSL does, so I had to generate the
strings manually.
Test 1008 and 206 don't show the disconnect since it happens when SWS
awaits a new request, but 503 does and so the verify section needs that
string added.
When doing CONNECT requests, libcurl must make sure the connection is
alive as much as possible. NTLM requires it and it is generally good for
other cases as well.
NTLM over CONNECT requests has been broken since this regression I
introduced in my CONNECT cleanup commits that started with 41b0237834,
included since 7.25.0.
Bug: http://curl.haxx.se/bug/view.cgi?id=3538625
Reported by: Marcel Raad
I moved out the servercmd parsing into a its own function called
parse_servercmd() and made sure it gets used also when the test number
is extracted from CONNECT requests. It turned out sws didn't do that
previously!
Using this, the server will output in the protocol log when the
connection gets disconnected and thus we will verify correctly in the
test cases that the connection doesn't get closed prematurely. This is
important for example NTLM to work.
Documentation added to FILEFORMAT, test 503 updated to use this.
Currently it is unknown if there is any version of clang that
actually supports -Wstrict-aliasing. What is known is that there
are several that don't support it.
With this commit, checks done in previous test2017 are now done in test2018.
Whole range test2017 to test2022 DISABLED until configure is capable of
requiring a new-enough metalink library.
Don't try these without mentioned check in place!
Print "parsing (...) OK" only when no warnings are generated. If
no file is found in Metalink, treat it FAILED.
If no digest is provided, print WARNING in parse_metalink().
Also print validating FAILED after download.
These changes make tests 2012 to 2016 pass.
Allow NTLM authentication when building using SecureTransport (Darwin) for SSL.
This uses CommonCrypto, a cryptography library that ships with all versions of
iOS and Mac OS X. It's like OpenSSL's libcrypto, except that it's missing a few
less-common cyphers and doesn't have a big number data structure.
Before commit 2dded8fedb (dec 2010) there was logic that used
RAND_screen() at times and now I remove the leftover #ifdef check for
it.
The seeding code that uses Curl_FormBoundary() in ossl_seed() is dubious
to keep since it hardly increases randomness but I fear I'll break
something if I remove it now...
- Renamed st_ function prefix to darwinssl_
- Renamed Curl_st_ function prefix to Curl_darwinssl_
- Moved the duplicated ssl_connect_done out of the #ifdef in lib/urldata.h
- Fixed a teensy little bug that made non-blocking connection attempts block
- Made it so that it builds cleanly against the iOS 5.1 SDK
Documented that --include will be ignored if both --metalink
and --include are specified.
Also documented that a Metalink file in the local file system
cannot be used if FILE protocol is disabled.
Including headers in response body will break Metalink XML parser.
If it is included in the file described in Metalink XML, hash check
will fail. Therefore, --include should be ignored if --metalink is
used.
The noprogress and isatty in Configurable are global, in a sense
that they persist in one curl invocation. Currently once one
download writes its response data to tty, they are set to FALSE
and they are not restored on successive downloads. This change
first backups the current noprogress and isatty, and restores
them when download does not write its data to tty.
In this change, --metalink option no longer takes argument. If
it is specified, given URIs are processed as Metalink XML file.
If given URIs are remote (e.g., http URI), curl downloads it
first. Regardless URI is local file (e.g., file URI scheme) or
remote, Metalink XML file is not written to local file system and
the received data is fed into Metalink XML parser directly. This
means with --metalink option, filename related options like -O
and -o are ignored.
Usage examples:
$ curl --metalink http://example.org/foo.metalink
This will download foo.metalink and parse it and then download
the URI described there.
$ curl --metalink file://foo.metalink
This will parse local file foo.metalink and then download the URI
described there.
When creating metalink_checksum from metalink_checksum_t, first
check hex digest is valid for the given hash function. We do
this check in the order of digest_aliases so that first good
match will be chosen (strongest hash function available). As a
result, the metalinkfile now only contains at most one
metalink_checksum because other entries are just redundant.
Removed two, not intended to exist, RESOURCE declarations.
Bug: http://curl.haxx.se/bug/view.cgi?id=3535977
And sorted configuration hunks to reflect same internal order
as the one shown in the usage message.
Increase decrypted and encrypted cache buffers using limitted
doubling strategy. More information on the mailinglist:
http://curl.haxx.se/mail/lib-2012-06/0255.html
It updates the two remaining reallocations that have already been there
and fixes the other one to use the same "do we need to increase the
buffer"-condition as the other two. CURL_SCHANNEL_BUFFER_STEP_SIZE was
renamed to CURL_SCHANNEL_BUFFER_FREE_SIZE since that is actually what it
is now. Since we don't know how much more data we are going to read
during the handshake, CURL_SCHANNEL_BUFFER_FREE_SIZE is used as the
minimum free space required in the buffer for the next operation.
CURL_SCHANNEL_BUFFER_STEP_SIZE was used for that before, too, but since
we don't have a step size now, the define was renamed.
Test definition section parts which needed to include xml-lingo as contents
of that part required that the xml-blurb was written as a single line. Now the
xml-data inside the part can be written multiline making it more readable.
Tested with <client><file> part which is written to disk before <command> runs.
Process extra data buffer before returning from schannel_connect_step2.
Without this change I've seen WinCE hang when schannel_connect_step2
returns and calls Curl_socket_ready.
If the encrypted handshake does not fit in the intial buffer (seen with
large certificate chain), increasing the encrypted data buffer is necessary.
Fixed warning in curl_schannel.c line 1215.
Implemented timeout loop in schannel_send while sending data. This
is as close as I think we can get to write buffering; I put a big
comment in to explain my thinking.
With some committer adjustments
Make the Schannel implementation use libcurl's default buffer size
for the initial received encrypted and decrypted data cache buffers.
The implementation still needs to handle more data since more data
might have already been received or decrypted during the handshake
or a read operation which needs to be cached for the next read.
curl_schannel.c - implemented graceful SSL shutdown. If we fail to
shutdown the connection gracefully, I've seen schannel try to use a
session ID for future connects and the server aborts the connection
during the handshake.
curl_schannel.c - auto certificate validation doesn't seem to work
right on CE. I added a method to perform the certificate validation
which uses CertGetCertificateChain and manually handles the result.
Coverity actually pointed out flawed logic in the previous call to
Curl_strntoupper() where the code used sizeof() of a pointer to pass in
a size argument. That code still worked since it only needed to
uppercase 4 letters. Still, the entire malloc/uppercase/free sequence
was pointless since the code has already matched the string once in the
condition that starts the block of code.
As spotted by Coverity, va_end() was not used previously. To make it
used I took away a bunch of return statements and made them into
assignments instead.
SSPI related code now compiles with ANSI and WCHAR versions of security
methods (WinCE requires WCHAR versions of methods).
Pulled UTF8 to WCHAR conversion methods out of idn_win32.c into their own file.
curl_sasl.c - include curl_memory.h to use correct memory functions.
getenv.c and telnet.c - WinCE compatibility fix
With some committer adjustments
The changes introduced in commit 2bfa57bc32 are not enough
to make it actually possible to use the USE_WINSSL option.
Makefile.vc was not updated and the configuration name which is
used in the build path did not match between both build files.
This patch fixes those issues and introduces the following changes:
- Replaced the -schannel name with -winssl in order to be consistent
with the other options
- Added ENABLE_WINSSL option to winbuild/Makefile.vc (default yes)
- Changed winbuild/MakefileBuild.vc to set USE_WINSSL to true if
USE_SSL is false and USE_WINSSL was not specified as a parameter
- Separated WINSSL handling from SSPI handling to be consistent with
the other options and their corresponding code path
Building with CyaSSL failed compilation. Reason being that OCSP_REQUEST and
OCSP_RESPONSE are enum values in CyaSSL and defines in <wincrypt.h> included
via <winldap.h> in ldap.c.
http://curl.haxx.se/mail/lib-2012-06/0196.html
This option may be used to build curl/libcurl using SSL/TLS support provided
by MS windows system libraries. Option is mutually exclusive with any other
SSL library. Default value is --without-winssl.
--with-winssl option implies --with-sspi option.
Option meaningful only for Windows builds.
Version number is removed in order to make this info consistent with
how we do it with other MS and Linux system libraries for which we don't
provide this info.
Identifier changed from 'WinSSPI' to 'schannel' given that this is the
actual provider of the SSL/TLS support. libcurl can still be built with
SSPI and without SCHANNEL support.
Removed obsolete minor status variable and parameter of status function
which was never used or set at all. Also Curl_sspi_strerror does support
only one status and there is no need for a second sub status.
Removed specific WITH_SSL=schannel paramter that did not fit the general
schema and complicated the parameters. For now Schannel will be enabled
if SSPI is enabled and OpenSSL is disabled.
Added Windows SSPI version information to the curl version string when
SCHANNEL SSL is not enabled, as the version of the library should also
be included when SSPI is used to generate security contexts.
Removed SSPI from the feature list as the features are GSS-Negotiate,
NTLM and SSL depending on the usage of the SSPI library.
Removed duplicate blank lines.
Removed spaces between the not and test in various if statements.
Removed explicit test of NULL in an if statement.
Placed function returns on same line as function declarations.
Replaced the use of curl_maprintf() with aprintf() as it is the
preprocessor job to do this substitution if ENABLE_CURLX_PRINTF
is set.
curl_sspi.c: Fixed mingw32-gcc compiler warnings
curl_sspi.c: Fixed length of error code hex output
The hex value was printed as signed 64-bit value on 64-bit systems:
SEC_E_WRONG_PRINCIPAL (0xFFFFFFFF80090322)
It is now correctly printed as the following:
SEC_E_WRONG_PRINCIPAL (0x80090322)
curl_sspi.c: Fallback to security function table version number
Instead of reporting an unknown version, the interface version is used.
curl_sspi.c: Removed SSPI/ version prefix from Curl_sspi_version
curl_schannel: Replaced static buffer sizes with defined names
curl_schannel.c: First brace when declaring functions on column 0
curl_schannel.c: Put the pointer sign directly at variable name
curl_schannel.c: Use structs directly instead of typedef'ed structs
curl_schannel.c: Removed space before opening brace
curl_schannel.c: Fixed lines being longer than 80 chars
Moved the error constant switch to curl_sspi.c and added two new helper
functions to curl_sspi.[ch] which either return the constant or a fully
translated message representing the SSPI security status.
Updated socks_sspi.c and curl_schannel.c to use the new functions.
Windows 2000 Professional: Schannel returns SEC_E_OK instead
of SEC_I_CONTEXT_EXPIRED. If the length of the output buffer
is zero and the first byte of the encrypted packet is 0x15,
the application can safely assume that the message was a
close_notify message and change the return value to
SEC_I_CONTEXT_EXPIRED.
Connection shutdown does not mean that there is no data to read
Correctly handle incomplete message and ask curl to re-read
Fixed buffer for decrypted being to small
Re-structured read condition to be more effective
Removed obsolete verbose messages
Changed memory reduction method to keep a minimum buffer of size 4096
Fixed warning: dereferencing pointer does break strict-aliasing rules
by using a union instead of separate pointer variables.
Internal union sockaddr_u could probably be moved to generic header.
Thanks to Paul Howarth for the hint about using unions for this.
Important for winbuild: Separate declaration of sockaddr_u pointer.
The pointer variable *sock cannot be declared and initialized right
after the union declaration. Therefore it has to be a separate statement.
Since Curl_pgrsDone() itself calls Curl_pgrsUpdate() which may return an
abort instruction or similar we need to return that info back and
subsequently properly handle return codes from Curl_pgrsDone() where
used.
(Spotted by a Coverity scan)
Fixed USE_IPV6 and USE_IDN not being passed
from Makefile.vc to MakefileBuild.vc
Fixed whitespace and formatting issues
Fixed typo and format in help message
Previously it would use a 256 byte buffer and thus cut off very long
subject names. The limit is now upped to the receive buffer size, 16K.
Bug: http://curl.haxx.se/bug/view.cgi?id=3533045
Reported by: Anthony G. Basile
Additionally, make hash checking ability mandatory in order to allow metalink
support in curl.
A command line option could be introduced to skip hash checking at runtime,
but the ability to check hashes should always be built-in when providing
metalink support.
Re-factored the smtp_state_*_resp() functions to 1) Match the constants
that were refactored in commit 00fddba672, 2) To be more readable and
3) To match their counterparties in pop3.c.
Verify that the "Saved to filename 'blabla'" message is only displayed when
the 'blabla' filename being used _actually_ has been specified by the server
in the Content-Disposition header.
Use relative path for unintended file creation postcheck.
When a <file> part is now specified with no contents at all, this
will actually verify that the specified file has no contents at all.
Previously file contents would be ignored.
Corrected lines longer than 78 characters.
Removed unnecessary braces in smtp_state_helo_resp().
Introduced some comments in data sending functions.
Tidied up comments to match changes made in pop3.c.
Corrected lines longer than 78 characters.
Changed POP3_AUTH_FINAL to POP3_AUTH to match SMTP code now that the
AUTH command is no longer sent on its own.
Introduced some comments in data sending functions.
Another attempt at trying to rational code and comment style.
Added a service type parameter to Curl_sasl_create_digest_md5_message()
to allow the function to be used by different services rather than being
hard coded to "smtp".
Not all SASL enabled POP3 servers support the AUTH command on its own
when trying to detect the supported mechanisms. As such changed the
mechanism detection to use the CAPA command instead.
- For all *FUNCTION options, they now all show the complete prototype in
the description. Previously some of them would just refer to a
typedef'ed function pointer in the curl.h header.
- I made the phrasing of that "Pass a pointer to a function that matches
the following prototype" the same for all *FUNCTION option descriptions.
- I removed some uses of 'should'. I think I sometimes over-use this
word as in many places I actually mean MUST or otherwise more specific
and not-so-optional synonyms.
Because pop3_endofresp() is called for each line of data yet is not
passed the line and line length, so we have to use the data pointed to
by pp->linestart_resp which contains the whole packet, the mechanisms
were being detected in one call yet the function would be called for
each line of data.
Using curl with verbose mode enabled would show that one line of data
would be received in response to the AUTH command, before the AUTH
<mechanism> command was sent to the server and then the next few lines
of the original AUTH command would be displayed before the response from
the AUTH <mechanism> command. This would then cause problems when
parsing the CRAM-MD5 challenge data as extra data was contained in the
buffer.
Changed the parsing so that each line is checked for the mechanisms
and the function returns FALSE until the whole of the AUTH response has
been processed.
Added new sections 11. IMAP and 12. LDAP to document adding SASL based
authentication.
Renumbered current sections 11 to 17 as 13 to 19.
Additionally added 19.10 Add CURLOPT_MAIL_CLIENT option.
Previously it wasn't possible to connect to POP3 and not specify the
user name as a CURLE_ACCESS_DENIED error would be returned. This error
occurred because USER would be sent to the server with a blank user name
if no mailbox user was specified as the server would reply with -ERR.
This wasn't a problem prior to the 7.26.0 release but with the
introduction of custom commands the user and/or application developer
might want to issue a CAPA command without having to log in as a
specific mailbox user.
Additionally this fix won't send the newly introduced AUTH command if no
user name is specified.
This is done introducing tags <file1> to <file4> besides existing <file> one,
as well as corresponding <stripfile1> to <stripfile4> ones, that can be used
in the <verify> section in the same way as the non-numbered ones.
Rather than encoding the password message itself the
smtp_state_authpasswd_resp() function now delegates the work to the same
function that smtp_state_authlogin_resp() and smtp_authenticate() use
when constructing the encoded user name.
In preparation for moving to the SASL module re-factored the
smtp_auth_login_user() function to smtp_auth_login() so that it can be
used for both user names and passwords as sending both of these under
the login authentication mechanism is the same.
Introduce SUPPORTCAPA and SUPPORTAUTH config commands to allow further
pop3 test server expansion for tests that require CAPA or AUTH support,
although this will need some extra work to make it fully functional.
The POP3 protocol doesn't really have the concept of error codes and
uses +, +OK and -ERR in response to commands to indicate continue,
success and error.
The AUTH command is one of those commands that requires multiple pieces
of data to be sent to the server where the server will respond with + as
part of the handshaking. This meant changing the values before
continuing with the next stage of adding authentication support.
Changed the order of the state machine to match the order of actual
events.
Reworked some comments and function parameter positioning that I missed
the other day.
Added support for detecting the supported SASL authentication mechanisms
via the AUTH command. There are two ways of detecting them, either by
using the AUTH command, that will return -ERR if not supported or by
using the CAPA command which will return SASL and the list of mechanisms
if supported, not include SASL if SASL authentication is not supported
or -ERR if the CAPA command is not supported. As such it seems simpler
to use the AUTH command and fallback to normal clear text authentication
if the the command is not supported.
Additionally updated the test cases to return -ERR when the AUTH command
is encountered. Additional test cases will be added when support for the
individual authentication mechanisms is added.
Moved EOB definition into header file.
Switched the logic around in pop3_endofresp() to allow for the
introduction of auth-mechanism detection.
Repositioned second and third function variables where they will fit
within the 78 character line limit.
Tidied up some comments.
Metalink file contains several hash types of checksums, such as
md5, sha-1, sha-256, etc. To deal with these checksums, I created
abstraction layer based on lib/curl_md5.h and
lib/md5.c. Basically, they are almost the same but I changed the
code so that it is not hash type dependent. Currently,
GNUTLS(nettle or gcrypt) and OpenSSL functions are supported.
Checksum checking is done by reopening download file. If there
is an I/O error, the current implementation just prints error
message and does not try next resource.
In this patch, the supported hash types are: md5, sha-1 and sha-256.
Filenames contained in Metalink file can include directory information.
Filenames are unique in Metalink file, taking into account the directory
information. So we need to create the directory hierarchy.
Curl has --create-dirs option, but we create directory hierarchy for
Metalink downloads regardless of the option value.
This patch also put metalink int variable outside of HAVE_LIBMETALINK
guard. This reduces the number of #ifdefs.
This change adds experimental Metalink support to curl.
To enable Metalink support, run configure with --with-libmetalink.
To feed Metalink file to curl, use --metalink option like this:
$ curl -O --metalink foo.metalink
We use libmetalink to parse Metalink files.
Setting bit 2 for this value was documented as having a constant value
defined as CURL_REDIR_POST_303 yet referenced a 302 request.
Additionally corrected the meaning of CURL_REDIR_POST_ALL for all three
bits and fixed problems with the bolding of keywords in this section.
Standardised how RFCs are referenced so that the website may autolink to
the correct documentation on ietf.org. Additionally removed the one link
to RFC3986 on curl.haxx.se.
To achieve this, first new structure HeaderData is defined to hold
necessary data to perform header-related work. Then tool_header_cb now
receives HeaderData pointer as userdata. All header-related work
(currently, dumping header and Content-Disposition inspection) are done
in this callback function. HeaderData.outs->config is used to determine
whether each work is done.
Unit tests were also updated because after this change, curl code always
sets CURLOPT_HEADERFUNCTION and CURLOPT_HEADERDATA.
Tested with -O -J -D, -O -J -i and -O -J -D -i and all worked fine.
Move the SMTP_AUTH constants into a separate header file in
preparation for adding SASL based authentication to POP3 as the two
protocols will need to share them.
Due to the result code being reset to CURLE_OK when smtp_dophase_done()
was called, postdata would incorrectly be sent to the server when the
MAIL FROM or RCPT command was rejected.
As such, libcurl would return the wrong result code from performing the
operation and additionally set CURLINFO_RESPONSE_CODE to be that
returned by the postdata command.
Bug: http://curl.haxx.se/mail/lib-2012-05/0108.html
Reported by: Gokhan Sengun
In nettle/md5.h, md5_init and md5_update are defined as macros to
nettle_md5_init and nettle_md5_update respectively. This causes
error when using MD5_params.md5_init and md5_update. This patch
renames these members as md5_init_func and md5_update_func to
avoid name conflict. For completeness, MD5_params.md5_final was
also renamed as md5_final_func.
The changes in curl_ntlm_core.c is conversion error and fixed by
casting to proper type.
The curl-config command must be used twice in the single command line to
work properly in some environments.
Bug: http://curl.haxx.se/bug/view.cgi?id=3528241
Reported by: Julian Taylor
A dot character at the beginning of a line would not be escaped to a
double dot as required by RFC-2821, instead it would be deleted by the
mail server. Please see section 4.5.2 of the RFC for more information.
Note: This fix also simplifies the detection of repeated CRLF.CRLF
combinations, such as CRLF.CRLF.CRLF, a little rather than having to
advance the eob counter to 2.
It checks whether versioned symbols should be enabled before checking
whether it is possible (i.e. the linker supports --version-script) or
not. This avoids a useless warning when building cURL on a platform that
does not use GNU ld.
Moreover, it fixes broken indentation of this chunk of code.
Roman Mamedov spotted (in
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=670126) that curl would
not complain when given a URL with an IPv6 numerical address without
brackets. It would simply cut off the last ":[hex]" part and thus not
work correctly.
That's a URL using an illegal syntax and now libcurl will instead return
a clear error code and error message detailing the error.
The above mentioned bug report claims this to be a regression but
libcurl does not guarantee functionality when given URLs that aren't
following the URL spec (RFC3986 mostly). I consider the fact that it
used to handle this differently a mere coincidence.
When doing a chunked-encoded POST with -d (CURLOPT_POSTFIELDS) and the
size of the POST was zero length, it made libcurl first send a zero
chunk and then the terminating one. This could confuse a receiver and it
should rather just send the terminating chunk as it does with this fix.
Test case 1333 is added to verify.
Bug: http://curl.haxx.se/mail/archive-2012-04/0060.html
Reported by: Arnaud Compan
Commit 9109cdec11 brought this regression (shipped since 7.24.0).
The singleipconnect() function must not return an error if Curl_socket()
returns an error. It should then simply return OK and pass a SOCKET_BAD
back simply because that is how the user of this function expects it to
work and something else is not fine.
Reported by: Blaise Potard
Bug: http://curl.haxx.se/bug/view.cgi?id=3516508
Explicit conversion to 'long' of curl_easy_setopt() third argument for options
CURLOPT_HTTPAUTH and CURLOPT_PROXYAUTH given that this is how its bitmasks are
docummented to be used.
By comparing if a different "progress point" is reached or not since the
previous update, the progress function callback for this now avoids many
superfluous screen updates. This has the nice side-effect that it fixes
a problem that causes a second progress meter line.
The second line output happened because when we use the -# progress
meter, we force a newline output after the transfer in the main loop in
curl, but when libcurl calls the progress callback from
curl_easy_cleanup() it would then output the progress display
again. Possibly the naive newline output is wrong but this optimization
was suitable anyway...
Reported by: Daniel Theron
Bug: http://curl.haxx.se/bug/view.cgi?id=3517418
Include stdbool.h only when it is available and configure is capable of
detecting a proper 'bool' data type when the header is included.
Compilation fix for old or unpatched versions of XL C compiler.
Report: http://curl.haxx.se/mail/archive-2012-04/0022.html
NSS_InitContext() was introduced in NSS 3.12.5 and helps to prevent
collisions on NSS initialization/shutdown with other libraries.
Bug: https://bugzilla.redhat.com/738456
Allow repeatable file name length reduction on file names with underscore or
dash characters. This is done in order to better support libcurl's existing
source file names and allow OS/400 package to build out of the box again.
configure script now provides conditional definitions for Makefile.am
that result in CURL_HIDDEN_SYMBOLS being defined by resulting makefiles
when appropriate.
Additionally, configure script option for symbol hiding control is now
named --enable-symbol-hiding --disable-symbol-hiding. While still valid,
old option name --enable-hidden-symbols --disable-hidden-symbols will
be deprecated in some future release.
BUILDING_LIBCURL and CURL_STATICLIB are no longer defined in curl_config.h,
configure will generate appropriate conditionals so that mentioned symbols
get defined and used in Makefiles at compilation time
Configuration files such as curl_config.h and all config-*.h no longer exist
nor are generated/copied into 'src' directory, now these only exist in 'lib'
directory from where curl tool sources uses them.
Additionally old src/setup.h has been refactored into src/tool_setup.h which
now pulls lib/setup.h
The possibility of a makefile needing an include path adjustment exists.
Don't set the "has_openssl" variable if yassl or polarssl is found as
they will simply not work as 100% drop-in replacements for some of the
stuff the "OpenSSL" feature is used for.
I spotted this problem when doing test runs with PolarSSL builds.
Curl_socket returns CURLE_COULDNT_CONNECT when the opensocket callback
returns CURL_SOCKET_BAD. Previous return value CURLE_FAILED_INIT
conveys incorrect information to the user.
Reworked the command sending from two specific LIST and RETR command
functions into a single command based function as well as the two
associated response handlers into a generic command handler.
By modifying the parameter list for ourWriteOut() and passing the
OutStruct that collects data in tool_operate, we get access to the
remote name that we're writing to. Shell scripters should find this
useful when used in conjuntion with the --remote-header-name option.
If an empty string is passed to CURLOPT_SSH_PUBLIC_KEYFILE, libcurl will
pass no public key to libssh2 which then tries to compute it from the
private key. This is known to work when libssh2 1.4.0+ is linked against
OpenSSL.
This change replaces RFC 2818 based hostname check in OpenSSL build with
RFC 6125 [1] based one.
The hostname check in RFC 2818 is ambiguous and each project implements
it in the their own way and they are slightly different. I check curl,
gnutls, Firefox and Chrome and they are all different.
I don't think there is a bug in current implementation of hostname
check. But it is not as strict as the modern browsers do. Currently,
curl allows multiple wildcard character '*' and it matches '.'. (as
described in the comment in ssluse.c).
Firefox implementation is also based on RFC 2818 but it only allows at
most one wildcard character and it must be in the left-most label in the
pattern and the wildcard must not be followed by any character in the
label.[2] Chromium implementation is based on RFC 6125 as my patch does.
Firefox and Chromium both require wildcard in the left-most label in the
presented identifier.
This patch is more strict than the current implementation, so there may
be some cases where old curl works but new one does not. But at the same
time I think it is good practice to follow the modern browsers do and
follow the newer RFC.
[1] http://tools.ietf.org/html/rfc6125#section-6.4.3
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=159483
With FOLLOWLOCATION enabled. When a 3xx page is downloaded and the
download size was known (like with a Content-Length header), but the
subsequent URL (transfered after the 3xx page) was chunked encoded, then
the previous "known download size" would linger and cause the progress
meter to get incorrect information, ie the former value would remain
being sent in. This could easily result in downloads that were WAY
larger than "expected" and would cause >100% outputs with the curl
command line tool.
Test case 599 was created and it was used to repeat the bug and then
verify the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3510057
Reported by: Michael Wallner
It is now possible to calculate the md5 sum as the stream of buffers
becomes known where as previously it was only possible to calculate the
md5 sum of a pre-prepared buffer.
This feature allows the user to specify and use additional POP3
commands such as UIDL and DELE via libcurl's CURLOPT_CUSTOMREQUEST or
curl's -X command line option.
Simplified the code to remove the need for a separate "LIST <msg id>"
command handler and state machine and instead use the LIST command
handler for both operations.
Moved the server greeting response handling code from the statemach_act
functions to separate response functions. This makes the code simpler
to follow and provides consistency with the other responses that are
handled here.
Original wording could lead users in thinking it tries to
somehow parse the filename for a date expression (like
news_2012_03_05.html). It never mentions that it actually
reads the mtime of the file in filesystem.
This allows building of libcurl on DOS using DJGPP 2.04 and Watt-32
sockets. I know there's already Makefile.djgpp, but I find this more
convenient since I'm used to using the ./configure script from other
platforms
# (From Daniel Stenberg) The cmake build selected to run gcc with -fPIC on my box while the plain configure script did not.
# (From Daniel Stenberg) The gcc command line use neither -g nor any -O options. As a developer, I also treasure our configure scripts's --enable-debug option that sets a long range of "picky" compiler options.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
