CURLOPT_DNS_USE_GLOBAL_CACHE broke in commit c43127414d (been
broken since the libcurl 7.29.0 release). While this option has been
documented as deprecated for almost a decade and nobody even reported
this bug, it should remain functional.
Added test case 1512 to verify
The previous naming scheme ftp_state_post_XXXX() wasn't really helpful
as it wasn't always immediately after 'xxxx' and it wasn't easy to
understand what it does based on such a name.
This new one is instead ftp_state_yyyy() where yyyy describes what it
does or sends.
Since the mk-ca-bundle tool itself isn't installed with make install,
there's no point in installing its documentation.
Bug: http://curl.haxx.se/mail/lib-2013-08/0057.html
Reported-by: Guenter Knauf
These are the files needed to build VMS distribution packages known as
PCSI kits.
Also minor update to the existing files, mainly to the documentation and
file clean up code.
This is the first version of this new document, detailing the seven
perhaps most important internal structs in libcurl source code:
1.1 SessionHandle
1.2 connectdata
1.3 Curl_multi
1.4 Curl_handler
1.5 conncache
1.6 Curl_share
1.7 CookieInfo
This is a regression as this logic used to work. It isn't clear when it
broke, but I'm assuming in 7.28.0 when we went all-multi internally.
This likely never worked with the multi interface. As the failed
connection is detected once the multi state has reached DO_MORE, the
Curl_do_more() function was now expanded somewhat so that the
ftp_do_more() function can request to go "back" to the previous state
when it makes another attempt - using PASV.
Added test case 1233 to verify this fix. It has the little issue that it
assumes no service is listening/accepting connections on port 1...
Reported-by: byte_bucket in the #curl IRC channel
For some reason, OS X 10.5's GCC suddenly stopped working correctly with
macros that change MD5_Init etc. in the code to CC_MD5_Init etc., so I
worked around this by removing use of the macros and inserting static
functions that just call CommonCrypto's implementations of the functions
instead.
This changes the previous check for untrusted certs to a check for
certs explicitely marked as trusted.
The change is backward-compatible (tested with certdata.txt v1.80).
The internal function that's used to detect known file extensions for
the default Content-Type got the the wrong pointer passed in when
CURLFORM_BUFFER + CURLFORM_BUFFERPTR were used. This had the effect that
strlen() would be used which could lead to an out-of-bounds read (and
thus segfault). In most cases it would only lead to it not finding or
using the correct default content-type.
It also showed that test 554 and test 587 were testing for the
previous/wrong behavior and now they're updated as well.
Bug: http://curl.haxx.se/bug/view.cgi?id=1262
Reported-by: Konstantin Isakov
Christian Heimes brought to our attention that the certdata.txt
format has recently changed [1], causing ca-bundle.crt created
with mk-ca-bundle.[pl|vbs] to include untrusted certs.
[1] http://lists.debian.org/debian-release/2012/11/msg00411.html
Instead of claiming it is an error, we call it a "note" to reduce the
severity level. But the following text now says the [variable] "*should*
only be used to specify"... instead of previously having said "may".
The motivation for having a separate struct that keep track of an easy
handle when using the multi handle was removed when we switched to
always using the multi interface internally. Now they were just two
separate struct that was always allocated for each easy handle.
This first step just moves the Curl_one_easy struct members into the
SessionHandle struct and hides this somehow (== keeps the source code
changes to a minimum) by defining Curl_one_easy to SessionHandle
The biggest changes in this commit are:
1 - the linked list of easy handles had to be changed somewhat due
to the new struct layout. This made the main linked list pointer
get renamed to 'easyp' and there's also a new pointer to the last
node, called easylp. It is no longer circular but ends with ->next
pointing to NULL. New nodes are still added last.
2 - easy->state is now called easy->mstate to avoid name collision
Update the packages/vms/readme file to be current.
Also some files for the GNV based build were either missing or needed an
update.
curl_crtl_init.c is a special file that is run before main() to
set up the proper C runtime behavior.
generate_vax_transfer.com generates the VAX transfer vector modules from
the gnv_libcurl_symbols.opt file.
gnv_conftest.c_first is a helper file needed for configure scripts to
come up with the expected answers on VMS.
gnv_libcurl_symbols.opt is the public symbols for the libcurl shared
image.
gnv_link_curl.com builds the shared libcurl image and rebuilds other
programs to use it.
macro32_exactcase.patch is a hack to make a local copy of the VMS Macro32
assembler case sensitive, which is needed to build the VAX transfer modules.
report_openssl_version.c is a tool for help verify that the libcurl
shared image is being built for a minium version of openssl.
Also, use memset() instead of a lame loop.
The previous logic that tried to avoid too many updates were very
ineffective for really fast transfers, as then it could easily end up
doing hundreds of updates per second that would make a significant
impact in transfer performance!
Bug: http://curl.haxx.se/mail/archive-2013-07/0031.html
Reported-by: Marc Doughty
(This doesn't need to appear in the release notes.) I noticed a few places
where infof() was called, and there should've been an LF at the end of the
string, but there wasn't.
It turns out Snow Leopard not only has SecItemCopyMatching() defined in
a header not included by the omnibus header, but it won't work for our
purposes, because searching for SecIdentityRef objects wasn't added
to that API until Lion. So we now use the old SecKeychainSearch API
instead if the user is building under, or running under, Snow Leopard.
Bug: http://sourceforge.net/p/curl/bugs/1255/
Reported by: Edward Rudd
Previously we used __MAC_10_X and __IPHONE_X to mark digest-generating
code that was specific to OS X and iOS. Now we use
__MAC_OS_X_VERSION_MAX_ALLOWED and __IPHONE_OS_VERSION_MAX_ALLOWED
instead of those macros.
Bug: http://sourceforge.net/p/curl/bugs/1255/
Reported by: Edward Rudd
For the standard VMS text file formats, VMS needs to read the file to
get the actual file size.
For the standard VMS binary file formats, VMS needs a special format of
fopen() call so that it stops reading at the logical end of file instead
of at the end of the blocks allocated to the file.
I structured the patch this way as I was not sure about changing the
structures or parameters to the routines, but would prefer to only call
the stat() function once and pass the information to where the fopen()
call is made.
Bug: https://sourceforge.net/p/curl/bugs/758/
The code for CURLFORM_FILECONTENT had its check for duplicate options
wrong so that it would reject CURLFORM_PTRNAME if used in combination
with it (but not CURLFORM_COPYNAME)! The flags field used for this
purpose cannot be interpreted that broadly.
Bug: http://curl.haxx.se/mail/lib-2013-07/0258.html
Reported-by: Byrial Jensen
Linking on Solaris 10 x86 with Sun Studio 12 failed when we upgraded
automake for the release builds.
Bug: http://curl.haxx.se/bug/view.cgi?id=1217
Reported-by: Dagobert Michelsen
Two fixes:
1. Force output file format to be stream-lf so that partial downloads
can be continued.
This should have minor impact as if the file does not exist, it was
created with stream-lf format. The only time this was an issue is if
there was already an existing file with a different format.
2. Fix file uploads are now fixed.
a. VMS binary files such as ZIP archives are now uploaded
correctly.
b. VMS text files are read once to get the correct size
and then converted to line-feed terminated records as
they are read into curl.
The default VMS text formats do not contain either line-feed or
carriage-return terminated records. Those delimiters are added by the
operating system file read calls if the application requests them.
Bug: http://curl.haxx.se/bug/view.cgi?id=496
We no longer pass our 'bool' data type variables nor constants as
an argument to my_setopt(), instead we use proper 1L or 0L values.
This also fixes macro used to pass string argument for CURLOPT_SSLCERT,
CURLOPT_SSLKEY and CURLOPT_EGDSOCKET using my_setopt_str() instead of
my_setopt().
This also casts enum or int argument data types to long when passed to
my_setopt_enum().
Commit 6d30f8ebed didn't work properly. First, it used the wrong
array index, but this fix also:
1 - only does the copying if indeed there was any activity
2 - makes sure to properly translate between internal and external
bitfields, which are not guaranteed to match
Reported-by: Evgeny Turnaev
Instead of going 50,100,150 etc millisecond delay time when nothing has
been found to do or wait for, we now start lower and double each loop as
in 4,8,16,32 etc.
This lowers the minimum wait without sacrifizing the longer wait too
much with unnecessary CPU cycles burnt.
Bug: http://curl.haxx.se/mail/lib-2013-07/0103.html
Reported-by: Andreas Malzahn
In the case of an active connection when ftp_do_more() detects that the
server has connected back, it must make sure to mark it as complete so
that the multi_runsingle() function will detect this and move on to the
next state.
Bug: http://curl.haxx.se/mail/lib-2013-07/0115.html
Reported-by: Clemens Gruber
CURLOPT_XFERINFOFUNCTION is now the preferred progress callback function
and CURLOPT_PROGRESSFUNCTION is considered deprecated.
This new callback uses pure 'curl_off_t' arguments to pass on full
resolution sizes. It otherwise retains the same characteristics: the
same call rate, the same meanings for the arguments and the return code
is used the same way.
The progressfunc.c example is updated to show how to use the new
callback for newer libcurls while supporting the older one if built with
an older libcurl or even built with a newer libcurl while running with
an older.
This reverts commit 7ed25cc, reinstating commit 8ec2cb5.
As of 18-jul-2013 we still do have code in libcurl that makes use of these
memory functions. Commit 8ec2cb5 comment still applies and is yet valid.
These memory functions are solely used in Windows builds, so all related
code is protected with '#ifdef WIN32' preprocessor conditional compilation
directives.
Specifically, wcsdup() _wcsdup() are used when building a Windows target with
UNICODE and USE_WINDOWS_SSPI preprocessor symbols defined. This is the case
when building a Windows UNICODE target with Windows native SSL/TLS support
enabled.
Realizing that wcsdup() _wcsdup() are used is a bit tricky given that usage
of these is hidden behind _tcsdup() which is MS way of dealing with code
that must tolerate UNICODE and non-UNICODE compilation. Additionally, MS
header files and those compatible from other compilers use this preprocessor
conditional compilation directive in order to select at compilation time
whether 'wide' or 'ansi' MS API functions are used.
Without this code, Windows build targets with Windows native SSL/TLS support
enabled and MemoryTracking support enabled misbehave in tracking memory usage,
regardless of being a UNICODE enabled build or not.
Fixed issue with static build for MSVC2010.
After some investigation I've discovered known issue
http://public.kitware.com/Bug/view.php?id=11240 When .rc file is linked
to static lib it fails with following linker error
LINK : warning LNK4068: /MACHINE not specified; defaulting to X86
file.obj : fatal error LNK1112: module machine type 'x64' conflicts with
target machine type 'X86'
Fix add target property /MACHINE: for MSVC generation.
Also removed old workarounds - it caused errors during msvc build.
Bug: http://curl.haxx.se/mail/lib-2013-07/0046.html
Implement wrappers around strtod to convert the user argument to a
double with sane error checking. Use this to allow --max-time and
--connect-timeout to accept decimal values instead of strictly integers.
The manpage is updated to make mention of this feature and,
additionally, forewarn that the actual timeout of the operation can
vary in its precision (particularly as the value increases in its
decimal precision).
strto* functions happily chomp off leading whitespace, so simply
checking for str[0] can lead to false negatives. Do the full parse and
check the out value instead.
Update the build_vms.com to detect and use zlib shared image installed
by the ZLIB kit produced by Jean-Francois Pieronne, and the also the
future ZLIB 1.2.8 kit in addition to the older ZLIB kits.
Also fix the indentation to match one of the common standards used for
VMS DCL command files and removed the hard tab characters.
Tested on OpenVMS 8.4 Alpha and IA64, and OpenVMS 7.3 VAX.
I just noticed that OS X no longer supports SSLv2. Other TLS engines return
an error if the requested protocol isn't supported by the underlying
engine, so we do that now for SSLv2 if the framework returns an error
when trying to turn on SSLv2 support. (Note: As always, SSLv2 support is
only enabled in curl when starting the app with the -2 argument; it's off
by default. SSLv2 is really old and insecure.)
When doing multi-part formposts, libcurl used a pseudo-random value that
was seeded with time(). This turns out to be bad for users who formpost
data that is provided with users who then can guess how the boundary
string will look like and then they can forge a different formpost part
and trick the receiver.
My advice to such implementors is (still even after this change) to not
rely on the boundary strings being cryptographically strong. Fix your
code and logic to not depend on them that much!
I moved the Curl_rand() function into the sslgen.c source file now to be
able to take advantage of the SSL library's random function if it
provides one. If not, try to use the RANDOM_FILE for seeding and as a
last resort keep the old logic, just modified to also add microseconds
which makes it harder to properly guess the exact seed.
The formboundary() function in formdata.c is now using 64 bit entropy
for the boundary and therefore the string of dashes was reduced by 4
letters and there are 16 hex digits following it. The total length is
thus still the same.
Bug: http://curl.haxx.se/bug/view.cgi?id=1251
Reported-by: "Floris"
When using %x, the number must be treated as unsigned as otherwise it
would get sign-extended on for example 64bit machines and do wrong
output. This problem showed when doing printf("%08x", 0xffeeddcc) on a
64bit host.
Follow-up fix from 7d80ed64e4.
The SessionHandle may not be around to use when we restore the sigpipe
sighandler so we store the no_signal boolean in the local struct to know
if/how to restore.
When the c-ares based resolver backend failed to resolve a name, it
tried to show the name that failed from existing structs. This caused
the wrong output and shown hostname when for example --interface
[hostname] was used and that name resolving failed.
Now we use the hostname used in the actual resolve attempt in the error
message as well.
Bug: http://curl.haxx.se/bug/view.cgi?id=1191
Reported-by: Kim Vandry
When we recently started to treat a zero return code from SSL_read() as
an error we also got false positives - which primarily looks to be
because the OpenSSL documentation is wrong and a zero return code is not
at all an error case in many situations.
Now ossl_recv() will check with ERR_get_error() to see if there is a
stored error and only then consider it to be a true error if SSL_read()
returned zero.
Bug: http://curl.haxx.se/bug/view.cgi?id=1249
Reported-by: Nach M. S.
Patch-by: Nach M. S.
Something (a recent security update maybe?) changed in Lion, and now it
has changed SSLCopyPeerTrust such that it may return noErr but also give
us a null trust, which caught us off guard and caused an eventual crash.
... and restore the ordinary handling again when it returns. This is
done for curl_easy_perform() and curl_easy_cleanup() only for now - and
only when built to use OpenSSL as backend as this is the known culprit
for the spurious SIGPIPEs people have received.
Bug: http://curl.haxx.se/bug/view.cgi?id=1180
Reported by: Lluís Batlle i Rossell
This doesn't need to be in the release notes. I cleaned up a lot of the #if
lines in the code to use MAC_OS_X_VERSION_MIN_REQUIRED and
MAC_OS_X_VERSION_MAX_ALLOWED instead of checking for whether things like
__MAC_10_6 or whatever were defined, because for some SDKs Apple has released
they were defined out of place.
RFC3986 details how a path part passed in as part of a URI should be
"cleaned" from dot sequences before getting used. The described
algorithm is now implemented in lib/dotdot.c with the accompanied test
case in test 1395.
Bug: http://curl.haxx.se/bug/view.cgi?id=1200
Reported-by: Alex Vinnik
Security problem: CVE-2013-2174
If a program would give a string like "%FF" to curl_easy_unescape() but
ask for it to decode only the first byte, it would still parse and
decode the full hex sequence. The function then not only read beyond the
allowed buffer but it would also deduct the *unsigned* counter variable
for how many more bytes there's left to read in the buffer by two,
making the counter wrap. Continuing this, the function would go on
reading beyond the buffer and soon writing beyond the allocated target
buffer...
Bug: http://curl.haxx.se/docs/adv_20130622.html
Reported-by: Timo Sirainen
As a remedy to the problem when a socket gets closed and a new one is
opened with the same file descriptor number and as a result
multi.c:singlesocket() doesn't detect the difference, the new function
Curl_multi_closed() gets told when a socket is closed so that it can be
removed from the socket hash. When the old one has been removed, a new
socket should be detected fine by the singlesocket() on next invoke.
Bug: http://curl.haxx.se/bug/view.cgi?id=1248
Reported-by: Erik Johansson
When performing COOKIELIST operations the cookie lock needs to be taken
for the cases where the cookies are shared among multiple handles!
Verified by Benjamin Gilbert's updated test 506
Bug: http://curl.haxx.se/bug/view.cgi?id=1215
Reported-by: Benjamin Gilbert
When curl_multi_wait() finds no file descriptor to wait for, it returns
instantly and this must be handled gracefully within curl_easy_perform()
or cause a busy-loop. Starting now, repeated fast returns without any
file descriptors is detected and a gradually increasing sleep will be
used (up to a max of 1000 milliseconds) before continuing the loop.
Bug: http://curl.haxx.se/bug/view.cgi?id=1238
Reported-by: Miguel Angel
The initial fix to only compare full path names were done in commit
04f52e9b4d but found out to be incomplete. This takes should make the
change more complete and there's now two additional tests to verify
(test 31 and 62).
By always returning the md5 for an empty body when auth-int is asked
for, libcurl now at least sometimes does the right thing.
Bug: http://curl.haxx.se/bug/view.cgi?id=1235
Patched-by: Nach M. S.
Allow less room for "triggered too early" mistakes by applications /
timers on non-windows platforms. Starting now, we assume that a timeout
call is never made earlier than 3 milliseconds before the actual
timeout. This greatly improves timeout accuracy on Linux.
Bug: http://curl.haxx.se/bug/view.cgi?id=1228
Reported-by: Hang Su
In the pkcs12 code, we get a list of x509 records returned from
PKCS12_parse but when iterating over the list and passing each to
SSL_CTX_add_extra_chain_cert() we didn't also properly remove them from
the "stack", which made them get freed twice (both in sk_X509_pop_free()
and then later in SSL_CTX_free).
This isn't really documented anywhere...
Bug: http://curl.haxx.se/bug/view.cgi?id=1236
Reported-by: Nikaiw
After curl_multi_wait() returns, this test checked that we got exactly
one file descriptor told to read from, but we cannot be sure that is
true. curl_multi_wait() will sometimes return earlier without any file
descriptor to handle, just just because it is a suitable time to call
*perform().
This problem showed up with commit 29bf0598.
Bug: http://curl.haxx.se/mail/lib-2013-06/0029.html
Reported-by: Fabian Keil
If an ipv6-address is provided to CONNECT, the last hexadecimal group in
the address will be used as the test number! For example the address
"[1234::ff]" would be treated as test case 255.
commit 29bf0598aa introduced a problem when the "internal" timeout is
prefered to the given if shorter, as it didn't consider the case where
-1 was returned. Now the internal timeout is only considered if not -1.
Reported-by: Tor Arntsen
Bug: http://curl.haxx.se/mail/lib-2013-06/0015.html
Also added a (correctly-escaped) backslash to the autoexec.bat
example file and a new Windows character device name with
a colon as examples of other characters that are special
and potentially dangerous (this reverts and reworks commit
7d8d2a54).
If the multi handle's pending timeout is less than what is passed into
this function, it will now opt to use the shorter time anyway since it
is a very good hint that the handle wants to process something in a
shorter time than what otherwise would happen.
curl_multi_wait.3 was updated accordingly to clarify
This is the reason for bug #1224
Bug: http://curl.haxx.se/bug/view.cgi?id=1224
Reported-by: Andrii Moiseiev
When sending the HTTP Authorization: header for digest, the user name
needs to be escaped if it contains a double-quote or backslash.
Test 1229 was added to verify
Reported and fixed by: Nach M. S
Bug: http://curl.haxx.se/bug/view.cgi?id=1230
We found that in specific cases if the connection is abruptly closed,
the underlying socket is listed in a close_wait state. We continue to
call the curl_multi_perform, curl_mutli_fdset etc. None of these APIs
report the socket closed / connection finished. Since we have cases
where the multi connection is only used once, this can pose a problem
for us. I've read that if another connection was to come in, curl would
see the socket as bad and attempt to close it at that time -
unfortunately, this does not work for us.
I found that in specific situations, if SSL_write returns 0, curl did
not recognize the socket as closed (or errored out) and did not report
it to the application. I believe we need to change the code slightly, to
check if ssl_write returns 0. If so, treat it as an error - the same as
a negative return code.
For OpenSSL - the ssl_write documentation is here:
http://www.openssl.org/docs/ssl/SSL_write.html
1 - don't skip host names with a colon in them in an attempt to bail out
on HTTP headers in the cookie file parser. It was only a shortcut anyway
and trying to parse a file with HTTP headers will still be handled, only
slightly slower.
2 - don't skip domain names based on number of dots. The original
netscape cookie spec had this oddity mentioned and while our code
decreased the check to only check for two, the existing cookie spec has
no such dot counting required.
Bug: http://curl.haxx.se/bug/view.cgi?id=1221
Reported-by: Stefan Neis
I found a bug which cURL sends cookies to the path not to aim at.
For example:
- cURL sends a request to http://example.fake/hoge/
- server returns cookie which with path=/hoge;
the point is there is NOT the '/' end of path string.
- cURL sends a request to http://example.fake/hogege/ with the cookie.
The reason for this old "feature" is because that behavior is what is
described in the original netscape cookie spec:
http://curl.haxx.se/rfc/cookie_spec.html
The current cookie spec (RFC6265) clarifies the situation:
http://tools.ietf.org/html/rfc6265#section-5.2.4
This reverts commit 8ec2cb5544.
We don't have any code anywhere in libcurl (or the curl tool) that use
wcsdup so there's no such memory use to track. It seems to cause mild
problems with the Borland compiler though that we may avoid by reverting
this change again.
Bug: http://curl.haxx.se/mail/lib-2013-05/0070.html
If the mail sent during the transfer contains a terminating <CRLF> then
we should not send the first <CRLF> of the EOB as specified in RFC-5321.
Additionally don't send the <CRLF> if there is "no mail data" as the
DATA command already includes it.
The code within #ifdef HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID wrongly had two
closing braces when it should only have one, so builds without that
define would fail.
Bug: http://curl.haxx.se/mail/lib-2013-05/0000.html
The curl command line utility would display the the completed progress
bar with a percentage of zero as the progress routines didn't know the
size of the transfer.
The emails that are sent to the server during these tests were
incorrectly formatted as they contained one or more LF terminated lines
rather than being CRLF terminated as per Section 2.3.7 of RFC-2821.
This wasn't a problem for the test suite as the <stdin> data matched the
<upload> data but anyone using these tests as reference would be sending
incorrect data to a server.
Removed the hard returns from imap and pop3 by using the same style for
sending the authentication string as smtp. Moved the "Other mechanisms
not supported" check in smtp to match that of imap and pop3 to provide
consistency between the three email protocols.
Users using the Secure Transport (darwinssl) back-end can now use a
certificate and private key to authenticate with a site using TLS. Because
Apple's security system is based around the keychain and does not have any
non-public function to create a SecIdentityRef data structure from data
loaded outside of the Keychain, the certificate and private key have to be
loaded into the Keychain first (using the certtool command line tool or
the Security framework's C API) before we can find it and use it.
Updated test903 and test904 following the addition of CURLOPT_SASL_IR
as the default behaviour of SMTP AUTH responses is now to not include
the initial response. New tests with --sasl-ir support to follow.
In addition to checking for the SASL-IR capability the user can override
the sending of the client's initial response in the AUTHENTICATION
command with the use of CURLOPT_SASL_IR should the server erroneously
not report SASL-IR when it does support it.
Updated the default behaviour of sending the client's initial response in the AUTH
command to not send it and added support for CURLOPT_SASL_IR to allow the user to
specify including the response.
Related Bug: http://curl.haxx.se/mail/lib-2012-03/0114.html
Reported-by: Gokhan Sengun
By introducing an internal alternative to curl_multi_init() that accepts
parameters to set the hash sizes, easy handles will now use tiny socket
and connection hash tables since it will only ever add a single easy
handle to that multi handle.
This decreased the number mallocs in test 40 (which is a rather simple
and typical easy interface use case) from 1142 to 138. The maximum
amount of memory allocated used went down from 118969 to 78805.
When connecting back to an FTP server after having sent PASV/EPSV,
libcurl sometimes didn't use the proxy properly even though the proxy
was used for the initial connect.
The function wrongly checked for the CURLOPT_PROXY variable to be set,
which made it act wrongly if the proxy information was set with an
environment variable.
Added test case 711 to verify (based on 707 which uses --socks5). Also
added test712 to verify another variation of setting the proxy: with
--proxy socks5://
Bug: http://curl.haxx.se/bug/view.cgi?id=1218
Reported-by: Zekun Ni
... in order to prevent an artificial timeout event based on stale
speed-check data from a previous network transfer. This commit fixes
a regression caused by 9dd85bced5.
Bug: https://bugzilla.redhat.com/906031
I couldn't figure out why the host key logic isn't working, but having
it set to yes prevents my SSH-based test cases to run. I also don't see
a strong need to use strict host key checking on this test server.
So I disabled it.
...when mentioning login options. Additional minor clarification of
"Windows builds" to be "Windows builds with SSPI"as a way of enabling
NTLM as Windows builds may be built with OpenSSL to enable NTLM or
without NTLM support altogether.
Fixed an issue in parse_proxy(), introduced in commit 11332577b3,
where an empty username or password (For example: http://:@example.com)
would cause a crash.
There is no need to perform separate clearing of data if a NULL option
pointer is passed in. Instead this operation can be performed by simply
not calling parse_login_details() and letting the rest of the code do
the work.
setstropt_userpwd() was calling setstropt() in commit fddb7b44a7 to
set each of the login details which would duplicate the strings and
subsequently cause a memory leak.
Fix to prevent the options from being displayed when curl requests the
user's password if the following command line is specified:
--user username;options
In addition to parsing the optional login options from the URL, added
support for parsing them from CURLOPT_USERPWD, to allow the following
supported command line:
--user username:password;options
Added bounds checking when searching for the separator characters within
the login string as this string may not be NULL terminated (For example
it is the login part of a URL). We do this in preference to allocating a
new string to copy the login details into which could then be passed to
parse_login_details() for performance reasons.
Apparently the previous usage didn't work with that implementation,
while this updated version works with at least both Parallel BZIP2
v1.1.8 and regular bzip "Version 1.0.6, 6-Sept-2010".
As well as parsing the username and password from the URL, added support
for parsing the optional options part from the login details, to allow
the following supported URL format:
schema://username:password;options@example.com/path?q=foobar
This will only be used by IMAP, POP3 and SMTP at present but any
protocol that may be given login options in the URL will be able to
add support for them.
...instead of the 220 we otherwise expect.
Made the ftpserver.pl support sending a custom "welcome" and then
created test 1219 to verify this fix with such a 230 welcome.
Bug: http://curl.haxx.se/mail/lib-2013-02/0102.html
Reported by: Anders Havn
For libc variants without a spearate pthread lib (like bionic), try
using pthreads without the pthreads lib first and only if that fails try
the -lpthread linker flag.
Bug: http://curl.haxx.se/bug/view.cgi?id=1216
Reported by: Duncan
Accessing a file with an absolute path in the root dir but with no
directory specified was not handled correctly. This fix comes with four
new test cases that verify it.
Bug: http://curl.haxx.se/mail/lib-2013-04/0142.html
Reported by: Sam Deane
... and make 1216 also verify it with a file input
These tests verify commit 3604fde3d3c9b0d, the fix for the "cookie
domain tailmatch" vulnerability. See
http://curl.haxx.se/docs/adv_20130412.html
Cookies set for 'example.com' could accidentaly also be sent by libcurl
to the 'bexample.com' (ie with a prefix to the first domain name).
This is a security vulnerabilty, CVE-2013-1944.
Bug: http://curl.haxx.se/docs/adv_20130412.html
The previously applied patch didnt work on Windows; we cant rely
on shell commands like 'echo' since they act diffently on each
platform and each shell.
In order to keep this script platform-independent the code must
only use pure Perl.
When doing PWD, there's a 257 response which apparently some servers
prefix with a comment before the path instead of after it as is
otherwise the norm.
Failing to parse this, several otherwise legitimate use cases break.
Bug: http://curl.haxx.se/mail/lib-2013-04/0113.html
The OpenSSL pipe wrote to the final CA bundle file, but the encoded PEM
output wrote to a temporary file. Consequently, the OpenSSL output was
lost when the temp file was renamed to the final file at script finish
(overwriting the final file written earlier by openssl).
Patch posted to the list by Richard Michael (rmichael edgeofthenet org).
I noticed that aria2's SecureTransport code disables insecure ciphers such
as NULL, anonymous, IDEA, and weak-key ciphers used by SSLv3 and later.
That's a good idea, and now we do the same thing in order to prevent curl
from accessing a "secure" site that only negotiates insecure ciphersuites.
1 - We don't use the results from the test and we never did. recvfrom()
is only used by the TFTP code and it has not caused any problems.
2 - the CURL_CHECK_FUNC_RECVFROM function is extremely slow
Previously it only compared credentials if the requested needle
connection wasn't using a proxy. This caused NTLM authentication
failures when using proxies as the authentication code wasn't send on
the connection where the challenge arrived.
Added test 1215 to verify: NTLM server authentication through a proxy
(This is a modified copy of test 67)
Since qsort implementations vary with regards to handling the order
of similiar elements, this change makes the internal sort function
more deterministic by comparing path length first, then domain length
and finally the cookie name. Spotted with testcase 62 on Windows.
WINSOCK only:
Since FD_CLOSE is only signaled once, it may trigger at the same
time as FD_READ. Data actually being available makes it impossible
to detect that the connection was closed by checking that recv returns
zero. Another recv attempt could block the connection if it was
not closed. This workaround abuses exceptfds in conjunction with
readfds to signal that the connection has actually closed.
Since commit 57aeabcc1a, it handles errors on the control connection
while waiting for the data connection better.
Test 591 and 592 are updated accordingly.
When doing PORT and upload (STOR), this function needs to extract the
file descriptor for both connections so that it will respond immediately
when the server eventually connects back.
This flaw caused active connections to become unnecessary slow but they
would still often work due to the normal polling on a timeout. The bug
also would not occur if the server connected back very fast, like when
testing on local networks.
Bug: http://curl.haxx.se/bug/view.cgi?id=1183
Reported by: Daniel Theron
I am using curl_easy_setopt(CURLOPT_INTERFACE, "if!something") to force
transfers to use a particular interface but the transfer fails with
CURLE_INTERFACE_FAILED, "Failed binding local connection end" if the
interface I specify has no IPv6 address. The cause is as follows:
The remote hostname resolves successfully and has an IPv6 address and an
IPv4 address.
cURL attempts to connect to the IPv6 address first.
bindlocal (in lib/connect.c) fails because Curl_if2ip cannot find an
IPv6 address on the interface.
This is a fatal error in singleipconnect()
This change will make cURL try the next IP address in the list.
Also included are two changes related to IPv6 address scope:
- Filter the choice of address in Curl_if2ip to only consider addresses
with the same scope ID as the connection address (mismatched scope for
local and remote address does not result in a working connection).
- bindlocal was ignoring the scope ID of addresses returned by
Curl_if2ip . Now it uses them.
Bug: http://curl.haxx.se/bug/view.cgi?id=1189
This workaround fixes an issue on MinGW/Msys regarding the Perl
testsuite scripts not being able to signal or control the server
processes. The MinGW Perl runtime only sees the Msys processes and
their corresponding PIDs, but sockfilt (and other servers) wrote the
Windows PID into their PID-files. Since this PID is useless to the
testsuite, the write_pidfile function was changed to search for the
Msys PID and write that into the PID-file.
At some point recently we lost the default value for the easy handle's
connection cache, and this change puts it back to 5 - which is the
former default value and it is documented in the curl_easy_setopt.3 man
page.
The new read and write wrapper functions support reading from stdin
and writing to stdout/stderr on Windows by using the appropriate
Windows API functions and data types.
The Microsoft knowledge-base article
http://support.microsoft.com/kb/823764 describes how to use SNDBUF to
overcome a performance shortcoming in winsock, but it doesn't apply to
Windows Vista and later versions. If the described SNDBUF magic is
applied when running on those more recent Windows versions, it seems to
instead have the reversed effect in many cases and thus make libcurl
perform less good on those systems.
This fix thus adds a run-time version-check that does the SNDBUF magic
conditionally depending if it is deemed necessary or not.
Bug: http://curl.haxx.se/bug/view.cgi?id=1188
Reported by: Andrew Kurushin
Tested by: Christian Hägele
The last remaining code piece that still used FTPSENDF now uses PPSENDF.
In the problematic case, a PREQUOTE series was done on a re-used
connection when Curl_pp_init() hadn't been called so it had messed up
pointers. The init call is done properly from Curl_pp_sendf() so this
change fixes this particular crash.
Bug: http://curl.haxx.se/mail/lib-2013-03/0319.html
Reported by: Sam Deane
Modified sws to support and use custom CONNECT responses instead of the
previously naive hard-coded version. Made the HTTP test server able to
extract test case number from the host name in a CONNECT request by
finding the number after the last dot. It makes 'machine.moo.123' use
test case 123.
Adapted a larger amount of tests to the new <connect> style.
Bug: http://curl.haxx.se/bug/view.cgi?id=1204
Reported by: Martin Jansen
As of 25-mar-2013 wcsdup() _wcsdup() and _tcsdup() are only used in
WIN32 specific code, so tracking of these has not been extended for
other build targets. Without this fix, memory tracking system on
WIN32 builds, when using these functions, would provide misleading
results.
In order to properly extend this support for all targets curl.h
would have to define curl_wcsdup_callback prototype and consequently
wchar_t should be visible before that in curl.h. IOW curl_wchar_t
defined in curlbuild.h and this pulling whatever system header is
required to get wchar_t definition.
Additionally a new curl_global_init_mem() function that also receives
user defined wcsdup() callback would be required.
Proxy servers tend to add their own headers at the beginning of
responses. The size of these headers was not taken into account by
CURLINFO_HEADER_SIZE before this change.
Bug: http://curl.haxx.se/bug/view.cgi?id=1204
This patch removes building curlbuild.h from the build_vms.com procedure
and uses the one in the daily or release tarball instead.
packages/vms/build_curlbuild_h.com is obsolete with this change.
Accessing the library module name "tool_main" needs different handling
when the optional extended parsing is enabled.
Tested on IA64/VMS 8.4 and VAX/VMS 7.3
After having done a POST over a CONNECT request, the 'rewindaftersend'
boolean could be holding the previous value which could lead to badness.
This should be tested for in a new test case!
Bug: https://groups.google.com/d/msg/msysgit/B31LNftR4BI/KhRTz0iuGmUJ
Moved SMTP, POP3, IMAP and New Protocol sections to be listed after the
other protocols (FTP, HTTP and TELNET) and SASL to be after SSL and
GnuTLS as these are all security related.
Additionally fixed numbering of the SSL and GnuTLS sections as they
weren't consecutive.
Fixed incorrect initial response generation for the NTLM and LOGIN SASL
authentication mechanisms when the SASL-IR was detected.
Introduced in commit: 6da7dc026c.
These verfy that the 'memory tracking' subsystem is actually doing its
job when using curl tool (#96), a test in libtest (#558) and also a unit
test (#1330), in order to prevent regressions in this functionallity.
curl has been accepting URLs using slightly wrong syntax for a long
time, such as when completely missing as slash "http://example.org" or
missing a slash when a query part is given
"http://example.org?q=foobar".
curl would translate these into a legitimate HTTP request to servers,
although as was shown in bug #1206 it was not adjusted properly in the
cases where a HTTP proxy was used.
Test 1213 and 1214 were added to the test suite to verify this fix.
The test HTTP server was adjusted to allow us to specify test number in
the host name only without using any slashes in a given URL.
Bug: http://curl.haxx.se/bug/view.cgi?id=1206
Reported by: ScottJi
Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.
Two new options for limiting the number of connections:
CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.
CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.
Several new options for pipelining:
CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).
CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.
CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.
CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.
CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.
See the curl_multi_setopt() man page for details.
Following commit e450f66a02 and the changes in the multi interface
being used internally, from 7.29.0, the transfer cancellation in
pop3_dophase_done() is no longer required.
Added missing slash in cc_full_list.
Removed unwanted extra quotes inside symbol tool_main
for non-VAX architectures that triggered link failure.
Replaced curl_sys_inc with sys_inc.
An extern submits a psect and a global reference to the linker to point
to it. Using "extern int vms_show = 0" also creates a globaldef.
The use of the extern by itself does declare a psect but does not declare
a globalsymbol. It does declare a globalref. But the linker needs one and
only one globaldef or there is an error.
Any IMAP parameter can come in escaped and in double quotes. Added a
simple function to unquote the command parameters and applied it to
the IMAP command handlers.
The majority of lines, that specify a test file for inclusion, end with
a tab character before the slash whilst some end with a space. Corrected
those that end with a space to end with a tab character as well.
Finished segregating the email protocol tests, into their own protocol
based ranges, in preparation of adding more e-mail related tests to the
test suite.
Continued segregating the email protocol tests, into their own protocol
based ranges, in preparation of adding more e-mail related tests to the
test suite.
Continued segregating the email protocol tests, into their own protocol
based ranges, in preparation of adding more e-mail related tests to the
test suite.
Started segregating the email protocol tests, into their own protocol
based ranges, in preparation of adding more e-mail related tests to the
test suite.
When Curl_do() returns failure, the connection pointer could be NULL so
the code path following needs to that that into account.
Bug: http://curl.haxx.se/mail/lib-2013-03/0062.html
Reported by: Eric Hu
No more use exit(3) but instead tell libcurl that no byte has been
written to let it return a `CURLE_WRITE_ERROR`. In addition, check
curl easy handle return code.
Moved the blocking state machine to the disconnect functions so that the
logout / quit functions are only responsible for sending the actual
command needed to logout or quit.
Additionally removed the hard return on failure.
Added an exception, for the STORE command, to the untagged response
processor in imap_endofresp() as servers will back respones containing
the FETCH keyword instead.
GCC specific adjustments:
- check __ILP32__ before 32 and 64bit processor architectures in
order to detect ILP32 programming model on 64 bit processors
which, of course, also support LP64 programming model, when using
gcc 4.7 or newer.
- keep 32bit processor architecture checks in order to support gcc
versions older than 4.7 which don't define __ILP32__
- check __LP64__ for gcc 3.3 and newer, while keeping 64bit processor
architecture checks for older versions which don't define __LP64__
The list of unsafe functions currently consists of sprintf, vsprintf,
strcat, strncat and gets.
Subsequently, some existing code needed updating to avoid warnings on
this.
Enriched IMAP capabilities of ftpserver.pl in order to be able to
add tests for the new IMAP features.
* Added support for APPEND - Saves uploaded data to log/upload.$testno
* Added support for LIST - Returns the contents of <reply/> section in
the current test, like e.g FETCH.
* Added support for STORE - Returns hardcoded updated flags
* Changed handling of SELECT - Returns much more information in the
usual set of untagged responses; uses hardcoded data from an example
in the IMAP RFC
* Changed handling of FETCH - Fixed response format
As the UID has to be specified by the user for the FETCH command to work
correctly, added a check to imap_fetch(), although strictly speaking it
is protected by the call from imap_perform().
The option needs to be set on the SSL socket. Setting it on the model
takes no effect. Note that the non-blocking mode is still not enabled
for the handshake because the code is not yet ready for that.
Commit 26eaa83830 introduces the use of S_ISDIR() yet some compilers,
such as MSVC don't support it, so we must define a substitute using
file flags and mask.
Commit f4cc54cb47 (shipped as part of the 7.29.0 release) was a
bug fix that introduced a regression in that while trying to avoid
allowing directory names, it also forbade "special" files like character
devices and more. like "/dev/null" as was used by Oliver who reported
this regression.
Reported by: Oliver Gondža
Bug: http://curl.haxx.se/mail/archive-2013-02/0040.html
If the server hung up the connection without sending a closure alert,
then we'd keep probing the socket for data even though it's dead. Now
we're ready for this situation.
Bug: http://curl.haxx.se/mail/lib-2013-03/0014.html
Reported by: Aki Koskinen
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Added imap_custom(), which initiates the custom command processing,
and an associated response handler imap_state_custom_resp(), which
handles any responses by sending them to the client as body data.
All untagged responses with the same name as the first word of the
custom request string are accepted, with the exception of SELECT and
EXAMINE which have responses that cannot be easily identified. An
extra check has been provided for them so that any untagged responses
are accepted for them.
Added imap_parse_custom_request() for parsing the CURLOPT_CUSTOMREQUEST
parameter which URL decodes the value and separates the request from
any parameters - This makes it easier to filter untagged responses
by the request command.
For consistency changed the logic of the imap_state_append_resp()
function to test for an unsucessful continuation response rather than a
succesful one.
The APPEND operation needs to be performed in several steps:
1) We send "<tag> APPEND <mailbox> <flags> {<size>}\r\n"
2) Server responds with continuation respose "+ ...\r\n"
3) We start the transfer and send <size> bytes of data
4) Only now we end the request command line by sending "\r\n"
5) Server responds with "<tag> OK ...\r\n"
This commit performs steps 4 and 5, in the DONE phase, as more
processing is required after the transfer.
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Not processing the final FETCH responses was not optimal, not only
because the response code would be ignored but it would also leave data
unread on the socket which would prohibit connection reuse.
A typical FETCH response can be broken down into four parts:
1) "* <uid> FETCH (<what> {<size>}\r\n", using continuation syntax
2) <size> bytes of the actual message
3) ")\r\n", finishing the untagged response
4) "<tag> OK ...", finishing the command
Part 1 is read in imap_fetch_resp(), part 2 is consumed in the PERFORM
phase by the transfer subsystem, parts 3 and 4 are currently ignored.
Added a loop to imap_statemach_act() in which Curl_pp_readresp() is
called until the cache is drained. Without this multiple responses
received in a single packet could result in a hang or delay.
RFC 3501 states that "the client MUST be prepared to accept any response
at all times" yet we assume anything received with "* " at the beginning
is the untagged response we want.
Introduced a helper function that checks whether the input looks like a
response to specified command, so that we may filter the ones we are
interested in according to the current state.
Introduced similar handling to the FETCH responses, where even the
untagged data responses are handled by the response handler of the
individual state.
Removed this pointer to a downloaded bytes counter because it was set in
smtp_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
Removed this pointer to a downloaded bytes counter because it was set in
pop3_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
Removed this pointer to a downloaded bytes counter because it was set in
imap_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
From a maintenance point of view the code reads better to view tagged
responses, then untagged followed by continuation responses.
Additionally, this matches the order of responses in POP3.
Updated the POP3 sub-section to refer to message ID rather than mailbox.
Added an IMAP sub-section with example URLs depicting the specification
of mailbox, uid and section.
Updated the mailbox variable to correctly reflect it's purpose. The
name mailbox was a leftover from when IMAP and POP3 support was
initially added to curl.
Updated the FETCH command to send the UID and SECTION parsed from the
URL. By default the BODY specifier doesn't include a section, BODY[] is
now sent whereas BODY[TEXT] was previously sent. In my opinion
retrieving just the message text is rarely useful when dealing with
emails, as the headers are required for example, so that functionality
is not retained. In can however be simulated by adding SECTION=TEXT to
the URL.
Also updated test801 and test1321 due to the BODY change.
Removed user and passwd from the SMTP struct as these cannot be set on
a per-request basis and are leftover from legacy FTP code.
Changed some comments still using FTP terminology.
Removed user and passwd from the POP3 struct as these cannot be set on
a per-request basis and are leftover from legacy FTP code.
Changed some comments still using FTP terminology.
Moved the mailbox and custom request variables from the per-connection
struct pop3_conn to the new per-request struct and fixed references
accordingly.
Created a new IMAP structure and changed the type of the imap proto
variable in connectdata from FTP* to the new IMAP*.
Moved the mailbox variable from the per-connection struct imap_conn to
the new per-request struct and fixed references accordingly.
Moved the clean-up of the mailbox variable from imap_disconnect() to
imap_done() as this variable is allocated in the do phase, yet would
have only been freed only once if multiple selects where preformed
on a single connection.
* Elaborates on default values of some curl_easy_setopt() options.
* Reminds the user to cast variadic arguments to curl_easy_setopt() to
'void *' where curl internally interprets them as such.
* Clarifies the working of the CURLOPT_SEEKFUNCTION option for
curl_easy_setopt().
* Fixes typo 'forth' → 'fourth'.
* Elaborates on CURL_SOCKET_TIMEOUT.
* Adds some missing periods.
* Notes that the return value of curl_version() must not be passed to
free().
Always interprets the pointer passed with the CURLOPT_WRITEDATA or
CURLOPT_READDATA options of curl_easy_setopt() as a void pointer in
order to avoid problems in environments where FILE and void pointers
have non-trivial conversion.
* Adds several links to documentation of library functions which were
missing.
* Marks documentation of deprecated library functions "(deprecated)".
* Removes spurious .html suffixes.
Use Curl_pp_moredata() in Curl_pp_multi_statemach() to check if there is
more data to be received, rather than the socket state, as a task could
hang waiting for more data from the socket itself.
A simple function to test whether the PP is not sending and there are
still more data in its receiver cache. This will be later utilized to:
1) Change Curl_pp_multi_statemach() and Curl_pp_easy_statemach() to
not test socket state and just call user's statemach_act() function
when there are more data to process, because otherwise the task would
just hang, waiting for more data from the socket.
2) Allow PP users to read multiple responses by looping as long as there
are more data available and current phase is not finished.
(Currently needed for correct processing of IMAP SELECT responses.)
The this_url pointer wasn't being initialized, so if strdup() would return
null when copying the filename in a metalink file, then hilarity would
ensue during the cleanup phase. This change was brought to you by clang,
which noticed this and raised a warning.
Schannel and darwinssl use the certificates built into the
OS to do vert verification instead of bundles. darwinssl
is thread-safe. Corrected typos in the NSS docs.
The attempt to use gai_strerror() or alternative function didn't work as
the 'sock_error' field didn't contain the proper error code. But since
this hasn't been reported and thus isn't really a big deal I decided to
just scrap the whole attempt to output the detailed resolver error and
instead remain with just stating that the resolving of the name failed.
It seems older gcc installations (at least) will cause warnings if we
name a variable 'wait'. Now changed to 'block' instead.
Reported by: Jiří Hruška
Bug: http://curl.haxx.se/mail/lib-2013-02/0247.html
Apple made a number of changes to Xcode 4. The SDKs were moved, the entire
Developer folder was moved, and PowerPC support was removed. The script
will now adapt to those changes and should be future-proofed against
additional changes in case Apple moves the Developer folder ever again.
Also, the minimum OS X version compiler option was removed, so that the
framework can be built against the latest SDK but still run in older cats.
... since they're not used by the easy interface really, I wanted to
remove the association. Also, I unified the pingpong statemachine driver
into a single function with a 'wait' argument: Curl_pp_statemach.
A call to Curl_ssl_connect() was accidentally left in when the SSL/TLS
connection layer was reworked in 7.29. Not only would this cause the
connection to block but had the additional overhead of calling the
non-blocking connect a little bit later.
This function was only used twice, both in places where performance
isn't crucial (socks + if2ip). Removing the use of this function removes
the need to have our private version for systems without it == reduced
amount of code.
Also, in the SOCKS case it is clearly better to fail gracefully rather
than to truncate the results.
This work was triggered by a bug report on the strcal prototype in
strequal.h.
strlcat was added in commit db70cd28 in February 2001!
Bug: http://curl.haxx.se/bug/view.cgi?id=1192
Reported by: Jeremy Huddleston
Version 7.29.0 uses Makefiles generated with a newer version of the
autotools than the previous 7.28.1. These Makefiles try to install
e.g. header files by calling install-sh with multiple source files as
arguments. The bundled install-sh is to old and does not support this.
The problem only occurs, if install-sh is actually being used, ie. the
platform install executable is to old or not usable. Example: Solaris
10.
The files install-sh and mkinstalldirs are now updated with the automake
1.11.3 versions. A better fix might be to completely remove them from
git and force the files to be added/created during buildconf.
Bug: http://curl.haxx.se/bug/view.cgi?id=1195
Reported by: Rainer Jung
Fixed a null pointer reference when an empty challenge is passed to the
Curl_sasl_create_digest_md5_message() function.
Bug: http://sourceforge.net/p/curl/bugs/1193/
Reported by: Saran Neti
An ambiguity in the SSLWrite() documentation lead to a bad inference in the
code where we assumed SSLWrite() returned the amount of bytes written to
the socket, when that is not actually true; it returns the amount of data
that is buffered for writing to the socket if it returns errSSLWouldBlock.
Now darwinssl_send() returns CURLE_AGAIN if data is buffered but not written.
Reference URL: http://curl.haxx.se/mail/lib-2013-02/0145.html
Reworked the pp->endofresp() function so that the conndata, line and
line length are passed down to it just as with Curl_client_write()
rather than each implementation of the function having to query
these values.
Additionally changed the int return type to bool as this is more
representative of the function's usage.
Corrected the order of the upgrade_tls() functions and moved the handler
upgrade and getsock() functions out from the middle of the state related
functions.
Corrected the order of the pop3_state_capa() / imap_state_capability()
and the pop3_state_capa_resp() / imap_state_capability_resp() functions
to match the execution order.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added sending of initial CAPABILITY command before STARTTLS is sent.
This allows for the detection of the capability before trying to
upgrade the connection.
Follow up fix to commit 62bd217464 to cater for servers that don't
respond with a 250 in their EHLO responses. Additionally updated the
SMTP tests to respond with a 250 response code as per RFC5321.
Fixed the SASL capability detection to include the space character
before the authentication mechanism list. Otherwise a capability such
as SASLSOMETHING would be interpreted as enabling SASL and potentially
trying to identify SOMETHING as a mechanism.
Previously if a problem was found with one of the server's certificates,
we'd log an OSStatus for the end user to look up. Now we explain what
was wrong with the site's certificate chain. Also un-did part of the
previous commit where the code wouldn't catch errSSLServerAuthCompleted
if built under Leopard.
Fixed a small issue where smtp_endofresp() would look for capabilities
in the description part of a failure response. In theory a server
shouldn't respond with SIZE or AUTH in an EHLO command's failure
response but if it did then capabilities would be unnecessarily set
before eventually failing.
Renamed the authstate1 and authstate2 variables in imap_authenticate()
as the old name was a left over from when there was only one state
variable which was named due to a clash with the state() function.
Additionally this provides consistency with the smtp module.
Running tests\libtest\libntlmconnect.exe reveals a 1 byte (!) leak in
./lib/curl_ntlm_msgs.c:
perl ..\memanalyze.pl c:memdebug.curl
Leak detected: memory still allocated: 1 bytes
At 9771e8, there's 1 bytes.
allocated by curl_ntlm_msgs.c:399
Snippet from curl_ntlm_msgs.c:
/* setup ntlm identity's domain and length */
dup_domain.tchar_ptr = malloc(sizeof(TCHAR) * (domlen + 1));
(my domlen == 0).
'dup_domain.tbyte_ptr' looks to be freed in Curl_ntlm_sspi_cleanup() via
'ntlm->identity.Domain'. But I see no freeing of 'dup_domain.tchar_ptr'.
This bug report properly identified that when doing SMTP and aborting
the transfer with a callback, it must be considered aborted prematurely
by the code to avoid QUIT etc to be attempted as that would cause a
hang.
The new test case 1507 verifies this behavior.
Reported by: Patricia Muscalu
Bug: http://curl.haxx.se/bug/view.cgi?id=1184
It turns out that Leopard (OS X 10.5) doesn't have constants for the ECDH
ciphers in its headers, so the cases for them have been taken out of the
build when building under Leopard. Also added a standard function for
getting a string description of a SecCertificateRef.
Changed the SMTP_AUTH_PASSWD state constant to SMTP_AUTH_LOGIN_PASSWD to
better describe the state as the second part of an AUTH LOGIN command,
as well as for consistency with the imap and pop3 modules.
Introduced detection of the SASL-IR capability, in order to add support
for sending the initial response with the AUTHENTICATE command, as per
RFC4959.
Updated the automatic response tag generation to follow the examples
given in RC3501, which list a 4 character string such as A001, A002,
etc.
As a unique identifier should be generated for each command the string
generation is based on the connection id and the incrementing command
id.
This is untested, but ought to be enough to still allow it
to work automatically when the entire curl source tree is
dropped into a full Android source tree.
VC6 is _very_ old and we provide working makefiles even for that
compiler. Users who build with the IDE never use that method and project
file anyway and it was just lingering in the root dir.
Added IDN and HTTP data compression as they were left out of the
document until now.
Added notes for qssl, schannel and Secure Transport supporting SSLv2,
Secure Transport supports NTLM, and axTLS does not support SSLv3.
There was also a typo; "AUTH TSL" should be "AUTH TLS".
When negotiating SASL DIGEST-MD5 authentication, the function
Curl_sasl_create_digest_md5_message() uses the data provided from the
server without doing the proper length checks and that data is then
appended to a local fixed-size buffer on the stack.
This vulnerability can be exploited by someone who is in control of a
server that a libcurl based program is accessing with POP3, SMTP or
IMAP. For applications that accept user provided URLs, it is also
thinkable that a malicious user would feed an application with a URL to
a server hosting code targetting this flaw.
Bug: http://curl.haxx.se/docs/adv_20130206.html
config_h.com is a new file that generates a config.h file based on the
curl_config.h.in file and a quick scan of the configure script. This is
actually a generic procedure that is shared with other VMS packages.
The existing pre-built config-vms.h had over 100 entries that were not
correct and in some cases conflicted with the build options available in
the build_vms.com.
generate_config_vms_h_curl.com is a helper procedure to the
config_h.com. It covers the cases that the generic config_h.com is not
able to figure out, and accepts input from the build_vms.com procedure.
build_curlbuild_h.com is a new file to generate the curlbuild.h file
that Curl is now using when it is using a curl_config.h file.
post-config-vms.h is a new file that is needed to provide VMS specific
definitions, and most of them need to be set before the system header
files are included.
The VMS build procedure is fixed:
1. Fixed to link in the correct HP ssl library.
2. Fixed to detect if HP Kerberos is installed.
3. Fixed to detect if HP LDAP is installed.
4. Fixed to detect if gnv$libzshr is installed.
5. Simplified the input parameter parsing to not use a loop.
6. Warn that 64 bit pointer option support is not complete
in comments.
7. Default to IEEE floating if platform supports it so
resulting libcurl will be compatible with other
open source projects on VMS.
8. Default to LARGEFILE if platform supports it.
9. Default to enable SSL, LDAP, Kerberos, libz
if the libraries are present.
10. Build with exact case global symbols for libcurl.
11. Generate linker option file needed.
12. Compiler list option only commonly needed items.
13. fulllist option for those who really want it.
14. Create debug symbol file on Alpha, IA64.
By doing this unconditionally, we infer a simpler and more defined
behavior. This also has the upside that test 1021 no longer fails for me
even if I run with valgrind.
Also fixed some wrong comments.
According KB975858 this flag may cause problems on Windows 7 and
Windows Server 2008 R2 systems. Extended error information is not
currently used by libcurl and therefore not a requirement.
The flag may improve the SSL-connection shutdown in case of an
error. This means it might be a good improvement in the future.
Fixes bug/issue #1187 - thanks for the report
When a connection is no longer used, it is kept in the cache. If the
cache is full, the oldest idle connection is closed. If no connection is
idle, the current one is closed instead.
Added recent additions and fixes following the changes to imap, pop3
and smtp. Additionally added another contributor that helped to test
the imap sasl changes.
Tidied up code from commit 6b6bdc83bdUpdated where a few instances of
the pop3c struct variable used the longer conndata struct rather than
matching what other code in pop3_authenticate() used.
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to Clear Text authentication.
Note: In order to fallback to Clear Text authentication properly this
fix adds support for the LOGINDISABLED server capability.
imap: Fixed no known authentication mechanism when fallback is required
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to Clear Text authentication.
Note: In order to fallback to Clear Text authentication properly this
fix adds support for the LOGINDISABLED server capability.
Related bug: http://curl.haxx.se/mail/lib-2013-02/0004.html
Reported by: Stanislav Ivochkin
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to APOP or Clear Text
authentication.
Bug: http://curl.haxx.se/mail/lib-2013-02/0004.html
Reported by: Stanislav Ivochkin
Remove timeout argument that's never used.
Make the actual connection get detected on a single spot to reduce code
duplication.
Store the IPv6 state already when the connection is attempted.
- Take into account that 'wc' may return leading spaces.
- Set internationalization behavior variables.
Tor Arntsen analyzed and reported the issue.
URL: http://curl.haxx.se/mail/lib-2013-01/0351.html
There was a bug where, if SSLWrite() returned errSSLWouldBlock but did
succeed in transmitting at least something, then we'd incorrectly
resend the packet. Now we never take errSSLWouldBlock as a sign that
nothing was transferred to/from the server.
Bug: http://curl.haxx.se/mail/lib-2013-01/0295.html
Reported by: Bruno de Carvalho
Minor code tidy up to add comments similar to those used in the pop3
and imap end of resp functions, in order to assist anyone reading the
code and highlight the similarities between each of these protocols.
smtp_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to smtps and start the EHLO command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
pop3_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to pop3s and start the CAPA command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
imap_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to imaps and start the CAPABILITY command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
Some basic checks we make were placed early enough in generated
configure script when using autoconf 2.5X versions. Newer autoconf
versions expand these checks much further into the configure script,
rendering them useless. Using XC_CONFIGURE_PREAMBLE fixes placement
of early intended checks across all our autoconf supported versions.
- document the double-quote and backslash need be escaped if quoting.
- libcurl formdata escape double-quote in filename by backslash.
- curl formparse can parse filename both contains '"' and ',' or ';'.
- curl now can uploading file with ',' or ';' in filename.
Bug: http://curl.haxx.se/bug/view.cgi?id=1171
Notes:
When running a configure script that has nested packages (for example
libcurl's configure with --enable-ares and c-ares sources embedded in
curl tree) and AC_CONFIG_SUBDIRS([nested-subdir]) machinery is used to
automatically run the nested configure script from within the parent
configure script, it happens that the nested _shell_ script will
inherit shell variables exported from the parent _shell_ script.
If for example parent configure script sets and exports LDFLAGS and LIBS
variables with proper values in order to link either a parent library or
program with a library which will be configured and built by a nested
package; It will happen that when the nested configure script runs, the
nested library does not exist yet and _any_ link-test done in the nested
configure will fail, such as those that autoconf macros perform in order
to detect existing compiler and its characteristics, the result is that
the nested configure script will fail with errors such as:
configure: error: C compiler cannot create executables
For now, we no longer export variables previously exported here.
On the other hand, AC_SUBST'ing them is appropriate and even with nested
packages each package's config.status gets its own package values.
So we reinstate AC_SUBST'ing previously AC_SUBST'ed variables.
Fixed an issue where Curl_ssl_connect_nonblocking() wouldn't complete
correctly and the ssldone flag wouldn't be set to true for pop3s based
connections.
Bug introduced in commit: 4ffb8a6398.
Remove internal separated behavior of the easy vs multi intercace.
curl_easy_perform() is now using the multi interface itself.
Several minor multi interface quirks and bugs have been fixed in the
process.
Much help with debugging this has been provided by: Yang Tse
Fixes initial proxy response being processed by the tunneled protocol
handler instead of the HTTP wrapper handler. This issue would trigger
upon delayed CONNECT response from the proxy.
Additionally fixes a multi interface code-path in which connections
would not time out properly.
This does not fix known bug #39.
URL: http://curl.haxx.se/mail/lib-2013-01/0191.html
If the default value for an option taking a long as its value is non
zero, and it is set by zero by a command line option, then that command
line option is not reflected in --libcurl's output. This is because line
520-521 of tool_setopt.c look like:
if(!lval)
skip = TRUE;
An example of a command-line option doing so is the -k option that sets
CURLOPT_SLL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST to 0L, when the
defaults are non-zero.
Automake documents that doing this will make it choose a different name
for intermediate object files even when sharing source files across
targets of same Makefile.am.
Up to automake 1.13.1 target's intermediate object files were placed
in the build subdirectory of the target. We depended on this, probably
undocumented behavior, to achieve same behavior as if a per-target flag
had been specified when building targets that actually belong to
different Makefile.am files.
It seems automake 1.13.2 is going to break behavior mentioned above.
So, lets use a documented behavior in order to achieve same purpose,
across automake versions, no matter where automake wishes to place
intermediate object files.
Our build targets that already were using a per-target '_CFLAGS' or
'_CPPFLAGS' need no 'fixing', these were already 'fixed'. The only
Makefile.am or Makefile.in files in libcurl's source tree touched by
this 'fix' are tests/libtest/Makefile.inc and tests/unit/Makefile.inc.
Tested with:
buildconf: autoconf version 2.69
buildconf: autom4te version 2.69
buildconf: autoheader version 2.69
buildconf: automake version 1.13.1
buildconf: aclocal version 1.13.1
buildconf: libtool version 2.4
buildconf: GNU m4 version 1.4.16
This commit fixes a regression introduced in 052a08ff.
NSS caches certs/keys returned by the SSL_GetClientAuthDataHook callback
and if we connect second time to the same server, the cached cert/key
pair is used. If we use multiple client certificates for different
paths on the same server, we need to clear the session cache to force
NSS to call the hook again. The commit 052a08ff prevented the session
cache from being cleared if a client certificate from file was used.
The condition is now fixed to cover both cases: consssl->client_nickname
is not NULL if a client certificate from the NSS database is used and
connssl->obj_clicert is not NULL if a client certificate from file is
used.
Review by: Kai Engert
gcc on DOS hasn't really supported COFF-debug (-gcoff) on djgpp for a
long time.
"Sounds like the COFF debug info generation has bit-rotted in GCC.
Nothing new here, no other platform uses COFF AFAIK."
So lets drop it too.
URL: http://curl.haxx.se/mail/lib-2013-01/0130.html
This commit renames lib/setup.h to lib/curl_setup.h and
renames lib/setup_once.h to lib/curl_setup_once.h.
Removes the need and usage of a header inclusion guard foreign
to libcurl. [1]
Removes the need and presence of an alarming notice we carried
in old setup_once.h [2]
----------------------------------------
1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard
up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H,
this single inclusion guard is enough to ensure that inclusion of
lib/setup_once.h done from lib/setup.h is only done once.
Additionally lib/setup.h has always used __SETUP_ONCE_H macro to
protect inclusion of setup_once.h even after commit ec691ca3, this
was to avoid a circular header inclusion triggered when building a
c-ares enabled version with c-ares sources available which also has
a setup_once.h header. Commit ec691ca3 exposes the real nature of
__SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard
foreign to libcurl belonging to c-ares's setup_once.h
The renaming this commit does, fixes the circular header inclusion,
and as such removes the need and usage of a header inclusion guard
foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl.
2 - Due to the circular interdependency of old lib/setup_once.h and the
c-ares setup_once.h header, old file lib/setup_once.h has carried
back from 2006 up to now days an alarming and prominent notice about
the need of keeping libcurl's and c-ares's setup_once.h in sync.
Given that this commit fixes the circular interdependency, the need
and presence of mentioned notice is removed.
All mentioned interdependencies come back from now old days when
the c-ares project lived inside a curl subdirectory. This commit
removes last traces of such fact.
This is a work-around for bug #1180 which is really libcurl's inability
to ignore SIGPIPE in a few cases. With this work-around at least curl
won't suffer from it!
Bug: http://curl.haxx.se/bug/view.cgi?id=1180
Reported by: Lluís Batlle i Rossell
I ran the 2.59 version of autoupdate that updates obsoleted configure.ac
constructs to the 2.59 standard. With a little hands-on fiddling I
prevented it from ruining the quoting in AS_HELP_STRING() uses.
I subsequently also bumped the required autoconf version to 2.59
(released in December 2003) as I don't have an older autoconf version
around to test with and I can't be bothered to install one either...
Inspired by: Björn Stenberg
Related blog post: http://cazfi.livejournal.com/195108.html
This commit fixes support for sockets that are ready to accept
a new connection and have previously been put into listening mode.
It also includes changes which are the result of investigation
regarding Windows STDIN. These changes are the preparation for further
improvements regarding support for reading data from STDIN on Windows.
Open issue: WaitForMultipleObjectsEx does not support PIPE handles
which are returned by GetStdHandle while running without a GUI.
This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:
f871de0... build: make use of 76 lib/*.h renamed files
ffd8e12... build: rename 76 lib/*.h files
This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:
c087374... curl_setup.h: remove redundant include guard
This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:
13606bb... build: make use of 93 lib/*.c renamed files
5b6e792... build: rename 93 lib/*.c files
7d83dff... build: commit 13606bbfde follow-up 1
Start of related discussion thread:
http://curl.haxx.se/mail/lib-2013-01/0012.html
Asking for confirmation on pushing this revertion commit:
http://curl.haxx.se/mail/lib-2013-01/0048.html
Confirmation summary:
http://curl.haxx.se/mail/lib-2013-01/0079.html
NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.
lib/curl_imap.h
lib/curl_smtp.h
1. When the downloaded data file from Mozilla is current, but the output
bundle does not exist: continue processing to create the bundle. The
goal is to have the output file - not just download the latest input.
2. added -f option to force re-processing the file. Useful for
debugging/testing the process.
3. added support for output to '-' (stdout), allowing the output to be
piped.
4. All progress and error messages go to STDERR rather than STDOUT (3)
5. The script opened and closed the output file many times
unnecessarily. It now opens it once, does the output and closes it.
6. Backup of the input files happens after successful processing, not
before.
7. The output is written to a temporary file, and renamed to the
requested name after backup - this greatly reduces the window where the
file can be seen partially written.
8. all die calls have a \n at the end to suppress perl's traceback - the
traceback isn't useful to end users.
Patch: http://curl.haxx.se/mail/lib-2013-01/0045.html
lib/objnames.inc provides definition of curl_10char_object_name() shell
function. The intended purpose of this function is to transliterate a
(*.c) source file name that may be longer than 10 characters, or not,
into a string with at most 10 characters which may be used as an OS/400
object name.
Test case 1221 does unit testng of this function and also verifies
that it is possible to generate distinct short object names for all
curl and libcurl *.c source file names.
lib/objnames-test.sh is the shell script used for test case 1221.
tests/runtests.pl modified to accept shell script test cases.
More details inside lib/objnames.inc and lib/objnames-test.sh
automake 1.13 errors if AM_CONFIG_HEADER is used in configure script.
automake 1.13 no longer autoupdates AM_CONFIG_HEADER to
AC_CONFIG_HEADERS, thing which automake has been doing since automake
version 1.7
Given that our first automake supported version is automake 1.7,
simply replacing AM_CONFIG_HEADER usage with AC_CONFIG_HEADERS seems
enough to yet support same automake versions.
Dave Reisner reported issue with 1.13 and provided patch.
http://curl.haxx.se/mail/lib-2012-12/0246.html
Provide a 'traceable' AC_CONFIG_MACRO_DIR definition only when using
an autoconf version that does not provide it, instead of what we were
doing up to now of providing and overriding AC_CONFIG_MACRO_DIR for
all autoconf versions.
* Changing the order of the state machine to represent the order in
which commands are sent to the server.
* Reworking the imap_endofresp() function as the FETCH response doesn't
include the command id and shouldn't be part of the length comparison
that takes into account the id string.
Fixed a problem with the state machine when attempting to log in with
invalid credentials. The server would report login failure but libcurl
would not read the response due to inappropriate IMAP_STOP states being
set after the login was sent.
Applied some of the comment and layout changes that had already been
applied to the pop3 and smtp code over the last 6 to 9 months.
This is in preparation of adding SASL based authentication.
... on Snow Leopard and Lion
Snow Leopard introduced the SSLSetSessionOption() function, but it
doesn't disable peer verification as expected on Snow Leopard or
Lion (it works as expected in Mountain Lion). So we now use sysctl()
to detect whether or not the user is using Snow Leopard or Lion,
and if that's the case, then we now use the deprecated
SSLSetEnableCertVerify() function instead to disable peer verification.
WinSock select() does not support standard file descriptors,
it can only check SOCKETs. The following function is an attempt
to create a select() function with support for other handles.
... it also clobbered the 'result' return value so that it wouldn't
return the error back to the parent function properly, which broke test
809 when run with 'multi-always'.
When prefixing a path with /~/ it is supposed to be used relative to the
user's home directory but it didn't work. Now we cut off the entire
three byte sequenct "/~/" which seems to be how OpenSSH does it.
Bug: http://curl.haxx.se/bug/view.cgi?id=1173
Reported by: Balaji Parasuram
Issue: When building a 32bit target with large file support HP-UX
<sys/socket.h> header file may simultaneously provide two different
sets of declarations for sendfile and sendpath functions, one with
static and another with external linkage. Given that we do not use
mentioned functions we really don't care which linkage is the
appropriate one, but on the other hand, the double declaration emmits
warnings when using the HP-UX compiler and errors when using modern
gcc versions resulting in fatal compilation errors.
Mentioned issue is now fixed as long as we don't use sendfile nor
sendpath functions.
When cross-compiling, CURL_CHECK_PKGCONFIG was checking for the cross
pkg-config using ${host}-pkg-config.
The gold standard for doing this correctly is pkg-config's own macro,
PKG_PROG_PKG_CONFIG. However, on the assumption that you have a good
reason not to use that directly (reduced dependencies for maintainer
builds?), the behaviour of cURL's version should at least match.
PKG_PROG_PKG_CONFIG uses AC_PATH_TOOL, which ultimately ends up trying
${host_alias}-pkg-config; this is not quite the same as what cURL does,
and may differ because ${host} has been run through config.sub. For
instance, when cross-building to the armhf architecture on Ubuntu,
${host_alias} is arm-linux-gnueabihf while ${host} is
arm-unknown-linux-gnueabihf. This may also have been the cause of the
problem reported at http://curl.haxx.se/mail/lib-2012-04/0224.html.
AC_PATH_TOOL is significantly simpler than cURL's current code, and
dates back to well before the current minimum of Autoconf 2.57, so let's
use it instead.
A bundle is a list of all persistent connections to the same host.
The connection cache consists of a hash of bundles, with the
hostname as the key.
The benefits may not be obvious, but they are two:
1) Faster search for connections to reuse, since the hash
lookup only finds connections to the host in question.
2) It lays out the groundworks for an upcoming patch,
which will introduce multiple HTTP pipelines.
This patch also removes the awkward list of "closure handles",
which were needed to send QUIT commands to the FTP server
when closing a connection.
Now we allocate a separate closure handle and use that
one to close all connections.
This has been tested in a live system for a few weeks, and of
course passes the test suite.
BLANK_AT_MAKETIME may be used in our Makefile.am files to blank
LIBS variable used in generated makefile at makefile processing
time. Doing this functionally prevents LIBS from being used for
all link targets in given makefile.
This handling already works with the easy-interface code. When a request
is sent on a re-used connection that gets closed by the server at the
same time as the request is sent, the situation may occur so that we can
send the request and we discover the broken connection as a RECV_ERROR
in the PERFORM state and then the request needs to be retried on a fresh
connection. Test 64 broke with 'multi-always-internally'.
Although it is not explicitly stated in the documentation, NSS uses
*pRetCert and *pRetKey even if the client authentication hook returns
a failure. Namely, if we destroy *pRetCert without clearing *pRetCert
afterwards, NSS destroys the certificate once again, which causes a
double free.
Reported by: Bob Relyea
.. that are sent when auth-negotiating before a chunked
upload or when setting the 'Transfer-Encoding: chunked'
header and intentionally sending no content.
Adjust test565 and test1333 accordingly.
Blocking connect on the socket has been removed from opensocket
callback. opensocket just opens a new socket and gives it back to
libcurl and libcurl will take care of the connect. sockopt_callback has
also been removed, as it is no longer required.
AIX sys/poll.h header file defines 'events' and 'revents' as C
preprocessor macros. Usage of these literals in libcurl's external
API was introduced in commit de24d7bd4c causing AIX build failures.
Appropriate inclusion of sys/poll.h by libcurl's external interface
fixes AIX build and usage issues while avoiding a SONAME bump.
runtests.pl -am now uses the "PASS/FAIL: [desc]" output for each
executed test. You can run 'make test-am' in the root build directory to
invoke that. The reason for this output style is to better allow generic
test suite parsers to also grok our test output.
The test Makefile now also tests that perl was indeed found and that the
PERL variable points to an executable before it tries to run the main
test perl script runtests.pl,
It makes no difference from curl's point of view but
makes it more convenient to use the tests with a
lws-normalizing proxy between curl and the test server.
Consistently use CRLF instead. The mixed endings weren't
documented so I assume they were unintentional.
This change doesn't matter for curl itself but makes using
the tests with a proxy between curl and the test server
more convenient.
Tests that consistently use no carriage returns were
left unmodified as one can easily work around this.
DNS cache entries populated with CURLOPT_RESOLVE were not properly freed
again when done using the multi interface.
Test case 1502 added to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=3575448
Reported by: Alex Gruz
If we use memory functions (malloc, free, strdup etc) in C sources in
libcurl and we fail to include curl_memory.h or memdebug.h we either
fail to properly support user-provided memory callbacks or the memory
leak system of the test suite fails.
After Ajit's report of a failure in the first category in http_proxy.c,
I spotted a few in the second category as well. These problems are now
tested for by test 1132 which runs a perl program that scans for and
attempts to check that we use the correct include files if a memory
related function is used in the source code.
Reported by: Ajit Dhumale
Bug: http://curl.haxx.se/mail/lib-2012-11/0125.html
When nothing is told to configure, we should not enforce switching off
debug options with -g0 (or similar). We instead don't use -g at all in
that situaion and therefore allow the user's CFLAGS settings possibly
dictate what to do.
The {MD5,SHA1,SHA256}_Init functions from OpenSSL are called directly
without any wrappers and they return 1 for success, 0 otherwise. Hence,
we have to use the same approach in all the wrapper functions that are
used for the other crypto libraries.
This commit fixes a regression introduced in commit dca8ae5f.
When using only 1 second precision, curl doesn't create new cnonce
values quickly enough for all uses.
For example, issuing the following command multiple times to a recent
Tomcat causes authentication failures:
curl --digest -utest:test http://tomcat.test.com:8080/manager/list
This is because curl uses the same cnonce for several seconds, but
doesn't increment the nonce counter. Tomcat correctly interprets
this as a replay attack and rejects the request.
When microsecond-precision is available, this commit causes curl to
change cnonce values much more frequently.
With microsecond resolution, increasing the nounce length used in the
headers to 32 was made to further reduce the risk of duplication.
axTLS:
This will make the axTLS backend perform the RFC2818 checks, honoring
the VERIFYHOST setting similar to the OpenSSL backend.
Generic for OpenSSL and axTLS:
Move the hostcheck and cert_hostcheck functions from the lib/ssluse.c
files to make them genericly available for both the OpenSSL, axTLS and
other SSL backends. They are now in the new lib/hostcheck.c file.
CyaSSL:
CyaSSL now also has the RFC2818 checks enabled by default. There is a
limitation that the verifyhost can not be enabled exclusively on the
Subject CN field comparison. This SSL backend will thus behave like the
NSS and the GnuTLS (meaning: RFC2818 ok, or bust). In other words:
setting verifyhost to 0 or 1 will disable the Subject Alt Names checks
too.
Schannel:
Updated the schannel information messages: Split the IP address usage
message from the verifyhost setting and changed the message about
disabling SNI (Server Name Indication, used in HTTP virtual hosting)
into a message stating that the Subject Alternative Names checks are
being disabled when verifyhost is set to 0 or 1. As a side effect of
switching off the RFC2818 related servername checks with
SCH_CRED_NO_SERVERNAME_CHECK
(http://msdn.microsoft.com/en-us/library/aa923430.aspx) the SNI feature
is being disabled. This effect is not documented in MSDN, but Wireshark
output clearly shows the effect (details on the libcurl maillist).
PolarSSL:
Fix the prototype change in PolarSSL of ssl_set_session() and the move
of the peer_cert from the ssl_context to the ssl_session. Found this
change in the PolarSSL SVN between r1316 and r1317 where the
POLARSSL_VERSION_NUMBER was at 0x01010100. But to accommodate the Ubuntu
PolarSSL version 1.1.4 the check is to discriminate between lower then
PolarSSL version 1.2.0 and 1.2.0 and higher. Note: The PolarSSL SVN
trunk jumped from version 1.1.1 to 1.2.0.
Generic:
All the SSL backends are fixed and checked to work with the
ssl.verifyhost as a boolean, which is an internal API change.
The text "additional stuff not fine" text was added for debug purposes a
while ago, but it isn't really helping anyone and for some reason some
Linux distributions provide their libcurls built with debug info still
present and thus (far too many) users get to read this info.
The logic previously checked for a started NTLM negotiation only for
host and not also with proxy, leading to problems doing POSTs over a
proxy NTLM that are larger than 2000 bytes. Now it includes proxy in the
check.
Bug: http://curl.haxx.se/bug/view.cgi?id=3582321
Reported by: John Suprock
The existing logic only cut off the fragment from the separate 'path'
buffer which is used when sending HTTP to hosts. The buffer that held
the full URL used for proxies were not dealt with. It is now.
Test case 5 was updated to use a fragment on a URL over a proxy.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579813
As a handle can be re-used after having done HTTP auth in a previous
request, it must make sure to clear out the HTTP types that aren't
wanted in this new request.
This reverts commit ce8311c7e4.
The commit made test 2024 work but caused a regression with repeated
Digest authentication. We need to fix this differently.
After a research team wrote a document[1] that found several live source
codes out there in the wild that misused the CURLOPT_SSL_VERIFYHOST
option thinking it was a boolean, this change now bans 1 as a value and
will make libcurl return error for it.
1 was never a sensible value to use in production but was introduced
back in the days to help debugging. It was always documented clearly
this way.
1 was never supported by all SSL backends in libcurl, so this cleanup
makes the treatment of it unified.
The report's list of mistakes for this option were all PHP code and
while there's a binding layer between libcurl and PHP, the PHP team has
decided that they have an as thin layer as possible on top of libcurl so
they will not alter or specifically filter a 'TRUE' value for this
particular option. I sympathize with that position.
[1] = http://daniel.haxx.se/blog/2012/10/25/libcurl-claimed-to-be-dangerous/
Since automake 1.12.4, the warnings are issued on running automake:
warning: 'INCLUDES' is the old name for 'AM_CPPFLAGS' (or '*_CPPFLAGS')
Avoid INCLUDES and roll these flags into AM_CPPFLAGS.
Compile tested on:
Ubuntu 10.04 (automake 1:1.11.1-1)
Ubuntu 12.04 (automake 1:1.11.3-1ubuntu2)
Arch Linux (automake 1.12.4)
As pointed out in Bug report #3579064, curl_multi_perform() would
wrongly use a blocking mechanism internally for some commands which
could lead to for example a very long block if the LIST response never
showed.
The solution was to make sure to properly continue to use the multi
interface non-blocking state machine.
The new test 1501 verifies the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579064
Reported by: Guido Berhoerster
The URL we used before is the one that goes directly to 'add' a bug
report, but since you can only do that after first having logged in to
sourceforge, the link often doesn't work for visitors.
Bug: http://curl.haxx.se/bug/view.cgi?id=3582408
Reported by: Oscar Norlander
When given a string as 'srp' it didn't work, but required 'SRP'.
Starting now, the check disregards casing.
Bug: http://curl.haxx.se/bug/view.cgi?id=3578418
Reported by: Jeff Connelly
Previously the Metalink code used Apple's CommonCrypto library only if
curl was built using the --with-darwinssl option. Now we use CommonCrypto
on all Apple operating systems including Tiger or later, or iOS 5 or
later, so you don't need to build --with-darwinssl anymore. Also rolled
out this change to libcurl's md5 code.
The iOS build was broken by a reference to a function that only existed
under OS X; fixed. Also fixed a hard-to-reproduce problem where, if the
server disconnected before libcurl got the chance to hang up first and
SecureTransport was in use, then we'd raise an error instead of failing
gracefully.
The makefile is designed to build against a libmetalink devel package;
therefore is does not matter what will change inside libmetalink.
Add OpenSSL includes and defines for libmetalink-aware OpenSSL builds.
This is a minor change in behavior after having been pointed out by Mark
Tully and discussed on the list. Initially this case would internally
call poll() with no sockets and a timeout which would equal a sleep for
that specified time.
Bug: http://curl.haxx.se/mail/lib-2012-10/0076.html
Reported by: Mark Tully
Since there are servers that seem to return very big encrypted
data packages, we need to be able to handle those without having
an internal size limit. To avoid the buffer growing to fast to
early the initial size was decreased and the minimum free space
in the buffer was decreased as well.
During the periods of rate limitation, the speedcheck function wasn't
called and thus the values weren't updated accordingly and it would then
easily trigger wrongly once data got transferred again.
Also, the progress callback's return code was not acknowledged in this
state so it could make an "abort" return code to get ignored and not
have the documented effect of aborting an ongoing transfer.
Bug: http://curl.haxx.se/mail/lib-2012-09/0081.html
Reported by: Jie He
In Metalink v3, the type attribute of url element indicates the
type of the resource the URL points to. It can include URL to the
meta data, such as BitTorrent metainfo file. In Curl, we are not
interested in these meta data URLs. Instead, we are only
interested in the HTTP and FTP URLs. This change filters out
non-HTTP and FTP URLs. If we don't filter out them, it will be
downloaded by curl and hash check will fail if hash is provided
and next URL will be tried. This change will cut this useless
network transfer.
The Curl_reconnect_request() function could end up returning a pointer
to a free()d struct when Curl_done() failed inside. Clearing the pointer
unconditionally after Curl_done() avoids this risk.
Reported by: Ho-chi Chen
Bug: http://curl.haxx.se/mail/lib-2012-09/0188.html
Selected socks proxy in Google's Chrome browser. Resulting in the
following environment variables:
NO_PROXY=localhost,127.0.0.0/8
ALL_PROXY=socks://localhost:1080/
all_proxy=socks://localhost:1080/
no_proxy=localhost,127.0.0.0/8
... and libcurl didn't treat 'socks://' as socks but instead picked HTTP
proxy.
Reported by: Scott Bailey
Bug: http://curl.haxx.se/bug/view.cgi?id=3566860
Since Metalink support requires a crypto library for hash functions
and Windows comes with the builtin CryptoAPI, this patch adds that
API as a fallback to the supported crypto libraries.
It is automatically used on Windows if no other library is provided.
Fixed tests/libtest/libntlmconnect.c:52: warning: call to
'_curl_easy_getinfo_err_long' declared with attribute warning:
curl_easy_getinfo expects a pointer to long for this info
Since Windows/MinGW threat 0x1A as the EOF character, reading binary
files which contain that byte does not work using text mode.
The read function will only read until the first 0x1A byte. This
means that the hash is not computed from the whole file and the
final validation check using hash comparision fails.
Since Simple and Protected GSSAPI Negotiation Mechanism
is already implemented in curl and supported by the MinGW
builds, this change adds build support to winbuild makefiles.
Cleaned up order of handled build options by ordering them
nearly alphabetically by using the order of the generated
config name. Preparation for future/more build options.
While validating a new Clang diagnostic (-Wnon-literal-null-conversion -
yes, the name isn't quite correct in this case, but it suffices) I found
a few violations of it in Curl.
Each certificate section of the input certdata.txt file has a trust
section following it with details.
This script failed to detect the start of the trust for at least one
cert[*], which made the script continue pass that section into the next
one where it found an 'untrusted' marker and as a result that certficate
was not included in the output.
[*] = "Hellenic Academic and Research Institutions RootCA 2011"
Bug: http://curl.haxx.se/mail/lib-2012-09/0019.html
SMTP client will send SIZE parameter in MAIL FROM command only if server
supports it. Without this patch server might say "504 Command parameter
not implemented" and reject the message.
Bug: http://curl.haxx.se/bug/view.cgi?id=3564114
/*
* Name: curl_multi_wait()
*
* Desc: Poll on all fds within a CURLM set as well as any
* additional fds passed to the function.
*
* Returns: CURLMcode type, general multi error code.
*/
CURL_EXTERN CURLMcode curl_multi_wait(CURLM *multi_handle,
struct curl_waitfd extra_fds[],
unsigned int extra_nfds,
int timeout_ms);
The test would hang and get aborted with a "ABORTING TEST, since it
seems that it would have run forever." until I prevented that from
happening.
I also fixed the data file which got broken CRLF line endings when I
sucked down the path from Joe's repo == my fault.
Removed #37 from KNOWN_BUGS as this fix and test case verifies exactly
this.
"Currently you can only share DNS and/or COOKIE data" is incorrect since
also SSL sessions can be shared.
Bug: http://curl.haxx.se/bug/view.cgi?id=3562261
Reported by: Joe Mason
It's conceivable that after the first time curl_multi_perform returns,
the outvalue still_running will be 0, but work will have been done. This
is shown by a workload of small, purely file:// based URLs. Ensure that
we always read pending messages off the multi handle by forcing the
while loop to run at least once.
In Mountain Lion, Apple added TLS 1.1 and 1.2, and deprecated a number
of SecureTransport functions, some of which we were using. We now check
to see if the replacement functions are present, and if so, we use them
instead. The old functions are still present for users of older
cats. Also fixed a build warning that started to appear under Mountain
Lion
Commit b91d29a28e170c16d65d956db79f2cd3a82372d2 introduces a bug and breaks Curl_closesocket function. sock_accepted flag for the second socket should be tagged as TRUE before the sockopt callback is called because in case the callback returns an error, Curl_closesocket function is going to call the - fclosesocket - callback for the accept()ed socket
For active FTP connections, applications may need setting the sockopt after accept() call returns successful. This fix gives a call to the callback registered with CURL_SOCKOPTFUNCTION option. Also a new sock type - CURLSOCKTYPE_ACCEPT - is added. This type is to be passed to application callbacks with - purpose - parameter. Applications may use this parameter to distinguish between socket types.
Removing this option as it currently only functions to lure people into
wrongly using it and falsely believing that libcurl will work fine
without using nonblocking sockets internally - which leads to hard to
track or understand errors.
Commit e351972bc8 brought in the ssh agent support but some uses of
the libssh2 agent API was done unconditionally which wasn't good enough
since that API hasn't always been present.
By reading the ->head pointer and using that instead of the ->size
number to figure out if there's a list remaining we avoid the (false
positive) clang-analyzer warning that we might dereference of a null
pointer.
I suspect this is a regression introduced in commit 207cf150, included
since 7.24.0.
Avoid showing '(nil)' as hostname in verbose output by making sure the
hostname fixup function is called early enough to set the pointers that
are used for this. The name data is set again for each request even for
re-used connections to handle multiple hostnames over the same
connection (like with proxy) or that the casing etc of the host name is
changed between requests (which has proven to be important at least once
in the past).
Test1011 was modified to use a redirect with a re-used a connection
since it then showed the bug and now lo longer does. There's currently
no easy way to have the test suite detect 'nil' texts in verbose ouputs
so no tests will detect if this problem gets reintroduced.
Bug: http://curl.haxx.se/mail/lib-2012-07/0111.html
Reported by: Gisle Vanem
We found a problem with ftp transfer using libcurl (7.23 and 7.25)
inside an application which is receiving unix signals (SIGUSR1,
SIGUSR2...) almost continuously. (Linux 2.4, PowerPC, HAVE_POLL_FINE
defined).
Curl_socket_check() uses poll() to wait for the socket, and retries it
when a signal is received (EINTR). However, if a signal is received and
it also happens that the timeout has been reached, Curl_socket_check()
returns -1 instead of 0 (indicating an error instead of a timeout).
In our case, the result is an aborted connection even before the ftp
banner is received from the server, and a return value of
CURLE_OUT_OF_MEMORY from curl_easy_perform() (Curl_pp_multi_statemach(),
in pingpong.c, actually returns OOM if Curl_socket_check() fails :-)
Funny to debug on a system on which OOM is a possible cause).
Bug: http://curl.haxx.se/mail/lib-2012-07/0122.html
Due to WSAPoll bugs, libcurl does not work as intended. When the cURL
library is used to setup a connection to an incorrect port, normally the
result is CURLE_COULDNT_CONNECT, /* 7 */, but due to the bug in WSAPoll,
the result now is CURLE_OPERATION_TIMEDOUT, /* 28 - the timeout time was
reached */.
On August 1, Jan Koen Annot opened a case for this to Microsoft Premier
Online (https://premier.microsoft.com/). The support engineer handling
the case wrote that the case description is quite clear. He will try to
reproduce the issue and then proceed with troubleshooting it.
Reported by: Jan Koen Annot
Bug: http://curl.haxx.se/mail/lib-2012-07/0310.html
When figuring out if the data stream needs to be rewound when the
request is to be resent, we must not access the HTTP struct unless the
protocol used is indeed HTTP...
Bug: http://curl.haxx.se/bug/view.cgi?id=3544688
Fix a bug where closed sockets (fd -1) were left in the all_sockets
list, because of missing parens in a pointer arithmetic expression
Reenable the tests that were locking up due to this bug.
SO_KEEPALIVE flag to all sockets. Note that several loops which used to continue on a return value
of 0 (theoretical since 0 would never be returned without O_NONBLOCK) now break on 0 so that they
won't continue reading until after poll is called again.
for non-blocking sockets: now -1 means error or connection finished, 1 means data was read, and 0
means there is no data available now so need to wait for poll (new return value)
half-finished.
Note the the req struct used to be re-initialized AFTER reading pipeline data, so now that we
initialize it from the caller we must be careful not to overwrite the pipeline data.
Also we now need to handle the case where the buffer is already full when get_request is called -
previously this never happened as it was always called with an empty buffer and looped until done.
Now get_request is called in a loop, so the next step is to run the loop on a socket only when poll
signals it is readable.
Removed APOP and SASL authentication from the POP3 section and metalink
support from the client section as these features were implemented in
this release.
Moved adding gssapi to SASL into it's own section rather than repeat it
for each protocol.
/* maximum number of entries in the connection cache */
CINIT(MAXCONNECTS,LONG,6),
/* maximum number of (pipelining) connections to one host */
CINIT(MAX_HOST_CONNECTIONS,LONG,7),
/* maximum number of requests in a pipeline */
CINIT(MAX_PIPELINE_LENGTH,LONG,8),
/* a connection with a content-length longer than this
will not be considered for pipelining */
CINIT(CONTENT_LENGTH_PENALTY_SIZE,OFF_T,9),
/* a connection with a chunk length longer than this
will not be considered for pipelining */
CINIT(CHUNK_LENGTH_PENALTY_SIZE,OFF_T,10),
/* a list of site names(+port) that are blacklisted from
pipelining */
CINIT(PIPELINING_SITE_BL,OBJECTPOINT,11),
/* a list of server types that are blacklisted from
pipelining */
CINIT(PIPELINING_SERVER_BL,OBJECTPOINT,12),
/* maximum number of open connections in total */
CINIT(MAX_TOTAL_CONNECTIONS,LONG,13),
CURLMOPT_LASTENTRY/* the last unused */
}CURLMoption;
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
