7.17.1!

Added generation of dummy ca-bundle.h. Sorted objects.

CHANGES

@@ -6,6 +6,233 @@
 
                                   Changelog
 
+Version 7.17.1 (29 October 2007)
+
+Dan F (25 October 2007)
+- Added the --static-libs option to curl-config
+
+Daniel S (25 October 2007)
+- Made libcurl built with NSS possible to ignore the peer verification.
+  Previously it would fail if the ca bundle wasn't present, even if the code
+  ignored the verification results.
+
+Patrick M (25 October 2007)
+- Fixed test server to allow null bytes in binary posts.
+_ Added tests 35, 544 & 545 to check binary data posts, both static (in place)
+  and dynamic (copied).
+
+Daniel S (25 October 2007)
+- Michal Marek fixed the test script to be able to use valgrind even when the
+  lib is built shared with libtool.
+
+- Fixed a few memory leaks when the same easy handle is re-used to request
+  URLs with different protocols. FTP and TFTP related leaks. Caught thanks to
+  Dan F's new test cases.
+
+Dan F (24 October 2007)
+- Fixed the test FTP and TFTP servers to support the >10000 test number
+  notation
+
+- Added test cases 2000 through 2003 which test multiple protocols using the
+  same easy handle
+
+- Fixed the filecheck: make target to work outside the source tree
+
+Daniel S (24 October 2007)
+- Vladimir Lazarenko pointed out that we should do some 'mt' magic when
+  building with VC8 to get the "manifest" embedded to make fine stand-alone
+  binaries. The maketgz and the src/Makefile.vc6 files were adjusted
+  accordingly.
+
+Daniel S (23 October 2007)
+- Bug report #1812190 (http://curl.haxx.se/bug/view.cgi?id=1812190) points out
+  that libcurl tried to re-use connections a bit too much when using non-SSL
+  protocols tunneled over a HTTP proxy.
+
+Daniel S (22 October 2007)
+- Michal Marek forwarded the bug report
+  https://bugzilla.novell.com/show_bug.cgi?id=332917 about a HTTP redirect to
+  FTP that caused memory havoc. His work together with my efforts created two
+  fixes:
+
+  #1 - FTP::file was moved to struct ftp_conn, because is has to be dealt with
+       at connection cleanup, at which time the struct HandleData could be
+       used by another connection.
+       Also, the unused char *urlpath member is removed from struct FTP.
+ 
+  #2 - provide a Curl_reset_reqproto() function that frees
+       data->reqdata.proto.* on connection setup if needed (that is if the
+       SessionHandle was used by a different connection).
+
+  A long-term goal is of course to somehow get rid of how the reqdata struct
+  is used, as it is too error-prone.
+ 
+- Bug report #1815530 (http://curl.haxx.se/bug/view.cgi?id=1815530) points out
+  that specifying a proxy with a trailing slash didn't work (unless it also
+  contained a port number).
+
+Patrick M (15 October 2007)
+- Fixed the dynamic CURLOPT_POSTFIELDS problem: this option is now static again
+  and option CURLOPT_COPYPOSTFIELDS has been added to support dynamic mode.
+
+Patrick M (12 October 2007)
+- Added per-protocol callback static tables, replacing callback ptr storage
+  in the connectdata structure by a single handler table ptr.
+
+Dan F (11 October 2007)
+- Fixed the -l option of runtests.pl
+
+- Added support for skipping tests based on key words.
+
+Daniel S (9 October 2007)
+- Michal Marek removed the no longer existing return codes from the curl.1
+  man page.
+
+Daniel S (7 October 2007)
+- Known bug #47, which confused libcurl if doing NTLM auth over a proxy with
+  a response that was larger than 16KB is now improved slightly so that now
+  the restriction at 16KB is for the headers only and it should be a rare
+  situation where the response-headers exceed 16KB. Thus, I consider #47 fixed
+  and the header limitation is now known as known bug #48.
+
+Daniel S (5 October 2007)
+- Michael Wallner made the CULROPT_COOKIELIST option support a new magic
+  string: "FLUSH". Using that will cause libcurl to flush its cookies to the
+  CURLOPT_COOKIEJAR file.
+
+- The new file docs/libcurl/ABI describes how we view ABI breakages, soname
+  bumps and what the version number's significance to all that is.
+
+Daniel S (4 October 2007)
+- I enabled test 1009 and made the --local-port use a wide range to reduce the
+  risk of failures.
+
+- Kim Rinnewitz reported that --local-port didn't work with TFTP transfers.
+  This happened because the tftp code always uncondionally did a bind()
+  without caring if one already had been done and then it failed. I wrote a
+  test case (1009) to verify this, but it is a bit error-prone since it will
+  have to pick a fixed local port number and since the tests are run on so
+  many different hosts in different situations I'll add it in disabled state.
+
+Yang Tse (3 October 2007)
+- Fixed issue related with the use of ares_timeout() result.
+
+Daniel S (3 October 2007)
+- Alexey Pesternikov introduced CURLOPT_OPENSOCKETFUNCTION and
+  CURLOPT_OPENSOCKETDATA to set a callback that allows an application to
+  replace the socket() call used by libcurl. It basically allows the app to
+  change address, protocol or whatever of the socket.
+
+- I renamed the CURLE_SSL_PEER_CERTIFICATE error code to
+  CURLE_PEER_FAILED_VERIFICATION (standard CURL_NO_OLDIES style), and made
+  this return code get used by the previous SSH MD5 fingerprint check in case
+  it fails.
+
+- Based on a patch brought by Johnny Luong, libcurl now offers
+  CURLOPT_SSH_HOST_PUBLIC_KEY_MD5 and the curl tool --hostpubmd5. They both
+  make the SCP or SFTP connection verify the remote host's md5 checksum of the
+  public key before doing a connect, to reduce the risk of a man-in-the-middle
+  attack.
+
+Daniel S (2 October 2007)
+- libcurl now handles chunked-encoded CONNECT responses
+
+Daniel S (1 October 2007)
+- Alex Fishman reported a curl_easy_escape() problem that was made the
+  function do wrong on all input bytes that are >= 0x80 (decimal 128) due to a
+  signed / unsigned mistake in the code. I fixed it and added test case 543 to
+  verify.
+
+Daniel S (29 September 2007)
+- Immanuel Gregoire fixed a problem with persistent transfers over SFTP.
+
+Daniel S (28 September 2007)
+- Adapted the c-ares code to the API change c-ares 1.5.0 brings in the
+  notifier callback(s).
+
+Dan F (26 September 2007)
+- Enabled a few more gcc warnings with --enable-debug.  Renamed a few
+  variables to avoid shadowing global declarations.
+
+Daniel S (26 September 2007)
+- Philip Langdale provided the new CURLOPT_POST301 option for
+  curl_easy_setopt() that alters how libcurl functions when following
+  redirects. It makes libcurl obey the RFC2616 when a 301 response is received
+  after a non-GET request is made. Default libcurl behaviour is to change
+  method to GET in the subsequent request (like it does for response code 302
+  - because that's what many/most browsers do), but with this CURLOPT_POST301
+  option enabled it will do what the spec says and do the next request using
+  the same method again. I.e keep POST after 301. 
+
+  The curl tool got this option as --post301
+
+  Test case 1011 and 1012 were added to verify.
+
+- Max Katsev reported that when doing a libcurl FTP request with
+  CURLOPT_NOBODY enabled but not CURLOPT_HEADER, libcurl wouldn't do TYPE
+  before it does SIZE which makes it less useful. I walked over the code and
+  made it do this properly, and added test case 542 to verify it.
+
+Daniel S (24 September 2007)
+- Immanuel Gregoire fixed KNOWN_BUGS #44: --ftp-method nocwd did not handle
+  URLs ending with a slash properly (it should list the contents of that
+  directory). Test case 351 brought back and also test 1010 was added.
+
+Daniel S (21 September 2007)
+- Mark Davies fixed Negotiate authentication over proxy, and also introduced
+  the --proxy-negotiate command line option to allow a user to explicitly
+  select it.
+
+Daniel S (19 September 2007)
+- Rob Crittenden provided an NSS update with the following highlights:
+
+  o It looks for the NSS database first in the environment variable SSL_DIR,
+    then in /etc/pki/nssdb, then it initializes with no database if neither of
+    those exist.
+
+  o If the NSS PKCS#11 libnspsem.so driver is available then PEM files may be
+    loaded, including the ca-bundle. If it is not available then only
+    certificates already in the NSS database are used.
+
+  o Tries to detect whether a file or nickname is being passed in so the right
+    thing is done
+
+  o Added a bit of code to make the output more like the OpenSSL module,
+    including displaying the certificate information when connecting in
+    verbose mode
+
+  o Improved handling of certificate errors (expired, untrusted, etc)
+
+  The libnsspem.so PKCS#11 module is currently only available in Fedora
+  8/rawhide. Work will be done soon to upstream it. The NSS module will work
+  with or without it, all that changes is the source of the certificates and
+  keys.
+
+Daniel S (18 September 2007)
+- Immanuel Gregoire pointed out that public key SSH auth failed if no
+  public/private key was specified and there was no HOME environment variable,
+  and then it didn't continue to try the other auth methods. Now it will
+  instead try to get the files id_dsa.pub and id_dsa from the current
+  directory if none of the two conditions were met.
+
+Dan F (17 September 2007)
+- Added hooks to the test suite to make it possible to test a curl running
+  on a remote host.
+
+- Changed some FTP tests to validate the format of the PORT and EPRT commands
+  sent by curl, if not the addresses themselves.
+
+Daniel S (15 September 2007)
+- Michal Marek made libcurl automatically append ";type=<a|i>" when using HTTP
+  proxies for FTP urls.
+
+- G<>nter Knauf fixed LDAP builds in the Windows makefiles and fixed LDAPv3
+  support on Windows.
+
+Dan F (13 September 2007)
+- Added LDAPS, SCP and SFTP to curl-config --protocols. Removed and
+  fixed some AC_SUBST configure entries.
+
 Version 7.17.0 (13 September 2007)
 
 Daniel S (12 September 2007)

RELEASE-NOTES

@@ -1,58 +1,53 @@
-Curl and libcurl 7.17.0
+Curl and libcurl 7.17.1
 
- Public curl release number:               101
- Releases counted from the very beginning: 127
- Available command line options:           118
- Available curl_easy_setopt() options:     143
+ Public curl release number:               102
+ Releases counted from the very beginning: 128
+ Available command line options:           121
+ Available curl_easy_setopt() options:     147
  Number of public functions in libcurl:    55
- Amount of public web site mirrors:        42
+ Amount of public web site mirrors:        43
  Number of known libcurl bindings:         36
- Number of contributors:                   572
+ Number of contributors:                   588
 
 This release includes the following changes:
  
- o support for OS/400 Secure Sockets Layer library
- o curl_easy_setopt() now allocates strings passed to it
- o SCP and SFTP support now requires libssh2 0.16 or later
- o LDAP libraries are now linked "regularly" and not with dlopen
- o HTTP transfers have the download size info "available" earlier
- o FTP transfers have the download size info "available" earlier
- o builds and runs on OS/400
- o several error codes and options were marked as obsolete and subject to
-   future removal (set CURL_NO_OLDIES to see if your application is using them)
- o SFTP errors can return more specific error codes 
+ o automatically append ";type=<a|i>" when using HTTP proxies for FTP urls
+ o improved NSS support
+ o added --proxy-negotiate
+ o added --post301 and CURLOPT_POST301
+ o builds with c-ares 1.5.0
+ o added CURLOPT_SSH_HOST_PUBLIC_KEY_MD5 and --hostpubmd5
+ o renamed CURLE_SSL_PEER_CERTIFICATE to CURLE_PEER_FAILED_VERIFICATION
+ o added CURLOPT_OPENSOCKETFUNCTION and CURLOPT_OPENSOCKETDATA
+ o CULROPT_COOKIELIST supports "FLUSH"
+ o added CURLOPT_COPYPOSTFIELDS
+ o added --static-libs to curl-config
 
 This release includes the following bugfixes:
 
- o test cases 31, 46, 61, 506, 517 now work in time zones that use leap seconds
- o problem with closed proxy connection during HTTP CONNECT auth negotiation
- o transfer-encoding skipping didn't ignore the 407 response bodies properly
- o CURLOPT_SSL_VERIFYHOST set to 1
- o CONNECT endless loop
- o krb5 support builds with Heimdal
- o added returned error string for connection refused case
- o re-use of dead FTP control connections
- o login to FTP servers that don't require (nor understand) PASS after the
-   USER command
- o bad free of memory from libssh2
- o the SFTP PWD command works
- o HTTP Digest auth on a re-used connection
- o FTPS data connection close
- o AIX 4 and 5 get to use non-blocking sockets
- o small POST with NTLM
- o resumed file:// transfers
- o CURLOPT_DNS_CACHE_TIMEOUT and CURLOPT_DNS_USE_GLOBAL_CACHE are 64 bit
-   "clean"
- o memory leak when handling compressed data streams from broken servers
- o no NTLM unicode response
- o resume HTTP PUT using Digest authentication
- o FTP NOBODY requests on directories sent "SIZE (null)"
- o FTP NOBODY request on file crash
- o excessively long FTP server responses and response lines
- o file:// upload then FTP:// upload crash
- o TFTP error 0 is no longer treated as success
- o uploading empty file over FTP on re-used connection
- o superfluous CWD command on re-used FTP connections without subdirs used
+ o curl-config --protocols now properly reports LDAPS, SCP and SFTP
+ o ldapv3 support on Windows
+ o ldap builds with the MSVC makefiles
+ o no HOME and no key given caused SSH auth failure
+ o Negotiate authentication over proxy
+ o --ftp-method nocwd on directory listings
+ o FTP, CURLOPT_NOBODY enabled and CURLOPT_HEADER disabled now does TYPE
+   before SIZE
+ o re-used handle transfers with SFTP
+ o curl_easy_escape() problem with byte values >= 128
+ o handles chunked-encoded CONNECT responses
+ o misuse of ares_timeout() result
+ o --local-port on TFTP transfers
+ o CURLOPT_POSTFIELDS could fail to send binary data
+ o specifying a proxy with a trailing slash didn't work (unless it also
+   contained a port number)
+ o redirect from HTTP to FTP or TFTP memory problems and leaks
+ o re-used connections a bit too much when using non-SSL protocols tunneled
+   over a HTTP proxy
+ o embed the manifest in VC8 builds
+ o use valgrind in the tests even when the lib is built shared with libtool
+ o libcurl built with NSS can now ignore the peer verification even when the
+   ca cert bundle is absent
 
 This release includes the following known bugs:
 
@@ -60,29 +55,18 @@ This release includes the following known bugs:
 
 Other curl-related news:
 
- o pycurl 7.16.4 was released http://pycurl.sf.net
- o TclCurl 7.16.4 was released
-   http://personal1.iddeo.es/andresgarci/tclcurl/english/
- o curlpp 0.7.1 was released
-   http://rrette.com/textpattern/index.php?s=cURLpp
- o A brand new binding for SP-Forth was written by ygrek:
-   http://www.forth.org.ru/~ac/lib/lin/curl/
+ o 
 
 New curl mirrors:
 
- o http://curl.freeby.pctools.cl is a new mirror in Santiago, Chile
- o http://curl.site2nd.org is a new mirror in Dallas, Texas, USA
- o http://curl.cheap.co.il is a new mirror in Tel-Aviv, Israel
- o http://curl.digimirror.nl is a new mirror in Amsterdam, the Netherlands
+ o http://curl.wetzlmayr.at/ is a new web mirror in Nuremberg, Germany
 
 This release would not have looked like this without help, code, reports and
 advice from friends like these:
 
- Dan Fandrich, Song Ma, Daniel Black, Giancarlo Formicuccia, Shmulik Regev,
- Daniel Cater, Colin Hogben, Jofell Gallardo, Daniel Johnson,
- Ralf S. Engelschall, James Housley, Chris Flerackers, Patrick Monnerat,
- Jayesh A Shah, Greg Zavertnik, Peter O'Gorman, Greg Morse, Dmitriy Sergeyev,
- Scott Cantor, Allen Pulsifer, Andrew Wansink, Robson Braga Araujo,
- Christian Vogt
+ Dan Fandrich, Michal Marek, G<>nter Knauf, Rob Crittenden, Immanuel Gregoire,
+ Mark Davies, Max Katsev, Philip Langdale, Alex Fishman, Johnny Luong,
+ Alexey Pesternikov, Yang Tse, Kim Rinnewitz, Michael Wallner,
+ Patrick Monnerat, Vladimir Lazarenko
  
         Thanks! (and sorry if I forgot to mention someone)

TODO-RELEASE

@@ -1,5 +1,4 @@
-To be addressed before 7.16.3 (planned release: June 2007)
+To be addressed before 7.17.1 (planned release: late October 2007)
 =============================
 
-93 - 
-
+106 - 

acinclude.m4

@@ -171,6 +171,515 @@ AC_DEFUN([CURL_CHECK_HEADER_WS2TCPIP], [
 ])
 
 
+dnl CURL_CHECK_HEADER_WINLDAP
+dnl -------------------------------------------------
+dnl Check for compilable and valid winldap.h header
+
+AC_DEFUN([CURL_CHECK_HEADER_WINLDAP], [
+  AC_REQUIRE([CURL_CHECK_HEADER_WINDOWS])dnl
+  AC_CACHE_CHECK([for winldap.h], [ac_cv_header_winldap_h], [
+    AC_COMPILE_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#endif
+#include <winldap.h>
+      ],[
+#ifdef __CYGWIN__
+        HAVE_WINLDAP_H shall not be defined.
+#else
+        LDAP *ldp = ldap_init("dummy", LDAP_PORT);
+        ULONG res = ldap_unbind(ldp);
+#endif
+      ])
+    ],[
+      ac_cv_header_winldap_h="yes"
+    ],[
+      ac_cv_header_winldap_h="no"
+    ])
+  ])
+  case "$ac_cv_header_winldap_h" in
+    yes)
+      AC_DEFINE_UNQUOTED(HAVE_WINLDAP_H, 1,
+        [Define to 1 if you have the winldap.h header file.])
+      ;;
+  esac
+])
+
+
+dnl CURL_CHECK_HEADER_WINBER
+dnl -------------------------------------------------
+dnl Check for compilable and valid winber.h header
+
+AC_DEFUN([CURL_CHECK_HEADER_WINBER], [
+  AC_REQUIRE([CURL_CHECK_HEADER_WINLDAP])dnl
+  AC_CACHE_CHECK([for winber.h], [ac_cv_header_winber_h], [
+    AC_COMPILE_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#endif
+#include <winldap.h>
+#include <winber.h>
+      ],[
+#ifdef __CYGWIN__
+        HAVE_WINBER_H shall not be defined.
+#else
+        BERVAL *bvp = NULL;
+        BerElement *bep = ber_init(bvp);
+        ber_free(bep, 1);
+#endif
+      ])
+    ],[
+      ac_cv_header_winber_h="yes"
+    ],[
+      ac_cv_header_winber_h="no"
+    ])
+  ])
+  case "$ac_cv_header_winber_h" in
+    yes)
+      AC_DEFINE_UNQUOTED(HAVE_WINBER_H, 1,
+        [Define to 1 if you have the winber.h header file.])
+      ;;
+  esac
+])
+
+
+dnl CURL_CHECK_HEADER_LBER
+dnl -------------------------------------------------
+dnl Check for compilable and valid lber.h header,
+dnl and check if it is needed even with ldap.h
+
+AC_DEFUN([CURL_CHECK_HEADER_LBER], [
+  AC_REQUIRE([CURL_CHECK_HEADER_WINDOWS])dnl
+  AC_CACHE_CHECK([for lber.h], [ac_cv_header_lber_h], [
+    AC_COMPILE_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#else
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#endif
+#ifndef NULL
+#define NULL (void *)0
+#endif
+#include <lber.h>
+      ],[
+        BerValue *bvp = NULL;
+        BerElement *bep = ber_init(bvp);
+        ber_free(bep, 1);
+      ])
+    ],[
+      ac_cv_header_lber_h="yes"
+    ],[
+      ac_cv_header_lber_h="no"
+    ])
+  ])
+  if test "$ac_cv_header_lber_h" = "yes"; then
+    AC_DEFINE_UNQUOTED(HAVE_LBER_H, 1,
+      [Define to 1 if you have the lber.h header file.])
+    #
+    AC_COMPILE_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#else
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#endif
+#ifndef NULL
+#define NULL (void *)0
+#endif
+#ifndef LDAP_DEPRECATED
+#define LDAP_DEPRECATED 1
+#endif
+#include <ldap.h>
+      ],[
+        BerValue *bvp = NULL;
+        BerElement *bep = ber_init(bvp);
+        ber_free(bep, 1);
+      ])
+    ],[
+      curl_cv_need_header_lber_h="no"
+    ],[
+      curl_cv_need_header_lber_h="yes"
+    ])
+    #
+    case "$curl_cv_need_header_lber_h" in
+      yes)
+        AC_DEFINE_UNQUOTED(NEED_LBER_H, 1,
+          [Define to 1 if you need the lber.h header file even with ldap.h])
+        ;;
+    esac
+  fi
+])
+
+
+dnl CURL_CHECK_HEADER_LDAP
+dnl -------------------------------------------------
+dnl Check for compilable and valid ldap.h header
+
+AC_DEFUN([CURL_CHECK_HEADER_LDAP], [
+  AC_REQUIRE([CURL_CHECK_HEADER_LBER])dnl
+  AC_CACHE_CHECK([for ldap.h], [ac_cv_header_ldap_h], [
+    AC_COMPILE_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#else
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#endif
+#ifndef LDAP_DEPRECATED
+#define LDAP_DEPRECATED 1
+#endif
+#ifdef NEED_LBER_H
+#include <lber.h>
+#endif
+#include <ldap.h>
+      ],[
+        LDAP *ldp = ldap_init("dummy", LDAP_PORT);
+        int res = ldap_unbind(ldp);
+      ])
+    ],[
+      ac_cv_header_ldap_h="yes"
+    ],[
+      ac_cv_header_ldap_h="no"
+    ])
+  ])
+  case "$ac_cv_header_ldap_h" in
+    yes)
+      AC_DEFINE_UNQUOTED(HAVE_LDAP_H, 1,
+        [Define to 1 if you have the ldap.h header file.])
+      ;;
+  esac
+])
+
+
+dnl CURL_CHECK_HEADER_LDAP_SSL
+dnl -------------------------------------------------
+dnl Check for compilable and valid ldap_ssl.h header
+
+AC_DEFUN([CURL_CHECK_HEADER_LDAP_SSL], [
+  AC_REQUIRE([CURL_CHECK_HEADER_LDAP])dnl
+  AC_CACHE_CHECK([for ldap_ssl.h], [ac_cv_header_ldap_ssl_h], [
+    AC_COMPILE_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#else
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#endif
+#ifndef LDAP_DEPRECATED
+#define LDAP_DEPRECATED 1
+#endif
+#ifdef NEED_LBER_H
+#include <lber.h>
+#endif
+#ifdef HAVE_LDAP_H
+#include <ldap.h>
+#endif
+#include <ldap_ssl.h>
+      ],[
+        LDAP *ldp = ldapssl_init("dummy", LDAPS_PORT, 1);
+      ])
+    ],[
+      ac_cv_header_ldap_ssl_h="yes"
+    ],[
+      ac_cv_header_ldap_ssl_h="no"
+    ])
+  ])
+  case "$ac_cv_header_ldap_ssl_h" in
+    yes)
+      AC_DEFINE_UNQUOTED(HAVE_LDAP_SSL_H, 1,
+        [Define to 1 if you have the ldap_ssl.h header file.])
+      ;;
+  esac
+])
+
+
+dnl CURL_CHECK_HEADER_LDAPSSL
+dnl -------------------------------------------------
+dnl Check for compilable and valid ldapssl.h header
+
+AC_DEFUN([CURL_CHECK_HEADER_LDAPSSL], [
+  AC_REQUIRE([CURL_CHECK_HEADER_LDAP])dnl
+  AC_CACHE_CHECK([for ldapssl.h], [ac_cv_header_ldapssl_h], [
+    AC_COMPILE_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#else
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#endif
+#ifndef NULL
+#define NULL (void *)0
+#endif
+#ifndef LDAP_DEPRECATED
+#define LDAP_DEPRECATED 1
+#endif
+#ifdef NEED_LBER_H
+#include <lber.h>
+#endif
+#ifdef HAVE_LDAP_H
+#include <ldap.h>
+#endif
+#include <ldapssl.h>
+      ],[
+        char *cert_label = NULL;
+        LDAP *ldp = ldap_ssl_init("dummy", LDAPS_PORT, cert_label);
+      ])
+    ],[
+      ac_cv_header_ldapssl_h="yes"
+    ],[
+      ac_cv_header_ldapssl_h="no"
+    ])
+  ])
+  case "$ac_cv_header_ldapssl_h" in
+    yes)
+      AC_DEFINE_UNQUOTED(HAVE_LDAPSSL_H, 1,
+        [Define to 1 if you have the ldapssl.h header file.])
+      ;;
+  esac
+])
+
+
+dnl CURL_CHECK_LIBS_WINLDAP
+dnl -------------------------------------------------
+dnl Check for libraries needed for WINLDAP support,
+dnl and prepended to LIBS any needed libraries.
+dnl This macro can take an optional parameter with a
+dnl white space separated list of libraries to check
+dnl before the WINLDAP default ones.
+
+AC_DEFUN([CURL_CHECK_LIBS_WINLDAP], [
+  AC_REQUIRE([CURL_CHECK_HEADER_WINBER])dnl
+  #
+  AC_MSG_CHECKING([for WINLDAP libraries])
+  #
+  u_libs=""
+  #
+  ifelse($1,,,[
+    for x_lib in $1; do
+      case "$x_lib" in
+        -l*)
+          l_lib="$x_lib"
+          ;;
+        *)
+          l_lib="-l$x_lib"
+          ;;
+      esac
+      if test -z "$u_libs"; then
+        u_libs="$l_lib"
+      else
+        u_libs="$u_libs $l_lib"
+      fi
+    done
+  ])
+  #
+  curl_cv_save_LIBS="$LIBS"
+  curl_cv_ldap_LIBS="unknown"
+  #
+  for x_nlibs in '' "$u_libs" \
+    '-lwldap32' ; do
+    if test -z "$x_nlibs"; then
+      LIBS="$curl_cv_save_LIBS"
+    else
+      LIBS="$x_nlibs $curl_cv_save_LIBS"
+    fi
+    AC_LINK_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#ifdef HAVE_WINLDAP_H
+#include <winldap.h>
+#endif
+#ifdef HAVE_WINBER_H
+#include <winber.h>
+#endif
+#endif
+      ],[
+        BERVAL *bvp = NULL;
+        BerElement *bep = ber_init(bvp);
+        LDAP *ldp = ldap_init("dummy", LDAP_PORT);
+        ULONG res = ldap_unbind(ldp);
+        ber_free(bep, 1);
+      ])
+    ],[
+       curl_cv_ldap_LIBS="$x_nlibs"
+       break
+    ])
+  done
+  #
+  LIBS="$curl_cv_save_LIBS"
+  #
+  case X-"$curl_cv_ldap_LIBS" in
+    X-unknown)
+      AC_MSG_RESULT([cannot find WINLDAP libraries])
+      ;;
+    X-)
+      AC_MSG_RESULT([no additional lib required])
+      ;;
+    *)
+      if test -z "$curl_cv_save_LIBS"; then
+        LIBS="$curl_cv_ldap_LIBS"
+      else
+        LIBS="$curl_cv_ldap_LIBS $curl_cv_save_LIBS"
+      fi
+      AC_MSG_RESULT([$curl_cv_ldap_LIBS])
+      ;;
+  esac
+  #
+])
+
+
+dnl CURL_CHECK_LIBS_LDAP
+dnl -------------------------------------------------
+dnl Check for libraries needed for LDAP support,
+dnl and prepended to LIBS any needed libraries.
+dnl This macro can take an optional parameter with a
+dnl white space separated list of libraries to check
+dnl before the default ones.
+
+AC_DEFUN([CURL_CHECK_LIBS_LDAP], [
+  AC_REQUIRE([CURL_CHECK_HEADER_LDAP])dnl
+  #
+  AC_MSG_CHECKING([for LDAP libraries])
+  #
+  u_libs=""
+  #
+  ifelse($1,,,[
+    for x_lib in $1; do
+      case "$x_lib" in
+        -l*)
+          l_lib="$x_lib"
+          ;;
+        *)
+          l_lib="-l$x_lib"
+          ;;
+      esac
+      if test -z "$u_libs"; then
+        u_libs="$l_lib"
+      else
+        u_libs="$u_libs $l_lib"
+      fi
+    done
+  ])
+  #
+  curl_cv_save_LIBS="$LIBS"
+  curl_cv_ldap_LIBS="unknown"
+  #
+  for x_nlibs in '' "$u_libs" \
+    '-lldap' \
+    '-llber -lldap' \
+    '-lldap -llber' \
+    '-lldapssl -lldapx -lldapsdk' \
+    '-lldapsdk -lldapx -lldapssl' ; do
+    if test -z "$x_nlibs"; then
+      LIBS="$curl_cv_save_LIBS"
+    else
+      LIBS="$x_nlibs $curl_cv_save_LIBS"
+    fi
+    AC_LINK_IFELSE([
+      AC_LANG_PROGRAM([
+#undef inline
+#ifdef HAVE_WINDOWS_H
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#else
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#endif
+#ifndef NULL
+#define NULL (void *)0
+#endif
+#ifndef LDAP_DEPRECATED
+#define LDAP_DEPRECATED 1
+#endif
+#ifdef NEED_LBER_H
+#include <lber.h>
+#endif
+#ifdef HAVE_LDAP_H
+#include <ldap.h>
+#endif
+      ],[
+        BerValue *bvp = NULL;
+        BerElement *bep = ber_init(bvp);
+        LDAP *ldp = ldap_init("dummy", LDAP_PORT);
+        int res = ldap_unbind(ldp);
+        ber_free(bep, 1);
+      ])
+    ],[
+       curl_cv_ldap_LIBS="$x_nlibs"
+       break
+    ])
+  done
+  #
+  LIBS="$curl_cv_save_LIBS"
+  #
+  case X-"$curl_cv_ldap_LIBS" in
+    X-unknown)
+      AC_MSG_RESULT([cannot find LDAP libraries])
+      ;;
+    X-)
+      AC_MSG_RESULT([no additional lib required])
+      ;;
+    *)
+      if test -z "$curl_cv_save_LIBS"; then
+        LIBS="$curl_cv_ldap_LIBS"
+      else
+        LIBS="$curl_cv_ldap_LIBS $curl_cv_save_LIBS"
+      fi
+      AC_MSG_RESULT([$curl_cv_ldap_LIBS])
+      ;;
+  esac
+  #
+])
+
+
 dnl CURL_CHECK_HEADER_MALLOC
 dnl -------------------------------------------------
 dnl Check for compilable and valid malloc.h header,
@@ -1316,6 +1825,9 @@ AC_TRY_RUN([
 #include <string.h>
 #include <sys/types.h>
 #include <netdb.h>
+#ifndef NULL
+#define NULL (void *)0
+#endif
 
 int
 main () {
@@ -1762,7 +2274,7 @@ AC_DEFUN([CURL_CC_DEBUG_OPTS],
            dnl only if the compiler is newer than 2.95 since we got lots of
            dnl "`_POSIX_C_SOURCE' is not defined" in system headers with
            dnl gcc 2.95.4 on FreeBSD 4.9!
-           WARN="$WARN -Wundef -Wno-long-long -Wsign-compare"
+           WARN="$WARN -Wundef -Wno-long-long -Wsign-compare -Wshadow -Wno-multichar"
          fi
 
          if test "$gccnum" -ge "296"; then

ares/CHANGES

@@ -1,10 +1,55 @@
   Changelog for the c-ares project
 
+* October 2 2007 (Daniel Stenberg)
+
+- ares_strerror() segfaulted if the input error number was out of the currently
+  supported range.
+
+- Yang Tse: Avoid a segfault when generating a DNS "Transaction ID" in
+  internal function init_id_key() under low memory conditions.
+
+* September 28 2007 (Daniel Stenberg)
+
+- Bumped version to 1.5.0 for next release and soname bumped to 2 due to ABI
+  and API changes in the progress callback (and possibly more coming up from
+  Steinar)
+
+* September 28 2007 (Steinar H. Gunderson)
+
+- Don't skip a server if it's the only one. (Bugfix from the Google tree.)
+
+- Made the query callbacks receive the number of timeouts that happened during
+  the execution of a query, and updated documentation accordingly. (Patch from
+  the Google tree.)
+
+- Support a few more socket options: ARES_OPT_SOCK_SNDBUF and
+  ARES_OPT_SOCK_RCVBUF
+
+- Always register for TCP events even if there are no outstanding queries, as
+  the other side could always close the connection, which is a valid event
+  which should be responded to.
+
+* September 22 2007 (Daniel Stenberg)
+
+- Steinar H. Gunderson fixed: Correctly clear sockets from the fd_set on in
+  several functions (write_tcp_data, read_tcp_data, read_udp_packets) so that
+  if it fails and the socket is closed the following code doesn't try to use
+  the file descriptor.
+
+- Steinar H. Gunderson modified c-ares to now also do to DNS retries even when
+  TCP is used since there are several edge cases where it still makes sense.
+
+- Brad House provided a fix for ares_save_options():
+
+  Apparently I overlooked something with the ares_save_options() where it
+  would try to do a malloc(0) when no options of that type needed to be saved.
+  On most platforms, this was fine because malloc(0) doesn't actually return
+  NULL, but on AIX it does, so ares_save_options would return ARES_ENOMEM.
+
 * July 14 2007 (Daniel Stenberg)
 
 - Vlad Dinulescu fixed two outstanding valgrind reports:
  
- 
   1. In ares_query.c , in find_query_by_id we compare q->qid (which is a short
   int variable) with qid, which is declared as an int variable.  Moreover,
   DNS_HEADER_SET_QID is used to set the value of qid, but DNS_HEADER_SET_QID

ares/Makefile.am

@@ -14,7 +14,7 @@ EXTRA_DIST = CHANGES README.cares Makefile.inc adig.c ahost.c $(man_MANS) \
  $(MSVCFILES) AUTHORS config-win32.h RELEASE-NOTES libcares.pc.in
 
 
-VER=-version-info 1:0:0
+VER=-version-info 2:0:0
 # This flag accepts an argument of the form current[:revision[:age]]. So,
 # passing -version-info 3:12:1 sets current to 3, revision to 12, and age to
 # 1.

ares/Makefile.dj

@@ -14,7 +14,8 @@ CFLAGS += -DWATT32 -DHAVE_AF_INET6 -DHAVE_PF_INET6 -DHAVE_FIONBIO \
           -DHAVE_STRUCT_IN6_ADDR -DHAVE_SOCKADDR_IN6_SIN6_SCOPE_ID \
           -DHAVE_SYS_TIME_H -DHAVE_STRUCT_SOCKADDR_IN6 -DHAVE_STRUCT_ADDRINFO \
           -DHAVE_SIGNAL_H -DHAVE_SIG_ATOMIC_T -DRETSIGTYPE='void' -DHAVE_PROCESS_H \
-          -DHAVE_ARPA_NAMESER_H -DNS_INADDRSZ=4 -DHAVE_RECV -DHAVE_SEND \
+          -DHAVE_ARPA_NAMESER_H -DHAVE_SYS_SOCKET_H -DHAVE_SYS_UIO_H -DHAVE_NETINET_IN_H \
+          -DHAVE_NETINET_TCP_H -DNS_INADDRSZ=4 -DHAVE_RECV -DHAVE_SEND \
           -DSEND_TYPE_ARG1='int'   -DSEND_QUAL_ARG2='const' \
           -DSEND_TYPE_ARG2='void*' -DSEND_TYPE_ARG3='int' \
           -DSEND_TYPE_ARG4='int'   -DSEND_TYPE_RETV='int' \

ares/Makefile.inc

@@ -6,7 +6,7 @@ ares_timeout.c ares_destroy.c ares_mkquery.c ares_version.c		\
 ares_expand_name.c ares_parse_a_reply.c windows_port.c			\
 ares_expand_string.c ares_parse_ptr_reply.c ares_parse_aaaa_reply.c	\
 ares_getnameinfo.c inet_net_pton.c bitncmp.c inet_ntop.c		\
-ares_parse_ns_reply.c
+ares_parse_ns_reply.c ares_llist.c
 
 HHEADERS = ares.h ares_private.h setup.h ares_dns.h ares_version.h          \
            nameser.h inet_net_pton.h inet_ntop.h ares_ipv6.h bitncmp.h      \

ares/Makefile.vc6

@@ -60,6 +60,7 @@ OBJECTS = $(OBJ_DIR)\ares_fds.obj              \
           $(OBJ_DIR)\ares_strerror.obj         \
           $(OBJ_DIR)\ares_cancel.obj           \
           $(OBJ_DIR)\ares_init.obj             \
+          $(OBJ_DIR)\ares_llist.obj            \
           $(OBJ_DIR)\ares_timeout.obj          \
           $(OBJ_DIR)\ares_destroy.obj          \
           $(OBJ_DIR)\ares_mkquery.obj          \
@@ -231,3 +232,6 @@ $(OBJ_DIR)\inet_ntop.obj: inet_ntop.c setup.h setup_once.h nameser.h           \
   ares_ipv6.h inet_ntop.h
 
 $(OBJ_DIR)\ares_getopt.obj: ares_getopt.c ares_getopt.h
+
+$(OBJ_DIR)\ares_llist.obj: ares_llist.c setup.h setup_once.h ares.h            \
+  ares_private.h ares_llist.h

ares/acinclude.m4

@@ -1085,7 +1085,7 @@ AC_DEFUN([CURL_CHECK_NONBLOCKING_SOCKET],
 #  define PLATFORM_SUNOS4
 # endif
 #endif
-#if (defined(_AIX) || defined(__xlC__)) && !defined(_AIX4)
+#if (defined(_AIX) || defined(__xlC__)) && !defined(_AIX41)
 # define PLATFORM_AIX_V3
 #endif
 
@@ -1369,7 +1369,7 @@ AC_DEFUN([CURL_CC_DEBUG_OPTS],
            dnl only if the compiler is newer than 2.95 since we got lots of
            dnl "`_POSIX_C_SOURCE' is not defined" in system headers with
            dnl gcc 2.95.4 on FreeBSD 4.9!
-           WARN="$WARN -Wundef -Wno-long-long -Wsign-compare"
+           WARN="$WARN -Wundef -Wno-long-long -Wsign-compare -Wshadow -Wno-multichar"
          fi
 
          if test "$gccnum" -ge "296"; then

ares/adig.c

@@ -127,7 +127,8 @@ static const char *rcodes[] = {
   "(unknown)", "(unknown)", "(unknown)", "(unknown)", "NOCHANGE"
 };
 
-static void callback(void *arg, int status, unsigned char *abuf, int alen);
+static void callback(void *arg, int status, int timeouts,
+                     unsigned char *abuf, int alen);
 static const unsigned char *display_question(const unsigned char *aptr,
                                              const unsigned char *abuf,
                                              int alen);
@@ -294,7 +295,8 @@ int main(int argc, char **argv)
   return 0;
 }
 
-static void callback(void *arg, int status, unsigned char *abuf, int alen)
+static void callback(void *arg, int status, int timeouts,
+                     unsigned char *abuf, int alen)
 {
   char *name = (char *) arg;
   int id, qr, opcode, aa, tc, rd, ra, rcode;

ares/ahost.c

@@ -47,7 +47,7 @@ struct in6_addr
 };
 #endif
 
-static void callback(void *arg, int status, struct hostent *host);
+static void callback(void *arg, int status, int timeouts, struct hostent *host);
 static void usage(void);
 
 int main(int argc, char **argv)
@@ -142,7 +142,7 @@ int main(int argc, char **argv)
   return 0;
 }
 
-static void callback(void *arg, int status, struct hostent *host)
+static void callback(void *arg, int status, int timeouts, struct hostent *host)
 {
   char **p;
 

ares/ares.h

@@ -98,6 +98,8 @@ extern "C" {
 #define ARES_OPT_LOOKUPS        (1 << 8)
 #define ARES_OPT_SOCK_STATE_CB  (1 << 9)
 #define ARES_OPT_SORTLIST       (1 << 10)
+#define ARES_OPT_SOCK_SNDBUF    (1 << 11)
+#define ARES_OPT_SOCK_RCVBUF    (1 << 12)
 
 /* Nameinfo flag values */
 #define ARES_NI_NOFQDN                  (1 << 0)
@@ -156,17 +158,10 @@ typedef int ares_socket_t;
 #define ares_socket_typedef
 #endif /* ares_socket_typedef */
 
-#ifdef WIN32
 typedef void (*ares_sock_state_cb)(void *data,
-                                   SOCKET socket,
+                                   ares_socket_t socket_fd,
                                    int readable,
                                    int writable);
-#else
-typedef void (*ares_sock_state_cb)(void *data,
-                                   int socket,
-                                   int readable,
-                                   int writable);
-#endif
 
 struct apattern;
 
@@ -177,6 +172,8 @@ struct ares_options {
   int ndots;
   unsigned short udp_port;
   unsigned short tcp_port;
+  int socket_send_buffer_size;
+  int socket_receive_buffer_size;
   struct in_addr *servers;
   int nservers;
   char **domains;
@@ -193,11 +190,11 @@ struct timeval;
 struct sockaddr;
 struct ares_channeldata;
 typedef struct ares_channeldata *ares_channel;
-typedef void (*ares_callback)(void *arg, int status, unsigned char *abuf,
-                              int alen);
-typedef void (*ares_host_callback)(void *arg, int status,
+typedef void (*ares_callback)(void *arg, int status, int timeouts,
+                              unsigned char *abuf, int alen);
+typedef void (*ares_host_callback)(void *arg, int status, int timeouts,
                                    struct hostent *hostent);
-typedef void (*ares_nameinfo_callback)(void *arg, int status,
+typedef void (*ares_nameinfo_callback)(void *arg, int status, int timeouts,
                                        char *node, char *service);
 
 int ares_init(ares_channel *channelptr);

ares/ares__close_sockets.c

@@ -35,6 +35,8 @@ void ares__close_sockets(ares_channel channel, struct server_state *server)
       /* Advance server->qhead; pull out query as we go. */
       sendreq = server->qhead;
       server->qhead = sendreq->next;
+      if (sendreq->data_storage != NULL)
+        free(sendreq->data_storage);
       free(sendreq);
     }
   server->qtail = NULL;
@@ -45,12 +47,16 @@ void ares__close_sockets(ares_channel channel, struct server_state *server)
   server->tcp_buffer = NULL;
   server->tcp_lenbuf_pos = 0;
 
+  /* Reset brokenness */
+  server->is_broken = 0;
+
   /* Close the TCP and UDP sockets. */
   if (server->tcp_socket != ARES_SOCKET_BAD)
     {
       SOCK_STATE_CALLBACK(channel, server->tcp_socket, 0, 0);
       closesocket(server->tcp_socket);
       server->tcp_socket = ARES_SOCKET_BAD;
+      server->tcp_connection_generation = ++channel->tcp_connection_generation;
     }
   if (server->udp_socket != ARES_SOCKET_BAD)
     {

ares/ares_cancel.c

@@ -14,29 +14,45 @@
  */
 
 #include "setup.h"
+#include <assert.h>
 #include <stdlib.h>
 #include "ares.h"
 #include "ares_private.h"
 
 /*
- * ares_cancel() cancels a ongoing request/resolve that might be going on on
- * the given channel. It does NOT kill the channel, use ares_destroy() for
+ * ares_cancel() cancels all ongoing requests/resolves that might be going on
+ * on the given channel. It does NOT kill the channel, use ares_destroy() for
  * that.
  */
 void ares_cancel(ares_channel channel)
 {
-  struct query *query, *next;
+  struct query *query;
+  struct list_node* list_head;
+  struct list_node* list_node;
   int i;
 
-  for (query = channel->queries; query; query = next)
+  list_head = &(channel->all_queries);
+  for (list_node = list_head->next; list_node != list_head; )
   {
-    next = query->next;
-    query->callback(query->arg, ARES_ETIMEOUT, NULL, 0);
-    free(query->tcpbuf);
-    free(query->skip_server);
-    free(query);
+    query = list_node->data;
+    list_node = list_node->next;  /* since we're deleting the query */
+    query->callback(query->arg, ARES_ETIMEOUT, 0, NULL, 0);
+    ares__free_query(query);
   }
-  channel->queries = NULL;
+#ifndef NDEBUG
+  /* Freeing the query should remove it from all the lists in which it sits,
+   * so all query lists should be empty now.
+   */
+  assert(ares__is_list_empty(&(channel->all_queries)));
+  for (i = 0; i < ARES_QID_TABLE_SIZE; i++)
+    {
+      assert(ares__is_list_empty(&(channel->queries_by_qid[i])));
+    }
+  for (i = 0; i < ARES_TIMEOUT_TABLE_SIZE; i++)
+    {
+      assert(ares__is_list_empty(&(channel->queries_by_timeout[i])));
+    }
+#endif
   if (!(channel->flags & ARES_FLAG_STAYOPEN))
   {
     if (channel->servers)

ares/ares_destroy.c

@@ -16,6 +16,7 @@
  */
 
 #include "setup.h"
+#include <assert.h>
 #include <stdlib.h>
 #include "ares.h"
 #include "ares_private.h"
@@ -37,13 +38,42 @@ void ares_destroy(ares_channel channel)
 {
   int i;
   struct query *query;
+  struct list_node* list_head;
+  struct list_node* list_node;
   
   if (!channel)
     return;
 
+  list_head = &(channel->all_queries);
+  for (list_node = list_head->next; list_node != list_head; )
+    {
+      query = list_node->data;
+      list_node = list_node->next;  /* since we're deleting the query */
+      query->callback(query->arg, ARES_EDESTRUCTION, 0, NULL, 0);
+      ares__free_query(query);
+    }
+#ifndef NDEBUG
+  /* Freeing the query should remove it from all the lists in which it sits,
+   * so all query lists should be empty now.
+   */
+  assert(ares__is_list_empty(&(channel->all_queries)));
+  for (i = 0; i < ARES_QID_TABLE_SIZE; i++)
+    {
+      assert(ares__is_list_empty(&(channel->queries_by_qid[i])));
+    }
+  for (i = 0; i < ARES_TIMEOUT_TABLE_SIZE; i++)
+    {
+      assert(ares__is_list_empty(&(channel->queries_by_timeout[i])));
+    }
+#endif
+
   if (channel->servers) {
     for (i = 0; i < channel->nservers; i++)
-      ares__close_sockets(channel, &channel->servers[i]);
+      {
+        struct server_state *server = &channel->servers[i];
+        ares__close_sockets(channel, server);
+        assert(ares__is_list_empty(&(server->queries_to_server)));
+      }
     free(channel->servers);
   }
 
@@ -59,16 +89,5 @@ void ares_destroy(ares_channel channel)
   if (channel->lookups)
     free(channel->lookups);
 
-  while (channel->queries) {
-    query = channel->queries;
-    channel->queries = query->next;
-    query->callback(query->arg, ARES_EDESTRUCTION, NULL, 0);
-    if (query->tcpbuf)
-      free(query->tcpbuf);
-    if (query->skip_server)
-      free(query->skip_server);
-    free(query);
-  }
-
   free(channel);
 }

ares/ares_expand_name.c

@@ -74,6 +74,15 @@ int ares_expand_name(const unsigned char *encoded, const unsigned char *abuf,
     return ARES_ENOMEM;
   q = *s;
 
+  if (len == 0) {
+    /* RFC2181 says this should be ".": the root of the DNS tree.
+     * Since this function strips trailing dots though, it becomes ""
+     */
+    q[0] = '\0';
+    *enclen = 1;  /* the caller should move one byte to get past this */
+    return ARES_SUCCESS;
+  }
+
   /* No error-checking necessary; it was all done by name_length(). */
   p = encoded;
   while (*p)

ares/ares_expand_string.3

@@ -28,7 +28,7 @@ ares_expand_string \- Expand a length encoded string
 .SH DESCRIPTION
 The
 .B ares_expand_string
-function converts a length encoded string to a NULL terminated C
+function converts a length encoded string to a NUL-terminated C
 string.  The argument
 .I encoded
 gives the beginning of the encoded string, and the arguments

ares/ares_fds.c

@@ -30,20 +30,26 @@ int ares_fds(ares_channel channel, fd_set *read_fds, fd_set *write_fds)
   ares_socket_t nfds;
   int i;
 
-  /* No queries, no file descriptors. */
-  if (!channel->queries)
-    return 0;
+  /* Are there any active queries? */
+  int active_queries = !ares__is_list_empty(&(channel->all_queries));
 
   nfds = 0;
   for (i = 0; i < channel->nservers; i++)
     {
       server = &channel->servers[i];
-      if (server->udp_socket != ARES_SOCKET_BAD)
+      /* We only need to register interest in UDP sockets if we have
+       * outstanding queries.
+       */
+      if (active_queries && server->udp_socket != ARES_SOCKET_BAD)
         {
           FD_SET(server->udp_socket, read_fds);
           if (server->udp_socket >= nfds)
             nfds = server->udp_socket + 1;
         }
+      /* We always register for TCP events, because we want to know
+       * when the other side closes the connection, so we don't waste
+       * time trying to use a broken connection.
+       */
       if (server->tcp_socket != ARES_SOCKET_BAD)
        {
          FD_SET(server->tcp_socket, read_fds);

ares/ares_gethostbyaddr.3

@@ -22,7 +22,7 @@ ares_gethostbyaddr \- Initiate a host query by address
 .B #include <ares.h>
 .PP
 .B typedef void (*ares_host_callback)(void *\fIarg\fP, int \fIstatus\fP,
-.B	struct hostent *\fIhostent\fP)
+.B	int \fItimeouts\fP, struct hostent *\fIhostent\fP)
 .PP
 .B void ares_gethostbyaddr(ares_channel \fIchannel\fP, const void *\fIaddr\fP,
 .B 	int \fIaddrlen\fP, int \fIfamily\fP, ares_host_callback \fIcallback\fP,
@@ -76,6 +76,11 @@ The name service channel
 .I channel
 is being destroyed; the query will not be completed.
 .PP
+The callback argument
+.I timeouts
+reports how many times a query timed out during the execution of the
+given request.
+.PP
 On successful completion of the query, the callback argument
 .I hostent
 points to a

ares/ares_gethostbyaddr.c

@@ -49,11 +49,12 @@ struct addr_query {
   void *arg;
 
   const char *remaining_lookups;
+  int timeouts;
 };
 
 static void next_lookup(struct addr_query *aquery);
-static void addr_callback(void *arg, int status, unsigned char *abuf,
-                          int alen);
+static void addr_callback(void *arg, int status, int timeouts,
+                          unsigned char *abuf, int alen);
 static void end_aquery(struct addr_query *aquery, int status,
                        struct hostent *host);
 static int file_lookup(union ares_addr *addr, int family, struct hostent **host);
@@ -65,21 +66,21 @@ void ares_gethostbyaddr(ares_channel channel, const void *addr, int addrlen,
 
   if (family != AF_INET && family != AF_INET6)
     {
-      callback(arg, ARES_ENOTIMP, NULL);
+      callback(arg, ARES_ENOTIMP, 0, NULL);
       return;
     }
 
   if ((family == AF_INET && addrlen != sizeof(struct in_addr)) ||
       (family == AF_INET6 && addrlen != sizeof(struct in6_addr)))
     {
-      callback(arg, ARES_ENOTIMP, NULL);
+      callback(arg, ARES_ENOTIMP, 0, NULL);
       return;
     }
 
   aquery = malloc(sizeof(struct addr_query));
   if (!aquery)
     {
-      callback(arg, ARES_ENOMEM, NULL);
+      callback(arg, ARES_ENOMEM, 0, NULL);
       return;
     }
   aquery->channel = channel;
@@ -91,6 +92,7 @@ void ares_gethostbyaddr(ares_channel channel, const void *addr, int addrlen,
   aquery->callback = callback;
   aquery->arg = arg;
   aquery->remaining_lookups = channel->lookups;
+  aquery->timeouts = 0;
 
   next_lookup(aquery);
 }
@@ -151,11 +153,13 @@ static void next_lookup(struct addr_query *aquery)
   end_aquery(aquery, ARES_ENOTFOUND, NULL);
 }
 
-static void addr_callback(void *arg, int status, unsigned char *abuf, int alen)
+static void addr_callback(void *arg, int status, int timeouts,
+                          unsigned char *abuf, int alen)
 {
   struct addr_query *aquery = (struct addr_query *) arg;
   struct hostent *host;
 
+  aquery->timeouts += timeouts;
   if (status == ARES_SUCCESS)
     {
       if (aquery->family == AF_INET)
@@ -175,7 +179,7 @@ static void addr_callback(void *arg, int status, unsigned char *abuf, int alen)
 static void end_aquery(struct addr_query *aquery, int status,
                        struct hostent *host)
 {
-  aquery->callback(aquery->arg, status, host);
+  aquery->callback(aquery->arg, status, aquery->timeouts, host);
   if (host)
     ares_free_hostent(host);
   free(aquery);

ares/ares_gethostbyname.3

@@ -22,7 +22,7 @@ ares_gethostbyname \- Initiate a host query by name
 .B #include <ares.h>
 .PP
 .B typedef void (*ares_host_callback)(void *\fIarg\fP, int \fIstatus\fP,
-.B	struct hostent *\fIhostent\fP)
+.B	int \fItimeouts\fP, struct hostent *\fIhostent\fP)
 .PP
 .B void ares_gethostbyname(ares_channel \fIchannel\fP, const char *\fIname\fP,
 .B 	int \fIfamily\fP, ares_host_callback \fIcallback\fP, void *\fIarg\fP)
@@ -80,6 +80,11 @@ The name service channel
 .I channel
 is being destroyed; the query will not be completed.
 .PP
+The callback argument
+.I timeouts
+reports how many times a query timed out during the execution of the
+given request.
+.PP
 On successful completion of the query, the callback argument
 .I hostent
 points to a

ares/ares_gethostbyname.c

@@ -54,11 +54,12 @@ struct host_query {
   void *arg;
   int family;
   const char *remaining_lookups;
+  int timeouts;
 };
 
-static void next_lookup(struct host_query *hquery);
-static void host_callback(void *arg, int status, unsigned char *abuf,
-                          int alen);
+static void next_lookup(struct host_query *hquery, int status);
+static void host_callback(void *arg, int status, int timeouts,
+                          unsigned char *abuf, int alen);
 static void end_hquery(struct host_query *hquery, int status,
                        struct hostent *host);
 static int fake_hostent(const char *name, int family, ares_host_callback callback,
@@ -81,7 +82,7 @@ void ares_gethostbyname(ares_channel channel, const char *name, int family,
   /* Right now we only know how to look up Internet addresses. */
   if (family != AF_INET && family != AF_INET6)
     {
-      callback(arg, ARES_ENOTIMP, NULL);
+      callback(arg, ARES_ENOTIMP, 0, NULL);
       return;
     }
 
@@ -92,7 +93,7 @@ void ares_gethostbyname(ares_channel channel, const char *name, int family,
   hquery = malloc(sizeof(struct host_query));
   if (!hquery)
     {
-      callback(arg, ARES_ENOMEM, NULL);
+      callback(arg, ARES_ENOMEM, 0, NULL);
       return;
     }
   hquery->channel = channel;
@@ -101,20 +102,20 @@ void ares_gethostbyname(ares_channel channel, const char *name, int family,
   if (!hquery->name)
     {
       free(hquery);
-      callback(arg, ARES_ENOMEM, NULL);
+      callback(arg, ARES_ENOMEM, 0, NULL);
       return;
     }
   hquery->callback = callback;
   hquery->arg = arg;
   hquery->remaining_lookups = channel->lookups;
+  hquery->timeouts = 0;
 
   /* Start performing lookups according to channel->lookups. */
-  next_lookup(hquery);
+  next_lookup(hquery, ARES_SUCCESS);
 }
 
-static void next_lookup(struct host_query *hquery)
+static void next_lookup(struct host_query *hquery, int status)
 {
-  int status;
   const char *p;
   struct hostent *host;
 
@@ -126,8 +127,8 @@ static void next_lookup(struct host_query *hquery)
           /* DNS lookup */
           hquery->remaining_lookups = p + 1;
           if (hquery->family == AF_INET6)
-            ares_search(hquery->channel, hquery->name, C_IN, T_AAAA, host_callback,
-                        hquery);
+            ares_search(hquery->channel, hquery->name, C_IN, T_AAAA,
+                        host_callback, hquery);
           else
             ares_search(hquery->channel, hquery->name, C_IN, T_A, host_callback,
                         hquery);
@@ -144,15 +145,17 @@ static void next_lookup(struct host_query *hquery)
           break;
         }
     }
-  end_hquery(hquery, ARES_ENOTFOUND, NULL);
+  end_hquery(hquery, status, NULL);
 }
 
-static void host_callback(void *arg, int status, unsigned char *abuf, int alen)
+static void host_callback(void *arg, int status, int timeouts,
+                          unsigned char *abuf, int alen)
 {
   struct host_query *hquery = (struct host_query *) arg;
   ares_channel channel = hquery->channel;
   struct hostent *host;
 
+  hquery->timeouts += timeouts;
   if (status == ARES_SUCCESS)
     {
       if (hquery->family == AF_INET)
@@ -179,13 +182,13 @@ static void host_callback(void *arg, int status, unsigned char *abuf, int alen)
   else if (status == ARES_EDESTRUCTION)
     end_hquery(hquery, status, NULL);
   else
-    next_lookup(hquery);
+    next_lookup(hquery, status);
 }
 
 static void end_hquery(struct host_query *hquery, int status,
                        struct hostent *host)
 {
-  hquery->callback(hquery->arg, status, host);
+  hquery->callback(hquery->arg, status, hquery->timeouts, host);
   if (host)
     ares_free_hostent(host);
   free(hquery->name);
@@ -206,7 +209,27 @@ static int fake_hostent(const char *name, int family, ares_host_callback callbac
   struct in6_addr in6;
 
   if (family == AF_INET)
-    result = ((in.s_addr = inet_addr(name)) == INADDR_NONE ? 0 : 1);
+    {
+      /* It only looks like an IP address if it's all numbers and dots. */
+      int numdots = 0;
+      const char *p;
+      for (p = name; *p; p++)
+        {
+          if (!ISDIGIT(*p) && *p != '.') {
+            return 0;
+          } else if (*p == '.') {
+            numdots++;
+          }
+        }
+    
+      /* if we don't have 3 dots, it is illegal 
+       * (although inet_addr doesn't think so).
+       */
+      if (numdots != 3)
+        result = 0;
+      else
+        result = ((in.s_addr = inet_addr(name)) == INADDR_NONE ? 0 : 1);
+    }
   else if (family == AF_INET6)
     result = (ares_inet_pton(AF_INET6, name, &in6) < 1 ? 0 : 1);
 
@@ -227,7 +250,7 @@ static int fake_hostent(const char *name, int family, ares_host_callback callbac
   hostent.h_name = strdup(name);
   if (!hostent.h_name)
     {
-      callback(arg, ARES_ENOMEM, NULL);
+      callback(arg, ARES_ENOMEM, 0, NULL);
       return 1;
     }
 
@@ -236,7 +259,7 @@ static int fake_hostent(const char *name, int family, ares_host_callback callbac
   hostent.h_aliases = aliases;
   hostent.h_addrtype = family;
   hostent.h_addr_list = addrs;
-  callback(arg, ARES_SUCCESS, &hostent);
+  callback(arg, ARES_SUCCESS, 0, &hostent);
 
   free((char *)(hostent.h_name));
   return 1;
@@ -416,4 +439,3 @@ static int get6_address_index(struct in6_addr *addr, struct apattern *sortlist,
     }
   return i;
 }
-

ares/ares_getnameinfo.3

@@ -22,7 +22,7 @@ ares_getnameinfo \- Address-to-nodename translation in protocol-independent mann
 .B #include <ares.h>
 .PP
 .B typedef void (*ares_nameinfo_callback)(void *\fIarg\fP, int \fIstatus\fP,
-.B	char *\fInode\fP, char *\fIservice\fP)
+.B	int \fItimeouts\fP, char *\fInode\fP, char *\fIservice\fP)
 .PP
 .B void ares_getnameinfo(ares_channel \fIchannel\fP, const struct sockaddr *\fIsa\fP,
 .B 	socklen_t \fIsalen\fP, int \fIflags\fP, ares_nameinfo_callback \fIcallback\fP,
@@ -120,6 +120,11 @@ The
 .I flags
 parameter contains an illegal value.
 .PP
+The callback argument
+.I timeouts
+reports how many times a query timed out during the execution of the
+given request.
+.PP
 On successful completion of the query, the callback argument
 .I node
 contains a string representing the hostname (assuming 

ares/ares_getnameinfo.c

@@ -59,6 +59,7 @@ struct nameinfo_query {
   } addr;
   int family;
   int flags;
+  int timeouts;
 };
 
 #ifdef HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID
@@ -67,7 +68,7 @@ struct nameinfo_query {
 #define IPBUFSIZ 40
 #endif
 
-static void nameinfo_callback(void *arg, int status, struct hostent *host);
+static void nameinfo_callback(void *arg, int status, int timeouts, struct hostent *host);
 static char *lookup_service(unsigned short port, int flags,
                             char *buf, size_t buflen);
 #ifdef HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID
@@ -90,7 +91,7 @@ void ares_getnameinfo(ares_channel channel, const struct sockaddr *sa, socklen_t
     addr6 = (struct sockaddr_in6 *)sa;
   else
     {
-      callback(arg, ARES_ENOTIMP, NULL, NULL);
+      callback(arg, ARES_ENOTIMP, 0, NULL, NULL);
       return;
     }
 
@@ -110,7 +111,7 @@ void ares_getnameinfo(ares_channel channel, const struct sockaddr *sa, socklen_t
         port = addr6->sin6_port;
       service = lookup_service((unsigned short)(port & 0xffff),
                                flags, buf, sizeof(buf));
-      callback(arg, ARES_SUCCESS, NULL, service);
+      callback(arg, ARES_SUCCESS, 0, NULL, service);
       return;
     }
 
@@ -131,7 +132,7 @@ void ares_getnameinfo(ares_channel channel, const struct sockaddr *sa, socklen_t
          */
         if (flags & ARES_NI_NAMEREQD)
           {
-            callback(arg, ARES_EBADFLAGS, NULL, NULL);
+            callback(arg, ARES_EBADFLAGS, 0, NULL, NULL);
             return;
           }
         if (salen == sizeof(struct sockaddr_in6))
@@ -152,7 +153,7 @@ void ares_getnameinfo(ares_channel channel, const struct sockaddr *sa, socklen_t
         if (flags & ARES_NI_LOOKUPSERVICE)
           service = lookup_service((unsigned short)(port & 0xffff),
                                    flags, srvbuf, sizeof(srvbuf));
-        callback(arg, ARES_SUCCESS, ipbuf, service);
+        callback(arg, ARES_SUCCESS, 0, ipbuf, service);
         return;
       }
     /* This is where a DNS lookup becomes necessary */
@@ -161,12 +162,13 @@ void ares_getnameinfo(ares_channel channel, const struct sockaddr *sa, socklen_t
         niquery = malloc(sizeof(struct nameinfo_query));
         if (!niquery)
           {
-            callback(arg, ARES_ENOMEM, NULL, NULL);
+            callback(arg, ARES_ENOMEM, 0, NULL, NULL);
             return;
           }
         niquery->callback = callback;
         niquery->arg = arg;
         niquery->flags = flags;
+        niquery->timeouts = 0;
         if (sa->sa_family == AF_INET)
           {
             niquery->family = AF_INET;
@@ -185,13 +187,13 @@ void ares_getnameinfo(ares_channel channel, const struct sockaddr *sa, socklen_t
     }
 }
 
-static void nameinfo_callback(void *arg, int status, struct hostent *host)
+static void nameinfo_callback(void *arg, int status, int timeouts, struct hostent *host)
 {
   struct nameinfo_query *niquery = (struct nameinfo_query *) arg;
   char srvbuf[33];
   char *service = NULL;
 
-
+  niquery->timeouts += timeouts;
   if (status == ARES_SUCCESS)
     {
       /* They want a service too */
@@ -220,7 +222,7 @@ static void nameinfo_callback(void *arg, int status, struct hostent *host)
                  *end = 0;
              }
         }
-      niquery->callback(niquery->arg, ARES_SUCCESS, (char *)(host->h_name),
+      niquery->callback(niquery->arg, ARES_SUCCESS, niquery->timeouts, (char *)(host->h_name),
                         service);
       return;
     }
@@ -247,10 +249,10 @@ static void nameinfo_callback(void *arg, int status, struct hostent *host)
             service = lookup_service(niquery->addr.addr6.sin6_port,
                                      niquery->flags, srvbuf, sizeof(srvbuf));
         }
-      niquery->callback(niquery->arg, ARES_SUCCESS, ipbuf, service);
+      niquery->callback(niquery->arg, ARES_SUCCESS, niquery->timeouts, ipbuf, service);
       return;
     }
-  niquery->callback(niquery->arg, status, NULL, NULL);
+  niquery->callback(niquery->arg, status, niquery->timeouts, NULL, NULL);
   free(niquery);
 }
 

ares/ares_getsock.c

@@ -34,16 +34,18 @@ int ares_getsock(ares_channel channel,
 
   ares_socket_t *socks = (ares_socket_t *)s;
 
-  /* No queries, no file descriptors. */
-  if (!channel->queries)
-    return 0;
+  /* Are there any active queries? */
+  int active_queries = !ares__is_list_empty(&(channel->all_queries));
 
   for (i = 0;
        (i < channel->nservers) && (sockindex < ARES_GETSOCK_MAXNUM);
        i++)
     {
       server = &channel->servers[i];
-      if (server->udp_socket != ARES_SOCKET_BAD)
+      /* We only need to register interest in UDP sockets if we have
+       * outstanding queries.
+       */
+      if (active_queries && server->udp_socket != ARES_SOCKET_BAD)
         {
           if(sockindex >= numsocks)
             break;
@@ -51,6 +53,10 @@ int ares_getsock(ares_channel channel,
           bitmap |= ARES_GETSOCK_READABLE(setbits, sockindex);
           sockindex++;
         }
+      /* We always register for TCP events, because we want to know
+       * when the other side closes the connection, so we don't waste
+       * time trying to use a broken connection.
+       */
       if (server->tcp_socket != ARES_SOCKET_BAD)
        {
          if(sockindex >= numsocks)
@@ -58,7 +64,7 @@ int ares_getsock(ares_channel channel,
          socks[sockindex] = server->tcp_socket;
          bitmap |= ARES_GETSOCK_READABLE(setbits, sockindex);
 
-         if (server->qhead)
+         if (server->qhead && active_queries)
            /* then the tcp socket is also writable! */
            bitmap |= ARES_GETSOCK_WRITABLE(setbits, sockindex);
 

ares/ares_init.c

@@ -75,7 +75,7 @@ static int config_nameserver(struct server_state **servers, int *nservers,
 static int set_search(ares_channel channel, const char *str);
 static int set_options(ares_channel channel, const char *str);
 static const char *try_option(const char *p, const char *q, const char *opt);
-static void init_id_key(rc4_key* key,int key_data_len);
+static int init_id_key(rc4_key* key,int key_data_len);
 
 #ifndef WIN32
 static int sortlist_alloc(struct apattern **sortlist, int *nsort, struct apattern *pat);
@@ -133,17 +133,32 @@ int ares_init_options(ares_channel *channelptr, struct ares_options *options,
   channel->ndots = -1;
   channel->udp_port = -1;
   channel->tcp_port = -1;
+  channel->socket_send_buffer_size = -1;
+  channel->socket_receive_buffer_size = -1;
   channel->nservers = -1;
   channel->ndomains = -1;
   channel->nsort = -1;
+  channel->tcp_connection_generation = 0;
   channel->lookups = NULL;
-  channel->queries = NULL;
   channel->domains = NULL;
   channel->sortlist = NULL;
   channel->servers = NULL;
   channel->sock_state_cb = NULL;
   channel->sock_state_cb_data = NULL;
 
+  channel->last_timeout_processed = (long)time(NULL);
+
+  /* Initialize our lists of queries */
+  ares__init_list_head(&(channel->all_queries));
+  for (i = 0; i < ARES_QID_TABLE_SIZE; i++)
+    {
+      ares__init_list_head(&(channel->queries_by_qid[i]));
+    }
+  for (i = 0; i < ARES_TIMEOUT_TABLE_SIZE; i++)
+    {
+      ares__init_list_head(&(channel->queries_by_timeout[i]));
+    }
+
   /* Initialize configuration by each of the four sources, from highest
    * precedence to lowest.
    */
@@ -172,6 +187,18 @@ int ares_init_options(ares_channel *channelptr, struct ares_options *options,
       DEBUGF(fprintf(stderr, "Error: init_by_defaults failed: %s\n",
                      ares_strerror(status)));
   }
+
+  /* Generate random key */
+
+  if (status == ARES_SUCCESS) {
+    status = init_id_key(&channel->id_key, ARES_ID_KEY_LEN);
+    if (status == ARES_SUCCESS)
+      channel->next_id = ares__generate_new_id(&channel->id_key);
+    else
+      DEBUGF(fprintf(stderr, "Error: init_id_key failed: %s\n",
+                     ares_strerror(status)));
+  }
+
   if (status != ARES_SUCCESS)
     {
       /* Something failed; clean up memory we may have allocated. */
@@ -201,17 +228,16 @@ int ares_init_options(ares_channel *channelptr, struct ares_options *options,
       server = &channel->servers[i];
       server->udp_socket = ARES_SOCKET_BAD;
       server->tcp_socket = ARES_SOCKET_BAD;
+      server->tcp_connection_generation = ++channel->tcp_connection_generation;
       server->tcp_lenbuf_pos = 0;
       server->tcp_buffer = NULL;
       server->qhead = NULL;
       server->qtail = NULL;
+      ares__init_list_head(&(server->queries_to_server));
+      server->channel = channel;
+      server->is_broken = 0;
     }
 
-  init_id_key(&channel->id_key, ARES_ID_KEY_LEN);
-
-  channel->next_id = ares__generate_new_id(&channel->id_key);
-  channel->queries = NULL;
-
   *channelptr = channel;
   return ARES_SUCCESS;
 }
@@ -238,46 +264,55 @@ int ares_save_options(ares_channel channel, struct ares_options *options,
   options->timeout = channel->timeout;
   options->tries   = channel->tries;
   options->ndots   = channel->ndots;
-  options->udp_port = channel->udp_port;
-  options->tcp_port = channel->tcp_port;
+  options->udp_port = (unsigned short)channel->udp_port;
+  options->tcp_port = (unsigned short)channel->tcp_port;
   options->sock_state_cb     = channel->sock_state_cb;
   options->sock_state_cb_data = channel->sock_state_cb_data;
 
   /* Copy servers */
-  options->servers =
-    malloc(channel->nservers * sizeof(struct server_state));
-  if (!options->servers && channel->nservers != 0)
-    return ARES_ENOMEM;
-  for (i = 0; i < channel->nservers; i++)
-    options->servers[i] = channel->servers[i].addr;
+  if (channel->nservers) {
+    options->servers =
+      malloc(channel->nservers * sizeof(struct server_state));
+    if (!options->servers && channel->nservers != 0)
+      return ARES_ENOMEM;
+    for (i = 0; i < channel->nservers; i++)
+      options->servers[i] = channel->servers[i].addr;
+  }
   options->nservers = channel->nservers;
 
   /* copy domains */
-  options->domains = malloc(channel->ndomains * sizeof(char *));
-  if (!options->domains)
-    return ARES_ENOMEM;
-  for (i = 0; i < channel->ndomains; i++)
-  {
-    options->ndomains = i;
-    options->domains[i] = strdup(channel->domains[i]);
-    if (!options->domains[i])
+  if (channel->ndomains) {
+    options->domains = malloc(channel->ndomains * sizeof(char *));
+    if (!options->domains)
       return ARES_ENOMEM;
+
+    for (i = 0; i < channel->ndomains; i++)
+    {
+      options->ndomains = i;
+      options->domains[i] = strdup(channel->domains[i]);
+      if (!options->domains[i])
+        return ARES_ENOMEM;
+    }
   }
   options->ndomains = channel->ndomains;
 
   /* copy lookups */
-  options->lookups = strdup(channel->lookups);
-  if (!options->lookups)
-    return ARES_ENOMEM;
+  if (channel->lookups) {
+    options->lookups = strdup(channel->lookups);
+    if (!options->lookups && channel->lookups)
+      return ARES_ENOMEM;
+  }
 
   /* copy sortlist */
-  options->sortlist = malloc(channel->nsort * sizeof(struct apattern));
-  if (!options->sortlist)
-    return ARES_ENOMEM;
-  for (i = 0; i < channel->nsort; i++)
-  {
-    memcpy(&(options->sortlist[i]), &(channel->sortlist[i]),
-           sizeof(struct apattern));
+  if (channel->nsort) {
+    options->sortlist = malloc(channel->nsort * sizeof(struct apattern));
+    if (!options->sortlist)
+      return ARES_ENOMEM;
+    for (i = 0; i < channel->nsort; i++)
+    {
+      memcpy(&(options->sortlist[i]), &(channel->sortlist[i]),
+             sizeof(struct apattern));
+    }
   }
   options->nsort = channel->nsort;
 
@@ -308,6 +343,12 @@ static int init_by_options(ares_channel channel,
       channel->sock_state_cb = options->sock_state_cb;
       channel->sock_state_cb_data = options->sock_state_cb_data;
     }
+  if ((optmask & ARES_OPT_SOCK_SNDBUF)
+      && channel->socket_send_buffer_size == -1)
+    channel->socket_send_buffer_size = options->socket_send_buffer_size;
+  if ((optmask & ARES_OPT_SOCK_RCVBUF)
+      && channel->socket_receive_buffer_size == -1)
+    channel->socket_receive_buffer_size = options->socket_receive_buffer_size;
 
   /* Copy the servers, if given. */
   if ((optmask & ARES_OPT_SERVERS) && channel->nservers == -1)
@@ -456,7 +497,7 @@ static int get_iphlpapi_dns_info (char *ret_buf, size_t ret_size)
   FIXED_INFO    *fi   = alloca (sizeof(*fi));
   DWORD          size = sizeof (*fi);
   typedef DWORD (WINAPI* get_net_param_func) (FIXED_INFO*, DWORD*);
-  get_net_param_func GetNetworkParams;  /* available only on Win-98/2000+ */
+  get_net_param_func fpGetNetworkParams;  /* available only on Win-98/2000+ */
   HMODULE        handle;
   IP_ADDR_STRING *ipAddr;
   int            i, count = 0;
@@ -473,16 +514,16 @@ static int get_iphlpapi_dns_info (char *ret_buf, size_t ret_size)
   if (!handle)
      return (0);
 
-  GetNetworkParams = (get_net_param_func) GetProcAddress (handle, "GetNetworkParams");
-  if (!GetNetworkParams)
+  fpGetNetworkParams = (get_net_param_func) GetProcAddress (handle, "GetNetworkParams");
+  if (!fpGetNetworkParams)
      goto quit;
 
-  res = (*GetNetworkParams) (fi, &size);
+  res = (*fpGetNetworkParams) (fi, &size);
   if ((res != ERROR_BUFFER_OVERFLOW) && (res != ERROR_SUCCESS))
      goto quit;
 
   fi = alloca (size);
-  if (!fi || (*GetNetworkParams) (fi, &size) != ERROR_SUCCESS)
+  if (!fi || (*fpGetNetworkParams) (fi, &size) != ERROR_SUCCESS)
      goto quit;
 
   if (debug)
@@ -1306,11 +1347,11 @@ static void randomize_key(unsigned char* key,int key_data_len)
 
   if ( !randomized ) {
     for (;counter<key_data_len;counter++)
-      key[counter]=rand() % 256;
+      key[counter]=(unsigned char)(rand() % 256);
   }
 }
 
-static void init_id_key(rc4_key* key,int key_data_len)
+static int init_id_key(rc4_key* key,int key_data_len)
 {
   unsigned char index1;
   unsigned char index2;
@@ -1319,25 +1360,28 @@ static void init_id_key(rc4_key* key,int key_data_len)
   unsigned char *key_data_ptr = 0;
 
   key_data_ptr = calloc(1,key_data_len);
+  if (!key_data_ptr)
+    return ARES_ENOMEM;
+
   randomize_key(key->state,key_data_len);
   state = &key->state[0];
   for(counter = 0; counter < 256; counter++)
     /* unnecessary AND but it keeps some compilers happier */
-    state[counter] = counter & 0xff;
+    state[counter] = (unsigned char)(counter & 0xff);
   key->x = 0;
   key->y = 0;
   index1 = 0;
   index2 = 0;
   for(counter = 0; counter < 256; counter++)
   {
-    index2 = (key_data_ptr[index1] + state[counter] +
-              index2) % 256;
+    index2 = (unsigned char)((key_data_ptr[index1] + state[counter] +
+                              index2) % 256);
     ARES_SWAP_BYTE(&state[counter], &state[index2]);
 
-    index1 = (index1 + 1) % key_data_len;
+    index1 = (unsigned char)((index1 + 1) % key_data_len);
   }
   free(key_data_ptr);
-
+  return ARES_SUCCESS;
 }
 
 short ares__generate_new_id(rc4_key* key)

ares/ares_llist.c

@@ -0,0 +1,87 @@
+/* $Id$ */
+
+/* Copyright 1998 by the Massachusetts Institute of Technology.
+ *
+ * Permission to use, copy, modify, and distribute this
+ * software and its documentation for any purpose and without
+ * fee is hereby granted, provided that the above copyright
+ * notice appear in all copies and that both that copyright
+ * notice and this permission notice appear in supporting
+ * documentation, and that the name of M.I.T. not be used in
+ * advertising or publicity pertaining to distribution of the
+ * software without specific, written prior permission.
+ * M.I.T. makes no representations about the suitability of
+ * this software for any purpose.  It is provided "as is"
+ * without express or implied warranty.
+ */
+
+#include "setup.h"
+
+#include "ares.h"
+#include "ares_private.h"
+
+/* Routines for managing doubly-linked circular linked lists with a
+ * dummy head.
+ */
+
+/* Initialize a new head node */
+void ares__init_list_head(struct list_node* head) {
+  head->prev = head;
+  head->next = head;
+  head->data = NULL;
+}
+
+/* Initialize a list node */
+void ares__init_list_node(struct list_node* node, void* d) {
+  node->prev = NULL;
+  node->next = NULL;
+  node->data = d;
+}
+
+/* Returns true iff the given list is empty */
+int ares__is_list_empty(struct list_node* head) {
+  return ((head->next == head) && (head->prev == head));
+}
+
+/* Inserts new_node before old_node */
+void ares__insert_in_list(struct list_node* new_node,
+                          struct list_node* old_node) {
+  new_node->next = old_node;
+  new_node->prev = old_node->prev;
+  old_node->prev->next = new_node;
+  old_node->prev = new_node;
+}
+
+/* Removes the node from the list it's in, if any */
+void ares__remove_from_list(struct list_node* node) {
+  if (node->next != NULL) {
+    node->prev->next = node->next;
+    node->next->prev = node->prev;
+    node->prev = NULL;
+    node->next = NULL;
+  }
+}
+
+/* Swap the contents of two lists */
+void ares__swap_lists(struct list_node* head_a,
+                      struct list_node* head_b) {
+  int is_a_empty = ares__is_list_empty(head_a);
+  int is_b_empty = ares__is_list_empty(head_b);
+  struct list_node old_a = *head_a;
+  struct list_node old_b = *head_b;
+
+  if (is_a_empty) {
+    ares__init_list_head(head_b);
+  } else {
+    *head_b = old_a;
+    old_a.next->prev = head_b;
+    old_a.prev->next = head_b;
+  }
+  if (is_b_empty) {
+    ares__init_list_head(head_a);
+  } else {
+    *head_a = old_b;
+    old_b.next->prev = head_a;
+    old_b.prev->next = head_a;
+  }
+}

ares/ares_llist.h

@@ -0,0 +1,43 @@
+#ifndef __ARES_LLIST_H
+#define __ARES_LLIST_H
+
+/* $Id$ */
+
+/* Copyright 1998 by the Massachusetts Institute of Technology.
+ *
+ * Permission to use, copy, modify, and distribute this
+ * software and its documentation for any purpose and without
+ * fee is hereby granted, provided that the above copyright
+ * notice appear in all copies and that both that copyright
+ * notice and this permission notice appear in supporting
+ * documentation, and that the name of M.I.T. not be used in
+ * advertising or publicity pertaining to distribution of the
+ * software without specific, written prior permission.
+ * M.I.T. makes no representations about the suitability of
+ * this software for any purpose.  It is provided "as is"
+ * without express or implied warranty.
+ */
+
+
+/* Node definition for circular, doubly-linked list */
+struct list_node {
+  struct list_node *prev;
+  struct list_node *next;
+  void* data;
+};
+
+void ares__init_list_head(struct list_node* head);
+
+void ares__init_list_node(struct list_node* node, void* d);
+
+int ares__is_list_empty(struct list_node* head);
+
+void ares__insert_in_list(struct list_node* new_node,
+                          struct list_node* old_node);
+
+void ares__remove_from_list(struct list_node* node);
+
+void ares__swap_lists(struct list_node* head_a,
+                      struct list_node* head_b);
+
+#endif /* __ARES_LLIST_H */

ares/ares_mkquery.c

@@ -88,6 +88,10 @@ int ares_mkquery(const char *name, int dnsclass, int type, unsigned short id,
   unsigned char *q;
   const char *p;
 
+  /* Set our results early, in case we bail out early with an error. */
+  *buflen = 0;
+  *buf = NULL;
+
   /* Compute the length of the encoded name so we can check buflen.
    * Start counting at 1 for the zero-length label at the end. */
   len = 1;
@@ -104,6 +108,23 @@ int ares_mkquery(const char *name, int dnsclass, int type, unsigned short id,
   if (*name && *(p - 1) != '.')
     len++;
 
+  /* Immediately reject names that are longer than the maximum of 255
+   * bytes that's specified in RFC 1035 ("To simplify implementations,
+   * the total length of a domain name (i.e., label octets and label
+   * length octets) is restricted to 255 octets or less."). We aren't
+   * doing this just to be a stickler about RFCs. For names that are
+   * too long, 'dnscache' closes its TCP connection to us immediately
+   * (when using TCP) and ignores the request when using UDP, and
+   * BIND's named returns ServFail (TCP or UDP). Sending a request
+   * that we know will cause 'dnscache' to close the TCP connection is
+   * painful, since that makes any other outstanding requests on that
+   * connection fail. And sending a UDP request that we know
+   * 'dnscache' will ignore is bad because resources will be tied up
+   * until we time-out the request.
+   */
+  if (len > MAXCDNAME)
+    return ARES_EBADNAME;
+
   *buflen = len + HFIXEDSZ + QFIXEDSZ;
   *buf = malloc(*buflen);
   if (!*buf)

ares/ares_private.h

@@ -83,12 +83,20 @@
 #define ARES_ID_KEY_LEN 31
 
 #include "ares_ipv6.h"
+#include "ares_llist.h"
+
+struct query;
 
 struct send_request {
   /* Remaining data to send */
   const unsigned char *data;
   size_t len;
 
+  /* The query for which we're sending this data */
+  struct query* owner_query;
+  /* The buffer we're using, if we have our own copy of the packet */
+  unsigned char *data_storage;
+
   /* Next request in queue */
   struct send_request *next;
 };
@@ -110,13 +118,42 @@ struct server_state {
   /* TCP output queue */
   struct send_request *qhead;
   struct send_request *qtail;
+
+  /* Which incarnation of this connection is this? We don't want to
+   * retransmit requests into the very same socket, but if the server
+   * closes on us and we re-open the connection, then we do want to
+   * re-send. */
+  int tcp_connection_generation;
+
+  /* Circular, doubly-linked list of outstanding queries to this server */
+  struct list_node queries_to_server;
+
+  /* Link back to owning channel */
+  ares_channel channel;
+
+  /* Is this server broken? We mark connections as broken when a
+   * request that is queued for sending times out.
+   */
+  int is_broken;
 };
 
+/* State to represent a DNS query */
 struct query {
   /* Query ID from qbuf, for faster lookup, and current timeout */
   unsigned short qid;
   time_t timeout;
 
+  /*
+   * Links for the doubly-linked lists in which we insert a query.
+   * These circular, doubly-linked lists that are hash-bucketed based
+   * the attributes we care about, help making most important
+   * operations O(1).
+   */
+  struct list_node queries_by_qid;    /* hopefully in same cache line as qid */
+  struct list_node queries_by_timeout;
+  struct list_node queries_to_server;
+  struct list_node all_queries;
+
   /* Query buf with length at beginning, for TCP transmission */
   unsigned char *tcpbuf;
   int tcplen;
@@ -130,12 +167,16 @@ struct query {
   /* Query status */
   int try;
   int server;
-  int *skip_server;
+  struct query_server_info *server_info;   /* per-server state */
   int using_tcp;
   int error_status;
+  int timeouts; /* number of timeouts we saw for this request */
+};
 
-  /* Next query in chain */
-  struct query *next;
+/* Per-server state for a query */
+struct query_server_info {
+  int skip_server;  /* should we skip server, due to errors, etc? */
+  int tcp_connection_generation;  /* into which TCP connection did we send? */
 };
 
 /* An IP address pattern; matches an IP address X if X & mask == addr */
@@ -173,6 +214,8 @@ struct ares_channeldata {
   int ndots;
   int udp_port;
   int tcp_port;
+  int socket_send_buffer_size;
+  int socket_receive_buffer_size;
   char **domains;
   int ndomains;
   struct apattern *sortlist;
@@ -188,8 +231,21 @@ struct ares_channeldata {
   /* key to use when generating new ids */
   rc4_key id_key;
 
-  /* Active queries */
-  struct query *queries;
+  /* Generation number to use for the next TCP socket open/close */
+  int tcp_connection_generation;
+
+  /* The time at which we last called process_timeouts() */
+  time_t last_timeout_processed;
+
+  /* Circular, doubly-linked list of queries, bucketed various ways.... */
+  /* All active queries in a single list: */
+  struct list_node all_queries;
+  /* Queries bucketed by qid, for quickly dispatching DNS responses: */
+#define ARES_QID_TABLE_SIZE 2048
+  struct list_node queries_by_qid[ARES_QID_TABLE_SIZE];
+  /* Queries bucketed by timeout, for quickly handling timeouts: */
+#define ARES_TIMEOUT_TABLE_SIZE 1024
+  struct list_node queries_by_timeout[ARES_TIMEOUT_TABLE_SIZE];
 
   ares_sock_state_cb sock_state_cb;
   void *sock_state_cb_data;
@@ -200,6 +256,7 @@ void ares__send_query(ares_channel channel, struct query *query, time_t now);
 void ares__close_sockets(ares_channel channel, struct server_state *server);
 int ares__get_hostent(FILE *fp, int family, struct hostent **host);
 int ares__read_line(FILE *fp, char **buf, int *bufsize);
+void ares__free_query(struct query *query);
 short ares__generate_new_id(rc4_key* key);
 
 #define ARES_SWAP_BYTE(a,b) \
@@ -220,4 +277,3 @@ short ares__generate_new_id(rc4_key* key);
 #endif
 
 #endif /* __ARES_PRIVATE_H */
-

ares/ares_process.c

@@ -21,13 +21,24 @@
 #include "nameser.h"
 
 #else
+#ifdef HAVE_SYS_SOCKET_H
 #include <sys/socket.h>
+#endif
 #ifdef HAVE_SYS_UIO_H
 #include <sys/uio.h>
 #endif
-#include <netinet/in.h>
+#ifdef HAVE_NETINET_IN_H
+#include <netinet/in.h> /* <netinet/tcp.h> may need it */
+#endif
+#ifdef HAVE_NETINET_TCP_H
+#include <netinet/tcp.h> /* for TCP_NODELAY */
+#endif
+#ifdef HAVE_NETDB_H
 #include <netdb.h>
+#endif
+#ifdef HAVE_ARPA_NAMESER_H
 #include <arpa/nameser.h>
+#endif
 #ifdef HAVE_ARPA_NAMESER_COMPAT_H
 #include <arpa/nameser_compat.h>
 #endif
@@ -43,6 +54,7 @@
 #include <sys/filio.h>
 #endif
 
+#include <assert.h>
 #include <string.h>
 #include <stdlib.h>
 #include <fcntl.h>
@@ -61,16 +73,22 @@ static void read_tcp_data(ares_channel channel, fd_set *read_fds,
                           ares_socket_t read_fd, time_t now);
 static void read_udp_packets(ares_channel channel, fd_set *read_fds,
                              ares_socket_t read_fd, time_t now);
+static void advance_tcp_send_queue(ares_channel channel, int whichserver,
+                                   ssize_t num_bytes);
 static void process_timeouts(ares_channel channel, time_t now);
+static void process_broken_connections(ares_channel channel, time_t now);
 static void process_answer(ares_channel channel, unsigned char *abuf,
                            int alen, int whichserver, int tcp, time_t now);
 static void handle_error(ares_channel channel, int whichserver, time_t now);
-static struct query *next_server(ares_channel channel, struct query *query, time_t now);
+static void skip_server(ares_channel channel, struct query *query,
+                        int whichserver);
+static void next_server(ares_channel channel, struct query *query, time_t now);
+static int configure_socket(int s, ares_channel channel);
 static int open_tcp_socket(ares_channel channel, struct server_state *server);
 static int open_udp_socket(ares_channel channel, struct server_state *server);
 static int same_questions(const unsigned char *qbuf, int qlen,
                           const unsigned char *abuf, int alen);
-static struct query *end_query(ares_channel channel, struct query *query, int status,
+static void end_query(ares_channel channel, struct query *query, int status,
                       unsigned char *abuf, int alen);
 
 /* Something interesting happened on the wire, or there was a timeout.
@@ -85,6 +103,7 @@ void ares_process(ares_channel channel, fd_set *read_fds, fd_set *write_fds)
   read_tcp_data(channel, read_fds, ARES_SOCKET_BAD, now);
   read_udp_packets(channel, read_fds, ARES_SOCKET_BAD, now);
   process_timeouts(channel, now);
+  process_broken_connections(channel, now);
 }
 
 /* Something interesting happened on the wire, or there was a timeout.
@@ -109,7 +128,7 @@ void ares_process_fd(ares_channel channel,
  * otherwise. This is mostly for HP-UX, which could return EAGAIN or
  * EWOULDBLOCK. See this man page
  *
- * 	http://devrsrc1.external.hp.com/STKS/cgi-bin/man2html?manpage=/usr/share/man/man2.Z/send.2
+ *      http://devrsrc1.external.hp.com/STKS/cgi-bin/man2html?manpage=/usr/share/man/man2.Z/send.2
  */
 static int try_again(int errnum)
 {
@@ -155,7 +174,7 @@ static void write_tcp_data(ares_channel channel,
       /* Make sure server has data to send and is selected in write_fds or
          write_fd. */
       server = &channel->servers[i];
-      if (!server->qhead || server->tcp_socket == ARES_SOCKET_BAD)
+      if (!server->qhead || server->tcp_socket == ARES_SOCKET_BAD || server->is_broken)
         continue;
 
       if(write_fds) {
@@ -167,6 +186,14 @@ static void write_tcp_data(ares_channel channel,
           continue;
       }
 
+      if(write_fds)
+        /* If there's an error and we close this socket, then open
+         * another with the same fd to talk to another server, then we
+         * don't want to think that it was the new socket that was
+         * ready. This is not disastrous, but is likely to result in
+         * extra system calls and confusion. */
+        FD_CLR(server->tcp_socket, write_fds);
+
       /* Count the number of send queue items. */
       n = 0;
       for (sendreq = server->qhead; sendreq; sendreq = sendreq->next)
@@ -194,27 +221,7 @@ static void write_tcp_data(ares_channel channel,
             }
 
           /* Advance the send queue by as many bytes as we sent. */
-          while (wcount)
-            {
-              sendreq = server->qhead;
-              if ((size_t)wcount >= sendreq->len)
-                {
-                  wcount -= sendreq->len;
-                  server->qhead = sendreq->next;
-                  if (server->qhead == NULL)
-                    {
-                      SOCK_STATE_CALLBACK(channel, server->tcp_socket, 1, 0);
-                      server->qtail = NULL;
-                    }
-                  free(sendreq);
-                }
-              else
-                {
-                  sendreq->data += wcount;
-                  sendreq->len -= wcount;
-                  break;
-                }
-            }
+          advance_tcp_send_queue(channel, i, wcount);
         }
       else
         {
@@ -230,25 +237,42 @@ static void write_tcp_data(ares_channel channel,
             }
 
           /* Advance the send queue by as many bytes as we sent. */
-          if ((size_t)scount == sendreq->len)
-            {
-              server->qhead = sendreq->next;
-              if (server->qhead == NULL)
-                {
-                  SOCK_STATE_CALLBACK(channel, server->tcp_socket, 1, 0);
-                  server->qtail = NULL;
-                }
-              free(sendreq);
-            }
-          else
-            {
-              sendreq->data += scount;
-              sendreq->len -= scount;
-            }
+          advance_tcp_send_queue(channel, i, scount);
         }
     }
 }
 
+/* Consume the given number of bytes from the head of the TCP send queue. */
+static void advance_tcp_send_queue(ares_channel channel, int whichserver,
+                                   ssize_t num_bytes)
+{
+  struct send_request *sendreq;
+  struct server_state *server = &channel->servers[whichserver];
+  while (num_bytes > 0)
+    {
+      sendreq = server->qhead;
+      if ((size_t)num_bytes >= sendreq->len)
+       {
+         num_bytes -= sendreq->len;
+         server->qhead = sendreq->next;
+         if (server->qhead == NULL)
+           {
+             SOCK_STATE_CALLBACK(channel, server->tcp_socket, 1, 0);
+             server->qtail = NULL;
+           }
+         if (sendreq->data_storage != NULL)
+           free(sendreq->data_storage);
+         free(sendreq);
+       }
+      else
+       {
+         sendreq->data += num_bytes;
+         sendreq->len -= num_bytes;
+         num_bytes = 0;
+       }
+    }
+}
+
 /* If any TCP socket selects true for reading, read some data,
  * allocate a buffer if we finish reading the length word, and process
  * a packet if we finish reading one.
@@ -268,7 +292,7 @@ static void read_tcp_data(ares_channel channel, fd_set *read_fds,
     {
       /* Make sure the server has a socket and is selected in read_fds. */
       server = &channel->servers[i];
-      if (server->tcp_socket == ARES_SOCKET_BAD)
+      if (server->tcp_socket == ARES_SOCKET_BAD || server->is_broken)
         continue;
 
       if(read_fds) {
@@ -280,6 +304,14 @@ static void read_tcp_data(ares_channel channel, fd_set *read_fds,
           continue;
       }
 
+      if(read_fds)
+        /* If there's an error and we close this socket, then open
+         * another with the same fd to talk to another server, then we
+         * don't want to think that it was the new socket that was
+         * ready. This is not disastrous, but is likely to result in
+         * extra system calls and confusion. */
+        FD_CLR(server->tcp_socket, read_fds);
+
       if (server->tcp_lenbuf_pos != 2)
         {
           /* We haven't yet read a length word, so read that (or
@@ -358,7 +390,7 @@ static void read_udp_packets(ares_channel channel, fd_set *read_fds,
       /* Make sure the server has a socket and is selected in read_fds. */
       server = &channel->servers[i];
 
-      if (server->udp_socket == ARES_SOCKET_BAD)
+      if (server->udp_socket == ARES_SOCKET_BAD || server->is_broken)
         continue;
 
       if(read_fds) {
@@ -370,30 +402,58 @@ static void read_udp_packets(ares_channel channel, fd_set *read_fds,
           continue;
       }
 
-      count = sread(server->udp_socket, buf, sizeof(buf));
-      if (count == -1 && try_again(SOCKERRNO))
-        continue;
-      else if (count <= 0)
-        handle_error(channel, i, now);
+      if(read_fds)
+        /* If there's an error and we close this socket, then open
+         * another with the same fd to talk to another server, then we
+         * don't want to think that it was the new socket that was
+         * ready. This is not disastrous, but is likely to result in
+         * extra system calls and confusion. */
+        FD_CLR(server->udp_socket, read_fds);
 
-      process_answer(channel, buf, (int)count, i, 0, now);
+      /* To reduce event loop overhead, read and process as many
+       * packets as we can. */
+      do {
+        count = sread(server->udp_socket, buf, sizeof(buf));
+        if (count == -1 && try_again(SOCKERRNO))
+          continue;
+        else if (count <= 0)
+          handle_error(channel, i, now);
+        else
+          process_answer(channel, buf, (int)count, i, 0, now);
+       } while (count > 0);
     }
 }
 
 /* If any queries have timed out, note the timeout and move them on. */
 static void process_timeouts(ares_channel channel, time_t now)
 {
-  struct query *query, *next;
+  time_t t;  /* the time of the timeouts we're processing */
+  struct query *query;
+  struct list_node* list_head;
+  struct list_node* list_node;
 
-  for (query = channel->queries; query; query = next)
+  /* Process all the timeouts that have fired since the last time we
+   * processed timeouts. If things are going well, then we'll have
+   * hundreds/thousands of queries that fall into future buckets, and
+   * only a handful of requests that fall into the "now" bucket, so
+   * this should be quite quick.
+   */
+  for (t = channel->last_timeout_processed; t <= now; t++)
     {
-      next = query->next;
-      if (query->timeout != 0 && now >= query->timeout)
+      list_head = &(channel->queries_by_timeout[t % ARES_TIMEOUT_TABLE_SIZE]);
+      for (list_node = list_head->next; list_node != list_head; )
         {
-          query->error_status = ARES_ETIMEOUT;
-          next = next_server(channel, query, now);
+          query = list_node->data;
+          list_node = list_node->next;  /* in case the query gets deleted */
+          if (query->timeout != 0 && now >= query->timeout)
+            {
+              query->error_status = ARES_ETIMEOUT;
+              ++query->timeouts;
+              next_server(channel, query, now);
+            }
         }
-    }
+     }
+  channel->last_timeout_processed = now;
 }
 
 /* Handle an answer from a server. */
@@ -403,6 +463,8 @@ static void process_answer(ares_channel channel, unsigned char *abuf,
   int tc, rcode;
   unsigned short id;
   struct query *query;
+  struct list_node* list_head;
+  struct list_node* list_node;
 
   /* If there's no room in the answer for a header, we can't do much
    * with it. */
@@ -414,11 +476,24 @@ static void process_answer(ares_channel channel, unsigned char *abuf,
   tc = DNS_HEADER_TC(abuf);
   rcode = DNS_HEADER_RCODE(abuf);
 
-  /* Find the query corresponding to this packet. */
-  for (query = channel->queries; query; query = query->next)
+  /* Find the query corresponding to this packet. The queries are
+   * hashed/bucketed by query id, so this lookup should be quick.
+   * Note that both the query id and the questions must be the same;
+   * when the query id wraps around we can have multiple outstanding
+   * queries with the same query id, so we need to check both the id and
+   * question.
+   */
+  query = NULL;
+  list_head = &(channel->queries_by_qid[id % ARES_QID_TABLE_SIZE]);
+  for (list_node = list_head->next; list_node != list_head;
+       list_node = list_node->next)
     {
-      if (query->qid == id)
-        break;
+      struct query *q = list_node->data;
+      if ((q->qid == id) && same_questions(q->qbuf, q->qlen, abuf, alen))
+        {
+          query = q;
+          break;
+        }
     }
   if (!query)
     return;
@@ -450,13 +525,7 @@ static void process_answer(ares_channel channel, unsigned char *abuf,
     {
       if (rcode == SERVFAIL || rcode == NOTIMP || rcode == REFUSED)
         {
-          query->skip_server[whichserver] = 1;
-          if (query->server == whichserver)
-            next_server(channel, query, now);
-          return;
-        }
-      if (!same_questions(query->qbuf, query->qlen, abuf, alen))
-        {
+          skip_server(channel, query, whichserver);
           if (query->server == whichserver)
             next_server(channel, query, now);
           return;
@@ -466,29 +535,72 @@ static void process_answer(ares_channel channel, unsigned char *abuf,
   end_query(channel, query, ARES_SUCCESS, abuf, alen);
 }
 
-static void handle_error(ares_channel channel, int whichserver, time_t now)
+/* Close all the connections that are no longer usable. */
+static void process_broken_connections(ares_channel channel, time_t now)
 {
-  struct query *query, *next;
-
-  /* Reset communications with this server. */
-  ares__close_sockets(channel, &channel->servers[whichserver]);
-
-  /* Tell all queries talking to this server to move on and not try
-   * this server again.
-   */
-
-  for (query = channel->queries; query; query = next)
+  int i;
+  for (i = 0; i < channel->nservers; i++)
     {
-      next = query->next;
-      if (query->server == whichserver)
+      struct server_state *server = &channel->servers[i];
+      if (server->is_broken)
         {
-          query->skip_server[whichserver] = 1;
-          next = next_server(channel, query, now);
+          handle_error(channel, i, now);
         }
     }
 }
 
-static struct query *next_server(ares_channel channel, struct query *query, time_t now)
+static void handle_error(ares_channel channel, int whichserver, time_t now)
+{
+  struct server_state *server;
+  struct query *query;
+  struct list_node list_head;
+  struct list_node* list_node;
+
+  server = &channel->servers[whichserver];
+
+  /* Reset communications with this server. */
+  ares__close_sockets(channel, server);
+
+  /* Tell all queries talking to this server to move on and not try
+   * this server again. We steal the current list of queries that were
+   * in-flight to this server, since when we call next_server this can
+   * cause the queries to be re-sent to this server, which will
+   * re-insert these queries in that same server->queries_to_server
+   * list.
+   */
+  ares__init_list_head(&list_head);
+  ares__swap_lists(&list_head, &(server->queries_to_server));
+  for (list_node = list_head.next; list_node != &list_head; )
+    {
+      query = list_node->data;
+      list_node = list_node->next;  /* in case the query gets deleted */
+      assert(query->server == whichserver);
+      skip_server(channel, query, whichserver);
+      next_server(channel, query, now);
+    }
+  /* Each query should have removed itself from our temporary list as
+   * it re-sent itself or finished up...
+   */
+  assert(ares__is_list_empty(&list_head));
+}
+
+static void skip_server(ares_channel channel, struct query *query,
+                        int whichserver) {
+  /* The given server gave us problems with this query, so if we have
+   * the luxury of using other servers, then let's skip the
+   * potentially broken server and just use the others. If we only
+   * have one server and we need to retry then we should just go ahead
+   * and re-use that server, since it's our only hope; perhaps we
+   * just got unlucky, and retrying will work (eg, the server timed
+   * out our TCP connection just as we were sending another request).
+   */
+  if (channel->nservers > 1)
+    {
+      query->server_info[whichserver].skip_server = 1;
+    }
+}
+
+static void next_server(ares_channel channel, struct query *query, time_t now)
 {
   /* Advance to the next server or try. */
   query->server++;
@@ -496,19 +608,33 @@ static struct query *next_server(ares_channel channel, struct query *query, time
     {
       for (; query->server < channel->nservers; query->server++)
         {
-          if (!query->skip_server[query->server])
+          struct server_state *server = &channel->servers[query->server];
+          /* We don't want to use this server if (1) we decided this
+           * connection is broken, and thus about to be closed, (2)
+           * we've decided to skip this server because of earlier
+           * errors we encountered, or (3) we already sent this query
+           * over this exact connection.
+           */
+          if (!server->is_broken &&
+               !query->server_info[query->server].skip_server &&
+               !(query->using_tcp &&
+                 (query->server_info[query->server].tcp_connection_generation ==
+                  server->tcp_connection_generation)))
             {
-              ares__send_query(channel, query, now);
-              return (query->next);
+               ares__send_query(channel, query, now);
+               return;
             }
         }
       query->server = 0;
 
-      /* Only one try if we're using TCP. */
-      if (query->using_tcp)
-        break;
+      /* You might think that with TCP we only need one try. However,
+       * even when using TCP, servers can time-out our connection just
+       * as we're sending a request, or close our connection because
+       * they die, or never send us a reply because they get wedged or
+       * tickle a bug that drops our request.
+       */
     }
-  return end_query(channel, query, query->error_status, NULL, 0);
+  end_query(channel, query, query->error_status, NULL, 0);
 }
 
 void ares__send_query(ares_channel channel, struct query *query, time_t now)
@@ -526,7 +652,7 @@ void ares__send_query(ares_channel channel, struct query *query, time_t now)
         {
           if (open_tcp_socket(channel, server) == -1)
             {
-              query->skip_server[query->server] = 1;
+              skip_server(channel, query, query->server);
               next_server(channel, query, now);
               return;
             }
@@ -537,8 +663,16 @@ void ares__send_query(ares_channel channel, struct query *query, time_t now)
         end_query(channel, query, ARES_ENOMEM, NULL, 0);
           return;
         }
+      /* To make the common case fast, we avoid copies by using the
+       * query's tcpbuf for as long as the query is alive. In the rare
+       * case where the query ends while it's queued for transmission,
+       * then we give the sendreq its own copy of the request packet
+       * and put it in sendreq->data_storage.
+       */
+      sendreq->data_storage = NULL;
       sendreq->data = query->tcpbuf;
       sendreq->len = query->tcplen;
+      sendreq->owner_query = query;
       sendreq->next = NULL;
       if (server->qtail)
         server->qtail->next = sendreq;
@@ -548,7 +682,8 @@ void ares__send_query(ares_channel channel, struct query *query, time_t now)
           server->qhead = sendreq;
         }
       server->qtail = sendreq;
-      query->timeout = 0;
+      query->server_info[query->server].tcp_connection_generation =
+        server->tcp_connection_generation;
     }
   else
     {
@@ -556,7 +691,7 @@ void ares__send_query(ares_channel channel, struct query *query, time_t now)
         {
           if (open_udp_socket(channel, server) == -1)
             {
-              query->skip_server[query->server] = 1;
+              skip_server(channel, query, query->server);
               next_server(channel, query, now);
               return;
             }
@@ -564,21 +699,36 @@ void ares__send_query(ares_channel channel, struct query *query, time_t now)
       if (swrite(server->udp_socket, query->qbuf, query->qlen) == -1)
         {
           /* FIXME: Handle EAGAIN here since it likely can happen. */
-          query->skip_server[query->server] = 1;
+          skip_server(channel, query, query->server);
           next_server(channel, query, now);
           return;
         }
-      query->timeout = now
-          + ((query->try == 0) ? channel->timeout
-             : channel->timeout << query->try / channel->nservers);
     }
+    query->timeout = now
+        + ((query->try == 0) ? channel->timeout
+           : channel->timeout << query->try / channel->nservers);
+    /* Keep track of queries bucketed by timeout, so we can process
+     * timeout events quickly.
+     */
+    ares__remove_from_list(&(query->queries_by_timeout));
+    ares__insert_in_list(
+        &(query->queries_by_timeout),
+        &(channel->queries_by_timeout[query->timeout %
+                                      ARES_TIMEOUT_TABLE_SIZE]));
+
+    /* Keep track of queries bucketed by server, so we can process server
+     * errors quickly.
+     */
+    ares__remove_from_list(&(query->queries_to_server));
+    ares__insert_in_list(&(query->queries_to_server),
+                         &(server->queries_to_server));
 }
 
 /*
- * nonblock() set the given socket to either blocking or non-blocking mode
+ * setsocknonblock sets the given socket to either blocking or non-blocking mode
  * based on the 'nonblock' boolean argument. This function is highly portable.
  */
-static int nonblock(ares_socket_t sockfd,    /* operate on this */
+static int setsocknonblock(ares_socket_t sockfd,    /* operate on this */
                     int nonblock   /* TRUE or FALSE */)
 {
 #undef SETBLOCK
@@ -646,9 +796,36 @@ static int nonblock(ares_socket_t sockfd,    /* operate on this */
 #endif
 }
 
+static int configure_socket(int s, ares_channel channel)
+{
+  setsocknonblock(s, TRUE);
+
+#if defined(FD_CLOEXEC) && !defined(MSDOS)
+  /* Configure the socket fd as close-on-exec. */
+  if (fcntl(s, F_SETFD, FD_CLOEXEC) == -1)
+    return -1;
+#endif
+
+  /* Set the socket's send and receive buffer sizes. */
+  if ((channel->socket_send_buffer_size > 0) &&
+      setsockopt(s, SOL_SOCKET, SO_SNDBUF,
+                 (void *)&channel->socket_send_buffer_size,
+                 sizeof(channel->socket_send_buffer_size)) == -1)
+    return -1;
+
+  if ((channel->socket_receive_buffer_size > 0) &&
+      setsockopt(s, SOL_SOCKET, SO_RCVBUF,
+                 (void *)&channel->socket_receive_buffer_size,
+                 sizeof(channel->socket_receive_buffer_size)) == -1)
+    return -1;
+
+  return 0;
+ }
+
 static int open_tcp_socket(ares_channel channel, struct server_state *server)
 {
   ares_socket_t s;
+  int opt;
   struct sockaddr_in sockin;
 
   /* Acquire a socket. */
@@ -656,8 +833,26 @@ static int open_tcp_socket(ares_channel channel, struct server_state *server)
   if (s == ARES_SOCKET_BAD)
     return -1;
 
-  /* Set the socket non-blocking. */
-  nonblock(s, TRUE);
+  /* Configure it. */
+  if (configure_socket(s, channel) < 0)
+    {
+       close(s);
+       return -1;
+    }
+
+  /*
+   * Disable the Nagle algorithm (only relevant for TCP sockets, and thus not in
+   * configure_socket). In general, in DNS lookups we're pretty much interested
+   * in firing off a single request and then waiting for a reply, so batching
+   * isn't very interesting in general.
+   */
+  opt = 1;
+  if (setsockopt(s, IPPROTO_TCP, TCP_NODELAY,
+                 (void *)&opt, sizeof(opt)) == -1)
+    {
+       close(s);
+       return -1;
+    }
 
   /* Connect to the server. */
   memset(&sockin, 0, sizeof(sockin));
@@ -676,6 +871,7 @@ static int open_tcp_socket(ares_channel channel, struct server_state *server)
   SOCK_STATE_CALLBACK(channel, s, 1, 0);
   server->tcp_buffer_pos = 0;
   server->tcp_socket = s;
+  server->tcp_connection_generation = ++channel->tcp_connection_generation;
   return 0;
 }
 
@@ -690,7 +886,11 @@ static int open_udp_socket(ares_channel channel, struct server_state *server)
     return -1;
 
   /* Set the socket non-blocking. */
-  nonblock(s, TRUE);
+  if (configure_socket(s, channel) < 0)
+    {
+       close(s);
+       return -1;
+    }
 
   /* Connect to the server. */
   memset(&sockin, 0, sizeof(sockin));
@@ -788,34 +988,92 @@ static int same_questions(const unsigned char *qbuf, int qlen,
   return 1;
 }
 
-static struct query *end_query (ares_channel channel, struct query *query, int status,
-                      unsigned char *abuf, int alen)
+static void end_query (ares_channel channel, struct query *query, int status,
+                       unsigned char *abuf, int alen)
 {
-  struct query **q, *next;
   int i;
 
-  query->callback(query->arg, status, abuf, alen);
-  for (q = &channel->queries; *q; q = &(*q)->next)
+  /* First we check to see if this query ended while one of our send
+   * queues still has pointers to it.
+   */
+  for (i = 0; i < channel->nservers; i++)
     {
-      if (*q == query)
-        break;
+      struct server_state *server = &channel->servers[i];
+      struct send_request *sendreq;
+      for (sendreq = server->qhead; sendreq; sendreq = sendreq->next)
+        if (sendreq->owner_query == query)
+          {
+            sendreq->owner_query = NULL;
+            assert(sendreq->data_storage == NULL);
+            if (status == ARES_SUCCESS)
+              {
+                /* We got a reply for this query, but this queued
+                 * sendreq points into this soon-to-be-gone query's
+                 * tcpbuf. Probably this means we timed out and queued
+                 * the query for retransmission, then received a
+                 * response before actually retransmitting. This is
+                 * perfectly fine, so we want to keep the connection
+                 * running smoothly if we can. But in the worst case
+                 * we may have sent only some prefix of the query,
+                 * with some suffix of the query left to send. Also,
+                 * the buffer may be queued on multiple queues. To
+                 * prevent dangling pointers to the query's tcpbuf and
+                 * handle these cases, we just give such sendreqs
+                 * their own copy of the query packet.
+                 */
+               sendreq->data_storage = malloc(sendreq->len);
+               if (sendreq->data_storage != NULL)
+                 {
+                   memcpy(sendreq->data_storage, sendreq->data, sendreq->len);
+                   sendreq->data = sendreq->data_storage;
+                 }
+              }
+            if ((status != ARES_SUCCESS) || (sendreq->data_storage == NULL))
+              {
+                /* We encountered an error (probably a timeout,
+                 * suggesting the DNS server we're talking to is
+                 * probably unreachable, wedged, or severely
+                 * overloaded) or we couldn't copy the request, so
+                 * mark the connection as broken. When we get to
+                 * process_broken_connections() we'll close the
+                 * connection and try to re-send requests to another
+                 * server.
+                 */
+               server->is_broken = 1;
+               /* Just to be paranoid, zero out this sendreq... */
+               sendreq->data = NULL;
+               sendreq->len = 0;
+             }
+          }
     }
-  *q = query->next;
-  if (*q)
-    next = (*q)->next;
-  else
-    next = NULL;
-  free(query->tcpbuf);
-  free(query->skip_server);
-  free(query);
+
+  /* Invoke the callback */
+  query->callback(query->arg, status, query->timeouts, abuf, alen);
+  ares__free_query(query);
 
   /* Simple cleanup policy: if no queries are remaining, close all
    * network sockets unless STAYOPEN is set.
    */
-  if (!channel->queries && !(channel->flags & ARES_FLAG_STAYOPEN))
+  if (!(channel->flags & ARES_FLAG_STAYOPEN) &&
+      ares__is_list_empty(&(channel->all_queries)))
     {
       for (i = 0; i < channel->nservers; i++)
         ares__close_sockets(channel, &channel->servers[i]);
     }
-  return (next);
+}
+
+void ares__free_query(struct query *query)
+{
+  /* Remove the query from all the lists in which it is linked */
+  ares__remove_from_list(&(query->queries_by_qid));
+  ares__remove_from_list(&(query->queries_by_timeout));
+  ares__remove_from_list(&(query->queries_to_server));
+  ares__remove_from_list(&(query->all_queries));
+  /* Zero out some important stuff, to help catch bugs */
+  query->callback = NULL;
+  query->arg = NULL;
+  /* Deallocate the memory associated with the query */
+  free(query->tcpbuf);
+  free(query->server_info);
+  free(query);
 }

ares/ares_query.3

@@ -22,7 +22,7 @@ ares_query \- Initiate a single-question DNS query
 .B #include <ares.h>
 .PP
 .B typedef void (*ares_callback)(void *\fIarg\fP, int \fIstatus\fP,
-.B	unsigned char *\fIabuf\fP, int \fIalen\fP)
+.B	int \fItimeouts\fP, unsigned char *\fIabuf\fP, int \fIalen\fP)
 .PP
 .B void ares_query(ares_channel \fIchannel\fP, const char *\fIname\fP,
 .B 	int \fIdnsclass\fP, int \fItype\fP, ares_callback \fIcallback\fP,
@@ -124,6 +124,11 @@ The name service channel
 .I channel
 is being destroyed; the query will not be completed.
 .PP
+The callback argument
+.I timeouts
+reports how many times a query timed out during the execution of the
+given request.
+.PP
 If the query completed (even if there was something wrong with it, as
 indicated by some of the above error codes), the callback argument
 .I abuf

ares/ares_query.c

@@ -37,7 +37,7 @@ struct qquery {
   void *arg;
 };
 
-static void qcallback(void *arg, int status, unsigned char *abuf, int alen);
+static void qcallback(void *arg, int status, int timeouts, unsigned char *abuf, int alen);
 
 void ares__rc4(rc4_key* key, unsigned char *buffer_ptr, int buffer_len)
 {
@@ -53,13 +53,13 @@ void ares__rc4(rc4_key* key, unsigned char *buffer_ptr, int buffer_len)
   state = &key->state[0];
   for(counter = 0; counter < buffer_len; counter ++)
   {
-	x = (x + 1) % 256;
-	y = (state[x] + y) % 256;
-	ARES_SWAP_BYTE(&state[x], &state[y]);
+    x = (unsigned char)((x + 1) % 256);
+    y = (unsigned char)((state[x] + y) % 256);
+    ARES_SWAP_BYTE(&state[x], &state[y]);
 
-	xorIndex = (state[x] + state[y]) % 256;
+    xorIndex = (unsigned char)((state[x] + state[y]) % 256);
 
-	buffer_ptr[counter] ^= state[xorIndex];
+    buffer_ptr[counter] = (unsigned char)(buffer_ptr[counter]^state[xorIndex]);
   }
   key->x = x;
   key->y = y;
@@ -68,15 +68,19 @@ void ares__rc4(rc4_key* key, unsigned char *buffer_ptr, int buffer_len)
 static struct query* find_query_by_id(ares_channel channel, int id)
 {
   unsigned short qid;
-  struct query* q;
+  struct list_node* list_head;
+  struct list_node* list_node;
   DNS_HEADER_SET_QID(((unsigned char*)&qid), id);
 
   /* Find the query corresponding to this packet. */
-  for (q = channel->queries; q; q = q->next)
-  {
-	if (q->qid == qid)
+  list_head = &(channel->queries_by_qid[qid % ARES_QID_TABLE_SIZE]);
+  for (list_node = list_head->next; list_node != list_head;
+       list_node = list_node->next)
+    {
+       struct query *q = list_node->data;
+       if (q->qid == qid)
 	  return q;
-  }
+    }
   return NULL;
 }
 
@@ -110,7 +114,8 @@ void ares_query(ares_channel channel, const char *name, int dnsclass,
                         &qlen);
   if (status != ARES_SUCCESS)
     {
-      callback(arg, status, NULL, 0);
+      if (qbuf != NULL) free(qbuf);
+      callback(arg, status, 0, NULL, 0);
       return;
     }
 
@@ -121,7 +126,7 @@ void ares_query(ares_channel channel, const char *name, int dnsclass,
   if (!qquery)
     {
       ares_free_string(qbuf);
-      callback(arg, ARES_ENOMEM, NULL, 0);
+      callback(arg, ARES_ENOMEM, 0, NULL, 0);
       return;
     }
   qquery->callback = callback;
@@ -132,14 +137,14 @@ void ares_query(ares_channel channel, const char *name, int dnsclass,
   ares_free_string(qbuf);
 }
 
-static void qcallback(void *arg, int status, unsigned char *abuf, int alen)
+static void qcallback(void *arg, int status, int timeouts, unsigned char *abuf, int alen)
 {
   struct qquery *qquery = (struct qquery *) arg;
   unsigned int ancount;
   int rcode;
 
   if (status != ARES_SUCCESS)
-    qquery->callback(qquery->arg, status, abuf, alen);
+    qquery->callback(qquery->arg, status, timeouts, abuf, alen);
   else
     {
       /* Pull the response code and answer count from the packet. */
@@ -168,7 +173,7 @@ static void qcallback(void *arg, int status, unsigned char *abuf, int alen)
           status = ARES_EREFUSED;
           break;
         }
-      qquery->callback(qquery->arg, status, abuf, alen);
+      qquery->callback(qquery->arg, status, timeouts, abuf, alen);
     }
   free(qquery);
 }

ares/ares_search.3

@@ -22,7 +22,7 @@ ares_search \- Initiate a DNS query with domain search
 .B #include <ares.h>
 .PP
 .B typedef void (*ares_callback)(void *\fIarg\fP, int \fIstatus\fP,
-.B	unsigned char *\fIabuf\fP, int \fIalen\fP)
+.B	int \fItimeouts\fP, unsigned char *\fIabuf\fP, int \fIalen\fP)
 .PP
 .B void ares_search(ares_channel \fIchannel\fP, const char *\fIname\fP,
 .B 	int \fIdnsclass\fP, int \fItype\fP, ares_callback \fIcallback\fP,
@@ -125,6 +125,11 @@ The name service channel
 .I channel
 is being destroyed; the query will not be completed.
 .PP
+The callback argument
+.I timeouts
+reports how many times a query timed out during the execution of the
+given request.
+.PP
 If a query completed successfully, the callback argument
 .I abuf
 points to a result buffer of length

ares/ares_search.c

@@ -41,10 +41,12 @@ struct search_query {
   int status_as_is;             /* error status from trying as-is */
   int next_domain;              /* next search domain to try */
   int trying_as_is;             /* current query is for name as-is */
+  int timeouts;                 /* number of timeouts we saw for this request */
+  int ever_got_nodata;          /* did we ever get ARES_ENODATA along the way? */
 };
 
-static void search_callback(void *arg, int status, unsigned char *abuf,
-                            int alen);
+static void search_callback(void *arg, int status, int timeouts,
+                            unsigned char *abuf, int alen);
 static void end_squery(struct search_query *squery, int status,
                        unsigned char *abuf, int alen);
 static int cat_domain(const char *name, const char *domain, char **s);
@@ -64,7 +66,7 @@ void ares_search(ares_channel channel, const char *name, int dnsclass,
   status = single_domain(channel, name, &s);
   if (status != ARES_SUCCESS)
     {
-      callback(arg, status, NULL, 0);
+      callback(arg, status, 0, NULL, 0);
       return;
     }
   if (s)
@@ -80,7 +82,7 @@ void ares_search(ares_channel channel, const char *name, int dnsclass,
   squery = malloc(sizeof(struct search_query));
   if (!squery)
     {
-      callback(arg, ARES_ENOMEM, NULL, 0);
+      callback(arg, ARES_ENOMEM, 0, NULL, 0);
       return;
     }
   squery->channel = channel;
@@ -88,7 +90,7 @@ void ares_search(ares_channel channel, const char *name, int dnsclass,
   if (!squery->name)
     {
       free(squery);
-      callback(arg, ARES_ENOMEM, NULL, 0);
+      callback(arg, ARES_ENOMEM, 0, NULL, 0);
       return;
     }
   squery->dnsclass = dnsclass;
@@ -96,6 +98,8 @@ void ares_search(ares_channel channel, const char *name, int dnsclass,
   squery->status_as_is = -1;
   squery->callback = callback;
   squery->arg = arg;
+  squery->timeouts = 0;
+  squery->ever_got_nodata = 0;
 
   /* Count the number of dots in name. */
   ndots = 0;
@@ -132,18 +136,20 @@ void ares_search(ares_channel channel, const char *name, int dnsclass,
         /* failed, free the malloc()ed memory */
         free(squery->name);
         free(squery);
-        callback(arg, status, NULL, 0);
+        callback(arg, status, 0, NULL, 0);
       }
     }
 }
 
-static void search_callback(void *arg, int status, unsigned char *abuf,
-                            int alen)
+static void search_callback(void *arg, int status, int timeouts,
+                            unsigned char *abuf, int alen)
 {
   struct search_query *squery = (struct search_query *) arg;
   ares_channel channel = squery->channel;
   char *s;
                 
+  squery->timeouts += timeouts;
+
   /* Stop searching unless we got a non-fatal error. */
   if (status != ARES_ENODATA && status != ARES_ESERVFAIL
       && status != ARES_ENOTFOUND)
@@ -153,6 +159,17 @@ static void search_callback(void *arg, int status, unsigned char *abuf,
       /* Save the status if we were trying as-is. */
       if (squery->trying_as_is)
         squery->status_as_is = status;
+
+      /* 
+       * If we ever get ARES_ENODATA along the way, record that; if the search
+       * should run to the very end and we got at least one ARES_ENODATA,
+       * then callers like ares_gethostbyname() may want to try a T_A search
+       * even if the last domain we queried for T_AAAA resource records
+       * returned ARES_ENOTFOUND.
+       */
+      if (status == ARES_ENODATA)
+        squery->ever_got_nodata = 1;
+
       if (squery->next_domain < channel->ndomains)
         {
           /* Try the next domain. */
@@ -176,15 +193,20 @@ static void search_callback(void *arg, int status, unsigned char *abuf,
           ares_query(channel, squery->name, squery->dnsclass, squery->type,
                      search_callback, squery);
         }
-      else
-        end_squery(squery, squery->status_as_is, NULL, 0);
+      else {
+        if (squery->status_as_is == ARES_ENOTFOUND && squery->ever_got_nodata) {
+          end_squery(squery, ARES_ENODATA, NULL, 0);
+        }
+        else
+          end_squery(squery, squery->status_as_is, NULL, 0);
+      }
     }
 }
 
 static void end_squery(struct search_query *squery, int status,
                        unsigned char *abuf, int alen)
 {
-  squery->callback(squery->arg, status, abuf, alen);
+  squery->callback(squery->arg, status, squery->timeouts, abuf, alen);
   free(squery->name);
   free(squery);
 }

ares/ares_send.3

@@ -22,7 +22,7 @@ ares_send \- Initiate a DNS query
 .B #include <ares.h>
 .PP
 .B typedef void (*ares_callback)(void *\fIarg\fP, int \fIstatus\fP,
-.B	unsigned char *\fIabuf\fP, int \fIalen\fP)
+.B	int \fItimeouts\fP, unsigned char *\fIabuf\fP, int \fIalen\fP)
 .PP
 .B void ares_send(ares_channel \fIchannel\fP, const unsigned char *\fIqbuf\fP,
 .B 	int \fIqlen\fP, ares_callback \fIcallback\fP, void *\fIarg\fP)
@@ -79,6 +79,11 @@ The name service channel
 .I channel
 is being destroyed; the query will not be completed.
 .PP
+The callback argument
+.I timeouts
+reports how many times a query timed out during the execution of the
+given request.
+.PP
 If the query completed, the callback argument
 .I abuf
 points to a result buffer of length

ares/ares_send.c

@@ -44,7 +44,7 @@ void ares_send(ares_channel channel, const unsigned char *qbuf, int qlen,
   /* Verify that the query is at least long enough to hold the header. */
   if (qlen < HFIXEDSZ || qlen >= (1 << 16))
     {
-      callback(arg, ARES_EBADQUERY, NULL, 0);
+      callback(arg, ARES_EBADQUERY, 0, NULL, 0);
       return;
     }
 
@@ -52,22 +52,23 @@ void ares_send(ares_channel channel, const unsigned char *qbuf, int qlen,
   query = malloc(sizeof(struct query));
   if (!query)
     {
-      callback(arg, ARES_ENOMEM, NULL, 0);
+      callback(arg, ARES_ENOMEM, 0, NULL, 0);
       return;
     }
   query->tcpbuf = malloc(qlen + 2);
   if (!query->tcpbuf)
     {
       free(query);
-      callback(arg, ARES_ENOMEM, NULL, 0);
+      callback(arg, ARES_ENOMEM, 0, NULL, 0);
       return;
     }
-  query->skip_server = malloc(channel->nservers * sizeof(int));
-  if (!query->skip_server)
+  query->server_info = malloc(channel->nservers *
+                              sizeof(query->server_info[0]));
+  if (!query->server_info)
     {
       free(query->tcpbuf);
       free(query);
-      callback(arg, ARES_ENOMEM, NULL, 0);
+      callback(arg, ARES_ENOMEM, 0, NULL, 0);
       return;
     }
 
@@ -93,13 +94,28 @@ void ares_send(ares_channel channel, const unsigned char *qbuf, int qlen,
   query->try = 0;
   query->server = 0;
   for (i = 0; i < channel->nservers; i++)
-    query->skip_server[i] = 0;
+    {
+      query->server_info[i].skip_server = 0;
+      query->server_info[i].tcp_connection_generation = 0;
+    }
   query->using_tcp = (channel->flags & ARES_FLAG_USEVC) || qlen > PACKETSZ;
   query->error_status = ARES_ECONNREFUSED;
+  query->timeouts = 0;
 
-  /* Chain the query into this channel's query list. */
-  query->next = channel->queries;
-  channel->queries = query;
+  /* Initialize our list nodes. */
+  ares__init_list_node(&(query->queries_by_qid),     query);
+  ares__init_list_node(&(query->queries_by_timeout), query);
+  ares__init_list_node(&(query->queries_to_server),  query);
+  ares__init_list_node(&(query->all_queries),        query);
+       
+  /* Chain the query into the list of all queries. */
+  ares__insert_in_list(&(query->all_queries), &(channel->all_queries));
+  /* Keep track of queries bucketed by qid, so we can process DNS
+   * responses quickly.
+   */
+  ares__insert_in_list(
+      &(query->queries_by_qid),
+      &(channel->queries_by_qid[query->qid % ARES_QID_TABLE_SIZE]));
 
   /* Perform the first query action. */
   time(&now);

ares/ares_strerror.c

@@ -46,6 +46,8 @@ const char *ares_strerror(int code)
     "Illegal hints flags specified"
   };
 
-  DEBUGASSERT(code >= 0 && code < (int)(sizeof(errtext) / sizeof(*errtext)));
-  return errtext[code];
+  if(code >= 0 && code < (int)(sizeof(errtext) / sizeof(*errtext)))
+    return errtext[code];
+  else
+    return "unknown";
 }

ares/ares_timeout.c

@@ -26,23 +26,34 @@
 #include "ares.h"
 #include "ares_private.h"
 
+/* WARNING: Beware that this is linear in the number of outstanding
+ * requests! You are probably far better off just calling ares_process()
+ * once per second, rather than calling ares_timeout() to figure out
+ * when to next call ares_process().
+ */
 struct timeval *ares_timeout(ares_channel channel, struct timeval *maxtv,
                              struct timeval *tvbuf)
 {
   struct query *query;
+  struct list_node* list_head;
+  struct list_node* list_node;
   time_t now;
   time_t offset, min_offset; /* these use time_t since some 32 bit systems
                                 still use 64 bit time_t! (like VS2005) */
 
   /* No queries, no timeout (and no fetch of the current time). */
-  if (!channel->queries)
+  if (ares__is_list_empty(&(channel->all_queries)))
     return maxtv;
 
   /* Find the minimum timeout for the current set of queries. */
   time(&now);
   min_offset = -1;
-  for (query = channel->queries; query; query = query->next)
+
+  list_head = &(channel->all_queries);
+  for (list_node = list_head->next; list_node != list_head;
+       list_node = list_node->next)
     {
+      query = list_node->data;
       if (query->timeout == 0)
         continue;
       offset = query->timeout - now;

ares/ares_version.h

@@ -4,12 +4,12 @@
 #define ARES__VERSION_H
 
 #define ARES_VERSION_MAJOR 1
-#define ARES_VERSION_MINOR 4
-#define ARES_VERSION_PATCH 1
+#define ARES_VERSION_MINOR 5
+#define ARES_VERSION_PATCH 0
 #define ARES_VERSION ((ARES_VERSION_MAJOR<<16)|\
                        (ARES_VERSION_MINOR<<8)|\
                        (ARES_VERSION_PATCH))
-#define ARES_VERSION_STR "1.4.1-CVS"
+#define ARES_VERSION_STR "1.5.0-CVS"
 
 #ifdef  __cplusplus
 extern "C" {

ares/config-win32.h

@@ -146,11 +146,6 @@
 #define ssize_t int
 #endif
 
-/* Define to 'int' if socklen_t is not an available 'typedefed' type */
-#ifndef HAVE_WS2TCPIP_H
-#define socklen_t int
-#endif
-
 /* ---------------------------------------------------------------- */
 /*                          STRUCT RELATED                          */
 /* ---------------------------------------------------------------- */

ares/configure.ac

@@ -117,6 +117,43 @@ dnl gethostbyname_r() version
 dnl **********************************************************************
 CURL_DETECT_ICC([CFLAGS="$CFLAGS -we 147"])
 
+dnl **********************************************************************
+dnl platform/compiler/architecture specific checks/flags
+dnl **********************************************************************
+
+case $host in
+  #
+  x86_64*linux*)
+    #
+    dnl find out if icc is being used
+    if test "z$ICC" = "z"; then
+      CURL_DETECT_ICC
+    fi
+    #
+    if test "$ICC" = "yes"; then
+      dnl figure out icc version
+      AC_MSG_CHECKING([icc version])
+      iccver=`$CC -dumpversion`
+      iccnhi=`echo $iccver | cut -d . -f1`
+      iccnlo=`echo $iccver | cut -d . -f2`
+      iccnum=`(expr $iccnhi "*" 100 + $iccnlo) 2>/dev/null`
+      AC_MSG_RESULT($iccver)
+      #
+      if test "$iccnum" -ge "900" && test "$iccnum" -lt "1000"; then
+        dnl icc 9.X specific
+        CFLAGS="$CFLAGS -i-dynamic"
+      fi
+      #
+      if test "$iccnum" -ge "1000"; then
+        dnl icc 10.X or later
+        CFLAGS="$CFLAGS -shared-intel"
+      fi
+      #
+    fi
+    ;;
+  #
+esac
+
 dnl **********************************************************************
 dnl Checks for libraries.
 dnl **********************************************************************
@@ -340,6 +377,7 @@ AC_CHECK_HEADERS(
        sys/param.h \
        netdb.h \
        netinet/in.h \
+       netinet/tcp.h \
        net/if.h \
        errno.h \
        stdbool.h \

ares/nameser.h

@@ -149,6 +149,11 @@ typedef enum __ns_opcode {
 
 #define T_CNAME                ns_t_cname
 
+#define NS_MAXDNAME   256     /* maximum domain name */
+#define MAXDNAME      NS_MAXDNAME
+
+#define NS_MAXCDNAME  255     /* maximum compressed domain name */
+#define MAXCDNAME     NS_MAXCDNAME
 
 #define NS_PACKETSZ   512     /* maximum packet size */
 #define PACKETSZ       NS_PACKETSZ

ares/setup.h

@@ -97,10 +97,6 @@
 #define ssize_t int
 #endif
 
-#ifndef HAVE_WS2TCPIP_H
-#define socklen_t int
-#endif
-
 #endif /* HAVE_CONFIG_H */
 
 /*

ares/setup_once.h

@@ -91,6 +91,31 @@ struct timeval {
 #endif
 
 
+/*
+ * Windows build targets have socklen_t definition in
+ * ws2tcpip.h but some versions of ws2tcpip.h do not
+ * have the definition. It seems that when the socklen_t
+ * definition is missing from ws2tcpip.h the definition
+ * for INET_ADDRSTRLEN is also missing, and that when one
+ * definition is present the other one also is available.
+ */
+
+#if defined(WIN32) && !defined(HAVE_SOCKLEN_T)
+#  if ( defined(_MSC_VER) && !defined(INET_ADDRSTRLEN) ) || \
+      (!defined(_MSC_VER) && !defined(HAVE_WS2TCPIP_H) )
+#    define socklen_t int
+#    define HAVE_SOCKLEN_T
+#  endif
+#endif
+
+
+#if defined(__minix)
+/* Minix doesn't support recv on TCP sockets */
+#define sread(x,y,z) (ssize_t)read((RECV_TYPE_ARG1)(x), \
+                                   (RECV_TYPE_ARG2)(y), \
+                                   (RECV_TYPE_ARG3)(z))
+
+#elif defined(HAVE_RECV)
 /*
  * The definitions for the return type and arguments types
  * of functions recv() and send() belong and come from the
@@ -113,7 +138,6 @@ struct timeval {
  * SEND_TYPE_RETV must also be defined.
  */
 
-#ifdef HAVE_RECV
 #if !defined(RECV_TYPE_ARG1) || \
     !defined(RECV_TYPE_ARG2) || \
     !defined(RECV_TYPE_ARG3) || \
@@ -136,7 +160,14 @@ struct timeval {
 #endif
 #endif /* HAVE_RECV */
 
-#ifdef HAVE_SEND
+
+#if defined(__minix)
+/* Minix doesn't support send on TCP sockets */
+#define swrite(x,y,z) (ssize_t)write((SEND_TYPE_ARG1)(x), \
+                                    (SEND_TYPE_ARG2)(y), \
+                                    (SEND_TYPE_ARG3)(z))
+
+#elif defined(HAVE_SEND)
 #if !defined(SEND_TYPE_ARG1) || \
     !defined(SEND_QUAL_ARG2) || \
     !defined(SEND_TYPE_ARG2) || \
@@ -358,5 +389,96 @@ typedef int sig_atomic_t;
 #endif
 
 
+/*
+ * We use this ZERO_NULL to avoid picky compiler warnings,
+ * when assigning a NULL pointer to a function pointer var.
+ */
+
+#define ZERO_NULL 0
+
+
+#if defined (__LP64__) && defined(__hpux) && !defined(_XOPEN_SOURCE_EXTENDED)
+#include <sys/socket.h>
+/* HP-UX has this oddity where it features a few functions that don't work
+   with socklen_t so we need to convert to ints
+
+   This is due to socklen_t being a 64bit int under 64bit ABI, but the
+   pre-xopen (default) interfaces require an int, which is 32bits.
+
+   Therefore, Anytime socklen_t is passed by pointer, the libc function
+   truncates the 64bit socklen_t value by treating it as a 32bit value.
+
+
+   Note that some socket calls are allowed to have a NULL pointer for
+   the socklen arg.
+*/
+
+inline static int Curl_hp_getsockname(int s, struct sockaddr *name,
+                                      socklen_t *namelen)
+{
+  int rc;
+  if(namelen) {
+     int len = *namelen;
+     rc = getsockname(s, name, &len);
+     *namelen = len;
+   }
+  else
+     rc = getsockname(s, name, 0);
+  return rc;
+}
+
+inline static int Curl_hp_getsockopt(int  s, int level, int optname,
+                                     void *optval, socklen_t *optlen)
+{
+  int rc;
+  if(optlen) {
+    int len = *optlen;
+    rc = getsockopt(s, level, optname, optval, &len);
+    *optlen = len;
+  }
+  else
+    rc = getsockopt(s, level, optname, optval, 0);
+  return rc;
+}
+
+inline static int Curl_hp_accept(int sockfd, struct sockaddr *addr,
+                                 socklen_t *addrlen)
+{
+  int rc;
+  if(addrlen) {
+     int len = *addrlen;
+     rc = accept(sockfd, addr, &len);
+     *addrlen = len;
+  }
+  else
+     rc = accept(sockfd, addr, 0);
+  return rc;
+}
+
+
+inline static ssize_t Curl_hp_recvfrom(int s, void *buf, size_t len, int flags,
+                                       struct sockaddr *from,
+                                       socklen_t *fromlen)
+{
+  ssize_t rc;
+  if(fromlen) {
+    int fromlen32 = *fromlen;
+    rc = recvfrom(s, buf, len, flags, from, &fromlen32);
+    *fromlen = fromlen32;
+  }
+  else {
+    rc = recvfrom(s, buf, len, flags, from, 0);
+  }
+  return rc;
+}
+
+#define getsockname(a,b,c) Curl_hp_getsockname((a),(b),(c))
+#define getsockopt(a,b,c,d,e) Curl_hp_getsockopt((a),(b),(c),(d),(e))
+#define accept(a,b,c) Curl_hp_accept((a),(b),(c))
+#define recvfrom(a,b,c,d,e,f) Curl_hp_recvfrom((a),(b),(c),(d),(e),(f))
+
+#endif /* HPUX work-around */
+
+
 #endif /* __SETUP_ONCE_H */
 

ares/vc/areslib/areslib.dsp

@@ -137,6 +137,10 @@ SOURCE=..\..\ares_init.c
 # End Source File
 # Begin Source File
 
+SOURCE=..\..\ares_llist.c
+# End Source File
+# Begin Source File
+
 SOURCE=..\..\ares_mkquery.c
 # End Source File
 # Begin Source File
@@ -213,6 +217,10 @@ SOURCE=..\..\ares_ipv6.h
 # End Source File
 # Begin Source File
 
+SOURCE=..\..\ares_llist.h
+# End Source File
+# Begin Source File
+
 SOURCE=..\..\ares_private.h
 # End Source File
 # Begin Source File

configure.ac

@@ -55,6 +55,7 @@ AC_SUBST(AR)
 if test "x$AR" = "xar-was-not-found-by-configure"; then
   AC_MSG_WARN([ar was not found, this may ruin your chances to build fine])
 fi
+AC_SUBST(libext)
 
 dnl figure out the libcurl version
 VERSION=`$SED -ne 's/^#define LIBCURL_VERSION "\(.*\)"/\1/p' ${srcdir}/include/curl/curlver.h`
@@ -221,6 +222,45 @@ CURL_CHECK_HEADER_WINSOCK
 CURL_CHECK_HEADER_WINSOCK2
 CURL_CHECK_HEADER_WS2TCPIP
 
+CURL_CHECK_HEADER_WINLDAP
+CURL_CHECK_HEADER_WINBER
+
+dnl **********************************************************************
+dnl platform/compiler/architecture specific checks/flags
+dnl **********************************************************************
+
+case $host in
+  #
+  x86_64*linux*)
+    #
+    dnl find out if icc is being used
+    if test "z$ICC" = "z"; then
+      CURL_DETECT_ICC
+    fi
+    #
+    if test "$ICC" = "yes"; then
+      dnl figure out icc version
+      AC_MSG_CHECKING([icc version])
+      iccver=`$CC -dumpversion`
+      iccnhi=`echo $iccver | cut -d . -f1`
+      iccnlo=`echo $iccver | cut -d . -f2`
+      iccnum=`(expr $iccnhi "*" 100 + $iccnlo) 2>/dev/null`
+      AC_MSG_RESULT($iccver)
+      #
+      if test "$iccnum" -ge "900" && test "$iccnum" -lt "1000"; then
+        dnl icc 9.X specific
+        CFLAGS="$CFLAGS -i-dynamic"
+      fi
+      #
+      if test "$iccnum" -ge "1000"; then
+        dnl icc 10.X or later
+        CFLAGS="$CFLAGS -shared-intel"
+      fi
+      #
+    fi
+    ;;
+  #
+esac
 
 dnl ************************************************************
 dnl switch off particular protocols
@@ -544,6 +584,11 @@ AC_HELP_STRING([--with-lber-lib=libname],[Specify name of lber lib file]),
 
 if test x$CURL_DISABLE_LDAP != x1 ; then
 
+  CURL_CHECK_HEADER_LBER
+  CURL_CHECK_HEADER_LDAP
+  CURL_CHECK_HEADER_LDAPSSL
+  CURL_CHECK_HEADER_LDAP_SSL
+
   if test -z "$LDAPLIBNAME" ; then
     case $host in
       *-*-cygwin* | *-*-mingw* | *-*-pw32*)
@@ -560,11 +605,15 @@ if test x$CURL_DISABLE_LDAP != x1 ; then
       AC_DEFINE(CURL_DISABLE_LDAP, 1, [to disable LDAP])
       AC_SUBST(CURL_DISABLE_LDAP, [1])])
   else
-    dnl Try to find the right ldap library name for this system
-    AC_SEARCH_LIBS(ldap_init, [ldap],, [
-      AC_MSG_WARN([Cannot find LDAP library: LDAP disabled])
-      AC_DEFINE(CURL_DISABLE_LDAP, 1, [to disable LDAP])
-      AC_SUBST(CURL_DISABLE_LDAP, [1])])
+    dnl Try to find the right ldap libraries for this system
+    CURL_CHECK_LIBS_LDAP
+    case X-"$curl_cv_ldap_LIBS" in
+      X-unknown)
+        AC_MSG_WARN([Cannot find libraries for LDAP support: LDAP disabled])
+        AC_DEFINE(CURL_DISABLE_LDAP, 1, [to disable LDAP])
+        AC_SUBST(CURL_DISABLE_LDAP, [1])
+        ;;
+    esac
   fi
 fi
 
@@ -579,12 +628,6 @@ if test x$CURL_DISABLE_LDAP != x1 ; then
         AC_DEFINE(CURL_DISABLE_LDAP, 1, [to disable LDAP])
         AC_SUBST(CURL_DISABLE_LDAP, [1])])
     fi
-  else
-    dnl Try to find the right lber library name for this system
-    AC_SEARCH_LIBS(ber_free, [lber],, [
-      AC_MSG_WARN([Cannot find a library defining ber_free(): LDAP disabled])
-      AC_DEFINE(CURL_DISABLE_LDAP, 1, [to disable LDAP])
-      AC_SUBST(CURL_DISABLE_LDAP, [1])])
   fi
 fi
 
@@ -1314,6 +1357,7 @@ if test X"$OPT_LIBSSH2" != Xno; then
     curl_ssh_msg="enabled (libSSH2)"
     LIBSSH2_ENABLED=1
     AC_DEFINE(USE_LIBSSH2, 1, [if libSSH2 is in use]))
+    AC_SUBST(USE_LIBSSH2, [1])
 
   if test X"$OPT_LIBSSH2" != Xoff &&
      test "$LIBSSH2_ENABLED" != "1"; then
@@ -1469,6 +1513,14 @@ if test "$OPENSSL_ENABLED" != "1" -a "$GNUTLS_ENABLED" != "1"; then
       version="unknown"
       gtlsprefix=$OPT_GNUTLS
     fi
+
+    dnl Check for functionPK11_CreateGenericObject
+    dnl this is needed for using the PEM PKCS#11 module
+    AC_CHECK_LIB(nss3, PK11_CreateGenericObject-d,
+     [
+     AC_DEFINE(HAVE_PK11_CREATEGENERICOBJECT, 1, [if you have the function PK11_CreateGenericObject])
+     AC_SUBST(HAVE_PK11_CREATEGENERICOBJECT, [1])
+     ])
     if test -n "$addlib"; then
 
       CLEANLIBS="$LIBS"
@@ -1520,7 +1572,7 @@ dnl **********************************************************************
 dnl Check for the CA bundle
 dnl **********************************************************************
 
-if test X"$USE_GNUTLS$OPENSSL_ENABLED" != "X"; then
+if test X"$USE_NSS$USE_GNUTLS$OPENSSL_ENABLED" != "X"; then
 
   AC_MSG_CHECKING([CA cert bundle install path])
 
@@ -1755,7 +1807,6 @@ AC_CHECK_HEADERS(
         assert.h \
         unistd.h \
         stdlib.h \
-        ldap_ssl.h \
         limits.h \
         arpa/inet.h \
         net/if.h \
@@ -1891,6 +1942,7 @@ AC_CHECK_FUNCS( strtoll \
                 select \
                 strdup \
                 strstr \
+                strcasestr \
                 strtok_r \
                 uname \
                 strcasecmp \
@@ -2189,7 +2241,6 @@ AC_HELP_STRING([--disable-verbose],[Disable verbose strings]),
   no)
        AC_MSG_RESULT(no)
        AC_DEFINE(CURL_DISABLE_VERBOSE_STRINGS, 1, [to disable verbose strings])
-       AC_SUBST(CURL_DISABLE_VERBOSE_STRINGS)
        curl_verbose_msg="no"
        ;;
   *)   AC_MSG_RESULT(yes)
@@ -2212,7 +2263,7 @@ AC_HELP_STRING([--disable-sspi],[Disable SSPI]),
        if test "$ac_cv_native_windows" = "yes"; then
          AC_MSG_RESULT(yes)
          AC_DEFINE(USE_WINDOWS_SSPI, 1, [to enable SSPI support])
-         AC_SUBST(USE_WINDOWS_SSPI)
+         AC_SUBST(USE_WINDOWS_SSPI, [1])
          curl_sspi_msg="yes"
        else
          AC_MSG_RESULT(no)
@@ -2262,7 +2313,6 @@ AC_HELP_STRING([--disable-crypto-auth],[Disable cryptographic authentication]),
   no)
        AC_MSG_RESULT(no)
        AC_DEFINE(CURL_DISABLE_CRYPTO_AUTH, 1, [to disable cryptographic authentication])
-       AC_SUBST(CURL_DISABLE_CRYPTO_AUTH)
        ;;
   *)   AC_MSG_RESULT(yes)
        ;;
@@ -2281,7 +2331,6 @@ AC_HELP_STRING([--disable-cookies],[Disable cookies support]),
   no)
        AC_MSG_RESULT(no)
        AC_DEFINE(CURL_DISABLE_COOKIES, 1, [to disable cookies support])
-       AC_SUBST(CURL_DISABLE_COOKIES)
        ;;
   *)   AC_MSG_RESULT(yes)
        ;;
@@ -2308,9 +2357,7 @@ AC_HELP_STRING([--disable-hidden-symbols],[Leave all symbols with default visibi
          if $CC --help --verbose 2>&1 | grep fvisibility= > /dev/null ; then
            AC_MSG_RESULT(yes)
            AC_DEFINE(CURL_HIDDEN_SYMBOLS, 1, [to enable hidden symbols])
-           AC_SUBST(CURL_HIDDEN_SYMBOLS)
            AC_DEFINE(CURL_EXTERN_SYMBOL, [__attribute__ ((visibility ("default")))], [to make a symbol visible])
-           AC_SUBST(CURL_EXTERN_SYMBOL)
            CFLAGS="$CFLAGS -fvisibility=hidden"
          else
             AC_MSG_RESULT(no)
@@ -2321,9 +2368,7 @@ AC_HELP_STRING([--disable-hidden-symbols],[Leave all symbols with default visibi
          if $CC 2>&1 | grep flags >/dev/null && $CC -flags | grep xldscope= >/dev/null ; then
            AC_MSG_RESULT(yes)
            AC_DEFINE(CURL_HIDDEN_SYMBOLS, 1, [to enable hidden symbols])
-           AC_SUBST(CURL_HIDDEN_SYMBOLS)
            AC_DEFINE(CURL_EXTERN_SYMBOL, [__global], [to make a symbol visible])
-           AC_SUBST(CURL_EXTERN_SYMBOL)
            CFLAGS="$CFLAGS -xldscope=hidden"
          else
            AC_MSG_RESULT(no)

curl-config.in

@@ -41,10 +41,11 @@ Available values for OPTION include:
   --cflags    pre-processor and compiler flags
   --checkfor [version] check for (lib)curl of the specified version
   --features  newline separated list of enabled features
-  --protocols newline separated list of enabled protocols
   --help      display this help and exit
   --libs      library linking information
   --prefix    curl install prefix
+  --protocols newline separated list of enabled protocols
+  --static-libs static libcurl library linking information
   --version   output version information
   --vernum    output the version information as a number (hexadecimal)
 EOF
@@ -131,12 +132,19 @@ while test $# -gt 0; do
 	if test "@CURL_DISABLE_LDAP@" != "1"; then
           echo "LDAP"
         fi
+	if test "@CURL_DISABLE_LDAPS@" != "1"; then
+          echo "LDAPS"
+        fi
 	if test "@CURL_DISABLE_DICT@" != "1"; then
           echo "DICT"
         fi
 	if test "@CURL_DISABLE_TFTP@" != "1"; then
           echo "TFTP"
         fi
+	if test "@USE_LIBSSH2@" = "1"; then
+          echo "SCP"
+          echo "SFTP"
+        fi
 	;;
     --version)
 	echo libcurl @VERSION@
@@ -193,6 +201,10 @@ while test $# -gt 0; do
 	fi
 	;;
 
+    --static-libs)
+	echo @libdir@/libcurl.@libext@ @LDFLAGS@ @LIBCURL_LIBS@ @LIBS@
+	;;
+
     *)
         echo "unknown option: $1"
 	usage 1

docs/BINDINGS

@@ -32,7 +32,7 @@ C
 C++
 
   Written by Jean-Philippe Barrette-LaPierre
-  http://rrette.com/curlpp.html
+  http://rrette.com/textpattern/index.php?s=cURLpp
 
 Ch
 
@@ -82,9 +82,12 @@ Lisp
 
 Lua
 
-  LuaCURL Written by Alexander Marinov
+  luacurl by Alexander Marinov
   http://luacurl.luaforge.net/
 
+  Lua-cURL by J<>rgen H<>tzel
+  http://luaforge.net/projects/lua-curl/
+
 Mono
 
   Written by Jeffrey Phillips
@@ -92,7 +95,7 @@ Mono
 
 .NET
 
-  libcurl-net Written by Jeffrey Phillips
+  libcurl-net by Jeffrey Phillips
   http://sourceforge.net/projects/libcurl-net/
 
 Object-Pascal
@@ -127,12 +130,12 @@ PostgreSQL
 
 Python
 
-  PycURL is written by Kjetil Jacobsen
+  PycURL by Kjetil Jacobsen
   http://pycurl.sourceforge.net/
 
 R
 
-  RCurl is written by Duncan Temple Lang
+  RCurl by Duncan Temple Lang
   http://www.omegahat.org/RCurl/
 
 Rexx
@@ -147,17 +150,17 @@ Ruby
 
 Scheme
 
-  Bigloo binding written by Kirill Lisovsky
+  Bigloo binding by Kirill Lisovsky
   http://curl.haxx.se/libcurl/scheme/
 
 S-Lang
 
-  S-Lang binding written by John E Davis
+  S-Lang binding by John E Davis
   http://www.jedsoft.org/slang/modules/curl.html
 
 Smalltalk
 
-  Smalltalk binding written by Danil Osipchuk
+  Smalltalk binding by Danil Osipchuk
   http://www.squeaksource.com/CurlPlugin/
 
 SP-Forth
@@ -166,17 +169,17 @@ SP-Forth
 
 SPL
 
-  SPL binding written by Clifford Wolf
+  SPL binding by Clifford Wolf
   http://www.clifford.at/spl/
 
 Tcl
 
-  Tclcurl is written by Andr<64>s Garc<72>a
+  Tclcurl by Andr<64>s Garc<72>a
   http://personal1.iddeo.es/andresgarci/tclcurl/english/docs.html
 
 Visual Basic
 
-  libcurl-vb is written by Jeffrey Phillips
+  libcurl-vb by Jeffrey Phillips
   http://sourceforge.net/projects/libcurl-vb/
 
 Q

docs/CONTRIBUTE

@@ -10,16 +10,46 @@
  mind when you decide to contribute to the project. This concerns new features
  as well as corrections to existing flaws or bugs.
 
-Join the Community
+ 1. Learning cURL
+ 1.1 Join the Community
+ 1.2 License
+ 1.3 What To Read
+
+ 2. cURL Coding Standards
+ 2.1 Naming
+ 2.2 Indenting
+ 2.3 Commenting
+ 2.4 Line Lengths
+ 2.5 General Style
+ 2.6 Non-clobbering All Over
+ 2.7 Platform Dependent Code
+ 2.8 Write Separate Patches
+ 2.9 Patch Against Recent Sources
+ 2.10 Document
+ 2.11 Test Cases
+
+ 3. Pushing Out Your Changes 
+ 3.1 Write Access to CVS Repository
+ 3.2 How To Make a Patch
+ 3.3 How to get your changes into the main sources
+
+==============================================================================
+
+1. Learning cURL
+
+1.1 Join the Community
 
  Skip over to http://curl.haxx.se/mail/ and join the appropriate mailing
  list(s).  Read up on details before you post questions. Read this file before
  you start sending patches! We prefer patches and discussions being held on
  the mailing list(s), not sent to individuals.
 
+ Before posting to one of the curl mailing lists, please read up on the mailing
+ list etiquette: http://curl.haxx.se/mail/etiquette.html
+
  We also hang out on IRC in #curl on irc.freenode.net
 
-License
+1.2. License
 
  When contributing with code, you agree to put your changes and new code under
  the same license curl and libcurl is already using unless stated and agreed
@@ -43,14 +73,16 @@ License
  give credit but also to keep a trace back to who made what changes. Please
  always provide us with your full real name when contributing!
 
-What To Read
+1.3 What To Read
 
  Source code, the man pages, the INTERNALS document, TODO, KNOWN_BUGS, the
  most recent CHANGES. Just lurking on the libcurl mailing list is gonna give
  you a lot of insights on what's going on right now. Asking there is a good
  idea too.
 
-Naming
+2. cURL Coding Standards
+
+2.1 Naming
 
  Try using a non-confusing naming scheme for your new functions and variable
  names. It doesn't necessarily have to mean that you should use the same as in
@@ -61,7 +93,7 @@ Naming
  See the INTERNALS document on how we name non-exported library-global
  symbols.
 
-Indenting
+2.2 Indenting
 
  Please try using the same indenting levels and bracing method as all the
  other code already does. It makes the source code a lot easier to follow if
@@ -70,7 +102,7 @@ Indenting
  using spaces only (no tabs) and having the opening brace ({) on the same line
  as the if() or while().
 
-Commenting
+2.3 Commenting
 
  Comment your source code extensively using C comments (/* comment */), DO NOT
  use C++ comments (// this style). Commented code is quality code and enables
@@ -78,12 +110,16 @@ Commenting
  replaced when someone wants to extend things, since other persons' source
  code can get quite hard to read.
 
-General Style
+2.4 Line Lengths
+
+ We try to keep source lines shorter than 80 columns.
+
+2.5 General Style
 
  Keep your functions small. If they're small you avoid a lot of mistakes and
  you don't accidentally mix up variables etc.
 
-Non-clobbering All Over
+2.6 Non-clobbering All Over
 
  When you write new functionality or fix bugs, it is important that you don't
  fiddle all over the source files and functions. Remember that it is likely
@@ -92,14 +128,14 @@ Non-clobbering All Over
  functionality, try writing it in a new source file. If you fix bugs, try to
  fix one bug at a time and send them as separate patches.
 
-Platform Dependent Code
+2.7 Platform Dependent Code
 
  Use #ifdef HAVE_FEATURE to do conditional code. We avoid checking for
  particular operating systems or hardware in the #ifdef lines. The
  HAVE_FEATURE shall be generated by the configure script for unix-like systems
  and they are hard-coded in the config-[system].h files for the others.
 
-Separate Patches
+2.8 Write Separate Patches
 
  It is annoying when you get a huge patch from someone that is said to fix 511
  odd problems, but discussions and opinions don't agree with 510 of them - or
@@ -110,14 +146,14 @@ Separate Patches
  description exactly what they correct so that all patches can be selectively
  applied by the maintainer or other interested parties.
 
-Patch Against Recent Sources
+2.9 Patch Against Recent Sources
 
  Please try to get the latest available sources to make your patches
  against. It makes the life of the developers so much easier. The very best is
  if you get the most up-to-date sources from the CVS repository, but the
  latest release archive is quite OK as well!
 
-Document
+2.10 Document
 
  Writing docs is dead boring and one of the big problems with many open source
  projects. Someone's gotta do it. It makes it a lot easier if you submit a
@@ -128,16 +164,7 @@ Document
  ASCII files. All HTML files on the web site and in the release archives are
  generated from the nroff/ASCII versions.
 
-Write Access to CVS Repository
-
- If you are a frequent contributor, or have another good reason, you can of
- course get write access to the CVS repository and then you'll be able to
- check-in all your changes straight into the CVS tree instead of sending all
- changes by mail as patches. Just ask if this is what you'd want. You will be
- required to have posted a few quality patches first, before you can be
- granted write access.
-
-Test Cases
+2.11 Test Cases
 
  Since the introduction of the test suite, we can quickly verify that the main
  features are working as they're supposed to. To maintain this situation and
@@ -146,7 +173,18 @@ Test Cases
  test case that verifies that it works as documented. If every submitter also
  posts a few test cases, it won't end up as a heavy burden on a single person!
 
-How To Make a Patch
+3. Pushing Out Your Changes 
+
+3.1 Write Access to CVS Repository
+
+ If you are a frequent contributor, or have another good reason, you can of
+ course get write access to the CVS repository and then you'll be able to
+ check-in all your changes straight into the CVS tree instead of sending all
+ changes by mail as patches. Just ask if this is what you'd want. You will be
+ required to have posted a few quality patches first, before you can be
+ granted write access.
+
+3.2 How To Make a Patch
 
  Keep a copy of the unmodified curl sources. Make your changes in a separate
  source tree. When you think you have something that you want to offer the
@@ -166,15 +204,15 @@ How To Make a Patch
 
  For unix-like operating systems:
 
-        http://www.fsf.org/software/patch/patch.html
-        http://www.gnu.org/directory/diffutils.html
+   http://www.gnu.org/software/patch/patch.html
+   http://www.gnu.org/directory/diffutils.html
 
  For Windows:
 
-        http://gnuwin32.sourceforge.net/packages/patch.htm
-        http://gnuwin32.sourceforge.net/packages/diffutils.htm
+   http://gnuwin32.sourceforge.net/packages/patch.htm
+   http://gnuwin32.sourceforge.net/packages/diffutils.htm
 
-How to get your patches into the libcurl sources
+3.3 How to get your changes into the main sources
 
  1. Submit your patch to the curl-library mailing list
 
@@ -189,5 +227,5 @@ How to get your patches into the libcurl sources
     simply drop such patches from my TODO list.
 
  5. If you've followed the above mentioned paragraphs and your patch still
-    hasn't been incorporated after some weeks, consider resubmitting them to
-    the list.
+    hasn't been incorporated after some weeks, consider resubmitting it to the
+    list.

docs/INSTALL

@@ -714,6 +714,9 @@ REDUCING SIZE
 
       ./configure CFLAGS='-Os' ...
 
+   Note that newer compilers often produce smaller code than older versions
+   due to better optimization.
+
    Be sure to specify as many --disable- and --without- flags on the configure
    command-line as you can to disable all the libcurl features that you
    know your application is not going to need.  Besides specifying the
@@ -740,9 +743,9 @@ REDUCING SIZE
    sections of the shared library using the -R option to objcopy (e.g. the
    .comment section).
 
-   Using these techniques it is possible to create an HTTP-only shared
-   libcurl library for i386 Linux platforms that is less than 90 KB in
-   size (as of version 7.15.4).
+   Using these techniques it is possible to create an HTTP-only shared libcurl
+   library for i386 Linux platforms that is only 96 KiB in size (as of libcurl
+   version 7.17.1, using gcc 4.2.2).
 
    You may find that statically linking libcurl to your application will
    result in a lower total size.

docs/INTERNALS

@@ -1,4 +1,3 @@
-                                    Updated for curl 7.9.1 on November 2, 2001
                                   _   _ ____  _     
                               ___| | | |  _ \| |    
                              / __| | | | |_) | |    
@@ -37,8 +36,8 @@ Windows vs Unix
 
  2. Windows requires a couple of init calls for the socket stuff.
 
-   Those must be made by the application that uses libcurl, in curl that means
-   src/main.c has some code #ifdef'ed to do just that.
+   That's taken care of by the curl_global_init() call, but if other libs also
+   do it etc there might be reasons for applications to alter that behaviour.
 
  3. The file descriptors for network communication and file operations are
     not easily interchangable as in unix.

docs/KNOWN_BUGS

@@ -3,12 +3,10 @@ join in and help us correct one or more of these! Also be sure to check the
 changelog of the current development status, as one or more of these problems
 may have been fixed since this was written!
 
-47. If a CONNECT response is larger than BUFSIZE when the connection is meant
-  to be kept alive, the function will return prematurely and will confuse the
-  rest of the HTTP protocol code.
-
-46. If a CONNECT response is chunked-encoded, the function may return
-  prematurely and will confuse the rest of the HTTP protocol code.
+48. If a CONNECT response-headers are larger than BUFSIZE (16KB) when the
+  connection is meant to be kept alive (like for NTLM proxy auth), the
+  function will return prematurely and will confuse the rest of the HTTP
+  protocol code. This should be very rare.
 
 45. libcurl built to support ipv6 uses getaddrinfo() to resolve host names.
   getaddrinfo() sorts the response list which effectively kills how libcurl
@@ -17,9 +15,6 @@ may have been fixed since this was written!
   initial suggested function to use for randomizing the response:
     http://curl.haxx.se/mail/lib-2007-07/0178.html
 
-44. --ftp-method nocwd does not handle URLs ending with a slash properly (it
-  should list the contents of that directory). See test case 351.
-
 43. There seems to be a problem when connecting to the Microsoft telnet server.
   http://curl.haxx.se/bug/view.cgi?id=1720605
 

docs/THANKS

@@ -21,6 +21,7 @@ Alexander Lazic
 Alexander Zhuravlev
 Alexey Simak
 Alexis Carvalho
+Allen Pulsifer
 Amol Pattekar
 Anders Gustafsson
 Andi Jahja
@@ -35,6 +36,7 @@ Andrew Biggs
 Andrew Bushnell
 Andrew Francis
 Andrew Fuller
+Andrew Wansink
 Andr<EFBFBD>s Garc<72>a
 Andy Cedilnik
 Andy Serpa
@@ -71,10 +73,12 @@ Casey O'Donnell
 Chih-Chung Chang
 Chris "Bob Bob"
 Chris Combes
+Chris Flerackers
 Chris Gaukroger
 Chris Maltby
 Christian Kurz
 Christian Robottom Reis
+Christian Vogt
 Christophe Demory
 Christophe Legry
 Christopher R. Palmer
@@ -82,6 +86,7 @@ Ciprian Badescu
 Clarence Gardner
 Clifford Wolf
 Cody Jones
+Colin Hogben
 Colin Watson
 Colm Buckley
 Cory Nelson
@@ -97,6 +102,7 @@ Dan Nelson
 Dan Torop
 Dan Zitter
 Daniel Black
+Daniel Cater
 Daniel Johnson
 Daniel Stenberg
 Daniel at touchtunes
@@ -186,6 +192,7 @@ Georg Wicherski
 Gerd v. Egidy
 Gerhard Herre
 Gerrit Bruchh<68>user
+Giancarlo Formicuccia
 Giaslas Georgios
 Gilad
 Gilbert Ramirez Jr.
@@ -195,7 +202,9 @@ Giuseppe D'Ambrosio
 Glen Nakamura
 Glen Scott
 Greg Hewgill
+Greg Morse
 Greg Onufer
+Greg Zavertnik
 Grigory Entin
 Guenole Bescon
 Guillaume Arluison
@@ -240,6 +249,7 @@ Jared Lundell
 Jari Sundell
 Jason S. Priebe
 Jay Austin
+Jayesh A Shah
 Jaz Fresh
 Jean Jacques Drouin
 Jean-Claude Chauve
@@ -256,6 +266,7 @@ Jesse Noller
 Jim Drash
 Joe Halpin
 Joel Chen
+Jofell Gallardo
 Johan Anderson
 Johan Nilsson
 John Crow
@@ -403,6 +414,7 @@ Olaf St
 Oren Tirosh
 P R Schaffner
 Patrick Bihan-Faou
+Patrick Monnerat
 Patrick Smith
 Paul Harrington
 Paul Marquis
@@ -435,6 +447,7 @@ Pierre
 Puneet Pawaia
 Quagmire
 Rafael Sagula
+Ralf S. Engelschall
 Ralph Beckmann
 Ralph Mitchell
 Ramana Mokkapati
@@ -486,6 +499,7 @@ Samuel D
 Samuel Listopad
 Sander Gates
 Saul good
+Scott Cantor
 Scott Davis
 Sebastien Willemijns
 Sergio Ballestrero

docs/curl-config.1

@@ -5,7 +5,7 @@
 .\" *                            | (__| |_| |  _ <| |___
 .\" *                             \___|\___/|_| \_\_____|
 .\" *
-.\" * Copyright (C) 1998 - 2006, Daniel Stenberg, <daniel@haxx.se>, et al.
+.\" * Copyright (C) 1998 - 2007, Daniel Stenberg, <daniel@haxx.se>, et al.
 .\" *
 .\" * This software is licensed as described in the file COPYING, which
 .\" * you should have received as part of this distribution. The terms
@@ -21,7 +21,7 @@
 .\" * $Id$
 .\" **************************************************************************
 .\"
-.TH curl-config 1 "25 Jan 2004" "Curl 7.15.4" "curl-config manual"
+.TH curl-config 1 "25 Oct 2007" "Curl 7.17.1" "curl-config manual"
 .SH NAME
 curl-config \- Get information about a libcurl installation
 .SH SYNOPSIS
@@ -62,6 +62,9 @@ the time of writing, this list may include HTTP, HTTPS, FTP, FTPS, FILE,
 TELNET, LDAP, DICT. Do not assume any particular order. The protocols will
 be listed using uppercase and are separated by newlines. There may be none,
 one or several protocols in the list. (Added in 7.13.0)
+.IP "--static-libs"
+Shows the complete set of libs and other linker options you will need in order
+to link your application with libcurl statically. (Added in 7.17.1)
 .IP "--version"
 Outputs version information about the installed libcurl.
 .IP "--vernum"

docs/curl.1

@@ -21,7 +21,7 @@
 .\" * $Id$
 .\" **************************************************************************
 .\"
-.TH curl 1 "28 Feb 2007" "Curl 7.16.2" "Curl Manual"
+.TH curl 1 "21 Sep 2007" "Curl 7.17.1" "Curl Manual"
 .SH NAME
 curl \- transfer a URL
 .SH SYNOPSIS
@@ -35,7 +35,7 @@ FILE).  The command is designed to work without user interaction.
 
 curl offers a busload of useful tricks like proxy support, user
 authentication, ftp upload, HTTP post, SSL connections, cookies, file transfer
-resume and more. As you will see below, the amount of features will make your
+resume and more. As you will see below, the number of features will make your
 head spin!
 
 curl is powered by libcurl for all transfer-related features. See
@@ -106,8 +106,8 @@ file instead of overwriting it. If the file doesn't exist, it will be created.
 If this option is used twice, the second one will disable append mode again.
 .IP "-A/--user-agent <agent string>"
 (HTTP) Specify the User-Agent string to send to the HTTP server. Some badly
-done CGIs fail if its not set to "Mozilla/4.0".  To encode blanks in the
-string, surround the string with single quote marks.  This can also be set
+done CGIs fail if this field isn't set to "Mozilla/4.0". To encode blanks in
+the string, surround the string with single quote marks. This can also be set
 with the \fI-H/--header\fP option of course.
 
 If this option is set more than once, the last one will be the one that's
@@ -330,7 +330,9 @@ them independently.
 
 If curl is built against the NSS SSL library then this option tells
 curl the nickname of the certificate to use within the NSS database defined
-by --cacert.
+by the environment variable SSL_DIR (or by default /etc/pki/nssdb). If the
+NSS PEM PKCS#11 module (libnsspem.so) is available then PEM files may be
+loaded.
 
 If this option is used several times, the last one will be used.
 .IP "--cert-type <type>"
@@ -352,7 +354,10 @@ The windows version of curl will automatically look for a CA certs file named
 Current Working Directory, or in any folder along your PATH.
 
 If curl is built against the NSS SSL library then this option tells
-curl the directory that the NSS certificate database resides in.
+curl the nickname of the CA certificate to use within the NSS database
+defined by the environment variable SSL_DIR (or by default /etc/pki/nssdb).
+If the NSS PEM PKCS#11 module (libnsspem.so) is available then PEM files
+may be loaded.
 
 If this option is used several times, the last one will be used.
 .IP "--capath <CA certificate directory>"
@@ -539,6 +544,11 @@ for you.
 See also the \fI-A/--user-agent\fP and \fI-e/--referer\fP options.
 
 This option can be used multiple times to add/replace/remove multiple headers.
+.IP "--hostpubmd5"
+Pass a string containing 32 hexadecimal digits. The string should be the 128
+bit MD5 cheksum of the remote host's public key, curl will refuse the
+connection with the host unless the md5sums match. This option is only for SCP
+and SFTP transfers. (Added in 7.17.1)
 .IP "--ignore-content-length"
 (HTTP)
 Ignore the Content-Length header. This is particularly useful for servers
@@ -769,6 +779,9 @@ meant as a support for Kerberos5 authentication but may be also used along
 with another authentication methods. For more information see IETF draft
 draft-brezak-spnego-http-04.txt.
 
+If you want to enable Negotiate for your proxy authentication, then use
+\fI--proxy-negotiate\fP.
+
 This option requires that the library was built with GSSAPI support. This is
 not very common. Use \fI-V/--version\fP to see if your version supports
 GSS-Negotiate.
@@ -839,6 +852,13 @@ You may use this option as many times as you have number of URLs.
 (SSL/SSH) Pass phrase for the private key
 
 If this option is used several times, the last one will be used.
+.IP "--post301"
+Tells curl to respect RFC 2616/10.3.2 and not convert POST requests into GET
+requests when following a 301 redirection. The non-RFC behaviour is ubiquitous
+in web browsers, so curl does the conversion by default to maintain
+consistency. However, a server may requires a POST to remain a POST after such
+a redirection. This option is meaningful only when using \fI-L/--location\fP
+(Added in 7.17.1)
 .IP "--proxy-anyauth"
 Tells curl to pick a suitable authentication method when communicating with
 the given proxy. This will cause an extra request/response round-trip. (Added
@@ -858,6 +878,13 @@ Tells curl to use HTTP Digest authentication when communicating with the given
 proxy. Use \fI--digest\fP for enabling HTTP Digest with a remote host.
 
 If this option is used twice, the second will again disable proxy HTTP Digest.
+.IP "--proxy-negotiate"
+Tells curl to use HTTP Negotiate authentication when communicating
+with the given proxy. Use \fI--negotiate\fP for enabling HTTP Negotiate
+with a remote host.
+
+If this option is used twice, the second will again disable proxy HTTP
+Negotiate. (Added in 7.17.1)
 .IP "--proxy-ntlm"
 Tells curl to use HTTP NTLM authentication when communicating with the given
 proxy. Use \fI--ntlm\fP for enabling NTLM with a remote host.
@@ -1381,8 +1408,6 @@ Unsupported protocol. This build of curl has no support for this protocol.
 Failed to initialize.
 .IP 3
 URL malformat. The syntax was not correct.
-.IP 4
-URL user malformatted. The user-part of the URL syntax was not correct.
 .IP 5
 Couldn't resolve proxy. The given proxy host could not be resolved.
 .IP 6
@@ -1395,21 +1420,14 @@ FTP weird server reply. The server sent data curl couldn't parse.
 FTP access denied. The server denied login or denied access to the particular
 resource or directory you wanted to reach. Most often you tried to change to a
 directory that doesn't exist on the server.
-.IP 10
-FTP user/password incorrect. Either one or both were not accepted by the
-server.
 .IP 11
 FTP weird PASS reply. Curl couldn't parse the reply sent to the PASS request.
-.IP 12
-FTP weird USER reply. Curl couldn't parse the reply sent to the USER request.
 .IP 13
 FTP weird PASV reply, Curl couldn't parse the reply sent to the PASV request.
 .IP 14
 FTP weird 227 format. Curl couldn't parse the 227-line the server sent.
 .IP 15
 FTP can't get host. Couldn't resolve the host IP we got in the 227-line.
-.IP 16
-FTP can't reconnect. Couldn't connect to the host we got in the 227-line.
 .IP 17
 FTP couldn't set binary. Couldn't change transfer method to binary.
 .IP 18
@@ -1417,8 +1435,6 @@ Partial file. Only a part of the file was transferred.
 .IP 19
 FTP couldn't download/access the given file, the RETR (or similar) command
 failed.
-.IP 20
-FTP write error. The transfer was reported bad by the server.
 .IP 21
 FTP quote error. A quote command returned error from the server.
 .IP 22
@@ -1427,8 +1443,6 @@ error with the HTTP error code being 400 or above. This return code only
 appears if \fI-f/--fail\fP is used.
 .IP 23
 Write error. Curl couldn't write data to a local filesystem or similar.
-.IP 24
-Malformed user. User name badly specified.
 .IP 25
 FTP couldn't STOR file. The server denied the STOR operation, used for FTP
 uploading.
@@ -1439,17 +1453,12 @@ Out of memory. A memory allocation request failed.
 .IP 28
 Operation timeout. The specified time-out period was reached according to the
 conditions.
-.IP 29
-FTP couldn't set ASCII. The server returned an unknown reply.
 .IP 30
 FTP PORT failed. The PORT command failed. Not all FTP servers support the PORT
 command, try doing a transfer using PASV instead!
 .IP 31
 FTP couldn't use REST. The REST command failed. This command is used for
 resumed FTP transfers.
-.IP 32
-FTP couldn't use SIZE. The SIZE command failed. The command is an extension
-to the original FTP spec RFC 959.
 .IP 33
 HTTP range error. The range "command" didn't work.
 .IP 34
@@ -1464,20 +1473,14 @@ FILE couldn't read file. Failed to open the file. Permissions?
 LDAP cannot bind. LDAP bind operation failed.
 .IP 39
 LDAP search failed.
-.IP 40
-Library not found. The LDAP library was not found.
 .IP 41
 Function not found. A required LDAP function was not found.
 .IP 42
 Aborted by callback. An application told curl to abort the operation.
 .IP 43
 Internal error. A function was called with a bad parameter.
-.IP 44
-Internal error. A function was called in a bad order.
 .IP 45
 Interface error. A specified outgoing interface could not be used.
-.IP 46
-Bad password entered. An error was signalled when the password was entered.
 .IP 47
 Too many redirects. When following redirects, curl hit the maximum amount.
 .IP 48
@@ -1485,7 +1488,7 @@ Unknown TELNET option specified.
 .IP 49
 Malformed telnet option.
 .IP 51
-The remote peer's SSL certificate wasn't ok
+The peer's SSL certificate or SSH MD5 fingerprint was not ok
 .IP 52
 The server didn't reply anything, which here is considered an error.
 .IP 53
@@ -1496,14 +1499,12 @@ Cannot set SSL crypto engine as default
 Failed sending network data
 .IP 56
 Failure in receiving network data
-.IP 57
-Share is in use (internal error)
 .IP 58
 Problem with the local certificate
 .IP 59
 Couldn't use specified SSL cipher
 .IP 60
-Problem with the CA cert (path? permission?)
+Peer certificate cannot be authenticated with known CA certificates
 .IP 61
 Unrecognized transfer encoding
 .IP 62
@@ -1536,6 +1537,14 @@ No such user (TFTP)
 Character conversion failed
 .IP 76
 Character conversion functions required
+.IP 77
+Problem with reading the SSL CA cert (path? access rights?)
+.IP 78
+The resource referenced in the URL does not exist
+.IP 79
+An unspecified error occurred during the SSH session
+.IP 80
+Failed to shut down the SSL connection
 .IP XX
 There will appear more error codes here in future releases. The existing ones
 are meant to never change.

docs/examples/Makefile.am

@@ -9,6 +9,7 @@ EXTRA_DIST = README Makefile.example makefile.dj $(COMPLICATED_EXAMPLES)
 INCLUDES = -I$(top_srcdir)/include
 
 LIBDIR = $(top_builddir)/lib
+CPPFLAGS = -DCURL_NO_OLDIES
 
 # Dependencies
 LDADD = $(LIBDIR)/libcurl.la

docs/examples/README

@@ -40,7 +40,6 @@ curlx.c        - getting file info from the remote cert data
 debug.c        - showing how to use the debug callback
 fileupload.c   - uploading to a file:// URL
 fopen.c        - fopen() layer that supports opening URLs and files
-ftp3rdparty.c  - FTP 3rd party transfer
 ftpget.c       - simple getting a file from FTP
 ftpgetresp.c   - get the response strings from the FTP server
 ftpupload.c    - upload a file to an FTP server
@@ -50,6 +49,7 @@ getinmemory.c  - download a file to memory only
 ghiper.c       - curl_multi_socket() using code with glib-2
 hiperfifo.c    - downloads all URLs written to the fifo, using
                  curl_multi_socket() and libevent
+htmltidy.c     - download a document and use libtidy to parse the HTML
 htmltitle.cc   - download a HTML file and extract the <title> tag from a HTML
                  page using libxml
 http-post.c    - HTTP POST
@@ -60,9 +60,9 @@ multi-debugcallback.c - a multi-interface app using the debug callback
 multi-double.c - a multi-interface app doing two simultaneous transfers
 multi-post.c   - a multi-interface app doing a multipart formpost
 multi-single.c - a multi-interface app getting a single file
-multithread.c  - an example using multi-treading transfering multiple files
+multithread.c  - an example using multi-treading transferring multiple files
 opensslthreadlock.c - show how to do locking when using OpenSSL multi-threaded
-persistant.c   - request two URLs with a persistant connection
+persistant.c   - request two URLs with a persistent connection
 post-callback.c - send a HTTP POST using a callback
 postit2.c      - send a HTTP multipart formpost
 sampleconv.c   - showing how a program on a non-ASCII platform would invoke
@@ -72,5 +72,5 @@ sepheaders.c   - download headers to a separate file
 simple.c       - the most simple download a URL source
 simplepost.c   - HTTP POST
 simplessl.c    - HTTPS example with certificates many options set
-synctime.c     - Sync local time by extracing date from remote HTTP servers
+synctime.c     - Sync local time by extracting date from remote HTTP servers
 10-at-a-time.c - Download many files simultaneously, 10 at a time.

docs/examples/makefile.dj

@@ -23,7 +23,7 @@ CSOURCES = fopen.c ftpget.c ftpgetresp.c ftpupload.c getinmemory.c  \
            multi-post.c multi-single.c persistant.c post-callback.c \
            postit2.c sepheaders.c simple.c simplepost.c simplessl.c \
            multi-debugcallback.c fileupload.c getinfo.c anyauthput.c \
-           10-at-a-time.c # ftpuploadresume.c ftp3rdparty.c cookie_interface.c
+           10-at-a-time.c # ftpuploadresume.c cookie_interface.c
 
 PROGRAMS = $(CSOURCES:.c=.exe)
 

docs/libcurl/ABI

@@ -0,0 +1,54 @@
+                                  _   _ ____  _
+                              ___| | | |  _ \| |
+                             / __| | | | |_) | |
+                            | (__| |_| |  _ <| |___
+                             \___|\___/|_| \_\_____|
+
+                          libcurl's binary interface
+
+ABI - Application Binary Interface
+
+  First, allow me to define the word for this context: ABI describes the
+  low-level interface between an application program a library. Calling
+  conventions, function arguments, return values, struct sizes/defines and
+  more.
+
+  For a longer descricption, see
+  http://en.wikipedia.org/wiki/Application_binary_interface
+
+Upgrades
+
+  In the vast majority of all cases, a typical libcurl upgrade does not break
+  the ABI at all. Your application can remain using libcurl just as before,
+  only with less bugs and possibly with added new features. You need to read
+  the release notes, and if they mention an ABI break/soname bump, you may
+  have to verify that your application still builds fine and uses libcurl as
+  it now is defined to work.
+
+Version Numbers
+
+  In libcurl land, you really can't tell by the libcurl version number if that
+  libcurl is binary compatible or not with another libcurl version.
+
+Soname Bumps
+
+  Whenever there are changes done to the library that will cause an ABI
+  breakage, that may require your application to get attention or possibly be
+  changed to adhere to new things, we will bump the soname. Then the library
+  will get a different output name and thus can in fact be installed in
+  parallell with an older installed lib (on most systems). Thus, old
+  applications built against the previous ABI version will remain working and
+  using the older lib, while newer applications build and use the newer one.
+
+  During the first seven years of libcurl releases, there have only been four
+  ABI breakages.
+
+Downgrades
+
+  Going to an older libcurl version from one you're currently using can be a
+  tricky thing. Mostly we add features and options to newer libcurls as that
+  won't break ABI or hamper existing applications. This has the implication
+  that going backwards may get you in a situation where you pick a libcurl
+  that doesn't support the options your application needs. Or possibly you
+  even downgrade so far so you cross an ABI break border and thus a different
+  soname, and then your application may need to adapt to the modified ABI.

docs/libcurl/Makefile.am

@@ -57,7 +57,7 @@ PDFPAGES = curl_easy_cleanup.pdf curl_easy_getinfo.pdf curl_easy_init.pdf \
 
 CLEANFILES = $(HTMLPAGES) $(PDFPAGES)
 
-EXTRA_DIST = $(man_MANS) $(HTMLPAGES) index.html $(PDFPAGES) libcurl.m4
+EXTRA_DIST = $(man_MANS) $(HTMLPAGES) index.html $(PDFPAGES) libcurl.m4 ABI
 
 MAN2HTML= roffit --mandir=. < $< >$@
 

docs/libcurl/curl_easy_setopt.3

@@ -184,6 +184,30 @@ unrecoverable error to the library and it will close the socket and return
 Pass a pointer that will be untouched by libcurl and passed as the first
 argument in the sockopt callback set with \fICURLOPT_SOCKOPTFUNCTION\fP.
 (Option added in 7.15.6.)
+.IP CURLOPT_OPENSOCKETFUNCTION
+Function pointer that should match the \fIcurl_opensocket_callback\fP
+prototype found in \fI<curl/curl.h>\fP. This function gets called by libcurl
+instead of the \fIsocket(2)\fP call. The callback's \fIpurpose\fP argument
+identifies the exact purpose for this particular socket, and currently only
+one value is supported: \fICURLSOCKTYPE_IPCXN\fP for the primary connection
+(meaning the control connection in the FTP case). Future versions of libcurl
+may support more purposes. It passes the resolved peer address as a
+\fIaddress\fP argument so the callback can modify the address or refuse to
+connect at all. The callback function should return the socket or
+\fICURL_SOCKET_BAD\fP in case no connection should be established or any error
+detected. Any additional \fIsetsockopt(2)\fP calls can be done on the socket
+at the user's discretion.  \fICURL_SOCKET_BAD\fP return value from the
+callback function will signal an unrecoverable error to the library and it
+will return \fICURLE_COULDNT_CONNECT\fP.  This return code can be used for IP
+address blacklisting.  The default behavior is:
+.Bd -literal -offset indent
+   return socket(addr->family, addr->socktype, addr->protocol);
+.Ed
+(Option added in 7.17.1.)
+.IP CURLOPT_OPENSOCKETDATA
+Pass a pointer that will be untouched by libcurl and passed as the first
+argument in the opensocket callback set with \fICURLOPT_OPENSOCKETFUNCTION\fP.
+(Option added in 7.17.1.)
 .IP CURLOPT_PROGRESSFUNCTION
 Function pointer that should match the \fIcurl_progress_callback\fP prototype
 found in \fI<curl/curl.h>\fP. This function gets called by libcurl instead of
@@ -362,8 +386,7 @@ POST/PUT and a 401 or 407 is received immediately afterwards.
 .SH NETWORK OPTIONS
 .IP CURLOPT_URL
 The actual URL to deal with. The parameter should be a char * to a zero
-terminated string. The string must remain present until curl no longer needs
-it, as it doesn't copy the string.
+terminated string.
 
 If the given URL lacks the protocol part ("http://" or "ftp://" etc), it will
 attempt to guess which protocol to use based on the given host name. If the
@@ -619,6 +642,13 @@ redirections have been followed, the next redirect will cause an error
 \fICURLOPT_FOLLOWLOCATION\fP is used at the same time. Added in 7.15.1:
 Setting the limit to 0 will make libcurl refuse any redirect. Set it to -1 for
 an infinite number of redirects (which is the default)
+.IP CURLOPT_POST301
+A non-zero parameter tells the library to respect RFC 2616/10.3.2 and not
+convert POST requests into GET requests when following a 301 redirection. The
+non-RFC behaviour is ubiquitous in web browsers, so the library does the
+conversion by default to maintain consistency. However, a server may requires
+a POST to remain a POST after such a redirection. This option is meaningful
+only when setting \fICURLOPT_FOLLOWLOCATION\fP.  (Added in 7.17.1)
 .IP CURLOPT_PUT
 A non-zero parameter tells the library to use HTTP PUT to transfer data. The
 data should be set with \fICURLOPT_READDATA\fP and \fICURLOPT_INFILESIZE\fP.
@@ -631,14 +661,16 @@ also make the library use the a "Content-Type:
 application/x-www-form-urlencoded" header. (This is by far the most commonly
 used POST method).
 
-Use the \fICURLOPT_POSTFIELDS\fP option to specify what data to post and
-\fICURLOPT_POSTFIELDSIZE\fP to set the data size.
+Use one of \fICURLOPT_POSTFIELDS\fP or \fICURLOPT_COPYPOSTFIELDS\fP options to
+specify what data to post and \fICURLOPT_POSTFIELDSIZE\fP or
+\fICURLOPT_POSTFIELDSIZE_LARGE\fP to set the data size.
 
 Optionally, you can provide data to POST using the \fICURLOPT_READFUNCTION\fP
 and \fICURLOPT_READDATA\fP options but then you must make sure to not set
 \fICURLOPT_POSTFIELDS\fP to anything but NULL. When providing data with a
 callback, you must transmit it using chunked transfer-encoding or you must set
-the size of the data with the \fICURLOPT_POSTFIELDSIZE\fP option.
+the size of the data with the \fICURLOPT_POSTFIELDSIZE\fP or
+\fICURLOPT_POSTFIELDSIZE_LARGE\fP option.
 
 You can override the default POST Content-Type: header by setting your own
 with \fICURLOPT_HTTPHEADER\fP.
@@ -659,11 +691,14 @@ If you issue a POST request and then want to make a HEAD or GET using the same
 re-used handle, you must explicitly set the new request type using
 \fICURLOPT_NOBODY\fP or \fICURLOPT_HTTPGET\fP or similar.
 .IP CURLOPT_POSTFIELDS
-Pass a char * as parameter, which should be the full data to post in an HTTP
+Pass a void * as parameter, which should be the full data to post in an HTTP
 POST operation. You must make sure that the data is formatted the way you want
 the server to receive it. libcurl will not convert or encode it for you. Most
 web servers will assume this data to be url-encoded. Take note.
 
+The pointed data are NOT copied by the library: as a consequence, they must
+be preserved by the calling application until the transfer finishes.
+
 This POST is a normal application/x-www-form-urlencoded kind (and libcurl will
 set that Content-Type by default when this option is used), which is the most
 commonly used one by HTML forms. See also the \fICURLOPT_POST\fP. Using
@@ -690,6 +725,22 @@ Pass a curl_off_t as parameter. Use this to set the size of the
 \fICURLOPT_POSTFIELDS\fP data to prevent libcurl from doing strlen() on the
 data to figure out the size. This is the large file version of the
 \fICURLOPT_POSTFIELDSIZE\fP option. (Added in 7.11.1)
+.IP CURLOPT_COPYPOSTFIELDS
+Pass a char * as parameter, which should be the full data to post in an HTTP
+POST operation. It behaves as the \fICURLOPT_POSTFIELDS\fP option, but the
+original data are copied by the library, allowing the application to overwrite
+the original data after setting this option.
+
+Because data are copied, care must be taken when using this option in
+conjunction with \fICURLOPT_POSTFIELDSIZE\fP or
+\fICURLOPT_POSTFIELDSIZE_LARGE\fP: If the size has not been set prior to
+\fICURLOPT_COPYPOSTFIELDS\fP, the data are assumed to be a NUL-terminated
+string; else the stored size informs the library about the data byte count to
+copy. In any case, the size must not be changed after
+\fICURLOPT_COPYPOSTFIELDS\fP, unless another \fICURLOPT_POSTFIELDS\fP or
+\fICURLOPT_COPYPOSTFIELDS\fP option is issued.
+(Added in 7.17.1)
+
 .IP CURLOPT_HTTPPOST
 Tells libcurl you want a multipart/formdata HTTP POST to be made and you
 instruct what data to pass on to the server.  Pass a pointer to a linked list
@@ -803,7 +854,9 @@ format or just regular HTTP-style header (Set-Cookie: ...) format. If cURL
 cookie engine was not enabled it will enable its cookie engine.  Passing a
 magic string \&"ALL" will erase all cookies known by cURL. (Added in 7.14.1)
 Passing the special string \&"SESS" will only erase all session cookies known
-by cURL. (Added in 7.15.4)
+by cURL. (Added in 7.15.4) Passing the special string \&"FLUSH" will write
+all cookies known by cURL to the file specified by \fICURLOPT_COOKIEJAR\fP.
+(Added in 7.17.1)
 .IP CURLOPT_HTTPGET
 Pass a long. If the long is non-zero, this forces the HTTP request to get back
 to GET. usable if a POST, HEAD, PUT or a custom request have been used
@@ -1030,12 +1083,19 @@ option to -1 to make the transfer start from the end of the target file
 Pass a curl_off_t as parameter. It contains the offset in number of bytes that
 you want the transfer to start from. (Added in 7.11.0)
 .IP CURLOPT_CUSTOMREQUEST
-Pass a pointer to a zero terminated string as parameter. It will be user
+Pass a pointer to a zero terminated string as parameter. It will be used
 instead of GET or HEAD when doing an HTTP request, or instead of LIST or NLST
 when doing an ftp directory listing. This is useful for doing DELETE or other
 more or less obscure HTTP requests. Don't do this at will, make sure your
 server supports the command first.
 
+Note that libcurl will still act and assume the keyword it would use if you
+didn't set your custom one is the one in use and it will act according to
+that. Thus, changing this to a HEAD when libcurl otherwise would do a GET
+might cause libcurl to act funny, and similar. To switch to a proper HEAD, use
+\fICURLOPT_NOBODY\fP, to switch to a proper POST, use \fICURLOPT_POST\fP or
+\fICURLOPT_POSTFIELDS\fP and so on.
+
 Restore to the internal default by setting this to NULL.
 
 Many people have wrongly used this option to replace the entire request with
@@ -1404,6 +1464,11 @@ Pass a long set to a bitmask consisting of one or more of
 CURLSSH_AUTH_PUBLICKEY, CURLSSH_AUTH_PASSWORD, CURLSSH_AUTH_HOST,
 CURLSSH_AUTH_KEYBOARD. Set CURLSSH_AUTH_ANY to let libcurl pick one.
 (Added in 7.16.1)
+.IP CURLOPT_SSH_HOST_PUBLIC_KEY_MD5
+Pass a char * pointing to a string containing 32 hexadecimal digits. The
+string should be the 128 bit MD5 cheksum of the remote host's public key, and
+libcurl will reject the connection to the host unless the md5sums match. This
+option is only for SCP and SFTP transfers. (Added in 7.17.1)
 .IP CURLOPT_SSH_PUBLIC_KEYFILE
 Pass a char * pointing to a file name for your public key. If not used,
 libcurl defaults to using \fB~/.ssh/id_dsa.pub\fP.

docs/libcurl/curl_formadd.3

@@ -44,7 +44,7 @@ parts.
 .IP CURLFORM_COPYNAME
 followed by a string which provides the \fIname\fP of this part. libcurl
 copies the string so your application doesn't need to keep it around after
-this function call. If the name isn't null terminated, or if you'd
+this function call. If the name isn't NUL-terminated, or if you'd
 like it to contain zero bytes, you must set its length with
 \fBCURLFORM_NAMELENGTH\fP. The copied data will be freed by
 \fIcurl_formfree(3)\fP.
@@ -53,7 +53,7 @@ like it to contain zero bytes, you must set its length with
 followed by a string which provides the \fIname\fP of this part. libcurl
 will use the pointer and refer to the data in your application, so you
 must make sure it remains until curl no longer needs it. If the name
-isn't null terminated, or if you'd like it to contain zero
+isn't NUL-terminated, or if you'd like it to contain zero
 bytes, you must set its length with \fBCURLFORM_NAMELENGTH\fP.
 
 .IP CURLFORM_COPYCONTENTS
@@ -68,7 +68,7 @@ data will be freed by \fIcurl_formfree(3)\fP.
 followed by a pointer to the contents of this part, the actual data
 to send away. libcurl will use the pointer and refer to the data in your
 application, so you must make sure it remains until curl no longer needs it.
-If the data isn't null terminated, or if you'd like it to contain zero bytes,
+If the data isn't NUL-terminated, or if you'd like it to contain zero bytes,
 you must set its length  with \fBCURLFORM_CONTENTSLENGTH\fP.
 
 .IP CURLFORM_CONTENTSLENGTH

docs/libcurl/libcurl-errors.3

@@ -148,8 +148,8 @@ An option set with CURLOPT_TELNETOPTIONS was not recognized/known. Refer to
 the appropriate documentation.
 .IP "CURLE_TELNET_OPTION_SYNTAX (49)"
 A telnet option string was Illegally formatted.
-.IP "CURLE_SSL_PEER_CERTIFICATE (51)"
-The remote server's SSL certificate was deemed not OK.
+.IP "CURLE_PEER_FAILED_VERIFICATION (51)"
+The remote server's SSL certificate or SSH md5 fingerprint was deemed not OK.
 .IP "CURLE_GOT_NOTHING (52)"
 Nothing was returned from the server, and under the circumstances, getting
 nothing is considered an error.

include/curl/curl.h

@@ -151,6 +151,7 @@ extern "C" {
 /* The check above prevents the winsock2 inclusion if winsock.h already was
    included, since they can't co-exist without problems */
 #include <winsock2.h>
+#include <ws2tcpip.h>
 #endif
 #else
 
@@ -246,6 +247,19 @@ typedef int (*curl_sockopt_callback)(void *clientp,
                                      curl_socket_t curlfd,
                                      curlsocktype purpose);
 
+struct curl_sockaddr {
+  int family;
+  int socktype;
+  int protocol;
+  socklen_t addrlen;
+  struct sockaddr addr;
+};
+
+typedef curl_socket_t
+(*curl_opensocket_callback)(void *clientp,
+                            curlsocktype purpose,
+                            struct curl_sockaddr *address);
+
 #ifndef CURL_NO_OLDIES
   /* not used since 7.10.8, will be removed in a future release */
 typedef int (*curl_passwd_callback)(void *clientp,
@@ -367,7 +381,8 @@ typedef enum {
   CURLE_UNKNOWN_TELNET_OPTION,   /* 48 - User specified an unknown option */
   CURLE_TELNET_OPTION_SYNTAX ,   /* 49 - Malformed telnet option */
   CURLE_OBSOLETE50,              /* 50 - NOT USED */
-  CURLE_SSL_PEER_CERTIFICATE,    /* 51 - peer's certificate wasn't ok */
+  CURLE_PEER_FAILED_VERIFICATION, /* 51 - peer's certificate or fingerprint
+                                     wasn't verified fine */
   CURLE_GOT_NOTHING,             /* 52 - when this is a specific error */
   CURLE_SSL_ENGINE_NOTFOUND,     /* 53 - SSL crypto engine not found */
   CURLE_SSL_ENGINE_SETFAILED,    /* 54 - can not set SSL crypto engine as
@@ -416,10 +431,14 @@ typedef enum {
                           the obsolete stuff removed! */
 
 /* Backwards compatibility with older names */
+
+/* The following were added in 7.17.1 */
 /* These are scheduled to disappear by 2009 */
+#define CURLE_SSL_PEER_CERTIFICATE CURLE_PEER_FAILED_VERIFICATION
 
 /* The following were added in 7.17.0 */
-#define CURLE_OBSOLETE CURLE_OBSOLETE50	/* noone should be using this! */
+/* These are scheduled to disappear by 2009 */
+#define CURLE_OBSOLETE CURLE_OBSOLETE50 /* noone should be using this! */
 #define CURLE_BAD_PASSWORD_ENTERED CURLE_OBSOLETE46
 #define CURLE_BAD_CALLING_ORDER CURLE_OBSOLETE44
 #define CURLE_FTP_USER_PASSWORD_INCORRECT CURLE_OBSOLETE10
@@ -640,7 +659,7 @@ typedef enum {
    */
   CINIT(INFILESIZE, LONG, 14),
 
-  /* POST input fields. */
+  /* POST static input fields. */
   CINIT(POSTFIELDS, OBJECTPOINT, 15),
 
   /* Set the referer page (needed by some CGIs) */
@@ -1124,6 +1143,22 @@ typedef enum {
   CINIT(NEW_FILE_PERMS, LONG, 159),
   CINIT(NEW_DIRECTORY_PERMS, LONG, 160),
 
+  /* Obey RFC 2616/10.3.2 and keep POSTs as POSTs after a 301 */
+  CINIT(POST301, LONG, 161),
+
+  /* used by scp/sftp to verify the host's public key */
+  CINIT(SSH_HOST_PUBLIC_KEY_MD5, OBJECTPOINT, 162),
+
+  /* Callback function for opening socket (instead of socket(2)). Optionally,
+     callback is able change the address or refuse to connect returning
+     CURL_SOCKET_BAD.  The callback should have type
+     curl_opensocket_callback */
+  CINIT(OPENSOCKETFUNCTION, FUNCTIONPOINT, 163),
+  CINIT(OPENSOCKETDATA, OBJECTPOINT, 164),
+
+  /* POST volatile input fields. */
+  CINIT(COPYPOSTFIELDS, OBJECTPOINT, 165),
+
   CURLOPT_LASTENTRY /* the last unused */
 } CURLoption;
 

include/curl/curlver.h

@@ -28,13 +28,13 @@
 
 /* This is the version number of the libcurl package from which this header
    file origins: */
-#define LIBCURL_VERSION "7.17.0-CVS"
+#define LIBCURL_VERSION "7.17.1-CVS"
 
 /* The numeric version number is also available "in parts" by using these
    defines: */
 #define LIBCURL_VERSION_MAJOR 7
 #define LIBCURL_VERSION_MINOR 17
-#define LIBCURL_VERSION_PATCH 0
+#define LIBCURL_VERSION_PATCH 1
 
 /* This is the numeric version of the libcurl version number, meant for easier
    parsing and comparions by programs. The LIBCURL_VERSION_NUM define will
@@ -51,7 +51,7 @@
    and it is always a greater number in a more recent release. It makes
    comparisons with greater than and less than work.
 */
-#define LIBCURL_VERSION_NUM 0x071100
+#define LIBCURL_VERSION_NUM 0x071101
 
 /*
  * This is the date and time when the full source package was created. The

lib/Makefile.Watcom

@@ -4,7 +4,7 @@
 #
 # $Id$
 
-TARGETS = libcurl_wc.lib libcurl_wc.dll libcurl_wc_imp.lib
+TARGETS = ca-bundle.h libcurl_wc.lib libcurl_wc.dll libcurl_wc_imp.lib
 
 CC = wcc386
 
@@ -12,41 +12,41 @@ CFLAGS = -3r -mf -d3 -hc -zff -zgf -zq -zm -zc -s -fr=con -w2 -fpi -oilrtfm -bt=
          -d+ -dWIN32 -dCURL_CA_BUNDLE=getenv("CURL_CA_BUNDLE")                     &
          -dBUILDING_LIBCURL -dWITHOUT_MM_LIB -dHAVE_SPNEGO=1 -dENABLE_IPV6         &
          -dDEBUG_THREADING_GETADDRINFO -dDEBUG=1 -dCURLDEBUG -d_WIN32_WINNT=0x0501 &
-         -I. -I..\include
+         -I. -I..\include -dCURL_DISABLE_LDAP
 
 OBJ_DIR  = Watcom_obj
 LIB_ARG  = $(OBJ_DIR)\wlib.arg
 LINK_ARG = $(OBJ_DIR)\wlink.arg
 
-OBJS = $(OBJ_DIR)\transfer.obj         $(OBJ_DIR)\file.obj       &
-       $(OBJ_DIR)\strequal.obj         $(OBJ_DIR)\timeval.obj    &
-       $(OBJ_DIR)\easy.obj             $(OBJ_DIR)\base64.obj     &
-       $(OBJ_DIR)\security.obj         $(OBJ_DIR)\hostip.obj     &
-       $(OBJ_DIR)\krb4.obj             $(OBJ_DIR)\progress.obj   &
-       $(OBJ_DIR)\memdebug.obj         $(OBJ_DIR)\formdata.obj   &
-       $(OBJ_DIR)\http_chunks.obj      $(OBJ_DIR)\cookie.obj     &
-       $(OBJ_DIR)\strtok.obj           $(OBJ_DIR)\http.obj       &
-       $(OBJ_DIR)\connect.obj          $(OBJ_DIR)\sendf.obj      &
-       $(OBJ_DIR)\llist.obj            $(OBJ_DIR)\ftp.obj        &
-       $(OBJ_DIR)\hash.obj             $(OBJ_DIR)\url.obj        &
-       $(OBJ_DIR)\multi.obj            $(OBJ_DIR)\dict.obj       &
-       $(OBJ_DIR)\content_encoding.obj $(OBJ_DIR)\if2ip.obj      &
-       $(OBJ_DIR)\share.obj            $(OBJ_DIR)\speedcheck.obj &
-       $(OBJ_DIR)\http_digest.obj      $(OBJ_DIR)\ldap.obj       &
-       $(OBJ_DIR)\md5.obj              $(OBJ_DIR)\ssluse.obj     &
-       $(OBJ_DIR)\http_negotiate.obj   $(OBJ_DIR)\version.obj    &
-       $(OBJ_DIR)\http_ntlm.obj        $(OBJ_DIR)\getenv.obj     &
-       $(OBJ_DIR)\inet_pton.obj        $(OBJ_DIR)\escape.obj     &
-       $(OBJ_DIR)\strtoofft.obj        $(OBJ_DIR)\mprintf.obj    &
-       $(OBJ_DIR)\strerror.obj         $(OBJ_DIR)\telnet.obj     &
-       $(OBJ_DIR)\hostares.obj         $(OBJ_DIR)\netrc.obj      &
-       $(OBJ_DIR)\hostasyn.obj         $(OBJ_DIR)\getinfo.obj    &
-       $(OBJ_DIR)\hostip4.obj          $(OBJ_DIR)\hostthre.obj   &
-       $(OBJ_DIR)\hostip6.obj          $(OBJ_DIR)\inet_ntop.obj  &
-       $(OBJ_DIR)\hostsyn.obj          $(OBJ_DIR)\parsedate.obj  &
-       $(OBJ_DIR)\select.obj           $(OBJ_DIR)\sslgen.obj     &
-       $(OBJ_DIR)\gtls.obj             $(OBJ_DIR)\tftp.obj       &
-       $(OBJ_DIR)\splay.obj	       $(OBJ_DIR)\socks.obj
+OBJS = $(OBJ_DIR)\base64.obj           $(OBJ_DIR)\connect.obj     &
+       $(OBJ_DIR)\content_encoding.obj $(OBJ_DIR)\cookie.obj      &
+       $(OBJ_DIR)\dict.obj             $(OBJ_DIR)\easy.obj        &
+       $(OBJ_DIR)\escape.obj           $(OBJ_DIR)\file.obj        &
+       $(OBJ_DIR)\formdata.obj         $(OBJ_DIR)\ftp.obj         &
+       $(OBJ_DIR)\getenv.obj           $(OBJ_DIR)\getinfo.obj     &
+       $(OBJ_DIR)\gtls.obj             $(OBJ_DIR)\hash.obj        &
+       $(OBJ_DIR)\hostares.obj         $(OBJ_DIR)\hostasyn.obj    &
+       $(OBJ_DIR)\hostip.obj           $(OBJ_DIR)\hostip4.obj     &
+       $(OBJ_DIR)\hostip6.obj          $(OBJ_DIR)\hostsyn.obj     &
+       $(OBJ_DIR)\hostthre.obj         $(OBJ_DIR)\http.obj        &
+       $(OBJ_DIR)\http_chunks.obj      $(OBJ_DIR)\http_digest.obj &
+       $(OBJ_DIR)\http_negotiate.obj   $(OBJ_DIR)\http_ntlm.obj   &
+       $(OBJ_DIR)\if2ip.obj            $(OBJ_DIR)\inet_ntop.obj   &
+       $(OBJ_DIR)\inet_pton.obj        $(OBJ_DIR)\krb4.obj        &
+       $(OBJ_DIR)\ldap.obj             $(OBJ_DIR)\llist.obj       &
+       $(OBJ_DIR)\md5.obj              $(OBJ_DIR)\memdebug.obj    &
+       $(OBJ_DIR)\mprintf.obj          $(OBJ_DIR)\multi.obj       &
+       $(OBJ_DIR)\netrc.obj            $(OBJ_DIR)\parsedate.obj   &
+       $(OBJ_DIR)\progress.obj         $(OBJ_DIR)\security.obj    &
+       $(OBJ_DIR)\select.obj           $(OBJ_DIR)\sendf.obj       &
+       $(OBJ_DIR)\share.obj            $(OBJ_DIR)\socks.obj       &
+       $(OBJ_DIR)\speedcheck.obj       $(OBJ_DIR)\splay.obj       &
+       $(OBJ_DIR)\sslgen.obj           $(OBJ_DIR)\ssluse.obj      &
+       $(OBJ_DIR)\strequal.obj         $(OBJ_DIR)\strerror.obj    &
+       $(OBJ_DIR)\strtok.obj           $(OBJ_DIR)\strtoofft.obj   &
+       $(OBJ_DIR)\telnet.obj           $(OBJ_DIR)\tftp.obj        &
+       $(OBJ_DIR)\timeval.obj          $(OBJ_DIR)\transfer.obj    &
+       $(OBJ_DIR)\url.obj              $(OBJ_DIR)\version.obj
 
 RESOURCE = $(OBJ_DIR)\libcurl.res
 
@@ -56,6 +56,9 @@ all: $(OBJ_DIR) $(TARGETS) .SYMBOLIC
 $(OBJ_DIR):
 	mkdir $(OBJ_DIR)
 
+ca-bundle.h:
+	@echo /* dummy ca-bundle.h. Not used */ > $@
+
 libcurl_wc.lib: $(OBJS) $(LIB_ARG)
 	wlib -q -b -c $@ @$(LIB_ARG)
 

lib/Makefile.am

@@ -50,7 +50,7 @@ INCLUDES = -I$(top_srcdir)/include \
            -I$(top_builddir)/lib   \
            -I$(top_srcdir)/lib
 
-VERSION=-version-info 4:0:0
+VERSION=-version-info 4:1:0
 
 # This flag accepts an argument of the form current[:revision[:age]]. So,
 # passing -version-info 3:12:1 sets current to 3, revision to 12, and age to

lib/Makefile.netware

@@ -309,7 +309,7 @@ ifndef DISABLE_LDAP
 	@echo $(DL)import @$(SDK_LDAP)/clib/imports/ldapsdk.imp$(DL) >> $@
 	@echo $(DL)import @$(SDK_LDAP)/clib/imports/ldapssl.imp$(DL) >> $@
 	@echo $(DL)import @$(SDK_LDAP)/clib/imports/ldapx.imp$(DL) >> $@
-	@echo $(DL)module ldapsdk$(DL) >> $@
+	@echo $(DL)module ldapsdk ldapssl ldapx$(DL) >> $@
 endif
 	@echo $(DL)module clib$(DL) >> $@
 else
@@ -323,7 +323,7 @@ ifndef DISABLE_LDAP
 	@echo $(DL)import @$(SDK_LDAP)/libc/imports/lldapsdk.imp$(DL) >> $@
 	@echo $(DL)import @$(SDK_LDAP)/libc/imports/lldapssl.imp$(DL) >> $@
 	@echo $(DL)import @$(SDK_LDAP)/libc/imports/lldapx.imp$(DL) >> $@
-	@echo $(DL)module lldapsdk$(DL) >> $@
+	@echo $(DL)module lldapsdk lldapssl lldapx$(DL) >> $@
 endif
 	@echo $(DL)module libc$(DL) >> $@
 endif

lib/Makefile.vc6

@@ -86,9 +86,9 @@ SSLLIBS    = libeay32.lib ssleay32.lib
 ZLIBLIBSDLL= zdll.lib
 ZLIBLIBS   = zlib.lib
 !IFDEF USEMM_LIBS
-WINLIBS    = wsock32.lib winmm.lib
+WINLIBS    = wsock32.lib wldap32.lib winmm.lib
 !ELSE
-WINLIBS    = wsock32.lib
+WINLIBS    = wsock32.lib wldap32.lib
 CFLAGS     = $(CFLAGS) /DWITHOUT_MM_LIB
 !ENDIF
 #  RSAglue.lib was formerly needed in the SSLLIBS

lib/config-os400.h

@@ -361,6 +361,9 @@
 /* The size of a `long double', as computed by sizeof. */
 #define SIZEOF_LONG_DOUBLE      8
 
+/* Define if 64 bit integers are supported. */
+#define HAVE_LONGLONG
+
 /* The size of a `long long', as computed by sizeof. */
 #define SIZEOF_LONG_LONG        8
 

lib/config-win32.h

@@ -279,6 +279,7 @@
 /* Define as the return type of signal handlers (int or void).  */
 #define RETSIGTYPE void
 
+#ifndef _SSIZE_T_DEFINED
 #if (defined(__WATCOMC__) && (__WATCOMC__ >= 1240)) || defined(__POCC__) || \
     defined(__MINGW32__)
 #elif defined(_WIN64)
@@ -286,10 +287,7 @@
 #else
 #define ssize_t int
 #endif
-
-/* Define to 'int' if socklen_t is not an available 'typedefed' type */
-#ifndef HAVE_WS2TCPIP_H
-#define socklen_t int
+#define _SSIZE_T_DEFINED
 #endif
 
 /* ---------------------------------------------------------------- */

lib/config-win32ce.h

@@ -273,11 +273,6 @@
 #define ssize_t int
 #endif
 
-/* Define to 'int' if socklen_t is not an available 'typedefed' type */
-#ifndef HAVE_WS2TCPIP_H
-#define socklen_t int
-#endif
-
 /* ---------------------------------------------------------------- */
 /*                            TYPE SIZES                            */
 /* ---------------------------------------------------------------- */

lib/connect.c

@@ -380,6 +380,7 @@ static CURLcode bindlocal(struct connectdata *conn,
         port = ntohs(((struct sockaddr_in6 *)&add)->sin6_port);
 #endif
       infof(data, "Local port: %d\n", port);
+      conn->bits.bound = TRUE;
       return CURLE_OK;
     }
     if(--portnum > 0) {
@@ -677,14 +678,44 @@ singleipconnect(struct connectdata *conn,
   struct SessionHandle *data = conn->data;
   curl_socket_t sockfd;
   CURLcode res;
+  /*
+   * Curl_sockaddr_storage, which is basically sockaddr_storage has a space
+   * for a largest possible struct sockaddr only. We should add some space for
+   * the other fields we are using. Hence the addr_storage size math.
+   */
+  char addr_storage[sizeof(struct curl_sockaddr)-
+                    sizeof(struct sockaddr)+
+                    sizeof(struct Curl_sockaddr_storage)];
+  struct curl_sockaddr *addr=(struct curl_sockaddr*)&addr_storage;
+  const void *iptoprint;
 
-  sockfd = socket(ai->ai_family, conn->socktype, ai->ai_protocol);
+  addr->family=ai->ai_family;
+  addr->socktype=conn->socktype;
+  addr->protocol=ai->ai_protocol;
+  addr->addrlen =
+    (ai->ai_addrlen < (socklen_t)sizeof(struct Curl_sockaddr_storage)) ?
+     ai->ai_addrlen : (socklen_t)sizeof(struct Curl_sockaddr_storage);
+  memcpy(&addr->addr, ai->ai_addr, addr->addrlen);
+
+  /* If set, use opensocket callback to get the socket */
+  if(data->set.fopensocket)
+    sockfd = data->set.fopensocket(data->set.opensocket_client,
+                                   CURLSOCKTYPE_IPCXN, addr);
+  else
+    sockfd = socket(addr->family, addr->socktype, addr->protocol);
   if (sockfd == CURL_SOCKET_BAD)
     return CURL_SOCKET_BAD;
 
   *connected = FALSE; /* default is not connected */
 
-  Curl_printable_address(ai, addr_buf, sizeof(addr_buf));
+  /* FIXME: do we have Curl_printable_address-like with struct sockaddr* as
+     argument? */
+  iptoprint = &((const struct sockaddr_in*)(&addr->addr))->sin_addr;
+#ifdef ENABLE_IPV6
+  if(addr->family==AF_INET6)
+    iptoprint= &((const struct sockaddr_in6*)(&addr->addr))->sin6_addr;
+#endif
+  Curl_inet_ntop(addr->family, iptoprint, addr_buf, sizeof(addr_buf));
   infof(data, "  Trying %s... ", addr_buf);
 
   if(data->set.tcp_nodelay)
@@ -715,7 +746,7 @@ singleipconnect(struct connectdata *conn,
 
   /* Connect TCP sockets, bind UDP */
   if(conn->socktype == SOCK_STREAM)
-    rc = connect(sockfd, ai->ai_addr, ai->ai_addrlen);
+    rc = connect(sockfd, &addr->addr, addr->addrlen);
   else
     rc = 0;
 
@@ -779,7 +810,7 @@ singleipconnect(struct connectdata *conn,
  */
 
 CURLcode Curl_connecthost(struct connectdata *conn,  /* context */
-                          const struct Curl_dns_entry *remotehost, /* use this one */
+                          const struct Curl_dns_entry *remotehost,
                           curl_socket_t *sockconn,   /* the connected socket */
                           Curl_addrinfo **addr,      /* the one we used */
                           bool *connected)           /* really connected? */

lib/curl_ldap.h

@@ -24,6 +24,11 @@
  * $Id$
  ***************************************************************************/
 #ifndef CURL_DISABLE_LDAP
-CURLcode Curl_ldap(struct connectdata *conn, bool *done);
+extern const struct Curl_handler Curl_handler_ldap;
+
+#ifdef HAVE_LDAP_SSL
+extern const struct Curl_handler Curl_handler_ldaps;
+#endif
+
 #endif
 #endif /* __CURL_LDAP_H */

lib/dict.c

@@ -82,6 +82,33 @@
 /* The last #include file should be: */
 #include "memdebug.h"
 
+
+/*
+ * Forward declarations.
+ */
+
+static CURLcode Curl_dict(struct connectdata *conn, bool *done);
+
+/*
+ * DICT protocol handler.
+ */
+
+const struct Curl_handler Curl_handler_dict = {
+  "DICT",                               /* scheme */
+  ZERO_NULL,                            /* setup_connection */
+  Curl_dict,                            /* do_it */
+  ZERO_NULL,                            /* done */
+  ZERO_NULL,                            /* do_more */
+  ZERO_NULL,                            /* connect_it */
+  ZERO_NULL,                            /* connecting */
+  ZERO_NULL,                            /* doing */
+  ZERO_NULL,                            /* proto_getsock */
+  ZERO_NULL,                            /* doing_getsock */
+  ZERO_NULL,                            /* disconnect */
+  PORT_DICT,                            /* defport */
+  PROT_DICT                             /* protocol */
+};
+
 static char *unescape_word(struct SessionHandle *data, const char *inp)
 {
   char *newp;
@@ -115,7 +142,7 @@ static char *unescape_word(struct SessionHandle *data, const char *inp)
   return dictp;
 }
 
-CURLcode Curl_dict(struct connectdata *conn, bool *done)
+static CURLcode Curl_dict(struct connectdata *conn, bool *done)
 {
   char *word;
   char *eword;

lib/dict.h

@@ -24,7 +24,6 @@
  * $Id$
  ***************************************************************************/
 #ifndef CURL_DISABLE_DICT
-CURLcode Curl_dict(struct connectdata *conn, bool *done);
-CURLcode Curl_dict_done(struct connectdata *conn);
+extern const struct Curl_handler Curl_handler_dict;
 #endif
 #endif

lib/escape.c

@@ -5,7 +5,7 @@
  *                            | (__| |_| |  _ <| |___
  *                             \___|\___/|_| \_\_____|
  *
- * Copyright (C) 1998 - 2006, Daniel Stenberg, <daniel@haxx.se>, et al.
+ * Copyright (C) 1998 - 2007, Daniel Stenberg, <daniel@haxx.se>, et al.
  *
  * This software is licensed as described in the file COPYING, which
  * you should have received as part of this distribution. The terms
@@ -59,7 +59,7 @@ char *curl_easy_escape(CURL *handle, const char *string, int inlength)
   size_t alloc = (inlength?(size_t)inlength:strlen(string))+1;
   char *ns;
   char *testing_ptr = NULL;
-  char in;
+  unsigned char in; /* we need to treat the characters unsigned */
   size_t newlen = alloc;
   int strindex=0;
   size_t length;

lib/file.c

@@ -89,6 +89,35 @@
 /* The last #include file should be: */
 #include "memdebug.h"
 
+
+/*
+ * Forward declarations.
+ */
+
+static CURLcode Curl_file(struct connectdata *, bool *done);
+static CURLcode Curl_file_done(struct connectdata *conn,
+                               CURLcode status, bool premature);
+
+/*
+ * FILE scheme handler.
+ */
+
+const struct Curl_handler Curl_handler_file = {
+  "FILE",                               /* scheme */
+  ZERO_NULL,                            /* setup_connection */
+  Curl_file,                            /* do_it */
+  Curl_file_done,                       /* done */
+  ZERO_NULL,                            /* do_more */
+  ZERO_NULL,                            /* connect_it */
+  ZERO_NULL,                            /* connecting */
+  ZERO_NULL,                            /* doing */
+  ZERO_NULL,                            /* proto_getsock */
+  ZERO_NULL,                            /* doing_getsock */
+  ZERO_NULL,                            /* disconnect */
+  0,                                    /* defport */
+  PROT_FILE                             /* protocol */
+};
+
 /*
  * Curl_file_connect() gets called from Curl_protocol_connect() to allow us to
  * do protocol-specific actions at connect-time.  We emulate a
@@ -96,8 +125,8 @@
  */
 CURLcode Curl_file_connect(struct connectdata *conn)
 {
-  char *real_path = curl_easy_unescape(conn->data, conn->data->reqdata.path, 0,
-                                       NULL);
+  struct SessionHandle *data = conn->data;
+  char *real_path = curl_easy_unescape(data, data->reqdata.path, 0, NULL);
   struct FILEPROTO *file;
   int fd;
 #if defined(WIN32) || defined(MSDOS) || defined(__EMX__)
@@ -108,16 +137,28 @@ CURLcode Curl_file_connect(struct connectdata *conn)
   if(!real_path)
     return CURLE_OUT_OF_MEMORY;
 
-  file = (struct FILEPROTO *)calloc(sizeof(struct FILEPROTO), 1);
-  if(!file) {
-    free(real_path);
-    return CURLE_OUT_OF_MEMORY;
+  /* If there already is a protocol-specific struct allocated for this
+     sessionhandle, deal with it */
+  Curl_reset_reqproto(conn);
+
+  if (!data->reqdata.proto.file) {
+    file = (struct FILEPROTO *)calloc(sizeof(struct FILEPROTO), 1);
+    if(!file) {
+      free(real_path);
+      return CURLE_OUT_OF_MEMORY;
+    }
+    data->reqdata.proto.file = file;
+  }
+  else {
+    /* file is not a protocol that can deal with "persistancy" */
+    file = data->reqdata.proto.file;
+    Curl_safefree(file->freepath);
+    if(file->fd != -1)
+      close(file->fd);
+    file->path = NULL;
+    file->freepath = NULL;
+    file->fd = -1;
   }
-
-  if (conn->data->reqdata.proto.file)
-    free(conn->data->reqdata.proto.file);
-
-  conn->data->reqdata.proto.file = file;
 
 #if defined(WIN32) || defined(MSDOS) || defined(__EMX__)
   /* If the first character is a slash, and there's
@@ -157,8 +198,8 @@ CURLcode Curl_file_connect(struct connectdata *conn)
   file->freepath = real_path; /* free this when done */
 
   file->fd = fd;
-  if(!conn->data->set.upload && (fd == -1)) {
-    failf(conn->data, "Couldn't open file %s", conn->data->reqdata.path);
+  if(!data->set.upload && (fd == -1)) {
+    failf(data, "Couldn't open file %s", data->reqdata.path);
     Curl_file_done(conn, CURLE_FILE_COULDNT_READ_FILE, FALSE);
     return CURLE_FILE_COULDNT_READ_FILE;
   }
@@ -166,8 +207,8 @@ CURLcode Curl_file_connect(struct connectdata *conn)
   return CURLE_OK;
 }
 
-CURLcode Curl_file_done(struct connectdata *conn,
-                        CURLcode status, bool premature)
+static CURLcode Curl_file_done(struct connectdata *conn,
+                               CURLcode status, bool premature)
 {
   struct FILEPROTO *file = conn->data->reqdata.proto.file;
   (void)status; /* not used */
@@ -316,7 +357,7 @@ static CURLcode file_upload(struct connectdata *conn)
  * opposed to sockets) we instead perform the whole do-operation in this
  * function.
  */
-CURLcode Curl_file(struct connectdata *conn, bool *done)
+static CURLcode Curl_file(struct connectdata *conn, bool *done)
 {
   /* This implementation ignores the host name in conformance with
      RFC 1738. Only local files (reachable via the standard file system)
@@ -373,12 +414,12 @@ CURLcode Curl_file(struct connectdata *conn, bool *done)
 
     if(fstated) {
       const struct tm *tm;
-      time_t clock = (time_t)statbuf.st_mtime;
+      time_t filetime = (time_t)statbuf.st_mtime;
 #ifdef HAVE_GMTIME_R
       struct tm buffer;
-      tm = (const struct tm *)gmtime_r(&clock, &buffer);
+      tm = (const struct tm *)gmtime_r(&filetime, &buffer);
 #else
-      tm = gmtime(&clock);
+      tm = gmtime(&filetime);
 #endif
       /* format: "Tue, 15 Nov 1994 12:45:26 GMT" */
       snprintf(buf, BUFSIZE-1,

lib/file.h

@@ -24,8 +24,8 @@
  * $Id$
  ***************************************************************************/
 #ifndef CURL_DISABLE_FILE
-CURLcode Curl_file(struct connectdata *, bool *done);
-CURLcode Curl_file_done(struct connectdata *, CURLcode, bool premature);
+extern const struct Curl_handler Curl_handler_file;
+
 CURLcode Curl_file_connect(struct connectdata *);
 #endif
 #endif

lib/formdata.c

@@ -950,21 +950,21 @@ int curl_formget(struct curl_httppost *form, void *arg,
   for (ptr = data; ptr; ptr = ptr->next) {
     if (ptr->type == FORM_FILE) {
       char buffer[8192];
-      size_t read;
+      size_t nread;
       struct Form temp;
 
       Curl_FormInit(&temp, ptr);
 
       do {
-        read = readfromfile(&temp, buffer, sizeof(buffer));
-        if ((read == (size_t) -1) || (read != append(arg, buffer, read))) {
+        nread = readfromfile(&temp, buffer, sizeof(buffer));
+        if ((nread == (size_t) -1) || (nread != append(arg, buffer, nread))) {
           if (temp.fp) {
             fclose(temp.fp);
           }
           Curl_formclean(&data);
           return -1;
         }
-      } while (read == sizeof(buffer));
+      } while (nread == sizeof(buffer));
     } else {
       if (ptr->length != append(arg, ptr->line, ptr->length)) {
         Curl_formclean(&data);

lib/ftp.c

@@ -90,6 +90,7 @@
 #include "parsedate.h" /* for the week day and month names */
 #include "sockaddr.h" /* required for Curl_sockaddr_storage */
 #include "multiif.h"
+#include "url.h"
 
 #if defined(HAVE_INET_NTOA_R) && !defined(HAVE_INET_NTOA_R_DECL)
 #include "inet_ntoa_r.h"
@@ -130,9 +131,25 @@ static CURLcode ftp_state_post_cwd(struct connectdata *conn);
 static CURLcode ftp_state_quote(struct connectdata *conn,
                                 bool init, ftpstate instate);
 static CURLcode ftp_nb_type(struct connectdata *conn,
-                            bool ascii, ftpstate state);
+                            bool ascii, ftpstate newstate);
 static int ftp_need_type(struct connectdata *conn,
                          bool ascii);
+static CURLcode Curl_ftp(struct connectdata *conn, bool *done);
+static CURLcode Curl_ftp_done(struct connectdata *conn,
+                              CURLcode, bool premature);
+static CURLcode Curl_ftp_connect(struct connectdata *conn, bool *done);
+static CURLcode Curl_ftp_disconnect(struct connectdata *conn);
+static CURLcode Curl_ftp_nextconnect(struct connectdata *conn);
+static CURLcode Curl_ftp_multi_statemach(struct connectdata *conn, bool *done);
+static int Curl_ftp_getsock(struct connectdata *conn,
+                            curl_socket_t *socks,
+                            int numsocks);
+static CURLcode Curl_ftp_doing(struct connectdata *conn,
+                               bool *dophase_done);
+static CURLcode Curl_ftp_setup_connection(struct connectdata * conn);
+#ifdef USE_SSL
+static CURLcode Curl_ftps_setup_connection(struct connectdata * conn);
+#endif
 
 /* easy-to-use macro: */
 #define FTPSENDF(x,y,z)    if ((result = Curl_ftpsendf(x,y,z)) != CURLE_OK) \
@@ -141,6 +158,95 @@ static int ftp_need_type(struct connectdata *conn,
                               return result
 
 
+/*
+ * FTP protocol handler.
+ */
+
+const struct Curl_handler Curl_handler_ftp = {
+  "FTP",                                /* scheme */
+  Curl_ftp_setup_connection,            /* setup_connection */
+  Curl_ftp,                             /* do_it */
+  Curl_ftp_done,                        /* done */
+  Curl_ftp_nextconnect,                 /* do_more */
+  Curl_ftp_connect,                     /* connect_it */
+  Curl_ftp_multi_statemach,             /* connecting */
+  Curl_ftp_doing,                       /* doing */
+  Curl_ftp_getsock,                     /* proto_getsock */
+  Curl_ftp_getsock,                     /* doing_getsock */
+  Curl_ftp_disconnect,                  /* disconnect */
+  PORT_FTP,                             /* defport */
+  PROT_FTP                              /* protocol */
+};
+
+
+#ifdef USE_SSL
+/*
+ * FTPS protocol handler.
+ */
+
+const struct Curl_handler Curl_handler_ftps = {
+  "FTPS",                               /* scheme */
+  Curl_ftps_setup_connection,           /* setup_connection */
+  Curl_ftp,                             /* do_it */
+  Curl_ftp_done,                        /* done */
+  Curl_ftp_nextconnect,                 /* do_more */
+  Curl_ftp_connect,                     /* connect_it */
+  Curl_ftp_multi_statemach,             /* connecting */
+  Curl_ftp_doing,                       /* doing */
+  Curl_ftp_getsock,                     /* proto_getsock */
+  Curl_ftp_getsock,                     /* doing_getsock */
+  Curl_ftp_disconnect,                  /* disconnect */
+  PORT_FTPS,                            /* defport */
+  PROT_FTP | PROT_FTPS | PROT_SSL       /* protocol */
+};
+#endif
+
+#ifndef CURL_DISABLE_HTTP
+/*
+ * HTTP-proxyed FTP protocol handler.
+ */
+
+const struct Curl_handler Curl_handler_ftp_proxy = {
+  "FTP",                                /* scheme */
+  ZERO_NULL,                            /* setup_connection */
+  Curl_http,                            /* do_it */
+  Curl_http_done,                       /* done */
+  ZERO_NULL,                            /* do_more */
+  ZERO_NULL,                            /* connect_it */
+  ZERO_NULL,                            /* connecting */
+  ZERO_NULL,                            /* doing */
+  ZERO_NULL,                            /* proto_getsock */
+  ZERO_NULL,                            /* doing_getsock */
+  ZERO_NULL,                            /* disconnect */
+  PORT_FTP,                             /* defport */
+  PROT_HTTP                             /* protocol */
+};
+
+
+#ifdef USE_SSL
+/*
+ * HTTP-proxyed FTPS protocol handler.
+ */
+
+const struct Curl_handler Curl_handler_ftps_proxy = {
+  "FTPS",                               /* scheme */
+  ZERO_NULL,                            /* setup_connection */
+  Curl_http,                            /* do_it */
+  Curl_http_done,                       /* done */
+  ZERO_NULL,                            /* do_more */
+  ZERO_NULL,                            /* connect_it */
+  ZERO_NULL,                            /* connecting */
+  ZERO_NULL,                            /* doing */
+  ZERO_NULL,                            /* proto_getsock */
+  ZERO_NULL,                            /* doing_getsock */
+  ZERO_NULL,                            /* disconnect */
+  PORT_FTPS,                            /* defport */
+  PROT_HTTP                             /* protocol */
+};
+#endif
+#endif
+
+
 /*
  * NOTE: back in the old days, we added code in the FTP code that made NOBODY
  * requests on files respond with headers passed to the client/stdout that
@@ -153,11 +259,8 @@ static int ftp_need_type(struct connectdata *conn,
  */
 #define CURL_FTP_HTTPSTYLE_HEAD 1
 
-static void freedirs(struct connectdata *conn)
+static void freedirs(struct ftp_conn *ftpc)
 {
-  struct ftp_conn *ftpc = &conn->proto.ftpc;
-  struct FTP *ftp = conn->data->reqdata.proto.ftp;
-
   int i;
   if(ftpc->dirs) {
     for (i=0; i < ftpc->dirdepth; i++){
@@ -169,9 +272,9 @@ static void freedirs(struct connectdata *conn)
     free(ftpc->dirs);
     ftpc->dirs = NULL;
   }
-  if(ftp->file) {
-    free(ftp->file);
-    ftp->file = NULL;
+  if(ftpc->file) {
+    free(ftpc->file);
+    ftpc->file = NULL;
   }
 }
 
@@ -311,6 +414,7 @@ static CURLcode ftp_readresp(curl_socket_t sockfd,
   int code = 0;
 
   *ftpcode = 0; /* 0 for errors or not done */
+  *size = 0;
 
   ptr=buf + ftpc->nread_resp;
 
@@ -633,7 +737,7 @@ CURLcode Curl_GetFTPResponse(ssize_t *nreadp, /* return number of bytes read */
 
 /* This is the ONLY way to change FTP state! */
 static void state(struct connectdata *conn,
-                  ftpstate state)
+                  ftpstate newstate)
 {
 #if defined(CURLDEBUG) && !defined(CURL_DISABLE_VERBOSE_STRINGS)
   /* for debug purposes */
@@ -674,11 +778,11 @@ static void state(struct connectdata *conn,
 #endif
   struct ftp_conn *ftpc = &conn->proto.ftpc;
 #if defined(CURLDEBUG) && !defined(CURL_DISABLE_VERBOSE_STRINGS)
-  if(ftpc->state != state)
+  if(ftpc->state != newstate)
     infof(conn->data, "FTP %p state change from %s to %s\n",
-          ftpc, names[ftpc->state], names[state]);
+          ftpc, names[ftpc->state], names[newstate]);
 #endif
-  ftpc->state = state;
+  ftpc->state = newstate;
 }
 
 static CURLcode ftp_state_user(struct connectdata *conn)
@@ -706,9 +810,9 @@ static CURLcode ftp_state_pwd(struct connectdata *conn)
 }
 
 /* For the FTP "protocol connect" and "doing" phases only */
-int Curl_ftp_getsock(struct connectdata *conn,
-                     curl_socket_t *socks,
-                     int numsocks)
+static int Curl_ftp_getsock(struct connectdata *conn,
+                            curl_socket_t *socks,
+                            int numsocks)
 {
   struct ftp_conn *ftpc = &conn->proto.ftpc;
 
@@ -1013,28 +1117,28 @@ static CURLcode ftp_state_use_port(struct connectdata *conn,
   unsigned short ip[4];
   bool freeaddr = TRUE;
   socklen_t sslen = sizeof(sa);
-  const char *ftpport = data->set.str[STRING_FTPPORT];
+  const char *ftpportstr = data->set.str[STRING_FTPPORT];
 
   (void)fcmd; /* not used in the IPv4 code */
-  if(ftpport) {
+  if(ftpportstr) {
     in_addr_t in;
 
     /* First check if the given name is an IP address */
-    in=inet_addr(ftpport);
+    in=inet_addr(ftpportstr);
 
     if(in != CURL_INADDR_NONE)
       /* this is an IPv4 address */
-      addr = Curl_ip2addr(in, ftpport, 0);
+      addr = Curl_ip2addr(in, ftpportstr, 0);
     else {
-      if(Curl_if2ip(ftpport, myhost, sizeof(myhost))) {
+      if(Curl_if2ip(ftpportstr, myhost, sizeof(myhost))) {
         /* The interface to IP conversion provided a dotted address */
         in=inet_addr(myhost);
         addr = Curl_ip2addr(in, myhost, 0);
       }
-      else if(strlen(ftpport)> 1) {
+      else if(strlen(ftpportstr)> 1) {
         /* might be a host name! */
         struct Curl_dns_entry *h=NULL;
-        int rc = Curl_resolv(conn, ftpport, 0, &h);
+        int rc = Curl_resolv(conn, ftpportstr, 0, &h);
         if(rc == CURLRESOLV_PENDING)
           /* BLOCKING */
           rc = Curl_wait_for_resolv(conn, &h);
@@ -1048,11 +1152,11 @@ static CURLcode ftp_state_use_port(struct connectdata *conn,
                                since it points to a DNS cache entry! */
         } /* (h) */
         else {
-          infof(data, "Failed to resolve host name %s\n", ftpport);
+          infof(data, "Failed to resolve host name %s\n", ftpportstr);
         }
       } /* strlen */
     } /* CURL_INADDR_NONE */
-  } /* ftpport */
+  } /* ftpportstr */
 
   if(!addr) {
     /* pick a suitable default here */
@@ -1212,7 +1316,7 @@ static CURLcode ftp_state_post_rest(struct connectdata *conn)
   struct FTP *ftp = conn->data->reqdata.proto.ftp;
   struct SessionHandle *data = conn->data;
 
-  if(ftp->no_transfer) {
+  if(ftp->transfer != FTPTRANSFER_BODY) {
     /* doesn't transfer any data */
 
     /* still possibly do PRE QUOTE jobs */
@@ -1234,8 +1338,9 @@ static CURLcode ftp_state_post_size(struct connectdata *conn)
 {
   CURLcode result = CURLE_OK;
   struct FTP *ftp = conn->data->reqdata.proto.ftp;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
 
-  if(ftp->no_transfer && ftp->file) {
+  if((ftp->transfer != FTPTRANSFER_BODY) && ftpc->file) {
     /* if a "head"-like request is being made (on a file) */
 
     /* Determine if server can respond to REST command and therefore
@@ -1254,12 +1359,13 @@ static CURLcode ftp_state_post_type(struct connectdata *conn)
 {
   CURLcode result = CURLE_OK;
   struct FTP *ftp = conn->data->reqdata.proto.ftp;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
 
-  if(ftp->no_transfer && ftp->file) {
+  if((ftp->transfer == FTPTRANSFER_INFO) && ftpc->file) {
     /* if a "head"-like request is being made (on a file) */
 
-    /* we know ftp->file is a valid pointer to a file name */
-    NBFTPSENDF(conn, "SIZE %s", ftp->file);
+    /* we know ftpc->file is a valid pointer to a file name */
+    NBFTPSENDF(conn, "SIZE %s", ftpc->file);
 
     state(conn, FTP_SIZE);
   }
@@ -1279,10 +1385,55 @@ static CURLcode ftp_state_post_listtype(struct connectdata *conn)
      way. It has turned out that the NLST list output is not the same on all
      servers either... */
 
-  NBFTPSENDF(conn, "%s",
-             data->set.str[STRING_CUSTOMREQUEST]?
-             data->set.str[STRING_CUSTOMREQUEST]:
-             (data->set.ftp_list_only?"NLST":"LIST"));
+  /*
+     if FTPFILE_NOCWD was specified, we are currently in
+     the user's home directory, so we should add the path
+     as argument for the LIST / NLST / or custom command.
+     Whether the server will support this, is uncertain.
+
+     The other ftp_filemethods will CWD into dir/dir/ first and
+     then just do LIST (in that case: nothing to do here)
+  */
+  char *cmd,*lstArg,*slashPos;
+
+  lstArg = NULL;
+  if((data->set.ftp_filemethod == FTPFILE_NOCWD) &&
+     data->reqdata.path &&
+     data->reqdata.path[0] &&
+     strchr(data->reqdata.path,'/')) {
+
+    lstArg = strdup(data->reqdata.path);
+    if(!lstArg)
+      return CURLE_OUT_OF_MEMORY;
+
+    /* Check if path does not end with /, as then we cut off the file part */
+    if(lstArg[strlen(lstArg) - 1] != '/')  {
+
+      /* chop off the file part if format is dir/dir/file */
+      slashPos = strrchr(lstArg,'/');
+      *(slashPos+1) = '\0';
+    }
+  }
+
+  cmd = aprintf( "%s%s%s",
+                 data->set.str[STRING_CUSTOMREQUEST]?
+                 data->set.str[STRING_CUSTOMREQUEST]:
+                 (data->set.ftp_list_only?"NLST":"LIST"),
+                 lstArg? " ": "",
+                 lstArg? lstArg: "" );
+
+  if(!cmd) {
+    if(lstArg)
+      free(lstArg);
+    return CURLE_OUT_OF_MEMORY;
+  }
+
+  NBFTPSENDF(conn, "%s",cmd);
+
+  if(lstArg)
+    free(lstArg);
+
+  free(cmd);
 
   state(conn, FTP_LIST);
 
@@ -1316,17 +1467,19 @@ static CURLcode ftp_state_post_mdtm(struct connectdata *conn)
   CURLcode result = CURLE_OK;
   struct FTP *ftp = conn->data->reqdata.proto.ftp;
   struct SessionHandle *data = conn->data;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
 
   /* If we have selected NOBODY and HEADER, it means that we only want file
      information. Which in FTP can't be much more than the file size and
      date. */
-  if(conn->bits.no_body && data->set.include_header && ftp->file &&
+  if(conn->bits.no_body && ftpc->file &&
      ftp_need_type(conn, data->set.prefer_ascii)) {
     /* The SIZE command is _not_ RFC 959 specified, and therefor many servers
        may not support it! It is however the only way we have to get a file's
        size! */
 
-    ftp->no_transfer = TRUE; /* this means no actual transfer will be made */
+    ftp->transfer = FTPTRANSFER_INFO;
+    /* this means no actual transfer will be made */
 
     /* Some servers return different sizes for different modes, and thus we
        must set the proper type before we check the size */
@@ -1345,15 +1498,15 @@ static CURLcode ftp_state_post_mdtm(struct connectdata *conn)
 static CURLcode ftp_state_post_cwd(struct connectdata *conn)
 {
   CURLcode result = CURLE_OK;
-  struct FTP *ftp = conn->data->reqdata.proto.ftp;
   struct SessionHandle *data = conn->data;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
 
   /* Requested time of file or time-depended transfer? */
-  if((data->set.get_filetime || data->set.timecondition) && ftp->file) {
+  if((data->set.get_filetime || data->set.timecondition) && ftpc->file) {
 
     /* we have requested to get the modified-time of the file, this is a white
        spot as the MDTM is not mentioned in RFC959 */
-    NBFTPSENDF(conn, "MDTM %s", ftp->file);
+    NBFTPSENDF(conn, "MDTM %s", ftpc->file);
 
     state(conn, FTP_MDTM);
   }
@@ -1371,6 +1524,7 @@ static CURLcode ftp_state_ul_setup(struct connectdata *conn,
   CURLcode result = CURLE_OK;
   struct FTP *ftp = conn->data->reqdata.proto.ftp;
   struct SessionHandle *data = conn->data;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
   curl_off_t passed=0;
 
   if((data->reqdata.resume_from && !sizechecked) ||
@@ -1390,7 +1544,7 @@ static CURLcode ftp_state_ul_setup(struct connectdata *conn,
 
     if(data->reqdata.resume_from < 0 ) {
       /* Got no given size to start from, figure it out */
-      NBFTPSENDF(conn, "SIZE %s", ftp->file);
+      NBFTPSENDF(conn, "SIZE %s", ftpc->file);
       state(conn, FTP_STOR_SIZE);
       return result;
     }
@@ -1433,9 +1587,9 @@ static CURLcode ftp_state_ul_setup(struct connectdata *conn,
         /* no data to transfer */
         result=Curl_setup_transfer(conn, -1, -1, FALSE, NULL, -1, NULL);
 
-        /* Set no_transfer so that we won't get any error in
+        /* Set ->transfer so that we won't get any error in
          * Curl_ftp_done() because we didn't transfer anything! */
-        ftp->no_transfer = TRUE;
+        ftp->transfer = FTPTRANSFER_NONE;
 
         state(conn, FTP_STOP);
         return CURLE_OK;
@@ -1445,7 +1599,7 @@ static CURLcode ftp_state_ul_setup(struct connectdata *conn,
   } /* resume_from */
 
   NBFTPSENDF(conn, data->set.ftp_append?"APPE %s":"STOR %s",
-             ftp->file);
+             ftpc->file);
 
   state(conn, FTP_STOR);
 
@@ -1505,10 +1659,10 @@ static CURLcode ftp_state_quote(struct connectdata *conn,
       result = ftp_state_cwd(conn);
       break;
     case FTP_RETR_PREQUOTE:
-      if (ftp->no_transfer)
+      if (ftp->transfer != FTPTRANSFER_BODY)
         state(conn, FTP_STOP);
       else {
-        NBFTPSENDF(conn, "SIZE %s", ftp->file);
+        NBFTPSENDF(conn, "SIZE %s", ftpc->file);
         state(conn, FTP_RETR_SIZE);
       }
       break;
@@ -1810,6 +1964,7 @@ static CURLcode ftp_state_mdtm_resp(struct connectdata *conn,
   CURLcode result = CURLE_OK;
   struct SessionHandle *data=conn->data;
   struct FTP *ftp = data->reqdata.proto.ftp;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
 
   switch(ftpcode) {
   case 213:
@@ -1835,17 +1990,16 @@ static CURLcode ftp_state_mdtm_resp(struct connectdata *conn,
          we "emulate" a HTTP-style header in our output. */
 
       if(conn->bits.no_body &&
-         data->set.include_header &&
-         ftp->file &&
+         ftpc->file &&
          data->set.get_filetime &&
          (data->info.filetime>=0) ) {
         struct tm *tm;
-        time_t clock = (time_t)data->info.filetime;
+        time_t filetime = (time_t)data->info.filetime;
 #ifdef HAVE_GMTIME_R
         struct tm buffer;
-        tm = (struct tm *)gmtime_r(&clock, &buffer);
+        tm = (struct tm *)gmtime_r(&filetime, &buffer);
 #else
-        tm = gmtime(&clock);
+        tm = gmtime(&filetime);
 #endif
         /* format: "Tue, 15 Nov 1994 12:45:26" */
         snprintf(buf, BUFSIZE-1,
@@ -1880,7 +2034,7 @@ static CURLcode ftp_state_mdtm_resp(struct connectdata *conn,
       default:
         if(data->info.filetime <= data->set.timevalue) {
           infof(data, "The requested document is not new enough\n");
-          ftp->no_transfer = TRUE; /* mark this to not transfer data */
+          ftp->transfer = FTPTRANSFER_NONE; /* mark this to not transfer data */
           state(conn, FTP_STOP);
           return CURLE_OK;
         }
@@ -1888,7 +2042,7 @@ static CURLcode ftp_state_mdtm_resp(struct connectdata *conn,
       case CURL_TIMECOND_IFUNMODSINCE:
         if(data->info.filetime > data->set.timevalue) {
           infof(data, "The requested document is not old enough\n");
-          ftp->no_transfer = TRUE; /* mark this to not transfer data */
+          ftp->transfer = FTPTRANSFER_NONE; /* mark this to not transfer data */
           state(conn, FTP_STOP);
           return CURLE_OK;
         }
@@ -1942,6 +2096,7 @@ static CURLcode ftp_state_post_retr_size(struct connectdata *conn,
   CURLcode result = CURLE_OK;
   struct SessionHandle *data=conn->data;
   struct FTP *ftp = data->reqdata.proto.ftp;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
 
   if (data->set.max_filesize && (filesize > data->set.max_filesize)) {
     failf(data, "Maximum file size exceeded");
@@ -1992,9 +2147,9 @@ static CURLcode ftp_state_post_retr_size(struct connectdata *conn,
       result = Curl_setup_transfer(conn, -1, -1, FALSE, NULL, -1, NULL);
       infof(data, "File already completely downloaded\n");
 
-      /* Set no_transfer so that we won't get any error in Curl_ftp_done()
+      /* Set ->transfer so that we won't get any error in Curl_ftp_done()
        * because we didn't transfer the any file */
-      ftp->no_transfer = TRUE;
+      ftp->transfer = FTPTRANSFER_NONE;
       state(conn, FTP_STOP);
       return CURLE_OK;
     }
@@ -2010,7 +2165,7 @@ static CURLcode ftp_state_post_retr_size(struct connectdata *conn,
   }
   else {
     /* no resume */
-    NBFTPSENDF(conn, "RETR %s", ftp->file);
+    NBFTPSENDF(conn, "RETR %s", ftpc->file);
     state(conn, FTP_RETR);
   }
 
@@ -2059,7 +2214,7 @@ static CURLcode ftp_state_rest_resp(struct connectdata *conn,
                                     ftpstate instate)
 {
   CURLcode result = CURLE_OK;
-  struct FTP *ftp = conn->data->reqdata.proto.ftp;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
 
   switch(instate) {
   case FTP_REST:
@@ -2081,7 +2236,7 @@ static CURLcode ftp_state_rest_resp(struct connectdata *conn,
       result = CURLE_FTP_COULDNT_USE_REST;
     }
     else {
-      NBFTPSENDF(conn, "RETR %s", ftp->file);
+      NBFTPSENDF(conn, "RETR %s", ftpc->file);
       state(conn, FTP_RETR);
     }
     break;
@@ -2249,7 +2404,7 @@ static CURLcode ftp_state_get_resp(struct connectdata *conn,
   else {
     if((instate == FTP_LIST) && (ftpcode == 450)) {
       /* simply no matching files in the dir listing */
-      ftp->no_transfer = TRUE; /* don't download anything */
+      ftp->transfer = FTPTRANSFER_NONE; /* don't download anything */
       state(conn, FTP_STOP); /* this phase is over */
     }
     else {
@@ -2774,8 +2929,8 @@ static long ftp_state_timeout(struct connectdata *conn)
 
 
 /* called repeatedly until done from multi.c */
-CURLcode Curl_ftp_multi_statemach(struct connectdata *conn,
-                                  bool *done)
+static CURLcode Curl_ftp_multi_statemach(struct connectdata *conn,
+                                         bool *done)
 {
   curl_socket_t sock = conn->sock[FIRSTSOCKET];
   int rc;
@@ -2883,8 +3038,8 @@ static CURLcode ftp_init(struct connectdata *conn)
  * phase is done when this function returns, or FALSE is not. When called as
  * a part of the easy interface, it will always be TRUE.
  */
-CURLcode Curl_ftp_connect(struct connectdata *conn,
-                          bool *done) /* see description above */
+static CURLcode Curl_ftp_connect(struct connectdata *conn,
+                                 bool *done) /* see description above */
 {
   CURLcode result;
 #ifndef CURL_DISABLE_HTTP
@@ -2897,11 +3052,9 @@ CURLcode Curl_ftp_connect(struct connectdata *conn,
 
   *done = FALSE; /* default to not done yet */
 
-  if (data->reqdata.proto.ftp) {
-    Curl_ftp_disconnect(conn);
-    free(data->reqdata.proto.ftp);
-    data->reqdata.proto.ftp = NULL;
-  }
+  /* If there already is a protocol-specific struct allocated for this
+     sessionhandle, deal with it */
+  Curl_reset_reqproto(conn);
 
   result = ftp_init(conn);
   if(result)
@@ -2972,8 +3125,8 @@ CURLcode Curl_ftp_connect(struct connectdata *conn,
  *
  * Input argument is already checked for validity.
  */
-CURLcode Curl_ftp_done(struct connectdata *conn, CURLcode status,
-                       bool premature)
+static CURLcode Curl_ftp_done(struct connectdata *conn, CURLcode status,
+                              bool premature)
 {
   struct SessionHandle *data = conn->data;
   struct FTP *ftp = data->reqdata.proto.ftp;
@@ -3033,10 +3186,10 @@ CURLcode Curl_ftp_done(struct connectdata *conn, CURLcode status,
     ftpc->prevpath = NULL; /* no path */
 
   } else {
-    size_t flen = ftp->file?strlen(ftp->file):0; /* file is "raw" already */
+    size_t flen = ftpc->file?strlen(ftpc->file):0; /* file is "raw" already */
     size_t dlen = strlen(path)-flen;
     if(!ftpc->cwdfail) {
-      if(dlen) {
+      if(dlen && (data->set.ftp_filemethod != FTPFILE_NOCWD)) {
         ftpc->prevpath = path;
         if(flen)
           /* if 'path' is not the whole string */
@@ -3055,7 +3208,7 @@ CURLcode Curl_ftp_done(struct connectdata *conn, CURLcode status,
     }
   }
   /* free the dir tree and file parts */
-  freedirs(conn);
+  freedirs(ftpc);
 
 #if defined(HAVE_KRB4) || defined(HAVE_GSSAPI)
   Curl_sec_fflush_fd(conn, conn->sock[SECONDARYSOCKET]);
@@ -3079,7 +3232,7 @@ CURLcode Curl_ftp_done(struct connectdata *conn, CURLcode status,
 
   conn->sock[SECONDARYSOCKET] = CURL_SOCKET_BAD;
 
-  if(!ftp->no_transfer && !status && !premature) {
+  if((ftp->transfer == FTPTRANSFER_BODY) && !status && !premature) {
     /*
      * Let's see what the server says about the transfer we just performed,
      * but lower the timeout as sometimes this connection has died while the
@@ -3120,7 +3273,7 @@ CURLcode Curl_ftp_done(struct connectdata *conn, CURLcode status,
     if((-1 != data->set.infilesize) &&
        (data->set.infilesize != *ftp->bytecountp) &&
        !data->set.crlf &&
-       !ftp->no_transfer) {
+       (ftp->transfer == FTPTRANSFER_BODY)) {
       failf(data, "Uploaded unaligned file size (%" FORMAT_OFF_T
             " out of %" FORMAT_OFF_T " bytes)",
             *ftp->bytecountp, data->set.infilesize);
@@ -3150,7 +3303,7 @@ CURLcode Curl_ftp_done(struct connectdata *conn, CURLcode status,
   }
 
   /* clear these for next connection */
-  ftp->no_transfer = FALSE;
+  ftp->transfer = FTPTRANSFER_BODY;
   ftpc->dont_check = FALSE;
 
   /* Send any post-transfer QUOTE strings? */
@@ -3222,7 +3375,7 @@ static CURLcode ftp_nb_type(struct connectdata *conn,
 {
   struct ftp_conn *ftpc = &conn->proto.ftpc;
   CURLcode result;
-  char want = ascii?'A':'I';
+  char want = (char)(ascii?'A':'I');
 
   if (ftpc->transfertype == want) {
     state(conn, newstate);
@@ -3323,9 +3476,10 @@ static CURLcode ftp_range(struct connectdata *conn)
  * connected.
  */
 
-CURLcode Curl_ftp_nextconnect(struct connectdata *conn)
+static CURLcode Curl_ftp_nextconnect(struct connectdata *conn)
 {
   struct SessionHandle *data=conn->data;
+  struct ftp_conn *ftpc = &conn->proto.ftpc;
   CURLcode result = CURLE_OK;
 
   /* the ftp struct is inited in Curl_ftp_connect() */
@@ -3333,12 +3487,12 @@ CURLcode Curl_ftp_nextconnect(struct connectdata *conn)
 
   DEBUGF(infof(data, "DO-MORE phase starts\n"));
 
-  if(!ftp->no_transfer) {
-    /* a transfer is about to take place */
+  if(ftp->transfer <= FTPTRANSFER_INFO) {
+    /* a transfer is about to take place, or if not a file name was given
+       so we'll do a SIZE on it later and then we need the right TYPE first */
 
     if(data->set.upload) {
-      result = ftp_nb_type(conn, data->set.prefer_ascii,
-                                      FTP_STOR_TYPE);
+      result = ftp_nb_type(conn, data->set.prefer_ascii, FTP_STOR_TYPE);
       if (result)
         return result;
     }
@@ -3349,13 +3503,18 @@ CURLcode Curl_ftp_nextconnect(struct connectdata *conn)
       result = ftp_range(conn);
       if(result)
         ;
-      else if((data->set.ftp_list_only) || !ftp->file) {
+      else if(data->set.ftp_list_only || !ftpc->file) {
         /* The specified path ends with a slash, and therefore we think this
            is a directory that is requested, use LIST. But before that we
            need to set ASCII transfer mode. */
-        result = ftp_nb_type(conn, 1, FTP_LIST_TYPE);
-        if (result)
-          return result;
+
+        /* But only if a body transfer was requested. */
+        if(ftp->transfer == FTPTRANSFER_BODY) {
+          result = ftp_nb_type(conn, 1, FTP_LIST_TYPE);
+          if (result)
+            return result;
+        }
+        /* otherwise just fall through */
       }
       else {
         result = ftp_nb_type(conn, data->set.prefer_ascii, FTP_RETR_TYPE);
@@ -3366,7 +3525,7 @@ CURLcode Curl_ftp_nextconnect(struct connectdata *conn)
     result = ftp_easy_statemach(conn);
   }
 
-  if(ftp->no_transfer)
+  if(ftp->transfer != FTPTRANSFER_BODY)
     /* no data to transfer. FIX: it feels like a kludge to have this here
        too! */
     result=Curl_setup_transfer(conn, -1, -1, FALSE, NULL, -1, NULL);
@@ -3400,7 +3559,7 @@ CURLcode ftp_perform(struct connectdata *conn,
   if(conn->bits.no_body) {
     /* requested no body means no transfer... */
     struct FTP *ftp = conn->data->reqdata.proto.ftp;
-    ftp->no_transfer = TRUE;
+    ftp->transfer = FTPTRANSFER_INFO;
   }
 
 
@@ -3435,7 +3594,7 @@ CURLcode ftp_perform(struct connectdata *conn,
  *
  * The input argument is already checked for validity.
  */
-CURLcode Curl_ftp(struct connectdata *conn, bool *done)
+static CURLcode Curl_ftp(struct connectdata *conn, bool *done)
 {
   CURLcode retcode = CURLE_OK;
 
@@ -3447,6 +3606,7 @@ CURLcode Curl_ftp(struct connectdata *conn, bool *done)
     make sure we have a good 'struct FTP' to play with. For new connections,
     the struct FTP is allocated and setup in the Curl_ftp_connect() function.
   */
+  Curl_reset_reqproto(conn);
   retcode = ftp_init(conn);
   if(retcode)
     return retcode;
@@ -3632,7 +3792,7 @@ static CURLcode ftp_quit(struct connectdata *conn)
  * Disconnect from an FTP server. Cleanup protocol-specific per-connection
  * resources. BLOCKING.
  */
-CURLcode Curl_ftp_disconnect(struct connectdata *conn)
+static CURLcode Curl_ftp_disconnect(struct connectdata *conn)
 {
   struct ftp_conn *ftpc= &conn->proto.ftpc;
 
@@ -3645,25 +3805,26 @@ CURLcode Curl_ftp_disconnect(struct connectdata *conn)
   */
 
   /* The FTP session may or may not have been allocated/setup at this point! */
-  if(conn->data->reqdata.proto.ftp) {
-    (void)ftp_quit(conn); /* ignore errors on the QUIT */
+  (void)ftp_quit(conn); /* ignore errors on the QUIT */
 
-    if(ftpc->entrypath) {
-      struct SessionHandle *data = conn->data;
+  if(ftpc->entrypath) {
+    struct SessionHandle *data = conn->data;
+    if (data->state.most_recent_ftp_entrypath == ftpc->entrypath) {
       data->state.most_recent_ftp_entrypath = NULL;
-      free(ftpc->entrypath);
-      ftpc->entrypath = NULL;
-    }
-    if(ftpc->cache) {
-      free(ftpc->cache);
-      ftpc->cache = NULL;
-    }
-    freedirs(conn);
-    if(ftpc->prevpath) {
-      free(ftpc->prevpath);
-      ftpc->prevpath = NULL;
     }
+    free(ftpc->entrypath);
+    ftpc->entrypath = NULL;
   }
+  if(ftpc->cache) {
+    free(ftpc->cache);
+    ftpc->cache = NULL;
+  }
+  freedirs(ftpc);
+  if(ftpc->prevpath) {
+    free(ftpc->prevpath);
+    ftpc->prevpath = NULL;
+  }
+
   return CURLE_OK;
 }
 
@@ -3695,7 +3856,26 @@ CURLcode ftp_parse_url_path(struct connectdata *conn)
   switch(data->set.ftp_filemethod) {
   case FTPFILE_NOCWD:
     /* fastest, but less standard-compliant */
-    ftp->file = data->reqdata.path;  /* this is a full file path */
+
+    /*
+      The best time to check whether the path is a file or directory is right
+      here. so:
+
+      the first condition in the if() right here, is there just in case
+      someone decides to set path to NULL one day
+   */
+    if(data->reqdata.path &&
+       data->reqdata.path[0] &&
+       (data->reqdata.path[strlen(data->reqdata.path) - 1] != '/') )
+      ftpc->file = data->reqdata.path;  /* this is a full file path */
+    else
+      ftpc->file = NULL;
+      /*
+        ftpc->file is not used anywhere other than for operations on a file.
+        In other words, never for directory operations.
+        So we can safely set it to NULL here and use it as a
+        argument in dir/file decisions.
+      */
     break;
 
   case FTPFILE_SINGLECWD:
@@ -3703,7 +3883,7 @@ CURLcode ftp_parse_url_path(struct connectdata *conn)
     if(!path_to_use[0]) {
       /* no dir, no file */
       ftpc->dirdepth = 0;
-      ftp->file = NULL;
+      ftpc->file = NULL;
       break;
     }
     slash_pos=strrchr(cur_pos, '/');
@@ -3716,14 +3896,14 @@ CURLcode ftp_parse_url_path(struct connectdata *conn)
                                          slash_pos?(int)(slash_pos-cur_pos):1,
                                          NULL);
       if(!ftpc->dirs[0]) {
-        freedirs(conn);
+        freedirs(ftpc);
         return CURLE_OUT_OF_MEMORY;
       }
       ftpc->dirdepth = 1; /* we consider it to be a single dir */
-      ftp->file = slash_pos ? slash_pos+1 : cur_pos; /* rest is file name */
+      ftpc->file = slash_pos ? slash_pos+1 : cur_pos; /* rest is file name */
     }
     else
-      ftp->file = cur_pos;  /* this is a file name only */
+      ftpc->file = cur_pos;  /* this is a file name only */
     break;
 
   default: /* allow pretty much anything */
@@ -3757,12 +3937,12 @@ CURLcode ftp_parse_url_path(struct connectdata *conn)
             curl_easy_unescape(conn->data, cur_pos - absolute_dir, len, NULL);
           if (!ftpc->dirs[ftpc->dirdepth]) { /* run out of memory ... */
             failf(data, "no memory");
-            freedirs(conn);
+            freedirs(ftpc);
             return CURLE_OUT_OF_MEMORY;
           }
           if (isBadFtpString(ftpc->dirs[ftpc->dirdepth])) {
             free(ftpc->dirs[ftpc->dirdepth]);
-            freedirs(conn);
+            freedirs(ftpc);
             return CURLE_URL_MALFORMAT;
           }
         }
@@ -3778,33 +3958,33 @@ CURLcode ftp_parse_url_path(struct connectdata *conn)
           ftpc->diralloc *= 2; /* double the size each time */
           bigger = realloc(ftpc->dirs, ftpc->diralloc * sizeof(ftpc->dirs[0]));
           if(!bigger) {
-            freedirs(conn);
+            freedirs(ftpc);
             return CURLE_OUT_OF_MEMORY;
           }
           ftpc->dirs = (char **)bigger;
         }
       }
     }
-    ftp->file = cur_pos;  /* the rest is the file name */
+    ftpc->file = cur_pos;  /* the rest is the file name */
   }
 
-  if(ftp->file && *ftp->file) {
-    ftp->file = curl_easy_unescape(conn->data, ftp->file, 0, NULL);
-    if(NULL == ftp->file) {
-      freedirs(conn);
+  if(ftpc->file && *ftpc->file) {
+    ftpc->file = curl_easy_unescape(conn->data, ftpc->file, 0, NULL);
+    if(NULL == ftpc->file) {
+      freedirs(ftpc);
       failf(data, "no memory");
       return CURLE_OUT_OF_MEMORY;
     }
-    if (isBadFtpString(ftp->file)) {
-      freedirs(conn);
+    if (isBadFtpString(ftpc->file)) {
+      freedirs(ftpc);
       return CURLE_URL_MALFORMAT;
     }
   }
   else
-    ftp->file=NULL; /* instead of point to a zero byte, we make it a NULL
+    ftpc->file=NULL; /* instead of point to a zero byte, we make it a NULL
                        pointer */
 
-  if(data->set.upload && !ftp->file && !ftp->no_transfer) {
+  if(data->set.upload && !ftpc->file && (ftp->transfer == FTPTRANSFER_BODY)) {
     /* We need a file name when uploading. Return error! */
     failf(data, "Uploading to a URL without a file name!");
     return CURLE_URL_MALFORMAT;
@@ -3817,11 +3997,11 @@ CURLcode ftp_parse_url_path(struct connectdata *conn)
        strings */
     char *path = curl_easy_unescape(conn->data, data->reqdata.path, 0, NULL);
     if(!path) {
-      freedirs(conn);
+      freedirs(ftpc);
       return CURLE_OUT_OF_MEMORY;
     }
 
-    dlen = strlen(path) - (ftp->file?strlen(ftp->file):0);
+    dlen = strlen(path) - (ftpc->file?strlen(ftpc->file):0);
     if((dlen == strlen(ftpc->prevpath)) &&
        curl_strnequal(path, ftpc->prevpath, dlen)) {
       infof(data, "Request has same path as previous transfer\n");
@@ -3851,7 +4031,7 @@ static CURLcode ftp_dophase_done(struct connectdata *conn,
     return result;
   }
 
-  if(ftp->no_transfer)
+  if(ftp->transfer != FTPTRANSFER_BODY)
     /* no data to transfer */
     result=Curl_setup_transfer(conn, -1, -1, FALSE, NULL, -1, NULL);
   else if(!connected)
@@ -3864,8 +4044,8 @@ static CURLcode ftp_dophase_done(struct connectdata *conn,
 }
 
 /* called from multi.c while DOing */
-CURLcode Curl_ftp_doing(struct connectdata *conn,
-                        bool *dophase_done)
+static CURLcode Curl_ftp_doing(struct connectdata *conn,
+                               bool *dophase_done)
 {
   CURLcode result;
   result = Curl_ftp_multi_statemach(conn, dophase_done);
@@ -3922,9 +4102,79 @@ CURLcode ftp_regular_transfer(struct connectdata *conn,
       return result;
   }
   else
-    freedirs(conn);
+    freedirs(ftpc);
 
   return result;
 }
 
+static CURLcode Curl_ftp_setup_connection(struct connectdata * conn)
+{
+  struct SessionHandle *data = conn->data;
+  char * type;
+  char command;
+
+  if(conn->bits.httpproxy && !data->set.tunnel_thru_httpproxy) {
+    /* Unless we have asked to tunnel ftp operations through the proxy, we
+       switch and use HTTP operations only */
+#ifndef CURL_DISABLE_HTTP
+    if (conn->handler == &Curl_handler_ftp)
+      conn->handler = &Curl_handler_ftp_proxy;
+    else {
+#ifdef USE_SSL
+      conn->handler = &Curl_handler_ftps_proxy;
+#else
+      failf(data, "FTPS not supported!");
+      return CURLE_UNSUPPORTED_PROTOCOL;
+#endif
+    }
+
+#else
+    failf(data, "FTP over http proxy requires HTTP support built-in!");
+    return CURLE_UNSUPPORTED_PROTOCOL;
+#endif
+  }
+
+  data->reqdata.path++;   /* don't include the initial slash */
+
+  /* FTP URLs support an extension like ";type=<typecode>" that
+   * we'll try to get now! */
+  type = strstr(data->reqdata.path, ";type=");
+
+  if (!type)
+    type = strstr(conn->host.rawalloc, ";type=");
+
+  if (type) {
+    *type = 0;                     /* it was in the middle of the hostname */
+    command = (char) toupper((int) type[6]);
+
+    switch (command) {
+    case 'A': /* ASCII mode */
+      data->set.prefer_ascii = TRUE;
+      break;
+
+    case 'D': /* directory mode */
+      data->set.ftp_list_only = TRUE;
+      break;
+
+    case 'I': /* binary mode */
+    default:
+      /* switch off ASCII */
+      data->set.prefer_ascii = FALSE;
+      break;
+    }
+  }
+
+  return CURLE_OK;
+}
+
+#ifdef USE_SSL
+static CURLcode Curl_ftps_setup_connection(struct connectdata * conn)
+{
+  struct SessionHandle *data = conn->data;
+
+  conn->ssl[SECONDARYSOCKET].use = data->set.ftp_ssl != CURLUSESSL_CONTROL;
+  return Curl_ftp_setup_connection(conn);
+}
+#endif
+
 #endif /* CURL_DISABLE_FTP */

lib/ftp.h

@@ -24,20 +24,23 @@
  ***************************************************************************/
 
 #ifndef CURL_DISABLE_FTP
-CURLcode Curl_ftp(struct connectdata *conn, bool *done);
-CURLcode Curl_ftp_done(struct connectdata *conn, CURLcode, bool premature);
-CURLcode Curl_ftp_connect(struct connectdata *conn, bool *done);
-CURLcode Curl_ftp_disconnect(struct connectdata *conn);
+extern const struct Curl_handler Curl_handler_ftp;
+
+#ifdef USE_SSL
+extern const struct Curl_handler Curl_handler_ftps;
+#endif
+
+#ifndef CURL_DISABLE_HTTP
+extern const struct Curl_handler Curl_handler_ftp_proxy;
+
+# ifdef USE_SSL
+extern const struct Curl_handler Curl_handler_ftps_proxy;
+# endif
+#endif
+
 CURLcode Curl_ftpsendf(struct connectdata *, const char *fmt, ...);
 CURLcode Curl_nbftpsendf(struct connectdata *, const char *fmt, ...);
 CURLcode Curl_GetFTPResponse(ssize_t *nread, struct connectdata *conn,
                              int *ftpcode);
-CURLcode Curl_ftp_nextconnect(struct connectdata *conn);
-CURLcode Curl_ftp_multi_statemach(struct connectdata *conn, bool *done);
-int Curl_ftp_getsock(struct connectdata *conn,
-                     curl_socket_t *socks,
-                     int numsocks);
-CURLcode Curl_ftp_doing(struct connectdata *conn,
-                        bool *dophase_done);
 #endif /* CURL_DISABLE_FTP */
 #endif /* __FTP_H */

lib/getenv.c

@@ -5,7 +5,7 @@
  *                            | (__| |_| |  _ <| |___
  *                             \___|\___/|_| \_\_____|
  *
- * Copyright (C) 1998 - 2005, Daniel Stenberg, <daniel@haxx.se>, et al.
+ * Copyright (C) 1998 - 2007, Daniel Stenberg, <daniel@haxx.se>, et al.
  *
  * This software is licensed as described in the file COPYING, which
  * you should have received as part of this distribution. The terms
@@ -48,19 +48,16 @@ char *GetEnv(const char *variable)
   env[0] = '\0';
   if (temp != NULL)
     ExpandEnvironmentStrings(temp, env, sizeof(env));
+  return (env[0] != '\0')?strdup(env):NULL;
 #else
-#ifdef  VMS
   char *env = getenv(variable);
-  if (env && strcmp("HOME",variable) == 0) {
-        env = decc$translate_vms(env);
-  }
-#else
-  /* no length control */
-  char *env = getenv(variable);
-#endif
+#ifdef VMS
+  if (env && strcmp("HOME",variable) == 0)
+    env = decc$translate_vms(env);
 #endif
   return (env && env[0])?strdup(env):NULL;
 #endif
+#endif
 }
 
 char *curl_getenv(const char *v)

lib/gtls.c

@@ -352,7 +352,7 @@ Curl_gtls_connect(struct connectdata *conn,
   if(!chainp) {
     if(data->set.ssl.verifyhost) {
       failf(data, "failed to get server cert");
-      return CURLE_SSL_PEER_CERTIFICATE;
+      return CURLE_PEER_FAILED_VERIFICATION;
     }
     infof(data, "\t common name: WARNING couldn't obtain\n");
   }
@@ -413,7 +413,7 @@ Curl_gtls_connect(struct connectdata *conn,
       failf(data, "SSL: certificate subject name (%s) does not match "
             "target host name '%s'", certbuf, conn->host.dispname);
       gnutls_x509_crt_deinit(x509_cert);
-      return CURLE_SSL_PEER_CERTIFICATE;
+      return CURLE_PEER_FAILED_VERIFICATION;
     }
     else
       infof(data, "\t common name: %s (does not match '%s')\n",
@@ -433,7 +433,7 @@ Curl_gtls_connect(struct connectdata *conn,
   if(clock < time(NULL)) {
     if (data->set.ssl.verifypeer) {
       failf(data, "server certificate expiration date has passed.");
-      return CURLE_SSL_PEER_CERTIFICATE;
+      return CURLE_PEER_FAILED_VERIFICATION;
     }
     else
       infof(data, "\t server certificate expiration date FAILED\n");
@@ -451,7 +451,7 @@ Curl_gtls_connect(struct connectdata *conn,
   if(clock > time(NULL)) {
     if (data->set.ssl.verifypeer) {
       failf(data, "server certificate not activated yet.");
-      return CURLE_SSL_PEER_CERTIFICATE;
+      return CURLE_PEER_FAILED_VERIFICATION;
     }
     else
       infof(data, "\t server certificate activation date FAILED\n");

lib/gtls.h

@@ -31,7 +31,7 @@ CURLcode Curl_gtls_connect(struct connectdata *conn, int sockindex);
 void Curl_gtls_close_all(struct SessionHandle *data);
 
  /* close a SSL connection */
-void Curl_gtls_close(struct connectdata *conn, int index);
+void Curl_gtls_close(struct connectdata *conn, int sockindex);
 
 /* return number of sent (non-SSL) bytes */
 ssize_t Curl_gtls_send(struct connectdata *conn, int sockindex,

lib/hash.c

@@ -117,12 +117,12 @@ mk_hash_element(const void *key, size_t key_len, const void *p)
     (struct curl_hash_element *) malloc(sizeof(struct curl_hash_element));
 
   if(he) {
-    void *dup = malloc(key_len);
-    if(dup) {
+    void *dupkey = malloc(key_len);
+    if(dupkey) {
       /* copy the key */
-      memcpy(dup, key, key_len);
+      memcpy(dupkey, key, key_len);
 
-      he->key = dup;
+      he->key = dupkey;
       he->key_len = key_len;
       he->ptr = (void *) p;
     }

lib/hostares.c

@@ -109,7 +109,8 @@ int Curl_resolv_getsock(struct connectdata *conn,
 
 {
   struct timeval maxtime;
-  struct timeval timeout;
+  struct timeval timebuf;
+  struct timeval *timeout;
   int max = ares_getsock(conn->data->state.areschannel,
                          (int *)socks, numsocks);
 
@@ -117,10 +118,10 @@ int Curl_resolv_getsock(struct connectdata *conn,
   maxtime.tv_sec = CURL_TIMEOUT_RESOLVE;
   maxtime.tv_usec = 0;
 
-  ares_timeout(conn->data->state.areschannel, &maxtime, &timeout);
+  timeout = ares_timeout(conn->data->state.areschannel, &maxtime, &timebuf);
 
   Curl_expire(conn->data,
-              (timeout.tv_sec * 1000) + (timeout.tv_usec/1000) );
+              (timeout->tv_sec * 1000) + (timeout->tv_usec/1000));
 
   return max;
 }
@@ -254,7 +255,7 @@ CURLcode Curl_wait_for_resolv(struct connectdata *conn,
     tvp = ares_timeout(data->state.areschannel, &store, &tv);
 
     /* use the timeout period ares returned to us above */
-    ares_waitperform(conn, tv.tv_sec * 1000 + tv.tv_usec/1000);
+    ares_waitperform(conn, (int)(tvp->tv_sec * 1000 + tvp->tv_usec/1000));
 
     if(conn->async.done)
       break;
@@ -278,7 +279,7 @@ CURLcode Curl_wait_for_resolv(struct connectdata *conn,
     /* a name was not resolved */
     if((timeout < 0) || (conn->async.status == ARES_ETIMEOUT)) {
       failf(data, "Resolving host timed out: %s", conn->host.dispname);
-      rc = CURLE_OPERATION_TIMEDOUT;
+      rc = CURLE_COULDNT_RESOLVE_HOST;
     }
     else if(conn->async.done) {
       failf(data, "Could not resolve host: %s (%s)", conn->host.dispname,

lib/hostasyn.c

@@ -151,19 +151,31 @@ static CURLcode addrinfo_callback(void *arg, /* "struct connectdata *" */
 
 CURLcode Curl_addrinfo4_callback(void *arg, /* "struct connectdata *" */
                                  int status,
+#ifdef HAVE_CARES_CALLBACK_TIMEOUTS
+                                 int timeouts,
+#endif
                                  struct hostent *hostent)
 {
+#ifdef HAVE_CARES_CALLBACK_TIMEOUTS
+  (void)timeouts; /* ignored */
+#endif
   return addrinfo_callback(arg, status, hostent);
 }
 
 #ifdef CURLRES_IPV6
 CURLcode Curl_addrinfo6_callback(void *arg, /* "struct connectdata *" */
                                  int status,
+#ifdef HAVE_CARES_CALLBACK_TIMEOUTS
+                                 int timeouts,
+#endif
                                  struct addrinfo *ai)
 {
  /* NOTE: for CURLRES_ARES, the 'ai' argument is really a
   * 'struct hostent' pointer.
   */
+#ifdef HAVE_CARES_CALLBACK_TIMEOUTS
+  (void)timeouts; /* ignored */
+#endif
   return addrinfo_callback(arg, status, ai);
 }
 #endif

lib/hostip.c

@@ -398,7 +398,6 @@ int Curl_resolv(struct connectdata *conn,
   char *entry_id = NULL;
   struct Curl_dns_entry *dns = NULL;
   size_t entry_len;
-  int wait;
   struct SessionHandle *data = conn->data;
   CURLcode result;
   int rc;
@@ -447,19 +446,20 @@ int Curl_resolv(struct connectdata *conn,
     /* The entry was not in the cache. Resolve it to IP address */
 
     Curl_addrinfo *addr;
+    int respwait;
 
     /* Check what IP specifics the app has requested and if we can provide it.
      * If not, bail out. */
     if(!Curl_ipvalid(data))
       return CURLRESOLV_ERROR;
 
-    /* If Curl_getaddrinfo() returns NULL, 'wait' might be set to a non-zero
-       value indicating that we need to wait for the response to the resolve
-       call */
-    addr = Curl_getaddrinfo(conn, hostname, port, &wait);
+    /* If Curl_getaddrinfo() returns NULL, 'respwait' might be set to a
+       non-zero value indicating that we need to wait for the response to the
+       resolve call */
+    addr = Curl_getaddrinfo(conn, hostname, port, &respwait);
 
     if (!addr) {
-      if(wait) {
+      if(respwait) {
         /* the response to our resolve call will come asynchronously at
            a later time, good or bad */
         /* First, check that we haven't received the info by now */

lib/hostip.h

@@ -7,7 +7,7 @@
  *                            | (__| |_| |  _ <| |___
  *                             \___|\___/|_| \_\_____|
  *
- * Copyright (C) 1998 - 2006, Daniel Stenberg, <daniel@haxx.se>, et al.
+ * Copyright (C) 1998 - 2007, Daniel Stenberg, <daniel@haxx.se>, et al.
  *
  * This software is licensed as described in the file COPYING, which
  * you should have received as part of this distribution. The terms
@@ -35,6 +35,10 @@
  * Setup comfortable CURLRES_* defines to use in the host*.c sources.
  */
 
+#ifdef USE_ARES
+#include <ares_version.h>
+#endif
+
 #ifdef USE_ARES
 #define CURLRES_ASYNCH
 #define CURLRES_ARES
@@ -85,6 +89,10 @@
 
 #ifdef CURLRES_ARES
 #define CURL_ASYNC_SUCCESS ARES_SUCCESS
+#if ARES_VERSION >= 0x010500
+/* c-ares 1.5.0 or later, the callback proto is modified */
+#define HAVE_CARES_CALLBACK_TIMEOUTS 1
+#endif
 #else
 #define CURL_ASYNC_SUCCESS CURLE_OK
 #define ares_cancel(x) do {} while(0)
@@ -216,11 +224,17 @@ int curl_dogetnameinfo(GETNAMEINFO_QUAL_ARG1 GETNAMEINFO_TYPE_ARG1 sa,
    resolve, ipv4 */
 CURLcode Curl_addrinfo4_callback(void *arg,
                                  int status,
+#ifdef HAVE_CARES_CALLBACK_TIMEOUTS
+                                 int timeouts,
+#endif
                                  struct hostent *hostent);
 /* This is the callback function that is used when we build with asynch
    resolve, ipv6 */
 CURLcode Curl_addrinfo6_callback(void *arg,
                                  int status,
+#ifdef HAVE_CARES_CALLBACK_TIMEOUTS
+                                 int timeouts,
+#endif
                                  struct addrinfo *ai);
 
 

lib/http.c

@@ -104,6 +104,59 @@
 /* The last #include file should be: */
 #include "memdebug.h"
 
+
+/*
+ * Forward declarations.
+ */
+
+static CURLcode Curl_https_connecting(struct connectdata *conn, bool *done);
+#ifdef USE_SSL
+static int Curl_https_getsock(struct connectdata *conn,
+                              curl_socket_t *socks,
+                              int numsocks);
+#endif
+
+/*
+ * HTTP handler interface.
+ */
+const struct Curl_handler Curl_handler_http = {
+  "HTTP",                               /* scheme */
+  ZERO_NULL,                            /* setup_connection */
+  Curl_http,                            /* do_it */
+  Curl_http_done,                       /* done */
+  ZERO_NULL,                            /* do_more */
+  Curl_http_connect,                    /* connect_it */
+  ZERO_NULL,                            /* connecting */
+  ZERO_NULL,                            /* doing */
+  ZERO_NULL,                            /* proto_getsock */
+  ZERO_NULL,                            /* doing_getsock */
+  ZERO_NULL,                            /* disconnect */
+  PORT_HTTP,                            /* defport */
+  PROT_HTTP,                            /* protocol */
+};
+
+#ifdef USE_SSL
+/*
+ * HTTPS handler interface.
+ */
+const struct Curl_handler Curl_handler_https = {
+  "HTTPS",                              /* scheme */
+  ZERO_NULL,                            /* setup_connection */
+  Curl_http,                            /* do_it */
+  Curl_http_done,                       /* done */
+  ZERO_NULL,                            /* do_more */
+  Curl_http_connect,                    /* connect_it */
+  Curl_https_connecting,                /* connecting */
+  ZERO_NULL,                            /* doing */
+  Curl_https_getsock,                   /* proto_getsock */
+  ZERO_NULL,                            /* doing_getsock */
+  ZERO_NULL,                            /* disconnect */
+  PORT_HTTPS,                           /* defport */
+  PROT_HTTP | PROT_HTTPS | PROT_SSL     /* protocol */
+};
+#endif
+
+
 /*
  * checkheaders() checks the linked list of custom HTTP headers for a
  * particular header (prefix).
@@ -424,6 +477,18 @@ Curl_http_output_auth(struct connectdata *conn,
   /* Send proxy authentication header if needed */
   if (conn->bits.httpproxy &&
       (conn->bits.tunnel_proxy == proxytunnel)) {
+#ifdef HAVE_GSSAPI
+    if((authproxy->picked == CURLAUTH_GSSNEGOTIATE) &&
+       data->state.negotiate.context &&
+       !GSS_ERROR(data->state.negotiate.status)) {
+      auth="GSS-Negotiate";
+      result = Curl_output_negotiate(conn, TRUE);
+      if (result)
+        return result;
+      authproxy->done = TRUE;
+    } 
+    else
+#endif
 #ifdef USE_NTLM
     if(authproxy->picked == CURLAUTH_NTLM) {
       auth="NTLM";
@@ -486,7 +551,7 @@ Curl_http_output_auth(struct connectdata *conn,
          data->state.negotiate.context &&
          !GSS_ERROR(data->state.negotiate.status)) {
         auth="GSS-Negotiate";
-        result = Curl_output_negotiate(conn);
+        result = Curl_output_negotiate(conn, FALSE);
         if (result)
           return result;
         authhost->done = TRUE;
@@ -593,7 +658,7 @@ CURLcode Curl_http_input_auth(struct connectdata *conn,
     authp->avail |= CURLAUTH_GSSNEGOTIATE;
     if(authp->picked == CURLAUTH_GSSNEGOTIATE) {
       /* if exactly this is wanted, go */
-      int neg = Curl_input_negotiate(conn, start);
+      int neg = Curl_input_negotiate(conn, (bool)(httpcode == 407), start);
       if (neg == 0) {
         data->reqdata.newurl = strdup(data->change.url);
         data->state.authproblem = (data->reqdata.newurl == NULL);
@@ -1131,6 +1196,7 @@ CURLcode Curl_proxyCONNECT(struct connectdata *conn,
   curl_socket_t tunnelsocket = conn->sock[sockindex];
   curl_off_t cl=0;
   bool closeConnection = FALSE;
+  bool chunked_encoding = FALSE;
   long check;
 
 #define SELECT_OK      0
@@ -1191,37 +1257,37 @@ CURLcode Curl_proxyCONNECT(struct connectdata *conn,
            data->set.str[STRING_USERAGENT])
           useragent = conn->allocptr.uagent;
 
-	/* Send the connect request to the proxy */
-	/* BLOCKING */
-	result =
-	  add_bufferf(req_buffer,
-		      "CONNECT %s:%d HTTP/1.0\r\n"
-		      "%s"  /* Host: */
-		      "%s"  /* Proxy-Authorization */
-		      "%s"  /* User-Agent */
-		      "%s", /* Proxy-Connection */
-		      hostname, remote_port,
-		      host,
-		      conn->allocptr.proxyuserpwd?
-		      conn->allocptr.proxyuserpwd:"",
-		      useragent,
-		      proxyconn);
+        /* Send the connect request to the proxy */
+        /* BLOCKING */
+        result =
+          add_bufferf(req_buffer,
+                      "CONNECT %s:%d HTTP/1.0\r\n"
+                      "%s"  /* Host: */
+                      "%s"  /* Proxy-Authorization */
+                      "%s"  /* User-Agent */
+                      "%s", /* Proxy-Connection */
+                      hostname, remote_port,
+                      host,
+                      conn->allocptr.proxyuserpwd?
+                      conn->allocptr.proxyuserpwd:"",
+                      useragent,
+                      proxyconn);
 
-	if(host && *host)
-	  free(host);
+        if(host && *host)
+          free(host);
 
-	if(CURLE_OK == result)
-	  result = add_custom_headers(conn, req_buffer);
+        if(CURLE_OK == result)
+          result = add_custom_headers(conn, req_buffer);
 
-	if(CURLE_OK == result)
-	  /* CRLF terminate the request */
-	  result = add_bufferf(req_buffer, "\r\n");
+        if(CURLE_OK == result)
+          /* CRLF terminate the request */
+          result = add_bufferf(req_buffer, "\r\n");
 
-	if(CURLE_OK == result) {
-	  /* Now send off the request */
-	  result = add_buffer_send(req_buffer, conn,
-				   &data->info.request_size, 0, sockindex);
-	}
+        if(CURLE_OK == result) {
+          /* Now send off the request */
+          result = add_buffer_send(req_buffer, conn,
+                                   &data->info.request_size, 0, sockindex);
+        }
         req_buffer = NULL;
         if(result)
           failf(data, "Failed sending CONNECT to proxy");
@@ -1328,13 +1394,35 @@ CURLcode Curl_proxyCONNECT(struct connectdata *conn,
             nread += gotbytes;
 
             if(keepon > TRUE) {
-              /* This means we are currently ignoring a response-body, so we
-                 simply count down our counter and make sure to break out of
-                 the loop when we're done! */
-              cl -= gotbytes;
-              if(cl<=0) {
-                keepon = FALSE;
-                break;
+              /* This means we are currently ignoring a response-body */
+
+              nread = 0; /* make next read start over in the read buffer */
+              if(cl) {
+                /* A Content-Length based body: simply count down the counter
+                   and make sure to break out of the loop when we're done! */
+                cl -= gotbytes;
+                if(cl<=0) {
+                  keepon = FALSE;
+                  break;
+                }
+              }
+              else {
+                /* chunked-encoded body, so we need to do the chunked dance
+                   properly to know when the end of the body is reached */
+                CHUNKcode r;
+                ssize_t tookcareof=0;
+
+                /* now parse the chunked piece of data so that we can
+                   properly tell when the stream ends */
+                r = Curl_httpchunk_read(conn, ptr, gotbytes, &tookcareof);
+                if(r == CHUNKE_STOP) {
+                  /* we're done reading chunks! */
+                  infof(data, "chunk reading DONE\n");
+                  keepon = FALSE;
+                }
+                else
+                  infof(data, "Read %d bytes of chunk, continue\n",
+                        tookcareof);
               }
             }
             else
@@ -1354,7 +1442,8 @@ CURLcode Curl_proxyCONNECT(struct connectdata *conn,
                   if(data->set.include_header)
                     writetype |= CLIENTWRITE_BODY;
 
-                  result = Curl_client_write(conn, writetype, line_start, perline);
+                  result = Curl_client_write(conn, writetype, line_start,
+                                             perline);
                   if(result)
                     return result;
 
@@ -1365,19 +1454,62 @@ CURLcode Curl_proxyCONNECT(struct connectdata *conn,
                   if(('\r' == line_start[0]) ||
                      ('\n' == line_start[0])) {
                     /* end of response-headers from the proxy */
-                    if(cl && (407 == k->httpcode) &&
-                       !data->state.authproblem) {
+                    nread = 0; /* make next read start over in the read
+                                  buffer */
+                    if((407 == k->httpcode) && !data->state.authproblem) {
                       /* If we get a 407 response code with content length
-                       * when we have no auth problem, we must ignore the
-                       * whole response-body */
+                         when we have no auth problem, we must ignore the
+                         whole response-body */
                       keepon = 2;
-                      infof(data, "Ignore %" FORMAT_OFF_T
-                            " bytes of response-body\n", cl);
-                      cl -= (gotbytes - i);/* remove the remaining chunk of
-                                              what we already read */
-                      if(cl<=0)
-                        /* if the whole thing was already read, we are done! */
+
+                      if(cl) {
+
+                        infof(data, "Ignore %" FORMAT_OFF_T
+                              " bytes of response-body\n", cl);
+                        /* remove the remaining chunk of what we already
+                           read */
+                        cl -= (gotbytes - i);
+
+                        if(cl<=0)
+                          /* if the whole thing was already read, we are done!
+                           */
+                          keepon=FALSE;
+                      }
+                      else if(chunked_encoding) {
+                        CHUNKcode r;
+                        /* We set ignorebody true here since the chunked
+                           decoder function will acknowledge that. Pay
+                           attention so that this is cleared again when this
+                           function returns! */
+                        k->ignorebody = TRUE;
+                        infof(data, "%d bytes of chunk left\n", gotbytes-i);
+
+                        if(line_start[1] == '\n') {
+                          /* this can only be a LF if the letter at index 0
+                             was a CR */
+                          line_start++;
+                          i++;
+                        }
+
+                        /* now parse the chunked piece of data so that we can
+                           properly tell when the stream ends */
+                        r = Curl_httpchunk_read(conn, line_start+1,
+                                                  gotbytes -i, &gotbytes);
+                        if(r == CHUNKE_STOP) {
+                          /* we're done reading chunks! */
+                          infof(data, "chunk reading DONE\n");
+                          keepon = FALSE;
+                        }
+                        else
+                          infof(data, "Read %d bytes of chunk, continue\n",
+                                gotbytes);
+                      }
+                      else {
+                        /* without content-length or chunked encoding, we
+                           can't keep the connection alive since the close is
+                           the end signal so we bail out at once instead */
                         keepon=FALSE;
+                      }
                     }
                     else
                       keepon = FALSE;
@@ -1403,6 +1535,13 @@ CURLcode Curl_proxyCONNECT(struct connectdata *conn,
                   else if(Curl_compareheader(line_start,
                                              "Connection:", "close"))
                     closeConnection = TRUE;
+                  else if(Curl_compareheader(line_start,
+                                             "Transfer-Encoding:", "chunked")) {
+                    infof(data, "CONNECT responded chunked\n");
+                    chunked_encoding = TRUE;
+                    /* init our chunky engine */
+                    Curl_httpchunk_init(conn);
+                  }
                   else if(Curl_compareheader(line_start,
                                              "Proxy-Connection:", "close"))
                     closeConnection = TRUE;
@@ -1460,6 +1599,7 @@ CURLcode Curl_proxyCONNECT(struct connectdata *conn,
   data->state.authproxy.done = TRUE;
 
   infof (data, "Proxy replied OK to CONNECT request\n");
+  k->ignorebody = FALSE; /* put it (back) to non-ignore state */
   return CURLE_OK;
 }
 
@@ -1532,7 +1672,7 @@ CURLcode Curl_http_connect(struct connectdata *conn, bool *done)
   return CURLE_OK;
 }
 
-CURLcode Curl_https_connecting(struct connectdata *conn, bool *done)
+static CURLcode Curl_https_connecting(struct connectdata *conn, bool *done)
 {
   CURLcode result;
   DEBUGASSERT((conn) && (conn->protocol & PROT_HTTPS));
@@ -1548,9 +1688,9 @@ CURLcode Curl_https_connecting(struct connectdata *conn, bool *done)
 #ifdef USE_SSLEAY
 /* This function is OpenSSL-specific. It should be made to query the generic
    SSL layer instead. */
-int Curl_https_getsock(struct connectdata *conn,
-                       curl_socket_t *socks,
-                       int numsocks)
+static int Curl_https_getsock(struct connectdata *conn,
+                              curl_socket_t *socks,
+                              int numsocks)
 {
   if (conn->protocol & PROT_HTTPS) {
     struct ssl_connect_data *connssl = &conn->ssl[FIRSTSOCKET];
@@ -1740,6 +1880,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
   CURLcode result=CURLE_OK;
   struct HTTP *http;
   char *ppath = data->reqdata.path;
+  char ftp_typecode[sizeof(";type=?")] = "";
   char *host = conn->host.name;
   const char *te = ""; /* transfer-encoding */
   char *ptr;
@@ -1753,13 +1894,16 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
      the rest of the request in the PERFORM phase. */
   *done = TRUE;
 
+  /* If there already is a protocol-specific struct allocated for this
+     sessionhandle, deal with it */
+  Curl_reset_reqproto(conn);
+
   if(!data->reqdata.proto.http) {
     /* Only allocate this struct if we don't already have it! */
 
-    http = (struct HTTP *)malloc(sizeof(struct HTTP));
+    http = (struct HTTP *)calloc(sizeof(struct HTTP), 1);
     if(!http)
       return CURLE_OUT_OF_MEMORY;
-    memset(http, 0, sizeof(struct HTTP));
     data->reqdata.proto.http = http;
   }
   else
@@ -1950,6 +2094,23 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
       }
     }
     ppath = data->change.url;
+    /* when doing ftp, append ;type=<a|i> if not present */
+    if (checkprefix("ftp://", ppath) || checkprefix("ftps://", ppath)) {
+      char *p = strstr(ppath, ";type=");
+      if (p && p[6] && p[7] == 0) {
+        switch (toupper((int)((unsigned char)p[6]))) {
+        case 'A':
+        case 'D':
+        case 'I':
+          break;
+        default:
+          p = NULL;
+        }
+      }
+      if (!p)
+        snprintf(ftp_typecode, sizeof(ftp_typecode), ";type=%c",
+                 data->set.prefer_ascii ? 'a' : 'i');
+    }
   }
   if(HTTPREQ_POST_FORM == httpreq) {
     /* we must build the whole darned post sequence first, so that we have
@@ -2093,7 +2254,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
     result =
       add_bufferf(req_buffer,
                   "%s " /* GET/HEAD/POST/PUT */
-                  "%s HTTP/%s\r\n" /* path + HTTP version */
+                  "%s%s HTTP/%s\r\n" /* path + HTTP version */
                   "%s" /* proxyuserpwd */
                   "%s" /* userpwd */
                   "%s" /* range */
@@ -2108,6 +2269,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
 
                 request,
                 ppath,
+                ftp_typecode,
                 httpstring,
                 conn->allocptr.proxyuserpwd?
                 conn->allocptr.proxyuserpwd:"",
@@ -2395,8 +2557,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
         /* figure out the size of the postfields */
         postsize = (data->set.postfieldsize != -1)?
           data->set.postfieldsize:
-          (data->set.str[STRING_POSTFIELDS]?
-           (curl_off_t)strlen(data->set.str[STRING_POSTFIELDS]):0);
+          (data->set.postfields? (curl_off_t)strlen(data->set.postfields):0);
 
       if(!conn->bits.upload_chunky) {
         /* We only set Content-Length and allow a custom Content-Length if
@@ -2421,7 +2582,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
           return result;
       }
 
-      if(data->set.str[STRING_POSTFIELDS]) {
+      if(data->set.postfields) {
 
         /* for really small posts we don't use Expect: headers at all, and for
            the somewhat bigger ones we allow the app to disable it */
@@ -2449,7 +2610,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
           if(!conn->bits.upload_chunky) {
             /* We're not sending it 'chunked', append it to the request
                already now to reduce the number if send() calls */
-            result = add_buffer(req_buffer, data->set.str[STRING_POSTFIELDS],
+            result = add_buffer(req_buffer, data->set.postfields,
                                 (size_t)postsize);
             included_body = postsize;
           }
@@ -2457,7 +2618,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
             /* Append the POST data chunky-style */
             result = add_bufferf(req_buffer, "%x\r\n", (int)postsize);
             if(CURLE_OK == result)
-              result = add_buffer(req_buffer, data->set.str[STRING_POSTFIELDS],
+              result = add_buffer(req_buffer, data->set.postfields,
                                   (size_t)postsize);
             if(CURLE_OK == result)
               result = add_buffer(req_buffer,
@@ -2471,7 +2632,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
         else {
           /* A huge POST coming up, do data separate from the request */
           http->postsize = postsize;
-          http->postdata = data->set.str[STRING_POSTFIELDS];
+          http->postdata = data->set.postfields;
 
           http->sending = HTTPSEND_BODY;
 

lib/http.h

@@ -24,6 +24,13 @@
  * $Id$
  ***************************************************************************/
 #ifndef CURL_DISABLE_HTTP
+
+extern const struct Curl_handler Curl_handler_http;
+
+#ifdef USE_SSL
+extern const struct Curl_handler Curl_handler_https;
+#endif
+
 bool Curl_compareheader(const char *headerline,  /* line to check */
                         const char *header,   /* header keyword _with_ colon */
                         const char *content); /* content string to find */
@@ -37,10 +44,6 @@ CURLcode Curl_proxyCONNECT(struct connectdata *conn,
 CURLcode Curl_http(struct connectdata *conn, bool *done);
 CURLcode Curl_http_done(struct connectdata *, CURLcode, bool premature);
 CURLcode Curl_http_connect(struct connectdata *conn, bool *done);
-CURLcode Curl_https_connecting(struct connectdata *conn, bool *done);
-int Curl_https_getsock(struct connectdata *conn,
-                       curl_socket_t *socks,
-                       int numsocks);
 
 /* The following functions are defined in http_chunks.c */
 void Curl_httpchunk_init(struct connectdata *conn);

lib/http_chunks.c

@@ -84,7 +84,7 @@
 
 void Curl_httpchunk_init(struct connectdata *conn)
 {
-  struct Curl_chunker *chunk = &conn->data->reqdata.proto.http->chunk;
+  struct Curl_chunker *chunk = &conn->chunk;
   chunk->hexindex=0; /* start at 0 */
   chunk->dataleft=0; /* no data left yet! */
   chunk->state = CHUNK_HEX; /* we get hex first! */
@@ -108,7 +108,7 @@ CHUNKcode Curl_httpchunk_read(struct connectdata *conn,
 {
   CURLcode result=CURLE_OK;
   struct SessionHandle *data = conn->data;
-  struct Curl_chunker *ch = &data->reqdata.proto.http->chunk;
+  struct Curl_chunker *ch = &conn->chunk;
   struct Curl_transfer_keeper *k = &data->reqdata.keep;
   size_t piece;
   size_t length = (size_t)datalen;
@@ -124,11 +124,11 @@ CHUNKcode Curl_httpchunk_read(struct connectdata *conn,
   while(length) {
     switch(ch->state) {
     case CHUNK_HEX:
-       /* Check for an ASCII hex digit.
-          We avoid the use of isxdigit to accommodate non-ASCII hosts. */
-       if((*datap >= 0x30 && *datap <= 0x39)    /* 0-9 */
-       || (*datap >= 0x41 && *datap <= 0x46)    /* A-F */
-       || (*datap >= 0x61 && *datap <= 0x66)) { /* a-f */
+      /* Check for an ASCII hex digit.
+         We avoid the use of isxdigit to accommodate non-ASCII hosts. */
+      if((*datap >= 0x30 && *datap <= 0x39)    /* 0-9 */
+         || (*datap >= 0x41 && *datap <= 0x46)    /* A-F */
+         || (*datap >= 0x61 && *datap <= 0x66)) { /* a-f */
         if(ch->hexindex < MAXNUM_SIZE) {
           ch->hexbuffer[ch->hexindex] = *datap;
           datap++;
@@ -218,39 +218,39 @@ CHUNKcode Curl_httpchunk_read(struct connectdata *conn,
 #ifdef HAVE_LIBZ
       switch (conn->data->set.http_ce_skip?
               IDENTITY : data->reqdata.keep.content_encoding) {
-        case IDENTITY:
+      case IDENTITY:
 #endif
-          if(!k->ignorebody) {
-            if ( !data->set.http_te_skip )
-              result = Curl_client_write(conn, CLIENTWRITE_BODY, datap,
-                                         piece);
-            else
-              result = CURLE_OK;
-          }
+        if(!k->ignorebody) {
+          if ( !data->set.http_te_skip )
+            result = Curl_client_write(conn, CLIENTWRITE_BODY, datap,
+                                       piece);
+          else
+            result = CURLE_OK;
+        }
 #ifdef HAVE_LIBZ
-          break;
+        break;
 
-        case DEFLATE:
-          /* update data->reqdata.keep.str to point to the chunk data. */
-          data->reqdata.keep.str = datap;
-          result = Curl_unencode_deflate_write(conn, &data->reqdata.keep,
-                                               (ssize_t)piece);
-          break;
+      case DEFLATE:
+        /* update data->reqdata.keep.str to point to the chunk data. */
+        data->reqdata.keep.str = datap;
+        result = Curl_unencode_deflate_write(conn, &data->reqdata.keep,
+                                             (ssize_t)piece);
+        break;
 
-        case GZIP:
-          /* update data->reqdata.keep.str to point to the chunk data. */
-          data->reqdata.keep.str = datap;
-          result = Curl_unencode_gzip_write(conn, &data->reqdata.keep,
-                                            (ssize_t)piece);
-          break;
+      case GZIP:
+        /* update data->reqdata.keep.str to point to the chunk data. */
+        data->reqdata.keep.str = datap;
+        result = Curl_unencode_gzip_write(conn, &data->reqdata.keep,
+                                          (ssize_t)piece);
+        break;
 
-        case COMPRESS:
-        default:
-          failf (conn->data,
-                 "Unrecognized content encoding type. "
-                 "libcurl understands `identity', `deflate' and `gzip' "
-                 "content encodings.");
-          return CHUNKE_BAD_ENCODING;
+      case COMPRESS:
+      default:
+        failf (conn->data,
+               "Unrecognized content encoding type. "
+               "libcurl understands `identity', `deflate' and `gzip' "
+               "content encodings.");
+        return CHUNKE_BAD_ENCODING;
       }
 #endif
 
@@ -319,7 +319,7 @@ CHUNKcode Curl_httpchunk_read(struct connectdata *conn,
       else {
         datap++;
         length--;
-     }
+      }
       break;
 
     case CHUNK_TRAILER_CR:
@@ -403,7 +403,6 @@ CHUNKcode Curl_httpchunk_read(struct connectdata *conn,
         return CHUNKE_BAD_CHUNK;
       }
 
-
     default:
       return CHUNKE_STATE_ERROR;
     }

lib/http_negotiate.c

@@ -49,7 +49,7 @@
 #include "memdebug.h"
 
 static int
-get_gss_name(struct connectdata *conn, gss_name_t *server)
+get_gss_name(struct connectdata *conn, bool proxy, gss_name_t *server)
 {
   struct negotiatedata *neg_ctx = &conn->data->state.negotiate;
   OM_uint32 major_status, minor_status;
@@ -69,11 +69,11 @@ get_gss_name(struct connectdata *conn, gss_name_t *server)
   else
     service = "HTTP";
 
-  token.length = strlen(service) + 1 + strlen(conn->host.name) + 1;
+  token.length = strlen(service) + 1 + strlen(proxy ? conn->proxy.name : conn->host.name) + 1;
   if (token.length + 1 > sizeof(name))
     return EMSGSIZE;
 
-  snprintf(name, sizeof(name), "%s@%s", service, conn->host.name);
+  snprintf(name, sizeof(name), "%s@%s", service, proxy ? conn->proxy.name : conn->host.name);
 
   token.value = (void *) name;
   major_status = gss_import_name(&minor_status,
@@ -113,7 +113,7 @@ log_gss_error(struct connectdata *conn, OM_uint32 error_status, char *prefix)
   infof(conn->data, "%s", buf);
 }
 
-int Curl_input_negotiate(struct connectdata *conn, const char *header)
+int Curl_input_negotiate(struct connectdata *conn, bool proxy, const char *header)
 {
   struct negotiatedata *neg_ctx = &conn->data->state.negotiate;
   OM_uint32 major_status, minor_status, minor_status2;
@@ -156,7 +156,7 @@ int Curl_input_negotiate(struct connectdata *conn, const char *header)
   }
 
   if (neg_ctx->server_name == NULL &&
-      (ret = get_gss_name(conn, &neg_ctx->server_name)))
+      (ret = get_gss_name(conn, proxy, &neg_ctx->server_name)))
     return ret;
 
   header += strlen(neg_ctx->protocol);
@@ -245,7 +245,7 @@ int Curl_input_negotiate(struct connectdata *conn, const char *header)
 }
 
 
-CURLcode Curl_output_negotiate(struct connectdata *conn)
+CURLcode Curl_output_negotiate(struct connectdata *conn, bool proxy)
 {
   struct negotiatedata *neg_ctx = &conn->data->state.negotiate;
   OM_uint32 minor_status;
@@ -299,7 +299,7 @@ CURLcode Curl_output_negotiate(struct connectdata *conn)
     return CURLE_OUT_OF_MEMORY;
 
   conn->allocptr.userpwd =
-    aprintf("Authorization: %s %s\r\n", neg_ctx->protocol, encoded);
+    aprintf("%sAuthorization: %s %s\r\n", proxy ? "Proxy-" : "", neg_ctx->protocol, encoded);
   free(encoded);
   gss_release_buffer(&minor_status, &neg_ctx->output_token);
   return (conn->allocptr.userpwd == NULL) ? CURLE_OUT_OF_MEMORY : CURLE_OK;

lib/http_negotiate.h

@@ -27,10 +27,10 @@
 #ifdef HAVE_GSSAPI
 
 /* this is for Negotiate header input */
-int Curl_input_negotiate(struct connectdata *conn, const char *header);
+int Curl_input_negotiate(struct connectdata *conn, bool proxy, const char *header);
 
 /* this is for creating Negotiate header output */
-CURLcode Curl_output_negotiate(struct connectdata *conn);
+CURLcode Curl_output_negotiate(struct connectdata *conn, bool proxy);
 
 void Curl_cleanup_negotiate(struct SessionHandle *data);
 

lib/http_ntlm.c

@@ -443,11 +443,11 @@ static CURLcode mk_nt_hash(struct SessionHandle *data,
 
   {
     /* Create NT hashed password. */
-    MD4_CTX MD4;
+    MD4_CTX MD4pw;
 
-    MD4_Init(&MD4);
-    MD4_Update(&MD4, pw, 2*len);
-    MD4_Final(ntbuffer, &MD4);
+    MD4_Init(&MD4pw);
+    MD4_Update(&MD4pw, pw, 2*len);
+    MD4_Final(ntbuffer, &MD4pw);
 
     memset(ntbuffer + 16, 0, 21 - 16);
   }
@@ -857,25 +857,25 @@ CURLcode Curl_output_ntlm(struct connectdata *conn,
       unsigned char ntbuffer[0x18];
       unsigned char tmp[0x18];
       unsigned char md5sum[MD5_DIGEST_LENGTH];
-      MD5_CTX MD5;
-      unsigned char random[8];
+      MD5_CTX MD5pw;
+      unsigned char entropy[8];
 
       /* Need to create 8 bytes random data */
       Curl_ossl_seed(conn->data); /* Initiate the seed if not already done */
-      RAND_bytes(random,8);
+      RAND_bytes(entropy,8);
 
       /* 8 bytes random data as challenge in lmresp */
-      memcpy(lmresp,random,8);
+      memcpy(lmresp,entropy,8);
       /* Pad with zeros */
       memset(lmresp+8,0,0x10);
 
-      /* Fill tmp with challenge(nonce?) + random */
+      /* Fill tmp with challenge(nonce?) + entropy */
       memcpy(tmp,&ntlm->nonce[0],8);
-      memcpy(tmp+8,random,8);
+      memcpy(tmp+8,entropy,8);
 
-      MD5_Init(&MD5);
-      MD5_Update(&MD5, tmp, 16);
-      MD5_Final(md5sum, &MD5);
+      MD5_Init(&MD5pw);
+      MD5_Update(&MD5pw, tmp, 16);
+      MD5_Final(md5sum, &MD5pw);
       /* We shall only use the first 8 bytes of md5sum,
          but the des code in lm_resp only encrypt the first 8 bytes */
       if (mk_nt_hash(conn->data, passwdp, ntbuffer) == CURLE_OUT_OF_MEMORY)

lib/ldap.c

@@ -45,8 +45,16 @@
 
 #ifdef CURL_LDAP_WIN            /* Use W$ LDAP implementation. */
 # include <winldap.h>
+# ifndef LDAP_VENDOR_NAME
+#  error Your Platform SDK is NOT sufficient for LDAP support! Update your Platform SDK, or disable LDAP LDAP support!
+# else
+#  include <winber.h>
+# endif
 #else
 #define LDAP_DEPRECATED 1       /* Be sure ldap_init() is defined. */
+#ifdef HAVE_LBER_H
+# include <lber.h>
+#endif
 # include <ldap.h>
 #if (defined(HAVE_LDAP_SSL) && defined(HAVE_LDAP_SSL_H))
 # include <ldap_ssl.h>
@@ -110,7 +118,52 @@ static void _ldap_free_urldesc (LDAPURLDesc *ludp);
 #endif
 
 
-CURLcode Curl_ldap(struct connectdata *conn, bool *done)
+static CURLcode Curl_ldap(struct connectdata *conn, bool *done);
+
+/*
+ * LDAP protocol handler.
+ */
+
+const struct Curl_handler Curl_handler_ldap = {
+  "LDAP",                               /* scheme */
+  ZERO_NULL,                            /* setup_connection */
+  Curl_ldap,                            /* do_it */
+  ZERO_NULL,                            /* done */
+  ZERO_NULL,                            /* do_more */
+  ZERO_NULL,                            /* connect_it */
+  ZERO_NULL,                            /* connecting */
+  ZERO_NULL,                            /* doing */
+  ZERO_NULL,                            /* proto_getsock */
+  ZERO_NULL,                            /* doing_getsock */
+  ZERO_NULL,                            /* disconnect */
+  PORT_LDAP,                            /* defport */
+  PROT_LDAP                             /* protocol */
+};
+
+#ifdef HAVE_LDAP_SSL
+/*
+ * LDAPS protocol handler.
+ */
+
+const struct Curl_handler Curl_handler_ldaps = {
+  "LDAPS",                              /* scheme */
+  ZERO_NULL,                            /* setup_connection */
+  Curl_ldap,                            /* do_it */
+  ZERO_NULL,                            /* done */
+  ZERO_NULL,                            /* do_more */
+  ZERO_NULL,                            /* connect_it */
+  ZERO_NULL,                            /* connecting */
+  ZERO_NULL,                            /* doing */
+  ZERO_NULL,                            /* proto_getsock */
+  ZERO_NULL,                            /* doing_getsock */
+  ZERO_NULL,                            /* disconnect */
+  PORT_LDAPS,                           /* defport */
+  PROT_LDAP | PROT_SSL                  /* protocol */
+};
+#endif
+
+
+static CURLcode Curl_ldap(struct connectdata *conn, bool *done)
 {
   CURLcode status = CURLE_OK;
   int rc = 0;
@@ -120,7 +173,7 @@ CURLcode Curl_ldap(struct connectdata *conn, bool *done)
   LDAPMessage *entryIterator;
   int num = 0;
   struct SessionHandle *data=conn->data;
-  int ldap_proto;
+  int ldap_proto = LDAP_VERSION3;
   int ldap_ssl = 0;
   char *val_b64;
   size_t val_b64_sz;
@@ -153,7 +206,6 @@ CURLcode Curl_ldap(struct connectdata *conn, bool *done)
 #ifdef LDAP_OPT_NETWORK_TIMEOUT
   ldap_set_option(NULL, LDAP_OPT_NETWORK_TIMEOUT, &ldap_timeout);
 #endif
-  ldap_proto = LDAP_VERSION3;
   ldap_set_option(NULL, LDAP_OPT_PROTOCOL_VERSION, &ldap_proto);
 
   if (ldap_ssl) {
@@ -289,6 +341,9 @@ CURLcode Curl_ldap(struct connectdata *conn, bool *done)
       goto quit;
     }
   }
+#ifdef CURL_LDAP_WIN
+  ldap_set_option(server, LDAP_OPT_PROTOCOL_VERSION, &ldap_proto);
+#endif
 
   rc = ldap_simple_bind_s(server,
                           conn->bits.user_passwd ? conn->user : NULL,

lib/msvcproj.head

@@ -54,7 +54,7 @@ BSC32=bscmake.exe
 # ADD BSC32 /nologo
 LINK32=link.exe
 # ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /dll /machine:I386
-# ADD LINK32 kernel32.lib ws2_32.lib winmm.lib /nologo /dll /map /debug /machine:I386 /out:"Release/libcurl.dll"
+# ADD LINK32 kernel32.lib ws2_32.lib wldap32.lib winmm.lib /nologo /dll /map /debug /machine:I386 /out:"Release/libcurl.dll"
 
 !ELSEIF  "$(CFG)" == "curllib - Win32 Debug"
 
@@ -81,7 +81,7 @@ BSC32=bscmake.exe
 # ADD BSC32 /nologo
 LINK32=link.exe
 # ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /dll /debug /machine:I386 /pdbtype:sept
-# ADD LINK32 kernel32.lib ws2_32.lib winmm.lib /nologo /dll /incremental:no /map /debug /machine:I386 /out:"Debug/libcurl.dll" /pdbtype:sept
+# ADD LINK32 kernel32.lib ws2_32.lib wldap32.lib winmm.lib /nologo /dll /incremental:no /map /debug /machine:I386 /out:"Debug/libcurl.dll" /pdbtype:sept
 # SUBTRACT LINK32 /nodefaultlib
 
 !ENDIF