1.3.1

library, so that the same application code can use mutex-locking
independently of OpenSSL or GnutTLS being used.

CHANGES

@@ -6,6 +6,30 @@
 
                                   Changelog
 
+Daniel (23 June 2006)
+- Arve Knudsen found a flaw in curl_multi_fdset() for systems where
+  curl_socket_t is unsigned (like Windows) that could cause it to wrongly
+  return a max fd of -1.
+
+Daniel (20 June 2006)
+- Peter Silva introduced CURLOPT_MAX_SEND_SPEED_LARGE and
+  CURLOPT_MAX_RECV_SPEED_LARGE that limit tha maximum rate libcurl is allowed
+  to send or receive data. This kind of adds the the command line tool's
+  option --limit-rate to the library.
+
+  The rate limiting logic in the curl app is now removed and is instead
+  provided by libcurl itself. Transfer rate limiting will now also work for -d
+  and -F, which it didn't before.
+
+Daniel (19 June 2006)
+- Made -K on a file that couldn't be read cause a warning to be displayed.
+
+Daniel (13 June 2006)
+- Dan Fandrich implemented --enable-hidden-symbols configure option to enable
+  -fvisibility=hidden on gcc >= 4.0.  This reduces the size of the libcurl
+  binary and speeds up dynamic linking by hiding all the internal symbols from
+  the symbol table.
+
 Version 7.15.4 (12 June 2006)
 
 Daniel (8 June 2006)

RELEASE-NOTES

@@ -1,72 +1,36 @@
-Curl and libcurl 7.15.4
+Curl and libcurl 7.15.5
 
- Public curl release number:               94
- Releases counted from the very beginning: 121
+ Public curl release number:               95
+ Releases counted from the very beginning: 122
  Available command line options:           112
  Available curl_easy_setopt() options:     132
  Number of public functions in libcurl:    49
  Amount of public web site mirrors:        33
  Number of known libcurl bindings:         32
- Number of contributors:                   492
+ Number of contributors:                   506
 
 This release includes the following changes:
 
- o NTLM2 session response support
- o CURLOPT_COOKIELIST set to "SESS" clears all session cookies
- o CURLINFO_LASTSOCKET returned sockets are now checked more before returned
- o curl-config got a --checkfor option to compare version numbers
- o line end conversions for FTP ASCII transfers
- o curl_multi_socket() API added (still mostly untested)
- o conversion callback options for EBCDIC <=> ASCII conversions
- o added CURLINFO_FTP_ENTRY_PATH
- o less blocking for the multi interface during (Open)SSL connect negotiation
+ o added CURLOPT_MAX_SEND_SPEED_LARGE and CURLOPT_MAX_RECV_SPEED_LARGE
+ o configure --enable-hidden-symbols
 
 This release includes the following bugfixes:
 
- o builds fine on cygwin
- o md5-sess with Digest authentication
- o dict with letters such as space in a word
- o dict with url-encoded words in the URL
- o libcurl.m4 when default=yes but no libcurl was found
- o numerous bugs fixed in the TFTP code
- o possible memory leak when adding easy handles to multi stack
- o TFTP works in a more portable fashion (== on more platforms)
- o WSAGetLastError() is now used (better) on Windows
- o GnuTLS non-block case that could cause data trashing
- o deflate code survives lack of zlib header
- o CURLOPT_INTERFACE works with hostname
- o configure runs fine with ICC
- o closed control connection with FTP when easy handle was removed from multi
- o curl --trace crash when built with VS2005
- o SSL connect time-out
- o improved NTLM functionality
- o following redirects with more than one question mark in source URL
- o fixed debug build crash with -d
- o generates a fine AIX Toolbox RPM spec
- o treat FTP AUTH failures properly
- o TFTP transfers could trash data
- o -d + -G combo crash
+ o curl_multi_fdset() could return a crazy max_fd value
+ o Made -K on a file that couldn't be read cause a warning to be displayed.
 
 Other curl-related news:
 
- o tclcurl 0.15.3 was released:
-   http://personal1.iddeo.es/andresgarci/tclcurl/english/
+ o cURLpp 0.6.0 was released: http://rrette.com/curlpp.html
+ o pycurl-7.15.4 was released: http://pycurl.sf.net
 
 New curl mirrors:
 
- o http://curl.webdesign-zdg.de/ in Frankfurt, Germany
- o http://curl.de-mirror.de/ in Aachen, Germany
- o http://curl.osmirror.nl/ in Amsterdam, the Netherlands
- o http://curl.usphp.com/ in Florida, US
- o http://curl.oslevel.de/ in Karlsruhe, Germany
+ o 
 
 This release would not have looked like this without help, code, reports and
 advice from friends like these:
 
- Dan Fandrich, Ilja van Sprundel, David McCreedy, Tor Arntsen, Xavier Bouchoux,
- David Byron, Michele Bini, Ates Goral, Katie Wang, Robson Braga Araujo,
- Ale Vesely, Paul Querna, Gisle Vanem, Mark Eichin, Roland Blom, Andreas
- Ntaflos, David Shaw, Michael Wallner, Olaf St<53>ben, Mikael Sennerholm,
- Brian Dessent
+ Dan Fandrich, Peter Silva, Arve Knudsen
 
         Thanks! (and sorry if I forgot to mention someone)

ares/CHANGES

@@ -1,5 +1,23 @@
   Changelog for the c-ares project
 
+Version 1.3.1 (June 24, 2006)
+
+* June 19, 2006
+
+- (wahern) Removed "big endian" DNS section and RR data integer parser
+  macros from ares_dns.h, which break c-ares on my Sparc64. Bit-wise
+  operations in C operate on logical values. And in any event the octets are
+  already in big-endian (aka network) byte order so they're being reversed
+  (thus the source of the breakage).
+
+* June 18, 2006
+
+- William Ahern handles EAGAIN/EWOULDBLOCK errors in most of the I/O calls
+  from area_process.c.
+
+  TODO: Handle one last EAGAIN for a UDP socket send(2) in
+  ares__send_query().
+
 * May 10, 2006
 
 - Bram Matthys brought my attention to a libtool peculiarity where detecting
@@ -49,7 +67,7 @@
 - configure fix for detecting a member in the sockaddr_in6 struct which failed
   on ipv6-enabled HP-UX 11.00
 
-Version 1.3.0 (August 29, 2004)
+Version 1.3.0 (August 29, 2005)
 
 * August 21
 

ares/ares_dns.h

@@ -18,18 +18,6 @@
 #ifndef ARES__DNS_H
 #define ARES__DNS_H
 
-#ifdef ARES_BIG_ENDIAN
-/* big-endian aware versions */
-#define DNS__16BIT(p)                   (((p)[1] << 8) | (p)[0])
-#define DNS__32BIT(p)                   (((p)[3] << 24) | ((p)[2] << 16) | \
-                                         ((p)[1] << 8) | (p)[0])
-#define DNS__SET16BIT(p, v)             (((p)[1] = ((v) >> 8) & 0xff), \
-                                         ((p)[0] = (v) & 0xff))
-#define DNS__SET32BIT(p, v)             (((p)[3] = ((v) >> 24) & 0xff), \
-                                         ((p)[2] = ((v) >> 16) & 0xff), \
-                                         ((p)[1] = ((v) >> 8) & 0xff), \
-                                         ((p)[0] = (v) & 0xff))
-#else
 #define DNS__16BIT(p)                   (((p)[0] << 8) | (p)[1])
 #define DNS__32BIT(p)                   (((p)[0] << 24) | ((p)[1] << 16) | \
                                          ((p)[2] << 8) | (p)[3])
@@ -39,7 +27,6 @@
                                          ((p)[1] = ((v) >> 16) & 0xff), \
                                          ((p)[2] = ((v) >> 8) & 0xff), \
                                          ((p)[3] = (v) & 0xff))
-#endif
 
 #if 0
 /* we cannot use this approach on systems where we can't access 16/32 bit

ares/ares_process.c

@@ -63,6 +63,7 @@
 #define GET_ERRNO()  errno
 #endif
 
+static int try_again(int errnum);
 static void write_tcp_data(ares_channel channel, fd_set *write_fds,
                            time_t now);
 static void read_tcp_data(ares_channel channel, fd_set *read_fds, time_t now);
@@ -94,6 +95,31 @@ void ares_process(ares_channel channel, fd_set *read_fds, fd_set *write_fds)
   process_timeouts(channel, now);
 }
 
+/* Return 1 if the specified errno describes a readiness error, or 0
+ * otherwise. This is mostly for HP-UX, which could return EAGAIN or
+ * EWOULDBLOCK. See this man page
+ *
+ * 	http://devrsrc1.external.hp.com/STKS/cgi-bin/man2html?manpage=/usr/share/man/man2.Z/send.2
+ */
+static int try_again(int errnum)
+{
+#if !defined EWOULDBLOCK && !defined EAGAIN
+#error "Neither EWOULDBLOCK nor EAGAIN defined"
+#endif
+  switch (errnum)
+    {
+#ifdef EWOULDBLOCK
+    case EWOULDBLOCK:
+      return 1;
+#endif
+#if defined EAGAIN && EAGAIN != EWOULDBLOCK
+    case EAGAIN:
+      return 1;
+#endif
+    }
+  return 0;
+}
+
 /* If any TCP sockets select true for writing, write out queued data
  * we have for them.
  */
@@ -136,7 +162,8 @@ static void write_tcp_data(ares_channel channel, fd_set *write_fds, time_t now)
           free(vec);
           if (wcount < 0)
             {
-              handle_error(channel, i, now);
+              if (!try_again(GET_ERRNO()))
+                  handle_error(channel, i, now);
               continue;
             }
 
@@ -173,7 +200,8 @@ static void write_tcp_data(ares_channel channel, fd_set *write_fds, time_t now)
 
           if (scount < 0)
             {
-              handle_error(channel, i, now);
+              if (!try_again(GET_ERRNO()))
+                  handle_error(channel, i, now);
               continue;
             }
 
@@ -224,7 +252,8 @@ static void read_tcp_data(ares_channel channel, fd_set *read_fds, time_t now)
                        2 - server->tcp_lenbuf_pos, 0);
           if (count <= 0)
             {
-              handle_error(channel, i, now);
+              if (!(count == -1 && try_again(GET_ERRNO())))
+                  handle_error(channel, i, now);
               continue;
             }
 
@@ -250,7 +279,8 @@ static void read_tcp_data(ares_channel channel, fd_set *read_fds, time_t now)
                        server->tcp_length - server->tcp_buffer_pos, 0);
           if (count <= 0)
             {
-              handle_error(channel, i, now);
+              if (!(count == -1 && try_again(GET_ERRNO())))
+                  handle_error(channel, i, now);
               continue;
             }
 
@@ -289,7 +319,9 @@ static void read_udp_packets(ares_channel channel, fd_set *read_fds,
         continue;
 
       count = recv(server->udp_socket, (void *)buf, sizeof(buf), 0);
-      if (count <= 0)
+      if (count == -1 && try_again(GET_ERRNO()))
+        continue;
+      else if (count <= 0)
         handle_error(channel, i, now);
 
       process_answer(channel, buf, count, i, 0, now);
@@ -479,6 +511,7 @@ void ares__send_query(ares_channel channel, struct query *query, time_t now)
       if (send(server->udp_socket, (void *)query->qbuf,
                query->qlen, 0) == -1)
         {
+          /* FIXME: Handle EAGAIN here since it likely can happen. */
           query->skip_server[query->server] = 1;
           next_server(channel, query, now);
           return;

configure.ac

@@ -1954,6 +1954,39 @@ AC_HELP_STRING([--disable-cookies],[Disable cookies support]),
        AC_MSG_RESULT(yes)
 )
 
+dnl ************************************************************
+dnl Enable hiding of internal symbols in library to reduce its size and
+dnl speed dynamic linking of applications.  This currently is only supported
+dnl on gcc >= 4.0
+dnl
+AC_MSG_CHECKING([whether to enable hidden symbols in the library])
+AC_ARG_ENABLE(hidden-symbols,
+AC_HELP_STRING([--enable-hidden-symbols],[Hide internal symbols in library (gcc>=4)])
+AC_HELP_STRING([--disable-hidden-symbols],[Leave all symbols with default visibility in library]),
+[ case "$enableval" in
+  no)
+       AC_MSG_RESULT(no)
+       ;;
+  *)   
+       if test "$GCC" = yes ; then
+         AC_MSG_CHECKING([whether $CC supports it])
+         if $CC --help --verbose 2>&1 | grep fvisibility= > /dev/null ; then
+	   AC_MSG_RESULT(yes)
+	   AC_DEFINE(CURL_HIDDEN_SYMBOLS, 1, [to enable hidden symbols])
+	   AC_SUBST(CURL_HIDDEN_SYMBOLS)
+	   CFLAGS="$CFLAGS -fvisibility=hidden"
+         else
+            AC_MSG_RESULT(no)
+          fi
+       else
+          AC_MSG_RESULT([no (not gcc)])
+       fi
+       ;;
+  esac ],
+       AC_MSG_RESULT(no)
+)
+
+dnl ************************************************************
 if test "x$ws2" = "xyes"; then
 
   dnl If ws2_32 is wanted, make sure it is the _last_ lib in LIBS (makes

docs/BINDINGS

@@ -47,7 +47,7 @@ Cocoa
 D
 
   Written by Charles Sanders and James Wavro
-  http://www.atari-soldiers.com/libcurl.html
+  http://www.atari-soldiers.com/libcurl.html (DEAD LINK)
 
 Dylan
 

docs/INSTALL

@@ -506,13 +506,17 @@ CROSS COMPILE
 
        ./configure --host=ARCH-OS
 
+REDUCING SIZE
+=============
+
    There are a number of configure options that can be used to reduce the
    size of libcurl for embedded applications where binary size is an
-   important factor.  First, be sure to set the CFLAGS environment variable
-   when configuring with any compiler optimization flags to reduce the
-   size of the binary.  For gcc, this would mean at minimum:
+   important factor.  First, be sure to set the CFLAGS variable when
+   configuring with any relevant compiler optimization flags to reduce the
+   size of the binary.  For gcc, this would mean at minimum the -Os option
+   and probably the -march=X option as well, e.g.:
 
-      env CFLAGS='-Os' ./configure ...
+      ./configure CFLAGS='-Os' ...
 
    Be sure to specify as many --disable- and --without- flags on the configure
    command-line as you can to disable all the libcurl features that you
@@ -526,10 +530,24 @@ CROSS COMPILE
      --disable-crypto-auth (disables HTTP cryptographic authentication)
      --disable-ipv6 (disables support for IPv6)
      --disable-verbose (eliminates debugging strings and error code strings)
+     --enable-hidden-symbols (eliminates unneeded symbols in library)
      --without-libidn (disables support for the libidn DNS library)
      --without-ssl (disables support for SSL/TLS)
      --without-zlib (disables support for on-the-fly decompression)
 
+   The GNU linker has a number of options to reduce the size of the libcurl
+   dynamic libraries on some platforms even further. Specify them by giving
+   the options -Wl,-Bsymbolic and -Wl,-s on the gcc command-line.  
+   Be sure also to strip debugging symbols from your binaries after
+   compiling using 'strip' (or the appropriate variant if cross-compiling).
+   If space is really tight, you may be able to remove some unneeded
+   sections of the library using the -R option to objcopy (e.g. the
+   .comment section).
+
+   Using these techniques it is possible to create an HTTP-only shared
+   libcurl library for i386 Linux platforms that is less than 90 KB in
+   size (as of version 7.15.4).
+
    You may find that statically linking libcurl to your application will
    result in a lower total size.
 

docs/KNOWN_BUGS

@@ -103,10 +103,6 @@ may have been fixed since this was written!
   http://curl.haxx.se/bug/view.cgi?id=1004841. How?
   http://curl.haxx.se/mail/lib-2004-08/0182.html
 
-9. --limit-rate using -d or -F does not work. This is because the limit logic
-  is provided by the curl app in its read/write callbacks, and when doing
-  -d/-F the callbacks aren't used! http://curl.haxx.se/bug/view.cgi?id=921395
-
 8. Doing resumed upload over HTTP does not work with '-C -', because curl
   doesn't do a HEAD first to get the initial size. This needs to be done
   manually for HTTP PUT resume to work, and then '-C [index]'.

docs/THANKS

@@ -8,6 +8,7 @@ Adrian Schuur
 Alan Pinstein
 Albert Chin-A-Young
 Albert Choy
+Ale Vesely
 Aleksandar Milivojevic
 Alex Neblett
 Alex Suykov
@@ -20,6 +21,7 @@ Alexis Carvalho
 Amol Pattekar
 Andi Jahja
 Andreas Damm
+Andreas Ntaflos
 Andreas Olsson
 Andreas Rieke
 Andres Garcia
@@ -34,6 +36,7 @@ Angus Mackay
 Antoine Calando
 Anton Kalmykov
 Arkadiusz Miskiewicz
+Ates Goral
 Augustus Saunders
 Avery Fay
 Ben Greear
@@ -47,6 +50,7 @@ Brad Burdick
 Bradford Bruce
 Brent Beardsley
 Brian Akins
+Brian Dessent
 Brian R Duffy
 Bruce Mitchener
 Bryan Henderson
@@ -189,6 +193,7 @@ Ian Wilkes
 Ignacio Vazquez-Abrams
 Igor Polyakov
 Ilguiz Latypov
+Ilja van Sprundel
 Ingo Ralf Blum
 Ingo Wilken
 Jacky Lam
@@ -249,6 +254,7 @@ Kai-Uwe Rommel
 Kang-Jin Lee
 Karl Moerder
 Karol Pietrzak
+Katie Wang
 Keith MacDonald
 Keith McGuigan
 Ken Hirsch
@@ -294,6 +300,7 @@ Marco G. Salvagno
 Marcus Webster
 Mario Schroeder
 Mark Butler
+Mark Eichin
 Markus Koetter
 Markus Moeller
 Markus Oberhumer
@@ -318,7 +325,9 @@ Michael Mealling
 Michael Wallner
 Michal Bonino
 Michal Marek
+Michele Bini
 Mihai Ionescu
+Mikael Sennerholm
 Mike Bytnar
 Mike Dobbs
 Miklos Nemeth
@@ -343,6 +352,7 @@ Nis Jorgensen
 Nodak Sodak
 Norbert Novotny
 Ofer
+Olaf St<53>ben
 Oren Tirosh
 P R Schaffner
 Patrick Bihan-Faou
@@ -351,6 +361,7 @@ Paul Harrington
 Paul Marquis
 Paul Moore
 Paul Nolan
+Paul Querna
 Pavel Cenek
 Pavel Orehov
 Pawel A. Gajda
@@ -400,7 +411,9 @@ Robert D. Young
 Robert Olson
 Robert Weaver
 Robin Kay
+Robson Braga Araujo
 Rodney Simmons
+Roland Blom
 Roland Krikava
 Roland Zimmermann
 Roman Koifman
@@ -490,6 +503,7 @@ Wesley Laxton
 Wez Furlong
 Wilfredo Sanchez
 Wojciech Zwiefka
+Xavier Bouchoux
 Yang Tse
 Yarram Sunil
 Zvi Har'El 

docs/TODO

@@ -135,6 +135,10 @@ TODO
 
  SSL
 
+ * Provide a libcurl API for setting mutex callbacks in the underlying SSL
+   library, so that the same application code can use mutex-locking
+   independently of OpenSSL or GnutTLS being used.
+
  * Anton Fedorov's "dumpcert" patch:
    http://curl.haxx.se/mail/lib-2004-03/0088.html
 

docs/curl.1

@@ -1240,7 +1240,7 @@ If this option is used twice, the second will again disable the progress bar.
 .SH FILES
 .I ~/.curlrc
 .RS
-Default config file.
+Default config file, see \fI-K/--config\fP for details.
 
 .SH ENVIRONMENT
 .IP "http_proxy [protocol://]<host>[:port]"

docs/libcurl/curl_easy_setopt.3

@@ -1051,6 +1051,14 @@ for the library to consider it too slow and abort.
 Pass a long as parameter. It contains the time in seconds that the transfer
 should be below the \fICURLOPT_LOW_SPEED_LIMIT\fP for the library to consider
 it too slow and abort.
+.IP CURLOPT_MAX_SEND_SPEED_LARGE
+Pass a curl_off_t as parameter.  If an upload exceeds this speed on cumulative 
+average during the transfer, the transfer will pause to keep the average rate 
+less than or equal to the parameter value.  (default: 0, unlimited)
+.IP CURLOPT_MAX_RECV_SPEED_LARGE
+Pass a curl_off_t as parameter.  If an upload exceeds this speed on cumulative 
+average during the transfer, the transfer will pause to keep the average rate 
+less than or equal to the parameter value.  (default: 0, unlimited)
 .IP CURLOPT_MAXCONNECTS
 Pass a long. The set number will be the persistent connection cache size. The
 set amount will be the maximum amount of simultaneously open connections that
@@ -1111,7 +1119,7 @@ Resolve to ipv4 addresses.
 .IP CURL_IPRESOLVE_V6
 Resolve to ipv6 addresses.
 .RE
-.SH CURLOPT_CONNECT_ONLY
+.IP CURLOPT_CONNECT_ONLY
 Pass a long. A non-zero parameter tells the library to perform any required
 proxy authentication and connection setup, but no data transfer.
 

docs/libcurl/libcurl-tutorial.3

@@ -212,7 +212,7 @@ opened for writing with the \fICURLOPT_WRITEDATA\fP option.
 Now, we need to take a step back and have a deep breath. Here's one of those
 rare platform-dependent nitpicks. Did you spot it? On some platforms[2],
 libcurl won't be able to operate on files opened by the program. Thus, if you
-use the default callback and pass in a an open file with
+use the default callback and pass in an open file with
 \fICURLOPT_WRITEDATA\fP, it will crash. You should therefore avoid this to
 make your program run fine virtually everywhere.
 

docs/libcurl/libcurl.3

@@ -10,8 +10,8 @@ specific man pages for each function mentioned in here. There are also the
 \fIlibcurl-share(3)\fP man page and the \fIlibcurl-tutorial(3)\fP man page for
 in-depth understanding on how to program with libcurl.
 
-There are more than a twenty custom bindings available that bring libcurl
-access to your favourite language. Look elsewhere for documentation on those.
+There are more than thirty custom bindings available that bring libcurl access
+to your favourite language. Look elsewhere for documentation on those.
 
 libcurl has a global constant environment that you must set up and
 maintain while using libcurl.  This essentially means you call

include/curl/curl.h

@@ -58,8 +58,18 @@ extern "C" {
 #define CURL_EXTERN  __declspec(dllimport)
 #endif
 #else
+
+#ifdef CURL_HIDDEN_SYMBOLS
+/*
+ * On gcc >= 4 if -fvisibility=hidden is given then this is used to cause
+ * external definitions to be put into the shared library.  It makes no
+ * difference to applications whether this is set or not, only the library.
+ */
+#define CURL_EXTERN __attribute__ ((visibility ("default")))
+#else
 #define CURL_EXTERN 
 #endif
+#endif
 
 /*
  * We want the typedef curl_off_t setup for large file support on all
@@ -963,6 +973,11 @@ typedef enum {
      Note that this is used only for SSL certificate processing */
   CINIT(CONV_FROM_UTF8_FUNCTION, FUNCTIONPOINT, 144),
 
+  /* if the connection proceeds too quickly then need to slow it down */
+  /* limit-rate: maximum number of bytes per second to send or receive */
+  CINIT(MAX_SEND_SPEED_LARGE, OFF_T, 145),
+  CINIT(MAX_RECV_SPEED_LARGE, OFF_T, 146),
+
   CURLOPT_LASTENTRY /* the last unused */
 } CURLoption;
 

include/curl/curlver.h

@@ -28,13 +28,13 @@
 
 /* This is the version number of the libcurl package from which this header
    file origins: */
-#define LIBCURL_VERSION "7.15.4-CVS"
+#define LIBCURL_VERSION "7.15.5-CVS"
 
 /* The numeric version number is also available "in parts" by using these
    defines: */
 #define LIBCURL_VERSION_MAJOR 7
 #define LIBCURL_VERSION_MINOR 15
-#define LIBCURL_VERSION_PATCH 4
+#define LIBCURL_VERSION_PATCH 5
 
 /* This is the numeric version of the libcurl version number, meant for easier
    parsing and comparions by programs. The LIBCURL_VERSION_NUM define will
@@ -51,6 +51,6 @@
    and it is always a greater number in a more recent release. It makes
    comparisons with greater than and less than work.
 */
-#define LIBCURL_VERSION_NUM 0x070f04
+#define LIBCURL_VERSION_NUM 0x070f05
 
 #endif /* __CURL_CURLVER_H */

lib/multi.c

@@ -68,6 +68,7 @@ typedef enum {
   CURLM_STATE_DOING,       /* sending off the request (part 1) */
   CURLM_STATE_DO_MORE,     /* send off the request (part 2) */
   CURLM_STATE_PERFORM,     /* transfer data */
+  CURLM_STATE_TOOFAST,     /* wait because limit-rate exceeded */
   CURLM_STATE_DONE,        /* post data transfer operation */
   CURLM_STATE_COMPLETED,   /* operation complete */
 
@@ -156,6 +157,7 @@ static void multistate(struct Curl_one_easy *easy, CURLMstate state)
     "DOING",
     "DO_MORE",
     "PERFORM",
+    "TOOFAST",
     "DONE",
     "COMPLETED",
   };
@@ -440,6 +442,7 @@ static int multi_getsock(struct Curl_one_easy *easy,
                          int numsocks)
 {
   switch(easy->state) {
+  case CURLM_STATE_TOOFAST:  /* returns 0, so will not select. */
   default:
     return 0;
 
@@ -501,7 +504,7 @@ CURLMcode curl_multi_fdset(CURLM *multi_handle,
         /* this socket is unused, break out of loop */
         break;
       else {
-        if(s > (curl_socket_t)this_max_fd)
+        if((int)s > this_max_fd)
           this_max_fd = (int)s;
       }
     }
@@ -771,7 +774,37 @@ static CURLMcode multi_runsingle(struct Curl_multi *multi,
       }
       break;
 
+    case CURLM_STATE_TOOFAST: /* limit-rate exceeded in either direction */
+      /* if both rates are within spec, resume transfer */
+      Curl_pgrsUpdate(easy->easy_conn);
+      if ( ( ( easy->easy_handle->set.max_send_speed == 0 ) ||
+             ( easy->easy_handle->progress.ulspeed <
+               easy->easy_handle->set.max_send_speed ) )  &&
+           ( ( easy->easy_handle->set.max_recv_speed == 0 ) ||
+             ( easy->easy_handle->progress.dlspeed <
+               easy->easy_handle->set.max_recv_speed ) )
+        )
+        multistate(easy, CURLM_STATE_PERFORM);
+
+      break;
+
     case CURLM_STATE_PERFORM:
+
+      /* check if over speed */
+      if ( (  ( easy->easy_handle->set.max_send_speed > 0 ) &&
+              ( easy->easy_handle->progress.ulspeed >
+                easy->easy_handle->set.max_send_speed ) )  ||
+           (  ( easy->easy_handle->set.max_recv_speed > 0 ) &&
+              ( easy->easy_handle->progress.dlspeed >
+                easy->easy_handle->set.max_recv_speed ) )
+        ) {
+        /* Transfer is over the speed limit. Change state.  TODO: Call
+         * Curl_expire() with the time left until we're targeted to be below
+         * the speed limit again. */
+        multistate(easy, CURLM_STATE_TOOFAST );
+        break;
+      }
+
       /* read/write data if it is ready to do so */
       easy->result = Curl_readwrite(easy->easy_conn, &done);
 
@@ -825,6 +858,7 @@ static CURLMcode multi_runsingle(struct Curl_multi *multi,
         }
       }
       break;
+
     case CURLM_STATE_DONE:
       /* post-transfer command */
       easy->result = Curl_done(&easy->easy_conn, CURLE_OK);

lib/transfer.c

@@ -285,7 +285,7 @@ CURLcode Curl_readwrite(struct connectdata *conn,
 
   curl_socket_t fd_read;
   curl_socket_t fd_write;
-  curl_socket_t select_res;
+  int select_res;
 
   curl_off_t contentlength;
 
@@ -1628,15 +1628,31 @@ Transfer(struct connectdata *conn)
 
     interval_ms = 1 * 1000;
 
-    if(k->keepon & KEEP_READ)
-      fd_read = conn->sockfd;
-    else
-      fd_read = CURL_SOCKET_BAD;
-
-    if(k->keepon & KEEP_WRITE)
-      fd_write = conn->writesockfd;
-    else
+    /* limit-rate logic: if speed exceeds threshold, then do not include fd in
+       select set */
+    if ( (conn->data->set.max_send_speed > 0) &&
+         (conn->data->progress.ulspeed > conn->data->set.max_send_speed) )  {
       fd_write = CURL_SOCKET_BAD;
+      Curl_pgrsUpdate(conn);
+    }
+    else {
+      if(k->keepon & KEEP_WRITE)
+        fd_write = conn->writesockfd;
+      else
+        fd_write = CURL_SOCKET_BAD;
+    }
+
+    if ( (conn->data->set.max_recv_speed > 0) &&
+         (conn->data->progress.dlspeed > conn->data->set.max_recv_speed) ) {
+      fd_read = CURL_SOCKET_BAD;
+      Curl_pgrsUpdate(conn);
+    }
+    else {
+      if(k->keepon & KEEP_READ)
+        fd_read = conn->sockfd;
+      else
+        fd_read = CURL_SOCKET_BAD;
+    }
 
     switch (Curl_select(fd_read, fd_write, interval_ms)) {
     case -1: /* select() error, stop reading */
@@ -1651,6 +1667,7 @@ Transfer(struct connectdata *conn)
       continue;
     case 0:  /* timeout */
     default: /* readable descriptors */
+
       result = Curl_readwrite(conn, &done);
       break;
     }

lib/url.c

@@ -1039,6 +1039,22 @@ CURLcode Curl_setopt(struct SessionHandle *data, CURLoption option,
      */
     data->set.low_speed_limit=va_arg(param, long);
     break;
+  case CURLOPT_MAX_SEND_SPEED_LARGE:
+    /*
+     * The max speed limit that sends transfer more than
+     * CURLOPT_MAX_SEND_PER_SECOND bytes per second the transfer is
+     * throttled..
+     */
+    data->set.max_send_speed=va_arg(param, curl_off_t);
+    break;
+  case CURLOPT_MAX_RECV_SPEED_LARGE:
+    /*
+     * The max speed limit that sends transfer more than
+     * CURLOPT_MAX_RECV_PER_SECOND bytes per second the transfer is
+     * throttled..
+     */
+    data->set.max_recv_speed=va_arg(param, curl_off_t);
+    break;
   case CURLOPT_LOW_SPEED_TIME:
     /*
      * The low speed time that if transfers are below the set

lib/urldata.h

@@ -1039,6 +1039,8 @@ struct UserDefined {
   curl_off_t infilesize;      /* size of file to upload, -1 means unknown */
   long low_speed_limit; /* bytes/second */
   long low_speed_time;  /* number of seconds */
+  curl_off_t max_send_speed; /* high speed limit in bytes/second for upload */
+  curl_off_t max_recv_speed; /* high speed limit in bytes/second for download */
   curl_off_t set_resume_from;  /* continue [ftp] transfer from here */
   char *cookie;         /* HTTP cookie string to send */
   struct curl_slist *headers; /* linked list of extra headers */

src/main.c

@@ -633,8 +633,8 @@ struct LongShort {
 /* global variable to hold info about libcurl */
 static curl_version_info_data *curlinfo;
 
-static void parseconfig(const char *filename,
-                        struct Configurable *config);
+static int parseconfig(const char *filename,
+                       struct Configurable *config);
 static char *my_get_line(FILE *fp);
 static int create_dir_hierarchy(char *outfile);
 
@@ -2069,7 +2069,9 @@ static ParameterError getparameter(char *flag, /* f or -long-flag */
       config->insecure_ok ^= TRUE;
       break;
     case 'K': /* parse config file */
-      parseconfig(nextarg, config);
+      if(parseconfig(nextarg, config))
+        warnf(config, "error trying read config from the '%s' file\n",
+              nextarg);
       break;
     case 'l':
       config->conf ^= CONF_FTPLISTONLY; /* only list the names of the FTP dir */
@@ -2402,9 +2404,9 @@ static ParameterError getparameter(char *flag, /* f or -long-flag */
   return PARAM_OK;
 }
 
-
-static void parseconfig(const char *filename,
-                        struct Configurable *config)
+/* return 0 on everything-is-fine, and non-zero otherwise */
+static int parseconfig(const char *filename,
+                       struct Configurable *config)
 {
   int res;
   FILE *file;
@@ -2613,6 +2615,9 @@ static void parseconfig(const char *filename,
     if(file != stdin)
       fclose(file);
   }
+  else
+    return 1; /* couldn't open the file */
+  return 0;
 }
 
 static void go_sleep(long ms)
@@ -2655,9 +2660,7 @@ static size_t my_fwrite(void *buffer, size_t sz, size_t nmemb, void *stream)
   size_t rc;
   struct OutStruct *out=(struct OutStruct *)stream;
   struct Configurable *config = out->config;
-  curl_off_t size = (curl_off_t)(sz * nmemb); /* typecast to prevent
-                                                 warnings when converting from
-                                                 unsigned to signed */
+
   if(out && !out->stream) {
     /* open file for writing */
     out->stream=fopen(out->filename, "wb");
@@ -2674,55 +2677,6 @@ static size_t my_fwrite(void *buffer, size_t sz, size_t nmemb, void *stream)
     }
   }
 
-  if(config->recvpersecond) {
-    /*
-     * We know when we received data the previous time. We know how much data
-     * we get now. Make sure that this is not faster than we are told to run.
-     * If we're faster, sleep a while *before* doing the fwrite() here.
-     */
-
-    struct timeval now;
-    long timediff;
-    long sleep_time;
-
-    static curl_off_t addit = 0;
-
-    now = curlx_tvnow();
-    timediff = curlx_tvdiff(now, config->lastrecvtime); /* milliseconds */
-
-    if((config->recvpersecond > CURL_MAX_WRITE_SIZE) && (timediff < 100) ) {
-      /* If we allow a rather speedy transfer, add this amount for later
-       * checking. Also, do not modify the lastrecvtime as we will use a
-       * longer scope due to this addition.  We wait for at least 100 ms to
-       * pass to get better values to do better math for the sleep. */
-      addit += size;
-    }
-    else {
-      size += addit; /* add up the possibly added bonus rounds from the
-                        zero timediff calls */
-      addit = 0; /* clear the addition pool */
-
-      if( size*1000 > config->recvpersecond*timediff) {
-        /* figure out how many milliseconds to rest */
-        sleep_time = (long)(size*1000/config->recvpersecond - timediff);
-
-        /*
-         * Make sure we don't sleep for so long that we trigger the speed
-         * limit.  This won't limit the bandwidth quite the way we've been
-         * asked to, but at least the transfer has a chance.
-         */
-        if (config->low_speed_time > 0)
-          sleep_time = MIN(sleep_time,(config->low_speed_time * 1000) / 2);
-
-        if(sleep_time > 0) {
-          go_sleep(sleep_time);
-          now = curlx_tvnow();
-        }
-      }
-      config->lastrecvtime = now;
-    }
-  }
-
   rc = fwrite(buffer, sz, nmemb, out->stream);
 
   if((sz * nmemb) == rc) {
@@ -2767,62 +2721,6 @@ static size_t my_fread(void *buffer, size_t sz, size_t nmemb, void *userp)
 {
   size_t rc;
   struct InStruct *in=(struct InStruct *)userp;
-  struct Configurable *config = in->config;
-  curl_off_t size = (curl_off_t)(sz * nmemb);  /* typecast to prevent warnings
-                                                  when converting from
-                                                  unsigned to signed */
-
-  if(config->sendpersecond) {
-    /*
-     * We know when we sent data the previous time. We know how much data
-     * we sent. Make sure that this was not faster than we are told to run.
-     * If we're faster, sleep a while *before* doing the fread() here.
-     * Also, make no larger fread() than should be sent this second!
-     */
-
-    struct timeval now;
-    long timediff;
-    long sleep_time;
-
-    static curl_off_t addit = 0;
-
-    now = curlx_tvnow();
-    timediff = curlx_tvdiff(now, config->lastsendtime); /* milliseconds */
-
-    if((config->sendpersecond > CURL_MAX_WRITE_SIZE) &&
-       (timediff < 100)) {
-      /*
-       * We allow very fast transfers, then allow at least 100 ms between
-       * each sleeping mile-stone to create more accurate long-term rates.
-       */
-      addit += size;
-    }
-    else {
-      /* If 'addit' is non-zero, it contains the total amount of bytes
-         uploaded during the last 'timediff' milliseconds. If it is zero,
-         we use the stored previous size. */
-      curl_off_t xfered = addit?addit:(curl_off_t)config->lastsendsize;
-      addit = 0; /* clear it for the next round */
-
-      if( xfered*1000 > config->sendpersecond*timediff) {
-        /* figure out how many milliseconds to rest */
-        sleep_time = (long)(xfered*1000/config->sendpersecond - timediff);
-        if(sleep_time > 0) {
-          go_sleep (sleep_time);
-          now = curlx_tvnow();
-        }
-      }
-      config->lastsendtime = now;
-
-      if(size > config->sendpersecond) {
-        /* lower the size to actually read */
-        nmemb = (size_t)config->sendpersecond;
-        sz = 1;
-      }
-    }
-
-    config->lastsendsize = sz*nmemb;
-  }
 
   rc = fread(buffer, sz, nmemb, in->stream);
 #if 0
@@ -3392,7 +3290,7 @@ operate(struct Configurable *config, int argc, char *argv[])
     ;
   }
   else {
-    parseconfig(NULL, config);
+    parseconfig(NULL, config); /* ignore possible failure */
   }
 
   if ((argc < 2)  && !config->url_list) {
@@ -3885,11 +3783,10 @@ operate(struct Configurable *config, int argc, char *argv[])
         curl_easy_setopt(curl, CURLOPT_IOCTLDATA, &input);
         curl_easy_setopt(curl, CURLOPT_IOCTLFUNCTION, my_ioctl);
 
-        if(config->recvpersecond) {
+        if(config->recvpersecond)
           /* tell libcurl to use a smaller sized buffer as it allows us to
              make better sleeps! 7.9.9 stuff! */
           curl_easy_setopt(curl, CURLOPT_BUFFERSIZE, config->recvpersecond);
-        }
 
         /* size of uploaded file: */
         curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, uploadfilesize);
@@ -3939,8 +3836,14 @@ operate(struct Configurable *config, int argc, char *argv[])
                          config->conf&CONF_AUTO_REFERER);
         curl_easy_setopt(curl, CURLOPT_USERAGENT, config->useragent);
         curl_easy_setopt(curl, CURLOPT_FTPPORT, config->ftpport);
-        curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, config->low_speed_limit);
+        curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT,
+                         config->low_speed_limit);
         curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, config->low_speed_time);
+        curl_easy_setopt(curl, CURLOPT_MAX_SEND_SPEED_LARGE,
+                         config->sendpersecond);
+        curl_easy_setopt(curl, CURLOPT_MAX_RECV_SPEED_LARGE,
+                         config->recvpersecond);
+
         curl_easy_setopt(curl, CURLOPT_RESUME_FROM_LARGE,
                          config->use_resume?config->resume_from:0);
         curl_easy_setopt(curl, CURLOPT_COOKIE, config->cookie);