7.6-pre1

describes, Bob Schader's research seems to agree.

CHANGES

@@ -7,6 +7,31 @@
                                History of Changes
 
 
+Daniel (29 January 2001)
+- Yet another Content-Range change. Ok now? Bob Schader checks from his end.
+
+Daniel (27 January 2001)
+- So the HTTP PUT resume fix wasn't good. There should appearantly be a
+  Content-Range header when resuming a PUT.
+
+- I noticed I broke the download-check that verifies that a resumed HTTP
+  download is actually resumed. It got broke because my new 'httpreq' field
+  in the main curl struct. I should get slapped. I added a test case for
+  this now, so I won't be able to ruin this again without noticing.
+
+- Added a test case for content-length verifying when downloading HTTP.
+
+- Made the progress meter title say if the transfer is being transfered. It
+  makes the output slightly better for resumes.
+
+- When dealing with Location: and HTTP return codes, libcurl will not attempt
+  to follow the spirit of RFC2616 better. It means that when POSTing to a
+  URL that is being following to a second place, the standard will judge on
+  what to do. All HTTP codes except 303 and 305 will cause curl to make a
+  second POST operation. 303 will make a GET and 305 is not yet supported.
+
+  I also wrote two test cases for this POST/GET/Location stuff.
+
 Version 7.6
 
 Daniel (26 January 2001)

docs/FAQ

@@ -1,4 +1,4 @@
-Updated: January 22, 2001 (http://curl.haxx.se/docs/faq.shtml)
+Updated: January 29, 2001 (http://curl.haxx.se/docs/faq.shtml)
                                   _   _ ____  _     
                               ___| | | |  _ \| |    
                              / __| | | | |_) | |    
@@ -30,6 +30,7 @@ FAQ
   3.6 Does curl support javascript, ASP, XML, XHTML or HTML version Y?
   3.7 Can I use curl to delete/rename a file through FTP?
   3.8 How do I tell curl to follow HTTP redirects?
+  3.9 How do I use curl in PHP?
 
  4. Running Problems
   4.1 Problems connecting to SSL servers.
@@ -53,6 +54,7 @@ FAQ
   5.3 How do I fetch multiple files with libcurl?
   5.4 Does libcurl do Winsock initing on win32 systems?
   5.5 Does CURLOPT_FILE work on win32 ?
+  5.6 What about Keep-Alive or persistant connections?
 
  6. License Issues
   6.1 I have a GPL program, can I use the libcurl library?
@@ -279,6 +281,19 @@ FAQ
 
      curl -L http://redirector.com
 
+  3.9 How do I use curl in PHP?
+
+  PHP4 has the ability to use libcurl as an internal module if built with that
+  option enabled. You then get a set of extra functions that can be used
+  within your PHP programs. You find all details about those functions in the
+  curl section in the PHP manual, see the online version at:
+
+        http://www.php.net/manual/ref.curl.php
+
+  PHP also offers the option to run a command line, and then you can of course
+  invoke the curl tool using a command line. This is the way to use curl if
+  you're using PHP3 or PHP4 built without curl module support.
+
 4. Running Problems
 
   4.1. Problems connecting to SSL servers.
@@ -310,6 +325,9 @@ FAQ
   In win32, the standard DOS shell treats the %-letter specially and you may
   need to quote the string properly when % is used in it.
 
+  Also note that if you want the literal %-letter to be part of the data you
+  pass in a POST using -d/--data you must encode it as '%25'.
+
   4.3. How can I use {, }, [ or ] to specify multiple URLs?
 
   Because those letters have a special meaning to the shell, and to be used in
@@ -482,6 +500,15 @@ FAQ
 
   (provided by Joel DeYoung)
 
+  5.6 What about Keep-Alive or persistant connections?
+
+  This is closely related to issue 5.3. Since libcurl has no real support
+  for doing multiple file transfers, there's no support for Keep-Alive or
+  persistant connections either.
+
+  This is of course subject to change as soon as libcurl gets support for
+  multiple files. Feel free to join in and make this change happen sooner!
+
 6. License Issues
 
   NOTE: This section is now updated to concern curl 7.5.2 or later!

include/curl/curl.h

@@ -452,8 +452,8 @@ char *curl_getenv(char *variable);
 char *curl_version(void);
 
 /* This is the version number */
-#define LIBCURL_VERSION "7.6"
+#define LIBCURL_VERSION "7.6.1-pre1"
-#define LIBCURL_VERSION_NUM 0x070600
+#define LIBCURL_VERSION_NUM 0x070601
 
 /* linked-list structure for the CURLOPT_QUOTE option (and other) */
 struct curl_slist {

lib/http.c

@@ -429,9 +429,6 @@ CURLcode Curl_http(struct connectdata *conn)
       }
     }
   }
-  if((data->bits.set_range) && !checkheaders(data, "Range:")) {
-    data->ptr_rangeline = aprintf("Range: bytes=%s\015\012", data->range);
-  }
   if((data->bits.http_set_referer) && !checkheaders(data, "Referer:")) {
     data->ptr_ref = aprintf("Referer: %s\015\012", data->referer);
   }
@@ -528,6 +525,34 @@ CURLcode Curl_http(struct connectdata *conn)
       /* we've passed, proceed as normal */
     }
   }
+  if(data->bits.set_range) {
+    /*
+     * A range is selected. We use different headers whether we're downloading
+     * or uploading and we always let customized headers override our internal
+     * ones if any such are specified.
+     */
+    if((data->httpreq == HTTPREQ_GET) &&
+       !checkheaders(data, "Range:")) {
+      data->ptr_rangeline = aprintf("Range: bytes=%s\r\n", data->range);
+    }
+    else if((data->httpreq != HTTPREQ_GET) &&
+            !checkheaders(data, "Content-Range:")) {
+
+      if(data->resume_from) {
+        /* This is because "resume" was selected */
+        long total_expected_size= data->resume_from + data->infilesize;
+        data->ptr_rangeline = aprintf("Content-Range: bytes %s%ld/%ld\r\n",
+                                      data->range, total_expected_size-1,
+                                      total_expected_size);
+      }
+      else {
+        /* Range was selected and then we just pass the incoming range and 
+           append total size */
+        data->ptr_rangeline = aprintf("Content-Range: bytes %s/%d\r\n",
+                                      data->range, data->infilesize);
+      }
+    }
+  }
 
   do {
     send_buffer *req_buffer;

lib/progress.c

@@ -203,6 +203,9 @@ int Curl_pgrsUpdate(struct UrlData *data)
          even when not displayed! */
   else if(!(data->progress.flags & PGRS_HEADERS_OUT)) {
     if (!data->progress.callback) {
+      if(data->resume_from)
+        fprintf(data->err, "** Resuming transfer from byte position %d\n",
+                data->resume_from);
       fprintf(data->err,
               "  %% Total    %% Received %% Xferd  Average Speed          Time             Curr.\n"
               "                                 Dload  Upload Total    Current  Left    Speed\n");

lib/transfer.c

@@ -725,13 +725,48 @@ CURLcode curl_transfer(CURL *curl)
         data->newurl = NULL; /* don't show! */
         data->bits.urlstringalloc = TRUE; /* the URL is allocated */
 
-        /* Disable both types of POSTs, since doing a second POST when
-           following isn't what anyone would want! */
-        data->bits.http_post = FALSE;
-        data->bits.http_formpost = FALSE;
-
         infof(data, "Follows Location: to new URL: '%s'\n", data->url);
 
+        /*
+         * We get here when the HTTP code is 300-399. We need to perform
+         * differently based on exactly what return code there was.
+         * Discussed on the curl mailing list and posted about on the 26th
+         * of January 2001.
+         */
+        switch(data->progress.httpcode) {
+        case 300: /* Multiple Choices */
+        case 301: /* Moved Permanently */
+        case 302: /* Found */
+        case 306: /* Not used */
+        case 307: /* Temporary Redirect */
+        default:  /* for all unknown ones */
+          /* These are explicitly mention since I've checked RFC2616 and they
+           * seem to be OK to POST to.
+           */
+          break;
+        case 303: /* See Other */
+          /* Disable both types of POSTs, since doing a second POST when
+           * following isn't what anyone would want! */
+          data->bits.http_post = FALSE;
+          data->bits.http_formpost = FALSE;
+          data->httpreq = HTTPREQ_GET; /* enfore GET request */
+          infof(data, "Disables POST\n");
+          break;
+        case 304: /* Not Modified */
+          /* 304 means we did a conditional request and it was "Not modified".
+           * We shouldn't get any Location: header in this response!
+           */
+          break;
+        case 305: /* Use Proxy */
+          /* (quote from RFC2616, section 10.3.6):
+           * "The requested resource MUST be accessed through the proxy given
+           * by the Location field. The Location field gives the URI of the
+           * proxy.  The recipient is expected to repeat this single request
+           * via the proxy. 305 responses MUST only be generated by origin
+           * servers."
+           */
+          break;
+        }
         curl_disconnect(c_connect);
         continue;
       }

lib/url.c

@@ -339,9 +339,13 @@ CURLcode curl_setopt(CURL *curl, CURLoption option, ...)
     break;
   case CURLOPT_UPLOAD:
     data->bits.upload = va_arg(param, long)?TRUE:FALSE;
+    if(data->bits.upload)
+      /* If this is HTTP, PUT is what's needed to "upload" */
+      data->httpreq = HTTPREQ_PUT;
     break;
   case CURLOPT_POST:
     data->bits.http_post = va_arg(param, long)?TRUE:FALSE;
+    if(data->bits.http_post)
       data->httpreq = HTTPREQ_POST;
     break;
   case CURLOPT_FILETIME:
@@ -364,6 +368,7 @@ CURLcode curl_setopt(CURL *curl, CURLoption option, ...)
     break;
   case CURLOPT_PUT:
     data->bits.http_put = va_arg(param, long)?TRUE:FALSE;
+    if(data->bits.http_put)
       data->httpreq = HTTPREQ_PUT;
     break;
   case CURLOPT_MUTE:
@@ -406,11 +411,13 @@ CURLcode curl_setopt(CURL *curl, CURLoption option, ...)
     break;
   case CURLOPT_CUSTOMREQUEST:
     data->customrequest = va_arg(param, char *);
+    if(data->customrequest)
       data->httpreq = HTTPREQ_CUSTOM;
     break;
   case CURLOPT_HTTPPOST:
     data->httppost = va_arg(param, struct HttpPost *);
     data->bits.http_formpost = data->httppost?1:0;
+    if(data->bits.http_formpost)
       data->httpreq = HTTPREQ_POST_FORM;
     break;
   case CURLOPT_INFILE:

src/main.c

@@ -109,7 +109,9 @@ typedef enum {
 #define CONF_FOLLOWLOCATION (1<<23) /* use Location: Luke! */
 #define CONF_GETTEXT  (1<<24) /* use ASCII/text for transfer */
 #define CONF_HTTPPOST (1<<25) /* multipart/form-data HTTP POST */
+#if 0
 #define CONF_PUT      (1<<27) /* PUT the input file */
+#endif
 #define CONF_MUTE     (1<<28) /* force NOPROGRESS */
 
 #ifndef HAVE_STRDUP
@@ -1749,7 +1751,9 @@ operate(struct Configurable *config, int argc, char *argv[])
         curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION,
                          config->conf&CONF_FOLLOWLOCATION);
         curl_easy_setopt(curl, CURLOPT_TRANSFERTEXT, config->conf&CONF_GETTEXT);
+#if 0
         curl_easy_setopt(curl, CURLOPT_PUT, config->conf&CONF_PUT);
+#endif
         curl_easy_setopt(curl, CURLOPT_MUTE, config->conf&CONF_MUTE);
         curl_easy_setopt(curl, CURLOPT_USERPWD, config->userpwd);
         curl_easy_setopt(curl, CURLOPT_PROXYUSERPWD, config->proxyuserpwd);

src/version.h

@@ -1,3 +1,3 @@
 #define CURL_NAME "curl"
-#define CURL_VERSION "7.6"
+#define CURL_VERSION "7.6.1-pre1"
 #define CURL_ID CURL_NAME " " CURL_VERSION " (" OS ") "

tests/data/Makefile.am

@@ -56,4 +56,9 @@ command26.txt  prot26.txt     command27.txt  prot27.txt \
 name26.txt     reply26.txt    name27.txt     stdout27.txt \
 command28.txt  name28.txt     prot28.txt     reply28.txt \
 command120.txt  name120.txt     prot120.txt     reply120.txt \
-command121.txt  name121.txt     prot121.txt     reply121.txt
+command121.txt  name121.txt     prot121.txt     reply121.txt \
+command29.txt  error30.txt  name30.txt  prot30.txt   reply30.txt \
+command30.txt  name29.txt   prot29.txt  reply29.txt \
+command31.txt  name32.txt  reply31.txt      reply32.txt \
+command32.txt  prot31.txt  reply310001.txt  reply320001.txt \
+name31.txt     prot32.txt  reply310002.txt  reply320002.txt

tests/data/command29.txt

@@ -0,0 +1 @@
+-C 200 http://%HOSTIP:%HOSTPORT/29

tests/data/command30.txt

@@ -0,0 +1 @@
+http://%HOSTIP:%HOSTPORT/30

tests/data/command31.txt

@@ -0,0 +1,2 @@
+http://%HOSTIP:%HOSTPORT/31 -d mooo=fooo -L
+

tests/data/command32.txt

@@ -0,0 +1,2 @@
+http://%HOSTIP:%HOSTPORT/32 -d mooo=fooo -L
+

tests/data/error30.txt

@@ -0,0 +1 @@
+18

tests/data/name29.txt

@@ -0,0 +1 @@
+HTTP download resume with Content-Length validity

tests/data/name30.txt

@@ -0,0 +1 @@
+HTTP GET uncomplete document

tests/data/name31.txt

@@ -0,0 +1 @@
+HTTP POST and follow Location: (error 301)

tests/data/name32.txt

@@ -0,0 +1 @@
+HTTP POST and follow Location: (error 303)

tests/data/prot29.txt

@@ -0,0 +1,7 @@
+GET /29 HTTP/1.0
+Range: bytes=200-
+User-Agent: curl/7.6 (i686-pc-linux-gnu) libcurl 7.6 (SSL 0.9.5) (ipv6 enabled)
+Host: 127.0.0.1:8999
+Pragma: no-cache
+Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
+

tests/data/prot30.txt

@@ -0,0 +1,6 @@
+GET /30 HTTP/1.0
+User-Agent: curl/7.6 (i686-pc-linux-gnu) libcurl 7.6 (SSL 0.9.5) (ipv6 enabled)
+Host: 127.0.0.1:8999
+Pragma: no-cache
+Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
+

tests/data/prot31.txt

@@ -0,0 +1,9 @@
+POST /moo/moo/moo/310002 HTTP/1.0
+User-Agent: curl/7.6 (i686-pc-linux-gnu) libcurl 7.6 (SSL 0.9.5) (ipv6 enabled)
+Host: 127.0.0.1:8999
+Pragma: no-cache
+Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
+Content-Length: 9
+Content-Type: application/x-www-form-urlencoded
+
+mooo=fooo

tests/data/prot32.txt

@@ -0,0 +1,6 @@
+GET /moo/moo/moo/320002 HTTP/1.0
+User-Agent: curl/7.6 (i686-pc-linux-gnu) libcurl 7.6 (SSL 0.9.5) (ipv6 enabled)
+Host: 127.0.0.1:8999
+Pragma: no-cache
+Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
+

tests/data/reply29.txt

@@ -0,0 +1,9 @@
+HTTP/1.1 200 OK
+Server: fake
+Content-Range: bytes 200-3526/3527
+Content-Length: 84
+
+
+The Content-Range header's contents above aren't really genuine for this
+content.
+

tests/data/reply30.txt

@@ -0,0 +1,7 @@
+HTTP/1.1 200 OK
+Server: fake
+Content-Length: 8400
+
+This file is a lot smaller than 8400 and thus curl should return an error
+for this.
+

tests/data/reply31.txt

@@ -0,0 +1,6 @@
+HTTP/1.1 301 Moved Permanently
+Server: fake
+Location: /moo/moo/moo/310002
+
+No contents
+

tests/data/reply310001.txt

@@ -0,0 +1,9 @@
+HTTP/1.1 301 Moved Permanently
+Server: fake
+Location: /moo/moo/moo/310002
+
+HTTP/1.1 200 Followed here fine
+Date: Thu, 09 Nov 2010 14:49:00 GMT
+Server: test-server/fake
+
+If this is received, the location following worked

tests/data/reply310002.txt

@@ -0,0 +1,5 @@
+HTTP/1.1 200 Followed here fine
+Date: Thu, 09 Nov 2010 14:49:00 GMT
+Server: test-server/fake
+
+If this is received, the location following worked

tests/data/reply32.txt

@@ -0,0 +1,7 @@
+HTTP/1.1 303 See Other
+Server: fake
+Location: /moo/moo/moo/320002
+
+This Location should be fetched with a GET!
+
+

tests/data/reply320001.txt

@@ -0,0 +1,9 @@
+HTTP/1.1 303 See Other
+Server: fake
+Location: /moo/moo/moo/320002
+
+HTTP/1.1 200 Followed here fine
+Date: Thu, 09 Nov 2010 14:49:00 GMT
+Server: test-server/fake
+
+If this is received, the location following worked

tests/data/reply320002.txt

@@ -0,0 +1,5 @@
+HTTP/1.1 200 Followed here fine
+Date: Thu, 09 Nov 2010 14:49:00 GMT
+Server: test-server/fake
+
+If this is received, the location following worked

tests/httpserver.pl

@@ -119,7 +119,7 @@ for ( $waitedpid = 0;
                 $testnum=$1;
 
                 if($verbose) {
-                    print STDERR "sending reply $testnum\n";
+                    print STDERR "OUT: sending reply $testnum\n";
                 }
             }
             else {

tests/runtests.pl

@@ -49,6 +49,7 @@ my $memanalyze="../memanalyze.pl";
 
 my $short;
 my $verbose;
+my $debugprotocol;
 my $anyway;
 
 #######################################################################
@@ -109,7 +110,8 @@ sub runhttpserver {
     }
 
     if ($RUNNING != 1) {
-        system("perl $srcdir/httpserver.pl $HOSTPORT &");
+        my $flag=$debugprotocol?"-v ":"";
+        system("perl $srcdir/httpserver.pl $flag $HOSTPORT &");
         sleep 1; # give it a little time to start
     }
     else {
@@ -150,7 +152,8 @@ sub runftpserver {
     }
 
     if ($RUNNING != 1) {
-        system("perl $srcdir/ftpserver.pl $FTPPORT &");
+        my $flag=$debugprotocol?"-v ":"";
+        system("perl $srcdir/ftpserver.pl $flag $FTPPORT &");
         sleep 1; # give it a little time to start
     }
     else {
@@ -504,6 +507,7 @@ sub singletest {
     unlink($STDOUT);
     unlink($STDERR);
 
+    unlink($CURLOUT); # remove the downloaded results
     unlink($FTPDCMD); # remove the instructions for this test
 
     if($memory_debug) {
@@ -551,6 +555,10 @@ do {
         # verbose output
         $verbose=1;
     }
+    elsif ($ARGV[0] eq "-d") {
+        # have the servers display protocol output 
+        $debugprotocol=1;
+    }
     elsif($ARGV[0] eq "-s") {
         # short output
         $short=1;
@@ -564,6 +572,7 @@ do {
         print <<EOHELP
 Usage: runtests.pl [-h][-s][-v][numbers]
   -a       continue even if a test fails
+  -d       display server debug info
   -h       this help text
   -s       short output
   -v       verbose output