curl/lib
Howard Chu d64bd82bdc sendrecv: split the I/O handling into private handler
Howard Chu brought the bulk work of this patch that properly
moves out the sending and recving of data to the parts of the
code that are properly responsible for the various ways of doing
so.

Daniel Stenberg assisted with polishing a few bits and fixed some
minor flaws in the original patch.

Another upside of this patch is that we now abuse CURLcodes less
with the "magic" -1 return codes and instead use CURLE_AGAIN more
consistently.
2010-05-07 15:05:34 +02:00
..
.gitignore ignore files generated by 'maketgz' 2010-04-14 17:12:34 +02:00
amigaos.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
amigaos.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
arpa_telnet.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
base64.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
CMakeLists.txt removed trailing whitespace 2010-02-14 19:40:18 +00:00
config-amigaos.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
config-mac.h Added SIZEOF_INT and SIZEOF_SHORT definitions for non-configure systems 2010-02-26 18:32:46 +00:00
config-os400.h Added SIZEOF_INT and SIZEOF_SHORT definitions for non-configure systems 2010-02-26 18:32:46 +00:00
config-riscos.h Added SIZEOF_INT and SIZEOF_SHORT definitions for non-configure systems 2010-02-26 18:32:46 +00:00
config-symbian.h Added SIZEOF_INT and SIZEOF_SHORT definitions for non-configure systems 2010-02-26 18:32:46 +00:00
config-tpf.h Added SIZEOF_INT and SIZEOF_SHORT definitions for non-configure systems 2010-02-26 18:32:46 +00:00
config-vxworks.h Added SIZEOF_INT and SIZEOF_SHORT definitions for non-configure systems 2010-02-26 18:32:46 +00:00
config-win32.h Added SIZEOF_INT and SIZEOF_SHORT definitions for non-configure systems 2010-02-26 18:32:46 +00:00
config-win32ce.h Added SIZEOF_INT and SIZEOF_SHORT definitions for non-configure systems 2010-02-26 18:32:46 +00:00
config.dos remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
connect.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
connect.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
content_encoding.c gzip: Value stored to 'data' is never read 2010-04-24 12:37:11 +02:00
content_encoding.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
cookie.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
cookie.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_addrinfo.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_addrinfo.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_base64.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_config.h.cmake PolarSSL: initial support added 2010-05-05 22:30:46 +02:00
curl_hmac.h Implement SMTP authentication 2010-04-19 11:16:30 +02:00
curl_ldap.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_md5.h Fix compilation problem: declare Curl_HMAC_MD5 as extern in include file. 2010-04-19 12:44:17 +02:00
curl_memory.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_memrchr.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_memrchr.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_rand.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_rand.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_sspi.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_sspi.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_threads.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curl_threads.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
curlx.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
dict.c Curl_setup_transfer: no longer returns anything 2010-04-16 23:43:04 +02:00
dict.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
easy.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
easyif.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
escape.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
escape.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
file.c file_range: remove unnecessary check for NULL pointer 2010-04-16 23:18:43 +02:00
file.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
firefox-db2pem.sh remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
formdata.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
formdata.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
ftp.c FTP PORT: Value stored to 'rc' is never read 2010-04-16 23:45:05 +02:00
ftp.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
getenv.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
getinfo.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
getinfo.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
gtls.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
gtls.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
hash.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
hash.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
hmac.c hmac.c related compilation adjustment 2010-04-20 21:35:31 +02:00
hostares.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
hostasyn.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
hostip4.c resolvers: no more using AI_CANONNAME 2010-04-16 15:03:52 +02:00
hostip6.c resolvers: no more using AI_CANONNAME 2010-04-16 15:03:52 +02:00
hostip.c fix compiler warning: variable might be clobbered by longjmp or vfork 2010-04-13 16:34:13 +02:00
hostip.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
hostsyn.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
hostthre.c resolvers: no more using AI_CANONNAME 2010-04-16 15:03:52 +02:00
http_chunks.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
http_chunks.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
http_digest.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
http_digest.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
http_negotiate.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
http_negotiate.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
http_ntlm.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
http_ntlm.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
http.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
http.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
if2ip.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
if2ip.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
imap.c Curl_setup_transfer: no longer returns anything 2010-04-16 23:43:04 +02:00
imap.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
inet_ntop.c Remove usage of inet_ntoa and inet_ntoa_r 2008-09-23 19:17:19 +00:00
inet_ntop.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
inet_pton.c backport fix for failures to reject certain malformed literals 2008-11-17 02:40:41 +00:00
inet_pton.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
krb4.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
krb4.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
krb5.c krb5_auth: fix my previous change to compile 2010-04-16 20:22:46 +02:00
ldap.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
libcurl.def Yang Tse: msvc7+ has deprecated the 'DESCRIPTION' section in 2005-11-14 07:48:05 +00:00
libcurl.imp added missing curl_easy_pause to export list. 2009-08-16 15:03:58 +00:00
libcurl.plist Matt Veenstra updated to 7.12.3. Starting now, we'll update the version number 2004-12-08 23:09:23 +00:00
libcurl.rc remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
llist.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
llist.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
Makefile.am removed README.cmake due to the improved situation 2010-03-31 23:05:16 +02:00
makefile.amiga hmac.c related compilation adjustment 2010-04-20 21:23:53 +02:00
Makefile.b32 remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
makefile.dj update to current state 2010-03-24 10:44:25 +01:00
Makefile.inc PolarSSL: initial support added 2010-05-05 22:30:46 +02:00
Makefile.m32 encourage users to take latest lib dependencies. 2010-04-24 17:48:09 +02:00
Makefile.netware encourage users to take latest lib dependencies. 2010-04-24 17:48:09 +02:00
Makefile.riscos hmac.c related compilation adjustment 2010-04-20 21:23:53 +02:00
Makefile.vc6 hmac.c related compilation adjustment 2010-04-20 21:23:53 +02:00
Makefile.vxworks remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
Makefile.Watcom Add compilation directives for hmac in Watcom,riscos and vc6 platform-specific makefiles. 2010-04-20 19:23:48 +02:00
md5.c Curl_HMAC_MD5: fix the array init to not warn with picky compilers 2010-04-19 16:19:36 +02:00
memdebug.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
memdebug.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
mk-ca-bundle.pl restore executable bits on some files 2010-03-24 11:07:35 +01:00
mprintf.c dprintf_formatf: Value stored to 'left' is never read 2010-04-16 23:50:39 +02:00
msvcproj.foot Store now this file in CVS with unix line endings. 2010-01-15 20:23:35 +00:00
msvcproj.head - I renamed everything in the windows builds files that used the name 'curllib' 2009-02-23 09:36:54 +00:00
multi.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
multiif.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
netrc.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
netrc.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
nonblock.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
nonblock.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
nss.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
nssg.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
nwlib.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
nwos.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
parsedate.c parsedate: Value stored to 'found' is never read 2010-04-17 18:57:39 +02:00
parsedate.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
pingpong.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
pingpong.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
polarssl.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
polarssl.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
pop3.c Curl_setup_transfer: no longer returns anything 2010-04-16 23:43:04 +02:00
pop3.h fixed compiler warnings 2010-04-02 21:02:35 +02:00
progress.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
progress.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
qssl.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
qssl.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
rawstr.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
rawstr.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
README.ares update URL and cut out wrong info on ipv6 2010-04-12 11:09:08 +02:00
README.curl_off_t spell! (most of it fixed by Tor Arntsen) 2008-08-27 06:10:47 +00:00
README.curlx remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
README.encoding replaced tabs with spaces 2010-02-16 13:32:45 +00:00
README.hostip Added README.hostip 2005-01-14 13:43:29 +00:00
README.httpauth HTTP "auth done right". See lib/README.httpauth 2004-11-24 16:11:35 +00:00
README.memoryleak remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
README.multi_socket update the section on timeouts 2010-04-12 11:09:55 +02:00
README.pingpong introducing IMAP, POP3 and SMTP support (still lots of polish left to do) 2009-12-12 21:54:01 +00:00
README.pipelining updated to reflect reality 2006-10-29 09:11:44 +00:00
rtsp.c Curl_setup_transfer: no longer returns anything 2010-04-16 23:43:04 +02:00
rtsp.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
security.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
select.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
select.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
sendf.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
sendf.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
setup_once.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
setup-os400.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
setup.h PolarSSL: initial support added 2010-05-05 22:30:46 +02:00
share.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
share.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
slist.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
slist.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
smtp.c Remove null-effect leftover code. 2010-04-19 17:30:20 +02:00
smtp.h Implement SMTP authentication 2010-04-19 11:16:30 +02:00
sockaddr.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
socks_gssapi.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
socks_sspi.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
socks.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
socks.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
speedcheck.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
speedcheck.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
splay.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
splay.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
ssh.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
ssh.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
sslgen.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
sslgen.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
ssluse.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
ssluse.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
strdup.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strdup.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strequal.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strequal.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strerror.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strerror.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strtok.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strtok.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strtoofft.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
strtoofft.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
telnet.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
telnet.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
tftp.c Curl_setup_transfer: no longer returns anything 2010-04-16 23:43:04 +02:00
tftp.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
timeval.c remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
timeval.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
transfer.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
transfer.h Curl_setup_transfer: no longer returns anything 2010-04-16 23:43:04 +02:00
url.c sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
url.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00
urldata.h sendrecv: split the I/O handling into private handler 2010-05-07 15:05:34 +02:00
vc6libcurl.dsw Fix CR 2009-05-08 19:09:36 +00:00
vc8proj.foot Use DOS line-endings. 2007-01-29 19:08:04 +00:00
vc8proj.head - I renamed everything in the windows builds files that used the name 'curllib' 2009-02-23 09:36:54 +00:00
version.c curl_version: remove superfluous assignments 2010-04-16 23:49:03 +02:00
warnless.c use CURL_SIZEOF_LONG instead of SIZEOF_LONG 2010-03-25 11:39:13 +01:00
warnless.h remove the CVSish $Id$ lines 2010-03-24 11:02:54 +01:00

HTTP Pipelining with libcurl
============================

Background

Since pipelining implies that one or more requests are sent to a server before
the previous response(s) have been received, we only support it for multi
interface use.

Considerations

When using the multi interface, you create one easy handle for each transfer.
Bascially any number of handles can be created, added and used with the multi
interface - simultaneously. It is an interface designed to allow many
simultaneous transfers while still using a single thread. Pipelining does not
change any of these details.

API

We've added a new option to curl_multi_setopt() called CURLMOPT_PIPELINING
that enables "attempted pipelining" and then all easy handles used on that
handle will attempt to use an existing pipeline.

Details

- A pipeline is only created if a previous connection exists to the same IP
  address that the new request is being made to use.

- Pipelines are only supported for HTTP(S) as no other currently supported
  protocol has features resemembling this, but we still name this feature
  plain 'pipelining' to possibly one day support it for other protocols as
  well.

- HTTP Pipelining is for GET and HEAD requests only.

- When a pipeline is in use, we must take precautions so that when used easy
  handles (i.e those who still wait for a response) are removed from the multi
  handle, we must deal with the outstanding response nicely.

- Explicitly asking for pipelining handle X and handle Y won't be supported.
  It isn't easy for an app to do this association. The lib should probably
  still resolve the second one properly to make sure that they actually _can_
  be considered for pipelining. Also, asking for explicit pipelining on handle
  X may be tricky when handle X get a closed connection.

- We need options to control max pipeline length, and probably how to behave
  if we reach that limit. As was discussed on the list, it can probably be
  made very complicated, so perhaps we can think of a way to pass all
  variables involved to a callback and let the application decide how to act
  in specific situations. Either way, these fancy options are only interesting
  to work on when everything is working and we have working apps to test with.