If asked to read data into a buffer and the buffer is too small to hold
the data, this function now returns an error instead of as previously
just copy as much as fits.
As discussed on the mailing list, it was wrong for win64 and using the
VC-provided type is the safest approach instead of second- guessing
which one it should be.
Added crypto.h that is the unified header to include when using crypto
functionality. It should be the only header that needs to adapt to the
underlying crypto library in use. It provides the set of prototypes that
are library agnostic.
Pass a NULL pointer for the publickey parameter of
libssh2_userauth_publickey_fromfile and
libssh2_userauth_hostbased_fromfile functions. In this case, the
functions recompute the public key from the private key file data.
This is work done by Jean-Louis CHARTON
<Jean-Louis.CHARTON@oikialog.com>, then adapted by Mark Smith and
slightly edited further by me Daniel.
WARNING: this does leave the feature NOT WORKING when libssh2 is built
to use libgcrypt instead of OpenSSL simply due to lack of
implementation.
By using a new separate struct for incoming SFTP packets and not sharing
the generic packet struct, we can get rid of an unused field and add a
new one dedicated for holding the request_id for the incoming
package. As sftp_packet_ask() is called fairly often, a "mere" integer
comparison is MUCH faster than the previous memcmp() of (typically) 5
bytes.
Make sure that we cleanup remainders when the handle is closed and when
the subsystem is shutdown.
Existing flaw: if a single handle sends packets that haven't been
replied to yet at the time when the handle is closed, those packets will
arrive later and end up in the generic packet brigade queue and they
will remain in there until flushed. They will use unnecessary memory,
make things slower and they will ruin the SFTP handling if the
request_id counter ever wraps (highly unlikely to every happen).
The SFTP read function now does transfers the same way the SFTP write
function was made to recently: it creates a list of many outgoing
FXP_READ packets that each asks for a small data chunk. The code then
tries to keep sending read request while collecting the acks for the
previous requests and returns the received data.
The loop that waits for remote.close to get set may end up looping
forever since session->socket_state gets set to
LIBSSH2_SOCKET_DISCONNECTED by the packet_add() function called from the
transport_read() function and after having been set to
LIBSSH2_SOCKET_DISCONNECTED, the transport_read() function will only
return 0.
Bug: http://trac.libssh2.org/ticket/198
Split off libssh2_sftp_seek64 from the libssh2_sftp_seek man page, and
mentioned that we consider the latter deprecated. Also added a mention
about the dangers of doing seek during writing or reading.
The new SFTP write code caused a regression as the seek function no
longer worked as it didn't set the write position properly.
It should be noted that seeking is STRONGLY PROHIBITED during upload, as
the upload magic uses two different offset positions and the multiple
outstanding packets etc make them sensitive to change in the midst of
operations.
This functionality was just verified with the new example code
sftp_append. This bug was filed as bug #202:
Bug: http://trac.libssh2.org/ticket/202
I ran SFTP upload tests against localhost. It showed that to make the
app reach really good speeds, I needed to do a little code tweak and
change MAX_SFTP_OUTGOING_SIZE from 4000 to 30000. The tests I did before
with the high latency tests didn't show any real difference whatever I
had that size set to.
This number is the size in bytes that libssh2 cuts off the large input
buffer and sends off as an individual sftp packet.
This is an example that is very similar to sftp_write_nonblock.c, with
the exception that this uses
1 - a larger upload buffer
2 - a sliding buffer mechnism to allow the app to keep sending lots of
data to libssh2 without having to first drain the buffer.
These are two key issues to make libssh2 SFTP uploads really perform
well at this point in time.
The attempts made to have _libssh2_channel_write() accept larger pieces
of data and split up the data by itself into 32700 byte chunks and pass
them on to channel_write() in a loop as a way to do faster operations on
larger data blocks was a failed attempt.
The reason why it is difficult:
The API only allows EAGAIN or a length to be returned. When looping over
multiple blocks to get sent, one block can get sent and the next might
not. And yet: when transport_send() has returned EAGAIN we must not call
it again with new data until it has returned OK on the existing data it
is still working on. This makes it a mess and we do get a much easier
job by simply returning the bytes or EAGAIN at once, as in the EAGAIN
case we can assume that we will be called with the same arguments again
and transport_send() will be happy.
Unfortunately, I think we take a small performance hit by not being able
to do this.
This is a new example snippet. The code is largely based on ssh2_exec,
and is written by Tommy Lindgren. I edited it into C90 compliance and to
conform to libssh2 indent style and some more.
When a piece of data is sent from the send_existing() function we must
make the parent function return afterwards. Otherwise we risk that the
parent function tries to send more data and ends up getting an EGAIN for
that more data and since it can only return one return code it doesn't
return info for the successfully sent data.
As this change is a regression I now added a larger comment explaining
why it has to work like this.
Starting now, we unconditionally use the internal replacement functions
for send() and recv() - creatively named _libssh2_recv() and
_libssh2_send().
On errors, these functions return the negative 'errno' value instead of
the traditional -1. This design allows systems that have no "natural"
errno support to not have to invent it. It also means that no code
outside of these two transfer functions should use the errno variable.
Some checks are better done in _libssh2_channel_write just once per
write instead of in channel_write() since the looping will call the
latter function multiple times per _libssh2_channel_write() invoke.
The SFTP handle struct now buffers number of acked bytes that haven't
yet been returned. The way this is used is as following:
1. sftp_write() gets called with a buffer of let say size 32000. We
split 32000 into 8 smaller packets and send them off one by one. One of
them gets acked before the function returns so 4000 is returned.
2. sftp_write() gets called again a short while after the previous one,
now with a much smaller size passed in to the function. Lets say 8000.
In the mean-time, all of the remaining packets from the previous call
have been acked (7*4000 = 28000). This function then returns 8000 as all
data passed in are already sent and it can't return any more than what
it got passed in. But we have 28000 bytes acked. We now store the
remaining 20000 in the handle->u.file.acked struct field to add up in
the next call.
3. sftp_write() gets called again, and now there's a backlogged 20000
bytes to return as fine and that will get skipped from the beginning
of the buffer that is passed in.
When SCP send or recv fails, it gets a special message from the server
with a warning or error message included. We have no current API to
expose that message but the foundation is there. Removed unnecessary use
of session struct fields.
I added size checks in several places. I fixed the code flow to be easier
to read in some places.
I removed unnecessary zeroing of structs. I removed unused struct fields.
