FTP Client: slow uploads

Discussion of bugs and problems found in Altap Salamander. In your reports, please be as descriptive as possible, and report one incident per report. Do not post crash reports here, send us the generated bug report by email instead, please.
ugglebot
Posts: 6
Joined: 10 Dec 2008, 13:00

Upload slow too

Post by ugglebot »

I'm having a similar problem, with slow uploads. I'm uploading a large single file (100 Mb), and in AS I get transfer times of about 50 k (so I don't even finish it).

I switch to Filezilla instead and get close to 300 k.

Log from FileZilla:

Status: Starting upload of ********.exe
Command: PASV
Response: 227 Entering Passive Mode (174,132,105,34,224,90)
Command: STOR **************.exe
Response: 150 Accepted data connection
Status: Disconnected from server
Response: 226-File successfully transferred
Response: 226 358.876 seconds (measured here), 298.22 Kbytes per second
Status: File transfer successful
Status: Disconnected from server

Obviously adding thread has no effect on a single large file. Are there any other settings that can affect the speed?
Petr Solin
ALTAP Staff
ALTAP Staff
Posts: 1112
Joined: 08 Dec 2005, 09:13
Location: Novy Bor, Czech Republic
Contact:

Post by Petr Solin »

Maybe Fillezilla uses MODEZ (compressed connection). Try to search in its FTP log. I have no other idea why it should be slower (maybe time variable connection speed), it's only simple socket, nothing magical. ;-)

More connections can help only if you have many small files.
ugglebot
Posts: 6
Joined: 10 Dec 2008, 13:00

Post by ugglebot »

Hmm, I really wished someone had a good solution to this. I've tried a barrage of things, to no avail.

First I
* set binary mode by default
* set passive mode by default

However, oddly enough from the AS ftp log I still don't see "type i", so I
* Send PASV manually (to be safe)
* Send "TYPE I" manually

But no difference in upload speed. I have a pretty good Cable/DSL line and usually no problems with upload speeds, and as I said, FileZilla is 10 times faster (at the same time, I mean, I try uploading from the two applications about 10 seconds apart in time only).

So sad, because AS is the BEST program I've ever owned (yes, I have a proper purchased license :-) ) and it would be so convenient to work also with FTP in it. But now I have to switch to FileZilla for that...

Haven't been able to check the FZ logs, where are they...? Do you have to turn debug on?
ugglebot
Posts: 6
Joined: 10 Dec 2008, 13:00

Post by ugglebot »

Same file (about 3.5 Mb), 10 seconds apart in transfer

FZ log
Response: 200 TYPE is now 8-bit binary
Command: PASV
Response: 227 Entering Passive Mode (174,132,105,34,146,142)
Command: STOR myfile.tar.bz2
Response: 150 Accepted data connection
Response: 226-File successfully transferred
Response: 226 10.355 seconds (measured here), 320.55 Kbytes per second

AS log
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (174,132,105,34,144,192)
STOR myfile.tar.bz2
150 Accepted data connection
226-File successfully transferred
226 65.055 seconds (measured here), 51.02 Kbytes per second
(21:46:47): Connection was returned to panel...

The only diff, except speed, is the port number, but that can't really have an influence, can it...?
Petr Solin
ALTAP Staff
ALTAP Staff
Posts: 1112
Joined: 08 Dec 2005, 09:13
Location: Novy Bor, Czech Republic
Contact:

Post by Petr Solin »

MODE Z can be only at the beginning of FZ log, it's turned on for all future transfers.

You can also try to upload some archive file (e.g. .RAR) which cannot be compressed anymore.
ugglebot
Posts: 6
Joined: 10 Dec 2008, 13:00

Post by ugglebot »

I don't see MODE Z in FileZilla. Here the header log after I connect

Status: Resolving IP-Address for xxxxxxxxxxxxxxxxxxxxx
Status: Connecting to xxxxxxxxx:21...
Status: Connection established, waiting for welcome message...
Response: 220---------- Welcome to Pure-FTPd [TLS] ----------
Response: 220-You are user number 3 of 50 allowed.
Response: 220-Local time is now 15:11. Server port: 21.
Response: 220 You will be disconnected after 15 minutes of inactivity.
Command: USER xxxxxx
Response: 331 User xxxxxxxx OK. Password required
Command: PASS *********
Response: 230-User xxxxxxxxx has group access to: xxxxxxx
Response: 230 OK. Current restricted directory is /
Command: SYST
Response: 215 UNIX Type: L8
Command: FEAT
Response: 211-Extensions supported:
Response: EPRT
Response: IDLE
Response: MDTM
Response: SIZE
Response: REST STREAM
Response: MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
Response: MLSD
Response: ESTP
Response: PASV
Response: EPSV
Response: SPSV
Response: ESTA
Response: AUTH TLS
Response: PBSZ
Response: PROT
Response: 211 End.
Status: Connected
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is your current location
Command: TYPE I
Response: 200 TYPE is now 8-bit binary
Command: PASV
Response: 227 Entering Passive Mode (174,132,105,34,184,84)
Command: LIST
Response: 150 Accepted data connection
Response: 226-ASCII
Response: 226-Options: -a -l
Response: 226 24 matches total
Status: Directory listing successful

The file I'm transfering is BZ2, which also cannot be compressed any further (similar to gzip).
Jan Rysavy
ALTAP Staff
ALTAP Staff
Posts: 5229
Joined: 08 Dec 2005, 06:34
Location: Novy Bor, Czech Republic
Contact:

Post by Jan Rysavy »

Could you please also test another FTP client? For example Windows command line FTP?
ugglebot
Posts: 6
Joined: 10 Dec 2008, 13:00

Post by ugglebot »

ftp> put xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
200 PORT command successful
150 Connecting to port 5001
226-File successfully transferred
226 8.888 seconds (measured here), 371.97 Kbytes per second
ftp: 3398880 bytes sent in 8.73Seconds 389.11Kbytes/sec.
ftp>

Good news or bad news?
Jan Rysavy
ALTAP Staff
ALTAP Staff
Posts: 5229
Joined: 08 Dec 2005, 06:34
Location: Novy Bor, Czech Republic
Contact:

Post by Jan Rysavy »

I don't know :)

Please try to transfer some 30MB (already compressed) file, so the time will be around one minute.
Measure the time in Salamander and in command line FTP client, so we can compare.

Do not transfer using both FTP clients at the same time -- measure one, then another. Also try to repeat your measurements to assure you can reproduce it.

Thank you.
Petr Solin
ALTAP Staff
ALTAP Staff
Posts: 1112
Joined: 08 Dec 2005, 09:13
Location: Novy Bor, Czech Republic
Contact:

Post by Petr Solin »

Good news, we can improve it (at least I hope). ;-) I have forgotten that uploading (sending data to socket) is very tricky, we have some heuristics there for calculating block size (512B to 32KB) used for sending data to socket. We have cases when upload speed goes from 5MB/s to 160KB/s when we increased this block size to 32KB, I really cannot explain this.

So, I would like to please you to do some testing for us: please download special version of ftp plugin below, just rename your current version of directory plugins\ftp to e.g. ftp.orig and create new ftp directory and unpack .rar archive to it.

What to do:
-restart Salamander (just to clear FTP logs)
-start your upload
-wait until upload finishes
-open FTP Logs window, menu Plugins > FTP Client > Show Logs
-in FTP Logs window: menu Logs > Save All As ... and review if there are some sensitive data (and replace them by e.g. "***DELETED***")
-pack this file and send it to my email petr.solin@altap.cz

Thanks in advance!
Attachments
ftp2.rar
(242.67 KiB) Downloaded 503 times
Petr Solin
ALTAP Staff
ALTAP Staff
Posts: 1112
Joined: 08 Dec 2005, 09:13
Location: Novy Bor, Czech Republic
Contact:

Post by Petr Solin »

File size about 3.5 MB is enough for this test (65 seconds in Salamander).
Petr Solin
ALTAP Staff
ALTAP Staff
Posts: 1112
Joined: 08 Dec 2005, 09:13
Location: Novy Bor, Czech Republic
Contact:

Post by Petr Solin »

Thanks for sent logs! It seems that upload speed is relatively constant for 4KB and 8KB blocks (as you can see from part of log below, 14 * 4KB or 7 * 8KB blocks per second which is both 56KB/s).

I have prepared another test version which tries many variants of block sizes, please test it again (all is the same as with previous test version) and send me results to my email again. This time please upload some bigger file, it should upload at least 5 minutes (e.g. 50MB should be enough). Thanks!

Code: Select all

(0:25:58): Keeping send block size 4096 bytes
(0:25:59): Sent size: 57344 bytes (in 14 blocks)
(0:26:00): Sent size: 57344 bytes (in 14 blocks)
(0:26:01): Sent size: 57344 bytes (in 14 blocks)
(0:26:02): Sent size: 57344 bytes (in 14 blocks)
(0:26:03): Sent size: 49152 bytes (in 12 blocks)
(0:26:03): Changing send block size to 8192 bytes
(0:26:04): Sent size: 65536 bytes (in 8 blocks)
(0:26:05): Sent size: 57344 bytes (in 7 blocks)
(0:26:06): Sent size: 57344 bytes (in 7 blocks)
(0:26:07): Sent size: 57344 bytes (in 7 blocks)
(0:26:08): Sent size: 40960 bytes (in 5 blocks)
Attachments
ftp3.rar
(242.54 KiB) Downloaded 507 times
ugglebot
Posts: 6
Joined: 10 Dec 2008, 13:00

Post by ugglebot »

Things are looking brighter! :-)

Total transfer speed was 106 kb/s with this version. Detailed log emailed separately.

Still a way to go to FZ, however, which registered
116.433 seconds (measured here), 423.83 Kbytes per second
on the same file just afterwards.
Petr Solin
ALTAP Staff
ALTAP Staff
Posts: 1112
Joined: 08 Dec 2005, 09:13
Location: Novy Bor, Czech Republic
Contact:

Post by Petr Solin »

Thanks! This version changes 14 block sizes in cycle, so please do not expect maximal performance. The maximal speed in this test was 256 KB/s with 58982 B blocks and 64KB blocks. So it seems we need even larger blocks to obtain FZ speed 423.83 KB/s.

I will return to this problem at the end of next week, it seems not trivia, so this patch will not be in version 2.51 beta 1 which is about to be released. I will prepare another test version which should be faster.

What is theoretical maximal upload speed on your DSL connection?
User avatar
Ether
Posts: 1471
Joined: 10 May 2007, 16:08
Location: Czech Republic
Contact:

Post by Ether »

Petr Solin wrote:version 2.51 beta 1 which is about to be released.
You mean v2.52 beta 1.
Ελληνικά rulez.
Post Reply