WebDAV transfer extremely slow compared to CarotDAV

Advertisement

Ossdevely
Joined:
Posts:
3

WebDAV transfer extremely slow compared to CarotDAV

Hi WinSCP Team,

I have used the latest WinSCP version on Windows 10 to transfer large files to a WebDAV server via 10Gbps connection. Iperf3 benches 9.2Gbps between client and server. WinSCP manages to do around 70MB/s there is one CPU core at 80% and a second core at around 30% while transferring to WebDAV using normal HTTP, HTTPS is even slower. On the same client I used CarotDAV which is able to transfer at around 400MB/s. I used Wireshark to check the difference in transfer between the two clients.

I'm attaching screenshots of the capture. As seen in the capture, CarotDAV sends more data in each packet. IMHO those little packages send to the server by WinSCP are most likely the reason why it is so slow. To me it looks like it uses a default socket send buffer of 8192, which can be seen in the Wireshark dump. Can you set SO_SNDBUF to something more reasonable like 65536?

P.S. I also tested throughput from a Linux client using mount.davfs2 which also uses libneon, it also delivers 400MB/s.

Description: Carrotdav transfer

carotdav.PNG

Description: Winscp transfer

winscp.PNG

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
38,438
Location:
Prague, Czechia

Re: WebDAV transfer extremely slow compared to CarotDAV

Your followup email (redacted):
Hi Martin,

I opened a thread in WinSCP forum regarding bad WinSCP WebDAV performance.
I was able to compile WinSCP and fix this myself.

In lib neon I changed NE_BUFSIZ in ne_defs.h, from 8192 to:
#ifndef NE_BUFSIZ
#define NE_BUFSIZ 65536
#endif
With this change my throughput went up from 70MB/s to 320MB/s hitting the limit of the HDD on this machine and hardly any CPU load. I did not experience any problems, downloading/uploading files. I can easily transfer 50GB+ files.

But with WebDAV using SSL I hit around 45MB/s. I can see one core completely maxed out. As far as I can see OpenSSL is not compiled with HW acceleration, e.g. Intel AES. Any special reason for that?

Cheers Richard

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
38,438
Location:
Prague, Czechia

Re: WebDAV transfer extremely slow compared to CarotDAV

Thanks for your investigation.

I'll increase the NE_BUFSIZ.
And I'll look into the OpenSSL hardware acceleration.

Reply with quote

Ossdevely

Re: WebDAV transfer extremely slow compared to CarotDAV

Great!

I had a quick try myself generating the asm files and run it through nasm for hw acceleration, this worked, but got linker errors in the end, and the messages that come from the embarcadero toolchain don't say much and I'm not really familiar with this toolchain.

Reply with quote

martin
Site Admin
martin avatar

Re: WebDAV transfer extremely slow compared to CarotDAV

Can you please post the steps you did so far? (to save me some time, when working on this)

Reply with quote

Advertisement

Ossdevely
Joined:
Posts:
3

Re: WebDAV transfer extremely slow compared to CarotDAV

I downloaded the same OpenSSL version 1.1.1q generated the asm files, e.g.:
./aesni-x86.pl win32 aesni_x86.asm
Copied the asm files e.g. libs\openssl\crypto\aes\asm

Added compile flag: -DAESNI_ASM

Added it to the makefile:
$(OBJ_D)\aesni_x86.obj: $(SRC_D)\crypto\aes\asm\aesni_x86.asm
    $(ASM) -o$(OBJ_D)\aesni_x86.obj $(SRC_D)\crypto\aes\asm\aesni_x86.asm
libssleay32 liblibeay32 compile

Reply with quote

Advertisement

You can post new topics in this forum