Topic "Slow text conversion when using SCP on large text file"

Author Message
John F

Guest


Hi,
I have a customer who wishes to use WinSCP with sudo for their operators, and it is all working great with the SCP protocol, however, we find that with large text files (100mb+), the dos to unix conversion time is horrendous.

A test on my Windows7 intel i5 4G, winscp 4.2.9, it took 6 minutes for conversion of a 100mb text file, before it started the uploading part (which was darn quick!). To compare, using the standard dos2unix tool, it took mere seconds..

We tested this on multiple machines and os'es and found the same, including using the tofromdos tool on windows. Any thoughts?

Btw, the WinSCP team is doing a fantastic job!

John F
Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
I admit the conversion with SCP protocol may not be efficient. Do you have to convert the file actually? Cannot you do with just binary protocol?Or convert it yourself before or after transfer.
_________________
Martin Prikryl
coffecup1978
[View user's profile]
Donor
Joined: 2011-02-13
Posts: 3
Location: Australia
Hi,
I indeed concur, personally I could use local or remote conversion using local tools like 'dos2unix', if sftp-4 was not available, however, for the users I support have already very complex procedures to follow, and I was hoping to avoid introducing this extra step for them.

I do believe you are doing a great job with WinSCP, and I have made a small personal donation, and hope you perhaps will consider review the implementation of this feature? Perhaps there are already routines available in the GNU 'unix2dos' libraries you could take advantage of? Anyway, your effort is much appreciated!

Thanks&Regards,
John F
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
I do not think you need SFTP4 to make it work better. Any SFTP should do.
I'm sorry, but I do not want to invest my time into improving implementation of obsolete protocol, what SCP is.
_________________
Martin Prikryl
coffecup1978
[View user's profile]
Donor
Joined: 2011-02-13
Posts: 3
Location: Australia
That's ok, unfortunately SFTP's shortcoming is that it does not support sudo, so we'll have to rethink the whole situation a bit. Thanks for your response anyway.

Thanks&Regards,
John Fjeldberg
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
You may be able to use sudo even with SFTP:
http://winscp.net/eng/docs/faq_su
_________________
Martin Prikryl
coffecup1978
[View user's profile]
Donor
Joined: 2011-02-13
Posts: 3
Location: Australia
Thanks Martin for your suggestion, it doesn't quite fit the model, but we will be implementing a procedure where the application user and operators are in the same group with the right umask instead. Again, your prompt response on this is much appreciated.

Thanks&Regards,
John F
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
I'm not sure on what you are expecting me to respond Smile
_________________
Martin Prikryl
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License