Slow text conversion when using SCP on large text file

Advertisement

John F
Guest

Slow text conversion when using SCP on large text file

Hi,
I have a customer who wishes to use WinSCP with sudo for their operators, and it is all working great with the SCP protocol, however, we find that with large text files (100mb+), the dos to unix conversion time is horrendous.

A test on my Windows7 intel i5 4G, winscp 4.2.9, it took 6 minutes for conversion of a 100mb text file, before it started the uploading part (which was darn quick!). To compare, using the standard dos2unix tool, it took mere seconds..

We tested this on multiple machines and os'es and found the same, including using the tofromdos tool on windows. Any thoughts?

Btw, the WinSCP team is doing a fantastic job!

John F

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
40,430
Location:
Prague, Czechia

Re: Slow text conversion when using SCP on large text file

I admit the conversion with SCP protocol may not be efficient. Do you have to convert the file actually? Cannot you do with just binary protocol?Or convert it yourself before or after transfer.

Reply with quote

coffecup1978
Donor
Joined:
Posts:
3
Location:
Australia

Hi,
I indeed concur, personally I could use local or remote conversion using local tools like 'dos2unix', if sftp-4 was not available, however, for the users I support have already very complex procedures to follow, and I was hoping to avoid introducing this extra step for them.

I do believe you are doing a great job with WinSCP, and I have made a small personal donation, and hope you perhaps will consider review the implementation of this feature? Perhaps there are already routines available in the GNU 'unix2dos' libraries you could take advantage of? Anyway, your effort is much appreciated!

Thanks&Regards,
John F

Reply with quote

martin
Site Admin
martin avatar

I do not think you need SFTP4 to make it work better. Any SFTP should do.
I'm sorry, but I do not want to invest my time into improving implementation of obsolete protocol, what SCP is.

Reply with quote

coffecup1978
Donor
Joined:
Posts:
3
Location:
Australia

That's ok, unfortunately SFTP's shortcoming is that it does not support sudo, so we'll have to rethink the whole situation a bit. Thanks for your response anyway.

Thanks&Regards,
John Fjeldberg

Reply with quote

Advertisement

coffecup1978
Donor
Joined:
Posts:
3
Location:
Australia

Thanks Martin for your suggestion, it doesn't quite fit the model, but we will be implementing a procedure where the application user and operators are in the same group with the right umask instead. Again, your prompt response on this is much appreciated.

Thanks&Regards,
John F

Reply with quote

Advertisement

You can post new topics in this forum