Topic "Unpredictable text file corruption during transfer?"

Author Message
Guest




I am transferring CSV files from a cloud server to my local server for import to SQL via SFTP. I am frequently getting corruption -- records cut off, records cut off with subsequent records appended to them, etc. I can look at these files on the server, prior to transfer, and they are correct -- but changed when I get them after WinSCP transfer.

Example of 2 records after transfer - note that the first record has a full date as the last "field," but the 2nd record's last field is missing the 0 on 3000 (but NOT the ending quotation!)
"9934237","Active","09/05/2015","Conversion","Cert FT","Y","02/20/2013","01/01/1900","07/01/2015","01/01/3000"
"9934238","Terminated","09/05/2015","T-Assignment Ended","Sub","","02/13/2013","11/30/2013","11/30/2013","01/01/300"

CRLF = end of line character.

WinSCP version 5.9 build 6786
Windows Server 2012 R2
SFTP protocol

I have a script, but am not sure if it's right:
@echo off
"\\appdev2012\c$\Program Files (x86)\WinSCP\WinSCP.com" ^
/refresh ^
/log="\\appdev2012\c$\users\sisdbuser\desktop\!Y!M!D_!T_GetFiles.log" /ini=nul ^
/command ^
"open sftp://my.name%%40mydomain:Kronos%%40%%21%%40%%23@xfer.remotesite.net/ -hostkey=""ssh-rsa 2048 keyhere"" -rawsettings Compression=1 CacheDirectories=0 CacheDirectoryChanges=0 PreserveDirectoryChanges=0 EOLType=1 SFTPMaxVersion=3" ^
"cd /Prod/WIM_OUT/Tech" ^
"lcd \\Appdev2012\c$\Users\sisdbuser\Downloads" ^
"get -transfer=ascii -neweronly CCSD_*.csv" ^
"exit"

set WINSCP_RESULT=%ERRORLEVEL%
if %WINSCP_RESULT% equ 0 (
echo Success
) else (
echo Error
)

exit /b %WINSCP_RESULT%

Log file (modified for security, I hope) is attached. I don't see "errors" in it, but maybe I missed them.

I am (usually) executing my script file from a SQL server query window, using EXEC master..xp_CMDShell "UNC path to script name"

Please point me in the right direction.
20160902_131311_GetFiles.log (23.22 KB) Private file

Description: (none)

Advertisements
cara
[View user's profile]

Joined: 2016-09-02
Posts: 2
I thought I was logged in when I submitted this -- it belongs to me.

An additional note: I have tried automatic, binary and ascii transfers.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Thanks for your report.

Quote:
I can look at these files on the server, prior to transfer

How exactly are you doing that?

If you re-download the same file again immediately, do you get the same corruption? Or is the problem temporary?

Are you sure you can reproduce the problem with a binary mode? Can you post a log file for that?
cara
[View user's profile]

Joined: 2016-09-02
Posts: 2
martin wrote:
How exactly are you doing that?

I have a Citrix connection to the vendor's cloud server. When I log into that, I can run Visual Studio over their citrix connection. From within visual, I can File > Open and point to the CSV file that resides on that server, and open it with Notepad.

martin wrote:
If you re-download the same file again immediately, do you get the same corruption? Or is the problem temporary?

The problem SEEMS to change each time the files are downloaded. Sometimes, no corruption. Sometimes, even if run quickly after another download, I'll get DIFFERENT corruption. This has not been well-tested, however.

martin wrote:
Are you sure you can reproduce the problem with a binary mode? Can you post a log file for that?

I have a log file for a previous run using the binary option. It takes some effort to "cleanse" those files of identifying info, so I'll try to work on that. I see no errors in it, however.

I've had to move to another FTP software, because I cannot spend any more time on bad transfers -- but I like WinSCP better, so I would still like to resolve this, if possible. So far, I'm not having the problem using the other software.

P.S. Don't know why my quote tags aren't working!
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
cara wrote:
because I cannot spend any more time on bad transfers

So can you help us debugging this or not?
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License