Partially uploading file - Perl sftp upload script

Advertisement

mallett76
Joined:
Posts:
4
Location:
MA

Partially uploading file - Perl sftp upload script

I have a Perl SFTP script that is only partially uploading the file.
For example, today it uploaded 13,737,981, yet the actual size was 13,740,583 bytes. And, I did confirm a few records were missing from the file.
I know the vendor alters the file after it is uploaded, could that have anything to do with the process being interrupted.

Below is the log data. Any ideas or suggestions would be appreciated.

. 2020-01-14 06:54:35.824 --------------------------------------------------------------------------
. 2020-01-14 06:54:35.824 WinSCP Version 5.11.3 (Build 7995) (OS 6.3.9600 - Windows Server 2012 R2 Standard)
. 2020-01-14 06:54:35.824 Configuration: nul
. 2020-01-14 06:54:35.824 Log level: Normal
. 2020-01-14 06:54:35.824 Local account: CABLE\!NEDVFCollections
. 2020-01-14 06:54:35.824 Working directory: C:\Program Files (x86)\WINSCP
. 2020-01-14 06:54:35.824 Process ID: 6528
. 2020-01-14 06:54:35.824 Command-line: "C:\Program Files (x86)\WINSCP\winscp.exe" /xmllog="C:\Users\!!NEDVFCollections\AppData\Local\Temp\wscp0F9C.033C0D9D.tmp" /xmlgroups /xmllogrequired /nointeractiveinput /dotnet=5.11.3 /ini=nul /log="X:\tGram\Freedom\NED_FRE\freTGramSessionLog.txt" /console /consoleinstance=_3996_18643596_126
. 2020-01-14 06:54:35.824 Time zone: Current: GMT-5, Standard: GMT-5 (Eastern Standard Time), DST: GMT-4 (Eastern Daylight Time), DST Start: 3/8/2020, DST End: 11/1/2020
. 2020-01-14 06:54:35.824 Login time: Tuesday, January 14, 2020 6:54:35 AM
. 2020-01-14 06:54:35.824 --------------------------------------------------------------------------
. 2020-01-14 06:54:35.824 Script: Retrospectively logging previous script records:
> 2020-01-14 06:54:35.824 Script: option batch on
< 2020-01-14 06:54:35.824 Script: batch on
< 2020-01-14 06:54:35.824 Script: reconnecttime 120
> 2020-01-14 06:54:35.824 Script: option confirm off
< 2020-01-14 06:54:35.824 Script: confirm off
> 2020-01-14 06:54:35.824 Script: option reconnecttime 120
< 2020-01-14 06:54:35.824 Script: reconnecttime 120
> 2020-01-14 06:54:35.824 Script: open sftp://C85616:***@ftpx.revspringinc.com -hostkey="ssh-dss 1024 15:e4:8b:06:88:0a:ee:c4:5e:d4:94:02:f2:44:c9:5f" -timeout=15
. 2020-01-14 06:54:35.824 --------------------------------------------------------------------------
. 2020-01-14 06:54:35.824 Session name: C85616@ftpx.revspringinc.com (Ad-Hoc site)
. 2020-01-14 06:54:35.824 Host name: ftpx.revspringinc.com (Port: 22)
. 2020-01-14 06:54:35.824 User name: C85616 (Password: Yes, Key file: No, Passphrase: No)
. 2020-01-14 06:54:35.824 Tunnel: No
. 2020-01-14 06:54:35.824 Transfer Protocol: SFTP
. 2020-01-14 06:54:35.824 Ping type: Off, Ping interval: 30 sec; Timeout: 15 sec
. 2020-01-14 06:54:35.824 Disable Nagle: No
. 2020-01-14 06:54:35.824 Proxy: None
. 2020-01-14 06:54:35.824 Send buffer: 262144
. 2020-01-14 06:54:35.824 SSH protocol version: 2; Compression: No
. 2020-01-14 06:54:35.824 Bypass authentication: No
. 2020-01-14 06:54:35.839 Try agent: Yes; Agent forwarding: No; TIS/CryptoCard: No; KI: Yes; GSSAPI: Yes
. 2020-01-14 06:54:35.839 GSSAPI: Forwarding: No
. 2020-01-14 06:54:35.839 Ciphers: aes,chacha20,blowfish,3des,WARN,arcfour,des; Ssh2DES: No
. 2020-01-14 06:54:35.839 KEX: ecdh,dh-gex-sha1,dh-group14-sha1,rsa,WARN,dh-group1-sha1
. 2020-01-14 06:54:35.839 SSH Bugs: Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto
. 2020-01-14 06:54:35.839 Simple channel: Yes
. 2020-01-14 06:54:35.839 Return code variable: Autodetect; Lookup user groups: Auto
. 2020-01-14 06:54:35.839 Shell: default
. 2020-01-14 06:54:35.839 EOL: LF, UTF: Auto
. 2020-01-14 06:54:35.839 Clear aliases: Yes, Unset nat.vars: Yes, Resolve symlinks: Yes; Follow directory symlinks: No
. 2020-01-14 06:54:35.839 LS: ls -la, Ign LS warn: Yes, Scp1 Comp: No
. 2020-01-14 06:54:35.839 SFTP Bugs: Auto,Auto
. 2020-01-14 06:54:35.839 SFTP Server: default
. 2020-01-14 06:54:35.839 Local directory: default, Remote directory: home, Update: Yes, Cache: Yes
. 2020-01-14 06:54:35.839 Cache directory changes: Yes, Permanent: Yes
. 2020-01-14 06:54:35.839 Recycle bin: Delete to: No, Overwritten to: No, Bin path:
. 2020-01-14 06:54:35.839 DST mode: Unix
. 2020-01-14 06:54:35.839 --------------------------------------------------------------------------
. 2020-01-14 06:54:35.839 Looking up host "ftpx.revspringinc.com" for SSH connection
. 2020-01-14 06:54:35.886 Connecting to 40.139.154.12 port 22
. 2020-01-14 06:54:35.902 We claim version: SSH-2.0-WinSCP_release_5.11.3
. 2020-01-14 06:54:35.933 Server version: SSH-2.0-9.99 sshlib
. 2020-01-14 06:54:35.933 Using SSH protocol version 2
. 2020-01-14 06:54:35.933 Have a known host key of type dss
. 2020-01-14 06:54:35.933 Doing Diffie-Hellman group exchange
. 2020-01-14 06:54:36.043 Doing Diffie-Hellman key exchange with hash SHA-256
. 2020-01-14 06:54:36.355 Host key fingerprint is:
. 2020-01-14 06:54:36.355 ssh-dss 1024 15:e4:8b:06:88:0a:ee:c4:5e:d4:94:02:f2:44:c9:5f
. 2020-01-14 06:54:36.355 Verifying host key dss 0x8abffbd9d2a22976 ce5d447d952ffcfb 1166292cdaa2e1ea abbb5060e4d81558 baf5606896a3e68b 48500af4b1ded297 7e8b0dd8129b8dad 6806f4414f61df77 36adb4756f7ee13e 79903573e7e51eac 0633a91f3c7b1ddb 61cc797db4d6a70a 22e3b26f4d388be9 55467ef6f5806837 dad10f45771422c5 515140d8eba76873 ,0x9e5eaf2de13da58f 31321ccab141e12b 749dedc5,0x4ff615f2ceb16957 8b893db905455e8a 659a2270a83191bc 14951eb66146e267 ab31e5890218a3db 9a0b55d1db564a71 d1fdbe25a9fa9544 9466601cc417e6f9 3d474d08e2cbb2ae 01c00cc83872fc4c ea6f37ee06c1c565 f8be3cb8d1a3c54f 7ea21446814a2921 b8564cc1ad136a14 f0b063c7cbb910cd 2bef2b33c8616e53 ,0x10478ad1c14eff85 549b6e6652a85d27 7418b34b0fc200ad 10d1ec0ab5a46bf5 94efdfa6822fc0f6 f82280edafb7bcd2 b555817dbbacdfeb 8ba7e58e7dffe804 c37f79c02044f5e7 9382deb3ac90a985 bed14d9b7ce5d90a 14ec466b7fe0d9d9 8f44f191a69c7938 9c411d9a543ec017 af7aa1b0004a0da1 775eab9462da426c with fingerprint ssh-dss 1024 15:e4:8b:06:88:0a:ee:c4:5e:d4:94:02:f2:44:c9:5f
. 2020-01-14 06:54:36.355 Host key matches configured key
. 2020-01-14 06:54:36.355 Initialised AES-256 SDCTR client->server encryption
. 2020-01-14 06:54:36.355 Initialised HMAC-SHA-256 client->server MAC algorithm
. 2020-01-14 06:54:36.355 Initialised AES-256 SDCTR server->client encryption
. 2020-01-14 06:54:36.355 Initialised HMAC-SHA-256 server->client MAC algorithm
! 2020-01-14 06:54:36.464 Using username "C85616".
. 2020-01-14 06:54:36.480 Server offered these authentication methods: publickey,password
. 2020-01-14 06:54:36.480 Prompt (password, "SSH password", <no instructions>, "&Password: ")
. 2020-01-14 06:54:36.480 Using stored password.
. 2020-01-14 06:54:36.480 Sent password
. 2020-01-14 06:54:36.746 Access granted
. 2020-01-14 06:54:36.746 Opening session as main channel
. 2020-01-14 06:54:36.781 Opened main channel
. 2020-01-14 06:54:36.875 Started a shell/command
. 2020-01-14 06:54:36.890 --------------------------------------------------------------------------
. 2020-01-14 06:54:36.890 Using SFTP protocol.
. 2020-01-14 06:54:36.890 Doing startup conversation with host.
> 2020-01-14 06:54:36.890 Type: SSH_FXP_INIT, Size: 5, Number: -1
< 2020-01-14 06:54:36.906 Type: SSH_FXP_VERSION, Size: 5, Number: -1
. 2020-01-14 06:54:36.906 SFTP version 3 negotiated.
. 2020-01-14 06:54:36.906 We believe the server has signed timestamps bug
. 2020-01-14 06:54:36.906 We will use UTF-8 strings until server sends an invalid UTF-8 string as with SFTP version 3 and older UTF-8 strings are not mandatory
. 2020-01-14 06:54:36.906 Getting current directory name.
. 2020-01-14 06:54:36.906 Getting real path for '.'
> 2020-01-14 06:54:36.906 Type: SSH_FXP_REALPATH, Size: 10, Number: 16
< 2020-01-14 06:54:36.937 Type: SSH_FXP_NAME, Size: 45, Number: 16
. 2020-01-14 06:54:36.937 Real path is '/Home/c85616'
. 2020-01-14 06:54:36.937 Startup conversation with host finished.
< 2020-01-14 06:54:36.937 Script: Active session: [1] C85616@ftpx.revspringinc.com
> 2020-01-14 06:54:37.469 Script: pwd
< 2020-01-14 06:54:37.469 Script: /Home/c85616
> 2020-01-14 06:54:37.547 Script: stat -- "/Home/c85616/In/COMCAST_FREEDOM_01142020A.txt"
. 2020-01-14 06:54:37.547 Listing file "/Home/c85616/In/COMCAST_FREEDOM_01142020A.txt".
> 2020-01-14 06:54:37.547 Type: SSH_FXP_LSTAT, Size: 54, Number: 263
< 2020-01-14 06:54:37.704 Type: SSH_FXP_STATUS, Size: 31, Number: 263
< 2020-01-14 06:54:37.704 Status code: 2, Message: 263, Server: No such file, Language: en
< 2020-01-14 06:54:37.704 Script: Can't get attributes of file '/Home/c85616/In/COMCAST_FREEDOM_01142020A.txt'.
< 2020-01-14 06:54:37.704 No such file or directory.
< 2020-01-14 06:54:37.704 Error code: 2
< 2020-01-14 06:54:37.704 Error message from server (en): No such file
. 2020-01-14 06:54:37.704 Script: Failed
> 2020-01-14 06:54:37.798 Script: put -nopermissions -nopreservetime -transfer="binary" -resumesupport="off" -- "V:\tGram\Freedom\Text_Files\COMCAST_FREEDOM_01142020A.txt" "/Home/c85616/In/"
. 2020-01-14 06:54:37.807 Copying 1 files/directories to remote directory "/Home/c85616/In/" - total size: 13,737,981
. 2020-01-14 06:54:37.807 PrTime: No; PrRO: No; Rght: rw-r--r--; PrR: No (No); FnCs: N; RIC: 0100; Resume: N (102400); CalcS: No; Mask:
. 2020-01-14 06:54:37.808 TM: B; ClAr: No; RemEOF: No; RemBOM: No; CPS: 0; NewerOnly: No; InclM: ; ResumeL: 0
. 2020-01-14 06:54:37.808 AscM: *.*html; *.htm; *.txt; *.php; *.php3; *.cgi; *.c; *.cpp; *.h; *.pas; *.bas; *.tex; *.pl; *.js; .htaccess; *.xtml; *.css; *.cfg; *.ini; *.sh; *.xml
. 2020-01-14 06:54:37.878 File: 'V:\tGram\Freedom\Text_Files\COMCAST_FREEDOM_01142020A.txt' [2020-01-14T11:54:35.122Z] [13737981]
. 2020-01-14 06:54:37.893 Copying "V:\tGram\Freedom\Text_Files\COMCAST_FREEDOM_01142020A.txt" to remote directory started.
. 2020-01-14 06:54:37.893 Binary transfer mode selected.
. 2020-01-14 06:54:37.893 Opening remote file.
> 2020-01-14 06:54:37.893 Type: SSH_FXP_OPEN, Size: 70, Number: 515
< 2020-01-14 06:54:37.971 Type: SSH_FXP_HANDLE, Size: 13, Number: 515
> 2020-01-14 06:54:37.971 Type: SSH_FXP_WRITE, Size: 32793, Number: 1030
. 2020-01-14 06:54:40.352 759 skipped SSH_FXP_WRITE, SSH_FXP_READ, SSH_FXP_DATA and SSH_FXP_STATUS packets.
> 2020-01-14 06:54:40.352 Type: SSH_FXP_CLOSE, Size: 13, Number: 99332
< 2020-01-14 06:54:40.352 Type: SSH_FXP_STATUS, Size: 26, Number: 97286
< 2020-01-14 06:54:40.352 Status code: 0
< 2020-01-14 06:54:40.352 Type: SSH_FXP_STATUS, Size: 26, Number: 97542
< 2020-01-14 06:54:40.367 Status code: 0
< 2020-01-14 06:54:40.367 Type: SSH_FXP_STATUS, Size: 26, Number: 97798
< 2020-01-14 06:54:40.367 Status code: 0
< 2020-01-14 06:54:40.367 Type: SSH_FXP_STATUS, Size: 26, Number: 98054
< 2020-01-14 06:54:40.367 Status code: 0
< 2020-01-14 06:54:40.367 Type: SSH_FXP_STATUS, Size: 26, Number: 98310
< 2020-01-14 06:54:40.367 Status code: 0
< 2020-01-14 06:54:40.367 Type: SSH_FXP_STATUS, Size: 26, Number: 98566
< 2020-01-14 06:54:40.367 Status code: 0
< 2020-01-14 06:54:40.383 Type: SSH_FXP_STATUS, Size: 26, Number: 98822
< 2020-01-14 06:54:40.383 Status code: 0
< 2020-01-14 06:54:40.383 Type: SSH_FXP_STATUS, Size: 26, Number: 99078
< 2020-01-14 06:54:40.383 Status code: 0
< 2020-01-14 06:54:40.461 Type: SSH_FXP_STATUS, Size: 26, Number: 99332
< 2020-01-14 06:54:40.461 Status code: 0
. 2020-01-14 06:54:40.461 Transfer done: 'V:\tGram\Freedom\Text_Files\COMCAST_FREEDOM_01142020A.txt' => '/Home/c85616/In/COMCAST_FREEDOM_01142020A.txt' [13737981]
. 2020-01-14 06:54:40.461 Copying finished: Transferred: 13,737,981, Elapsed: 0:00:02, CPS: 5,517,396/s
> 2020-01-14 06:54:41.105 Script: stat -- "/Home/c85616/In/COMCAST_FREEDOM_01142020A.txt"
. 2020-01-14 06:54:41.105 Listing file "/Home/c85616/In/COMCAST_FREEDOM_01142020A.txt".
> 2020-01-14 06:54:41.105 Type: SSH_FXP_LSTAT, Size: 54, Number: 99591
< 2020-01-14 06:54:41.214 Type: SSH_FXP_ATTRS, Size: 37, Number: 99591
. 2020-01-14 06:54:41.214 COMCAST_FREEDOM_01142020A.txt;-;13737981;2020-01-14T11:54:40.000Z;3;"" [0];"" [0];rwxrwxrwx;1
< 2020-01-14 06:54:41.214 Script: -rwxrwxrwx 0 13737981 Jan 14 6:54:40 2020 COMCAST_FREEDOM_01142020A.txt

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,518
Location:
Prague, Czechia

Re: Partially uploading file - Perl sftp upload script

If the file is being written to, while it is being uploaded (or even after), then it is perfectly possible that it is uploaded incomplete.

Reply with quote

mallett76

I found the solution. It looks like after I created the file I didn’t close it. I went from creating the file then right to the Egyptians upload. I needed to create the file, close the file, then upload it to the Afro site. When I did that it solved the problem.

Reply with quote

mallett76
Joined:
Posts:
4
Location:
MA

mallett76 wrote:

I found the solution. It looks like after I created the file I didn’t close it. I went from creating the file then right to the Sftp upload. I needed to create the file, close the file, then upload it to the Sftp site. When I did that it solved the problem.
sorry, these were weird typos I didn’t mean to reference Egyptians or Afros before, please disregard. Long story short problem is solved now

Reply with quote

Advertisement

You can post new topics in this forum