file picked up before fully downloaded.

Hi All/Martin

I have Process-A which drops the files txt files to a directory. There is a scheduled task setup to execute WinSCP FTP file transfer every 5 mins.

Problem: it so happens once in a while there is an overlap the moment file is dropped and at the same time the task kicking in which seems to pick up the partial txt file and perform the FTP transfer. How to prevent the partial file transfer?

WinSCP 5.11.3

I did look into the documentation but not sure which feature I can use to restrict it.

Thank you
Sai

You might check out disabling transfer resume - I'm looking for a solution to this on a Session level, not at the root Preferences level for the entire application. See: https://winscp.net/eng/docs/ui_pref_resume

sjagadish2001 wrote:

Hi All/Martin

I have Process-A which drops the files txt files to a directory. There is a scheduled task setup to execute WinSCP FTP file transfer every 5 mins.

Problem: it so happens once in a while there is an overlap the moment file is dropped and at the same time the task kicking in which seems to pick up the partial txt file and perform the FTP transfer. How to prevent the partial file transfer?

WinSCP 5.11.3

I did look into the documentation but not sure which feature I can use to restrict it.

Thank you
Sai

sjagadish2001 wrote:

Problem: it so happens once in a while there is an overlap the moment file is dropped and at the same time the task kicking in which seems to pick up the partial txt file and perform the FTP transfer. How to prevent the partial file transfer?

You can use a file mask to transfer only files which are e.g. older than few minutes.
See https://winscp.net/eng/docs/file_mask#size_time

Hi Martin,

I used the file mask as shown in the logs - put -filemask=*.hl7>3H * (so include files modified before 3 hrs from current time.)
There were two files in the directory
1. "New Text Document.hl7" 26-Aug-2019 8:51 pm (date modified)
2. "test3.hl7" 05-Sep-2019 2:53 am (date modified)

Ran script 3:23 am so, test3.hl7 should have been skipped.
it skipped the 'New Text Document.hl7' file not sure why.. it should be the other way round I would think.

Ideally I would want to copy all files that were modified 2 minute ago ( making sure its downloaded fully before transferring).


[code]
. 2019-09-05 03:23:52.179 Using FTP protocol.
. 2019-09-05 03:23:52.179 Doing startup conversation with host.
> 2019-09-05 03:23:52.179 PWD
< 2019-09-05 03:23:52.180 257 "/" is current directory.
. 2019-09-05 03:23:52.180 Getting current directory name.
. 2019-09-05 03:23:52.180 Startup conversation with host finished.
> 2019-09-05 03:23:52.180 Script: cd /Acceptance/temp/OUT
. 2019-09-05 03:23:52.180 Changing directory to "/Acceptance/temp/OUT".
> 2019-09-05 03:23:52.180 CWD /Acceptance/temp/OUT
< 2019-09-05 03:23:52.181 250 CWD command successful.
. 2019-09-05 03:23:52.181 Getting current directory name.
> 2019-09-05 03:23:52.181 PWD
< 2019-09-05 03:23:52.182 257 "/Acceptance/temp/OUT" is current directory.
< 2019-09-05 03:23:52.182 Script: /Acceptance/temp/OUT
> 2019-09-05 03:23:52.182 Script: lcd D:\Shares\COMP\Acceptance\temp\OUT
< 2019-09-05 03:23:52.183 Script: D:\Shares\COMP\Acceptance\temp\OUT
> 2019-09-05 03:23:52.183 Script: put -filemask=*.hl7>3H *
. 2019-09-05 03:23:52.183 Copying 2 files/directories to remote directory "/Acceptance/temp/OUT" - total size: 5
. 2019-09-05 03:23:52.184 PrTime: Yes; PrRO: No; Rght: rw-r--r--; PrR: No (No); FnCs: N; RIC: 0100; Resume: S (102400); CalcS: No; Mask:
. 2019-09-05 03:23:52.184 TM: B; ClAr: No; RemEOF: No; RemBOM: No; CPS: 0; NewerOnly: No; InclM: *.hl7>3H; ResumeL: 0
. 2019-09-05 03:23:52.184 AscM: *.*html; *.htm; *.txt; *.php; *.php3; *.cgi; *.c; *.cpp; *.h; *.pas; *.bas; *.tex; *.pl; *.js; .htaccess; *.xtml; *.css; *.cfg; *.ini; *.sh; *.xml
. 2019-09-05 03:23:52.184 Retrieving listing to detect timezone difference
. 2019-09-05 03:23:52.184 Retrieving directory listing...
> 2019-09-05 03:23:52.184 TYPE A
< 2019-09-05 03:23:52.185 200 Type set to A.
> 2019-09-05 03:23:52.185 PASV
< 2019-09-05 03:23:52.187 227 Entering Passive Mode (10,99,250,43,82,117).
> 2019-09-05 03:23:52.187 LIST
< 2019-09-05 03:23:52.189 150 Opening ASCII mode data connection.
. 2019-09-05 03:23:52.189 Session ID reused
. 2019-09-05 03:23:52.189 Using TLSv1, cipher TLSv1/SSLv3: AES128-SHA, 2048 bit RSA, AES128-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA1
. 2019-09-05 03:23:52.189 TLS connection established
< 2019-09-05 03:23:52.191 226 Transfer complete.
. 2019-09-05 03:23:52.191 Data connection closed
. 2019-09-05 03:23:52.191 <Empty directory listing>
. 2019-09-05 03:23:52.191 Directory listing successful
. 2019-09-05 03:23:52.191 Detecting timezone difference...
. 2019-09-05 03:23:52.191 Found no file to use for detecting timezone difference
. 2019-09-05 03:23:52.191 File "New Text Document.hl7" excluded from transfer
* 2019-09-05 03:23:52.191 (EScpSkipFile)
. 2019-09-05 03:23:52.192 File: 'test3.hl7' [2019-09-05T00:53:42.042Z] [5]
. 2019-09-05 03:23:52.192 Copying "test3.hl7" to remote directory started.
. 2019-09-05 03:23:52.192 Binary transfer mode selected.
. 2019-09-05 03:23:52.193 Starting upload of test3.hl7
> 2019-09-05 03:23:52.193 TYPE I
< 2019-09-05 03:23:52.193 200 Type set to I.
> 2019-09-05 03:23:52.193 PASV
< 2019-09-05 03:23:52.196 227 Entering Passive Mode (10,99,250,43,82,118).
> 2019-09-05 03:23:52.196 STOR test3.hl7
< 2019-09-05 03:23:52.199 125 Data connection already open; Transfer starting.
. 2019-09-05 03:23:52.200 Session ID reused
. 2019-09-05 03:23:52.200 Using TLSv1, cipher TLSv1/SSLv3: AES128-SHA, 2048 bit RSA, AES128-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA1
. 2019-09-05 03:23:52.200 TLS connection established
< 2019-09-05 03:23:52.202 226 Transfer complete.
> 2019-09-05 03:23:52.202 MDTM 20190905005342 test3.hl7
< 2019-09-05 03:23:52.204 213 20190905005342
. 2019-09-05 03:23:52.204 Upload successful
. 2019-09-05 03:23:52.204 Transfer done: 'test3.hl7' => '/Acceptance/temp/OUT/test3.hl7' [5]
. 2019-09-05 03:23:52.204 Copying finished: Transferred: 5, Elapsed: 0:00:00, CPS: 333/s
> 2019-09-05 03:23:52.204 Script: cd /
. 2019-09-05 03:23:52.204 Changing directory to "/".
> 2019-09-05 03:23:52.204 CWD /
< 2019-09-05 03:23:52.205 250 CWD command successful.
[/code]

Can you attach a full log file, using the latest version of WinSCP?

martin wrote:

Can you attach a full log file, using the latest version of WinSCP?

Hi Martin,
My current ver. 5.11.3; will upgrade have any issues with the existing simple command line scripts? Just wanted to make sure.. and I should be able to use the Help->check for upgrades and update it directly which will keep all my settings? Let me know thanks ! Sai.

WinSCP is mostly backward compatible. New backward incompatible changes are logged:
https://winscp.net/eng/docs/incompatible_changes
Though your simple script should not break with the update.
All settings are preserved when upgrading.
https://winscp.net/eng/docs/installation#upgrade