Topic "Synchronization loops copying remote 0-length file to local"

Author Message
klausner
[View user's profile]
Donor
Joined: 2004-02-18
Posts: 18
Using WinSCP and logging on through an intranet (internal LAN, no internet WAN required). When a remote sub-directry in a directory tree contains a zero-length file (/dev/zero), and I start a synchronization (remote to local) of the top directory, this file is copied forever.
Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
So far I've never heard about /dev/zero file. But reading about it now it seems that what you experience is natural consequence of the file properties. It seems that the file is not zero-length, it is endless. Correct me if I'm wrong.

BTW, solution would be to add the file to exclusion list.
_________________
Martin Prikryl
klausner
[View user's profile]
Donor
Joined: 2004-02-18
Posts: 18
What is an endess file?

An "ls -l" of "/dev/zero" reports
0 crw-rw-rw- 1 root root 1, 5 Dec 8 2003 zero

/dev/zero is certainly treated as endless by WinSCP, as I have experienced. But with a zero length reported by both Linux and Windows, why would WinSCP treat it as endless? Sould it not treat it just like it would treat a non-existent filename that has been "touch"ed and thereby created?

David

prikryl wrote:
So far I've never heard about /dev/zero file. But reading about it now it seems that what you experience is natural consequence of the file properties. It seems that the file is not zero-length, it is endless. Correct me if I'm wrong.

BTW, solution would be to add the file to exclusion list.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
klausner wrote:
What is an endess file?

The /dev/zero file is endless source of zeroes.

Note that WinSCP disregards the file size. It must, as for most OS'es the directory listing file size is informational only and may not be accurate. Particularly SFTP specification says:
Quote:
The 'size' field specifies the size of the file on disk, in bytes. If it is present during file creation, it SHOULD be considered a hint as to the file's eventual size.

.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
Sorry, the quote from SFTP specification is irrelevant.

Maybe one obvious note: /dev/zero is not a file, it is device (program). The device, when read produces endless stream of zeroes. As a consequence WinSCP never encounters it's end-of-file.

You can try to do "cp /dev/zero tempfile" on your server. You will end with full filesystem Smile
_________________
Martin Prikryl
klausner
[View user's profile]
Donor
Joined: 2004-02-18
Posts: 18
Yes, you are right, it is a device and the cp produces an endless stream of bytes. I will add "*/dev/zero" (I assume that's the format based on the WinSCP Documentation) to the exclusion list.

Thank you.
David
klausner
[View user's profile]
Donor
Joined: 2004-02-18
Posts: 18
Dear Martin,

Synchronizing directories that contain "zero" and "null" endless files cause WinSCP to never complete. I know that these are "endless" files, but I have created a preset to exclude them, and the exclusion does not work. My preset exclusion is as follows:
"*.bak; *.tmp; ~$*; *.wbk; *~; #*; .#* zero null"

Can you assist me?

David
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
You need to separate masks with semicolon.
_________________
Martin Prikryl
klausner
[View user's profile]
Donor
Joined: 2004-02-18
Posts: 18
Dear Martin,

I have separated the masks with semicolons ... it still tries to synch the /dev/null and /dev/zero.

mask=(without quotes)
"/dev/null; /dev/zero"

David
klausner
[View user's profile]
Donor
Joined: 2004-02-18
Posts: 18
Posted: 25 Sep 2007 13:37
PostSynchronization loops copying remote 0-length file to local
[Reply with quote] [Edit this post] [Delete this post]
Dear Martin,

I have separated the masks with semicolons ... it still tries to synch the /dev/null and /dev/zero.

mask=(without quotes)
"/dev/null; /dev/zero"

I think, perhaps, WinSCP should check for zero-length files and not try toi open them -- merely create the same file on output and move on to the next operation. Reading these files and performing a copy seems to lead to trouble.

David
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
klausner wrote:
I have separated the masks with semicolons ... it still tries to synch the /dev/null and /dev/zero.

mask=(without quotes)
"/dev/null; /dev/zero"

Please a complete comand you use to set the mask.
_________________
Martin Prikryl
klausner
[View user's profile]
Donor
Joined: 2004-02-18
Posts: 18
without the enclosing quotes,

"*.bak; *.tmp; ~$*; *.wbk; *~; #*; .#*; /dev/null; /dev/zero;"
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
Maybe problem is elsewhere. After reading the thread from beginning I've noticed that you refer to "preset". Transfer setting presets does not apply for synchronisation! For synchronisation you need to set mask on the synchronization dialog.
_________________
Martin Prikryl
Guest




That seemed to do it. It's a bit confusing because one can easily assume that the (Text, Binary ...) Presets apply to synchronization (as I did) because the pulldown on the Synchronization window for "Transfer settings" contains choices for (Presets) Text and Binary, as well as for Custom; and also the Configure pulldown takes us to the Presets window! Best to have the synchronization apply the default presets and not require the user to always specify a Custom mask to be applied to each synchronization separately.

Also, how about not trying to copy zero-length files, just "touch" them on the output side of the synchronization?
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
Anonymous wrote:
That seemed to do it. It's a bit confusing because one can easily assume that the (Text, Binary ...) Presets apply to synchronization (as I did) because the pulldown on the Synchronization window for "Transfer settings" contains choices for (Presets) Text and Binary, as well as for Custom; and also the Configure pulldown takes us to the Presets window! Best to have the synchronization apply the default presets and not require the user to always specify a Custom mask to be applied to each synchronization separately.

Presets can be per-directory. So if it would apply for synchronisation too, different settings should/may be applied for each subdirectory being synchronized. It may be confusing too. I do not know what is better.

Quote:
Also, how about not trying to copy zero-length files, just "touch" them on the output side of the synchronization?

I do not think, this is right approach. First size of the file in directory listing is informational only. One should not rely on it. The file maybe growing while opened, etc... If fact what WinSCP does is correct. The files you are having a problem with are there exactly for the reason to generate endless stream of bytes. You may have different "zero-length" file that actually generates a sensible (and finite) content.
_________________
Martin Prikryl
Guest




I understand. How about a "default" preset for synchronization? The user can still change it, but the default can be other than nothing.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
Quote:
How about a "default" preset for synchronization? The user can still change it, but the default can be other than nothing.

There is a configurable default. The default transfer settings still apply to synchronization.
_________________
Martin Prikryl
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License