Topic "Copying files open for writing possible data loss"

Author Message
trybowski
[View user's profile]

Joined: 2010-01-13
Posts: 3
Location: Krakw
Hello everybody,
so I've been struggling with this problem with WinSCP. I'm using it to transfer files from a financial application to a banking system and every now and then accountants would complain that some information is missing from those files. It turned out that sometimes transfer would be attempted on a file that is still being generated (it takes on average 3-4 minutes to generate a file) and in such case WinSCP copies a part of the file generated sofar, and doesn't issue any warning (even though I use the -delete switch, meaning that I'm actually moving the file). This is a classical case of data loss.

I asked Martin whether this could be fixed, and his answer was Maybe if more people ask for it.. So, here I am, asking if you could support me in this case.


Description of an issue:

WinSCP allows to copy a file that is opened for writing. This can be useful at times (for example you can copy an Excel sheet that is open), but it can lead to data loss: imagine a situation where a program fetches data from database and saves it to file. If you copy a file while it's not yet ready, you'll get a copy that is missing some part of data, but since you got no warning, you may think that the file is complete. This is even trickier when a transfer is done automatically, periodically from script (which WinSCP is capable of doing and thus is predestined to support).


Proposed solution:

1) On attempt to copy a file that is opened for writing, issue a warning, with possibility to continue or skip.
2) Point 1 should be configurable with possibility to disable it, but enabled by default (I don't insist on the last one).
3) On attempt to move a file that is opened for writing, issue a warning, with possibility to continue or skip. If user presses skip, file shouldn't be copied at all (treat move as one logical operation, not two separate copy+delete).


Out of many file managers I tested (in their default settings):
1) File Commander by Brian Havard: warns when copying a file open for writing (allows to continue), and errors when moving such file (copy is not created too).
2) Free Commander by Marek Jasinski: allows copying such file (without a warning), disallows moving such file (copy is not created too)
3) FAR Manager: allows copying such file (without a warning), disallows moving such file (copy is not created too)
4) Unreal Commander: allows copying such file (without a warning), disallows moving such file (copy is not created too)
5) Total Commander: allows copying such file (without a warning), while moving such file, a copy is created, original not deleted, no warning
6) Windows Explorer: allows copying such file (without a warning), disallows moving such file (copy is not created too)

As I'm not a programmer (except for PHP webpages, which doesn't really count in this case, does it?), I'm unable to code a patch myself. But from what I know the fix is rather simple: shouldn't take much time to implement and shouldn't interfere with current functionality.

Regards for all.
Advertisements
Stajkowski

Guest


trybowski wrote:
Hello everybody,
so I've been struggling with this problem with WinSCP. I'm using it to transfer files from a financial application to a banking system and every now and then accountants would complain that some information is missing

Hello,

I have exactly the same problem in our application. We loose part of data. Definietely a file open for writing should not be available for copying.
Such fix would be great relief.

Kind Regards
Stajkowski
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Stajkowski wrote:
I have exactly the same problem in our application. We loose part of data. Definietely a file open for writing should not be available for copying.
Such fix would be great relief.

Do you mean that WinSCP should fails uploading a file that is opened for writing by another application? It's up to the writing application to decide whether to prevent other applications from reading it.
_________________
Martin Prikryl
gaurav.dutt

Guest


We are facing the same situation, the transfer should fail if the file being written.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
gaurav.dutt wrote:
We are facing the same situation, the transfer should fail if the file being written.

Again, the writing application, when opening the file decides whether it allows other application reading it or not. If the application did not protect the file against reading, although it's not safe to (what should be done almost always, except maybe for a log files), it's fault of the writing application.
_________________
Martin Prikryl
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License