Copying files open for writing possible data loss

Advertisement

trybowski
Joined:
Posts:
3
Location:
Krakw

Copying files open for writing possible data loss

Hello everybody,
so I've been struggling with this problem with WinSCP. I'm using it to transfer files from a financial application to a banking system and every now and then accountants would complain that some information is missing from those files. It turned out that sometimes transfer would be attempted on a file that is still being generated (it takes on average 3-4 minutes to generate a file) and in such case WinSCP copies a part of the file generated sofar, and doesn't issue any warning (even though I use the -delete switch, meaning that I'm actually moving the file). This is a classical case of data loss.

I asked Martin whether this could be fixed, and his answer was Maybe if more people ask for it.. So, here I am, asking if you could support me in this case.


Description of an issue:

WinSCP allows to copy a file that is opened for writing. This can be useful at times (for example you can copy an Excel sheet that is open), but it can lead to data loss: imagine a situation where a program fetches data from database and saves it to file. If you copy a file while it's not yet ready, you'll get a copy that is missing some part of data, but since you got no warning, you may think that the file is complete. This is even trickier when a transfer is done automatically, periodically from script (which WinSCP is capable of doing and thus is predestined to support).


Proposed solution:

1) On attempt to copy a file that is opened for writing, issue a warning, with possibility to continue or skip.
2) Point 1 should be configurable with possibility to disable it, but enabled by default (I don't insist on the last one).
3) On attempt to move a file that is opened for writing, issue a warning, with possibility to continue or skip. If user presses skip, file shouldn't be copied at all (treat move as one logical operation, not two separate copy+delete).


Out of many file managers I tested (in their default settings):
1) File Commander by Brian Havard: warns when copying a file open for writing (allows to continue), and errors when moving such file (copy is not created too).
2) Free Commander by Marek Jasinski: allows copying such file (without a warning), disallows moving such file (copy is not created too)
3) FAR Manager: allows copying such file (without a warning), disallows moving such file (copy is not created too)
4) Unreal Commander: allows copying such file (without a warning), disallows moving such file (copy is not created too)
5) Total Commander: allows copying such file (without a warning), while moving such file, a copy is created, original not deleted, no warning
6) Windows Explorer: allows copying such file (without a warning), disallows moving such file (copy is not created too)

As I'm not a programmer (except for PHP webpages, which doesn't really count in this case, does it?), I'm unable to code a patch myself. But from what I know the fix is rather simple: shouldn't take much time to implement and shouldn't interfere with current functionality.

Regards for all.

Reply with quote

Advertisement

Stajkowski
Guest

Re: Copying files open for writing � possible data loss

trybowski wrote:

Hello everybody,
so I've been struggling with this problem with WinSCP. I'm using it to transfer files from a financial application to a banking system and every now and then accountants would complain that some information is missing
Hello,

I have exactly the same problem in our application. We loose part of data. Definietely a file open for writing should not be available for copying.
Such fix would be great relief.

Kind Regards
Stajkowski

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
28,507
Location:
Prague, Czechia

Re: Copying files open for writing � possible data loss

Stajkowski wrote:

I have exactly the same problem in our application. We loose part of data. Definietely a file open for writing should not be available for copying.
Such fix would be great relief.
Do you mean that WinSCP should fails uploading a file that is opened for writing by another application? It's up to the writing application to decide whether to prevent other applications from reading it.
_________________
Martin Prikryl

Reply with quote

gaurav.dutt
Guest

The tranfer should fail if the file is being written

We are facing the same situation, the transfer should fail if the file being written.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
28,507
Location:
Prague, Czechia

Re: The tranfer should fail if the file is being written

gaurav.dutt wrote:

We are facing the same situation, the transfer should fail if the file being written.
Again, the writing application, when opening the file decides whether it allows other application reading it or not. If the application did not protect the file against reading, although it's not safe to (what should be done almost always, except maybe for a log files), it's fault of the writing application.
_________________
Martin Prikryl

Reply with quote

Advertisement

You can post new topics in this forum