Option to compare file contents during sync (+ time/size)

Advertisement

Ric
Guest

Option to compare file contents during sync (+ time/size)

I need an option to compare file contents (binary) when performing a sync/mirror operation - instead of only having "modification time" and "file size" in the comparison criteria section.

(The extra overhead to perform this accurately is not an issue for me, as all use is only within my LAN - nothing goes over the internet.)

I need to be able to confirm with 100% certainty that the data integrity is good and matches exactly for all files, and that no possible file variations have been missed due to RAM/system/drive failure/glitch, network corruption, etc. So actually downloading every file from the FTP server and doing a full contents comparison to the local files is the only way to assure this.

Thanks for the great program! Cheers, Ric.

Reply with quote

Advertisement

Ric
Guest

Re: Option to compare file contents during sync (+ time/size)

martin wrote:

Thanks for your post. This request is being tracked already.

Hi Martin. Thanks for your reply.
I've read over the messages about this at the tracking page. There seem to be two different reasons why people want these features:

(1) For determining what should be synched/updated - which as has been commented there - may as well just copy the whole lot anyway for a mirror operaton as that may be faster! (And also tends to make this feature pointless.)

(2) My purpose: checking/verifying data integrity. This forbids copying over the current data - it is first to be fully/thoroughly compared with the other data then prompt user what to do (if anything) over any differences found - same as with the current time/size methods.

Implementation of this would be quickest if the remote files are not saved to the local disk at all yet - only buffered in RAM (full or part depending on size) and reading in the local files (likewise full or parts of it at a time depending on size) to RAM and compare them in RAM - this would prevent thrashing the hard-drive over both files. If after a comparison is done, the user wants to copy the remote data over the local file(s) or vice-versa, then it has to be read in full again as they are not cached. This may sound inefficient, but in most cases few or no duplicate file-transfers would actually be done when using this full contents comparison method, because the other file/size method is adequate to be used before this one, in order to copy over the obvious differences first, so it's never going to be a problem. (If the user says to compare time, size and contents, then the full contents comparison may as well be skipped if time or size are obviously different.) And if there are any differences, then it may need further looking into via other means, since there's possibly something going seriously wrong, so any further acion inside winSCP may be cancelled, then check the problem file(s), then re-run WinSCP and possibly copy over any file(s) with issues earlier after knowing which one is correct.

Also as has been discussed, file-hashing at the remote server is not an option with a typical FTP-server - and that method is not how I would want to do it anyway, though hashing is potentially useful for some people and could be added as a fourth comparison method - where available - so then everyone's requirements/preferences can be met with this.

I have 260GB (and growing) of important data used on a primary system which I regularly mirror to laptops and "dock" HDDs, and am a bit paranoid about data loss/corruption. (After having issues with (often intermittent) faulty hardware, dodgy connections, and software bugs, it's paid to be like this. ;-)

It'd be fantastic for me if WinSCP supported a full binary contents comparison as I've suggested. Until such time, I also use other software for this which isn't as good as yours, but which has some other/extra feature, thus I can't give them up entirely just yet (but I'd like to :-).

Cheers, Ric.

Reply with quote

TC
Guest

Hello,

it would be very nice for me too..., because I need also this feature (binary file compare)!

Both other cases (criteria= size & time) don't meet my requirements..!

Is the implementation of this feature already planned?

Thank you for your answer!

Regards
TC

Reply with quote

martin
Site Admin
martin avatar

TC wrote:

Is the implementation of this feature already planned?
It is in the tracker, with low priority. That's all can say about it.

Reply with quote

Advertisement

You can post new topics in this forum