Topic "Applying filters to directories in "Keep Remote Dir Updated""

Author Message
gavbaa
[View user's profile]
Donor
Joined: 2005-02-10
Posts: 3
First and foremost, thank you for this amazing product. I've already donated for my personal use, and want to bring it in for the business I've recently started at, pending a fix that may or may not be possible.

We use Subversion here, a source code revisioning tool. This tool leaves a ".svn" folder in every directory in our working source code tree (it's web development, so we have a working copy on our local machines, and we upload on save to the server for testing. We do it this way because it works both in the office and on the road, so emergency development can happen anywhere we can check out the source code from). So, when we're using "Keep Remote Directory Up to Date", every time we do something with the repository (checkout/update/delete/etc.), WinSCP goes insane, uploading lots of little files that are stored in those .svn directories. It doesn't crash, it just takes a long while to deal with the update.

To give you an example of the directory structure we're talking about:


Code:
(root)
 |->file_a.txt
 |->file_b.txt
 |->.svn
   |->entries
   |->...(more files here)...
 |->dir_c
   |->file_c.txt
   |->.svn
     |->entries
     |->...(more files here)...


Every subdirectory have its own ".svn" folder with stuff inside of it.

We don't really care about those files, and would prefer if the .svn folders and everything in them weren't even uploaded to the servers. We've already tried the filters significantly, and there does not seem to be a way to make the filters ignore .svn *directories*.

The other problem these directories create is that inside these .svn directories are many more directories that we also don't care about. They cause the entire site tree directory count to bloat severely. Now, I've already built my own slightly modified version from your source code that allows me to use many MANY more than 500 directories (and it works just fine, doesn't seem to bog the system down too much). If we didn't have to even check those .svn folders at all, the directory count would be sooo much lower, and definitely within the safety range of 500 directories. But all those .svn/* files changing all the time are causing definite lags of minutes before we can continue getting our code pushed up. Regular working and saving goes just fine and snappy.

If there is some way that we can get the filter to apply to directories as well, it's the last piece that I can find to make our development environment magical. Smile

Thanks,
George
Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24530
Location: Prague, Czechia
If you are willing to apply the patch yourself, here is quick and dirty fix to TCustomScpExplorerForm::DoSynchronize in CustomScpExplorer.cpp:

Code:
void __fastcall TCustomScpExplorerForm::DoSynchronize(
  TSynchronizeController * /*Sender*/, const AnsiString LocalDirectory,
  const AnsiString RemoteDirectory, const TSynchronizeParamType & Params)
{
  try
  {
    TCopyParamType CopyParam = GUIConfiguration->CopyParam;
    CopyParam.PreserveTime = true;

    AnsiString Path = LocalDirectory;
    bool Allow = true;
    while (Allow && !Path.IsEmpty())
    {
      Path = ExcludeTrailingBackslash(Path);
      Allow = CopyParam.AllowTransfer(ExtractFileName(Path));
      AnsiString ParentPath = ExtractFilePath(Path);
      if (Path == ParentPath)
      {
        Path = "";
      }
      else
      {
        Path = ParentPath;
      }
    }
   
    if (Allow)
    {
      Synchronize(LocalDirectory, RemoteDirectory, smRemote, CopyParam,
        Params.Params | TTerminal::spNoRecurse | TTerminal::spUseCache |
        TTerminal::spDelayProgress);
    }
  }
  catch(Exception & E)
  {
    ShowExtendedExceptionEx(Terminal, &E);
    throw;
  }
}


This way it will still watch for changes in filtered out directories (that's why it's dirty), but it won't upload them. BTW: I suppose that you are using GUI, not scripting, and this won't fix "keep up to date" in scripting.
_________________
Martin Prikryl
gavbaa
[View user's profile]
Donor
Joined: 2005-02-10
Posts: 3
First, I added the filter of ".svn". That didn't work, it still checked all the .svn folders. So I connected to the server, removed all the .svn folders on there. Be-yoooo-tiful! I don't know why that made a difference, but I believe that was the only change I had made up to that point.

I do notice that there's more drive activity than I would suspect out of the small file tree, but it's a small price for now for the functionality.

Thank you!
George
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24530
Location: Prague, Czechia
gavbaa wrote:
First, I added the filter of ".svn". That didn't work, it still checked all the .svn folders. So I connected to the server, removed all the .svn folders on there. Be-yoooo-tiful! I don't know why that made a difference, but I believe that was the only change I had made up to that point.

Hm, strange, I do not think that it should make difference.
_________________
Martin Prikryl
plsnhat

Guest


Hey there,

I am also very thankful for this great product (: At the moment I am looking for a possibility to keep more than 500 directories up to date and it seems that you, gavbaa, implemented a solution (: Would you mind to tell me how to do this?

You would do me a great favor!

Thanks,
Phil
gavbaa
[View user's profile]
Donor
Joined: 2005-02-10
Posts: 3
Unfortunately, I have since moved on from the company where I developed that, so I will be unable to produce any additional patches besides what was in this thread. However, I recall 99% of the work outside of applying the patch listed above just being finding the 500 constant in the source and increasing it, then re-compiling. Should be straightforward.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24530
Location: Prague, Czechia
plsnhat wrote:
I am also very thankful for this great product (: At the moment I am looking for a possibility to keep more than 500 directories up to date and it seems that you, gavbaa, implemented a solution (: Would you mind to tell me how to do this?

Where does the 500 limit restrict you? I believe WinSCP asks you if you want to continue nevertheless.
_________________
Martin Prikryl
Guest




prikryl wrote:
plsnhat wrote:
I am also very thankful for this great product (: At the moment I am looking for a possibility to keep more than 500 directories up to date and it seems that you, gavbaa, implemented a solution (: Would you mind to tell me how to do this?

Where does the 500 limit restrict you? I believe WinSCP asks you if you want to continue nevertheless.


I store my projects locally and on a webserver which needs to be updated everytime I want to see an online version of a project. It may happen that just one file in any subdirectory of the project changes. So in order to not have to copy the whole project again I use WinSCP which determines the files which changed.

Well, it asks me whether I want to select other directories to be monitored and also offers the possibility to say "no", but doing this just closes the "keep up to date"-dialogue.

Thanks a lot for your help ! (:
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24530
Location: Prague, Czechia
I believe there's "Never ask me again" checkbox on the confirmation dialog.
_________________
Martin Prikryl
plsnhat

Guest


Oh, didn't see that, I'll try to look more precisely next time, thanks! (:
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License