Problem transferring many files

Advertisement

n3c086
Joined:
Posts:
2

Problem transferring many files

I have a folder with about 118,000 pdfs. This involves extremely slow opening of this folder, so I would like to move the files prior to 2022 to a different folder. This operation stops shortly after starting.
Is there a solution to be able to complete the operation?
tnx

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,040
Location:
Prague, Czechia

Re: problem transferring many files

So do you have problem with transferring or moving?

Please attach a full session log file showing the problem (using the latest version of WinSCP).

To generate the session log file, enable logging, log in to your server and do the operation and only the operation that causes the error. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

n3c086
Joined:
Posts:
2

Re: problem transferring many files

It's a file moving.

I have attached the log file.
I deleted some lines from the file list as the file was too large.
tnx
  • FTP@.rar (6.16 MB, Private file)

Reply with quote

martin
Site Admin
martin avatar

Re: problem transferring many files

Please post complete log. Or at least the part of the log after the file list.

Reply with quote

Muso
Joined:
Posts:
4

Can only transfer certain filetypes without error - help!

Hi all!

I've set up a musicbox on a Raspberry Pi for an oldie neighbour. As the SD card holding the mp*three*s is ext4 his Windows PC can't write to that (when you pull the SD card out of the Pi & into USB-reader on PC). So went down the SSH route (PC and Pi communicating via their wifi network). Login, no probs, commandline, no probs. WinSCP connection, no probs. I can see local PC & remote Pi file directory no problem. But I can't transfer .mp*three*s from PC to Pi via WinSCP.

I can:
  • copy .txt, .xls, .jpg (tried a few types, all worked)
  • delete files on the remote Pi
  • I can copy .mp*three* files via commandline 'scp' command outside WinSCP
I can not:
  • copy .png or .mp*three* (tried a few, these didn't work)
I've tried to vary the transfer settings, but no change.

Can you help? I can't ask oldie neighbour to muck around with command line....

Thanks!

Reply with quote

Advertisement

martin
Site Admin
martin avatar

Re: Can only transfer certain filetypes without error - help!

@Muso: Well, if you can transfer some files and some not, then it does not look like WinSCP problem. What protocol are you using in WinSCP? SFTP or SCP? Did you try the other one? Can you transfer all files using any other (GUI) client using the same protocol? You didn't really tell us anything about the "error" that you are experiencing. Please attach a full session log file showing the problem (using the latest version of WinSCP).

Reply with quote

Muso
Joined:
Posts:
4

Still can't transfer certain files from local windows machine to raspPi...

Hi Martin,

- thanks for replying!

Trying to trouble-shoot I have discovered it is not the filetype, but rather the filesize. Small files (<20kB) I can copy no problem, larger ones (>20kB) cause the connection to abort & only a 0kB part of the file I try to copy over gets created (after reconnecting session I can see that on the raspPi receiver side. Have tried FTP & SCP in WinSCP – both have the same issue. Commandline SCP copying (in windows cmd terminal) works no problem.

Attached log for 2 files, a small one (MG.jpg - 18kB, works) and a bigger one (MS.jpg - 90kB, does not work).

Help much appreciated!

Reply with quote

Muso
Joined:
Posts:
4

Buffer optimisation - still does not solve it.

Hi Martin,

thanks for the pointer, unfortunately same issue (file transfer > ca. 20kB) persists. Tried SCP & SFTP (both with 'buffer optimisation' unchecked).

Attached log with 1 (small) file working & 1 (bigger) file causing connection to abort.

Help much appreciated!

Reply with quote

Advertisement

martin
Site Admin
martin avatar

Re: Buffer optimisation - still does not solve it.

Can you enable Debug 1-level session logging?
Why is the SFTP/SSH server "dropbear_2012.55"? That does not look very up-to-date. Doesn't Raspberry Pi have OpenSSH by default?

Reply with quote

Muso
Joined:
Posts:
4

Debug 1 level log.

Hi again,
sorry about the delay.
Attached the requested log. I've got a few small (kB-sized) test files, the small ones (< ca. 20kB) work, above that threshold, the connection error comes. Happy to to do anything to the raspberry pi that helps.
Thanks for your help! (my oldie neighbour will thank you as well!)

Reply with quote

Advertisement

You can post new topics in this forum