@Flaster: Your connection latency is probably high.
If you have a shell access to the server, you might use shell rm command to to delete the files remotely. If this works for you, you can configure a custom command to make it easier to run the rm:
https://winscp.net/eng/docs/custom_commands#delete

Probably going to be one of two possibilities.

1. Increase connection count.
   
2. Server doesn't allow multiple connections.
   

It's also possible that WinSCP, or the connection type, or the way its used, won't do anything further when another way might. Such as command line tends to be more restricted than GUI.

You really didn't provide enough info.

I am busy removing files, but it go's 1 by 1.

How can we fix this to make it faster?

Thanks!