That's indeed true.

A request to improvement is being tracked already:
Issue 1027 – Refresh remote panel after background transfer finishes even when directory caching is off
You can vote for it there.

OK, I can see that remote directory listening is not refreshed only when background transfer is used. I'm just moving one or more files from/to remote directory, and when queue processing is finished remote directory listening is not refreshed. Maybe in this case it is intended behaviour? Can you reproduce that?

I've just tested it and it seems to work. The directory is refreshed after operation (upload, delete, rename), even with both caching options turned off. Can you give us more detailed instructions for reproducing the problem? Also, please post a log file.

The option "Automatically refresh directory after operation (Ctrl+Alt+R)" don't work when options
"Cache visited remote directories"
and "Cache directory changes"
are disabled.

That's a little contradiction in my opinion. Is it bug or intended behaviour?
Tested on 6.5.6 and 6.6.1 beta.