Ahh, I just found out it IS possible, by pasting in the path. I am happy that my ignorance was the resolution for this!

And what prevents you from browsing them?

It would be wonderful to be able to transfer files created in WSL2, which appear in the Windows File System as the wsl$ network share. As it is I need to copy my output directory to another directory on my C:\ drive before I can do the transfer. Either that or fight with Filezilla, which isn't fun.