S3 Transfer Automation > The specified bucket does not exist :: Support Forum

blackbishop313 Guest

S3 Transfer Automation > The specified bucket does not exist

2018-11-14 11:26

Hello,
I'm trying to use WinSCPnet to upload files to S3 using script task in SSIS.
This is my code (using Winscp version 5.14.1.beta) :

// Set up session options
SessionOptions sessionOptions = new SessionOptions
{
    Protocol = Protocol.S3,
    HostName = "<bucketName>.s3-eu-west-1.amazonaws.com",
    UserName = "*****************",
    Password = "***************************",
};
using (Session session = new Session())
{
    session.Open(sessionOptions);
    //logic here
}

The session is established but when I try tu put files I get this error :

The specified bucket does not exist

Here's my code :

string  LocalDirectoryPath = "C:\path\to\file";
string  RemoteDirectoryPath = "/<BucketName>/";
bool    DeleteSourceFiles = false;
TransferOptions transferOptions = new TransferOptions();
session.PutFiles(LocalDirectoryPath, RemoteDirectoryPath, DeleteSourceFiles, transferOptions);

When I change RemoteDirectoryPath to "/" I get another error :

WinSCP.SessionRemoteException: Copying files to remote side failed. Specify target bucket

In the WinSCP UI I have the same behaviour. If I use the following configuration it works :

HostName = "s3-eu-west-1.amazonaws.com"
Remote directory = "/<bucketName>"

But if I use :

HostName = "<bucketName>.s3-eu-west-1.amazonaws.com"
Remote directory = "/"

And try to drag some file I get this error :

Copying files to remote side failed. Specify target bucket

Could you please take a look at this ? Thank's

The logs generated by WinSCP :

logs.log (27.3 KB)

Description: logs generated by Winscp

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,441
Location:: Prague, Czechia

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-16

blackbishop313 wrote:
In the WinSCP UI I have the same behaviour. If I use the following configuration it works :
HostName = "s3-eu-west-1.amazonaws.com"
Remote directory = "/<bucketName>"

This is the correct solution. So use it in .NET assembly too.

Reply with quote

blackbishop
Joined:: 2018-11-14
Posts:: 5
Location:: Paris

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-16 10:08

martin wrote:
blackbishop313 wrote:
In the WinSCP UI I have the same behaviour. If I use the following configuration it works :
HostName = "s3-eu-west-1.amazonaws.com"
Remote directory = "/<bucketName>"
This is the correct solution. So use it in .NET assembly too.

Thank you for your reply.
I could not find the equivalent "Initial Remote Directory" in the .NET assembly.
If I try to connect using only :

HostName = "s3-eu-west-1.amazonaws.com"

I get this error :

WinSCP.SessionRemoteException: Access denied. Access Denied Extra Details: RequestId: 280C45C8BB312493, HostId: plUn**************rMlQA/pN************CEc/LR8******= Connection failed

With UI this is set in Advanced Site Settings dialog (https://winscp.net/eng/docs/ui_login_directories)
How could one do the same with the assembly code?
Thanks

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,441
Location:: Prague, Czechia

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-16

You already specify your bucket name in RemoteDirectoryPath that you pass to Session.PutFiles. That should be enough.
If that does not work, please attach a full session log file showing the problem (using the latest version of WinSCP).

To generate the session log file, set Session.SessionLogPath. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

blackbishop
Joined:: 2018-11-14
Posts:: 5
Location:: Paris

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-16 10:39

No, it is not enough because the connection isn't even established.
The error occurs when I call session.Open(sessionOptions);, so before the Session.PutFiles is executed.
The same happens in the UI ; if I do not specify initial remote directory I get the same error.

I tried to use <bucketName>.s3-eu-west-1.amazonaws.com to bypass that behaviour but it resulted in another weird behaviour which is explained in the first post.

Here's the session log generated by WinSCP 5.14.1 beta :

full_session.log (9.43 KB)

Description: full session log file showing the problem

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,441
Location:: Prague, Czechia

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-20

Sorry, but I got lost. You wrote before:

blackbishop313 wrote:
In the WinSCP UI I have the same behaviour. If I use the following configuration it works :
HostName = "s3-eu-west-1.amazonaws.com"
Remote directory = "/<bucketName>"

So does it work or not? Or does it work partially only? Can you elaborate?

Anyway, try this:

sessionOptions.AddRawSettings("RemoteDirectory", "/bucket");

See https://winscp.net/eng/docs/faq_script_vs_gui

Reply with quote

blackbishop
Joined:: 2018-11-14
Posts:: 5
Location:: Paris

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-26 16:27

Sorry if I haven't give good explanation of the problem.

IN WinSCP UI
Working configuration : Everything works fine (connection and files transfer)

HostName = "s3-eu-west-1.amazonaws.com"
Initial Remote directory = "/<bucketName>"

Not working configuration : The connection works but error occurs when I try to put files

HostName = "<bucketName>.s3-eu-west-1.amazonaws.com"
Initial Remote directory = "/"

IN .NET ASSEMBLY
The connection is established correctly but when the PutFiles is executed it generates an exception.
I tried sessionOptions.AddRawSettings("RemoteDirectory", "/bucket"); but with no success too.

SessionOptions sessionOptions = new SessionOptions
{
    Protocol = Protocol.S3,
    HostName = "s3-eu-west-1.amazonaws.com",
    UserName = "*****************",
    Password = "***************************",
};
sessionOptions.AddRawSettings("RemoteDirectory", "/<bucketName>");
using (Session session = new Session())
{
    session.Open(sessionOptions);
    session.PutFiles(LocalDirectoryPath, "/", DeleteSourceFiles, transferOptions);
}

Hope this is clear for you.
Thank you for your help.

Reply with quote

blackbishop
Joined:: 2018-11-14
Posts:: 5
Location:: Paris

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-26 17:53

I finally got it to work by changing this ligne :
session.PutFiles(LocalDirectoryPath, "/", DeleteSourceFiles, transferOptions);

with this one :
session.PutFiles(LocalDirectoryPath, "*", DeleteSourceFiles, transferOptions);

The * did the trick.
So it's OK for sessionOptions.AddRawSettings("RemoteDirectory", "/bucketName");

Thank you :)

blackbishop wrote:
IN .NET ASSEMBLY
The connection is established correctly but when the PutFiles is executed it generates an exception.
I tried sessionOptions.AddRawSettings("RemoteDirectory", "/bucket"); but with no success too.
SessionOptions sessionOptions = new SessionOptions
{
    Protocol = Protocol.S3,
    HostName = "s3-eu-west-1.amazonaws.com",
    UserName = "*****************",
    Password = "***************************",
};
sessionOptions.AddRawSettings("RemoteDirectory", "/<bucketName>");
using (Session session = new Session())
{
    session.Open(sessionOptions);
    session.PutFiles(LocalDirectoryPath, "/", DeleteSourceFiles, transferOptions);
}

Last edited by blackbishop on 2018-11-27 14:45; edited 1 time in total

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,441
Location:: Prague, Czechia

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-27

blackbishop wrote:

I finally got it to work by changing this ligne :
session.PutFiles(LocalDirectoryPath, "/", DeleteSourceFiles, transferOptions);

with this one :
session.PutFiles(LocalDirectoryPath, "/bucketName/*", DeleteSourceFiles, transferOptions);

The * did the trick.

Are you sure it's the * that "did the trick"? It should work even without the *:
session.PutFiles(LocalDirectoryPath, "/bucketName/", DeleteSourceFiles, transferOptions);

Reply with quote

blackbishop
Joined:: 2018-11-14
Posts:: 5
Location:: Paris

Re: S3 Transfer Automation > The specified bucket does not exist

2018-11-27 14:51

martin wrote:

Are you sure it's the * that "did the trick"? It should work even without the *:
session.PutFiles(LocalDirectoryPath, "/bucketName/", DeleteSourceFiles, transferOptions);

Sorry I meant :
session.PutFiles(LocalDirectoryPath, "*", DeleteSourceFiles, transferOptions);

Anyway this also works fine (just tested it):
session.PutFiles(LocalDirectoryPath, "/bucketName/", DeleteSourceFiles, transferOptions);

The only one that doesn't work is when using "/" as remote directory in the PutFiles method.

Reply with quote