Allow Login to AWS Athena with Profile from Credentials File

Advertisement

Stusstrupp
Joined:
Posts:
4
Location:
Cologne, Germany

Allow Login to AWS Athena with Profile from Credentials File

For security reasons, organisations prefer users to access AWS services via an credentials provider and the assumption of a role rather than with static credentials with a user role. For locally installed applications, this is possible by copying the credentials provided by the organisation's SSO service into the local AWS configuration file:
# temporary credentials to access the AWS account for
# user role arn:aws:iam::123456789012:role/user-role-<userid>
# from here:
# https://myappcallingcognito.mydomain.com/ssocredentials
 
[my_temp_aws_account_credentials]
aws_access_key_id = ...
aws_secret_access_key = ...
aws_session_token = ...
 
[athena_access_project_x]
# user role arn:aws:iam::123456789012:role/user-role-<userid>
# needs to be set up to be able to assum efollowing role:
role_arn = arn:aws:iam::123456789012:role/project-x-read
source_profile = my_temp_aws_account_credential
region = eu-central-1
In order to login, the user then simply needs to enter the profile name ("athena_access_project_x" in this example) and is connected until the credentials expire. This also allows the organisation in question to manage their users in just one directory rather than also having to manage them in AWS IAM.

An option to connect to AWS Athena from WinSCP by entering the profile name from the credentials file would be greatly appreciated.

Reply with quote

Advertisement

Stusstrupp
Joined:
Posts:
4
Location:
Cologne, Germany

Re: Allow Login to AWS Athena with Profile from Credentials File

Hi,

Indeed we need to connect not just with the temporary credentials of the sso role (as assumed with [my_temp_aws_account_credentials] in the example) but with a particular role.

Our S3 repository is organised along projects. Rather than managing as many different user roles as we have different access permutations (just project a, just project b, both project a and b, project c, ...), we manage one role per project. Each of those project roles are then assigned the proper SSO roles in their trust relationship. That means that when a user logs in, he has to select the proper project role (athena_access_project_x in the example above).

AWS CLI allows to use preconfigured role assumption with the credentials file, as shown in the example and described in https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-role.html, and our developers use this with VSCode. Data analysts however do not need a full fledged development IDE, so we wanted to offer them WinSCP.

Reply with quote

Stusstrupp
Joined:
Posts:
4
Location:
Cologne, Germany

Re: Allow Login to AWS Athena with Profile from Credentials File

Hi,

And thank you for you keeping at this request. Reading the corresponding forum entry to the request you mention (Allow S3 connection with IAM role instead of credentials), it seems to me that that requirement is similar, but not as simple as mine: It sounds as if that requirement is asking for the user to enter a role for an account that his access key points to.

I am just asking for WinSCP to allow entering the name of a profile out of his credentials file – maybe even select it from a list of profiles in that file. The role name would then be taken from the profile description within the credentials file (see example in original post).

In other words, I would like WinSCP to be able to use the --profile option of AWS CLI, e.g. like
aws s3api list-buckets --profile <profile name>

There were two feature requests I thought covered this, S3 - Use credentials from environment variables and Security Token Service support for AWS S3.

However, the implemented solution (Issue 1941 – Support reading S3 credentials from AWS CLI configuration) does not allow entering the profile name from the credentials file. It seems to me that just the default profile is being read, and then without any roles it is supposed to assume.

I am aware that you are using libs3 rather than AWS CLI and I have found no support of reading profiles or logging in with a role in its documentation.

Reply with quote

Advertisement

martin
Site Admin
martin avatar

Re: Allow Login to AWS Athena with Profile from Credentials File

Thanks for the clarification. I'll look into it.

Reply with quote

martin
Site Admin
martin avatar

Re: Allow Login to AWS Athena with Profile from Credentials File

Would you be able to setup a test account, role and bucket for me? Or provide me step-by-step instructions (like aws commands) to do that myself?

Btw, I've implemented the profile selection already. But the "role" support is still pending.

Reply with quote

Stusstrupp

Re: Allow Login to AWS Athena with Profile from Credentials File

I have tried to get this approved by our IT security, however setting you up for our corporate AWS accounts would require a huge effort.

Reply with quote

Advertisement

virshu
Joined:
Posts:
7
Location:
US

@Stusstrupp - and I thought my request was the simplest one ;)
All I envisioned that when WinSCP runs on EC2 instance, the user does not need to enter any credentials, assuming that the instance has the role that allows it to access S3

Reply with quote

virshu
Joined:
Posts:
7
Location:
US

Hi @Martin -
I thought I explained it in my forum post last year. Obviously, I wasn't successful. First, let me repeat – When S3 connection runs on AWS instance (called EC2), it is best practice to *not* use the credentials but use instance profile instead.
("instance profile" and "role" are synonymous for the purposes of this conversation).

Instance Profile is attached to the instance itself; not to individual user that is logged on to the instance. Practically, this means that the user DOES NOT specify any credentials. AWS takes care of the authentication. AWS CLI and APIs work exactly the same. Obviously, it only works on EC2 instances – but copying files between EC2 and S3 are very comment scenario.

I am happy to explain more and in more details (I wrote a number of programs in .NET doing exactly that...) – just ask!

Reply with quote

martin
Site Admin
martin avatar

@virshu: Please explain more. Or even better, post a link to API that needs to be implemented. Or is it just enough not to include the credentials to the AWS requests?

Reply with quote

Advertisement

virshu
Joined:
Posts:
7
Location:
US

Sorry – I missed the latest post.
API is exactly the same – but without specifying any credentials. For example, CLI command would be something like
aws s3 cp myfile.txt s3://mybucket/dir1/
and if I am using ASP.NET Core, then here is an example from GitHub:
https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/S3/S3_Basics
IAmazonS3 client = new AmazonS3Client();
var request = new PutObjectRequest
    {
        BucketName = bucketName,
        Key = objectName,
        FilePath = filePath
     };
var response = await client.PutObjectAsync(request);
The important part is that the code doesn't have to specify any credentials! Instance profile takes care of that.

Reply with quote

martin
Site Admin
martin avatar

@virshu: Thanks. Though I was referring to the protocol-level API. WinSCP is not using AWS SDK.

Anyway, can you help me setup the instance profile, so I can test this myself? What do I do?

Reply with quote

virshu
Joined:
Posts:
7
Location:
US

There are two parts.
First, you need to create a role. Here is a very good description of how to do it. Essentially, you create a new IAM role, and then attach existing AmazonS3FullAccess policy to it. Obviously, for testing at some point you may want to try a different policy – AmazonS3ReadOnlyAccess or create your own bucket-specific policy. But it's perfect to start.

Second, you launch an instance, and under IAM Role you choose the role that you created in step 2. Here is more detailed walkthrough

Once you have the instance with the role, you can log in to it and access S3 without any additional credentials. Note, that any changes that you make in the role, take effect immediately.

Hopefully, that helps

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
40,587
Location:
Prague, Czechia

What why? Why do I have to look at it? Because I had to implement the retrieval of credentials from AWS metadata service. It does not "just work". I've explained that above already. It just works for you, because you use AWS SDK, which implements the retrieval for you. But WinSCP has its own implementation of AWS API.

Anyway, I'm sending you an email with a development version of WinSCP to the address you have used to register on this forum.

Reply with quote

antpas
Guest

WinSCP with IAM Role

I see that access to S3 with IAM Role has been addressed in 6.2 beta. I've downloaded it for testing. However, I am unable to find any setting related to setting up IAM role. Can you please post instructions on how to perform IAM Role authentication with WinSCP 6.2 beta?

Reply with quote

martin
Site Admin
martin avatar

Re: WinSCP with IAM Role

@antpas: Just check Credentials from AWS environment on the Login dialog. If the machine has an IAM role, its ephemeral credentials will be retrieved.

Reply with quote

Advertisement

You can post new topics in this forum