Add support for connecting to AWS S3 with IAM Roles that use IMDSv2 (6 hours server session keys)

Advertisement

goslackware
Joined:
Posts:
6
Location:
USA

Add support for connecting to AWS S3 with IAM Roles that use IMDSv2 (6 hours server session keys)

Requesting adding IAM-Role with IMDSv2 support for WinSCP to retrieve the AWS AccessKeyID and SecretAccessKey automatically when accessing S3 buckets. Such as adding a check box to try connecting with S3 with IAM-Role with IMDS-v2 support, or automatically detecting iam-role keys, or trying, etc.
Related issue: Issue 2089 – Allow S3 connection with IAM role instead of credentials

However, when instead using IMDSv2 with temporary 6 hour session keys (that is the server you're RDP into has an assigned IAM role to it with S3 permissions to your bucket), it would be beneficial if WinSCP would instead automatically manage the retrieving these keys.

As an example in PowerShell, here's the code to retrieve the temporary session credentials for connecting to S3 bucket.
# the below command will show the last 4 characters of the access_key and secret_key for the iam-role.  It also includes profile (if set), and AWS region.
aws configure list
 
# To get the full iam-role access_key and secret_key the below powershell code works:
# Step 1: Get IMDSv2 token
$token = (Invoke-WebRequest -Uri "http://169.254.169.254/latest/api/token" -Method PUT -Headers @{ "X-aws-ec2-metadata-token-ttl-seconds" = "21600" }).Content
 
# Step 2: Get IAM role name
$roleName = (Invoke-WebRequest -Uri "http://169.254.169.254/latest/meta-data/iam/security-credentials/" -Headers @{ "X-aws-ec2-metadata-token" = $token }).Content
 
# Step 3: Get temporary credentials
$credentials = (Invoke-WebRequest -Uri "http://169.254.169.254/latest/meta-data/iam/security-credentials/$roleName" -Headers @{ "X-aws-ec2-metadata-token" = $token }).Content | ConvertFrom-Json
 
# Output
$credentials
 
# In which $credentials will contain like below (creds redacted of course)
$credentials | gm
   TypeName: System.Management.Automation.PSCustomObject
Name            MemberType   Definition                                                                                                              
----            ----------   ----------                                                                                                              
Equals          Method       bool Equals(System.Object obj)
GetHashCode     Method       int GetHashCode()
GetType         Method       type GetType()
AccessKeyId     NoteProperty string AccessKeyId=*****
ToString        Method       string ToString()
Code            NoteProperty string Code=Success
Expiration      NoteProperty string Expiration=2025-01-23T00:30:21Z
LastUpdated     NoteProperty string LastUpdated=2025-01-22T18:19:59Z 
SecretAccessKey NoteProperty string SecretAccessKey=*****
Token           NoteProperty string Token=*****
Type            NoteProperty string Type=AWS-HMAC
 
# I haven't tested below, but it looks like it may work in powershell by setting $credentials to the below environmental variables.  Also could set a profile, etc.
$env:AWS_ACCESS_KEY_ID = $credentials.AccessKeyId
$env:AWS_SECRET_ACCESS_KEY = $credentials.SecretAccessKey
$env:AWS_SESSION_TOKEN = $credentials.Token
 
# Verify
aws sts get-caller-identity

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,717
Location:
Prague, Czechia

Re: Add support for connecting to AWS S3 with IAM Roles that use IMDSv2 (6 hours server session keys)

Sorry, I'm not too familiar with all this. Is what WinSCP is currently doing IMDSv1? What is the advantage in using IMDSv2? Is IMDSv1 not supported in some scenarios?

Reply with quote

goslackware
Joined:
Posts:
6
Location:
USA

Re: Add support for connecting to AWS S3 with IAM Roles that use IMDSv2 (6 hours server session keys)

Hi Martin!

Thank you for the reply.
To your question, there's security protections in IMDSv2.
Many security concerned environments are required to disable IMDSv1, and use only IMDSv2.

From AWS Docs
https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/
With IMDSv2, every request is now protected by session authentication. A session begins and ends a series of requests that software running on an EC2 instance uses to access the locally-stored EC2 instance metadata and credentials. The software starts a session with a simple HTTP PUT request to IMDSv2. IMDSv2 returns a secret token to the software running on the EC2 instance, which will use the token as a password to make requests to IMDSv2 for metadata and credentials. Unlike traditional passwords, you don’t need to worry about getting the token to the software, because the software gets it for itself with the PUT request. The token is never stored by IMDSv2 and can never be retrieved by subsequent calls, so a session and its token are effectively destroyed when the process using the token terminates. There’s no limit on the number of requests within a single session, and there’s no limit on the number of IMDSv2 sessions. Sessions can last up to six hours and, for added security, a session token can only be used directly from the EC2 instance where that session began.
References:

Here's some examples of IMDSv1 and IMDSv2 for curl and PowerShell:

IMDSv1 Examples:

IMDSv2 Examples:

Reply with quote

goslackware
Joined:
Posts:
6
Location:
USA

possible code generated from AI for IMDSv2 feature

I asked https://chat.deepseek.com/ with "deep think" and "search" checked, what code changes would be needed to add IMDSv2 support, and below is the reply. No idea if the below code works or not.

To add IMDSv2 support to WinSCP, follow these steps:
  1. UI Changes (Login.dfm)
    Add radio buttons to select IMDS version within the login dialog.
    object MetadataVersionGroup: TGroupBox
      Caption = 'AWS Metadata Service Version'
      TabOrder = ... // Adjust based on existing layout
      object IMDSv1Radio: TRadioButton
        Caption = 'IMDSv1'
        Checked = True
        OnClick = MetadataVersionChange
      end
      object IMDSv2Radio: TRadioButton
        Caption = 'IMDSv2'
        OnClick = MetadataVersionChange
      end
    end

  2. Header File (Login.h)
    Declare UI components and variables in the TLoginDialog class.
    private:
      TRadioButton *IMDSv1Radio;
      TRadioButton *IMDSv2Radio;
      TGroupBox *MetadataVersionGroup;
     
      enum TMetadataVersion { mv1, mv2 };
      TMetadataVersion FMetadataVersion;
     
      void __fastcall MetadataVersionChange(TObject *Sender);

  3. Implementation (Login.cpp)
    Initialization in Constructor
    __fastcall TLoginDialog::TLoginDialog(TComponent* AOwner)
      : TForm(AOwner)
    {
      // Existing code
      FMetadataVersion = mv1; // Default to IMDSv1
    }

    Event Handler for Radio Buttons
    void __fastcall TLoginDialog::MetadataVersionChange(TObject *Sender)
    {
      FMetadataVersion = IMDSv1Radio->Checked ? mv1 : mv2;
    }

    Modify Metadata Retrieval Code
    Update the function fetching EC2 metadata to handle IMDSv2 token.
    UnicodeString GetEC2InstanceMetadata()
    {
      UnicodeString Token;
      if (FMetadataVersion == mv2)
      {
        THTTPClient *Client = new THTTPClient();
        TStringStream *Response = new TStringStream();
        try
        {
          Client->Request->Method = L"PUT";
          Client->Request->CustomHeaders->AddValue(L"X-aws-ec2-metadata-token-ttl-seconds", L"21600");
          Client->Put(L"http://169.254.169.254/latest/api/token", Response);
          Token = Response->DataString.Trim();
        }
        __finally
        {
          delete Response;
          delete Client;
        }
      }
     
      THTTPClient *Client = new THTTPClient();
      try
      {
        if (FMetadataVersion == mv2)
        {
          Client->Request->CustomHeaders->AddValue(L"X-aws-ec2-metadata-token", Token);
        }
        // Proceed with existing metadata requests (e.g., /latest/meta-data/...)
        TStringStream *Response = new TStringStream();
        Client->Get(L"http://169.254.169.254/latest/meta-data/...", Response);
        return Response->DataString;
      }
      __finally
      {
        delete Client;
      }
    }
Key Considerations
  • Error Handling: Implement try/catch around HTTP requests to handle failures (e.g., fallback to IMDSv1 if v2 fails).
  • Token TTL: Ensure tokens are requested fresh each session to avoid expiration.
  • Testing: Validate both IMDS versions under different AWS configurations.

Summary
  • UI: Added radio buttons for IMDS version selection.
  • Code: Integrated IMDSv2 token retrieval and header inclusion in metadata requests.
  • Robustness: Enhanced error handling for AWS metadata service interactions.

These changes enable WinSCP to support the more secure IMDSv2 protocol while maintaining compatibility with IMDSv1.

Here's the query I used:
What changes need to be made to the winscp source code to add support for IMDSv2?
The source code is at: https://github.com/winscp/winscp/tree/master/source/forms
The source code files of interest are:
source/forms/Login.cpp
source/forms/Login.dfm
source/forms/Login.h

Here's the "DeepThink R1" thoughts:
Okay, so I need to figure out what changes are required to add support for IMDSv2 in WinSCP. Let me start by understanding what IMDSv2 is. IMDS stands for Instance Metadata Service, and it's used in AWS EC2 instances. Version 2 of IMDS is more secure because it requires a session token that's obtained via a PUT request, which then needs to be included in subsequent requests as a header.

First, I should check how WinSCP currently handles IMDSv1. From what I know, IMDSv1 uses a simple GET request without any tokens. So, for IMDSv2, the process is different: you have to make a PUT request to get a token, then use that token in a header for the metadata requests.

Looking at the source files provided: Login.cpp, Login.dfm, and Login.h. These are part of the login form, so maybe the changes involve adding UI elements to enable IMDSv2 and modifying the code that retrieves the instance metadata.

Starting with the UI changes. In Login.dfm, which defines the form layout, there's probably a checkbox or a radio button group for selecting IMDS version. If there's already an option for IMDSv1, I might need to add a new option for v2. Alternatively, if there's no existing UI for IMDS, adding a new checkbox or a dropdown to choose between v1 and v2 would be necessary. Let's assume there's a section for AWS metadata settings. So, maybe adding a radio button with options for IMDSv1 and IMDSv2.

In the Login.h file, there would be variables or controls associated with the UI elements. For example, if adding a radio button group, there might be a TGroupBox or TRadioButton controls. So, adding new members to the TLoginDialog class for the radio buttons.

In Login.cpp, the code that handles the login process would need to be modified. When retrieving instance metadata, instead of just making a GET request, for IMDSv2, the code needs to first make a PUT request to http://169.254.169.254/latest/api/token with the header X-aws-ec2-metadata-token-ttl-seconds (usually set to 21600 seconds). Then, the returned token is used in subsequent GET requests with the header X-aws-ec2-metadata-token.

So, the steps would involve:
  1. Adding UI elements to select IMDSv2.

  2. Modifying the metadata retrieval code to check if IMDSv2 is selected.

  3. If so, perform the PUT request to get the token.

  4. Use that token in the headers when making the GET requests for metadata.
Now, looking into the existing code in Login.cpp. There might be a function that retrieves the EC2 instance metadata. Let's say there's a function like GetEC2InstanceMetadata(). This function would need to be updated to handle both IMDSv1 and v2.

In the case of IMDSv2, the code should first send a PUT request to get the token. The PUT request requires a specific header. The response from this request is the token string. Then, each subsequent metadata request (like getting instance ID, region, etc.) must include this token in the headers.

Potential issues: Handling timeouts for the token, error checking if the PUT request fails (fallback to IMDSv1?), and ensuring that the headers are correctly added to the metadata requests.

Also, the session token has a TTL, so maybe the code should generate a new token each time it's needed rather than reusing an old one, to avoid expiration issues.

For the UI changes, adding a radio group in the Login.dfm file with two options: "IMDSv1" and "IMDSv2". Let's say the current version uses IMDSv1 by default, so the radio button for v1 should be checked initially. The group could be labeled "AWS Metadata Service Version".

In Login.h, the class TLoginDialog might have a member like FMetadataVersion of an enum type (e.g., mdv1, mdv2). The radio buttons' OnClick events would set this variable accordingly.

In the code that retrieves the metadata (Login.cpp), check the FMetadataVersion variable. If it's mdv2, perform the token retrieval first. Then use that token in the headers for subsequent requests.

Testing would involve ensuring that when IMDSv2 is selected, the PUT and GET requests are correctly formed, and that the token is properly included. Also, ensuring backward compatibility with IMDSv1.

Another consideration: The PUT request to get the token might require a specific timeout. The code should handle cases where the PUT request fails (maybe because the instance isn't configured for IMDSv2) and fall back to IMDSv1 if possible, or show an error.

Additionally, some AWS setups enforce IMDSv2, so the code must not default to v1 in such cases. However, the UI would let the user choose, but maybe there's a way to detect if the server requires v2 and adjust accordingly. But that might be more complex.

So, the main changes are:
  • UI elements to choose IMDS version.

  • Code to handle token retrieval for v2.

  • Modify existing metadata requests to include the token header when v2 is selected.

  • Error handling for token retrieval failures.
Now, looking into the specific files:

In Login.dfm, adding the radio buttons. For example:
object MetadataVersionGroup: TGroupBox
  Caption = 'AWS Metadata Service Version'
  object IMDSv1Radio: TRadioButton
    Caption = 'IMDSv1'
    Checked = True
    OnClick = MetadataVersionChange
  end
  object IMDSv2Radio: TRadioButton
    Caption = 'IMDSv2'
    OnClick = MetadataVersionChange
  end
end
In Login.h, adding the radio button variables and the event handler:
private:
  TRadioButton *IMDSv1Radio;
  TRadioButton *IMDSv2Radio;
 
  void __fastcall MetadataVersionChange(TObject *Sender);
And a variable to track the selected version:
enum TMetadataVersion { mv1, mv2 };
 
TMetadataVersion FMetadataVersion;
In Login.cpp, in the constructor or initialization, setting the default:
FMetadataVersion = mv1;
The event handler:
void __fastcall TLoginDialog::MetadataVersionChange(TObject *Sender)
{
  if (IMDSv1Radio->Checked)
    FMetadataVersion = mv1;
  else
    FMetadataVersion = mv2;
}
Then, in the function that retrieves metadata:
UnicodeString token;
 
if (FMetadataVersion == mv2) {
  // Make PUT request to get token
  THTTPClient client;
  client.Request->Method = "PUT";
  client.Request->CustomHeaders->Add("X-aws-ec2-metadata-token-ttl-seconds: 21600");
  UnicodeString tokenUrl = "http://169.254.169.254/latest/api/token";
  TStringStream *response = new TStringStream;
  try {
    client.Put(tokenUrl, response);
    token = response->DataString.Trim();
  }
  __finally {
    delete response;
  }
}
 
// Now, for each metadata request, include the token if using v2
THTTPClient client;
if (FMetadataVersion == mv2) {
  client.Request->CustomHeaders->Add(UnicodeString().sprintf(L"X-aws-ec2-metadata-token: %s", token));
}
 
// Then proceed with GET requests as before
This is a simplified example. The actual code might have helper functions or different error handling.

Potential issues to handle: The PUT request might fail (e.g., if IMDSv2 isn't available), so catching exceptions and possibly falling back to v1 if allowed. Also, ensuring that the HTTP client is configured correctly (timeouts, retries, etc.).

Testing is crucial here. The user would need to test both scenarios where IMDSv2 is enforced and where it's not, to ensure the code works as expected.

So, the required changes are adding UI elements, modifying metadata retrieval to handle the token for v2, and including the necessary headers.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
41,717
Location:
Prague, Czechia

Re: Add support for connecting to AWS S3 with IAM Roles that use IMDSv2 (6 hours server session keys)

Thanks for your detailed response. So I assume it implies that what the mechanism WinSCP is currently using to retrieve the credentials is IMDSv1.

The only difference in IMDSv2 seems to be that a token needs to be acquired first.

Do you think it's even necessary to make IMDSv2 configurable? Shouldn't WinSCP just try it implicitly? The way aws seems to be doing.

Reply with quote

Advertisement

goslackware
Joined:
Posts:
6
Location:
USA

Retrieving token

Yes, I think you're correct that the main thing is that WinSCP just needs to also retrieve the token.

I've verified that winscp can connect to S3 from my IMDSv2 required server when I manually enter the AccessKeyId, SecretAccessKey, Token into WinSCP. To get those strings, I ran below PowerShell code, then copy\paste into WinSCP, and edit the S3 region.
# To get the full iam-role access_key and secret_key the below powershell code works:
# Step 1: Get IMDSv2 token
$token = (Invoke-WebRequest -Uri "http://169.254.169.254/latest/api/token" -Method PUT -Headers @{ "X-aws-ec2-metadata-token-ttl-seconds" = "21600" }).Content
 
# Step 2: Get IAM role name
$roleName = (Invoke-WebRequest -Uri "http://169.254.169.254/latest/meta-data/iam/security-credentials/" -Headers @{ "X-aws-ec2-metadata-token" = $token }).Content
 
# Step 3: Get temporary credentials
$credentials = (Invoke-WebRequest -Uri "http://169.254.169.254/latest/meta-data/iam/security-credentials/$roleName" -Headers @{ "X-aws-ec2-metadata-token" = $token }).Content | ConvertFrom-Json
 
# Output
$credentials.AccessKeyId
$credentials.SecretAccessKey
$credentials.Token
$credentials.Expiration
Make note of expiration.
Copy paste each string (AccessKeyId, SecretAccessKey, Token) into WinSCP. The Token is in Advanced > S3 settings.

Reply with quote

goslackware
Joined:
Posts:
6
Location:
USA

works!

Martin, thanks the dev portable version you sent works to resolve this issue!

All I had to do was:
* File protocol: Amazon S3
* Encryption: TLS/SSL Implicit encryption
* Hostname: # replace "s3.amazonaws.com" with the specific region endpoint listed at: https://docs.aws.amazon.com/general/latest/gr/s3.html
* Check the box "Credentials from AWS environment", then "General" is selected
(After checking that box, then the "Access key ID" was displayed and the "Secret access key" was shown as ***************************)

Thank you again!

Reply with quote

Advertisement

You can post new topics in this forum