AWS S3 secret key variable in script

Advertisement

kroberson
Joined:
Posts:
2
Location:
Edmond, Oklahoma

AWS S3 secret key variable in script

Hello, I have a Windows batch file that is called by a PowerShell script that retrieves the AWS access key and secret key from a file and places them in environment variables for the batch file to retrieve. The batch file uses WinSCP to connect to an S3 bucket and retrieve data. We are not using any AWS tools for the credentials as we only want to install the script files on a PC and no other installs. When I use the variables with the retrieved credentials, the access key works fine, but the secret does not and the connection fails. If I use the access key variable and provide the secret key directly in the script (not in a variable), it works fine. I assume there is some sort of translation that needs to take place on the secret key string in the variable, but I haven't been able to figure it out. Here are the portions of the scripts. Attached are log files with and without the secret key variable. Thanks for your help. Kevin

PowerShell file:
if(Test-Path $PWFile) {
    $json = Get-Content -Raw $PWFile | ConvertFrom-Json
    $env:ACCESS_KEY_ID = $json.key
    $env:SECRET_ACCESS_KEY = $json.secret
 
    Start-Process -FilePath "C:\kevin\projects\passwords\gw-bc-checks.bat" -Wait -NoNewWindow
}
BATCH FILE:
set logfile = c:\kevin\logs\winscp.log
"C:\Program Files (x86)\WinSCP\WinSCP.com" ^
  /log=%logfile% /ini=nul ^
  /command ^
    "open s3://%ACCESS_KEY_ID%:%SECRET_ACCESS_KEY%@s3.amazonaws.com/tenant-naico-gwcpdev-orange-dev-storage/qa/bc/inbound-files/pending/smartcomm/output/print/ -rawsettings CacheDirectories=0 CacheDirectoryChanges=0 S3DefaultRegion=""us-east-1""" ^
    "get *.ps %dlpath%" ^
    "close" ^
    "exit"
Description: Secret key is provided in a variable.
Description: Secret key is provided directly in the script.

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,517
Location:
Prague, Czechia

Re: AWS S3 secret key variable in script

The special characters in credentials in session URL need to be URL-encoded:
https://winscp.net/eng/docs/session_url#special

To avoid need for the encoding, you can use -password switch instead:
-password="%SECRET_ACCESS_KEY%"
See https://winscp.net/eng/docs/scriptcommand_open#password

Or set the standard AWS variables (i.e. AWS_SECRET_ACCESS_KEY) and use S3CredentialsEnv raw-session settings:
open s3://s3.amazonaws.com/ -rawsettings S3CredentialsEnv=on
See https://winscp.net/eng/docs/rawsettings#s3credentialsenv

Reply with quote

kroberson

I was using %%2F for the / character, which worked before I changed the script to retrieve the credentials from a file. I switched to the AWS variables and got rid of one of the %'s (now %2F) and that did the trick!
Thanks for your help Martin.

Reply with quote

PatrickHolmes
Joined:
Posts:
1
Location:
alaska

Ensure there are no extra characters or spaces in the SECRET_ACCESS_KEY variable. Try enclosing it in double quotes and trimming any spaces. Echo variables for debugging. Consider security best practices for key storage.

Reply with quote

Advertisement

You can post new topics in this forum