Differences
This shows you the differences between the selected revisions of the page.
| guide_amazon_s3_sftp 2017-03-04 | guide_amazon_s3_sftp 2023-05-30 (current) | ||
| Line 1: | Line 1: | ||
| ====== Setting up an SFTP Access to Amazon S3 ====== | ====== Setting up an SFTP Access to Amazon S3 ====== | ||
| - | If you need to access/manage files stored on Amazon S3 (Simple Storage Service) bucket via SFTP, you can mount the bucket to a file system on a Linux server and access the files using the SFTP as any other files on the server. | + | //Note that WinSCP supports a [[guide_amazon_s3|direct access to S3 storage]]//. |
| - | This guide shows how to mount the S3 bucket using ''[[https://github.com/s3fs-fuse/s3fs-fuse|s3fs]]'' file system to an Amazon EC2 server and access it using WinSCP. | + | If you need to access/manage files stored on Amazon S3 (Simple Storage Service) bucket via SFTP, there are two options. You can use a native managed SFTP service recently added by Amazon (which is easier to set up). Or you can mount the bucket to a file system on a Linux server and access the files using the SFTP as any other files on the server (which gives you greater control). |
| - | ===== Creating Access Server ===== | + | ===== [[managed]] Managed SFTP Service ===== |
| - | If you do not have a Linux server available for the mounting, [[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/launching-instance.html|launch a new Amazon EC2 server]]. | + | |
| - | A basic Amazon Linux AMI (free tier eligible) server will generally suffice and the following instructions are tested on this distribution. Instructions for other distributions may differ. | + | ==== Creating Managed SFTP Server ==== |
| - | ===== Installing s3fs ===== | + | * To create a Managed SFTP server for S3, in your Amazon AWS Console, go to [[https://console.aws.amazon.com/transfer|AWS Transfer for SFTP]] and create a new server (you can keep server options to their defaults for a start). |
| + | * In SFTP server page, add a new SFTP user (or users). | ||
| + | * Permissions of users are governed by an associated [[https://console.aws.amazon.com/iamv2/#/roles|AWS role]] in IAM service. To create a role which has a full access to all your S3 buckets, just create an //S3// service role with //AmazonS3FullAccess// policy. \\ The role must have trust relationship to ''transfer.amazonaws.com''. On a role page, select //Trust relationships// tab, click //Edit trust relationship// button, and in the access control policy JSON document, change ''Statement[].Principal.Service'' value to ''transfer.amazonaws.com'':((Based on the answer by @ChristopherTull to [[https://stackoverflow.com/q/53642132/850848#53648926|Connecting to AWS Transfer for SFTP]] on Stack Overflow.)) \\ <code php> | ||
| + | { | ||
| + | "Version": "2012-10-17", | ||
| + | "Statement": [ | ||
| + | { | ||
| + | "Effect": "Allow", | ||
| + | "Principal": { | ||
| + | "Service": "transfer.amazonaws.com" | ||
| + | }, | ||
| + | "Action": "sts:AssumeRole" | ||
| + | } | ||
| + | ] | ||
| + | } | ||
| + | </code> | ||
| + | * [[public_key#generate|Generate a key pair]] for your new user and paste public key fingerprint to //%%SSH%% public keys// box (use the format you would otherwise use for [[guide_public_key#configure_openssh|OpenSSH ''authorized_keys'' file]]). | ||
| + | |||
| + | ==== Connecting to Managed SFTP Server ==== | ||
| + | |||
| + | You can connect to the managed SFTP server as to any other SFTP server. | ||
| + | |||
| + | The host name of the server can be found on the server page as //Endpoint// in a format ''server_id.server.transfer.region.amazonaws.com''. | ||
| + | |||
| + | ===== [[mounting]] Mounting Bucket to Linux Server ===== | ||
| + | |||
| + | This guide shows how to mount the %%S3%% bucket using [[https://github.com/s3fs-fuse/s3fs-fuse|''s3fs'' file system]] to an Amazon EC2 server and access it using WinSCP. | ||
| + | |||
| + | ==== Creating Access Server ==== | ||
| + | If you do not have a Linux server available for the mounting, [[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-instance-wizard.html|launch a new Amazon EC2 server]]. | ||
| + | |||
| + | A basic Amazon Linux AMI (free tier eligible) server will generally suffice and the following instructions are tested on this distribution. Instructions for other distributions may differ. | ||
| - | To install the ''s3fs'' file system:((These instructions are based on [[https://github.com/s3fs-fuse/s3fs-fuse#user-content-installation|the official installation instructions]] for ''s3fs''. Except for added ''fuse'' dependency.)) | + | ==== Installing s3fs ==== |
| - | * Login to your Linux server via SSH. | + | Start by [[https://github.com/s3fs-fuse/s3fs-fuse#installation|installing ''s3fs'' file system]]. |
| - | * Install dependencies: \\ ''sudo yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel'' | + | |
| - | * Compile and install ''s3fs'': \\ ''git clone %%https://github.com/s3fs-fuse/s3fs-fuse.git%%'' \\ ''cd s3fs-fuse'' \\ ''./autogen.sh'' \\ ''./configure'' \\ ''make'' \\ ''sudo make install'' | + | |
| - | ===== Mounting S3 Bucket to File System ===== | + | ==== Mounting S3 Bucket to File System ==== |
| * Switch to ''root'': \\ ''sudo su'' | * Switch to ''root'': \\ ''sudo su'' | ||
| - | * Store security credentials that will be used to access the S3 bucket to ''/etc/passwd-s3fs'': \\ ''echo <access-key-id>:<secret-access-key> > /etc/passwd-s3fs'' \\ ''chmod 600 /etc/passwd-s3fs'' \\ (Replace the ''<access-key-id>'' and ''<secret-access-key>'' with the actual values) | + | * Store security credentials that will be used to access the %%S3%% bucket to ''/etc/passwd-s3fs'': \\ ''echo <access-key-id>:<secret-access-key> > /etc/passwd-s3fs'' \\ ''chmod 600 /etc/passwd-s3fs'' \\ (Replace the ''<access-key-id>'' and ''<secret-access-key>'' with the actual values) |
| * Create mount point (example): \\ ''mkdir /mnt/<bucket>'' | * Create mount point (example): \\ ''mkdir /mnt/<bucket>'' | ||
| - | * Add entry to ''[[&man_ref(5,fstab)|fstab]]'' to mount the bucket: \\ ''echo s3fs#<bucket> /mnt/<bucket> fuse _netdev,rw,nosuid,nodev,allow_other 0 0 %%>>%% /etc/fstab'' \\ (Replace the leading ''<bucket>'' with your bucket name and the ''/mnt/<bucket>'' with the mount point) | + | * Add entry to ''[[&man_ref(5,fstab)|fstab]]'' to mount the bucket: \\ ''echo s3fs#<bucket> /mnt/<bucket> fuse _netdev,rw,nosuid,nodev,allow_other,nonempty 0 0 %%>>%% /etc/fstab'' \\ (Replace the leading ''<bucket>'' with your bucket name and the ''/mnt/<bucket>'' with the mount point) |
| * Mount the bucket: \\ ''mount -a'' | * Mount the bucket: \\ ''mount -a'' | ||
| - | ===== Connecting to the Access Server to Manage the Bucket ===== | + | ==== Connecting to the Access Server to Manage the Bucket ==== |
| * Learn how to [[guide_amazon_ec2|connect securely to Amazon EC2 server with SFTP]]. | * Learn how to [[guide_amazon_ec2|connect securely to Amazon EC2 server with SFTP]]. | ||