Installation
Please refer to Amazon's documentation for instructions on how to install AWS CLI for Linux, MacOS, or Windows.
Correctly installed, you should be able to check the version by issuing:
$ aws --version aws-cli/2.13.23
Using the tool
The aws tool has a vast array of options, but for the purpose of communicating with s3 you will only need to consider aws s3 or aws s3api. Manuals can be checked like this:
aws s3 help aws s3api help
In short, aws s3 is used to perform simple high-level tasks treating the bucket like it's a folder, while aws s3api allows the use of any AWS REST API endpoint for specific, lower-level requests. You can read more about the differences here.
This guide will cover some common use cases of both options.
Configuration
First, you'll need to set up your keys in ~/.aws like described on the index page.
This should be sufficient to test your connection by listing available buckets with:
aws s3 ls --endpoint https://s3-oslo.educloud.no
Or by using --profile, if you have not configured a default profile.
In theory it's possible to set a default endpoint in ~/.aws/config, but aws cli has a known bug which does not pick it up, instead defaulting to an AWS endpoint if the --endpoint option is not provided.
To avoid having to type the endpoint with every request, a simple workaround is to make an alias for the aws command in .bashrc (or .zshrc):
alias aws='aws --endpoint https://s3-oslo.educloud.no'
This will work for both aws s3 and aws s3api.
High-level file interaction
Using the aws s3 option, files can be managed with your usual Unix shell commands:
ls, rm, cp, and mv. The path to the object(s) must be prefixed with s3://
Listing all your buckets
aws s3 ls [--profile <profile_name>] Ex. $ aws s3 ls --profile markusor 2023-03-15 11:40:04 1003-green-markusor-test 2023-02-20 10:35:55 1003-markusor-benchmark
Listing contents of a bucket
aws s3 ls s3://<bucket_name> Ex. $ aws s3 ls s3://1003-green-markusor-test PRE output/ PRE target/ 2023-04-20 15:04:07 578885 printerfeil.jpeg 2023-10-03 20:34:28 4897 scanner.sh (PRE are "directories")
Furthermore, you can list contents of a directory like this:
aws s3 ls s3://<bucket_name>/<directory>/ Ex. $ aws s3 ls s3://1003-green-markusor-test/output/ 2023-10-03 20:34:28 301111 test-2023-09-29-raw.tmp 2023-10-03 20:34:28 1675 test-2023-09-29-report.out
Note: Path must end with "/", otherwise you'll only list the directory itself
Moving/copying files to or from bucket
aws s3 mv ./local_file.txt s3://<bucket_name>/ or aws s3 mv s3://<bucket_name>/remote_file.txt ./ Similar for copying files with aws s3 cp. In practice, this will upload or download files from your bucket.
Removing files
aws s3 rm s3://<bucket_name>/remote_file.txt