S3 which stands for "Simple Storage Service" is Amazon's storage service that provides IT teams with a secure, scalable, and reliable way to store and retrieve files and folders on the cloud. S3 helps you make the most out of your data by making sure that it is available when needed and scalable as demand grows.
Conventionally, S3 is accessed from a web browser after signing in to your AWS account. This can also be achieved on command-line using a nifty tool called s3cmd. The s3cmd utility is an opensource command-line tool that allows you to access and manage your S3 service. It allows you to create/delete buckets, add/remote files and folders and list the contents of your S3 buckets.
Here at Ibmi Media, as part of our Server Management Services, we regularly help our Customers to perform related Amazon s3 Buckets queries.
In this context, we shall look into how to install s3cmd on Linux and help you make the most of it to manage your S3 account.
How to install s3cmd on Linux via commandline ?
For Debian / Ubuntu distributions use the APT package manage as follows:
$ sudo apt install s3cmdOn CentOS / RHEL distros install s3cmd using the command below:
$ sudo dnf install s3cmdFor SUSE Enterprise server , start by adding the repository and using the zypper command-line tool below:
$ sudo zypper addrepo http://s3tools.org/repo/SLE_11/s3tools.repo
$ zypper install s3cmdHow to Install s3cmd from source ?
If your Linux distribution is not listed above, don't worry. You can install from source which will work in all distributions.
i. First, download the zip file of the latest s3cmd version, which by this time is s3cmd 2.1.0.
$ wget https://sourceforge.net/projects/s3tools/files/s3cmd/2.1.0/s3cmd-2.1.0.zipIn your home directory, you should see the s3cmd-2.1.0.zip zip file.
ii. Next, unzip the file.
$ unzip s3cmd-2.1.0iii. Navigate to the uncompressed folder
$ cd s3cmd-2.1.0iv. And execute the command shown to install from source.
$ sudo python3 setup.py installHow to Set up the s3cmd environment ?
Before we begin interacting with Amazon S3, we need to set up the s3cmd tool by providing the necessary credentials such as Access Key & Secret access to our account. So, run the command below to configure the s3cmd environment.
$ s3cmd --configureProvide the access key, secret access , region and encryption password.
Be sure to use the HTTPS protocol for enhanced security.
How to List s3 buckets ?
We have successfully configured the s3cmd tool, and we can now begin interacting with Amazon S3. Before you start interacting with S3, ensure that the user on your AWS has been configured with Amazon S3 IAM role.
To list the existing S3 buckets on your account, execute the command:
$ s3cmd lsSo far, we have none since we are just getting started out.
Now, we will create one.
How to Create an S3 bucket ?
To create a new S3 bucket, run the command shown. The command below creates a bucket in S3 called linuxapt.
$ s3cmd mb s3://linuxaptYou can head over to your AWS account and confirm that the bucket is there.
How to Upload a file to S3 bucket ?
To upload a regular file to your bucket, invoke the put option followed by the file and the path to S3 bucket. The command below uploads the file hello.sh to S3.
$ s3cmd put hello.sh s3://linuxapt/How to Upload a directory to S3 bucket ?
To upload a directory, use the -r flag for adding the directory recursively alongside its contents. Take care not to add the leading / as this will only add the directory's contents and not the directory itself.
Here, we are uploading the Pictures directory that contains a JPG image file.
$ s3cmd put -r Pictures s3://linuxapt/How to List contents of S3 bucket ?
To view or display what your S3 bucket contains, run the command.
$ s3cmd ls s3://linuxapt/How to Delete a file or directory from an S3 bucket ?
If you wish to remove or delete a file you no longer need, use the del option as seen here.
$ s3cmd del s3://linuxapt/hello.shTo delete a directory, use the same syntax as follows. However, this only works if the directory is empty.
$ s3cmd del s3://linuxapt/PicturesIf a directory is not empty, use the -r option to recursively delete all its contents as shown.
$ s3cmd del -r s3://linuxapt/Pictures/How to Remove a bucket from S3 ?
If you no longer require your bucket, you can delete it as follows.
$ s3cmd rb s3://linuxaptIf the bucket still contains some files and folders, you will run into the error below informing you that the bucket is not empty.
Now, all you need to do is to go back and delete all the files and directories in your bucket and try again.
[Need assistance in fixing Linux System errors? We can help you. ]
Conclusion
This article covers an overview of the s3cmd command-line tool and how you can use it to manage your Amazon S3 storage feature. S3cmd is an Open Source tool that is free for both commercial and private use, but you need to pay only for Amazon resources. so most of the users are using this tool for managing AWS S3 Buckets. S3cmd is a free command-line tool written in Python
How to use the s3cmd command line to manage the Amazon s3 bucket in Linux ?
To use s3cmd command to list s3 bucket, run the command:
# s3cmd lsThis command will list s3bucket from Amazon S3, If you are unable to get the list of the bucket then there will be the following reasons:
- There is no bucket created in Amazon s3.
- As you have used IAM user Access Key and Secret Key while configuring the s3cmd tool, please verify IAM user has required permission to access the s3 bucket.
- If you are using an IAM role then check whether the s3 bucket policy is attached to the EC2 instance or not.
This article covers an overview of the s3cmd command-line tool and how you can use it to manage your Amazon S3 storage feature. S3cmd is an Open Source tool that is free for both commercial and private use, but you need to pay only for Amazon resources. so most of the users are using this tool for managing AWS S3 Buckets. S3cmd is a free command-line tool written in Python
How to use the s3cmd command line to manage the Amazon s3 bucket in Linux ?
To use s3cmd command to list s3 bucket, run the command:
# s3cmd lsThis command will list s3bucket from Amazon S3, If you are unable to get the list of the bucket then there will be the following reasons:
- There is no bucket created in Amazon s3.
- As you have used IAM user Access Key and Secret Key while configuring the s3cmd tool, please verify IAM user has required permission to access the s3 bucket.
- If you are using an IAM role then check whether the s3 bucket policy is attached to the EC2 instance or not.
