s3cmd – Installation and Management of AWS S3 Buckets on Linux System

s3cmd-installation
mm
Written By:- Santosh Prasad

s3cmd is used to upload, download, and manage directory and file with s3 (amazon simple storage). It is storage service in AWS. It is command line utility. In this article I am going to explain how we can create, remove, upload, download and delete file with s3cmd tool.

S3CMD PACKAGE INSTALLATION

You can download s3cmd package from CentOS, RHEL and Ubuntu default repository.  Simply follow the below command to install it on your Linux system.

ON RHEL/CENTOS SYSTEM

#yum install s3cmd

ON UBUNTU/DEBIAN SYSTEM

#apt-get install s3cmd

LATEST S3CMD INSTALLATION USING SOURCE

If you are unable to install s3cmd latest version with apt-get and  yum command you directly download and install it from source code. Follow the below command to install s3cmd using source code.

# wget http://ufpr.dl.sourceforge.net/project/s3tools/s3cmd/1.6.1/s3cmd-1.6.1.tar.gz
# tar xzf s3cmd-1.6.1.tar.gz

After download and extracting tar file you will need to install it.

# cd s3cmd-1.6.1
# python setup.py install

S3CMD CONFIGURATION

To configure s3cmd you will need to S3 Amazon account Access Key and Secret Key. To get these security keys go to
your AWS security Credential page.

Follow the below command to configure s3cmd after getting Keys file.

# s3cmd --configure
Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3
Access Key: xxxxxxxxxxxxxxxxxxxxxx
Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: xxxxxxxxxx
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP and can't be used if you're behind a proxy
Use HTTPS protocol [No]: Yes

New settings:
Access Key: xxxxxxxxxxxxxxxxxxxxxx
Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Encryption password: xxxxxxxxxx
Path to GPG program: /usr/bin/gpg
Use HTTPS protocol: True
HTTP Proxy server name:
HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :

Now verifying that encryption works...
Success. Encryption and decryption worked fine :

Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'

COMMAND LINE USAGE OF S3CMD TOOL

After configuring s3cmd successfully. Follow the below command to manage s3 buckets.

All S3 BUCKET LISTING

Follow the below command to list all s3 buckets in your AWS account.

# s3cmd ls

NEW BUCKET CREATION

Use below command to create new bucket in AWS S3. In your account “looklinux” named bucket will be create using below command.

# s3cmd mb s3://looklinux

Bucket 's3://looklinux/' created

FILE UPLOADING IN S3 BUCKET

Follow the below command to upload file in s3 bucket.

# s3cmd put file_name.txt s3://looklinux/

file_name.txt -> s3://looklinux/file_name.txt [1 of 1]
190216 of 190216 100% in 0s 1668.35 kB/s done>

DIRECTORY UPLOADING IN S3 BUCKET

If you want to upload entire directory in s3 bucket use -r to upload recursively.

# s3cmd put -r backup_file s3://looklinux/

backup_file/file1.txt -> s3://tecadmin/backup_file/file1.txt [1 of 2]
9984 of 9984 100% in 0s 18.78 kB/s done
backup_file/file2.txt -> s3://tecadmin/backup_file/file2.txt [2 of 2]
0 of 0 0% in 0s 0.00 B/s done

Adding trailing slash with backup_file/ it will only upload it content.

# s3cmd put -r backup_file/ s3://looklinux/

backup_file/file1.txt -> s3://looklinux/file1.txt [1 of 2]
9984 of 9984 100% in 0s 21.78 kB/s done
backup_file/file2.txt -> s3://looklinux/file2.txt [2 of 2]
0 of 0 0% in 0s 0.00 B/s done

LISTING ALL DATA OF S3 BUCKET

With ls option you can list all objects of s3 bucket.

# s3cmd ls s3://looklinux/

DIR s3://looklinux/backup_file/
2017-17-01 10:58 190216 s3://looklinux/file.txt>

FILE DOWNLOADING FROM S3 BUCKET

If you want to download file from s3 bucket, Follow below command to download it.

# s3cmd get s3://looklinux/file.txt

s3://looklinux/file.txt -> ./file.txt [1 of 1]
4 of 4 100% in 0s 10.84 B/s done

REMOVING DATA FROM S3 BUCKET

Follow the below command to remove folder and file from s3 bucket.

Removing file

# s3cmd del s3://looklinux/file.txt

File s3://looklinux/file.txt deleted

Removing directory
# s3cmd del s3://looklinux/backup_file

File s3://looklinux/backup_file deleted

DELETE/REMOVE S3 BUCKET

Follow the below command to delete s3 bucket. Before removing the bucket make sure it is empty.

# s3cmd rb s3://looklinux

ERROR: S3 error: 409 (BucketNotEmpty): The bucket you tried to delete is not empty

Above you can see error because looklinux bucket is not empty.

You will need to remove bucket’s all objects before remove it. Execute above command again.

# s3cmd rb s3://looklinux

Bucket 's3://looklinux/' removed

MOVING BUCKET OBJECTS

Follow the below command to move buckets objects.

# s3cmd mv s3://looklinux/backup_file s3://looklinux[/backup_file2]

I hope this article will be helpful to understand to s3cmd tools usage on Linux system. Read our another article Bash:crontab: Command not found on (Redhat.x,CentOS.x) system and Easy Steps to Protect Your WordPress wp-admin(Admin) URL. If you have any queries and problem please comment in comment section or you can also ask your question.

About Author

mm

Santosh Prasad

Hi! I'm Santosh and I'm here to post some cool article for you. If you have any query and suggestion please comment in comment section.

Other Post by Santosh Prasad

Visit All Post

Related Article

You may also Like

Leave a Comment

Shares