Cloud backup with MCrypt and S3cmd
The Name's Ex, Regex
A very appealing feature of S3cmd is its ability to use regular expressions. This functionality means you can exclude certain files from upload or download commands. Try using --rexclude=.*\.log
if you want to catch all files with the extension .log
. This functionality opens up several other possibilities, especially within scripts. You can effectively run cron jobs and grab precisely the files you want (whether for upload or download) without fear of accidentally adding an extra 10GB of data to your payload.
Examining the Viscera
A directory is called a bucket in the AWS environment. To create a bucket with mb
, where linux-is-the-best
is the bucket name, do:
# s3cmd mb s3://linux-is-the-best
Use rb
to delete a bucket. List the contents of your remote filesystem as follows:
# s3cmd ls s3://linux-is-the-best/chris-says
This command lists the contents of the chris-says
subdirectory. You can also use s3cmd la
to show all the files in all buckets.
Put It There
You might be surprised at how absurdly simple it is to upload and download with S3cmd. For uploading to Amazon S3, you simply use the put
command:
# s3cmd put chrisbinnie.tar.gz.nc s3://linux-is-the-best
And, if you can't guess, you can use get
for downloading:
# s3cmd get chris_backup_14411441.nc s3://only-use-linux
A final basic option is deleting a single file using del
:
# s3cmd del rubbish.nc s3://temp-bucket
Another very handy function is du
; for those unfamiliar, du
is a command-line disk usage tool on Unix-like systems that lists the disk space used by each bucket. The output from this command is well suited to scripts for quashing the hungry appetites of certain filesystem hogs. You can use it as follows:
# s3cmd du
Additionally, you can optionally add a file path to the end of that command.
A useful pair of options let you move files within the Amazon S3 filesystem without uploading or downloading the file. As in Unix, you can move files (using mv
) or duplicate them (by using cp
). The move and copy functions are as follows:
# s3cmd mv s3://linux_bucket/chris.png s3://older_linux_bucket # s3cmd cp s3://linux_bucket/binnie.jpg s3://older_linux_bucket
Note that you can also change the name of a file in one swift move with an elegant maneuver by adding something like /newer_chrisbinnie.jpg
as a object name melded to the s3://older_linux_bucket
destination.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.