The Name's Ex, Regex

A very appealing feature of S3cmd is its ability to use regular expressions. This functionality means you can exclude certain files from upload or download commands. Try using --rexclude=.*\.log if you want to catch all files with the extension .log. This functionality opens up several other possibilities, especially within scripts. You can effectively run cron jobs and grab precisely the files you want (whether for upload or download) without fear of accidentally adding an extra 10GB of data to your payload.

Examining the Viscera

A directory is called a bucket in the AWS environment. To create a bucket with mb, where linux-is-the-best is the bucket name, do:

# s3cmd mb s3://linux-is-the-best

Use rb to delete a bucket. List the contents of your remote filesystem as follows:

# s3cmd ls s3://linux-is-the-best/chris-says

This command lists the contents of the chris-says subdirectory. You can also use s3cmd la to show all the files in all buckets.

Put It There

You might be surprised at how absurdly simple it is to upload and download with S3cmd. For uploading to Amazon S3, you simply use the put command:

# s3cmd put chrisbinnie.tar.gz.nc s3://linux-is-the-best

And, if you can't guess, you can use get for downloading:

# s3cmd get chris_backup_14411441.nc s3://only-use-linux

A final basic option is deleting a single file using del:

# s3cmd del rubbish.nc s3://temp-bucket

Another very handy function is du; for those unfamiliar, du is a command-line disk usage tool on Unix-like systems that lists the disk space used by each bucket. The output from this command is well suited to scripts for quashing the hungry appetites of certain filesystem hogs. You can use it as follows:

# s3cmd du

Additionally, you can optionally add a file path to the end of that command.

A useful pair of options let you move files within the Amazon S3 filesystem without uploading or downloading the file. As in Unix, you can move files (using mv) or duplicate them (by using cp). The move and copy functions are as follows:

# s3cmd mv s3://linux_bucket/chris.png s3://older_linux_bucket
# s3cmd cp s3://linux_bucket/binnie.jpg s3://older_linux_bucket

Note that you can also change the name of a file in one swift move with an elegant maneuver by adding something like /newer_chrisbinnie.jpg as a object name melded to the s3://older_linux_bucket destination.