Amazon S3 Tools: Command Line S3 Client Software and S3 Backup

AWS S3 Command Line Clients for Windows, Linux, Mac. Backup to S3, upload, retrieve, query data on Amazon S3.


S3cmd Home   |   S3cmd Download   |   FAQ / KB   

S3cmd: FAQ and Knowledge Base

Main Page > Browse Categories > FAQ > Show All
FAQ
 Does s3cmd support Amazon S3 server-side encryption?
 Does s3cmd support multipart uploads?
 Does s3cmd work on Windows?
 Does s3cmd work with CloudFront in Frankfurt (eu-central-1)?
 Does S3cmd work with Frankfurt region (eu-central-1)?
 Is there a mailing list for s3cmd?


FAQ


Does s3cmd support Amazon S3 server-side encryption?

Yes, file encryption can optionally be used to make a backup/upload to S3 more secure. Files can be stored on the Amazon S3 servers encrypted (i.e. at rest).

Server-side encryption is only available starting with s3cmd 1.5.0-beta1.


S3cmd provides two types of file encryption: server-side encryption and client-side encryption.


Server-Side encryption
is about data encryption at rest, that is, Amazon S3 encrypts your data as it writes it to disks in its data centers and decrypts it for you when you access it. As long as you authenticate your request and you have access permissions, there is no difference in the way you access encrypted or unencrypted objects. Amazon S3 manages encryption and decryption for you. For example, if you share your objects using a pre-signed URL, the pre-signed URL works the same way for both encrypted and unencrypted objects.

Amazon S3 Server Side Encryption employs strong multi-factor encryption. Amazon S3 encrypts each object with a unique key. As an additional safeguard, it encrypts the key itself with a master key that it regularly rotates. Amazon S3 Server Side Encryption uses one of the strongest block ciphers available, 256-bit Advanced Encryption Standard (AES-256), to encrypt your data.

When you upload one or more objects with S3cmd, you can specify in your request if you want Amazon S3 to save your object data encrypted. To specify that you want Amazon S3 to save your object data encrypted use the flag --server-side-encryption. Server-side encryption is optional. Your bucket might contain both encrypted and unencrypted objects. Encrypted objects are marked automatically with the metadata header x-amz-server-side-encryption set to AES256.



With Client-Side encryption, you add an extra layer of security by encrypting data locally BEFORE uploading the files to Amazon S3. Client-side encryption and server-side encryption can be combined and used together. In S3cmd, client-side encryption is applied by specifying the flag -e or --encrypt.



Does s3cmd support multipart uploads?

Yes, the latest version of s3cmd supports Amazon S3 multipart uploads.

Multipart uploads are automatically used when a file to upload is larger than 15MB.
In that case the file is split into multiple parts, with each part of 15MB in size (the last part can be smaller). Each part is then uploaded separately and then reconstructed at destination when the transfer is completed.

With this new feature, if an upload of a part fails, it can be restarted without affecting any of the other parts already uploaded.

There are two options related to multipart uploads in s3cmd. They are:

--disable-multipart

Disable multipart uploads for all files

and

--multipart-chunk-size-mb=SIZE
Size of each chunk of a multipart upload. Files bigger than SIZE are automatically uploaded as multithreaded-multipart, smaller files are uploaded using the traditional method. SIZE is in Mega-Bytes, default chunk size is 15MB, minimum allowed chunk size is 5MB, maximum is 5GB.



Does s3cmd work on Windows?

Yes, however, being written in Python, s3cmd requires Python 2.4+ for Windows to be installed and also it requires the Python library date-util.

Alternatively, you can try S3Express. S3Express is a "native" Windows command line program, that works out of the box and does not require any additional library or software to run. S3Express is a commercial program. It's very compact and has very small footprint: the entire program is less than 5MB. S3Express is perfect for uploading, querying, listing Amazon S3 objects via the command line on Windows.



Does s3cmd work with CloudFront in Frankfurt (eu-central-1)?

No. CloudFront in Frankfurt currently requires newer signing code than 1.5.0 includes. This issue will be addressed in a future release. Other S3 operations do work for Frankfurt.



Does S3cmd work with Frankfurt region (eu-central-1)?

Yes. S3cmd supports the new Frankfurt S3 region since version 1.5



Is there a mailing list for s3cmd?

Yes, there are 4 s3cmd mailing lists available. Old e-mails are archived and searchable and can be accessed for reference.

The mailing lists are available here:

http://sourceforge.net/p/s3tools/mailman/


 A printable version of the entire FAQ and Knowledge Base is also available.
 For further queries or questions, please contact us.