About S3tools project

S3tools project offers several Open source tools for accessing Amazon S3 – Simple Storage Service.

  • Amazon S3 is a reasonably priced data storage service. Ideal for off-site backups, archiving and other data storage needs.
  • S3tools project is here to give you tools to upload / archive / store your data in S3 and download / retrieve / restore them later.

That’s basically it, not a rocket science, eh? Currently we have two sub-projects to offer:

S3cmd

Command line S3 client that can be used in scripts, backup cron jobs, etc. This is your best choice if you want to quickly get up to speed with S3. Requires Python 2.4 or newer and some pretty common Python modules.

S3browser

Server-side PHP application that allows managing your web storage and data in it using a web browser. To use S3browser you’ll need a web server with direct access to the internet and support for PHP5 scripts. And obviously a web browser too :-)

Both tools are developed on Linux and should run just fine anywhere else where a Python interpreter or a webserver with PHP5 support is available. Yes, that includes Mac OS X, Solaris, and various BSD flavours as well as, believe it or not, MS Windows :-)

Our TODO list currently contains are these sub-projects:

S3fuse

Filesystem driver for Linux FUSE (Filesystem in USErspace). It will allow for mounting the S3 storage as a fully fledged filesystem to your Linux box.

S3kio

KDE IO slave for handling S3-URIs. It will allow for browsing Amazon S3 storage in Konqueror and using it in most KDE file dialogs.

S3gui

GUI extension to S3cmd.

So these are our plans for the future. For now, however, you’ll have to decide only between S3cmd and S3browser :-)

By Michal Ludvig on 21 August 2008

Tags:

---

Comment

  1. Gollano Bropol Pöbelan wrote:

    your s3-browser repository gives a
    svn: Target path does not exist
    error. this is very annoying. please fix it.
    also there are already 2 different s3-fuse projects out there. do not reinvent the wheel if there is already something working.
    thanks.

    ( 3 February 2009, 06:36 · #)

  2. Michal Ludvig wrote:

    Hi Gollano,

    1) The link to S3browser SVN is now fixed.
    2) Whether or not I spend my free time reinventing the wheel is solely my choice ;-)

    Michal

    ( 4 February 2009, 13:51 · #)

  3. Dylan Oliver wrote:

    @gollum: none of the other s3fs options are compatible with tools like s3cmd.

    @michal: is there an option to make sync create local directories from s3 ‘paths’? i just tried:

    s3cmd sync s3://bucket /mnt/data-store/

    and got:

    File ‘s3://bucket/My Folder’ stored as /mnt/data-store/My Folder (38 bytes in 0.0 seconds, 334.01 kB/s) [1 of 36122]
    WARNING: /mnt/data-store/Folder: can not make directory: File exists
    WARNING: My Folder/subfolder: destination directory not writable: /mnt/data-store/My Folder

    :(

    back to the other s3cmd for now, but thanks a lot and how many goats do i need to sacrifice to get you moving on s3fuse?

    ( 6 February 2009, 05:24 · #)

  4. Dylan Oliver wrote:

    @gollum: none of the other s3fs options are compatible with tools like s3cmd.

    @michal: is there an option to make sync create local directories from s3 ‘paths’? i just tried:

    s3cmd sync s3://bucket /mnt/data-store/

    and got:

    File ‘s3://bucket/My Folder’ stored as /mnt/data-store/My Folder (38 bytes in 0.0 seconds, 334.01 kB/s) [1 of 36122]
    WARNING: /mnt/data-store/Folder: can not make directory: File exists
    WARNING: My Folder/subfolder: destination directory not writable: /mnt/data-store/My Folder

    :(

    back to the other s3cmd for now, but thanks a lot and how many goats do i need to sacrifice to get you moving on s3fuse?

    ( 6 February 2009, 05:26 · #)

  5. mkhalemi wrote:

    Nice update on the new version 0.9.9.

    (21 February 2009, 18:37 · #)

  6. Henry McGilton wrote:

    I’m obviously missing the point about S3.

    If I am developing an iPhone client and wish to make use of S3 storage/compute/network facilities, what are the APIs by which I communicate with the cloud? My only comparable point of reference is Google App Engine.

    Many Thanks for any Pointers to the correct place.

    . . . . . . . . Henry

    (27 March 2010, 12:17 · #)

  7. Black wrote:

    Hey ..
    Can I use this for accessing WayBack Machine?

    (20 May 2010, 00:10 · #)

  8. gwern wrote:

    I was trying to update my laptop’s old version of a folder with the usual ‘s3cmd sync s3://foo/bar bar’ command, and was puzzled that a bunch of the new files were showing up, but many of the old files were not being updated. After pondering a while, I tried adding a -f, and that seemed to do the trick.

    This doesn’t seem to be mentioned in the man page.

    (29 June 2010, 21:10 · #)

  9. Jessica wrote:

    Nice content very helpful info..

    (27 July 2010, 00:44 · #)

  10. alldayauto transport wrote:

    useful info

    (27 July 2010, 00:47 · #)

 
---