Configure Image Uploader to Upload Files to Amazon S3 from a Web Application

Introduction

If you develop some online service and have a significant number of users, you've probably found out that your existing network infrastructure can be insufficient to maintain huge upload volumes. So you have a choice: to add more servers, load balancing devices, and other elements of server infrastructure, or outsource storage to a third party company. In the latter case, most likely you will stick with a cloud storage provider.

Amazon S3 is one of the leading cloud storage provider. It has a reputation of fast, reliable and scalable file hosting solution. In addition, its straightforward API makes working with stored files easy. When used together, Image Uploader and Amazon S3 offer a really powerful solution: the ability to send files as a POST request, bypassing an intermediate web server. This article describes how to send files from a client directly to a cloud storage platform in your web application.

Amazon S3 Concepts

Before you start, you need to get an Amazon Web Services (AWS) account. If you don't have one already, you can do it here:

http://aws.amazon.com/s3/

After signing up, you will get an Access Key Id which will identify your account. In addition, a Secret Access Key will be issued for you. You have to use it to sign all requests to your account. It is used to avoid malicious attacks and misuse of this service.

All objects in Amazon S3 are stored in so-called buckets. A bucket is an analogue of a folder in a file system. Typically, you would create a separate bucket for each web application. Each bucket is identified with a unique domain name. A file within a bucket should be identified with a key.

For better understanding, let's examine how objects are identified in Amazon S3. Imagine, you have a bucket called mybucket, and a file myphoto.jpg is stored in a myfolder folder. In this case, the key for this file is myfolder/myphoto.jpg and the URL with which the file can be accessed is http://mybucket.s3.amazonaws.com/myfolder/myphoto.jpg.

To manage access to objects, Amazon S3 uses access control lists (ACL). They are used to verify what exact operations can be done with a file for a specific user (if any are allowed).

Additional technical documentation for Amazon S3 can be found at http://docs.amazonwebservices.com/AmazonS3/latest/index.html?UsingHTTPPOST.html

Using Aurigma Image Uploader with Amazon S3

To deliver files to Amazon S3 account, you should send an especially prepared HTTP POST request to a bucket. Although Image Uploader uses a different format of a POST request, it allows users to reconfigure the names of POST variables. Therefore, Image Uploader can submit requests to Amazon S3 in the format it can parse.

To do that, there is no need to learn API of Amazon S3 or something like this. All necessary code is separated into a JavaScript class called AmazonS3Extender. All you need is to load this code, link it with Image Uploader, and specify all your Amazon S3 parameters there (Access Key Id, Secret Access Key, ACL, bucket, etc).

There are ASP.NET and PHP wrappers for this extender for both ASP.NET and PHP developers.

Upload to Amazon S3 From ASP.NET

Let's create a quick ASP.NET application in C# which will allow users to upload files to your Amazon S3 account.

1. Create a new ASP.NET web site (in Visual Studio 2008 menu File -> New -> Web Site...).
2. Add Uploader and AmazonS3Extender controls to the toolbox. These components are implemented in the Aurigma.ImageUploader.dll which is located in Image Uploader installation folder (typically this is C:\Program Files\Aurigma\Image Uploader 7.0.11 - the version number may vary!).
3. Open your default.aspx in the design mode and drag and drop the Uploader and AmazonS3Extender items into the desired positions.
4. Bind the AmazonS3Extender to Uploader via its identifier using the TargetControlIDproperty.
5. Specify your authentication details (Access Key Id and Secret Access Key) and Bucket as values of the corresponding properties.
6. Set FileSettings-Acl param to define a level of permissions. For example, you can use public-read value to allow everyone to download a file.
7. Set Filesettings-Key param to specify the object key (i.e. how this file will be identified inside the bucket). If you want to keep the original file name, just use the ${filename} placeholder.
8. Add metadata fields using PredefinedMetaProperty and/or CustomMetaProperty items.

A code example follows.

<%@ Page Language="C#" AutoEventWireup="true" %>
<%@ Register Assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader" TagPrefix="cc1" %>
<%@ Register Assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader.AmazonS3" TagPrefix="cc2" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >
<head id="Head1" runat="server">
        <title>Amazon S3 Demo</title>
</head>
<body>
    <form id="form1" runat="server">
        <cc1:Uploader ID="Uploader1" runat="server" >
            <Converters>
                <cc1:Converter Mode="*.*=SourceFile" />
            </Converters>
        </cc1:Uploader>
        
        <cc2:AmazonS3Extender runat="server" ID="AmazonS3Extender1" TargetControlID="Uploader1"
            AWSAccessKeyId="<%$ _AWSAccessKeyId %>"
            Bucket="<%$ _Bucket %>" 
            SecretAccessKey="<%$ _SecretAccessKey %>"
            PolicyExpiration="6000" >
            <cc2:FileSettings Acl="public-read" Key="${filename}">
                <cc2:PredefinedMetaProperty Field="SourceWidth_[itemIndex]" Name="width" />
                <cc2:PredefinedMetaProperty Field="SourceHeight_[itemIndex]" Name="height" />
                <cc2:CustomMetaProperty Name="Author" Value="John Smith" />
            </cc2:FileSettings>
        </cc2:AmazonS3Extender>
   </form>
</body>
</html>

Here we presume that _AWSAccessKeyId, _Bucket, and _SecretAccessKey parameters are set beforehand as server-side variables.

Upload to Amazon S3 From PHP

Let’s examine how to the same thing in PHP.

1. Create a new PHP file, say index.php.
2. Add Image Uploader PHP to the server directory where your index.php resides. You can find this library in the ImageUploaderPHP folder within the SDK installation directory (typically C:\Program Files\Aurigma\Image Uploader 7.0.11 - the version number may vary).
3. Include Uploader.class.php and AmazonS3Extender.class.php files to the index.php.
4. Create Uploader and AmazonS3Extender objects and link them together. You can do it by passing the Uploader instance to the AmazonS3Extender constructor.
5. Specify your authentication details (Access Key Id and Secret Access Key) and Bucket as values of the corresponding properties.
6. Create a FileSettings instance and configure it as follows.
7. Use the setAcl($value) property of the FileSettings instance to define a level of permissions. For instance, if you specify the public-read value, everyone will be allowed to download a file.
8. Use the setKey($value) property of the FileSettings instance to specify the object key. This key will uniquely identify a file within a bucket. Here you can use the ${filename} variable to set an original filename.
9. Add metadata fields to the FileSettings->Meta array.

<?php
require_once 'ImageUploaderPHP/Uploader.class.php';
require_once 'ImageUploaderPHP/AmazonS3Extender.class.php';
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <title>Amazon S3 Demo</title>
</head>
<body>
  <?php
    $uploader = new Uploader("Uploader1");
    $converters = &$uploader->getConverters();
    $converter = new Converter();
    $converter->setMode("*.*=SourceFile");
    $converters[] = $converter;


    $as3 = new AmazonS3Extender($uploader);
    $as3->setAWSAccessKeyId($AWSAccessKeyId);
    $as3->setBucket($Bucket);
    $as3->setSecretAccessKey($SecretAccessKey);
    $files = &$as3->getFiles();
    $file = new FileSettings();
    $file->setAcl("public-read");
    $file->setKey('${filename}');
    $file->setMeta(array(
      array("name" => "Author", "value" => "John Smith"),
      array("name" => "width", "field" => "SourceWidth_[itemIndex]"),
      array("name" => "height", "field" => "SourceHeight_[itemIndex]")
    ));
    $files[] = $file;

    $uploader->render();
  ?>
</body>
</html>

Here we presume that the $AWSAccessKeyId, $Bucket, and $SecretAccessKey parameters are set beforehand as server-side variables.

Upload to Amazon S3 Without Helper Libraries (Any Server Platform)

This approach is more complicated, but it can be used with any other server platform beside PHP and ASP.NET. It requires performing several additional steps, such as creating the security policy and signature. So, let’s start.

Step 1 - Create a policy
The policy is a UTF-8 and Base64 encoded JSON document which specifies the date when this policy expires and conditions the request must meet. The expiration date must be in ISO8601 GMT date format. The conditions are used to validate the uploaded object and include definitions for accessing the control list, bucket, key, and metadata fields. Here is the code sample (C#) that creates a policy document which expires in 100 minutes and defines an access control list option, bucket name, and uploaded file.

Here we presume that the _AWSAccessKeyId, _bucket, and _secretAccessKey parameters are set beforehand.

public string ConstructPolicy()
{
    StringBuilder policy = new StringBuilder();
    DateTime expDate = DateTime.Now.ToUniversalTime().AddSeconds(6000);

    policy.AppendLine("{ \"expiration\": \"" + expDate.ToString("s") + ".000Z\"");
    policy.AppendLine(", \"conditions\": [");
    policy.AppendLine(" {\"acl\": \"public-read\" }");
    policy.AppendLine(" , {\"bucket\": \"" + _bucket + "\" }");
    policy.AppendLine(" , {\"success_action_status\": \"200\"}");
    policy.AppendLine(" , [\"starts-with\", \"$key\", \"\"]");

    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-width\", \"\"]");
    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-height\", \"\"]");
    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-author\", \"\"]");

    policy.AppendLine("]");
    policy.AppendLine("}");

    // Encode the policy using UTF-8.
    byte[] bytes = Encoding.UTF8.GetBytes(policy.ToString());
    
    // Encode those UTF-8 bytes using Base64.
    return Convert.ToBase64String(bytes);
}

Step 2 - Create a signature
To create a signature, encode your policy document using UTF-8 and then encode these bytes using Base64. After that sign the policy with your Secret Access Key using HMAC SHA-1 and encode the SHA-1 signature using Base64. See the code sample below (C#).

public string CreateSignature()
{
    _policy = ConstructPolicy();

    // Sign the policy with your Secret Access Key using HMAC SHA-1.
    System.Security.Cryptography.HMACSHA1 hmac = new System.Security.Cryptography.HMACSHA1();
    hmac.Key = System.Text.Encoding.UTF8.GetBytes(_secretAccessKey);

    byte[] signb = hmac.ComputeHash(System.Text.Encoding.UTF8.GetBytes(_policy));

    // Encode the SHA-1 signature using Base64.
    _signature = Convert.ToBase64String(signb);
}

Step 3 - Pass the policy and signature to the Image Uploader page
You should initialize JavaScript variables with the policy and signature values generated by the server code we described in the previous steps. The simplest way is just to insert server-side code calls into the HTML. Let's see how you would do it in ASP.NET. 

<script type="text/javascript">
    var _policy = '<%= _policy %>';
    var _signature = '<%= _signature %>';
    var _bucket = '<%= _bucket %>';
    var _AWSAccessKeyId = '<%= _AWSAccessKeyId %>';
</script>

Step 4 - Add Image Uploader to the page and initialize it for the upload to Amazon S3

1. Link the aurigma.uploader.js and aurigma.uploader.amazons3.js files with the page. These files are located in the Scripts folder within the SDK installation directory. Typically these files are located in C:\Program Files\Aurigma\Image Uploader 7.0.11\Scripts - the version number may vary! 
2. Add a JavaScript block in the position where you want to insert Image Uploader and create a new Uploader object here.
3. Create AmazonS3Extender objects and link it with Uploader together. You can do it by passing the Uploader instance to the AmazonS3Extender constructor.
4. Specify your Access Key Id and Bucket as values of the corresponding properties.
5. Create a fileSettings instance.
6. Define a level of permissions via the acl property.
7. Specify the object key via the key property. Here you can use the ${filename} variable to set a name of this file.
8. Specify the policy and signature constructed and encoded before as values of the policy and signature properties.

The code example follows.

<script type="text/javascript" src="Scripts/aurigma.uploader.js">  </script>
<script type="text/javascript" src="Scripts/aurigma.uploader.amazons3.js">  </script>
<script type="text/javascript">
var u = $au.uploader({
    id: 'Uploader1',
    converters: [
        {mode: '*.*=SourceFile'},
    ]
});
    
var as3 = $au.amazonS3Extender(u);
as3.accessKeyId(_AWSAccessKeyId);
as3.bucket(_bucket);
as3.bucketHostName('http://yourdomain.com/');
as3.converters([
    {
        acl: 'public-read',
        key: '${filename}',
        policy: _policy,
        signature: _signature,
        meta: [
            { name: 'Width', field: 'SourceWidth_[itemIndex]'},
            { name: 'Height', field: 'SourceHeight_[itemIndex]'},
            { name: 'Author', value: 'John Smith'}
        ]
    },
]);

u.writeHtml();
</script>