Uploading to Amazon S3 Cloud Storages

Uploading to Amazon S3

When you create an account in the Amazon Web Service (AWS) you obtain your Access Key Id and Secret Access Key. These parameters are used for authentication purposes:

• AWS Access Key Id identifies your AWS account. This key is included in every AWS request to identify the user.
• Secret Access Key is never included into requests and used to sign AWS requests. Each request must contain a valid signature calculated using the Secret Access Key, otherwise authentication will fail.

To store objects Amazon S3 uses buckets which are quite similar to Internet domain names and must be unique within the Amazon S3. For example, if the object named myphotos/holiday.jpg is stored in the mybucket bucket, the URL to access this object is http://mybucket.s3.amazonaws.com/myphotos/holiday.jpg. Every bucket has an associated access control list used to verify whether the sender is able to access this bucket. Each object represents a file and metadata stored within a bucket using a unique key. A bucket and key together uniquely identify each object stored in the Amazon S3.

In the simplest case, to create a new Amazon S3 object you need to send the POST request which identifies the sender and defines the object to be created. It means that ActiveX/Java Uploader can be used for this purpose. Thus, to implement this functionality you can take an advantage of one of the approaches described below.

To be sure that all files are uploaded successfully, you can initiate data integrity check. Find short description of this feature along with code snippets in the Checking Data Integrity part of this article.

Using ActiveX/Java Uploader JavaScript

This approach is more complicated but can be used with any ActiveX/Java Uploader supported server platform. It lies in using the amazonS3Extender class defined in the /Scripts/aurigma.uploader.amazons3.js file. However, before you initialize this class instance you should prepare additional data required for uploading to Amazon S3 storage: the security policy and signature.

The policy is a UTF-8 and Base64 encoded JSON document which specifies the date when this policy expires and conditions the request must meet. The expiration date must be in ISO8601 GMT date format. The conditions are used to validate the uploaded object and include definitions for access control list, bucket, key, and metadata fields (both default and custom).

Here is the code sample that creates a policy document which expires in 100 minutes and defines an access control list option, bucket name, uploaded file, two default fields (width and height), and one custom (author).

public string ConstructPolicy()
{
    StringBuilder policy = new StringBuilder();
    DateTime expDate = DateTime.Now.ToUniversalTime().AddSeconds(6000);

    policy.AppendLine("{ \"expiration\": \"" + expDate.ToString("s") + ".000Z\"");
    policy.AppendLine(", \"conditions\": [");
    policy.AppendLine(" {\"acl\": \"public-read\" }");
    policy.AppendLine(" , {\"bucket\": \"" + _bucket + "\" }");
    policy.AppendLine(" , {\"success_action_status\": \"200\"}");
    policy.AppendLine(" , [\"starts-with\", \"$key\", \"\"]");

    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-width\", \"\"]");
    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-height\", \"\"]");
    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-author\", \"\"]");

    policy.AppendLine("]");
    policy.AppendLine("}");

    // Encode the policy using UTF-8.
    byte[] bytes = Encoding.UTF8.GetBytes(policy.ToString());
    
    // Encode those UTF-8 bytes using Base64.
    return Convert.ToBase64String(bytes);
}

To create a signature, sign the policy with your Secret Access Key using HMAC SHA-1 and encode the SHA-1 signature using Base64. See the code sample below, it demonstrates how to create a signature in .NET.

public string CreateSignature()
{
    _policy = ConstructPolicy();

    // Sign the policy with your Secret Access Key using HMAC SHA-1.
    System.Security.Cryptography.HMACSHA1 hmac = new System.Security.Cryptography.HMACSHA1();
    hmac.Key = System.Text.Encoding.UTF8.GetBytes(_secretAccessKey);

    byte[] signb = hmac.ComputeHash(System.Text.Encoding.UTF8.GetBytes(_policy));

    // Encode the SHA-1 signature using Base64.
    _signature = Convert.ToBase64String(signb);
}

Then initialize an amazonS3Extender object and specify your AWSAccessKeyId and bucket using the accessKeyId and bucket properties. However, if you have a host name associated with an Amazon S3 bucket you should additionally specify this host name using the bucketHostName property.

Then for each converted file add a fileSettings instance specifying:

• access control list which should be equal to the acl value defined in the policy (acl)
• key of Amazon S3 object to be created for this converted file (key); here you can use the ${filename} variable to set a name of this file
• policy constructed and encoded before (policy)
• signature (signature)
• set of predefined and custom metadata fields which will be uploaded within this file (meta); these fields should be defined in the policy

<script type="text/javascript" src="Scripts/aurigma.uploader.js">  </script>
<script type="text/javascript" src="Scripts/aurigma.uploader.amazons3.js">  </script>
<script type="text/javascript">
var u = $au.uploader({
    id: 'Uploader1',
    converters: [
        {mode: '*.*=SourceFile'},
        // ... Other converters
    ]
});
    
var as3 = $au.amazonS3Extender(u);
as3.accessKeyId(_AWSAccessKeyId);
as3.bucket(_bucket);
as3.bucketHostName('http://yourdomain.com/');
as3.converters([
    {
        acl: 'public-read',
        key: '${filename}',
        policy: _policy,
        signature: _signature,
        meta: [
            { name: 'Width', field: 'SourceWidth_[itemIndex]'},
            { name: 'Height', field: 'SourceHeight_[itemIndex]'},
            { name: 'Author', value: 'John Smith'}
        ]
    },
    // ... Other converters
]);

u.writeHtml();
</script>

See the Amazon S3 Documentation for the detailed information on how to upload files to Amazon S3 using POST.

Using ActiveX/Java Uploader ASP.NET

This approach is simpler but it is applicable for ASP.NET applications only. It saves you the trouble of preparing the policy and signature yourself and familiarizing with the Amazon S3 API. You just need to perform the following steps to force ActiveX/Java Uploader to upload files directly to Amazon S3 storage:

1. Initialize an AmazonS3Extender object.
2. Specify your AWSAccessKeyId, secretAccessKey, and bucket as values of the AWSAccessKeyId, SecretAccessKey, and Bucket properties.

3. For each converted file add a FileSettings instance and configure it as follows:

   1. Set acl and key for this file using the Acl and Key properties.
   2. Add as many metadata fields as you like by adding PredefinedMetaProperty and CustomMetaProperty objects to the FileSettings.Meta collection.

<%@ Register assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader" tagprefix="cc1" %>
<%@ Register assembly="Aurigma.ImageUploader" namespace="Aurigma.ImageUploader.AmazonS3" tagprefix="cc1" %>
<!--... Omitted for brevity ...-->
<form id="form1" runat="server">
    <cc1:Uploader ID="Uploader1" runat="server">
        <Converters>
            <cc1:Converter Mode="*.*=SourceFile" />
            <!-- Other converters -->
        </Converters>
    </cc1:Uploader>
    <cc1:AmazonS3Extender runat="server" ID="AmazonS3Extender1" TargetControlID="Uploader1"
        AWSAccessKeyId="<%$ _AWSAccessKeyId %>"
        Bucket="<%$ _Bucket %>" 
        SecretAccessKey="<%$ _SecretAccessKey %>"
        PolicyExpiration="6000">
        <cc1:FileSettings Acl="public-read" Key="${filename}">
            <cc1:PredefinedMetaProperty Field="SourceWidth_[itemIndex]" Name="width" />
            <cc1:PredefinedMetaProperty Field="SourceHeight_[itemIndex]" Name="height" />
            <cc1:CustomMetaProperty Name="Author" Value="John Smith" />
        </cc1:FileSettings>
        <!-- Other files -->
    </cc1:AmazonS3Extender>
</form>

Using ActiveX/Java Uploader PHP

The present approach is to use ActiveX/Java Uploader PHP to upload files to Amazon S3 storage. To implement this functionality ActiveX/Java Uploader PHP exposes the same name classes and the same object model as the ASP.NET control (see the section above). So, to extend ActiveX/Java Uploader with a direct upload to Amazon S3 storage, follow the steps below:

1. Create an AmazonS3Extender object.
2. Specify your AWSAccessKeyId, bucket, and secretAccessKey as values of the AWSAccessKeyId, Bucket, and SecretAccessKey properties respectively.

3. For each converted file add a FileSettings instance and configure it as follows:

   1. Set acl and key for this file using the Acl and Key properties.
   2. Add predefined and custom metadata fields to the FileSettings.Meta array.

require_once "ImageUploaderPHP/Uploader.class.php";
require_once "ImageUploaderPHP/AmazonS3Extender.class.php";

$uploader = new Uploader("Uploader1");
$converters = &$uploader->getConverters();
$converter = new Converter();
$converter->setMode("*.*=SourceFile");
$converters[] = $converter;
//Other converters

$as3 = new AmazonS3Extender($uploader);
$as3->setAWSAccessKeyId($AWSAccessKeyId);
$as3->setBucket($Bucket);
$as3->setSecretAccessKey($SecretAccessKey);
$files = &$as3->getFiles();
$file = new FileSettings();
$file->setAcl("public-read");
$file->setKey('${filename}');
$file->setMeta(array(
  array("name" => "Author", "value" => "John Smith"),
  array("name" => "width", "field" => "SourceWidth_[itemIndex]"),
  array("name" => "height", "field" => "SourceHeight_[itemIndex]")
));
$files[] = $file;
//Other files

$uploader->render();

Checking Upload Integrity

Amazon S3 supports checking integrity of uploaded files using MD5 hash sums. To initiate this check set the AmazonS3Extender.CheckIntegrityCheckIntegrity (ASP.NET)CheckIntegrity (PHP)checkIntegrity (JavaScript) property to true. In this case a Content-MD5 header will be added to your upload, and S3 will verify the MD5 value in the header and actual MD5 value of the uploaded file before storing the file. If the values do not match, S3 will not save the file and will return an error code in the response.

The following code snippet demonstrates how to turn on integrity check.

<%@ Register assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader" tagprefix="cc1" %>
<%@ Register assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader.AmazonS3" tagprefix="cc1" %>

<body>
    <form id="form1" runat="server">
        <cc1:Uploader ID="Uploader1" runat="server">
        <%--... Converters and other settings--%>
        </cc1:Uploader>
        <cc1:AmazonS3Extender runat="server" ID="AmazonS3Extender1" TargetControlID="Uploader1"
           CheckIntegrity=True>
        <%--... File settings--%>
        </cc1:AmazonS3Extender>
    </form>
</body>

require_once "ImageUploaderPHP/Uploader.class.php";
require_once "ImageUploaderPHP/AmazonS3Extender.class.php";

$uploader = new Uploader("Uploader1");
//... Converters and other settings
$as3 = new AmazonS3Extender($uploader);
//... Authentication settings
$as3->setCheckIntegrity(true);
//... File settings

$uploader->render();

<script type="text/javascript" src="Scripts/aurigma.uploader.js">  </script>
<script type="text/javascript" src="Scripts/aurigma.uploader.amazons3.js">  </script>
<script type="text/javascript">
var u = $au.uploader({
    id: 'Uploader1',
    //... Converters and other settings
});
    
var as3 = $au.amazonS3Extender(u);
//... Authentication settings
as3.checkIntegrity(true);
//... File settings

u.writeHtml();
</script>

See Also

Reference

• AmazonS3Extender ClassAmazonS3Extender (ASP.NET)AmazonS3Extender (PHP)amazonS3Extender (JavaScript)
• FileSettings ClassFileSettings (ASP.NET)FileSettings (PHP)fileSettings (JavaScript)
• POST Field Reference

Manual

• Configuring Files to be Uploaded in ActiveX/Java Uploader

Other Resources

• Amazon Simple Storage Service API Reference