S3 Uploads with Pre-signed URLs

In case you didn’t know we also do consulting services for Amazon Web Services users. (One team member has achieved all 5 AWS certifications!) One thing clients commonly want to do is allow users to upload images (and other files but typically images) directly to S3. By uploading directly to S3, you don’t have to make a custom webservice to handle the upload and then send the file to S3. Direct uploads by the client to S3 are much more efficient.

An easy way to accomplish direct client uploads to S3 is through the use of pre-signed URLs. Originally, pre-signed URLs were available just for downloading files from S3 but they can be used for uploads, too. (A pre-signed URL for a download is a time-expiring link that allows anyone with that URL to download the file. They are very useful.) For uploading, a pre-signed URL allows anyone with the URL to upload a file to your S3 bucket. You define the target location in S3 and set some options and the URL is generated. You can then use that URL in an HTML upload form and the user’s file will upload directly to S3.

Here’s some sample code from the AWS documentation that shows how to generate a pre-signed URL for uploading:

AmazonS3 s3Client = new AmazonS3Client(new ProfileCredentialsProvider()); 

java.util.Date expiration = new java.util.Date();
long msec = expiration.getTime();
msec += 1000 * 60 * 60; // Add 1 hour.
expiration.setTime(msec);

GeneratePresignedUrlRequest generatePresignedUrlRequest = new GeneratePresignedUrlRequest(bucketName, objectKey);
generatePresignedUrlRequest.setMethod(HttpMethod.PUT); 
generatePresignedUrlRequest.setExpiration(expiration);
             
URL s = s3client.generatePresignedUrl(generatePresignedUrlRequest); 

// Use the pre-signed URL to upload an object.

In the code we set the expiration for the URL and the HTTP method and then we generate the URL. (See the full code example.)

And that’s where the hold up comes. There’s a slight gotcha that is very often missed by our clients. Our clients encounter the problem when a user goes to upload a file using the pre-signed URL and receive a forbidden error. Here’s the important point:

The user who generates the pre-signed URL must have permissions to upload a file to S3. 

This piece of information is in the AWS documentation but it is missed a lot. Here’s the text:

A pre-signed URL gives you access to the object identified in the URL, provided that the creator of the pre-signed URL has permissions to access that object. That is, if you receive a pre-signed URL to upload an object, you can upload the object only if the creator of the pre-signed URL has the necessary permissions to upload that object.

Hopefully this will save you a few minutes of head bashing in case you run into this the next time you’re implementing S3 uploads with pre-signed URLs.

Leave a Reply