Mobile App to Upload a Photo or a Document to S3 + Aws

In web and mobile applications, it'south common to provide users with the power to upload information. Your application may allow users to upload PDFs and documents, or media such as photos or videos. Every modern web server engineering has mechanisms to let this functionality. Typically, in the server-based environment, the process follows this flow:

1. The user uploads the file to the application server.
2. The application server saves the upload to a temporary space for processing.
3. The application transfers the file to a database, file server, or object store for persistent storage.

While the procedure is simple, it can have meaning side-furnishings on the performance of the web-server in busier applications. Media uploads are typically large, so transferring these can represent a large share of network I/O and server CPU time. You lot must also manage the country of the transfer to ensure that the entire object is successfully uploaded, and manage retries and errors.

This is challenging for applications with spiky traffic patterns. For instance, in a web awarding that specializes in sending vacation greetings, it may experience near traffic only around holidays. If thousands of users effort to upload media around the aforementioned time, this requires you to scale out the application server and ensure that there is sufficient network bandwidth bachelor.

By directly uploading these files to Amazon S3, you lot tin avert proxying these requests through your application server. This tin can significantly reduce network traffic and server CPU usage, and enable your application server to handle other requests during busy periods. S3 as well is highly bachelor and durable, making it an ideal persistent shop for user uploads.

In this blog post, I walk through how to implement serverless uploads and show the benefits of this approach. This pattern is used in the Happy Path web application. Yous can download the code from this weblog postal service in this GitHub repo.

Overview of serverless uploading to S3

When you lot upload directly to an S3 saucepan, you must first request a signed URL from the Amazon S3 service. You can so upload directly using the signed URL. This is two-step process for your application front end end:

1. Call an Amazon API Gateway endpoint, which invokes the getSignedURL Lambda office. This gets a signed URL from the S3 bucket.
2. Directly upload the file from the awarding to the S3 bucket.

To deploy the S3 uploader example in your AWS account:

1. Navigate to the S3 uploader repo and install the prerequisites listed in the README.md.
2. In a final window, run:
   git clone https://github.com/aws-samples/amazon-s3-presigned-urls-aws-sam
cd amazon-s3-presigned-urls-aws-sam
sam deploy --guided

3. At the prompts, enter s3uploader for Stack Proper name and select your preferred Region. Once the deployment is complete, notation the APIendpoint output.The API endpoint value is the base URL. The upload URL is the API endpoint with /uploads appended. For instance: https://ab123345677.execute-api.us-w-2.amazonaws.com/uploads.

Testing the application

I show two ways to exam this application. The first is with Postman, which allows you to direct phone call the API and upload a binary file with the signed URL. The second is with a basic frontend application that demonstrates how to integrate the API.

To exam using Postman:

1. First, copy the API endpoint from the output of the deployment.
2. In the Postman interface, paste the API endpoint into the box labeled Enter asking URL.
3. Choose Send.
4. After the request is complete, the Torso section shows a JSON response. The uploadURL attribute contains the signed URL. Copy this aspect to the clipboard.
5. Select the + icon adjacent to the tabs to create a new asking.
6. Using the dropdown, change the method from GET to PUT. Paste the URL into the Enter asking URL box.
7. Choose the Body tab, so the binary radio button.
8. Choose Select file and choose a JPG file to upload.
   Choose Send. You see a 200 OK response afterwards the file is uploaded.
9. Navigate to the S3 console, and open the S3 bucket created by the deployment. In the saucepan, you run across the JPG file uploaded via Postman.

To test with the sample frontend application:

1. Re-create index.html from the example's repo to an S3 saucepan.
2. Update the object'southward permissions to make it publicly readable.
3. In a browser, navigate to the public URL of index.html file.
4. Select Choose file and and then select a JPG file to upload in the file picker. Choose Upload image. When the upload completes, a confirmation bulletin is displayed.
5. Navigate to the S3 panel, and open the S3 bucket created by the deployment. In the bucket, yous see the second JPG file you uploaded from the browser.

Understanding the S3 uploading process

When uploading objects to S3 from a web awarding, you must configure S3 for Cantankerous-Origin Resource Sharing (CORS). CORS rules are defined as an XML document on the bucket. Using AWS SAM, you lot can configure CORS every bit office of the resource definition in the AWS SAM template:

                      S3UploadBucket:     Blazon: AWS::S3::Saucepan     Properties:       CorsConfiguration:         CorsRules:         - AllowedHeaders:             - "*"           AllowedMethods:             - GET             - PUT             - Caput           AllowedOrigins:             - "*"                  

The preceding policy allows all headers and origins – it's recommended that you use a more restrictive policy for production workloads.

In the first stride of the procedure, the API endpoint invokes the Lambda function to make the signed URL asking. The Lambda function contains the following code:

          const AWS = require('aws-sdk') AWS.config.update({ region: process.env.AWS_REGION }) const s3 = new AWS.S3() const URL_EXPIRATION_SECONDS = 300  // Main Lambda entry betoken exports.handler = async (event) => {   return look getUploadURL(event) }  const getUploadURL = async function(event) {   const randomID = parseInt(Math.random() * 10000000)   const Central = `${randomID}.jpg`    // Become signed URL from S3   const s3Params = {     Bucket: procedure.env.UploadBucket,     Cardinal,     Expires: URL_EXPIRATION_SECONDS,     ContentType: 'image/jpeg'   }   const uploadURL = await s3.getSignedUrlPromise('putObject', s3Params)   return JSON.stringify({     uploadURL: uploadURL,     Key   }) }                  

This role determines the name, or primal, of the uploaded object, using a random number. The s3Params object defines the accepted content type and also specifies the expiration of the key. In this case, the key is valid for 300 seconds. The signed URL is returned as function of a JSON object including the key for the calling application.

The signed URL contains a security token with permissions to upload this single object to this bucket. To successfully generate this token, the code calling getSignedUrlPromise must take s3:putObject permissions for the bucket. This Lambda function is granted the S3WritePolicy policy to the saucepan past the AWS SAM template.

The uploaded object must match the same file proper name and content type equally defined in the parameters. An object matching the parameters may be uploaded multiple times, providing that the upload process starts earlier the token expires. The default expiration is 15 minutes only you may want to specify shorter expirations depending upon your employ case.

One time the frontend application receives the API endpoint response, information technology has the signed URL. The frontend awarding then uses the PUT method to upload binary data direct to the signed URL:

          permit blobData = new Hulk([new Uint8Array(array)], {type: 'image/jpeg'}) const result = await fetch(signedURL, {   method: 'PUT',   body: blobData })                  

At this betoken, the caller awarding is interacting directly with the S3 service and not with your API endpoint or Lambda office. S3 returns a 200 HTML status code in one case the upload is complete.

For applications expecting a large number of user uploads, this provides a uncomplicated way to offload a big amount of network traffic to S3, away from your backend infrastructure.

Adding authentication to the upload procedure

The current API endpoint is open, available to whatever service on the cyberspace. This ways that anyone can upload a JPG file once they receive the signed URL. In most production systems, developers desire to use hallmark to control who has access to the API, and who can upload files to your S3 buckets.

You can restrict access to this API by using an authorizer. This sample uses HTTP APIs, which support JWT authorizers. This allows you to control access to the API via an identity provider, which could be a service such every bit Amazon Cognito or Auth0.

The Happy Path application just allows signed-in users to upload files, using Auth0 as the identity provider. The sample repo contains a second AWS SAM template, templateWithAuth.yaml, which shows how you can add an authorizer to the API:

                      MyApi:     Type: AWS::Serverless::HttpApi     Backdrop:       Auth:         Authorizers:           MyAuthorizer:             JwtConfiguration:               issuer: !Ref Auth0issuer               audience:                 - https://auth0-jwt-authorizer             IdentitySource: "$request.header.Authority"         DefaultAuthorizer: MyAuthorizer                  

Both the issuer and audience attributes are provided by the Auth0 configuration. By specifying this authorizer as the default authorizer, it is used automatically for all routes using this API. Read function i of the Enquire Around Me series to acquire more than most configuring Auth0 and authorizers with HTTP APIs.

Subsequently authentication is added, the calling web awarding provides a JWT token in the headers of the asking:

          const response = await axios.get(API_ENDPOINT_URL, {   headers: {     Say-so: `Bearer ${token}`         } })                  

API Gateway evaluates this token before invoking the getUploadURL Lambda role. This ensures that only authenticated users tin upload objects to the S3 bucket.

Modifying ACLs and creating publicly readable objects

In the current implementation, the uploaded object is non publicly accessible. To make an uploaded object publicly readable, you must ready its access command listing (ACL). In that location are preconfigured ACLs available in S3, including a public-read option, which makes an object readable by anyone on the internet. Ready the appropriate ACL in the params object earlier calling s3.getSignedUrl:

          const s3Params = {   Bucket: procedure.env.UploadBucket,   Key,   Expires: URL_EXPIRATION_SECONDS,   ContentType: 'epitome/jpeg',   ACL: 'public-read' }                  

Since the Lambda role must have the appropriate bucket permissions to sign the request, you lot must besides ensure that the function has PutObjectAcl permission. In AWS SAM, you tin can add the permission to the Lambda function with this policy:

                      - Statement:           - Result: Permit             Resource: !Sub 'arn:aws:s3:::${S3UploadBucket}/'             Action:               - s3:putObjectAcl                  

Conclusion

Many spider web and mobile applications allow users to upload data, including large media files like images and videos. In a traditional server-based awarding, this can create heavy load on the application server, and too employ a considerable corporeality of network bandwidth.

By enabling users to upload files to Amazon S3, this serverless pattern moves the network load away from your service. This can brand your application much more scalable, and capable of handling spiky traffic.

This blog post walks through a sample application repo and explains the process for retrieving a signed URL from S3. It explains how to the test the URLs in both Postman and in a spider web application. Finally, I explain how to add hallmark and brand uploaded objects publicly accessible.

To learn more, see this video walkthrough that shows how to upload directly to S3 from a frontend web application. For more than serverless learning resource, visit https://serverlessland.com.

gatewoodillaitty.blogspot.com

Source: https://aws.amazon.com/blogs/compute/uploading-to-amazon-s3-directly-from-a-web-or-mobile-application/

Share this post