SmartSave - A File Sharing Android App using s3 and Lambda based backend

7 min readFeb 13, 2021

I recently created an Android App using AWS cloud services at the backend to store the files directly to the s3 storage and wanted to share my experiences about the problems faced by me while developing it. In this post, I am going to describe the functioning of the app along with its architecture with links to the other posts related to it. This app provides users with the options to share the files with other users by using a sharable URL (That is accessible through this app) to other users and options to control the accessibility of the files as public or private with access to configured users only.

Smart Save — File Sharing operation

Serverless Architecture Used

The bellow diagram describes the Serverless Architecture used by in this project.

The workflow for used in different operations by the applications is as shown bellow.

For Login and Signup Operations:

For File Operations (Create Folder, delete a file, share files):

For File Upload and Download Operations:

Configuring AWS Gateway

First, all of the requests are mapped using the AWS Gateway to the Lambda Services. Now in case of a large project where there will be many different operations such as Login, SignUp, etc. Here, we have the option of creating a separate lambda function and endpoint (in API Gateway) for each of them and then linking the lambda functions by invoking the others when needed. This approach is suitable for a large scale project where each part is handled by a different team. The main advantage of this is that since the endpoints are separate for each functionality, they can be uniquely configured for each and hence it provides better flexibility.

The other option is to use a single endpoint that will be pointing towards a lambda function that can be used to route the request to different functions as needed. I have used this method while designing the lambda function and decided to keep only the function for sending the SMS using SNS as a separate module.

For this project, I have configured the API with the default URL path parameter ‘upload’ and a dynamic path variable optype. Hence all the requests to this API will be of format https://***.execute-api.ap-southeast-1.amazonaws.com/v2/upload/{optype}where optype is the type of operation. For example, for Login (URL will be https://***.execute-api.ap-southeast-1.amazonaws.com/v2/upload/Login).

The integration Request section is used to map the requests before it is sent to the backend Lambda function. The lambda function with the handler to map the requests is ‘Lambda-S3-Connect’ as shown in the above picture.

The query string parameters configured are fileName, fileType, id which are used for creating uploading and downloading presigned URLs to S3.

A Preview Of the Integration Request Mapping Option available in API Gateway

The request parameters along with the URL query strings are overridden in the Integration Request option for the post method. The Request mapping used for this project is:

#set($params = $util.parseJson($input.json(‘$’))) 
{
 “opType” : “$input.params(‘optype’)”,
 #foreach($paramName in $params.keySet())
 “$paramName” : “$util.escapeJavaScript($params.get($paramName))”
 #if($foreach.hasNext),#end
 #end
 #foreach($qparam in $input.params().querystring.keySet())
 #if($foreach.index == 0), #end
 “$qparam” : “$util.escapeJavaScript($input.params().querystring.get($qparam))”
 #if($foreach.hasNext),#end
 #end
}

Here, I am simply iterating over the request body and request parameters (if any) into a simple json request before it is being sent to the Lambda Functions.

AWS Lambda Integration:

AWS Lambda is a serverless service offered by Amazon in which the user is charged based on the number of resources used by the code ( running time and memory used during execution) without the need to set up a server or other resources ( such as base infrastructure).

AWS Lambda Endpoint:

The requests are mapped to a single Lambda function - Lambda-S3-Connect that acts as a gateway to the different operations based on the ‘opType’ path parameters in the URL.

Here, three separate files are used to group different operations into separate groups. They are:

LoginOrc.py - This is used to group the functions for Login, SignUp, validate and Generate OTP operations.
S3ConnOrc.py - This is used to group the functions for File Creation, Deletion, upload and download operations.
FileShareOrc.py - This is used to group the functions for File Sharing operations.

Lambda Function For Sending OTP

A separate Lambda function SendOTPSmS is used to send the OTPs based on the type variable in the request. This function is written in node js to send the OTPs using the AWS SNS (Simple Notification Service).

Invoking the OTP Generation Function

This function can be invoked from the required methods to send OTP using the invoke function in the boot package in case of python.

Invoking a different lambda function in python

Preview of the OTP Being Sent Along with the pop-up in the app to capture it

OTP Generation And Validation

All of the requests are only processed after the authentication token in the request is validated. This was done to increase the security of the services. All of the requests contain a six-digit OTP/Authentication Token that is validated. These are valid for a period of 24 hours and must be regenerated. They can be regenerated as a part of the login operation. The user will be prompted to enter the OTP and will be logged in again using it.

There are two different types of OTP generated for this application. They are:

A Six-Digit numeric OTP generated to be used as an auth key before performing the requested operation.
A Four-Digit numeric OTP generated to be used for Phone number validation and Login Operations to login in to more than one devices.

The bellow code contains the two methods used to validate and generate the OTPs. The testMethod contains the code to call upon the methods to generate and validate methods for login and other operations.

Establishing MySQL Connections

A db.t2.micro class Amazon RDS ( Relational Database Service ) Mysql instance was used to provide the backend database support for the project and the database connections were handled by using the pymysql module in python.

Establishing a connection to the database:

try:
 conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
 except pymysql.MySQLError as e:
 logger.error(“ERROR: Unexpected error: Could not connect to MySQL instance.”)
 logger.error(e)
 sys.exit()

2. Executing queries using pymysql module:

Manipulating s3 content

The objects in the s3 bucket are manipulated using the client/resource object created from boto3 package.

Generating a sharable URL

The files uploaded can be shared with other users using this app. There are two options to share a file:

Making it Public: Anyone with the shared URL can access it using the shared app as long as the file remains public.
Sharing it to specific users: The file can also be shared with specific users alone. In this case, the file can be accessed using the URL by only those users who have access to it.

The shared File generated using the base URL http://www.m.smartsave.com/loadSharedFile with the query string parameters specifying the file details such as fileId, fileName, fileType etc. The URL is configured to be accessible by the app which re-routes it to the required API.

1. Backend Code used in Lambda Function

2. Android Side Code

The below code is used to prompt the user to share the URL for the file via any message sharing application installed in the mobile along with some of the details about the file being shared.

A Sample URL created for Sharing a file sent using WhatsApp

AWS IAM Policies for the Lambda Function

Six policies are attached to a role named Lamda-S3-Access created for the Lambda function. These policies are needed to execute certain functions by Lambda. They are:

AmazonRDS Access: This is needed for access to the Relational database services which hosts the MySQL Database instance used to store additional data.
AmazonS3 Access: This is needed for access to the S3 services which store all of the data of the files and to perform upload, download and delete operations.
AWSLambdaBasicExecutionRole: This policy is needed for monitoring and debugging the functioning of the Lambda application by integrating Cloudwatch logs.
AmazonSNS Access: This is used for accessing the AWS Simple Notification Services for pushing and publication notifications to users via SMS and push notifications.
Invoke_Function: This policy is needed to provide permission to the Lambda function to invoke other Lambda functions.