Setting up Supertokens with AWS - A comprehensive guide

This is a comprehensive guide that describes how to set up Supertokens with AWS. The goal is to have a single, high-available and fast end point that all clients can authenticate their users with.

tl;dr

At the end of this guide you will have a url like https://auth.example.com that is distributed around the world and ready for use for any of your clients.

This guide covers:

• Setting up a database for all your user data
• Setting up a dedicated VPC with private subnets for maximum security of your user database
• Launching Supertokens server instances using AWS ECS
• Creating an api for authentication using AWS Api Gateway and AWS Lambda
• Distributing reads to your authentication api around the world using AWS Cloudfront
• Setting up DNS routing with Route 53

"An abstract painting of a computer with a lock on the screen" - DALL-E

Table of contents

0. Motivation

There are a couple of great providers that offer an easy way of authenticating, authorizing and managing users for your application (e.g. auth0, AWS Cognito). They all come with a free tier or at least are fairly inexpensive for as long as your application only has a few users. But these services get expensive very quickly as your user base grows.

When we were looking for a free, open-source solution to solve the same problem we found Supertokens. Supertokens has some guidelines and recipes on how to deploy their service on AWS, but it is far from describing an end-to-end real-life situation.

In this guide we are going to describe how you can set up your own Supertokens service with AWS step-by-step using AWS SAM. Warning ahead: It is going to get complicated (at least if your not familiar with Cloudformation and/or how AWS services are connected to each other).

Instead of showing you the final result and walking you through each component, this guide is organized in the same order that we followed to understand and explore how every piece works together. We will update our solutions along the way as the pieces get more complex.

If you are interested in only the final result, please jump to the end of the page. If you really want to understand what the final template is doing, please follow the guide step-by-step.

1. Setting up project

1.1. Prerequisites

Before getting started you will need to have:

1. An AWS account
2. AWS SAM cli installed

1.2. Creating samconfig.toml and template.yaml

In the root of the project directory we set up a samconfig.yaml. This file is automatically created for you when you run sam deploy --guided, but you can also create that file in advance.

samconfig.toml

version = 0.1
[default]
[default.deploy]
[default.deploy.parameters]
stack_name = "example-stack-name"
s3_bucket = "example-bucket-name"
s3_prefix = "optional-prefix"
region = "eu-central-1"
capabilities = "CAPABILITY_IAM"

You will need to create your S3 bucket in advance either through the console or the CLI.

template.yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Supertokens service

2. Setting up VPC

We are going to host our api and EC2 instance(s) in a single public subnet and our database and Supertokens service in two private subnets. We need two private subnets, because we are going to use a load balancer and a load balancer requires at least two different subnets.

If a subnet is associated with a route table that has a route to an internet gateway, it’s known as a public subnet. If a subnet is associated with a route table that does not have a route to an internet gateway, it’s known as a private subnet.

In your public subnet’s route table, you can specify a route for the internet gateway to all destinations not explicitly known to the route table (0.0.0.0/0 for IPv4 or ::/0 for IPv6)

[Source]

With the following Cloudformation snippet we are going to accomplish exactly that. The snippet describes the following steps in that order:

1. Creating a parameter called StVpcAZs that lets us define in what availability zones we want to deploy our private subnets. It is a best practice to not hardcode AZs in your Cloudformation template.
2. Creating a VPC
3. Creating an Internet Gateway
4. Attaching the Internet Gateway to the VPC
5. Creating two route tables for our VPC, one for public traffic and one for private traffic
6. Adding a route to the “public” route table that routes 0.0.0.0/0 traffic to the Internet Gateway
7. Creating one public and two private subnets
8. Associating our route tables to our subnets

Parameters:
  StVpcAZs:
    Type: CommaDelimitedList
    Default: 'us-east-1a, us-east-1b'

Resources:

  StVpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: "10.1.0.0/16"

  StInternetGateway:
    Type: AWS::EC2::InternetGateway

  StAttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref StVpc
      InternetGatewayId: !Ref StInternetGateway

  StPublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref StVpc

  StPrivateRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref StVpc

  StRouteToIGW:
    Type: AWS::EC2::Route
    DependsOn:
      - StAttachGateway
    Properties:
      RouteTableId: !Ref StPublicRouteTable
      DestinationCidrBlock: "0.0.0.0/0"
      GatewayId: !Ref StInternetGateway

  StPublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: "10.1.0.0/24"
      VpcId: !Ref StVpc
      AvailabilityZone: !Select [ 0, !Ref StVpcAZs ]

  StPrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: "10.1.1.0/24"
      VpcId: !Ref StVpc
      AvailabilityZone: !Select [ 0, !Ref StVpcAZs ]

  StPrivateSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: "10.1.2.0/24"
      VpcId: !Ref StVpc
      AvailabilityZone: !Select [ 1, !Ref StVpcAZs ]

  StAssignRouteTable1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref StPublicRouteTable
      SubnetId: !Ref StPublicSubnet1

  StAssignRouteTable2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref StPrivateSubnet1
      RouteTableId: !Ref StPrivateRouteTable

  StAssignRouteTable3:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref StPrivateSubnet2
      RouteTableId: !Ref StPrivateRouteTable

3. Setting up database

Supertokens let you chose between MySQL and Postgres. For this project, we select Postgres. The database instance is going to be the most expensive component of this project. At the moment of this writing, the most inexpensive instance is a Postgres t3.micro single-AZ instance with no proxy and 20 GB storage hosted in us-east-1 costing 15,44 $/month. But if you already have an RDS instance running you can obviously use that instance for this project to save costs.

When setting up a database there are a couple of things to keep in mind:

Your VPC must have at least two subnets. These subnets must be in two different Availability Zones in the AWS Region where you want to deploy your DB instance.

Your VPC must have a DB subnet group that you create. You create a DB subnet group by specifying the subnets you created.

Your VPC must have a VPC security group that allows access to the DB instance.

The CIDR blocks in each of your subnets must be large enough to accommodate spare IP addresses for Amazon RDS to use during maintenance activities, including failover and compute scaling.

[Source]

3.1. Creating a DB subnet group

Subnets are segments of a VPC’s IP address range that you designate to group your resources based on security and operational needs. A DB subnet group is a collection of subnets (typically private) that you create in a VPC and that you then designate for your DB instances.

Each DB subnet group should have subnets in at least two Availability Zones in a given AWS Region. When creating a DB instance in a VPC, you choose a DB subnet group for it. From the DB subnet group, Amazon RDS chooses a subnet and an IP address within that subnet to associate with the DB instance. The DB uses the Availability Zone that contains the subnet.

[Source]

We found a lot of information about what DB subnet groups are and how to use them, but we could not figure out why we need them.

The following snippet creates our subnet group that groups our two public subnets together. For some reason the DBSubnetGroupDescription is mandatory.

  StDBSubnetGroup:
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupName: st-db-subnet-group
      DBSubnetGroupDescription: This is our subnet group
      SubnetIds:
        - !Ref StPublicSubnet1
        - !Ref StPublicSubnet2

3.2. Creating security groups

We only want access to our database from our Supertokens service. We have not created our Suptertokens service yet, but we are going to create the security group for it in this step. That allows us to create a security group for our database that only allows access from the Supertokens service’s security group.

  StSecurityGroupService:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for our Supertokens service
      VpcId: !Ref StVpc
  
  StSecurityGroupDatabase:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for our database instance
      VpcId: !Ref StVpc

  StSecurityGroupIngress1:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      IpProtocol: "-1"
      SourceSecurityGroupId: !GetAtt ["StSecurityGroupService", "GroupId"]
      GroupId: !GetAtt ["StSecurityGroupDatabase", "GroupId"]

3.3. Creating a DB instance

The following snippet is going to create our DB instance. There are a couple of things to note:

• We are going for the cheapest solution possible. Smallest instance, minimum storage, gp2 storage, single AZ.
• We store the password of the database in the Parameter store and pass the key via parameters
• We have to reference the subnet group by name. Unfortunately AWS::RDS::DBSubnetGroup does not return its Name, therefore we have to copy the name to the DBSubnetGroupName attribute either manually or using parameters.
• We name the database supertokens because that is what the Supertokens service expects.
• We are assigning the security group that we have created in the previous step.
• DeletionProtection is set to true so that data is not lost when this stack gets accidentally deleted.
• There are dozens of other attributes on a DBInstance resource in Cloudformation that you definitely want to consider in a production-ready version. We kept it at a bare minimum.

Parameters:

  # ... 

  StPostgresPassword:
    Type: "AWS::SSM::Parameter::Value<String>"
    Default: "suptertokens-postgres-password"

  #...

Resources:

  #...

  StDatabase:
    Type: AWS::RDS::DBInstance
    DependsOn:
      - StDBSubnetGroup
    Properties:
      DBName: supertokens
      DBSubnetGroupName: st-db-subnet-group
      VPCSecurityGroups:
        - !Ref StSecurityGroupDatabase
      DBInstanceClass: "db.t3.micro"
      AllocatedStorage: 20
      MultiAZ: false
      Engine: postgres
      EngineVersion: "13.7"
      StorageType: gp2
      DeletionProtection: true
      MasterUsername: postgres
      MasterUserPassword: !Ref StPostgresPassword

4. Launching Supertokens server instances

Launching Supertokens server instances might sound simple, but there are a couple of different sets of configuration that we need to consider. We can group them in three different categories and we will tackle every category one by one:

1. Setting up a load balancer to have a single end point from where traffic is distributed.
2. Setting up an ECS cluster with task definitions that are able to run the Supertokens docker image.
3. Setting up an auto scaling group and a launch template that guarantees that there is always an EC2 instance running.

4.1. Setting up load balancer

We are going to set up a load balancer and a target group. This is not strictly required, but it makes a lot of sense to register all of our future Supertokens instances with a target group and only have to think about communicating with the load balancer moving forward.

The target groups uses a health check by calling the /hello end point on the services (Source).

We are also adding a security group for the load balancer and an ingress rule for the Supertokens security group.

  StSecurityGroupLoadBalancer:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for our load balancer
      VpcId: !Ref StVpc

  StLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      IpAddressType: ipv4
      Scheme: internal
      SecurityGroups:
        - Ref: StSecurityGroupLoadBalancer
      Subnets:
        - Ref: StPrivateSubnet1
        - Ref: StPrivateSubnet2
      Type: application

  StTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      VpcId: !Ref StVpc
      Protocol: HTTP
      Port: 3567
      HealthCheckEnabled: true
      HealthCheckIntervalSeconds: 30
      HealthCheckPath: /hello
      HealthCheckPort: traffic-port
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 5
      UnhealthyThresholdCount: 2
      IpAddressType: ipv4
      Matcher:
        HttpCode: "200"
      ProtocolVersion: HTTP1
      TargetType: ip

  StListener:
    Type: 'AWS::ElasticLoadBalancingV2::Listener'
    Properties:
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref StTargetGroup
      LoadBalancerArn: !Ref StLoadBalancer
      Port: '3567'
      Protocol: HTTP

  StSecurityGroupIngress2:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      IpProtocol: "-1"
      SourceSecurityGroupId: !GetAtt ["StSecurityGroupLoadBalancer", "GroupId"]
      GroupId: !GetAtt ["StSecurityGroupService", "GroupId"]

4.2. Setting up ECS cluster

We use a TaskDefinition to define what Docker image we want to launch. In our case it is registry.supertokens.io/supertokens/supertokens-postgresql. We pass connection details from our Postgres database from one of the previous steps to our service as environment variables.

For additional security we are using api keys for our Supertokens service. They are used to authenticate requests from the backend to the service. We believe that the communication between our backend and the service is already very secure, but it does not hurt to add another level of security. We store the api key also in the parameter store and pass it to the service. We will also pass it to the backend in a later step.

Note: As you can see we have disabled telemetry for the Supertokens service. As much we would like to send telemetric data to support that open source project, our Supertokens service runs in an environment without access to the internet. Therefore it cannot send anything out. Actually, the service crashes when it cannot send out telemetric data. We had to disable it.

Parameters:

  # ...

  StApiKey:
    Type: "AWS::SSM::Parameter::Value<String>"
    Description: Parameter store key that holds the api key to the Supertokens service
    Default: "supertokens-api-key"

Resources:

  # ...

  StCluster:
    Type: AWS::ECS::Cluster

  StEcsTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service: ecs-tasks.amazonaws.com
          Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

  StTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      ContainerDefinitions:
        - Essential: true
          Image: registry.supertokens.io/supertokens/supertokens-postgresql
          Environment:
            - Name: POSTGRESQL_USER
              Value: postgres
            - Name: POSTGRESQL_PASSWORD
              Value: !Ref StPostgresPassword
            - Name: POSTGRESQL_HOST
              Value: !GetAtt ["StDatabase", "Endpoint.Address"]
            - Name: POSTGRESQL_PORT
              Value: !GetAtt ["StDatabase", "Endpoint.Port"]
            - Name: API_KEYS
              Value: !Ref StApiKey
            - Name: DISABLE_TELEMETRY
              Value: 'true'
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: "/ecs/supertokens-core"
              awslogs-region: !Ref "AWS::Region"
              awslogs-stream-prefix: ecs
          Name: supertokens-core
          PortMappings:
            - HostPort: 3567
              Protocol: tcp
              ContainerPort: 3567
      Cpu: '256'
      Memory: '512'
      RuntimePlatform:
        OperatingSystemFamily: LINUX
      NetworkMode: awsvpc

  StServiceDefinition:
    Type: AWS::ECS::Service
    DependsOn:
      - StListener
    Properties:
      Cluster: !Ref StCluster
      DesiredCount: 1
      LaunchType: EC2
      NetworkConfiguration:
        AwsvpcConfiguration:
          Subnets:
          - Ref: StPrivateSubnet1
          - Ref: StPrivateSubnet2
          SecurityGroups:
          - Ref: StSecurityGroupService
      TaskDefinition:
        Ref: StTaskDefinition
      LoadBalancers:
        - TargetGroupArn: !Ref StTargetGroup
          ContainerPort: 3567
          ContainerName: supertokens-core

4.3. Setting up auto scaling group

Getting the auto scaling group and launch templates right to launch EC2 instances that we can use for our cluster was probably the most difficult thing to put together.

There are a few things that kept us busy before getting everything right:

• You can install the ecs agent on your EC2 instances yourself. But you can also use a specific AMI that is optimized for ECS.
• Since AMIs are different from region to region we had to add a mapping that maps to the correct AMI base on the region. In our template we only added support for two regions, but you can add more regions: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_windows_AMI.html
• Our EC2 instance has to have a public ip address. Otherwise the ECS service cannot talk to the agent.
• EC2 instances have to also be able to talk to the ECS service. Assigning a public ip address is not enough. In order for EC2 instances to talk to the ECS service they have to be either placed in a public subnet so that they have access to the internet or we have to create a VPC endpoint for the ECS service. We went for placing the EC2 instance in our public subnet.
• In a lot of tutorials online you find examples where they use launch configurations. Launch configuration are deprecated and AWS recommends to use launch templates instead. Therefore we use launch templates.
• We had to create an instance profile for EC2 in order to assume roles. We followed the following instruction to create that profile: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/instance_IAM_role.html

Mappings:
  AWSRegionToAMI:
    us-east-1:
      AMIID: ami-05e7fa5a3b6085a75
    eu-central-1:
      AMIID: ami-0348a4a91adc75319

Resources:

  #...

  StEc2Role:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: ecs-service
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - 'ec2:DescribeTags'
                  - 'ecs:CreateCluster'
                  - 'ecs:DeregisterContainerInstance'
                  - 'ecs:DiscoverPollEndpoint'
                  - 'ecs:Poll'
                  - 'ecs:RegisterContainerInstance'
                  - 'ecs:StartTelemetrySession'
                  - 'ecs:UpdateContainerInstancesState'
                  - 'ecs:Submit*'
                  - 'ecr:GetAuthorizationToken'
                  - 'ecr:BatchCheckLayerAvailability'
                  - 'ecr:GetDownloadUrlForLayer'
                  - 'ecr:BatchGetImage'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvent'
                Resource: '*'

  StEc2InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref StEc2Role

  StLaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties: 
      LaunchTemplateData:
        ImageId: !FindInMap 
        - AWSRegionToAMI
        - !Ref 'AWS::Region'
        - AMIID
        InstanceType: t2.micro
        IamInstanceProfile:
          Arn: !GetAtt StEc2InstanceProfile.Arn
        NetworkInterfaces:
          - DeviceIndex: 0 
            AssociatePublicIpAddress: true
            Groups:
              - !GetAtt StSecurityGroupLoadBalancer.GroupId
        UserData:
          Fn::Base64:
            !Sub |
              #!/bin/bash
              echo ECS_CLUSTER=${StCluster} >> /etc/ecs/ecs.config

  StAsg:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      LaunchTemplate:
        LaunchTemplateId: !Ref StLaunchTemplate
        Version: !GetAtt StLaunchTemplate.LatestVersionNumber
      MaxSize: '1'
      MinSize: '1'
      DesiredCapacity: '1'
      VPCZoneIdentifier:
        - Ref: StPublicSubnet1

5. Setting up API gateway

Supertokens offers three different backend sdks. We went with the NodeJS implementation.

When implementing the NodeJS SDK we loosely followed a guide provided by Supertokens. We made a couple of changes to it:

• We put everything in one file
• We set apiBasePath to /. The default is /auth, but we want the backend to listen on auth.example.com and there is no path.
• We have enabled jwt, because we want to be able to authenticate access tokens via JWT.
• We set the internal DNS name of the load balancer as connectionURI.
• We added an apiKey that we defined in a previous step

5.1. Setting up code

The following steps will transpile Typescript code to a /dist folder. The contents of that /dist folder will be deployed using AWS SAM in the next step.

1. Create /src
2. Run npm init --yes
3. Run npm install @middy/core @middy/http-cors aws-lambda supertokens-node typescript
4. Creating /src/index.ts with following content:

import supertokens from "supertokens-node";
import { middleware } from "supertokens-node/framework/awsLambda";
import middy from "@middy/core";
import cors from "@middy/http-cors";
import EmailPassword from "supertokens-node/recipe/emailpassword";
import Session from "supertokens-node/recipe/session";
import { TypeInput } from "supertokens-node/lib/build/types";

const getBackendConfig = (): TypeInput => ({
    appInfo: {
        apiDomain: "https://auth.example.com",
        appName: "example.com",
        websiteDomain: "https://example.com",
        apiBasePath: "/"
    },
    framework: "awsLambda",
    isInServerlessEnv: true,
    recipeList: [EmailPassword.init(), Session.init({
        jwt: {
            enable: true,
            issuer: "https://auth.example.com"
        },
    })],
    supertokens: {
        connectionURI: `http://${process.env.SUPERTOKENS_NAME}:3567`,
        apiKey: process.env.API_KEY
    },
});

supertokens.init(getBackendConfig());

const handler = middy(middleware())
    .use(
        cors({
            credentials: true,
            headers: ["Content-Type", ...supertokens.getAllCORSHeaders()].join(", "),
            methods: "OPTIONS,POST,GET,PUT,DELETE",
            origin: getBackendConfig().appInfo.websiteDomain,
        }),
    )
    .onError((request: { error: any }) => {
        throw request.error;
    });

export default handler;

5. Running npx tsc ./src/index.ts --outDir ./dist --esModuleInterop --module commonjs --target es2017 && cp package.json ./dist

5.2. Setting up infrastructure

We are going to add a new security that we are going to assign to our api. We update our ingress rules for the load balancer security group to allow any traffic from the api’s security group.

When creating our api end point we point to the /dist folder that we have created in the previous step. We pass the api key and the load balancer’s DNS name as environment variables.

  StSecurityGroupApi:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for Supertokens api
      VpcId: !Ref StVpc

  StSecurityGroupIngress3:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      IpProtocol: "-1"
      SourceSecurityGroupId: !GetAtt ["StSecurityGroupApi", "GroupId"]
      GroupId: !GetAtt ["StSecurityGroupLoadBalancer", "GroupId"]

  StApiGateway:
    Type: AWS::Serverless::Api
    Properties:
      StageName: prod

  StSupertokensApi:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: dist
      Handler: index.default
      Runtime: nodejs16.x
      Environment:
        Variables:
          SUPERTOKENS_NAME: !GetAtt ["StLoadBalancer", "DNSName"]
          API_KEY: !Ref StApiKey
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: any
            RestApiId: !Ref StApiGateway
      VpcConfig:
        SecurityGroupIds:
          - !GetAtt ["StSecurityGroupApi", "GroupId"]
        SubnetIds:
          - !Ref StPublicSubnet1
      Timeout: 10

6. Setting up Cloudfront

We are going to create a Cloudfront distribution that uses our API gateway as origin. There are a few things to note:

• We are going to use auth.example.com as an alternate domain name. We pass this information as Aliases
• If we want to use an alternate domain name, we also need to pass a certificate that includes that domain name. We have that certificate in our account and pass it as a parameter to the distribution
• We have to update the default behavior to allow for all http verbs. Otherwise users cannot sign up to Supertokens using POST requests.
• For our default behavior we use our own caching policy. We do not want to cache anything at all, but we want to enable cookies, because we want Cloudfront to forward the Set-Cookie headers.
• We add another cache behavior that uses the CachingOptimized for the jwt/jwks.json end point. That information is something that we want to be cached and distributed to the edge locations

Parameters:

  # ...

  CertificateArn:
    Type: String
    Description: Arn of certificate for domain that is connected to Cloudfront

Resources:

  # ...

  StCloudfront:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Aliases:
          - "auth.example.com"
        Comment: Supertokens
        CacheBehaviors:
          - AllowedMethods:
              - "GET"
              - "HEAD"
              - "OPTIONS"
            CachedMethods:
              - "HEAD"
              - "GET"
            CachePolicyId: "658327ea-f89d-4fab-a63d-7e88639e58f6"
            Compress: true
            PathPattern: "jwt/jwks.json"
            SmoothStreaming: false
            TargetOriginId: !Sub "${StApiGateway}.execute-api.${AWS::Region}.amazonaws.com"
            ViewerProtocolPolicy: "allow-all"
        DefaultCacheBehavior:
          AllowedMethods:
            - "HEAD"
            - "DELETE"
            - "POST"
            - "GET"
            - "OPTIONS"
            - "PUT"
            - "PATCH"
          CachedMethods:
            - "HEAD"
            - "GET"
          CachePolicyId: !Ref StCachePolicy
          Compress: true
          SmoothStreaming: false
          TargetOriginId: !Sub "${StApiGateway}.execute-api.${AWS::Region}.amazonaws.com"
          ViewerProtocolPolicy: "allow-all"
        Enabled: true
        HttpVersion: http2
        IPV6Enabled: true
        PriceClass: PriceClass_All
        Staging: false
        Origins:
          - DomainName: !Sub "${StApiGateway}.execute-api.${AWS::Region}.amazonaws.com"
            Id: !Sub "${StApiGateway}.execute-api.${AWS::Region}.amazonaws.com"
            OriginPath: "/prod"
            CustomOriginConfig:
              HTTPPort: 80
              HTTPSPort: 443
              OriginProtocolPolicy: "https-only"
              OriginSSLProtocols:
                - "TLSv1.2"
              OriginReadTimeout: 30
              OriginKeepaliveTimeout: 5
        ViewerCertificate:
          AcmCertificateArn: !Ref CertificateArn
          SslSupportMethod: "sni-only"
          MinimumProtocolVersion: "TLSv1.2_2021"

  StCachePolicy:
    Type: AWS::CloudFront::CachePolicy
    Properties: 
      CachePolicyConfig:
        DefaultTTL: 1
        MinTTL: 1
        MaxTTL: 1
        Name: AllowCookies
        ParametersInCacheKeyAndForwardedToOrigin:
          CookiesConfig:
            CookieBehavior: all
          EnableAcceptEncodingGzip: false
          HeadersConfig:
            HeaderBehavior: none
          QueryStringsConfig:
            QueryStringBehavior: none

7. Setting up Route53

Setting up a record set for https://auth.example.com with Route 53 is straight forward. There are just a few things to keep in mind.

• We pass the hosted zone id of example.com as a parameter, because that is different in your situation.
• The Name has to have a trailing .
• When the AliasTarget is a Cloudfront distribution the HostedZoneId of the alias is always Z2FDTNDATAQYW2
• The cloudfront distribution needs to contain auth.example.com as alternate domain name. We set this up in the previous step

Parameters:

  # ...

  HostedZoneId:
    Type: String
    Description: Id of hosted zone for Route 53

Resources:

  # ...

  StRecordSet:
    Type: AWS::Route53::RecordSet
    Properties:
      Name: "auth.example.com."
      Type: A
      AliasTarget:
        DNSName: !GetAtt ["StCloudfront", "DomainName"]
        HostedZoneId: Z2FDTNDATAQYW2
        EvaluateTargetHealth: false
      HostedZoneId: !Ref HostedZoneId

8. Costs

There are a couple of components that will cost you money if you deploy the final template as is. These services are:

• Based on your configuration of size, type, AZ etc. the costs of our Postgres RDS instance may vary.
• Based on the size and amount of your EC2 instances your costs will increase as well.
• The load balancer in front of the Supertokens service costs money as well
• If you decide to not deploy your EC2 instances in your public subnet, but use VPC endpoints for the ECS service instead, you are going to pay for VPC endpoints as well.

BUT if you already have all of those resources in your AWS account you can reuse them for this project. You will need to update the template yourself, but you will not pay any additional costs for using Supertokens.

Other services used in this template like Cloudfront or API Gateway are also not free, but they will only start making a difference on your monthly bill once your application has thousands of daily active users. And if you have thousands of daily active users this whole costs section of this tutorial is irrelevant, because you have a serious business going on :)

9. Final thoughts & template

When you followed all steps correctly or if you use the final template at the bottom of this page you have your Supertokens server now running on https://auth.example.com.

When considering to what AWS region you want to deploy your template to you can consider where most of your users come from and deploy to that region. But very likely your users come from all over the world or you do not know where they will be coming from. The good news is that it does not really matter to what region you are going to deploy.

Depending on where your template is deployed and where your user is coming from signing up or logging in with like POST https://auth.example.com/signin might take a little longer. But signing up or logging in are actions that are relatively rare compared to authenticating requests. You might even argue that you would like those actions to take a little bit longer to prevent brute force attacks.

We have set up the template for JWT authentication. For that you will only need GET https://auth.example.com/jwt/jwks.json and we set up Cloudfront up to cache that end point at its edge locations. So no matter where your users come from your authorizer function will only ever need ~20ms to authenticate JWT tokens.

Feel free to copy & paste & change the following template to your needs:

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Supertokens 

Parameters:
  StVpcAZs:
    Type: CommaDelimitedList
    Description: Availability zones to host private subnets in
    Default: 'us-east-1a, us-east-1b'

  StPostgresPassword:
    Type: "AWS::SSM::Parameter::Value<String>"
    Description: Parameter store key that holds the password to the RDS Postgres instance
    Default: "suptertokens-postgres-password"
    NoEcho: true

  StApiKey:
    Type: "AWS::SSM::Parameter::Value<String>"
    Description: Parameter store key that holds the api key to the Supertokens service
    Default: "supertokens-api-key"
    NoEcho: true

  HostedZoneId:
    Type: String
    Description: Id of hosted zone for Route 53

  CertificateArn:
    Type: String
    Description: Arn of certificate for domain that is connected to Cloudfront

Mappings:
  AWSRegionToAMI:
    us-east-1:
      AMIID: ami-05e7fa5a3b6085a75
    eu-central-1:
      AMIID: ami-0348a4a91adc75319

Resources:

  StVpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: "10.1.0.0/16"

  StInternetGateway:
    Type: AWS::EC2::InternetGateway

  StAttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref StVpc
      InternetGatewayId: !Ref StInternetGateway

  StRouteToIGW:
    Type: AWS::EC2::Route
    DependsOn:
      - StAttachGateway
    Properties:
      RouteTableId: !Ref StPublicRouteTable
      DestinationCidrBlock: "0.0.0.0/0"
      GatewayId: !Ref StInternetGateway

  StPublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref StVpc

  StPrivateRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref StVpc

  StPublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: "10.1.0.0/24"
      VpcId: !Ref StVpc
      AvailabilityZone: !Select [ 0, !Ref StVpcAZs ]

  StPrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: "10.1.1.0/24"
      VpcId: !Ref StVpc
      AvailabilityZone: !Select [ 0, !Ref StVpcAZs ]

  StPrivateSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: "10.1.2.0/24"
      VpcId: !Ref StVpc
      AvailabilityZone: !Select [ 1, !Ref StVpcAZs ]

  StAssignRouteTable1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref StPublicRouteTable
      SubnetId: !Ref StPublicSubnet1

  StAssignRouteTable2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref StPrivateSubnet1
      RouteTableId: !Ref StPrivateRouteTable

  StAssignRouteTable3:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref StPrivateSubnet2
      RouteTableId: !Ref StPrivateRouteTable

  StDBSubnetGroup:
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupName: st-db-subnet-group
      DBSubnetGroupDescription: This is our subnet group
      SubnetIds:
        - !Ref StPrivateSubnet1
        - !Ref StPrivateSubnet2

  StSecurityGroupService:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for our Supertokens service
      VpcId: !Ref StVpc
  
  StSecurityGroupDatabase:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for our database instance
      VpcId: !Ref StVpc

  StSecurityGroupIngress1:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      IpProtocol: "-1"
      SourceSecurityGroupId: !GetAtt ["StSecurityGroupService", "GroupId"]
      GroupId: !GetAtt ["StSecurityGroupDatabase", "GroupId"]

  StDatabase:
    Type: AWS::RDS::DBInstance
    DependsOn:
      - StDBSubnetGroup
    Properties:
      DBName: supertokens
      DBSubnetGroupName: st-db-subnet-group
      VPCSecurityGroups:
        - !Ref StSecurityGroupDatabase
      DBInstanceClass: "db.t3.micro"
      AllocatedStorage: 20
      MultiAZ: false
      Engine: postgres
      EngineVersion: "13.7"
      StorageType: gp2
      DeletionProtection: false
      MasterUsername: postgres
      MasterUserPassword: !Ref StPostgresPassword
  
  StSecurityGroupLoadBalancer:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for our load balancer
      VpcId: !Ref StVpc

  StLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      IpAddressType: ipv4
      Scheme: internal
      SecurityGroups:
        - Ref: StSecurityGroupLoadBalancer
      Subnets:
        - Ref: StPrivateSubnet1
        - Ref: StPrivateSubnet2
      Type: application

  StTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      VpcId: !Ref StVpc
      Protocol: HTTP
      Port: 3567
      HealthCheckEnabled: true
      HealthCheckIntervalSeconds: 60
      HealthCheckPath: /hello
      HealthCheckPort: traffic-port
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 30
      UnhealthyThresholdCount: 5
      IpAddressType: ipv4
      Matcher:
        HttpCode: "200"
      ProtocolVersion: HTTP1
      TargetType: ip

  StListener:
    Type: 'AWS::ElasticLoadBalancingV2::Listener'
    Properties:
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref StTargetGroup
      LoadBalancerArn: !Ref StLoadBalancer
      Port: '3567'
      Protocol: HTTP

  StSecurityGroupIngress2:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      IpProtocol: "-1"
      SourceSecurityGroupId: !GetAtt ["StSecurityGroupLoadBalancer", "GroupId"]
      GroupId: !GetAtt ["StSecurityGroupService", "GroupId"]

  StCluster:
    Type: AWS::ECS::Cluster

  StEcsTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service: ecs-tasks.amazonaws.com
          Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

  StTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      ContainerDefinitions:
        - Essential: true
          Image: registry.supertokens.io/supertokens/supertokens-postgresql
          Environment:
            - Name: POSTGRESQL_USER
              Value: postgres
            - Name: POSTGRESQL_PASSWORD
              Value: !Ref StPostgresPassword
            - Name: POSTGRESQL_HOST
              Value: !GetAtt ["StDatabase", "Endpoint.Address"]
            - Name: POSTGRESQL_PORT
              Value: !GetAtt ["StDatabase", "Endpoint.Port"]
            - Name: API_KEYS
              Value: !Ref StApiKey
            - Name: DISABLE_TELEMETRY
              Value: 'true'
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: "/ecs/supertokens-core"
              awslogs-region: !Ref "AWS::Region"
              awslogs-stream-prefix: ecs
          Name: supertokens-core
          PortMappings:
            - HostPort: 3567
              Protocol: tcp
              ContainerPort: 3567
      Cpu: '256'
      Memory: '512'
      RuntimePlatform:
        OperatingSystemFamily: LINUX
      NetworkMode: awsvpc
      ExecutionRoleArn: !GetAtt ["StEcsTaskExecutionRole", "Arn"]

  StServiceDefinition:
    Type: AWS::ECS::Service
    DependsOn:
      - StListener
    Properties:
      Cluster: !Ref StCluster
      DesiredCount: 1
      LaunchType: EC2
      NetworkConfiguration:
        AwsvpcConfiguration:
          Subnets:
          - Ref: StPrivateSubnet1
          - Ref: StPrivateSubnet2
          SecurityGroups:
          - Ref: StSecurityGroupService
      TaskDefinition:
        Ref: StTaskDefinition
      LoadBalancers:
        - TargetGroupArn: !Ref StTargetGroup
          ContainerPort: 3567
          ContainerName: supertokens-core

  StEc2Role:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: ecs-service
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - 'ec2:DescribeTags'
                  - 'ecs:CreateCluster'
                  - 'ecs:DeregisterContainerInstance'
                  - 'ecs:DiscoverPollEndpoint'
                  - 'ecs:Poll'
                  - 'ecs:RegisterContainerInstance'
                  - 'ecs:StartTelemetrySession'
                  - 'ecs:UpdateContainerInstancesState'
                  - 'ecs:Submit*'
                  - 'ecr:GetAuthorizationToken'
                  - 'ecr:BatchCheckLayerAvailability'
                  - 'ecr:GetDownloadUrlForLayer'
                  - 'ecr:BatchGetImage'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvent'
                Resource: '*'

  StEc2InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref StEc2Role

  StLaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties: 
      LaunchTemplateData:
        ImageId: !FindInMap 
        - AWSRegionToAMI
        - !Ref 'AWS::Region'
        - AMIID
        InstanceType: t2.micro
        IamInstanceProfile:
          Arn: !GetAtt StEc2InstanceProfile.Arn
        NetworkInterfaces:
          - DeviceIndex: 0 
            AssociatePublicIpAddress: true
            Groups:
              - !GetAtt StSecurityGroupLoadBalancer.GroupId
        UserData:
          Fn::Base64:
            !Sub |
              #!/bin/bash
              echo ECS_CLUSTER=${StCluster} >> /etc/ecs/ecs.config

  StAsg:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      LaunchTemplate:
        LaunchTemplateId: !Ref StLaunchTemplate
        Version: !GetAtt StLaunchTemplate.LatestVersionNumber
      MaxSize: '1'
      MinSize: '1'
      DesiredCapacity: '1'
      VPCZoneIdentifier:
        - Ref: StPublicSubnet1

  StSecurityGroupApi:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for Supertokens api
      VpcId: !Ref StVpc

  StSecurityGroupIngress3:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      IpProtocol: "-1"
      SourceSecurityGroupId: !GetAtt ["StSecurityGroupApi", "GroupId"]
      GroupId: !GetAtt ["StSecurityGroupLoadBalancer", "GroupId"]

  StApiGateway:
    Type: AWS::Serverless::Api
    Properties:
      StageName: prod
      Cors: "'*'"

  StSupertokensApi:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: dist
      Handler: index.default
      Runtime: nodejs16.x
      Environment:
        Variables:
          SUPERTOKENS_NAME: !GetAtt ["StLoadBalancer", "DNSName"]
          API_KEY: !Ref StApiKey
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: any
            RestApiId: !Ref StApiGateway
      VpcConfig:
        SecurityGroupIds:
          - !GetAtt ["StSecurityGroupApi", "GroupId"]
        SubnetIds:
          - !Ref StPublicSubnet1
      Timeout: 10

  StCloudfront:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Aliases:
          - "auth.example.com"
        Comment: Supertokens
        CacheBehaviors:
          - AllowedMethods:
              - "GET"
              - "HEAD"
              - "OPTIONS"
            CachedMethods:
              - "HEAD"
              - "GET"
            CachePolicyId: "658327ea-f89d-4fab-a63d-7e88639e58f6"
            Compress: true
            PathPattern: "jwt/jwks.json"
            SmoothStreaming: false
            TargetOriginId: !Sub "${StApiGateway}.execute-api.${AWS::Region}.amazonaws.com"
            ViewerProtocolPolicy: "allow-all"
        DefaultCacheBehavior:
          AllowedMethods:
            - "HEAD"
            - "DELETE"
            - "POST"
            - "GET"
            - "OPTIONS"
            - "PUT"
            - "PATCH"
          CachedMethods:
            - "HEAD"
            - "GET"
          CachePolicyId: !Ref StCachePolicy
          Compress: true
          SmoothStreaming: false
          TargetOriginId: !Sub "${StApiGateway}.execute-api.${AWS::Region}.amazonaws.com"
          ViewerProtocolPolicy: "allow-all"
        Enabled: true
        HttpVersion: http2
        IPV6Enabled: true
        PriceClass: PriceClass_All
        Staging: false
        Origins:
          - DomainName: !Sub "${StApiGateway}.execute-api.${AWS::Region}.amazonaws.com"
            Id: !Sub "${StApiGateway}.execute-api.${AWS::Region}.amazonaws.com"
            OriginPath: "/prod"
            CustomOriginConfig:
              HTTPPort: 80
              HTTPSPort: 443
              OriginProtocolPolicy: "https-only"
              OriginSSLProtocols:
                - "TLSv1.2"
              OriginReadTimeout: 30
              OriginKeepaliveTimeout: 5
        ViewerCertificate:
          AcmCertificateArn: !Ref CertificateArn
          SslSupportMethod: "sni-only"
          MinimumProtocolVersion: "TLSv1.2_2021"

  StCachePolicy:
    Type: AWS::CloudFront::CachePolicy
    Properties: 
      CachePolicyConfig:
        DefaultTTL: 1
        MinTTL: 1
        MaxTTL: 1
        Name: AllowCookies
        ParametersInCacheKeyAndForwardedToOrigin:
          CookiesConfig:
            CookieBehavior: all
          EnableAcceptEncodingGzip: false
          HeadersConfig:
            HeaderBehavior: none
          QueryStringsConfig:
            QueryStringBehavior: none

  StRecordSet:
    Type: AWS::Route53::RecordSet
    Properties:
      Name: "auth.example.com."
      Type: A
      AliasTarget:
        DNSName: !GetAtt ["StCloudfront", "DomainName"]
        HostedZoneId: Z2FDTNDATAQYW2
        EvaluateTargetHealth: false
      HostedZoneId: !Ref HostedZoneId