How to build a newsletter system for your blog from scratch

Posted on:March 27, 2023 at 09:21 AM

Every blog needs a newsletter these days. There are a lot of systems out there that offer an end-to-end experience for successfully creating, sending and managing newsletter for blogs or any other application. We decided to write our own newsletter system, because we wanted to be in control of our data and not depend on third-party applications.

tl;dr

The guide is divided into six different parts:

  1. Enabling users to subscribe to newsletter
  2. Enabling users to unsubscribe from newsletter (10th of April)
  3. Creating development environment to write, preview and test newsletter emails (24th of April)
  4. Adding tracking to newsletter events and clicks (8th of May)
  5. Publishing newsletters to subscribers (22nd of May)
  6. Automating the creation and release of newsletters (5th of June)

"An abstract painting of a lot of envelopes with wings flying around" - DALL-E

"An abstract painting of a lot of envelopes with wings flying around" - DALL-E

Table of contents

Open Table of contents

1. Enabling users to subscribe to newsletter

The first step is to enable users to subscribe to the newsletter. We decided to use a simple form on the blog to subscribe to the newsletter. The form will send a POST request to an API endpoint that will store the email address in a database. We accomplish this in two parts:

  1. Creating a /emails endpoint that will store the email address in a database
  2. Creating a frontend component in the blog to subscribe to the newsletter

1.1 Creating a /emails endpoint that will store the email address in a database

We write a very simple endpoint with AWS Lambda and Api Gateway. The endpoint will receive a POST request with the email address in the body. The endpoint will then store the email address in a database. We use DynamoDB to store the email addresses. We use email as primary key and create a global secondary index on status. We will use the status to query all emails that are subscribed to the newsletter. Additionally we will enable Global Tables to store the data in multiple regions. If you want to learn more about how to set up a global table, you can read our blog post about it.

table.yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Newsletter table

Parameters:

  TableName:
    Type: String
    Description: Name of newsletter Dynamodb table

Resources:

  NewsletterTable:
    Type: AWS::DynamoDB::GlobalTable
    Properties: 
      AttributeDefinitions:
        - AttributeName: email
          AttributeType: "S"
        - AttributeName: status
          AttributeType: "S"
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: email
          KeyType: HASH
      GlobalSecondaryIndexes:
        - IndexName: status
          KeySchema:
            - AttributeName: status
              KeyType: HASH
          Projection:
            ProjectionType: ALL
      Replicas:
        - Region: us-east-1
        - Region: us-west-2
        - Region: eu-west-1
        - Region: eu-north-1
        - Region: eu-central-1
        - Region: ap-northeast-2
        - Region: ap-southeast-1
      StreamSpecification:
        StreamViewType: NEW_AND_OLD_IMAGES
      TableName: !Ref TableName

We write our end point in Typescript.

import { APIGatewayProxyHandler, APIGatewayProxyResult } from "aws-lambda";
import * as DynamoDb from "aws-sdk/clients/dynamodb";
import * as z from "zod";

const payloadSchema = z.object({
    email: z.string().email({ message: "Invalid email address" },)
});

const tableName = process.env.DYNAMODB_TABLE as string;

const parseBody = (body: string): unknown | undefined => {
    try {
        return JSON.parse(body);
    } catch {
        return undefined;
    }
}

const response = (result: APIGatewayProxyResult): APIGatewayProxyResult => ({
    ...result,
    headers: {
        ...(result.headers || {}),
        "Access-Control-Allow-Headers" : "*",
        "Access-Control-Allow-Origin": "*",
        "Access-Control-Allow-Methods": "OPTIONS,POST,GET"
    }
});

const handler: APIGatewayProxyHandler = async (event) => {
    const body = event.body;
    if (!body) {
        return response({ statusCode: 400, body: "No payload provided" });
    }
    const parsedBody = parseBody(body);
    if (!parsedBody) {
        return response({ statusCode: 400, body: "Invalid JSON payload" });
    }
    const validation = payloadSchema.safeParse(parsedBody);
    if (validation.success) {
        const { email } = validation.data;
        await new DynamoDb.DocumentClient().put({
            Item: {
                email,
                status: "subscribed"
            },
            TableName: tableName
        }).promise();
        return response({ statusCode: 201, body: "OK" });
    } else {
        return response({ statusCode: 400, body: validation.error.message });
    }
};

export default handler;

We are going to deploy that endpoint to all regions using the following template and command. The endpoint will be available at https://newsletter-blog.taskli.st/emails.

api.yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Newsletter api

Parameters:

  TableName:
    Type: String
    Description: Name of newsletter Dynamodb table

  DomainName:
    Type: String
    Description: The domain name that we want to deploy our api to

  RegionalCertificateArn:
    Type: String
    Description: Arn of the regional certificate

Resources:

  ApiGatewayApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: prod
        
  CustomDomain:
    Type: AWS::ApiGateway::DomainName
    Properties:
      DomainName: !Ref DomainName
      EndpointConfiguration:
        Types:
          - REGIONAL
      RegionalCertificateArn: !Ref RegionalCertificateArn
      SecurityPolicy: TLS_1_2

  CustomDomainMapping:
    Type: AWS::ApiGateway::BasePathMapping
    Properties:
      DomainName: !Ref DomainName
      RestApiId: !Ref ApiGatewayApi
      Stage: !Ref ApiGatewayApi.Stage
    DependsOn:
      - CustomDomain
  
  EmailsEndpoint:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: dist/emails
      Handler: index.default
      Runtime: nodejs16.x
      Environment:
        Variables:
          DYNAMODB_TABLE: !Ref TableName
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /emails
            Method: post
            RestApiId:
              Ref: ApiGatewayApi
      Timeout: 6
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref TableName

Outputs:

  RegionalDomainName:
    Value: !GetAtt CustomDomain.RegionalDomainName

  RegionalHostedZoneId:
    Value: !GetAtt CustomDomain.RegionalHostedZoneId

command

sam deploy --stack-name newsletter-blog-api --s3-bucket [INFRASTRUCTURE_BUCKET]-us-east-1 --s3-prefix newsletter-blog-api --region us-east-1 --capabilities CAPABILITY_IAM --parameter-overrides TableName=newsletter-blog-table RegionalCertificateArn=[REGIONAL_ARN] DomainName=newsletter-blog.taskli.st --no-fail-on-empty-changeset

We need to run this command for every region that we want to support. The outputs are relevant for our final template. Our final template is going to set up the Route 53 records for our domain.

routing.yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Newsletter routing

Resources:

  NewsletterRouteUsEast1:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: [HOSTED_ZONE_ID]
      Name: "newsletter-blog.taskli.st."
      Type: A 
      AliasTarget:
        DNSName: [DOMAIN_NAME_FROM_API_TEMPLATE]
        HostedZoneId: [HOSTED_ZONE_ID_FROM_API_TEMPLATE]
        EvaluateTargetHealth: true
      Region: us-east-1
      SetIdentifier: us-east-1

  NewsletterRouteUsWest2:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: [HOSTED_ZONE_ID]
      Name: "newsletter-blog.taskli.st."
      Type: A 
      AliasTarget:
        DNSName: [DOMAIN_NAME_FROM_API_TEMPLATE]
        HostedZoneId: [HOSTED_ZONE_ID_FROM_API_TEMPLATE]
        EvaluateTargetHealth: true
      Region: us-west-2
      SetIdentifier: us-west-2

  NewsletterRouteEuWest1:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: [HOSTED_ZONE_ID]
      Name: "newsletter-blog.taskli.st."
      Type: A 
      AliasTarget:
        DNSName: [DOMAIN_NAME_FROM_API_TEMPLATE]
        HostedZoneId: [HOSTED_ZONE_ID_FROM_API_TEMPLATE]
        EvaluateTargetHealth: true
      Region: eu-west-1
      SetIdentifier: eu-west-1

  NewsletterRouteEuNorth1:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: [HOSTED_ZONE_ID]
      Name: "newsletter-blog.taskli.st."
      Type: A 
      AliasTarget:
        DNSName: [DOMAIN_NAME_FROM_API_TEMPLATE]
        HostedZoneId: [HOSTED_ZONE_ID_FROM_API_TEMPLATE]
        EvaluateTargetHealth: true
      Region: eu-north-1
      SetIdentifier: eu-north-1

  NewsletterRouteEuCentral1:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: [HOSTED_ZONE_ID]
      Name: "newsletter-blog.taskli.st."
      Type: A 
      AliasTarget:
        DNSName: [DOMAIN_NAME_FROM_API_TEMPLATE]
        HostedZoneId: [HOSTED_ZONE_ID_FROM_API_TEMPLATE]
        EvaluateTargetHealth: true
      Region: eu-central-1
      SetIdentifier: eu-central-1

  NewsletterRouteApNortheast2:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: [HOSTED_ZONE_ID]
      Name: "newsletter-blog.taskli.st."
      Type: A 
      AliasTarget:
        DNSName: [DOMAIN_NAME_FROM_API_TEMPLATE]
        HostedZoneId: [HOSTED_ZONE_ID_FROM_API_TEMPLATE]
        EvaluateTargetHealth: true
      Region: ap-northeast-2
      SetIdentifier: ap-northeast-2

  NewsletterRouteApSoutheast1:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: [HOSTED_ZONE_ID]
      Name: "newsletter-blog.taskli.st."
      Type: A 
      AliasTarget:
        DNSName: [DOMAIN_NAME_FROM_API_TEMPLATE]
        HostedZoneId: [HOSTED_ZONE_ID_FROM_API_TEMPLATE]
        EvaluateTargetHealth: true
      Region: ap-southeast-1
      SetIdentifier: ap-southeast-1

1.2 Creating a frontend component in the blog to subscribe to the newsletter

We use Astro with React to create our blog. Our new component that uses our API is going to be a simple form that takes an email address and sends it to our API. We use inline styles and some basic error handling. It looks like this:

import { useState, useRef } from "react";

type NewsletterState =
    | {
          type: "idle";
      }
    | {
          type: "loading";
      }
    | {
          type: "success";
      }
    | {
          type: "error";
          message: string;
      };

const Newsletter: React.FC = () => {
    const [state, setState] = useState<NewsletterState>({ type: "idle" });
    const input = useRef<HTMLInputElement>();

    const submitEmail = async (): Promise<void> => {
        if (input.current?.value && state.type !== "loading") {
            setState({ type: "loading" });
            try {
                const response = await fetch("https://newsletter-blog.taskli.st/emails", {
                    body: JSON.stringify({ email: input.current.value }),
                    method: "post",
                });
                if (response.status === 201) {
                    setState({ type: "success" });
                } else {
                    setState({
                        message: "Email address is not valid. Please try again!",
                        type: "error",
                    });
                }
            } catch {
                setState({
                    message: "We had some connection problem. Please try to submit again!",
                    type: "error",
                });
            }
        }
    };

    return (
        <div className="flex gap-2 w-full flex-col">
            <div className="flex gap-2">
                <input
                    onKeyUp={(event) => {
                        if (event.key === "Enter") {
                            submitEmail();
                        }
                    }}
                    ref={(field) => {
                        if (field) {
                            input.current = field;
                        }
                    }}
                    style={{
                        border: "2px solid #000",
                        borderRadius: 4,
                        flex: "auto",
                        outline: "none",
                        padding: "10px 15px",
                    }}
                    placeholder="Enter your email address for our newsletter..."
                ></input>
                <button
                    style={{
                        backgroundColor: "#000",
                        borderRadius: "5px",
                        color: "#fff",
                        fontWeight: "bold",
                        opacity: state.type === "loading" ? 0.5 : 1,
                        padding: "0px 20px",
                    }}
                    onClick={submitEmail}
                    disabled={state.type === "loading"}
                >
                    Subscribe
                </button>
            </div>
            {state.type === "error" && (
                <div style={{ color: "#c00", padding: "0px 10px" }}>{state.message}</div>
            )}
            {state.type === "success" && (
                <div style={{ padding: "0px 10px" }}>
                    Email submitted successfully! Thank you for subscribing to our newsletter!
                </div>
            )}
        </div>
    );
};

export default Newsletter;