Data Management

Introduction#

This section contains key information on how we manage data within Rebuild for Email.

Keywords:

DTU: Database Throughput Unit. Describes the performance capability of a database. It is a grouping of CPU, I/O, and memory. Increase the DTU increases performance.
Elastic pool: This is a technology within Azure SQL that allows the generation of a single pool of DTU resources that multiple databases can use. This is designed to reduce costs by being more efficient in resource usage.

Azure SQL#

We use Azure SQL to store relational data within the FileTrust email. The following databases are used within the product:

Policy: This stores the policy configuration of a tenant.
Tenman: This stores the tenant's details including who the admins are and who the tenant is.
Reports: This stores the transactional reports of the tenants. Transactional reports are defined as emails that flow through the system.
CAS: This stores the user's login details and manages sign up and login authentication.

Redundancy#

All live customer production SQL databases have automatic Geo-replication set to create a geo-redundancy.

UKSouth Geo-Replicates to UKWest

USEast Geo-Replicates to USWest

Security#

The following security measure has been taken to protect our customer's data:

Each database is encrypted at rest with AES 256 symmetric encryption.
Each production database can only be accessed via a specific set of IPs.
Only accessible by SRE team and key database developers.

Resource management#

Each of the key databases except Reports has a single Azure SQL server with a single Azure SQL database.

Reports have a separate database per live tenant because transactional data ingest and egress can vary dramatically per tenant. Having a separate database can be costly

Each database current resource configuration is:

- Policy, Tenman, and CAS:
    - S0 10 DTUs
        - Geo-Replication

- Reports:
     - PoC Customers:
        - Single 200 DTU Database
                - No Geo-Replication
     - Live Customers:
        - Elasticpools of 200 eDTUs:
            - 10 databases per pool.
            - 0 - 20 eDTUs set per database.
        - Geo-Replication

Blob Storage#

We use Blob storage to store customers' transactional data such as emails, analysis reports, and protected files.

Redundancy#

All live customer production blob storages have automatic Geo-replication set to create a geo-redundancy.

UKSouth Geo-Replicates to UKWest

USEast Geo-Replicates to USWest

Security#

The following security measure has been taken to protect our customer's data:

Each blob storage is encrypted at rest with AES 256 symmetric encryption.
Only accessible by SRE team and key database developers.