Installation guide

This guide is for the platform team installing Cloud Certainty Memora in their AWS account. The install is one CloudFormation stack (the hub). Every other account whose ElastiCache caches Memora should reach gets a small role stack (the spoke), and every VPC that holds caches gets a connector: the hub stack can create one in its own VPC, and a connector stack adds one to any other VPC, account or Region. With IAM Identity Center or another SAML IdP, the hub takes two steps: deploy the stack, configure the SAML application, then update the stack. For the short version, see Getting started.

#Before you start

• Region. Any Region with Amazon ElastiCache, AWS Lambda, Amazon API Gateway (HTTP APIs), Amazon DynamoDB, AWS Step Functions, Amplify Hosting, Amazon Cognito, AWS License Manager and Amazon ECS on Fargate. The templates are always loaded from the vendor's public bucket in us-east-1 (CloudFormation accepts a template URL from any Region), and all product code comes from the container image. The hub's Region doesn't limit which Regions you can work in: list them in LocalRegions and in each connection.
• Account. Deploy the hub in the account that subscribed on AWS Marketplace: the licence is delivered to that account through AWS License Manager. To run the hub in another account of your organisation, distribute the licence to it with a License Manager grant first.
• Permissions. You need permission to create the resources in the stack: IAM roles (some have fixed names), Lambda, Step Functions, ECS, EC2 security groups, Cognito, API Gateway, Amplify, DynamoDB, S3, SNS, EventBridge, KMS and CloudWatch Logs.
• Network. A VPC with at least one subnet with outbound HTTPS access, for the one-off installer task (see Networking). If your caches are in that VPC, choose the subnets for the hub's connector too. Nothing in your VPC accepts inbound connections.
• Identity. Decide how users will sign in: IAM Identity Center, another SAML 2.0 IdP (Entra ID, Okta and others), an OIDC IdP, or Cognito-managed users. SAML IdPs other than IAM Identity Center, and OIDC, need the Team plan or above.
• Your caches. Decide the access level per account (Read, Operate or Full, see What Memora may do), note the VPCs, subnets and security groups of your caches, and how each cache authenticates (IAM, an AUTH token or users with passwords, or none): that decides how Memora signs in to its data (Data access).

#1. Subscribe

1. Open Memora in AWS Marketplace and choose Continue to Subscribe. Choose the plan and the contract length (Team is the base plan; Business and Enterprise are add-ons to it) and accept the offer. AWS License Manager now holds your licence.
2. Choose Continue to Configuration, select the fulfilment option and the Region, then open the CloudFormation launch link from the usage instructions.

#2. Launch the hub stack

In the CloudFormation console, fill in the parameters (see the reference). Acknowledge "AWS CloudFormation might create IAM resources with custom names" and create the stack. Creation takes about 10 minutes.

The quick-create link, with the release version filled in:

https://console.aws.amazon.com/cloudformation/home#/stacks/quickcreate?templateURL=https://cloudcertainty-memora-us-east-1.s3.us-east-1.amazonaws.com/pro/<version>/main.yaml&stackName=memora

From the AWS CLI (--region is the Region you deploy to):

aws cloudformation create-stack --region <your-region> --stack-name memora \
  --template-url https://cloudcertainty-memora-us-east-1.s3.us-east-1.amazonaws.com/pro/<version>/main.yaml \
  --parameters file://params.json \
  --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

params.json lists the parameters as [{"ParameterKey": "VpcId", "ParameterValue": "vpc-..."}, ...].

#What happens during creation

The product's code (the Lambda functions, the connector and the web app) ships inside the subscription-protected container image, not as public downloads. Early in stack creation:

1. CloudFormation creates a private S3 bucket in your account (output ArtifactsBucketName; all public access blocked, encrypted, TLS only).
2. The ArtifactInstall custom resource starts one Fargate task from the product image in your InstallerSubnetIds. The task checks your AWS Marketplace licence in AWS License Manager, verifies every file of the code bundle against the SHA-256 checksums in the image, and copies it into the bucket under the release's version. It runs for about a minute, with a security group without inbound rules.
3. The Lambda functions (the API, the bulk job worker and, with ConnectorSubnetIds, the connector's read and write functions), Memora's own bulk jobs workflow (a Step Functions Standard state machine named <NamePrefix>-bulk) and the web app are then created from that bucket.

If the installer task fails, stack creation fails and rolls back. The error message gives the task's exit code and its log stream (log group /<NamePrefix>/installer); see Troubleshooting.

#3. Connect your identity provider

Until SamlMetadataUrl is set, the portal falls back to Cognito sign-in, which has no users. Nobody can sign in at that point, so the stack is safe in that state.

#4. First sign-in

Open the PortalUrl output and sign in as BootstrapAdminEmail. The first account that signs in with that (verified) email becomes the bootstrap administrator and stays bound to it (see the admin guide).

The local connection to the hub's own account exists from the start. It uses the stack's local role, which may do what LocalAccessLevel allows, in the Regions of LocalRegions. With ConnectorSubnetIds, the local connector exists from the start too, for the caches of VpcId in the stack's Region. Administrators see every cache. Other users see nothing until an administrator grants them access (admin guide).

#What Memora may do

The local role and every spoke role are the upper limit of what Memora can do in an account, for everyone, administrators included. Memora then narrows each request to the person's grants with a session policy.

With Read, nobody can change a cache or its data through Memora, whatever their grants: the role can't even invoke a connector's write function.

The roles have no IAM, CloudFormation, DynamoDB, S3 or Lambda change permissions. They reach cache data only by invoking connector functions named memora-connector-*, which hold no state, so Memora can't change its own stack.

#The connector

ElastiCache caches have no public endpoints, so Memora reaches their data through a connector: a pair of Lambda functions inside the caches' VPC, memora-connector-<name>-read and memora-connector-<name>-write, each with its own role and security group. They are the only part of Memora that connects to your caches, and they are invoked only through lambda:InvokeFunction by Memora's roles under /memora/.

• The read function runs read commands only and signs in to caches as the ElastiCache user memora-read (or with secrets tagged memora:access = read). The write function runs every command Memora allows and signs in as memora-write (or with secrets tagged write). A person without the write permission gets an AWS session that can invoke only the read function.
• They connect only to ElastiCache endpoints, on the address Memora builds from ElastiCache's own description of the cache.
• Their security group has no inbound rules, and outbound rules for TCP 6379 to 6380 (the cache ports) and 443 (Secrets Manager, only needed for caches that sign in with secrets).
• Your caches' security groups must allow inbound TCP 6379 and 6380 from the connector's security group. Set CacheSecurityGroupId and the stack adds that rule for one security group; for others, add it yourself (output ConnectorSecurityGroupId).
• IAM data access needs nothing else: the function signs a short-lived IAM token itself. Secrets need a route from the subnets to Secrets Manager: a NAT gateway, or an interface VPC endpoint for Secrets Manager.

The hub stack creates the connector local in VpcId (in the stack's Region) when you set ConnectorSubnetIds. For caches in any other VPC, account or Region, deploy a connector stack. Without any connector, Memora still shows your caches, their metrics, findings and backups, and operates them, but can't read or change their data.

#5. Deploy the spoke role in other accounts

spoke-role.yaml creates one role, /memora/<RoleName>, that only the hub's API role and its bulk job worker role (stack output SpokeTrustedPrincipal) can assume, only with your external ID, and only with a user's email as the source identity. IAM is global, so deploy it once per account; the connection then covers every Region you list. Its template URL is the stack output SpokeTemplateUrl:

https://cloudcertainty-memora-us-east-1.s3.us-east-1.amazonaws.com/pro/<version>/spoke-role.yaml

The role is the upper limit of what Memora can do in that account; grants narrow it per user or group but never widen it. The stack's outputs are RoleArn (enter it in the connection) and AccessLevel (choose the same level for the connection).

#A single account

cat > spoke-params.json <<'EOF'
[
  {"ParameterKey": "TrustedPrincipal", "ParameterValue": "<SpokeTrustedPrincipal output>"},
  {"ParameterKey": "ExternalId", "ParameterValue": "<a secret of at least 16 characters>"},
  {"ParameterKey": "AccessLevel", "ParameterValue": "Operate"}
]
EOF
aws cloudformation create-stack --region eu-west-1 --stack-name memora-spoke \
  --template-url https://cloudcertainty-memora-us-east-1.s3.us-east-1.amazonaws.com/pro/<version>/spoke-role.yaml \
  --parameters file://spoke-params.json --capabilities CAPABILITY_NAMED_IAM

#Every account of an AWS Organization (StackSets)

With service-managed StackSets, new accounts in the target OUs get the role automatically. Use one external ID for the whole StackSet, and enter that same external ID for every connection you create for those accounts.

# From the management account or a delegated StackSets administrator.
aws cloudformation create-stack-set --region us-east-1 --stack-set-name memora-spoke \
  --template-url https://cloudcertainty-memora-us-east-1.s3.us-east-1.amazonaws.com/pro/<version>/spoke-role.yaml \
  --parameters file://spoke-params.json \
  --capabilities CAPABILITY_NAMED_IAM \
  --permission-model SERVICE_MANAGED \
  --auto-deployment Enabled=true,RetainStacksOnAccountRemoval=false

# One Region is enough: IAM is global.
aws cloudformation create-stack-instances --region us-east-1 --stack-set-name memora-spoke \
  --deployment-targets OrganizationalUnitIds=ou-abcd-11111111 --regions eu-west-1

Service-managed StackSets don't deploy to the management account; use a normal stack there if Memora should reach it. To change AccessLevel later, update the StackSet, then the access level of the connections.

#Add the connection

In Administration › Connections › Add connection, enter an id (used in links, it can't be changed later), a name, the AWS account id, the access level, the spoke stack's RoleArn, the external ID and the Regions to show. Choose Save connection, then Test: Memora assumes the role and counts the caches in each Region. See the admin guide.

#6. Connectors in other VPCs, accounts and Regions

connector.yaml adds a connector to any VPC that holds caches, in the hub account or another one, in any Region. Deploy one per VPC. Its template URL is the hub's ConnectorTemplateUrl output:

https://cloudcertainty-memora-us-east-1.s3.us-east-1.amazonaws.com/pro/<version>/connector.yaml

The connector's code is not public: at create and update, a small custom resource in the connector stack copies <version>/connector.zip from the hub's private artifacts bucket into a bucket of the connector stack's own, so it works in any Region.

1. For another account, allow it to copy the code first. Update the hub stack and add the account id to ConnectorAccountIds. That lets the account read the connector's code (and only that file) from the hub's artifacts bucket. Connectors in the hub's own account need nothing here. The account also needs a connection.
2. Get the quick-create link. In Administration › Connectors, expand Deploy a connector, choose the connection and the Region of the caches, and choose Get the quick-create link. Open the quick-create page in the CloudFormation console while signed in to that account: the link fills in HubArtifactsBucket, HubRegion, Version and TrustedRoleArn, and the page also lists them to copy.
3. Fill in the rest (see the table below): a ConnectorName, the caches' VpcId and SubnetIds, and optionally CacheSecurityGroupId and CreateSecretsEndpoint. Acknowledge the IAM capabilities and create the stack (it creates named IAM roles).
4. Register it. Back in Administration › Connectors, choose Register connector (or Register the connector under the deploy steps) and enter an id, a name, the connection, the Region, the VPC id and the stack's ReadFunctionArn and WriteFunctionArn outputs. Choose Save connector, then Test: Memora invokes both functions with the connection's role.

Outputs: ReadFunctionArn, WriteFunctionArn, VpcId and SecurityGroupId.

Memora picks the connector for a cache automatically: the one registered for the cache's connection, Region and VPC. A cache's data access can also name a connector explicitly. When no connector can reach a cache, its data tabs say so and name the VPC to deploy one in.

#Parameter reference

#Networking

The installer task runs in your VPC for about a minute at install and at each upgrade. It needs outbound HTTPS (443) to:

• Amazon ECR in us-east-1, to pull the image from the AWS Marketplace registry (cross-Region, so this needs internet egress: a NAT gateway or public subnets);
• AWS License Manager in your Region, for the licence check;
• Amazon S3 in your Region, to write the code into the artifacts bucket (a gateway endpoint works);
• CloudWatch Logs in your Region, for its log stream.

Its security group has no inbound rules and one outbound rule, TCP 443.

The connector functions run in the subnets you choose, with no inbound rules (see The connector). With IAM data access they need no route out of the VPC at all; with secrets they need a route to Secrets Manager.

Everything else is outside your VPC: the API and the worker reach ElastiCache, CloudWatch, EC2, STS, Lambda (to invoke the connectors), AWS License Manager and the AWS Price List API over AWS's public endpoints.

#Encryption and KMS

Without KmsKeyArn, the DynamoDB tables, the log groups, the update SNS topic, the artifacts bucket and the bulk jobs workflow use AWS owned keys and SSE-S3.

With KmsKeyArn, your key encrypts all of them, except the connector's code in the artifacts bucket, which always uses SSE-S3 so that connector stacks in other accounts can copy it. The key must be a symmetric customer managed key in the stack's Region (give its ARN, not an alias), and its key policy must:

• let the account's IAM policies grant its use (the default key policy statement for the account root does this). The stack's roles use it only through DynamoDB, CloudWatch Logs, SNS, S3 and Step Functions;
• allow logs.<region>.amazonaws.com to use it for the stack's log groups, and delivery.logs.amazonaws.com for the bulk jobs workflow's log delivery;
• allow sns.amazonaws.com to use it for the update topic.

The identity that creates or updates the stack also needs kms:Decrypt on the key, because Lambda reads the function code from the encrypted artifacts bucket with the deploying identity's permissions.

The key protects Memora's own data. Your caches keep their own encryption. Secrets that hold cache credentials may use your own KMS keys: the connector roles may decrypt only through Secrets Manager, so the key policy must let the account's IAM policies grant its use.

#Stack outputs

#Updating

Each release is a new template; the hub has no version parameter. To upgrade, update the stack with the new release's template (Update stack › Replace current template) and keep every other parameter value, then update each connector stack's Version parameter. See the upgrading guide.

To change a setting such as LocalAccessLevel, LocalRegions, ConnectorSubnetIds, ConnectorAccountIds or SamlMetadataUrl, update the stack with the current template and change only that parameter.

#Deleting

1. Delete the connector stacks in your VPCs.

2. Delete the hub stack. The DynamoDB table (<NamePrefix>-table) is retained on purpose, with its settings, grants, data access settings, scripts, command records, analyses, bulk jobs and audit log. Everything else is deleted: the cache table (with the usage counters), the Cognito user pool (Cognito-only users are lost), the artifacts bucket, the hub's connector, the bulk jobs workflow and the log groups. A new stack with the same NamePrefix can't be created while the retained table exists.

   To delete the table when you no longer need it, turn off its deletion protection, then delete it. Your caches and their data are never touched by deleting Memora. The ElastiCache users memora-read and memora-write, and any secrets you created for Memora, stay until you delete them.

3. Delete the spoke stacks or the StackSets in the other accounts.

4. Delete the SAML or OIDC application in your identity provider.

#Troubleshooting

#Installer exit codes

When stack creation or an update fails at ArtifactInstall, the status reason gives the installer's exit code and log stream (log group /<NamePrefix>/installer).

#Other problems