Code integrations

Cloud Certainty Watchtower connects AWS Security Hub findings to the code that created the resources. It indexes the Terraform and CloudFormation in your GitHub repositories, shows each finding's resource "defined in infra/s3.tf:12", previews and opens fix pull requests for common controls, and brings code scanner results (built-in Trivy, SonarCloud and SonarQube, and SARIF uploads from any CI scanner) next to the cloud findings of the same control.

Integrations, repositories and ingest tokens are managed by administrators under Administration. This page explains how to set them up and how each part works.

#What Watchtower does on GitHub

Watchtower never merges, pushes to an existing branch, approves, or changes repository settings. Indexes keep resource names, types, tags and line numbers, not the files themselves.

#GitHub and GitHub Enterprise Server

Add an integration in Administration › Integrations with:

• Base URL: https://api.github.com for GitHub.com, or https://<host>/api/v3 for GitHub Enterprise Server.
• Authentication: a fine-grained personal access token, or a GitHub App.

Then test it: Watchtower reports the identity the token acts as, or GitHub's error. The token or the app's private key is stored in AWS Secrets Manager in your account, encrypted with the stack's key, and is never shown again, logged or written to the audit log.

#Permissions to grant

Give only what the features you use need, on the repositories Watchtower should index:

So a read-only setup is Contents: Read and Metadata: Read. Add Contents: Read and write and Pull requests: Read and write only if you want fix pull requests, and Issues: Read and write only for GitHub issues. No organisation or account permissions are needed.

Protect your default branches so that Watchtower's pull requests need review, like anyone else's.

#Fine-grained personal access token

1. In GitHub, open Settings › Developer settings › Personal access tokens › Fine-grained tokens and generate a new token. Use a dedicated machine user if you have one: pull requests and issues are created in the token owner's name.
2. Resource owner: your organisation (it may need to approve the token). Repository access: Only select repositories, with the repositories Watchtower should index.
3. Repository permissions: as in the table above.
4. Choose an expiry, generate the token and paste it into the integration. Before it expires, create a new one and paste it into the same integration.

A classic token with the repo scope also works, but it can reach every repository its owner can; prefer a fine-grained token or a GitHub App.

#GitHub App

A GitHub App acts as itself rather than as a person, and its installation tokens last at most an hour.

1. In your organisation, open Settings › Developer settings › GitHub Apps and create a new app. Give it a name such as Watchtower (acme); any homepage URL will do. Turn off the webhook: Watchtower doesn't use it.
2. Repository permissions: as in the table above.
3. Create the app, note its App ID, and generate a private key (a .pem file).
4. Install the app on your organisation with Only select repositories, and choose the repositories. The installation id is the number at the end of the installation's settings address (.../settings/installations/<id>).
5. In the integration, choose GitHub App and enter the app id, the installation id and the contents of the private key file.

Watchtower signs a short-lived token with the private key, exchanges it for an installation token, and renews it before it expires.

#GitHub Enterprise Server

Use the base URL https://<host>/api/v3 and a GitHub App or a token, as above. The server must be reachable over HTTPS from the stack's AWS Lambda functions, which run outside your VPC, and, for Trivy scans, from the scanner's subnets (see Networking). Its certificate must be trusted by public certificate authorities.

#Repositories

Add each repository in Administration › Repositories: up to 10 on Team, 100 on Business, unlimited on Enterprise.

#Indexing

When you add a repository, and then every 24 hours (a setting, 1 to 168 hours) or on demand, the sync worker:

1. reads the head commit of the branch and the repository's file tree;
2. picks the files that match the paths and not the excludes: *.tf files, and *.yaml, *.yml, *.json and *.template files that are CloudFormation templates (with AWSTemplateFormatVersion, or a Resources section whose resources have a Type of AWS::...);
3. parses Terraform with HashiCorp's HCL parser and CloudFormation YAML (short-form tags such as !Sub included) or JSON, and records each resource: its file and lines, type, name or logical id, Terraform address, literal physical names (such as bucket, name, function_name, BucketName, TableName) and literal tags.

Limits: 5,000 files and 50 MB per repository, 1 MB per file. A repository whose tree is too large for GitHub to list in one response is reported as an error: narrow the paths, or split the repository. Parse errors in single files are listed with the repository and don't stop the rest. AWS CDK apps aren't indexed in this release.

#How mapping works

For each finding, Watchtower looks for the code resource that defines the finding's resource, in every repository whose Accounts match the finding's account. Four rules are tried, best first, and the first that matches in a repository wins; each match has a confidence.

At most 10 matches are kept per finding. "Compatible" comes from a table of about 60 Security Hub resource types and their Terraform and CloudFormation types (for example AwsS3Bucket, aws_s3_bucket and AWS::S3::Bucket).

To get more high-confidence matches:

• Tag your resources with their source. Run yor in CI (yor tag -d .), or set watchtower:source on resources or in a module's tags. This is the most reliable rule.
• Add stack mappings for CloudFormation repositories, so a logical id such as LogsBucket in stack payments-prod maps to templates/payments.yaml and not to every template with a LogsBucket.
• Set Accounts on every repository, so code for production accounts is never matched to findings of sandbox accounts, and rule 4 has something to go on.
• Write names literally where you can. A name passed in from a variable defined in another file can't be resolved.

#Fix previews and pull requests

Business and above, with the fix permission on the findings. For some controls Watchtower can edit your code for you. A fix exists for a finding when its control has an automatic fix for the file's kind and the finding has a high or medium confidence match.

CloudFormation JSON templates and AWS CDK get no automatic fixes in this release; their guides still have the snippets.

Replacements. Turning on encryption for an existing RDS DB instance (RDS.3), EBS volume (EC2.3) or EFS file system (EFS.1) can't be done in place: Terraform plans to destroy and create the resource, and CloudFormation replaces it, with a new, empty resource. Read the plan or change set, and migrate the data first, before you merge such a pull request.

Formatting. Fixes edit the text of your file in place: they set or insert one attribute or block, or add a companion resource after the one they fix, and never reformat the rest of the file. The result is valid, but alignment may not match terraform fmt. If your pipeline checks formatting, run terraform fmt on the branch before merging.

#Preview

From an issue or from selected findings, preview the fix: Watchtower reads each file at the indexed commit and shows a unified diff per file, combining every finding mapped to the same file. Findings it can't fix are listed with the reason, for example no confident code match, the setting is already applied, or the resource uses a configuration the fix doesn't handle.

#Pull request

Opening the pull request:

1. recomputes the patches against the head of the branch, so the change applies to the current code;
2. creates a new branch watchtower/fix-<control>-<id> from the head commit and commits each file to it;
3. opens a pull request (optionally as a draft) against the branch, whose description lists the controls and the findings it addresses (titles, account ids and resource ids, never notes) and names the Watchtower user who opened it;
4. links it to the task you choose.

The pull request is created by the token's owner or the GitHub App. Watchtower never merges it: your review and pipeline decide. It follows its state (open, merged, closed) at every sync, and the findings close in Security Hub once the change is deployed. Every pull request is audited (fix.pr).

#Built-in Trivy scans

Business and above, with EnableScanner = Enabled on the hub stack (the default). Watchtower can scan the repositories it indexes with Trivy (opens in a new tab), without any CI changes.

• What runs. A one-off AWS Fargate task from the Watchtower image (2 vCPU, 4 GB of memory, 50 GB of ephemeral storage) downloads the branch as an archive with the integration's token, extracts it safely (no absolute paths, .., links or devices), and runs trivy config (Terraform, CloudFormation and other IaC misconfigurations), trivy fs --scanners secret (secrets in files) and, when Trivy can download its vulnerability database, trivy fs --scanners vuln (vulnerable dependencies in lock files). The task is discarded afterwards.
• When. For each repository with Trivy scans turned on, once every 24 hours by default (the scan interval setting, 6 to 720 hours), and whenever someone starts a scan from Code. One scan runs per repository at a time.
• Where results go. Into the stack's data bucket (scans/<repository>/), replacing the previous Trivy results of that repository, and onto Code, the issues of the controls they relate to, and the findings they correlate with. Each run is listed with its status and counts per severity for 90 days.
• Network. The task runs in the hub stack's SubnetIds and needs outbound HTTPS to GitHub (or your GitHub Enterprise Server) and to the Trivy database (mirror.gcr.io, ghcr.io); see Networking. Its logs are in /<NamePrefix>/scanner.

With EnableScanner = Disabled, scans answer "Scanning is turned off in this stack."; SonarCloud and SARIF uploads still work.

#SonarCloud and SonarQube

Business and above. Watchtower pulls the open vulnerabilities, bugs and security hotspots of your SonarCloud (SonarQube Cloud) or SonarQube projects, so they appear next to the repository's other scanner findings.

1. Create a token in SonarCloud or SonarQube (My Account › Security) for a user who can browse the projects.
2. Add an integration in Administration › Integrations with the kind SonarCloud, the base URL (https://sonarcloud.io, or your SonarQube server's URL, which must be reachable from AWS Lambda over HTTPS), the token and, for SonarCloud, your organisation key.
3. On each repository, choose that integration and enter the project key.

Every 24 hours by default (the scan interval setting), the sync worker reads the project's unresolved vulnerabilities and bugs and its security hotspots to review. Severities map to Watchtower's: for issues with Clean Code impacts, the highest impact (blocker to critical, high, medium, low, info to informational); otherwise the older severities (blocker to critical, critical to high, major to medium, minor to low, info to informational). Hotspots take their review priority (high, medium or low).

#SARIF uploads from CI

Business and above. Any scanner that writes SARIF 2.1.0 (Checkov, Trivy, tfsec, KICS, Semgrep, CodeQL and others) can send its results to Watchtower from your pipeline. Uploads are linked to a repository configured in Administration › Repositories.

1. Create an ingest token in Administration › Ingest tokens, scoped to the repository, and store it as a CI secret, for example WATCHTOWER_INGEST_TOKEN (admin guide).
2. Store the hub stack's IngestUrl output (https://<api id>.execute-api.<region>.amazonaws.com/ingest/sarif) as a CI variable, for example WATCHTOWER_INGEST_URL.
3. After the scan, POST the file:

Each upload replaces the previous results of the same tool for that repository. The tool is the SARIF file's tool name (Trivy, Checkov and so on), or the tool query parameter. Upload one file per tool, and only from the branch Watchtower indexes, so that results from pull request branches don't replace those of your main branch. A missing, expired or revoked token, or one not scoped to the repository, is refused. The upload route doesn't use the IP allowlist, is limited to a few requests per second, and every upload is audited (scan.ingest).

#GitHub Actions

name: iac-scan
on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read

jobs:
  scan:
    runs-on: ubuntu-latest
    env:
      WATCHTOWER_INGEST_URL: ${{ vars.WATCHTOWER_INGEST_URL }}      # the IngestUrl stack output
      WATCHTOWER_INGEST_TOKEN: ${{ secrets.WATCHTOWER_INGEST_TOKEN }}
      TRIVY_IMAGE: aquasec/trivy:0.56.2                             # pin the version you have tested
    steps:
      - uses: actions/checkout@v4

      - name: Trivy (Terraform and CloudFormation)
        run: |
          docker run --rm -v "$PWD:/src" -w /src "$TRIVY_IMAGE" \
            config --format sarif --output trivy.sarif .

      - name: Checkov
        run: |
          pipx install checkov
          checkov -d . --framework terraform cloudformation \
            --output sarif --output-file-path . --soft-fail
          # writes results_sarif.sarif

      - name: Upload to Watchtower
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        run: |
          upload() {
            gzip -c "$1" > "$1.gz"
            if [ "$(stat -c %s "$1.gz")" -gt 4000000 ]; then
              echo "$1.gz is larger than 4 MB"; exit 1
            fi
            curl --fail-with-body --silent --show-error -X POST \
              "$WATCHTOWER_INGEST_URL?repo=$GITHUB_REPOSITORY&commit=$GITHUB_SHA" \
              -H "Authorization: Bearer $WATCHTOWER_INGEST_TOKEN" \
              -H "Content-Type: application/json" \
              -H "Content-Encoding: gzip" \
              --data-binary "@$1.gz"
            echo
          }
          upload trivy.sarif
          upload results_sarif.sarif

On pull requests the scans still run and fail nothing (--soft-fail); only pushes to main upload. Remove --soft-fail, or set Trivy's --exit-code 1, if findings should fail the build.

#GitLab CI

Watchtower indexes GitHub repositories, so this suits a repository that GitLab CI builds from a GitHub repository (or a mirror of it). Set WATCHTOWER_REPO to the GitHub owner/name configured in Watchtower, and WATCHTOWER_INGEST_URL and WATCHTOWER_INGEST_TOKEN (masked) as CI/CD variables.

iac-scan:
  stage: test
  image: python:3.12-slim
  variables:
    TRIVY_VERSION: "0.56.2"   # pin the version you have tested
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - apt-get update -qq && apt-get install -y -qq curl gzip > /dev/null
    - curl -sSfL "https://raw.githubusercontent.com/aquasecurity/trivy/v${TRIVY_VERSION}/contrib/install.sh"
      | sh -s -- -b /usr/local/bin "v${TRIVY_VERSION}"
    - pip install --quiet checkov
    - trivy config --format sarif --output trivy.sarif .
    - checkov -d . --framework terraform cloudformation --output sarif --output-file-path . --soft-fail
    - |
      for f in trivy.sarif results_sarif.sarif; do
        gzip -c "$f" > "$f.gz"
        curl --fail-with-body --silent --show-error -X POST \
          "$WATCHTOWER_INGEST_URL?repo=$WATCHTOWER_REPO&commit=$CI_COMMIT_SHA" \
          -H "Authorization: Bearer $WATCHTOWER_INGEST_TOKEN" \
          -H "Content-Type: application/json" \
          -H "Content-Encoding: gzip" \
          --data-binary "@$f.gz"
        echo
      done

#How scanner findings meet cloud findings

The remediation catalogue maps scanner rules to Security Hub controls: for example Trivy AVD-AWS-0086 to AVD-AWS-0093 and Checkov CKV_AWS_53 to CKV_AWS_56 all check what S3.8 checks. A scanner finding and a cloud finding are linked when the scanner rule maps to the finding's control and the scanner finding lies inside the code resource the cloud finding is mapped to (same repository, file and lines), or, without a code mapping, when both name the same Terraform address.

So a cloud finding shows "also flagged by Trivy AVD-AWS-0086 in infra/s3.tf:12", a scanner finding shows the cloud findings it is already causing, and each issue of the fix list lists the scanner findings of its control in every repository: problems that will become cloud findings at the next deployment. Scanner findings only inform: they change nothing in AWS or GitHub.