10000 feat(export): async, multi-service export by kschelonka · Pull Request #1042 · Pocket/pocket-monorepo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(export): async, multi-service export #1042

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 22 commits into from
Mar 6, 2025
Merged

feat(export): async, multi-service export #1042

merged 22 commits into from
Mar 6, 2025

Conversation

kschelonka
Copy link
Contributor
@kschelonka kschelonka commented Feb 27, 2025 •
edited
Loading

Creates a distributed export service with state management. This changes our single-service list export to a distributed, asynchronous, multi-service export that includes collections and annotations. Collections are exported as json files (one per collection).

Partial request diagram
(Braze components omitted; shareable-lists is a separate work queue with analogous pattern and is omitted)
image

Changes:

  • Move shared code from account-data-deleter into aws-utils (consolidating also where it had already been copied in list-api)
  • Add export processes for shareable-lists and annotations data, along with relevant queues, etc.
  • Add a new event type - "export-part-complete" for service-specific work completion notifications; this is consumed by
  • the new state handler, which uses dynamodb to store state and checks whether or not an export is complete. this is also responsible for queuing up the initial messages in the self-propagating work queues

@kschelonka kschelonka requested a review from a team as a code owner February 27, 2025 18:38
@kschelonka kschelonka requested review from Gio2018 and removed request for a team February 27, 2025 18:38
@github-actions GitHub Actions
Copy link
github-actions bot commented Feb 27, 2025 •
edited
Loading

Plan Result (@infrastructure/transactional-emails-production)

CI link

Plan: 0 to add, 1 to change, 0 to destroy.
  • Update
    • aws_sns_topic_subscription.ListExportReadyEvents-sns-subscription
Change Result (Click me) 
  # aws_sns_topic_subscription.ListExportReadyEvents-sns-subscription will be updated in-place
  ~ resource "aws_sns_topic_subscription" "ListExportReadyEvents-sns-subscription" {
      + filter_policy                   = jsonencode(
            {
              + detail-type = [
                  + "list-export-ready",
                ]
            }
        )
      + filter_policy_scope             = "MessageBody"
        id                              = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListExportReadyEvents:d382f154-b61a-4219-99c2-93b7bd8f1096"
        # (14 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

⚠️ Errors

@github-actions GitHub Actions
Copy link
github-actions bot commented Feb 27, 2025 •
edited
Loading

Plan Result (@infrastructure/pocket-event-bridge-production)

CI link

Plan: 0 to add, 1 to change, 0 to destroy.
  • Update
    • aws_cloudwatch_event_rule.list-export-ready-event-topic_PocketEventBridge-Prod-ListExportReadyEvents-EventBridge-Rule_event-bridge-rule_62C0F581
Change Result (Click me) 
  # aws_cloudwatch_event_rule.list-export-ready-event-topic_PocketEventBridge-Prod-ListExportReadyEvents-EventBridge-Rule_event-bridge-rule_62C0F581 will be updated in-place
  ~ resource "aws_cloudwatch_event_rule" "list-export-ready-event-topic_PocketEventBridge-Prod-ListExportReadyEvents-EventBridge-Rule_event-bridge-rule_62C0F581" {
      ~ event_pattern       = jsonencode(
          ~ {
              ~ detail-type = [
                    "list-export-ready",
                  + "export-part-complete",
                ]
              ~ source      = [
                    "account-data-deleter",
                  + "shareable-lists",
                ]
            }
        )
        id                  = "PocketEventBridge-Prod-Shared-Event-Bus/PocketEventBridge-Prod-ListExportReadyEvents-Rule-Rule"
        name                = "PocketEventBridge-Prod-ListExportReadyEvents-Rule-Rule"
        tags                = {}
        # (10 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

⚠️ Errors

@github-actions GitHub Actions
Copy link
github-actions bot commented Feb 27, 2025 •
edited
Loading

Plan Result (@infrastructure/shareable-lists-api-production)

CI link

⚠️ Resource Deletion will happen ⚠️

This plan contains resource delete operation. Please check the plan result very carefully!

Plan: 7 to add, 1 to change, 1 to destroy.
  • Create
    • aws_sns_topic_subscription.list-events-sns-subscription_D086E37D
    • aws_sqs_queue.list-events-sns-subscription_sns-topic-dql_4D7AC050
    • aws_sqs_queue.sharelists-export-consumer-queue_redrive_sqs_queue_0CF22BB0
    • aws_sqs_queue.sharelists-export-consumer-queue_sqs_queue_402C0DF5
    • aws_sqs_queue_policy.list-events-sns-subscription_sns-dlq-policy_4905D91A
    • aws_sqs_queue_policy.list-events-sns-subscription_sns-sqs-policy_593FA96C
  • Update
    • aws_iam_policy.application_ecs_service_ecs-iam_ecs-task-role-policy_6FC89FB6
  • Replace
    • aws_ecs_task_definition.application_ecs_service_ecs-task_461CC9D4
Change Result (Click me) 
  # data.aws_iam_policy_document.application_ecs_service_ecs-iam_data-ecs-task-role-policy_090CC3AD will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "application_ecs_service_ecs-iam_data-ecs-task-role-policy_090CC3AD" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)
      + version       = "2012-10-17"

      + statement {
          + actions   = [
              + "xray:GetSamplingRules",
              + "xray:GetSamplingStatisticSummaries",
              + "xray:GetSamplingTargets",
              + "xray:PutTelemetryRecords",
              + "xray:PutTraceSegments",
            ]
          + effect    = "Allow"
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "sqs:DeleteMessage",
              + "sqs:ReceiveMessage",
              + "sqs:SendMessage",
              + "sqs:SendMessageBatch",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
            ]
        }
      + statement {
          + actions   = [
              + "events:PutEvents",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:events:us-east-1:996905175585:event-bus/PocketEventBridge-Prod-Shared-Event-Bus",
            ]
        }
      + statement {
          + actions   = [
              + "s3:ListBucket",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::com.getpocket-prod.list-exports",
            ]
        }
      + statement {
          + actions   = [
              + "s3:*Object",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::com.getpocket-prod.list-exports/*",
            ]
        }
    }

  # data.aws_iam_policy_document.list-events-sns-subscription_sns-dlq-policy-document_3BAC76F9 will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "list-events-sns-subscription_sns-dlq-policy-document_3BAC76F9" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions   = [
              + "sqs:SendMessage",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
            ]

          + condition {
              + test     = "ArnEquals"
              + values   = [
                  + "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents",
                ]
              + variable = "aws:SourceArn"
            }

          + principals {
              + identifiers = [
                  + "sns.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
    }

  # data.aws_iam_policy_document.list-events-sns-subscription_sns-sqs-policy-document_AFC69F1A will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "list-events-sns-subscription_sns-sqs-policy-document_AFC69F1A" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions   = [
              + "sqs:SendMessage",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
            ]

          + condition {
              + test     = "ArnEquals"
              + values   = [
                  + "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents",
                ]
              + variable = "aws:SourceArn"
            }

          + principals {
              + identifiers = [
                  + "sns.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
    }

  # aws_ecs_task_definition.application_ecs_service_ecs-task_461CC9D4 must be replaced
-/+ resource "aws_ecs_task_definition" "application_ecs_service_ecs-task_461CC9D4" {
      ~ arn                      = "arn:aws:ecs:us-east-1:996905175585:task-definition/ShareableListsApi-Prod:434" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:us-east-1:996905175585:task-definition/ShareableListsApi-Prod" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [
              ~ {
                  ~ environment            = [
                        # (1 unchanged element hidden)
                        {
                            name  = "EVENT_BUS_NAME"
                            value = "PocketEventBridge-Prod-Shared-Event-Bus"
                        },
                      + {
                          + name  = "EXPORT_BUCKET"
                          + value = "com.getpocket-prod.list-exports"
                        },
                      + {
                          + name  = "EXPORT_QUEUE_URL"
                          + value = "https://sqs.us-east-1.amazonaws.com/996905175585/ShareableListsApi-Prod-SharedList-Export"
                        },
                        {
                            name  = "NODE_ENV"
                            value = "production"
                        },
                        # (4 unchanged elements hidden)
                    ]
                    name                   = "app"
                    # (10 unchanged attributes hidden)
                },
            ] # forces replacement
        )
      ~ id                       = "ShareableListsApi-Prod" -> (known after apply)
      ~ revision                 = 434 -> (known after apply)
        tags                     = {
            "app_code"       = "pocket"
            "component_code" = "pocket-shareablelistsapi"
            "costCenter"     = "Pocket"
            "env_code"       = "prod"
            "environment"    = "Prod"
            "owner"          = "Pocket"
            "service"        = "ShareableListsApi"
        }
        # (12 unchanged attributes hidden)
    }

  # aws_iam_policy.application_ecs_service_ecs-iam_ecs-task-role-policy_6FC89FB6 will be updated in-place
  ~ resource "aws_iam_policy" "application_ecs_service_ecs-iam_ecs-task-role-policy_6FC89FB6" {
        id               = "arn:aws:iam::996905175585:policy/ShareableListsApi-Prod-TaskRolePolicy"
        name             = "ShareableListsApi-Prod-TaskRolePolicy"
      ~ policy           = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "xray:PutTraceSegments",
                          - "xray:PutTelemetryRecords",
                          - "xray:GetSamplingTargets",
                          - "xray:GetSamplingStatisticSummaries",
                          - "xray:GetSamplingRules",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = ""
                    },
                  - {
                      - Action   = "events:PutEvents"
                      - Effect   = "Allow"
                      - Resource = "arn:aws:events:us-east-1:996905175585:event-bus/PocketEventBridge-Prod-Shared-Event-Bus"
                      - Sid      = ""
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags             = {
            "app_code"       = "pocket"
            "component_code" = "pocket-shareablelistsapi"
            "costCenter"     = "Pocket"
            "env_code"       = "prod"
            "environment"    = "Prod"
            "owner"          = "Pocket"
            "service"        = "ShareableListsApi"
        }
        # (7 unchanged attributes hidden)
    }

  # aws_sns_topic_subscription.list-events-sns-subscription_D086E37D will be created
  + resource "aws_sns_topic_subscription" "list-events-sns-subscription_D086E37D" {
      + arn                             = (known after apply)
      + confirmation_timeout_in_minutes = 1
      + confirmation_was_authenticated  = (known after apply)
      + endpoint                        = (known after apply)
      + endpoint_auto_confirms          = false
      + filter_policy                   = jsonencode(
            {
              + detail-type = [
                  + "list-export-requested",
                ]
            }
        )
      + filter_policy_scope             = "MessageBody"
      + id                              = (known after apply)
      + owner_id                        = (known after apply)
      + pending_confirmation            = (known after apply)
      + protocol                        = "sqs"
      + raw_message_delivery            = false
      + redrive_policy                  = (known after apply)
      + topic_arn                       = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents"
    }

  # aws_sqs_queue.list-events-sns-subscription_sns-topic-dql_4D7AC050 will be created
  + resource "aws_sqs_queue" "list-events-sns-subscription_sns-topic-dql_4D7AC050" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = (known after apply)
      + max_message_size                  = 262144
      + message_retention_seconds         = 345600
      + name                              = "ShareableListsApi-Prod-SharedList-Export-SNS-SNS-Topic-DLQ"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + sqs_managed_sse_enabled           = (known after apply)
      + tags                              = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-shareablelistsapi"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "ShareableListsApi"
        }
      + tags_all                          = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-shareablelistsapi"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "ShareableListsApi"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 30
    }

  # aws_sqs_queue.sharelists-export-consumer-queue_redrive_sqs_queue_0CF22BB0 will be created
  + resource "aws_sqs_queue" "sharelists-export-consumer-queue_redrive_sqs_queue_0CF22BB0" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = (known after apply)
      + max_message_size                  = 262144
      + message_retention_seconds         = 345600
      + name                              = "ShareableListsApi-Prod-SharedList-Export-Deadletter"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + sqs_managed_sse_enabled           = (known after apply)
      + tags                              = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-shareablelistsapi"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "ShareableListsApi"
        }
      + tags_all                          = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-shareablelistsapi"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "ShareableListsApi"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 30
    }

  # aws_sqs_queue.sharelists-export-consumer-queue_sqs_queue_402C0DF5 will be created
  + resource "aws_sqs_queue" "sharelists-export-consumer-queue_sqs_queue_402C0DF5" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = (known after apply)
      + max_message_size                  = 262144
      + message_retention_seconds         = 1209600
      + name                              = "ShareableListsApi-Prod-SharedList-Export"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + sqs_managed_sse_enabled           = (known after apply)
      + tags                              = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-shareablelistsapi"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "ShareableListsApi"
        }
      + tags_all                          = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-shareablelistsapi"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "ShareableListsApi"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 1800
    }

  # aws_sqs_queue_policy.list-events-sns-subscription_sns-dlq-policy_4905D91A will be created
  + resource "aws_sqs_queue_policy" "list-events-sns-subscription_sns-dlq-policy_4905D91A" {
      + id        = (known after apply)
      + policy    = (known after apply)
      + queue_url = (known after apply)
    }

  # aws_sqs_queue_policy.list-events-sns-subscription_sns-sqs-policy_593FA96C will be created
  + resource "aws_sqs_queue_policy" "list-events-sns-subscription_sns-sqs-policy_593FA96C" {
      + id        = (known after apply)
      + policy    = (known after apply)
      + queue_url = (known after apply)
    }

Plan: 7 to add, 1 to change, 1 to destroy.

Changes to Outputs:
  ~ ecs-task-arn                                                           = "arn:aws:ecs:us-east-1:996905175585:task-definition/ShareableListsApi-Prod:434" -> (known after apply)

⚠️ Errors

@github-actions GitHub Actions
Copy link
github-actions bot commented Feb 27, 2025 •
edited
Loading

Plan Result (@infrastructure/account-data-deleter-production)

CI link

⚠️ Resource Deletion will happen ⚠️

This plan contains resource delete operation. Please check the plan result very carefully!

Plan: 11 to add, 1 to change, 5 to destroy.
  • Create
    • aws_dynamodb_table.export-request-state_dynamodb_table_E0092AB2
    • aws_sns_topic_subscription.sns-subscription
    • aws_sns_topic_subscription.sns-subscription-status
    • aws_sqs_queue.annotations-export-consumer-queue_redrive_sqs_queue_98F4AFB7
    • aws_sqs_queue.annotations-export-consumer-queue_sqs_queue_392D8280
    • aws_sqs_queue.export-request-consumer-queue_redrive_sqs_queue_EA1B2058
    • aws_sqs_queue.export-request-consumer-queue_sqs_queue_56CE0360
    • aws_sqs_queue.sns-topic-dql
    • aws_sqs_queue_policy.sns-dlq-policy
    • aws_sqs_queue_policy.sns-sqs-policy
  • Update
    • aws_iam_policy.data-deleter-app_application_ecs_service_ecs-iam_ecs-task-role-policy_0A0BC3A4
  • Delete
    • aws_sns_topic_subscription.list-events-sns-subscription_D086E37D
    • aws_sqs_queue.list-events-sns-subscription_sns-topic-dql_4D7AC050
    • aws_sqs_queue_policy.list-events-sns-subscription_sns-dlq-policy_4905D91A
    • aws_sqs_queue_policy.list-events-sns-subscription_sns-sqs-policy_593FA96C
  • Replace
    • aws_ecs_task_definition.data-deleter-app_application_ecs_service_ecs-task_243DED07
Change Result (Click me) 
  # data.aws_iam_policy_document.data-deleter-app_application_ecs_service_ecs-iam_data-ecs-task-role-policy_A007BDF5 will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "data-deleter-app_application_ecs_service_ecs-iam_data-ecs-task-role-policy_A007BDF5" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)
      + version       = "2012-10-17"

      + statement {
          + actions   = [
              + "xray:GetSamplingRules",
              + "xray:GetSamplingStatisticSummaries",
              + "xray:GetSamplingTargets",
              + "xray:PutTelemetryRecords",
              + "xray:PutTraceSegments",
            ]
          + effect    = "Allow"
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "sqs:DeleteMessage",
              + "sqs:ReceiveMessage",
              + "sqs:SendMessage",
              + "sqs:SendMessageBatch",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Export",
              + "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Import-Files",
              + "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-Sqs-Batch-Delete-Consumer-Queue",
              + (known after apply),
              + (known after apply),
            ]
        }
      + statement {
          + actions   = [
              + "sqs:SendMessage",
              + "sqs:SendMessageBatch",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Import-Batches",
            ]
        }
      + statement {
          + actions   = [
              + "s3:ListBucket",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::com.getpocket-prod.list-exports",
            ]
        }
      + statement {
          + actions   = [
              + "s3:*Object",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::com.getpocket-prod.list-exports/*",
            ]
        }
      + statement {
          + actions   = [
              + "s3:GetObject",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::com.getpocket-prod.list-imports/*",
            ]
        }
      + statement {
          + actions   = [
              + "events:PutEvents",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:events:us-east-1:996905175585:event-bus/PocketEventBridge-Prod-Shared-Event-Bus",
            ]
        }
      + statement {
          + actions   = [
              + "dynamodb:DescribeTable",
              + "dynamodb:Get*",
              + "dynamodb:UpdateItem",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
              + (known after apply),
            ]
        }
    }

  # data.aws_iam_policy_document.sns-dlq-policy-document will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "sns-dlq-policy-document" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions   = [
              + "sqs:SendMessage",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
            ]

          + condition {
              + test     = "ArnEquals"
              + values   = [
                  + "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents",
                  + "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListExportReadyEvents",
                ]
              + variable = "aws:SourceArn"
            }

          + principals {
              + identifiers = [
                  + "sns.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
    }

  # data.aws_iam_policy_document.sns-sqs-policy-document will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "sns-sqs-policy-document" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions   = [
              + "sqs:SendMessage",
            ]
          + effect    = "Allow"
          + resources = [
        
10000
      + (known after apply),
            ]

          + condition {
              + test     = "ArnEquals"
              + values   = [
                  + "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents",
                  + "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListExportReadyEvents",
                ]
              + variable = "aws:SourceArn"
            }

          + principals {
              + identifiers = [
                  + "sns.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
    }

  # aws_dynamodb_table.export-request-state_dynamodb_table_E0092AB2 will be created
  + resource "aws_dynamodb_table" "export-request-state_dynamodb_table_E0092AB2" {
      + arn              = (known after apply)
      + billing_mode     = "PAY_PER_REQUEST"
      + hash_key         = "requestId"
      + id               = (known after apply)
      + name             = "ADD-Prod-Export-Request-State"
      + read_capacity    = (known after apply)
      + stream_arn       = (known after apply)
      + stream_label     = (known after apply)
      + stream_view_type = (known after apply)
      + tags             = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + tags_all         = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + write_capacity   = (known after apply)

      + attribute {
          + name = "requestId"
          + type = "S"
        }

      + point_in_time_recovery {
          + enabled = true
        }
    }

  # aws_ecs_task_definition.data-deleter-app_application_ecs_service_ecs-task_243DED07 must be replaced
-/+ resource "aws_ecs_task_definition" "data-deleter-app_application_ecs_service_ecs-task_243DED07" {
      ~ arn                      = "arn:aws:ecs:us-east-1:996905175585:task-definition/AccountDataDeleter-Prod:914" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:us-east-1:996905175585:task-definition/AccountDataDeleter-Prod" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [
              ~ {
                  ~ environment            = [
                        # (3 unchanged elements hidden)
                        {
                            name  = "EVENT_BUS_NAME"
                            value = "PocketEventBridge-Prod-Shared-Event-Bus"
                        },
                      + {
                          + name  = "EXPORT_REQUEST_QUEUE_URL"
                          + value = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-Export-Request"
                        },
                      + {
                          + name  = "EXPORT_REQUEST_STATE_TABLE"
                          + value = "ADD-Prod-Export-Request-State"
                        },
                        {
                            name  = "LIST_EXPORT_ARCHIVE_PREFIX"
                            value = "archives"
                        },
                        # (4 unchanged elements hidden)
                        {
                            name  = "OTLP_COLLECTOR_URL"
                            value = "https://otel-collector.readitlater.com:443"
                        },
                      + {
                          + name  = "SQS_ANNOTATIONS_EXPORT_QUEUE_URL"
                          + value = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-Annotations-Export"
                        },
                        {
                            name  = "SQS_BATCH_DELETE_QUEUE_URL"
                            value = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-Sqs-Batch-Delete-Consumer-Queue"
                        },
                        # (3 unchanged elements hidden)
                    ]
                    name                   = "app"
                    # (10 unchanged attributes hidden)
                },
            ] # forces replacement
        )
      ~ id                       = "AccountDataDeleter-Prod" -> (known after apply)
      ~ revision                 = 914 -> (known after apply)
        tags                     = {
            "app_code"       = "pocket"
            "component_code" = "pocket-accountdatadeleter"
            "costCenter"     = "Pocket"
            "env_code"       = "prod"
            "environment"    = "Prod"
            "owner"          = "Pocket"
            "service"        = "AccountDataDeleter"
        }
        # (12 unchanged attributes hidden)
    }

  # aws_iam_policy.data-deleter-app_application_ecs_service_ecs-iam_ecs-task-role-policy_0A0BC3A4 will be updated in-place
  ~ resource "aws_iam_policy" "data-deleter-app_application_ecs_service_ecs-iam_ecs-task-role-policy_0A0BC3A4" {
        id               = "arn:aws:iam::996905175585:policy/AccountDataDeleter-Prod-TaskRolePolicy"
        name             = "AccountDataDeleter-Prod-TaskRolePolicy"
      ~ policy           = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "xray:PutTraceSegments",
                          - "xray:PutTelemetryRecords",
                          - "xray:GetSamplingTargets",
                          - "xray:GetSamplingStatisticSummaries",
                          - "xray:GetSamplingRules",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = [
                          - "sqs:SendMessageBatch",
                          - "sqs:SendMessage",
                          - "sqs:ReceiveMessage",
                          - "sqs:DeleteMessage",
                        ]
                      - Effect   = "Allow"
                      - Resource = [
                          - "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-Sqs-Batch-Delete-Consumer-Queue",
                          - "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Import-Files",
                          - "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Export",
                        ]
                    },
                  - {
                      - Action   = [
                          - "sqs:SendMessageBatch",
                          - "sqs:SendMessage",
                        ]
                      - Effect   = "Allow"
                      - Resource = "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Import-Batches"
                    },
                  - {
                      - Action   = "s3:ListBucket"
                      - Effect   = "Allow"
                      - Resource = "arn:aws:s3:::com.getpocket-prod.list-exports"
                    },
                  - {
                      - Action   = "s3:*Object"
                      - Effect   = "Allow"
                      - Resource = "arn:aws:s3:::com.getpocket-prod.list-exports/*"
                    },
                  - {
                      - Action   = "s3:GetObject"
                      - Effect   = "Allow"
                      - Resource = "arn:aws:s3:::com.getpocket-prod.list-imports/*"
                    },
                  - {
                      - Action   = "events:PutEvents"
                      - Effect   = "Allow"
                      - Resource = "arn:aws:events:us-east-1:996905175585:event-bus/PocketEventBridge-Prod-Shared-Event-Bus"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags             = {
            "app_code"       = "pocket"
            "component_code" = "pocket-accountdatadeleter"
            "costCenter"     = "Pocket"
            "env_code"       = "prod"
            "environment"    = "Prod"
            "owner"          = "Pocket"
            "service"        = "AccountDataDeleter"
        }
        # (7 unchanged attributes hidden)
    }

  # aws_sns_topic_subscription.list-events-sns-subscription_D086E37D will be destroyed
  # (because aws_sns_topic_subscription.list-events-sns-subscription_D086E37D is not in configuration)
  - resource "aws_sns_topic_subscription" "list-events-sns-subscription_D086E37D" {
      - arn                             = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents:fcdea3e6-985b-41dd-9133-60b5abd49f70" -> null
      - confirmation_timeout_in_minutes = 1 -> null
      - confirmation_was_authenticated  = true -> null
      - endpoint                        = "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Export" -> null
      - endpoint_auto_confirms          = false -> null
      - filter_policy                   = jsonencode(
            {
              - detail-type = [
                  - "list-export-requested",
                ]
            }
        ) -> null
      - filter_policy_scope             = "MessageBody" -> null
      - id                              = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents:fcdea3e6-985b-41dd-9133-60b5abd49f70" -> null
      - owner_id                        = "996905175585" -> null
      - pending_confirmation            = false -> null
      - protocol                        = "sqs" -> null
      - raw_message_delivery            = false -> null
      - redrive_policy                  = jsonencode(
            {
              - deadLetterTargetArn = "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ"
            }
        ) -> null
      - topic_arn                       = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents" -> null
        # (3 unchanged attributes hidden)
    }

  # aws_sns_topic_subscription.sns-subscription will be created
  + resource "aws_sns_topic_subscription" "sns-subscription" {
      + arn                             = (known after apply)
      + confirmation_timeout_in_minutes = 1
      + confirmation_was_authenticated  = (known after apply)
      + endpoint                        = (known after apply)
      + endpoint_auto_confirms          = false
      + filter_policy                   = jsonencode(
            {
              + detail-type = [
                  + "list-export-requested",
                ]
            }
        )
      + filter_policy_scope             = "MessageBody"
      + id                              = (known after apply)
      + owner_id                        = (known after apply)
      + pending_confirmation            = (known after apply)
      + protocol                        = "sqs"
      + raw_message_delivery            = false
      + redrive_policy                  = (known after apply)
      + topic_arn                       = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents"
    }

  # aws_sns_topic_subscription.sns-subscription-status will be created
  + resource "aws_sns_topic_subscription" "sns-subscription-status" {
      + arn                             = (known after apply)
      + confirmation_timeout_in_minutes = 1
      + confirmation_was_authenticated  = (known after apply)
      + endpoint                        = (known after apply)
      + endpoint_auto_confirms          = false
      + filter_policy                   = jsonencode(
            {
              + detail-type = [
                  + "export-part-complete",
                ]
            }
        )
      + filter_policy_scope             = "MessageBody"
      + id                              = (known after apply)
      + owner_id                        = (known after apply)
      + pending_confirmation            = (known after apply)
      + protocol                        = "sqs"
      + raw_message_delivery            = false
      + redrive_policy                  = (known after apply)
      + topic_arn                       = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListExportReadyEvents"
    }

  # aws_sqs_queue.annotations-export-consumer-queue_redrive_sqs_queue_98F4AFB7 will be created
  + resource "aws_sqs_queue" "annotations-export-consumer-queue_redrive_sqs_queue_98F4AFB7" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = (known after apply)
      + max_message_size                  = 262144
      + message_retention_seconds         = 345600
      + name                              = "AccountDataDeleter-Prod-Annotations-Export-Deadletter"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + sqs_managed_sse_enabled           = (known after apply)
      + tags                              = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + tags_all                          = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 30
    }

  # aws_sqs_queue.annotations-export-consumer-queue_sqs_queue_392D8280 will be created
  + resource "aws_sqs_queue" "annotations-export-consumer-queue_sqs_queue_392D8280" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = (known after apply)
      + max_message_size                  = 262144
      + message_retention_seconds         = 1209600
      + name                              = "AccountDataDeleter-Prod-Annotations-Export"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + sqs_managed_sse_enabled           = (known after apply)
      + tags                              = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + tags_all                          = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 1800
    }

  # aws_sqs_queue.export-request-consumer-queue_redrive_sqs_queue_EA1B2058 will be created
  + resource "aws_sqs_queue" "export-request-consumer-queue_redrive_sqs_queue_EA1B2058" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = (known after apply)
      + max_message_size                  = 262144
      + message_retention_seconds         = 345600
      + name                              = "AccountDataDeleter-Prod-Export-Request-Deadletter"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + sqs_managed_sse_enabled           = (known after apply)
      + tags                              = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + tags_all                          = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 30
    }

  # aws_sqs_queue.export-request-consumer-queue_sqs_queue_56CE0360 will be created
  + resource "aws_sqs_queue" "export-request-consumer-queue_sqs_queue_56CE0360" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = (known after apply)
      + max_message_size                  = 262144
      + message_retention_seconds         = 1209600
      + name                              = "AccountDataDeleter-Prod-Export-Request"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + sqs_managed_sse_enabled           = (known after apply)
      + tags                              = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + tags_all                          = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 1800
    }

  # aws_sqs_queue.list-events-sns-subscription_sns-topic-dql_4D7AC050 will be destroyed
  # (because aws_sqs_queue.list-events-sns-subscription_sns-topic-dql_4D7AC050 is not in configuration)
  - resource "aws_sqs_queue" "list-events-sns-subscription_sns-topic-dql_4D7AC050" {
      - arn                               = "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ" -> null
      - content_based_deduplication       = false -> null
      - delay_seconds                     = 0 -> null
      - fifo_queue                        = false -> null
      - id                                = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ" -> null
      - kms_data_key_reuse_period_seconds = 300 -> null
      - max_message_size                  = 262144 -> null
      - message_retention_seconds         = 345600 -> null
      - name                              = "AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ" -> null
      - policy                            = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sqs:SendMessage"
                      - Condition = {
                          - ArnEquals = {
                              - "aws:SourceArn" = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "sns.amazonaws.com"
                        }
                      - Resource  = "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
      - receive_wait_time_seconds         = 0 -> null
      - sqs_managed_sse_enabled           = true -> null
      - tags                              = {
          - "app_code"       = "pocket"
          - "component_code" = "pocket-accountdatadeleter"
          - "costCenter"     = "Pocket"
          - "env_code"       = "prod"
          - "environment"    = "Prod"
          - "owner"          = "Pocket"
          - "service"        = "AccountDataDeleter"
        } -> null
      - tags_all                          = {
          - "app_code"       = "pocket"
          - "component_code" = "pocket-accountdatadeleter"
          - "costCenter"     = "Pocket"
          - "env_code"       = "prod"
          - "environment"    = "Prod"
          - "owner"          = "Pocket"
          - "service"        = "AccountDataDeleter"
        } -> null
      - url                               = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ" -> null
      - visibility_timeout_seconds        = 30 -> null
        # (6 unchanged attributes hidden)
    }

  # aws_sqs_queue.sns-topic-dql will be created
  + resource "aws_sqs_queue" "sns-topic-dql" {
      + arn                               = (known after apply)
      + content_based_deduplication       = false
      + deduplication_scope               = (known after apply)
      + delay_seconds                     = 0
      + fifo_queue                        = false
      + fifo_throughput_limit             = (known after apply)
      + id                                = (known after apply)
      + kms_data_key_reuse_period_seconds = (known after apply)
      + max_message_size                  = 262144
      + message_retention_seconds         = 345600
      + name                              = "AccountDataDeleter-Prod-List-Export-SNS-Topic-DLQ"
      + name_prefix                       = (known after apply)
      + policy                            = (known after apply)
      + receive_wait_time_seconds         = 0
      + redrive_allow_policy              = (known after apply)
      + redrive_policy                    = (known after apply)
      + sqs_managed_sse_enabled           = (known after apply)
      + tags                              = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + tags_all                          = {
          + "app_code"       = "pocket"
          + "component_code" = "pocket-accountdatadeleter"
          + "costCenter"     = "Pocket"
          + "env_code"       = "prod"
          + "environment"    = "Prod"
          + "owner"          = "Pocket"
          + "service"        = "AccountDataDeleter"
        }
      + url                               = (known after apply)
      + visibility_timeout_seconds        = 30
    }

  # aws_sqs_queue_policy.list-events-sns-subscription_sns-dlq-policy_4905D91A will be destroyed
  # (because aws_sqs_queue_policy.list-events-sns-subscription_sns-dlq-policy_4905D91A is not in configuration)
  - resource "aws_sqs_queue_policy" "list-events-sns-subscription_sns-dlq-policy_4905D91A" {
      - id        = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ" -> null
      - policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sqs:SendMessage"
                      - Condition = {
                          - ArnEquals = {
                              - "aws:SourceArn" = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "sns.amazonaws.com"
                        }
                      - Resource  = "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
      - queue_url = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-List-Export-SNS-SNS-Topic-DLQ" -> null
    }

  # aws_sqs_queue_policy.list-events-sns-subscription_sns-sqs-policy_593FA96C will be destroyed
  # (because aws_sqs_queue_policy.list-events-sns-subscription_sns-sqs-policy_593FA96C is not in configuration)
  - resource "aws_sqs_queue_policy" "list-events-sns-subscription_sns-sqs-policy_593FA96C" {
      - id        = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-List-Export" -> null
      - policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sqs:SendMessage"
                      - Condition = {
                          - ArnEquals = {
                              - "aws:SourceArn" = "arn:aws:sns:us-east-1:996905175585:PocketEventBridge-Prod-ListEvents"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "sns.amazonaws.com"
                        }
                      - Resource  = "arn:aws:sqs:us-east-1:996905175585:AccountDataDeleter-Prod-List-Export"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
      - queue_url = "https://sqs.us-east-1.amazonaws.com/996905175585/AccountDataDeleter-Prod-List-Export" -> null
    }

  # aws_sqs_queue_policy.sns-dlq-policy will be created
  + resource "aws_sqs_queue_policy" "sns-dlq-policy" {
      + id        = (known after apply)
      + policy    = (known after apply)
      + queue_url = (known after apply)
    }

  # aws_sqs_queue_policy.sns-sqs-policy will be created
  + resource "aws_sqs_queue_policy" "sns-sqs-policy" {
      + id        = (known after apply)
      + policy    = (known after apply)
      + queue_url = (known after apply)
    }

Plan: 11 to add, 1 to change, 5 to destroy.

Changes to Outputs:
  ~ ecs-task-arn                                                                              = "arn:aws:ecs:us-east-1:996905175585:task-definition/AccountDataDeleter-Prod:914" -> (known after apply)

⚠️ Errors

@github-actions github-actions bot deployed to @server/otel-collector-dev March 5, 2025 01:08 Active
@github-actions github-actions bot deployed to @server/braze-content-proxy-dev March 5, 2025 01:08 Active
@github-actions github-actions bot deployed to @server/v3-proxy-api-dev March 5, 2025 01:09 Active
@github-actions github-actions bot deployed to @server/shared-snowplow-consumer-dev March 5, 2025 01:09 Active
@github-actions github-actions bot deployed to @server/parser-graphql-wrapper-dev March 5, 2025 01:09 Active
@github-actions github-actions bot deployed to @server/image-api-dev March 5, 2025 01:09 Active
@github-actions github-actions bot deployed to @server/annotations-api-dev March 5, 2025 01:09 Active
@github-actions github-actions bot deployed to @server/user-api-dev March 5, 2025 01:09 Active
@github-actions github-actions bot deployed to @server/shares-api-dev March 5, 2025 01:09 Active
@github-actions github-actions bot deployed to @server/feature-flags-dev March 5, 2025 01:09 Active
@github-actions github-actions bot deployed to @server/account-data-deleter-dev March 5, 2025 01:10 Active
@github-actions github-actions bot deployed to @server/shareable-lists-api-dev March 5, 2025 01:10 Active
@github-actions github-actions bot deployed to @server/list-api-dev March 5, 2025 01:10 Active
@github-actions github-actions bot deployed to @server/account-data-deleter-dev March 6, 2025 19:10 Active
@kschelonka
feat(export): new infrastructure for multi-service export
27bd35f
@kschelonka
feat(export): set up multi-service export capabilities
9fdf2d4 
At this time the behavior should be the same, but
there is an additional layer acting like a state machine
to schedule the work, determine if it's finished, and
notify the user if it is.
@kschelonka
chore: fix some test mocks
01a3a6b
@kschelonka
chore: init queue handler
794517d 
remove dev fxa secret env vars
(they don't exist)
@kschelonka
chore: add dynamodb permissions
70646b2
@kschelonka
fix: re-throw unhandled error for sqs poller
785c433
@kschelonka
chore(temp): add dev data
03da9c8
@kschelonka
fix(export): use attribute substitution for reserved keywords
73c220e 
in DynamoDB

feat(terraform): Add new method for subscribing SQS to multiple SNS

Need to be able to update the SQS access policy to allow two ARNs;
creating subscriptions with existing method results with the policy
being chosen randomly since only one can be applied to a queue.

Since we are already using the wrapped helper stacks, just make
a new one. Alternatively just write it all manually.
@kschelonka
feat(export): export for annotations and shareable-lists
aa0aa86 
pulling out shared code into package
@kschelonka kschelonka force-pushed the multi-export-infrastructure branch from f95fe7e to 4a4af65 Compare March 6, 2025 19:50
@kschelonka kschelonka changed the title wip: infrastructure for multi-export feat(export): async, multi-service export Mar 6, 2025
@github-actions github-actions bot deployed to @server/otel-collector-dev March 6, 2025 20:56 Active
@github-actions github-actions bot deployed to @server/braze-content-proxy-dev March 6, 2025 20:57 Active
@github-actions github-actions bot deployed to @server/image-api-dev March 6, 2025 20:57 Active
@github-actions github-actions bot deployed to @server/shares-api-dev March 6, 2025 20:57 Active
@github-actions github-actions bot deployed to @server/user-api-dev March 6, 2025 20:57 Active
@github-actions github-actions bot deployed to @server/shared-snowplow-consumer-dev March 6, 2025 20:57 Active
@github-actions github-actions bot deployed to @server/v3-proxy-api-dev March 6, 2025 20:58 Active
@github-actions github-actions bot deployed to @server/parser-graphql-wrapper-dev March 6, 2025 20:58 Active
@github-actions github-actions bot deployed to @server/list-api-dev March 6, 2025 20:58 Active
@github-actions github-actions bot deployed to @server/feature-flags-dev March 6, 2025 20:58 Active
@github-actions github-actions bot deployed to @server/shareable-lists-api-dev March 6, 2025 20:58 Active
@github-actions github-actions bot deployed to @server/account-data-deleter-dev March 6, 2025 20:58 Active
@github-actions github-actions bot deployed to @server/annotations-api-dev March 6, 2025 20:58 Active
@kschelonka kschelonka merged commit 383e417 into main Mar 6, 2025
387 of 391 checks passed
@kschelonka kschelonka deleted the multi-export-infrastructure branch March 6, 2025 21:56
@github-actions GitHub Actions
Copy link
github-actions bot commented Mar 6, 2025

🎉 This PR is included in version @pocket-tools/terraform-modules-v5.20.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions GitHub Actions
Copy link
github-actions bot commented Mar 6, 2025

🎉 This PR is included in version @pocket-tools/event-bridge-v1.11.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions GitHub Actions
Copy link
github-actions bot commented Mar 6, 2025

🎉 This PR is included in version @pocket-tools/aws-utils-v1.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Gio2018 Gio2018 Awaiting requested review from Gio2018 Gio2018 is a code owner automatically assigned from Pocket/pocket

Assignees
No one assigned
Labels
released
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@kschelonka
0