Skip to content

Conversation

@DecFox
Copy link
Contributor

@DecFox DecFox commented Nov 20, 2025

We experienced an increase in the avg INSERT query latency during high measurement volume after scaling the ecs cluster to include more machines. This could be due to the db being a bottleneck in terms of the number of client machines in the cluster. To this end, this diff reduces the number of machines and spins up larger ones instead so the total count of machines stays low.

Screenshot 2025-11-19 at 22 00 41

@DecFox DecFox requested a review from LDiazN November 20, 2025 13:01
@github-actions
Copy link

github-actions bot commented Nov 20, 2025

Terraform Run Output 🤖

Format and Style 🖌failure

Initialization ⚙️success

Validation 🤖success

Validation Output

$ terraform validate

Warning: Available Write-only Attribute Alternative

  with module.ooni_monitoring.aws_ssm_parameter.ooni_monitoring_access_key,
  on ../../modules/ooni_monitoring/main.tf line 47, in resource "aws_ssm_parameter" "ooni_monitoring_access_key":
  47:   value = aws_iam_access_key.ooni_monitoring.id

The attribute value has a write-only alternative value_wo available. Use the
write-only alternative of the attribute when possible.

(and one more similar warning elsewhere)
Success! The configuration is valid, but there were some validation warnings
as shown above.

Plan 📖success

  • Plan: 2 to add, 3 to change, 6 to destroy.
Show Plan

$ terraform plan
Acquiring state lock. This may take a few moments...
module.ooni_monitoring_proxy.data.cloudinit_config.ooni_ec2: Reading...
module.ansible_inventory.local_file.ansible_inventory: Refreshing state... [id=b6de844ed8d384f890fa6f467502390de843f758]
module.adm_iam_roles.tls_private_key.oonidevops: Refreshing state... [id=b49a9fdb9f720320340226016efe24808dd68203]
random_id.artifact_id: Refreshing state... [id=8Ujqew]
module.ooniapi_frontend.random_id.artifact_id: Refreshing state... [id=_3cQlA]
module.ooni_monitoring_proxy.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ooni_anonc.data.cloudinit_config.ooni_ec2: Reading...
module.ooni_fastpath.data.cloudinit_config.ooni_ec2: Reading...
data.dns_a_record_set.monitoring_host: Reading...
module.ooni_clickhouse_proxy.data.cloudinit_config.ooni_ec2: Reading...
module.ooni_fastpath.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ooni_anonc.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ooni_clickhouse_proxy.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ansible_inventory.null_resource.ansible_update_known_hosts: Refreshing state... [id=236461505953331670]
data.dns_a_record_set.monitoring_host: Read complete after 0s [id=monitoring.ooni.org]
module.ooniapi_user.aws_ses_email_identity.ooniapi: Refreshing state... [id=admin+dev@ooni.org]
module.ooni_clickhouse_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.adm_iam_roles.aws_secretsmanager_secret.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe]
module.ooniapi_ooniprobe.aws_appautoscaling_policy.policies["memory"]: Refreshing state... [id=memory]
module.ooniapi_cluster.aws_iam_role.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role]
module.ooniapi_oonimeasurements.aws_appautoscaling_policy.policies["memory"]: Refreshing state... [id=memory]
module.ooniapi_ooniprobe.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniprobe]
module.ooniapi_oonimeasurements.aws_appautoscaling_target.ecs_target[0]: Refreshing state... [id=service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements]
module.oonidevops_github_user.aws_secretsmanager_secret.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd]
module.ooniapi_ooniprobe.aws_appautoscaling_target.ecs_target[0]: Refreshing state... [id=service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe]
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonimeasurements_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_reverseproxy.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-reverseproxy]
module.ooni_clickhouse_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 1s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.oonidevops_github_user.aws_iam_user.oonidevops_github: Refreshing state... [id=oonidevops-github]
module.ooniapi_oonimeasurements_deployer.data.aws_caller_identity.current: Read complete after 1s [id=905418398257]
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
aws_s3_bucket.ooniapi_codepipeline_bucket: Refreshing state... [id=codepipeline-ooniapi-eu-central-1-f148ea7b]
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Reading...
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Read complete after 0s [id=367960279]
module.adm_iam_roles.aws_key_pair.oonidevops: Refreshing state... [id=oonidevops]
module.oonidevops_github_user.aws_iam_policy.oonidevops_github: Refreshing state... [id=arn:aws:iam::905418398257:policy/oonidevops-github-policy]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_reverseproxy.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.oonitier1plus_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Reading...
module.ooniapi_user.aws_secretsmanager_secret.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx]
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.oonitier1plus_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Read complete after 0s [id=/aws/service/ecs/optimized-ami/amazon-linux-2/recommended]
module.ooniapi_ooniauth.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniauth]
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 1s [id=ooniapi-service-ooniauth-td/ooniapi-service-ooniauth]
aws_acm_certificate.ooniapi_frontend: Refreshing state... [id=arn:aws:acm:eu-central-1:905418398257:certificate/7f60c068-9bd3-4251-8468-6583b131afe4]
data.aws_ssm_parameter.jwt_secret_legacy: Reading...
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=ooniapi-service-oonifindings-td/ooniapi-service-oonifindings]
aws_s3_bucket.ooniprobe_failed_reports: Refreshing state... [id=ooniprobe-failed-reports-eu-central-1]
data.aws_ssm_parameter.oonipg_url: Reading...
data.aws_ssm_parameter.jwt_secret_legacy: Read complete after 0s [id=/oonidevops/secrets/ooni_services/jwt_secret_legacy]
module.ooniapi_oonirun.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=ooniapi-service-ooniprobe-td/ooniapi-service-ooniprobe]
module.ooni_monitoring_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.ooniapi_reverseproxy.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=ooniapi-service-reverseproxy-td/ooniapi-service-reverseproxy]
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Reading...
module.ooni_monitoring.aws_iam_user.ooni_monitoring: Refreshing state... [id=oonidevops-monitoring]
module.ooniapi_user.aws_iam_user.ooniapi: Refreshing state... [id=oonidevops-ooniapi]
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=ooniapi-service-oonirun-td/ooniapi-service-oonirun]
data.aws_availability_zones.available: Reading...
data.aws_ssm_parameter.clickhouse_readonly_url: Reading...
data.aws_ssm_parameter.oonipg_url: Read complete after 0s [id=/oonidevops/secrets/ooni-tier0-postgres/postgresql_write_url]
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Reading...
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Reading...
data.aws_ssm_parameter.do_token: Reading...
module.ooniapi_oonifindings.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonifindings]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooni_monitoring_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooniapi_oonimeasurements.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonimeasurements]
module.oonitier1plus_cluster.aws_iam_role.container_host: Refreshing state... [id=oonitier1plus-ecs-cluster-container-host-role]
data.aws_availability_zones.available: Read complete after 0s [id=eu-central-1]
module.ooni_anonc.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Read complete after 0s [id=/aws/service/ecs/optimized-ami/amazon-linux-2/recommended]
module.ooniapi_cluster.aws_cloudwatch_log_group.ooniapi_services: Refreshing state... [id=ooni-ecs-group/ooniapi-ecs-cluster]
data.aws_ssm_parameter.clickhouse_readonly_url: Read complete after 0s [id=/oonidevops/secrets/clickhouse_readonly_url]
module.ooniapi_oonirun.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonirun]
data.aws_ssm_parameter.clickhouse_readonly_test_url: Reading...
data.aws_ssm_parameter.prometheus_metrics_password: Reading...
module.ooniapi_reverseproxy_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-reverseproxy]
data.aws_ssm_parameter.do_token: Read complete after 0s [id=/oonidevops/secrets/digitalocean_access_token]
module.ooniapi_oonimeasurements_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonimeasurements]
module.ooniapi_frontend.aws_s3_bucket.athena_results: Refreshing state... [id=ooni-athena-results-ff771094]
module.ooni_anonc.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Reading...
module.adm_iam_roles.aws_iam_policy.oonidevops: Refreshing state... [id=arn:aws:iam::905418398257:policy/OONIDevopsPolicy]
data.aws_ssm_parameter.clickhouse_readonly_test_url: Read complete after 1s [id=/oonidevops/secrets/clickhouse_readonly_test_url]
module.ooniapi_frontend.aws_s3_bucket.load_balancer_logs: Refreshing state... [id=lb-logs-eu-central-1-ff771094]
module.ooniapi_reverseproxy_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Read complete after 1s [id=905418398257]
aws_s3_bucket.oonith_codepipeline_bucket: Refreshing state... [id=codepipeline-oonith-eu-central-1-f148ea7b]
data.aws_ssm_parameter.prometheus_metrics_password: Read complete after 1s [id=/oonidevops/ooni_services/prometheus_metrics_password]
data.aws_ssm_parameter.jwt_secret: Reading...
module.ooniapi_oonifindings_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonifindings]
module.ooniapi_ooniauth.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role]
module.ooniapi_oonifindings.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role]
module.ooniapi_reverseproxy_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
aws_secretsmanager_secret.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ]
module.ooniapi_oonimeasurements.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonimeasurements-task-role]
data.aws_ssm_parameter.jwt_secret: Read complete after 0s [id=/oonidevops/secrets/ooni_services/jwt_secret]
module.ooniapi_user.aws_secretsmanager_secret.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr]
module.ooniapi_ooniprobe.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role]
module.oonitier1plus_cluster.aws_cloudwatch_log_group.ooniapi_services: Refreshing state... [id=ooni-ecs-group/oonitier1plus-ecs-cluster]
module.ooniapi_reverseproxy.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-reverseproxy-task-role]
module.ooniapi_oonirun_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonirun]
module.ooniapi_ooniauth_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniauth]
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniprobe]
module.fastpath_builder.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-oonidkr-fastpath]
module.ooni_fastpath.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Reading...
module.fastpath_builder.data.aws_caller_identity.current: Reading...
module.ooniapi_cluster.aws_iam_role_policy.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role:ooniapi-ecs-cluster-instance-role-policy]
module.ooni_fastpath.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooniapi_cluster.aws_iam_instance_profile.container_host: Refreshing state... [id=ooniapi-ecs-cluster]
module.adm_iam_roles.aws_secretsmanager_secret_version.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|terraform-20240925140131946100000002]
module.oonidevops_github_user.aws_iam_access_key.oonidevops_github: Refreshing state... [id=AKIA5FTZELIYXDN55SMS]
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.oonidevops_github_user.aws_iam_user_policy_attachment.oonidevops_github: Refreshing state... [id=oonidevops-github-20240313195612421500000001]
module.fastpath_builder.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooni_monitoring.aws_iam_user_policy.ooni_monitoring: Refreshing state... [id=oonidevops-monitoring:oonidevops-monitoring-policy]
module.ooni_monitoring.aws_iam_access_key.ooni_monitoring: Refreshing state... [id=AKIA5FTZELIYWULOT65S]
module.ooniapi_user.aws_iam_access_key.ooniapi: Refreshing state... [id=AKIA5FTZELIYSK2XEVOT]
module.ooniapi_user.aws_iam_user_policy.ooniapi: Refreshing state... [id=oonidevops-ooniapi:oonidevops-ooniapi-policy]
module.ooniapi_oonirun.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role:ooniapi-service-oonirun-task-role]
aws_route53_record.ooniapi_frontend_cert_validation["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__ef17825e5fd9713f596344bdd9626f5e.8.th.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["oonimeasurements.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__8fb10887c4ca7af87e33703c03c4c82e.oonimeasurements.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__a064be8aa084a037ff9fa5e3e541c87d.ooniprobe.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__05c891caeb4509d4cd7f9c24d8b6dbd0.oonirun.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["api.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__cd4729fc0c282e771d056e719a7bdf4f.api.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__48cd4e71cee9930614228176b7deefb9.ooniauth.dev.ooni.io._CNAME]
module.oonitier1plus_cluster.aws_iam_role_policy.container_host: Refreshing state... [id=oonitier1plus-ecs-cluster-container-host-role:oonitier1plus-ecs-cluster-instance-role-policy]
module.oonitier1plus_cluster.aws_iam_instance_profile.container_host: Refreshing state... [id=oonitier1plus-ecs-cluster]
module.ooniapi_reverseproxy_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-reverseproxy]
module.ooniapi_oonimeasurements_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonimeasurements]
module.adm_iam_roles.aws_iam_role.oonidevops: Refreshing state... [id=oonidevops]
module.ooniapi_cluster.aws_ecs_cluster.main: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:cluster/ooniapi-ecs-cluster]
module.ooniapi_oonifindings_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonifindings]
module.ooniapi_ooniauth.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role:ooniapi-service-ooniauth-task-role]
module.ooniapi_oonifindings.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role:ooniapi-service-oonifindings-task-role]
module.ooniapi_oonimeasurements.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonimeasurements-task-role:ooniapi-service-oonimeasurements-task-role]
module.ooniapi_ooniprobe.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role:ooniapi-service-ooniprobe-task-role]
module.ooniapi_oonirun_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonirun]
module.ooniapi_reverseproxy.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-reverseproxy-task-role:ooniapi-service-reverseproxy-task-role]
module.ooniapi_ooniauth_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniauth]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniprobe]
module.oonitier1plus_cluster.aws_ecs_cluster.main: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:cluster/oonitier1plus-ecs-cluster]
module.fastpath_builder.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-oonidkr-fastpath]
module.oonidevops_github_user.aws_secretsmanager_secret_version.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd|terraform-20240519071250187000000004]
module.ooni_monitoring.aws_ssm_parameter.ooni_monitoring_access_key: Refreshing state... [id=/oonidevops/secrets/ooni_monitoring/access_key]
module.ooni_monitoring.aws_ssm_parameter.ooni_monitoring_secret_key: Refreshing state... [id=/oonidevops/secrets/ooni_monitoring/secret_key]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr|terraform-20240314200140914600000006]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx|terraform-20240314200140918400000007]
module.ooniapi_reverseproxy.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-reverseproxy-td]
module.ooniapi_oonirun.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonirun-td]
module.ooniapi_oonifindings.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonifindings-td]
module.ooniapi_oonimeasurements.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonimeasurements-td]
aws_iam_role_policy.ooniprobe_role: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role:oonidevops-dev-task-role]
module.ooniapi_ooniauth.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniauth-td]
module.ooniapi_ooniprobe.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniprobe-td]
data.aws_secretsmanager_secret_version.deploy_key: Reading...
aws_codestarconnections_connection.oonidevops: Refreshing state... [id=arn:aws:codestar-connections:eu-central-1:905418398257:connection/6bd492f6-c11d-43ec-92b0-24c47700d528]
module.terraform_state_backend.data.aws_region.current: Reading...
module.terraform_state_backend.data.aws_region.current: Read complete after 0s [id=eu-central-1]
module.network.aws_vpc.main: Refreshing state... [id=vpc-0e382f3ad89286de9]
module.ooniapi_frontend.aws_athena_database.load_balancer_logs: Refreshing state... [id=load_balancer_logs]
module.ooniapi_frontend.aws_athena_workgroup.ooni_workgroup: Refreshing state... [id=ooni-workgroup]
module.ooniapi_frontend.aws_s3_bucket_lifecycle_configuration.athena_results: Refreshing state... [id=ooni-athena-results-ff771094]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Read complete after 0s [id=2666303363]
module.terraform_state_backend.aws_s3_bucket.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
data.aws_secretsmanager_secret_version.deploy_key: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|AWSCURRENT]
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Read complete after 0s [id=2666303363]
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Reading...
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Read complete after 0s [id=1194028725]
module.terraform_state_backend.aws_dynamodb_table.with_server_side_encryption[0]: Refreshing state... [id=oonidevops-dev-terraform-state-lock]
module.ooni_th_droplet.digitalocean_droplet.ooni_th_docker[0]: Refreshing state... [id=459912318]
module.ooniapi_frontend.aws_s3_bucket_policy.alb_logs_policy: Refreshing state... [id=lb-logs-eu-central-1-ff771094]
module.ooniapi_frontend.aws_s3_bucket_ownership_controls.load_balancer_logs: Refreshing state... [id=lb-logs-eu-central-1-ff771094]
module.ooniapi_frontend.aws_s3_bucket_lifecycle_configuration.load_balancer_logs: Refreshing state... [id=lb-logs-eu-central-1-ff771094]
aws_acm_certificate_validation.ooniapi_frontend: Refreshing state... [id=2025-09-05 10:01:09.971 +0000 UTC]
module.ooniapi_ooniauth_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniauth-eu-central-1]
module.ooniapi_reverseproxy_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-reverseproxy-eu-central-1]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonifindings-eu-central-1]
module.ooniapi_oonirun_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonirun-eu-central-1]
module.fastpath_builder.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-fastpath-eu-central-1]
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniprobe-eu-central-1]
module.ooniapi_oonimeasurements_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonimeasurements-eu-central-1]
module.ooniapi_oonifindings_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonifindings]
module.ooniapi_reverseproxy_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-reverseproxy]
module.ooniapi_ooniauth_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniauth]
module.ooniapi_oonirun_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonirun]
module.fastpath_builder.aws_iam_role.codebuild: Refreshing state... [id=codebuild-oonidkr-fastpath]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniprobe]
module.ooniapi_oonimeasurements_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonimeasurements]
module.ooni_th_droplet.aws_route53_record.ooni_th["0"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_0.do.th.dev.ooni.io_A]
module.ooniapi_oonifindings_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonifindings]
module.ooniapi_reverseproxy_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-reverseproxy]
module.ooniapi_oonirun_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonirun]
module.ooniapi_ooniauth_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniauth]
module.fastpath_builder.aws_codebuild_project.oonidkr: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/oonidkr-fastpath]
module.ooniapi_ooniprobe_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniprobe]
module.ooniapi_oonimeasurements_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonimeasurements]
module.network.aws_internet_gateway.gw: Refreshing state... [id=igw-0c080e9b235ed29d1]
module.ooniapi_ooniauth.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OautM-20250115122624347200000004/6e746a968782a49f]
module.ooni_clickhouse_proxy.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oockpr20250116192249626700000002/2e9dada4dd22c268]
module.ooniapi_cluster.aws_security_group.web: Refreshing state... [id=sg-0187eedfe39538357]
module.ooniapi_oonirun.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrunM-20250115122624347100000003/17e1664b99b708a5]
module.ooni_clickhouse_proxy.aws_security_group.ec2_sg: Refreshing state... [id=sg-0903c108a44c922a5]
module.ooni_monitoring_proxy.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oomnpr20250423083217708600000002/90babad6f0c8b903]
module.ooni_monitoring_proxy.aws_security_group.ec2_sg: Refreshing state... [id=sg-00c4199ae6a658579]
module.oonitier1plus_cluster.aws_security_group.web: Refreshing state... [id=sg-07090c14e80a5def2]
module.ooniapi_oonimeasurements.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OmeaM-20250116160254864500000001/4d88cb32eb2f381c]
module.ooniapi_oonifindings.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OfinM-20250115122624350600000005/ad715c6e26dd616c]
module.ooni_fastpath.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oofstp20250724100921781100000001/153128e00c90a683]
module.ooni_fastpath.aws_security_group.ec2_sg: Refreshing state... [id=sg-03f565bff4dac580b]
module.ooniapi_ooniprobe.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OproM-20250115122624346700000001/9f9264a4e53931d3]
module.ooni_anonc.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/ooanon20251003085918842900000002/3d14866336282a65]
module.ooni_anonc.aws_security_group.ec2_sg: Refreshing state... [id=sg-063668ca077d07d17]
module.ooniapi_reverseproxy.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrevM-20250115122624347000000002/32c2f9b4e4d3b8c4]
module.fastpath_builder.aws_codepipeline.oonidkr: Refreshing state... [id=oonidkr-fastpath]
module.network.aws_subnet.public[1]: Refreshing state... [id=subnet-0b18966cccfc9d5ef]
module.network.aws_subnet.public[0]: Refreshing state... [id=subnet-0e7a4478be988463f]
module.network.aws_route_table.private: Refreshing state... [id=rtb-011463437da96c77b]
module.network.aws_route_table.public: Refreshing state... [id=rtb-0ccb0852e6a365a95]
module.network.aws_subnet.private[0]: Refreshing state... [id=subnet-09314a43ec89d6331]
module.network.aws_subnet.private[1]: Refreshing state... [id=subnet-0b899a7ad10406d06]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-1281654482]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-1099643652]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-2756751855]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-2383513485]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-316337242]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-3806784481]
module.ooni_monitoring_proxy.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-4288788045]
module.terraform_state_backend.aws_s3_bucket_server_side_encryption_configuration.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_versioning.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_public_access_block.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooni_fastpath.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-3270433048]
module.ooni_fastpath.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-697669294]
module.ooni_anonc.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-2372809180]
module.ooniapi_ooniauth.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniauth]
module.ooniapi_oonirun.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonirun]
module.ooni_anonc.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-3803885271]
module.ooniapi_oonifindings.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonifindings]
module.ooniapi_oonimeasurements.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements]
module.ooniapi_ooniprobe.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe]
module.network.aws_route_table_association.public[0]: Refreshing state... [id=rtbassoc-0dbd7fb16801ee049]
module.network.aws_route_table_association.public[1]: Refreshing state... [id=rtbassoc-08ab18165bf481054]
module.ooniapi_reverseproxy.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-reverseproxy]
module.network.aws_route_table_association.private[1]: Refreshing state... [id=rtbassoc-0c9cc0f117ef15fe7]
module.network.aws_route_table_association.private[0]: Refreshing state... [id=rtbassoc-0e7933e6b804ff2c1]
module.terraform_state_backend.aws_s3_bucket_policy.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooniapi_cluster.aws_security_group.container_host: Refreshing state... [id=sg-0aa6a97400b619de3]
module.oonitier1plus_cluster.aws_security_group.container_host: Refreshing state... [id=sg-0e74a206196727883]
module.ooniapi_frontend.aws_alb.ooniapi: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:loadbalancer/app/ooni-api-frontend/4a50f3dd46584390]
module.ooni_fastpath.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-0e2815252815b8d33]
module.ooni_anonc.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-0a87839eed39f7476]
module.oonipg.aws_db_subnet_group.pg: Refreshing state... [id=ooni-tier0-postgres-dbsng]
module.ooni_monitoring_proxy.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-0c9dddb576a4f71a3]
module.ooni_clickhouse_proxy.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-0855bc6373ff4c75b]
module.oonipg.aws_security_group.pg: Refreshing state... [id=sg-005ca579eb9c08cda]
module.terraform_state_backend.time_sleep.wait_for_aws_s3_bucket_settings[0]: Refreshing state... [id=2024-03-10T15:06:17Z]
module.ooniapi_cluster.aws_launch_template.container_host: Refreshing state... [id=lt-0e328a8671f870c64]
module.ooniapi_ooniauth_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniauth]
module.ooniapi_oonirun_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonirun]
module.oonitier1plus_cluster.aws_launch_template.container_host: Refreshing state... [id=lt-0eb432177b5a9f2aa]
module.ooniapi_oonifindings_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonifindings]
module.ooniapi_oonimeasurements_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonimeasurements]
module.ooniapi_ooniprobe_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniprobe]
module.terraform_state_backend.aws_s3_bucket_ownership_controls.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooni_fastpath.aws_instance.ooni_ec2: Refreshing state... [id=i-0f120ad4f1b95c697]
module.ooni_anonc.aws_instance.ooni_ec2: Refreshing state... [id=i-058b0fd97a772f7e1]
module.ooni_monitoring_proxy.aws_instance.ooni_ec2: Refreshing state... [id=i-067b337ada2d9cc00]
module.ooni_clickhouse_proxy.aws_instance.ooni_ec2: Refreshing state... [id=i-0f308c94682614973]
module.ooniapi_cluster.aws_autoscaling_group.container_host: Refreshing state... [id=ooniapi-ecs-cluster20240310192644083800000003]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_http: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-api-frontend/4a50f3dd46584390/664a34cfb30f72e8]
module.ooniapi_reverseproxy_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-reverseproxy]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_https: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45]
module.oonitier1plus_cluster.aws_autoscaling_group.container_host: Refreshing state... [id=oonitier1plus-ecs-cluster20251022145227179100000007]
module.oonipg.aws_db_instance.pg: Refreshing state... [id=db-27N7Q6XIBNASFCOXN4N7C762L4]
aws_route53_record.ooniapi_frontend_alt["oonimeasurements.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_oonimeasurements.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_oonirun.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_main: Refreshing state... [id=Z055356431RGCLK3JXZDL_api.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniprobe.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_8.th.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniauth.dev.ooni.io_A]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/9af03e886f8803f2]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/cc29701b6ed6aa2e]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/f4bf91203c7ca76e]
module.ooniapi_frontend.aws_alb_listener_rule.ooniapi_th: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/775cd6d0dc062fd3]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/583471b0bdc1c388]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/54cda6e694a0103f]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/5f2394ffa8b71f98]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/36d49e835c0b81c5]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/178511e1b6ae89c5]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_host[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/f3d75d5d93fd6903]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/82069bb29bca6af1]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_2[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/e6dbe09be108b001]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_1[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-api-frontend/4a50f3dd46584390/9ef650e256f41d45/1cf3d6a7a694eec9]
aws_route53_record.postgres_dns: Refreshing state... [id=Z091407123AEJO90Z3H6D_postgres.dev.ooni.nu_CNAME]
data.aws_secretsmanager_secret_version.pg_login: Reading...
data.aws_secretsmanager_secret_version.pg_login: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:rds!db-5fe27151-3a37-44e0-a5bd-3517363fa2e8-BDI0KI|AWSCURRENT]
aws_secretsmanager_secret_version.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ|terraform-20251119100733811400000001]
module.ooni_anonc.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/ooanon20251003085918842900000002/3d14866336282a65-20251003085941554000000006]
aws_route53_record.anonc_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_anonc.dev.ooni.io_CNAME]
module.ooni_fastpath.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oofstp20250724100921781100000001/153128e00c90a683-20250814085348689200000002]
aws_route53_record.fastpath_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_fastpath.dev.ooni.io_CNAME]
module.ooni_monitoring_proxy.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oomnpr20250423083217708600000002/90babad6f0c8b903-20250423083239704200000006]
module.ooni_fastpath.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-2156590276]
module.ooni_fastpath.aws_security_group_rule.ec2_sg_ingress[4]: Refreshing state... [id=sgrule-1337977241]
module.ooni_fastpath.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-1675080911]
module.ooni_fastpath.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-3445203843]
module.ooni_fastpath.aws_security_group_rule.ec2_sg_ingress[3]: Refreshing state... [id=sgrule-556872261]
aws_route53_record.monitoring_proxy_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_monitoringproxy.dev.ooni.io_CNAME]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[3]: Refreshing state... [id=sgrule-3453785268]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-3208669716]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-164247457]
module.ooni_anonc.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-2843886495]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[4]: Refreshing state... [id=sgrule-3520426823]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-1921217342]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-3288936075]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-4131337951]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[3]: Refreshing state... [id=sgrule-3953292375]
module.ooni_clickhouse_proxy.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oockpr20250116192249626700000002/2e9dada4dd22c268-20250423080341595800000003]
aws_route53_record.clickhouse_proxy_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_clickhouseproxy.dev.ooni.io_CNAME]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place
  - destroy
+/- create replacement and then destroy

Terraform planned the following actions, but then encountered a problem:

  # module.ooniapi_cluster.aws_autoscaling_group.container_host will be updated in-place
  ~ resource "aws_autoscaling_group" "container_host" {
      ~ desired_capacity                 = 4 -> 2
        id                               = "ooniapi-ecs-cluster20240310192644083800000003"
        name                             = "ooniapi-ecs-cluster20240310192644083800000003"
        # (30 unchanged attributes hidden)

      - tag {
          - key                 = "AmazonECSManaged" -> null
          - propagate_at_launch = true -> null
            # (1 unchanged attribute hidden)
        }

        # (3 unchanged blocks hidden)
    }

  # module.ooniapi_oonimeasurements.aws_appautoscaling_policy.policies["memory"] will be destroyed
  # (because aws_appautoscaling_policy.policies is not in configuration)
  - resource "aws_appautoscaling_policy" "policies" {
      - alarm_arns         = [
          - "arn:aws:cloudwatch:eu-central-1:905418398257:alarm:TargetTracking-service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements-AlarmHigh-14f71b7e-75ec-4de6-9b65-399e0f4cb897",
          - "arn:aws:cloudwatch:eu-central-1:905418398257:alarm:TargetTracking-service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements-AlarmLow-e528a6d2-cf3f-42be-8964-63c59f8780f2",
        ] -> null
      - arn                = "arn:aws:autoscaling:eu-central-1:905418398257:scalingPolicy:0f79caab-728b-4625-860a-a6cfd3e2233d:resource/ecs/service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements:policyName/memory" -> null
      - id                 = "memory" -> null
      - name               = "memory" -> null
      - policy_type        = "TargetTrackingScaling" -> null
      - resource_id        = "service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements" -> null
      - scalable_dimension = "ecs:service:DesiredCount" -> null
      - service_namespace  = "ecs" -> null

      - target_tracking_scaling_policy_configuration {
          - disable_scale_in   = false -> null
          - scale_in_cooldown  = 60 -> null
          - scale_out_cooldown = 60 -> null
          - target_value       = 60 -> null

          - predefined_metric_specification {
              - predefined_metric_type = "ECSServiceAverageMemoryUtilization" -> null
                # (1 unchanged attribute hidden)
            }
        }
    }

  # module.ooniapi_oonimeasurements.aws_appautoscaling_target.ecs_target[0] will be destroyed
  # (because aws_appautoscaling_target.ecs_target is not in configuration)
  - resource "aws_appautoscaling_target" "ecs_target" {
      - arn                = "arn:aws:application-autoscaling:eu-central-1:905418398257:scalable-target/0ec50f79caab728b4625860aa6cfd3e2233d" -> null
      - id                 = "service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements" -> null
      - max_capacity       = 4 -> null
      - min_capacity       = 2 -> null
      - resource_id        = "service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements" -> null
      - role_arn           = "arn:aws:iam::905418398257:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService" -> null
      - scalable_dimension = "ecs:service:DesiredCount" -> null
      - service_namespace  = "ecs" -> null
      - tags               = {} -> null
      - tags_all           = {} -> null

      - suspended_state {
          - dynamic_scaling_in_suspended  = false -> null
          - dynamic_scaling_out_suspended = false -> null
          - scheduled_scaling_suspended   = false -> null
        }
    }

  # module.ooniapi_oonimeasurements.aws_ecs_service.ooniapi_service will be updated in-place
  ~ resource "aws_ecs_service" "ooniapi_service" {
      ~ desired_count                      = 4 -> 2
        id                                 = "arn:aws:ecs:eu-central-1:905418398257:service/oonitier1plus-ecs-cluster/ooniapi-service-oonimeasurements"
        name                               = "ooniapi-service-oonimeasurements"
        tags                               = {
            "Environment" = "dev"
            "Name"        = "ooni-tier0-oonimeasurements"
            "Repository"  = "https://github.com/ooni/devops"
        }
      ~ task_definition                    = "arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonimeasurements-td:90" -> (known after apply)
        # (16 unchanged attributes hidden)

        # (5 unchanged blocks hidden)
    }

  # module.ooniapi_oonimeasurements.aws_ecs_task_definition.ooniapi_service must be replaced
+/- resource "aws_ecs_task_definition" "ooniapi_service" {
      ~ arn                      = "arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonimeasurements-td:90" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonimeasurements-td" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [
              ~ {
                  ~ memoryReservation = 128 -> 64
                  - mountPoints       = []
                    name              = "ooniapi-service-oonimeasurements"
                  ~ portMappings      = [
                      ~ {
                          - hostPort      = 0
                          - protocol      = "tcp"
                            # (1 unchanged attribute hidden)
                        },
                    ]
                  - systemControls    = []
                  - volumesFrom       = []
                    # (5 unchanged attributes hidden)
                },
            ] # forces replacement
        )
      ~ enable_fault_injection   = false -> (known after apply)
      ~ id                       = "ooniapi-service-oonimeasurements-td" -> (known after apply)
      - requires_compatibilities = [] -> null
      ~ revision                 = 90 -> (known after apply)
        tags                     = {
            "Environment" = "dev"
            "Name"        = "ooni-tier0-oonimeasurements"
            "Repository"  = "https://github.com/ooni/devops"
        }
        # (11 unchanged attributes hidden)
    }

  # module.ooniapi_ooniprobe.aws_appautoscaling_policy.policies["memory"] will be destroyed
  # (because aws_appautoscaling_policy.policies is not in configuration)
  - resource "aws_appautoscaling_policy" "policies" {
      - alarm_arns         = [
          - "arn:aws:cloudwatch:eu-central-1:905418398257:alarm:TargetTracking-service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe-AlarmHigh-6aeca043-4a6c-4042-999b-b56095d2d77c",
          - "arn:aws:cloudwatch:eu-central-1:905418398257:alarm:TargetTracking-service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe-AlarmLow-fd8cf48e-bf7f-4d9d-8123-04f50289dc5c",
        ] -> null
      - arn                = "arn:aws:autoscaling:eu-central-1:905418398257:scalingPolicy:36007eb5-79e9-48ef-981b-ad13a9e56afa:resource/ecs/service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe:policyName/memory" -> null
      - id                 = "memory" -> null
      - name               = "memory" -> null
      - policy_type        = "TargetTrackingScaling" -> null
      - resource_id        = "service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe" -> null
      - scalable_dimension = "ecs:service:DesiredCount" -> null
      - service_namespace  = "ecs" -> null

      - target_tracking_scaling_policy_configuration {
          - disable_scale_in   = false -> null
          - scale_in_cooldown  = 60 -> null
          - scale_out_cooldown = 60 -> null
          - target_value       = 60 -> null

          - predefined_metric_specification {
              - predefined_metric_type = "ECSServiceAverageMemoryUtilization" -> null
                # (1 unchanged attribute hidden)
            }
        }
    }

  # module.ooniapi_ooniprobe.aws_appautoscaling_target.ecs_target[0] will be destroyed
  # (because aws_appautoscaling_target.ecs_target is not in configuration)
  - resource "aws_appautoscaling_target" "ecs_target" {
      - arn                = "arn:aws:application-autoscaling:eu-central-1:905418398257:scalable-target/0ec536007eb579e948ef981bad13a9e56afa" -> null
      - id                 = "service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe" -> null
      - max_capacity       = 20 -> null
      - min_capacity       = 12 -> null
      - resource_id        = "service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe" -> null
      - role_arn           = "arn:aws:iam::905418398257:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService" -> null
      - scalable_dimension = "ecs:service:DesiredCount" -> null
      - service_namespace  = "ecs" -> null
      - tags               = {} -> null
      - tags_all           = {} -> null

      - suspended_state {
          - dynamic_scaling_in_suspended  = false -> null
          - dynamic_scaling_out_suspended = false -> null
          - scheduled_scaling_suspended   = false -> null
        }
    }

  # module.ooniapi_ooniprobe.aws_ecs_service.ooniapi_service will be updated in-place
  ~ resource "aws_ecs_service" "ooniapi_service" {
      ~ desired_count                      = 12 -> 2
        id                                 = "arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe"
        name                               = "ooniapi-service-ooniprobe"
        tags                               = {
            "Environment" = "dev"
            "Name"        = "ooni-tier0-ooniprobe"
            "Repository"  = "https://github.com/ooni/devops"
        }
      ~ task_definition                    = "arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniprobe-td:132" -> (known after apply)
        # (16 unchanged attributes hidden)

        # (5 unchanged blocks hidden)
    }

  # module.ooniapi_ooniprobe.aws_ecs_task_definition.ooniapi_service must be replaced
+/- resource "aws_ecs_task_definition" "ooniapi_service" {
      ~ arn                      = "arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniprobe-td:132" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniprobe-td" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [
              ~ {
                  ~ memoryReservation = 256 -> 64
                  - mountPoints       = []
                    name              = "ooniapi-service-ooniprobe"
                  ~ portMappings      = [
                      ~ {
                          - hostPort      = 0
                          - protocol      = "tcp"
                            # (1 unchanged attribute hidden)
                        },
                    ]
                  - systemControls    = []
                  - volumesFrom       = []
                    # (5 unchanged attributes hidden)
                },
            ] # forces replacement
        )
      ~ enable_fault_injection   = false -> (known after apply)
      ~ id                       = "ooniapi-service-ooniprobe-td" -> (known after apply)
      - requires_compatibilities = [] -> null
      ~ revision                 = 132 -> (known after apply)
        tags                     = {
            "Environment" = "dev"
            "Name"        = "ooni-tier0-ooniprobe"
            "Repository"  = "https://github.com/ooni/devops"
        }
        # (11 unchanged attributes hidden)
    }

Plan: 2 to add, 3 to change, 6 to destroy.

Warning: Argument is deprecated

  with module.adm_iam_roles.aws_iam_role.oonidevops,
  on ../../modules/adm_iam_roles/main.tf line 69, in resource "aws_iam_role" "oonidevops":
  69:   managed_policy_arns = [aws_iam_policy.oonidevops.arn]

The managed_policy_arns argument is deprecated. Use the
aws_iam_role_policy_attachment resource instead. If Terraform should
exclusively manage all managed policy attachments (the current behavior of
this argument), use the aws_iam_role_policy_attachments_exclusive resource as
well.

(and 4 more similar warnings elsewhere)

Warning: Available Write-only Attribute Alternative

  with module.ooni_monitoring.aws_ssm_parameter.ooni_monitoring_access_key,
  on ../../modules/ooni_monitoring/main.tf line 47, in resource "aws_ssm_parameter" "ooni_monitoring_access_key":
  47:   value = aws_iam_access_key.ooni_monitoring.id

The attribute value has a write-only alternative value_wo available. Use the
write-only alternative of the attribute when possible.

(and one more similar warning elsewhere)
Pusher @DecFox
Action pull_request
Environment dev
Workflow .github/workflows/check_terraform.yml
Last updated Thu, 20 Nov 2025 13:16:00 GMT

asg_min = 2
asg_max = 10
asg_desired = 7
asg_min = 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

min should be 2 to have one instance per AZ

asg_min = 2
asg_max = 5
asg_desired = 3
asg_min = 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

key_name = module.adm_iam_roles.oonidevops_key_name
ecs_cluster_id = module.ooniapi_cluster.cluster_id
task_memory = 128
task_memory = 512
Copy link
Contributor

@LDiazN LDiazN Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would try with 256 first to see how it behaves, ooniprobe doesn't needs that much memory

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants