8000 roachtest: cluster_creation failed · Issue #144214 · cockroachdb/cockroach · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
8000

roachtest: cluster_creation failed #144214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
cockroach-teamcity opened this issue Apr 10, 2025 · 391 comments
Open

roachtest: cluster_creation failed #144214

cockroach-teamcity opened this issue Apr 10, 2025 · 391 comments
Labels
branch-release-25.2 O-roachtest O-robot Originated from a bot. T-testeng TestEng Team X-infra-flake the automatically generated issue was closed due to an infrastructure problem not a product issue

Comments

@cockroach-teamcity
Copy link
Member
cockroach-teamcity commented Apr 10, 2025 •
edited by cockroach-jira-scripts
Loading

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test kv0/enc=false/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19383170-1744263552-42-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-42-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-42-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-04-10T06:33:00Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19383170-1744263552-42-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-42-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-42-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-04-10T06:33:00Z}] --user-data file:///tmp/aws-startup-script3780207060 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1839657968 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-49275
@cockroach-teamcity cockroach-teamcity added branch-release-25.2 O-roachtest O-robot Originated from a bot. T-testeng TestEng Team X-infra-flake the automatically generated issue was closed due to an infrastructure problem not a product issue labels Apr 10, 2025
This was referenced Apr 10, 2025
Closed
Open
@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test kv0/enc=false/nodes=3/cpu=32/size=64kb failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-49-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T06:40:41Z},{Key=Name,Value=teamcity-19383170-1744263552-49-n4cpu32-0003},{Key=Cluster,Value=teamcity-19383170-1744263552-49-n4cpu32},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-49-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T06:40:41Z},{Key=Name,Value=teamcity-19383170-1744263552-49-n4cpu32-0003},{Key=Cluster,Value=teamcity-19383170-1744263552-49-n4cpu32},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script3734556791 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3729930211 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test kv0/enc=false/nodes=3/cpu=32/size=64kb/splt=100 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19383170-1744263552-52-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-52-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T06:42:09Z},{Key=Name,Value=teamcity-19383170-1744263552-52-n4cpu32-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19383170-1744263552-52-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-52-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T06:42:09Z},{Key=Name,Value=teamcity-19383170-1744263552-52-n4cpu32-0002}] --user-data file:///tmp/aws-startup-script209855428 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1756783757 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity cockroach-teamcity mentioned this issue Apr 10, 2025
Closed
@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-04-10T07:40:19Z},{Key=Name,Value=teamcity-19383170-1744263552-62-n4cpu96-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-62-n4cpu96},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-62-n4cpu96}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-04-10T07:40:19Z},{Key=Name,Value=teamcity-19383170-1744263552-62-n4cpu96-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-62-n4cpu96},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-62-n4cpu96}] --user-data file:///tmp/aws-startup-script3747398356 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1304245406 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/B/nodes=3/cpu=32/isolation-level=read-committed failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-96-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-04-10T09:10:16Z},{Key=Name,Value=teamcity-19383170-1744263552-96-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-96-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-96-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-04-10T09:10:16Z},{Key=Name,Value=teamcity-19383170-1744263552-96-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-96-n4cpu32}] --user-data file:///tmp/aws-startup-script1928464400 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3037361999 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/B/nodes=3/cpu=32/uniform failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-04-10T09:14:16Z},{Key=Name,Value=teamcity-19383170-1744263552-100-n4cpu32-0002},{Key=Cluster,Value=teamcity-19383170-1744263552-100-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-100-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-04-10T09:14:16Z},{Key=Name,Value=teamcity-19383170-1744263552-100-n4cpu32-0002},{Key=Cluster,Value=teamcity-19383170-1744263552-100-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-100-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script1126300057 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3168352983 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/D/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-104-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T09:57:11Z},{Key=Name,Value=teamcity-19383170-1744263552-104-n4cpu32-0003},{Key=Cluster,Value=teamcity-19383170-1744263552-104-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-104-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T09:57:11Z},{Key=Name,Value=teamcity-19383170-1744263552-104-n4cpu32-0003},{Key=Cluster,Value=teamcity-19383170-1744263552-104-n4cpu32}] --user-data file:///tmp/aws-startup-script1348692842 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1762630119 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/D/nodes=3/cpu=32/uniform failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19383170-1744263552-107-n4cpu32-0004},{Key=Cluster,Value=teamcity-19383170-1744263552-107-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-107-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T09:58:36Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,
8000
Value=teamcity-19383170-1744263552-107-n4cpu32-0004},{Key=Cluster,Value=teamcity-19383170-1744263552-107-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-107-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T09:58:36Z}] --user-data file:///tmp/aws-startup-script133282999 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3833754965 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/E/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T09:59:54Z},{Key=Name,Value=teamcity-19383170-1744263552-110-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-110-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-110-n4cpu32},{Key=lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T09:59:54Z},{Key=Name,Value=teamcity-19383170-1744263552-110-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-110-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-110-n4cpu32},{Key=lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script3344292831 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping136197355 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/E/nodes=3/cpu=32/isolation-level=read-committed failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T10:01:12Z},{Key=Name,Value=teamcity-19383170-1744263552-113-n4cpu32-0002},{Key=Cluster,Value=teamcity-19383170-1744263552-113-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-113-n4cpu32},{Key=lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T10:01:12Z},{Key=Name,Value=teamcity-19383170-1744263552-113-n4cpu32-0002},{Key=Cluster,Value=teamcity-19383170-1744263552-113-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-113-n4cpu32},{Key=lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script2649976978 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping303544043 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/E/nodes=3/cpu=32/uniform failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T10:02:30Z},{Key=Name,Value=teamcity-19383170-1744263552-116-n4cpu32-0003},{Key=Cluster,Value=teamcity-19383170-1744263552-116-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-116-n4cpu32},{Key=lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T10:02:30Z},{Key=Name,Value=teamcity-19383170-1744263552-116-n4cpu32-0003},{Key=Cluster,Value=teamcity-19383170-1744263552-116-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-116-n4cpu32},{Key=lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script2528738562 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3597665119 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c7gd.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c7gd.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/F/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=cluster,Value=teamcity-19383170-1744263552-119-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T10:03:51Z},{Key=Name,Value=teamcity-19383170-1744263552-119-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-119-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=cluster,Value=teamcity-19383170-1744263552-119-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-10T10:03:51Z},{Key=Name,Value=teamcity-19383170-1744263552-119-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-119-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true}] --user-data file:///tmp/aws-startup-script2702139741 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping840711021 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

test ycsb/F/nodes=3/cpu=32/uniform failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-b-03oK2MEN55XVIHfOsrFYX52Kk= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-04-10T10:06:42Z},{Key=Name,Value=teamcity-19383170-1744263552-123-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-123-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-123-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-04-10T10:06:42Z},{Key=Name,Value=teamcity-19383170-1744263552-123-n4cpu32-0001},{Key=Cluster,Value=teamcity-19383170-1744263552-123-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19383170-1744263552-123-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64}] --user-data file:///tmp/aws-startup-script3307841244 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1929021454 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-d zone. Alternatively, you can try your request
      again with a different VM hardware configuration or at a later time. For more
      information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-d
      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-d zone. Alternatively, you can try your request
      again with a different VM hardware configuration or at a later time. For more
      information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-d
      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-d' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1 [owner=test-eng]

Parameters:

  • cloud=gce
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 9b3b43414e8032a1adab5425dd88fe32a09fc803:

      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-c zone. Alternatively, you can try your request
      again with a different VM hardware configuration or at a later time. For more
      information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-c
      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-c zone. Alternatively, you can try your request
      again with a different VM hardware configuration or at a later time. For more
      information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-c
      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1 [owner=test-eng]

Parameters:

  • cloud=gce
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 0e37e8c11132a7175d402a2a03ecbcf2091d3781:

test cdc/tpcc-1000/sink=kafka failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.4xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-0W5zf-GOBooUczfOIwKIZZ4fXaA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-11T06:17:25Z},{Key=Name,Value=teamcity-19395765-1744349969-32-n4cpu16-0002},{Key=Cluster,Value=teamcity-19395765-1744349969-32-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-32-n4cpu16}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-11T06:17:25Z},{Key=Name,Value=teamcity-19395765-1744349969-32-n4cpu16-0002},{Key=Cluster,Value=teamcity-19395765-1744349969-32-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-32-n4cpu16}] --user-data file:///tmp/aws-startup-script3518332221 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping70323651 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 0e37e8c11132a7175d402a2a03ecbcf2091d3781:

test kv0/enc=false/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-0W5zf-GOBooUczfOIwKIZZ4fXaA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-04-11T06:30:58Z},{Key=Name,Value=teamcity-19395765-1744349969-44-n4cpu32-0003},{Key=Cluster,Value=teamcity-19395765-1744349969-44-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-44-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-04-11T06:30:58Z},{Key=Name,Value=teamcity-19395765-1744349969-44-n4cpu32-0003},{Key=Cluster,Value=teamcity-19395765-1744349969-44-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-44-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script3466079665 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping4114981078 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 0e37e8c11132a7175d402a2a03ecbcf2091d3781:

test restore/tpce/15GB/aws/nodes=4/cpus=8 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7g.2xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-0W5zf-GOBooUczfOIwKIZZ4fXaA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-73-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-11T08:04:20Z},{Key=Name,Value=teamcity-19395765-1744349969-73-n4cpu8-0002},{Key=Cluster,Value=teamcity-19395765-1744349969-73-n4cpu8},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-73-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-11T08:04:20Z},{Key=Name,Value=teamcity-19395765-1744349969-73-n4cpu8-0002},{Key=Cluster,Value=teamcity-19395765-1744349969-73-n4cpu8},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script4255492118 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping881880208 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 0e37e8c11132a7175d402a2a03ecbcf2091d3781:

test restore/tpce/400GB/aws/inc-count=48/nodes=4/cpus=8 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7g.2xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-0W5zf-GOBooUczfOIwKIZZ4fXaA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19395765-1744349969-76-n4cpu8-0002},{Key=Cluster,Value=teamcity-19395765-1744349969-76-n4cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-76-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-11T08:05:44Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19395765-1744349969-76-n4cpu8-0002},{Key=Cluster,Value=teamcity-19395765-1744349969-76-n4cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-76-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-11T08:05:44Z}] --user-data file:///tmp/aws-startup-script3965292605 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2297923543 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 0e37e8c11132a7175d402a2a03ecbcf2091d3781:

test tpcc/headroom/n4cpu16 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.4xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-0W5zf-GOBooUczfOIwKIZZ4fXaA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-04-11T08:09:37Z},{Key=Name,Value=teamcity-19395765-1744349969-81-n4cpu16-0003},{Key=Cluster,Value=teamcity-19395765-1744349969-81-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-81-n4cpu16},{Key=lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-04-11T08:09:37Z},{Key=Name,Value=teamcity-19395765-1744349969-81-n4cpu16-0003},{Key=Cluster,Value=teamcity-19395765-1744349969-81-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19395765-1744349969-81-n4cpu16},{Key=lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script4259770235 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2746372831 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 0e37e8c11132a7175d402a2a03ecbcf2091d3781:

      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-c zone. Alternatively, you can try your request
      again with a different VM hardware configuration or at a later time. For more
      information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-c
      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-c zone. Alternatively, you can try your request
      again with a different VM hardware configuration or at a later time. For more
      information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-c
      zonesAvailable: ''
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1 [owner=test-eng]

Parameters:

  • cloud=gce
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 0e37e8c11132a7175d402a2a03ecbcf2091d3781:

message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-b zone. Consider trying your request in the us-east1-c
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-b
      zonesAvailable: us-east1-c
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
    links:
    - description: Troubleshooting documentation
      url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
    locale: en-US
    message: A n2d-custom-96-196608 VM instance with 16 local SSD(s) is currently
      unavailable in the us-east1-b zone. Consider trying your request in the us-east1-c
      zone(s), which currently has capacity to accommodate your request. Alternatively,
      you can try your request again with a different VM hardware configuration or
      at a later time. For more information, see the troubleshooting documentation.
- errorInfo:
    domain: compute.googleapis.com
    metadatas:
      attachment: local-ssd:16
      vmType: n2d-custom-96-196608
      zone: us-east1-b
      zonesAvailable: us-east1-c
    reason: resource_availability
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-b' does not have enough
  resources available to fulfill the request.  '(resource type:compute)'.: exit status 1 [owner=test-eng]

Parameters:

  • cloud=gce
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c6c16191f659d3507fb3d4b8a23279af03f5fb5f:

test kv0/enc=false/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-GEC6NNzyCT-tc3_7Oa7Pf2Rfpl8= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19404325-1744436455-43-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-12T07:34:01Z},{Key=Name,Value=teamcity-19404325-1744436455-43-n4cpu32-0003},{Key=Cluster,Value=teamcity-19404325-1744436455-43-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19404325-1744436455-43-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-04-12T07:34:01Z},{Key=Name,Value=teamcity-19404325-1744436455-43-n4cpu32-0003},{Key=Cluster,Value=teamcity-19404325-1744436455-43-n4cpu32}] --user-data file:///tmp/aws-startup-script1472897433 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping851858589 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 1b1501f18fe79c46c9244f83cbe13d1b94cad70a:

test ycsb/B/nodes=3/cpu=32/uniform failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-kIkMtqNJvanG6z3M7atmEAes3l4= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19761526-1747978730-80-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-23T07:07:05Z},{Key=Name,Value=teamcity-19761526-1747978730-80-n4cpu32-0001},{Key=Cluster,Value=teamcity-19761526-1747978730-80-n4cpu32},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19761526-1747978730-80-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-23T07:07:05Z},{Key=Name,Value=teamcity-19761526-1747978730-80-n4cpu32-0001},{Key=Cluster,Value=teamcity-19761526-1747978730-80-n4cpu32},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script2972093637 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping512908104 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ 1b1501f18fe79c46c9244f83cbe13d1b94cad70a:

test ycsb/C/nodes=3/cpu=32/uniform failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-kIkMtqNJvanG6z3M7atmEAes3l4= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-05-23T07:10:40Z},{Key=Name,Value=teamcity-19761526-1747978730-84-n4cpu32-0001},{Key=Cluster,Value=teamcity-19761526-1747978730-84-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19761526-1747978730-84-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-05-23T07:10:40Z},{Key=Name,Value=teamcity-19761526-1747978730-84-n4cpu32-0001},{Key=Cluster,Value=teamcity-19761526-1747978730-84-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19761526-1747978730-84-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64}] --user-data file:///tmp/aws-startup-script2023821466 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1101410335 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test kv0/enc=false/nodes=3/cpu=32/mt-shared-process failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-GU4hK1SzQOpCsyU_us9F27IlWWA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19770511-1748065186-23-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-23-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:07:17Z},{Key=Name,Value=teamcity-19770511-1748065186-23-n4cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19770511-1748065186-23-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-23-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:07:17Z},{Key=Name,Value=teamcity-19770511-1748065186-23-n4cpu32-0001}] --user-data file:///tmp/aws-startup-script244958332 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping443707307 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test kv0/enc=false/nodes=3/mt-shared-process failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.2xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-GU4hK1SzQOpCsyU_us9F27IlWWA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-29-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-24T06:13:36Z},{Key=Name,Value=teamcity-19770511-1748065186-29-n4cpu8-0002},{Key=Cluster,Value=teamcity-19770511-1748065186-29-n4cpu8},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-29-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-24T06:13:36Z},{Key=Name,Value=teamcity-19770511-1748065186-29-n4cpu8-0002},{Key=Cluster,Value=teamcity-19770511-1748065186-29-n4cpu8},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script3833704529 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3779178067 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test kv95/enc=false/nodes=3/cpu=32/seq failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-GU4hK1SzQOpCsyU_us9F27IlWWA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-41-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:41:52Z},{Key=Name,Value=teamcity-19770511-1748065186-41-n4cpu32-0004},{Key=Cluster,Value=teamcity-19770511-1748065186-41-n4cpu32},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-41-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:41:52Z},{Key=Name,Value=teamcity-19770511-1748065186-41-n4cpu32-0004},{Key=Cluster,Value=teamcity-19770511-1748065186-41-n4cpu32},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script9945771 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping346923294 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test restore/tpce/400GB/aws/nodes=4/cpus=8 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7g.2xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-GU4hK1SzQOpCsyU_us9F27IlWWA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-50-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:51:05Z},{Key=Name,Value=teamcity-19770511-1748065186-50-n4cpu8-0003},{Key=Cluster,Value=teamcity-19770511-1748065186-50-n4cpu8}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-50-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:51:05Z},{Key=Name,Value=teamcity-19770511-1748065186-50-n4cpu8-0003},{Key=Cluster,Value=teamcity-19770511-1748065186-50-n4cpu8}] --user-data file:///tmp/aws-startup-script2255983365 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping934928067 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test restore/tpce/8TB/aws/nodes=10/cpus=8 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7g.2xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-GU4hK1SzQOpCsyU_us9F27IlWWA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:56:37Z},{Key=Name,Value=teamcity-19770511-1748065186-56-n10cpu8-0006},{Key=Cluster,Value=teamcity-19770511-1748065186-56-n10cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-56-n10cpu8}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:56:37Z},{Key=Name,Value=teamcity-19770511-1748065186-56-n10cpu8-0006},{Key=Cluster,Value=teamcity-19770511-1748065186-56-n10cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-56-n10cpu8}] --user-data file:///tmp/aws-startup-script613152548 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2319169992 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test ycsb/A/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-GU4hK1SzQOpCsyU_us9F27IlWWA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-60-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:59:41Z},{Key=Name,Value=teamcity-19770511-1748065186-60-n4cpu32-0003},{Key=Cluster,Value=teamcity-19770511-1748065186-60-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-60-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-24T06:59:41Z},{Key=Name,Value=teamcity-19770511-1748065186-60-n4cpu32-0003},{Key=Cluster,Value=teamcity-19770511-1748065186-60-n4cpu32}] --user-data file:///tmp/aws-startup-script64252226 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping711250029 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test ycsb/B/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-GU4hK1SzQOpCsyU_us9F27IlWWA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-05-24T07:09:34Z},{Key=Name,Value=teamcity-19770511-1748065186-71-n4cpu32-0001},{Key=Cluster,Value=teamcity-19770511-1748065186-71-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-71-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-05-24T07:09:34Z},{Key=Name,Value=teamcity-19770511-1748065186-71-n4cpu32-0001},{Key=Cluster,Value=teamcity-19770511-1748065186-71-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19770511-1748065186-71-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64}] --user-data file:///tmp/aws-startup-script3367858495 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1845125468 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test costfuzz/rand-multi-region failed: (test_runner.go:864).func4: in provider: gce: T2A instances are not supported outside of [asia-southeast1-b,asia-southeast1-c,europe-west4-a,europe-west4-b,europe-west4-c,us-central1-a,us-central1-b,us-central1-f] [owner=test-eng]

Parameters:

  • cloud=gce
  • coverageBuild=false
  • cpu=4
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test kv95/enc=false/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-05-25T06:14:08Z},{Key=Name,Value=teamcity-19775447-1748151510-29-n4cpu32-0001},{Key=Cluster,Value=teamcity-19775447-1748151510-29-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-29-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-05-25T06:14:08Z},{Key=Name,Value=teamcity-19775447-1748151510-29-n4cpu32-0001},{Key=Cluster,Value=teamcity-19775447-1748151510-29-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-29-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script3046655950 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1029358490 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test kv95/enc=false/nodes=3/cpu=32/seq failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-32-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:15:46Z},{Key=Name,Value=teamcity-19775447-1748151510-32-n4cpu32-0002},{Key=Cluster,Value=teamcity-19775447-1748151510-32-n4cpu32},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-32-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:15:46Z},{Key=Name,Value=teamcity-19775447-1748151510-32-n4cpu32-0002},{Key=Cluster,Value=teamcity-19775447-1748151510-32-n4cpu32},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script3205566487 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3349149895 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test kv95/enc=false/nodes=3/lease=expiration failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.2xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19775447-1748151510-37-n4cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-37-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:20:05Z},{Key=Name,Value=teamcity-19775447-1748151510-37-n4cpu8-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19775447-1748151510-37-n4cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-37-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:20:05Z},{Key=Name,Value=teamcity-19775447-1748151510-37-n4cpu8-0002}] --user-data file:///tmp/aws-startup-script377288793 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping809431862 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=cluster,Value=teamcity-19775447-1748151510-44-n4cpu96},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-25T06:33:46Z},{Key=Name,Value=teamcity-19775447-1748151510-44-n4cpu96-0001},{Key=Cluster,Value=teamcity-19775447-1748151510-44-n4cpu96},{Key=spot,Value=true},{Key=roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=cluster,Value=teamcity-19775447-1748151510-44-n4cpu96},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-25T06:33:46Z},{Key=Name,Value=teamcity-19775447-1748151510-44-n4cpu96-0001},{Key=Cluster,Value=teamcity-19775447-1748151510-44-n4cpu96},{Key=spot,Value=true},{Key=roachprod,Value=true}] --user-data file:///tmp/aws-startup-script2114858704 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2788845082 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test ycsb/A/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19775447-1748151510-63-n4cpu32-0002},{Key=Cluster,Value=teamcity-19775447-1748151510-63-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-63-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:55:10Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19775447-1748151510-63-n4cpu32-0002},{Key=Cluster,Value=teamcity-19775447-1748151510-63-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-63-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:55:10Z}] --user-data file:///tmp/aws-startup-script3956906158 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping4227640229 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test ycsb/A/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-66-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:56:49Z},{Key=Name,Value=teamcity-19775447-1748151510-66-n4cpu32-0003},{Key=Cluster,Value=teamcity-19775447-1748151510-66-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-66-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:56:49Z},{Key=Name,Value=teamcity-19775447-1748151510-66-n4cpu32-0003},{Key=Cluster,Value=teamcity-19775447-1748151510-66-n4cpu32}] --user-data file:///tmp/aws-startup-script3286524185 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1824462209 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test ycsb/B/nodes=3 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.2xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19775447-1748151510-69-n4cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-69-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:58:35Z},{Key=Name,Value=teamcity-19775447-1748151510-69-n4cpu8-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19775447-1748151510-69-n4cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-69-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T06:58:35Z},{Key=Name,Value=teamcity-19775447-1748151510-69-n4cpu8-0001}] --user-data file:///tmp/aws-startup-script4117268340 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1609312451 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test ycsb/B/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-72-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T07:00:26Z},{Key=Name,Value=teamcity-19775447-1748151510-72-n4cpu32-0001},{Key=Cluster,Value=teamcity-19775447-1748151510-72-n4cpu32},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-72-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-25T07:00:26Z},{Key=Name,Value=teamcity-19775447-1748151510-72-n4cpu32-0001},{Key=Cluster,Value=teamcity-19775447-1748151510-72-n4cpu32},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script52654187 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3614675514 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ c616af291289bef46024b55af834079028b955d1:

test ycsb/E/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-J2cO9l9hOBaZvzLIC0wPaaJ9kQU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-05-25T07:14:17Z},{Key=Name,Value=teamcity-19775447-1748151510-83-n4cpu32-0002},{Key=Cluster,Value=teamcity-19775447-1748151510-83-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-83-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=created,Value=2025-05-25T07:14:17Z},{Key=Name,Value=teamcity-19775447-1748151510-83-n4cpu32-0002},{Key=Cluster,Value=teamcity-19775447-1748151510-83-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19775447-1748151510-83-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64}] --user-data file:///tmp/aws-startup-script3259495903 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2846691150 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test kv95/enc=false/nodes=1/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19780251-1748237910-25-n2cpu32-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-25-n2cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-25-n2cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:11:25Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19780251-1748237910-25-n2cpu32-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-25-n2cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-25-n2cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:11:25Z}] --user-data file:///tmp/aws-startup-script241966707 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1552122826 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test kv95/enc=false/nodes=1/cpu=32/mt-shared-process failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19780251-1748237910-28-n2cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-28-n2cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:13:04Z},{Key=Name,Value=teamcity-19780251-1748237910-28-n2cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19780251-1748237910-28-n2cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-28-n2cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:13:04Z},{Key=Name,Value=teamcity-19780251-1748237910-28-n2cpu32-0001}] --user-data file:///tmp/aws-startup-script1648431592 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping930140918 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test kv95/enc=false/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:17:16Z},{Key=Name,Value=teamcity-19780251-1748237910-34-n4cpu32-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-34-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-34-n4cpu32},{Key=lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:17:16Z},{Key=Name,Value=teamcity-19780251-1748237910-34-n4cpu32-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-34-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-34-n4cpu32},{Key=lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script1042328684 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping509956064 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-26T06:21:47Z},{Key=Name,Value=teamcity-19780251-1748237910-39-n4cpu96-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-39-n4cpu96},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-39-n4cpu96}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-26T06:21:47Z},{Key=Name,Value=teamcity-19780251-1748237910-39-n4cpu96-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-39-n4cpu96},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-39-n4cpu96}] --user-data file:///tmp/aws-startup-script2011922909 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3046282325 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test restore/tpce/400GB/aws/nodes=4/cpus=16 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7g.4xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=cluster,Value=teamcity-19780251-1748237910-52-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:43:33Z},{Key=Name,Value=teamcity-19780251-1748237910-52-n4cpu16-0002},{Key=Cluster,Value=teamcity-19780251-1748237910-52-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=cluster,Value=teamcity-19780251-1748237910-52-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:43:33Z},{Key=Name,Value=teamcity-19780251-1748237910-52-n4cpu16-0002},{Key=Cluster,Value=teamcity-19780251-1748237910-52-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true}] --user-data file:///tmp/aws-startup-script961270124 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping855884678 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test restore/tpce/8TB/aws/nodes=10/cpus=8 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6i.2xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-58-n10cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-26T06:49:47Z},{Key=Name,Value=teamcity-19780251-1748237910-58-n10cpu8-0005},{Key=Cluster,Value=teamcity-19780251-1748237910-58-n10cpu8},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-58-n10cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-26T06:49:47Z},{Key=Name,Value=teamcity-19780251-1748237910-58-n10cpu8-0005},{Key=Cluster,Value=teamcity-19780251-1748237910-58-n10cpu8},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script447753031 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1568473975 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test tpccbench/nodes=3/cpu=16/enc=true failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.4xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19780251-1748237910-61-n4cpu16-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-61-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-61-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-26T06:51:41Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19780251-1748237910-61-n4cpu16-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-61-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-61-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-05-26T06:51:41Z}] --user-data file:///tmp/aws-startup-script2984471054 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping815070961 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation (reached max retries: 2): Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test ycsb/A/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-64-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:53:40Z},{Key=Name,Value=teamcity-19780251-1748237910-64-n4cpu32-0002},{Key=Cluster,Value=teamcity-19780251-1748237910-64-n4cpu32},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-64-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:53:40Z},{Key=Name,Value=teamcity-19780251-1748237910-64-n4cpu32-0002},{Key=Cluster,Value=teamcity-19780251-1748237910-64-n4cpu32},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script3490434587 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1610918585 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test ycsb/B/nodes=3 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.2xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-68-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:57:19Z},{Key=Name,Value=teamcity-19780251-1748237910-68-n4cpu8-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-68-n4cpu8}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-68-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:57:19Z},{Key=Name,Value=teamcity-19780251-1748237910-68-n4cpu8-0001},{Key=Cluster,Value=teamcity-19780251-1748237910-68-n4cpu8}] --user-data file:///tmp/aws-startup-script898260098 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2108745557 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test ycsb/B/nodes=3/cpu=32 failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-71-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:59:06Z},{Key=Name,Value=teamcity-19780251-1748237910-71-n4cpu32-0003},{Key=Cluster,Value=teamcity-19780251-1748237910-71-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-71-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T06:59:06Z},{Key=Name,Value=teamcity-19780251-1748237910-71-n4cpu32-0003},{Key=Cluster,Value=teamcity-19780251-1748237910-71-n4cpu32}] --user-data file:///tmp/aws-startup-script4058156783 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping998879472 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.2 @ cfe971ca2dbac85aed9e65577feab2c87af2bc1d:

test ycsb/B/nodes=3/cpu=32/isolation-level=read-committed failed: (test_runner.go:864).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-htrOtKWj3FzsV9T0dY29GNz1cZA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19780251-1748237910-74-n4cpu32-0002},{Key=Cluster,Value=teamcity-19780251-1748237910-74-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-74-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T07:00:59Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19780251-1748237910-74-n4cpu32-0002},{Key=Cluster,Value=teamcity-19780251-1748237910-74-n4cpu32},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-19780251-1748237910-74-n4cpu32},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-05-26T07:00:59Z}] --user-data file:///tmp/aws-startup-script3405909565 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping4207519975 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch-release-25.2 O-roachtest O-robot Originated from a bot. T-testeng TestEng Team X-infra-flake the automatically generated issue was closed due to an infrastructure problem not a product issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cockroach-teamcity
0