S3 backend configuration does not react the same way whether endpoint or endpoints.s3 is specified

[Warkdev]

Terraform Version

Terraform v1.9.6
on darwin_amd64

Terraform Configuration Files

terraform {
  backend "s3" {
    region                      = "eu-fr2"
    skip_region_validation      = true
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    skip_requesting_account_id  = true
    bucket = "my-bucket-12345"
    endpoints = {
      s3 = "https://{s3_suffix}.{region}.{dnsSuffix}:{custom_port}"
    }
  }
}

Debug Output

N/A

Expected Behavior

Given the configuration above, I'm expecting the terraform init command to try connecting to the S3 endpoint mentioned in the configuration.

Actual Behavior

This is not the case, it is trying to contact an endpoint with the suffix amazonaws.com instead of the configured suffix.

Error refreshing state: RequestError: send request failed
caused by: Get "https://{s3_suffix}.{region}.amazonaws.com/<key_path_to_tf_state>": dial tcp: lookup {s3_suffix}.{region}.amazonaws.com on : no such host

Please note that using the deprecated attribute endpoint works like a charm. Therefore, I'm suspecting that endpoints.s3 is not handled in the same way.

Steps to Reproduce

terraform init with valid access/secret keys

Additional Context

No response

References

No response

[bschaatsbergen]

Hey @Warkdev,

Thank you for reporting this! The S3 backend is managed by the AWS Provider team at HashiCorp, and this issue has been added to their triage queue. Thanks again!

[MohammedFairoz1]

To resolve the issue where Terraform was trying to connect to the default amazonaws.com endpoint instead of the custom endpoint specified in the configuration, setting use_path_style = true in the backend "s3" block solved the problem.

Explanation:

Issue: Terraform's endpoints configuration wasn't correctly resolving the custom endpoint, and it defaulted to AWS's amazonaws.com domain.

Solution: By enabling use_path_style = true, Terraform constructs the URL using the path-style format, which is compatible with custom or non-AWS S3-compatible services (like MinIO). This ensures the correct endpoint structure is used.

`terraform {

  backend "s3" {

    region                      = "eu-fr2"

    skip_region_validation      = true

    skip_credentials_validation = true

    skip_metadata_api_check     = true

    skip_requesting_account_id  = true

    bucket = "my-bucket-12345"

    endpoints = {

      s3 = "https://{s3_suffix}.{region}.{dnsSuffix}:{custom_port}"

    }

   use_path_style = true

  }

}`

This configuration allows Terraform to connect to the custom S3 endpoint instead of defaulting to amazonaws.com

[Warkdev]

Thank you for the update. I'll try it out. Confusing at first to see that endpoint and endpoints.s3 are not a 1-to-1 replacement :-)

[MohammedFairoz1]

You're welcome! I understand the confusion—it’s not immediately obvious that endpoints.s3 behaves differently, especially when transitioning from the deprecated endpoint parameter. Enabling use_path_style = true ensures compatibility with custom S3-like endpoints by enforcing the correct URL structure.

Feel free to reach out if you encounter any issues while testing it out. I'd be happy to help!