Storage Insights Datasets terraform support #14564

sameer-google · 2025-07-18T11:25:05Z

Adding terraform support for Storage Insights Datasets.

`google_storage_insights_dataset_config`

modular-magician · 2025-07-18T11:37:54Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 9 files changed, 2617 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 9 files changed, 2617 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 113 insertions(+))

Missing doc report (experimental)

The following data sources are missing documents:

google_storage_insights_dataset_config, expect file /website/docs/d/storage_insights_dataset_config.html.markdown to exist.

modular-magician · 2025-07-18T11:40:43Z

Tests analytics

Total tests: 10
Passed tests: 3
Skipped tests: 0
Affected tests: 7

Click here to see the affected service packages

#### Action taken

Found 7 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

Get to know how VCR tests work

modular-magician · 2025-07-18T11:44:43Z

🟢 Tests passed during RECORDING mode:
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_folder_to_org [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_folder_to_project [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_org_to_folder [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_org_to_project [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_project_to_folder [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_project_to_org [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

🟢 All tests passed!

View the build log or the debug log for each test

modular-magician · 2025-07-18T12:09:07Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 9 files changed, 2617 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 9 files changed, 2617 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 113 insertions(+))

Missing doc report (experimental)

The following data sources are missing documents:

google_storage_insights_dataset_config, expect file /website/docs/d/storage_insights_dataset_config.html.markdown to exist.

modular-magician · 2025-07-18T12:12:24Z

Tests analytics

Total tests: 10
Passed tests: 10
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

🟢 All tests passed!

View the build log

github-actions · 2025-07-21T07:01:33Z

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

Googlers: For automatic test runs see go/terraform-auto-test-runs.

@shuyama1, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

modular-magician · 2025-07-21T07:14:53Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 9 files changed, 2624 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 9 files changed, 2624 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 113 insertions(+))

Missing doc report (experimental)

The following data sources are missing documents:

google_storage_insights_dataset_config, expect file /website/docs/d/storage_insights_dataset_config.html.markdown to exist.

modular-magician · 2025-07-21T07:19:08Z

Tests analytics

Total tests: 10
Passed tests: 10
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

🟢 All tests passed!

View the build log

shuyama1

Could you please add the corresponding documentation for the data source? Thanks! https://googlecloudplatform.github.io/magic-modules/develop/add-handwritten-datasource/#add-documentation

sameer-google · 2025-07-22T05:42:51Z

Done, thank you

modular-magician · 2025-07-22T05:55:14Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 10 files changed, 2674 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 10 files changed, 2674 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 113 insertions(+))

modular-magician · 2025-07-22T05:58:18Z

Tests analytics

Total tests: 10
Passed tests: 10
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

🟢 All tests passed!

View the build log

kautikdk

I would recommend creating a GitHub issue in the Public Issue tracker here: https://github.com/hashicorp/terraform-provider-google/issues for the reference so that someone else won't start working on the same resource as long as it is assigned to you.

Another suggestion would be to follow the update test guidelines mentioned here: https://googlecloudplatform.github.io/magic-modules/test/test/#add-an-update-test

mmv1/products/storageinsights/DatasetConfig.yaml

mmv1/templates/terraform/post_create/storage_insights_dataset_config.go.tmpl

mmv1/templates/terraform/pre_update/storage_insights_dataset_config.go.tmpl

...rd_party/terraform/services/storageinsights/resource_storage_insights_dataset_config_test.go

modular-magician · 2025-07-22T18:06:33Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 10 files changed, 2651 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 10 files changed, 2651 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 112 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_storage_insights_dataset_config (13 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_storage_insights_dataset_config" "primary" {
  organization_number = # value needed
}

modular-magician · 2025-07-22T18:09:47Z

Tests analytics

Total tests: 10
Passed tests: 3
Skipped tests: 0
Affected tests: 7

Click here to see the affected service packages

#### Action taken

Found 7 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

Get to know how VCR tests work

modular-magician · 2025-07-22T18:13:27Z

🟢 Tests passed during RECORDING mode:
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_folder_to_org [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_folder_to_project [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_org_to_folder [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_org_to_project [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_project_to_folder [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample_update_project_to_org [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

🟢 All tests passed!

View the build log or the debug log for each test

modular-magician · 2025-07-22T18:54:03Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 10 files changed, 2661 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 10 files changed, 2661 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 112 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_storage_insights_dataset_config (13 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_storage_insights_dataset_config" "primary" {
  organization_number = # value needed
}

kautikdk

Overall looks good to me!
Will approve once we get clarity on the project field from the offline discussion.

mmv1/templates/terraform/pre_update/storage_insights_dataset_config.go.tmpl

...rd_party/terraform/services/storageinsights/resource_storage_insights_dataset_config_test.go

kautikdk · 2025-07-23T13:39:44Z

Hi @shuyama1, can you please start review as the feature is planned for release in the next week if possible?

modular-magician · 2025-07-23T13:44:50Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 10 files changed, 2642 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 10 files changed, 2642 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 112 insertions(+))

modular-magician · 2025-07-23T13:48:06Z

Tests analytics

Total tests: 7
Passed tests: 7
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

🟢 All tests passed!

View the build log

shuyama1 · 2025-07-23T22:37:14Z

/gcbrun

modular-magician · 2025-07-23T22:51:39Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 10 files changed, 2642 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 10 files changed, 2642 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 4 files changed, 112 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_storage_insights_dataset_config (8 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_storage_insights_dataset_config" "primary" {
  description = # value needed
  exclude_cloud_storage_buckets {
    cloud_storage_buckets {
      bucket_name         = # value needed
      bucket_prefix_regex = # value needed
    }
  }
  exclude_cloud_storage_locations {
    locations = # value needed
  }
  include_cloud_storage_buckets {
    cloud_storage_buckets {
      bucket_name         = # value needed
      bucket_prefix_regex = # value needed
    }
  }
  include_cloud_storage_locations {
    locations = # value needed
  }
  include_newly_created_buckets = # value needed
  link_dataset                  = # value needed
  organization_number           = # value needed
}

modular-magician · 2025-07-23T22:54:31Z

Non-exercised tests

🔴 Tests were added that are skipped in VCR:

TestAccStorageInsightsDatasetConfigExample_update_scope
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExample

Tests analytics

Total tests: 0
Passed tests: 0
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

🔴 Errors occurred during REPLAYING mode. Please fix them to complete your PR.

View the build log

...rd_party/terraform/services/storageinsights/resource_storage_insights_dataset_config_test.go

...hird_party/terraform/services/storageinsights/data_source_storage_insights_dataset_config.go

mmv1/products/storageinsights/DatasetConfig.yaml

...rd_party/terraform/services/storageinsights/resource_storage_insights_dataset_config_test.go

mmv1/templates/terraform/post_create/storage_insights_dataset_config.go.tmpl

…anged documentations

sameer-google · 2025-07-24T07:03:32Z

@shuyama1 , @kautikdk , addressed all the comments, you can proceed for review

modular-magician · 2025-07-24T07:13:57Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 10 files changed, 2740 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 10 files changed, 2740 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 8 files changed, 247 insertions(+))

modular-magician · 2025-07-24T07:16:57Z

Tests analytics

Total tests: 8
Passed tests: 3
Skipped tests: 0
Affected tests: 5

Click here to see the affected service packages

#### Action taken

Found 5 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

Get to know how VCR tests work

modular-magician · 2025-07-24T07:21:09Z

🟢 Tests passed during RECORDING mode:
TestAccStorageInsightsDatasetConfigExample_update_filters [Debug log]
TestAccStorageInsightsDatasetConfigExample_update_link [Debug log]
TestAccStorageInsightsDatasetConfigExample_update_scope [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigExcludesExample [Debug log]
TestAccStorageInsightsDatasetConfig_storageInsightsDatasetConfigIncludesExample [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

🟢 All tests passed!

View the build log or the debug log for each test

modular-magician · 2025-07-24T08:44:51Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 8 files changed, 2632 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 8 files changed, 2632 insertions(+), 2 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 391 insertions(+))
Open in Cloud Shell: Diff ( 8 files changed, 247 insertions(+))

modular-magician · 2025-07-24T08:48:16Z

Tests analytics

Total tests: 8
Passed tests: 8
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

🟢 All tests passed!

View the build log

sameer-google · 2025-07-24T08:51:36Z

/gcbrun

kautikdk

LGTM as all the comments have been addressed and project field concern is now resolved.

shuyama1

Thank you!

Storage Insights Datasets initial terraform support

d75fd4c

modular-magician added the service/storage label Jul 18, 2025

modular-magician requested a review from kautikdk July 18, 2025 11:38

fixed lint

52ccf4f

updated documentation

8a833e2

sameer-google marked this pull request as ready for review July 21, 2025 07:00

github-actions bot requested a review from shuyama1 July 21, 2025 07:01

sameer-google changed the title ~~Storage Insights Datasets initial terraform support~~ Storage Insights Datasets terraform support Jul 21, 2025

shuyama1 reviewed Jul 21, 2025

View reviewed changes

Added documentation for datasource

661c9bb

sameer-google requested a review from shuyama1 July 22, 2025 05:42

Update with modified organization number behaviour

7fec508

kautikdk reviewed Jul 22, 2025

View reviewed changes

modular-magician requested a review from kautikdk July 22, 2025 18:06

update resource yaml with extra notes

afe4b9e

Refactor update tests

9de2b8f

kautikdk reviewed Jul 23, 2025

View reviewed changes

renamed the update tests

12cba00

modular-magician requested a review from kautikdk July 23, 2025 13:44

shuyama1 reviewed Jul 24, 2025

View reviewed changes

Fixed update_test names, added extra examples, fixed indentations, ch…

1c78ea3

…anged documentations

github-actions bot requested a review from shuyama1 July 24, 2025 07:01

Seperated Datasource

b77dd0f

kautikdk approved these changes Jul 24, 2025

View reviewed changes

shuyama1 approved these changes Jul 24, 2025

View reviewed changes

shuyama1 added this pull request to the merge queue Jul 24, 2025

Merged via the queue into GoogleCloudPlatform:main with commit 2f55f02 Jul 24, 2025
28 checks passed

Storage Insights Datasets terraform support #14564

Storage Insights Datasets terraform support #14564

Uh oh!

Conversation

sameer-google commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

modular-magician commented Jul 18, 2025

Diff report

Missing doc report (experimental)

Uh oh!

modular-magician commented Jul 18, 2025

Tests analytics

Uh oh!

modular-magician commented Jul 18, 2025

Uh oh!

modular-magician commented Jul 18, 2025

Diff report

Missing doc report (experimental)

Uh oh!

modular-magician commented Jul 18, 2025

Tests analytics

Uh oh!

github-actions bot commented Jul 21, 2025

Uh oh!

modular-magician commented Jul 21, 2025

Diff report

Missing doc report (experimental)

Uh oh!

modular-magician commented Jul 21, 2025

Tests analytics

Uh oh!

shuyama1 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sameer-google commented Jul 22, 2025

Uh oh!

modular-magician commented Jul 22, 2025

Diff report

Uh oh!

modular-magician commented Jul 22, 2025

Tests analytics

Uh oh!

kautikdk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

modular-magician commented Jul 22, 2025

Diff report

Missing test report

Uh oh!

modular-magician commented Jul 22, 2025

Tests analytics

Uh oh!

modular-magician commented Jul 22, 2025

Uh oh!

modular-magician commented Jul 22, 2025

Diff report

Missing test report

Uh oh!

kautikdk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kautikdk commented Jul 23, 2025

Uh oh!

modular-magician commented Jul 23, 2025

Diff report

Uh oh!

modular-magician commented Jul 23, 2025

sameer-google commented Jul 18, 2025 •

edited

Loading

shuyama1 left a comment •

edited

Loading