这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@tolgaozen
Copy link
Member

@tolgaozen tolgaozen commented Sep 9, 2025

Summary by CodeRabbit

  • New Features

    • None.
  • Bug Fixes

    • Improved PostgreSQL repair robustness by safely handling missing/null transaction IDs and preventing counter regressions.
  • Tests

    • Added comprehensive integration tests for the repair workflow using real PostgreSQL containers.
    • Streamlined test database setup across suites; removed some redundant error-path tests.
  • Refactor

    • Consolidated test database utilities into a shared test helper for consistency.
  • Chores

    • Minor naming and documentation cleanups related to the repair logic.

@coderabbitai
Copy link

coderabbitai bot commented Sep 9, 2025

Walkthrough

Tests across internal/storage/postgres now use pkg/testinstance.PostgresDB instead of internal/storage/postgres/instance.PostgresDB. Some error-path tests were removed in bundleReader_test.go and gc/gc_test.go. Postgres repair helper was renamed and adjusted to COALESCE NULLs. A comprehensive repair_test.go suite was added under pkg/database/postgres. pkg/testinstance package name was corrected.

Changes

Cohort / File(s) Summary
Test DB factory migration
internal/storage/postgres/bundleWriter_test.go, .../dataReader_test.go, .../dataWriter_test.go, .../schemaReader_test.go, .../schemaWriter_test.go, .../tenantReader_test.go, .../tenantWriter_test.go, .../utils/version_test.go, .../watch_test.go, internal/storage/postgres/gc/options_test.go
Replace internal/storage/postgres/instance.PostgresDB with github.com/Permify/permify/pkg/testinstance.PostgresDB; update imports; no test logic changes.
Error-path test pruning + factory swap
internal/storage/postgres/bundleReader_test.go, internal/storage/postgres/gc/gc_test.go
Switch to testinstance.PostgresDB and remove several error-handling tests; retain limited failure cases.
Repair internals
pkg/database/postgres/repair.go
Rename getCurrentPostgreSQLXID → getCurrentPostgreXID; use COALESCE in XID queries; update call site in Repair.
Repair integration tests
pkg/database/postgres/repair_test.go
Add extensive Ginkgo/Gomega tests using TestContainers covering defaults, XID retrieval, dry-run/actual repair, and migrations.
Test helper package rename
pkg/testinstance/postgres.go
Change package declaration from instance to testinstance.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Tester
  participant Repair as Postgres.Repair
  participant DB as PostgreSQL

  Tester->>Repair: Repair(ctx, cfg)
  Repair->>DB: SELECT COALESCE(pg_current_xact_id()::text,'0')::bigint
  DB-->>Repair: currentXID
  Repair->>DB: SELECT COALESCE(MAX(id)::text,'0')::bigint FROM transactions
  DB-->>Repair: maxRefXID or error
  alt dry-run
    Repair-->>Tester: RepairResult (no writes)
  else actual run
    Repair->>DB: UPDATE/INSERT fix transactions (as needed)
    DB-->>Repair: rows affected
    Repair-->>Tester: RepairResult (fixed count, XID bounds)
  end
  note over Repair,DB: COALESCE prevents NULL → 0 regressions
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • tolgaOzen

Pre-merge checks (2 passed, 1 warning)

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Description Check ⚠️ Warning No pull request description was provided to articulate the goals or summarize the changes, leaving reviewers without context beyond the title and diff. Add a descriptive summary that outlines the scope of the refactoring, the helper renaming in pkg/database/postgres/repair.go, and the addition of the repair test suite to give reviewers clear context for all changes.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The title succinctly highlights the primary change—refactoring test setup and imports in the internal/storage/postgres package—using clear and specific wording without unnecessary detail, which aligns with the bulk of the modifications in this pull request.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

Poem

A whisk of whiskers, a hop through the logs,
I nibbled on tests, moved DBs from the bogs.
COALESCE for crumbs when XIDs are shy,
New repairs binky—oh my, oh my!
With containers afloat and queries so neat,
This bunny stamps “tested” with thumpity feet. 🐰🛠️

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

  • Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
  • Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix-postgres-schemaReader-test

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
pkg/database/postgres/repair.go (1)

124-129: Measure the right “referenced XID”: use commit_tx_id/created_tx_id, not the row id.

MAX(id) may diverge from the actual XID columns and produce incorrect advancement decisions. Query the greatest of commit_tx_id and created_tx_id.

Apply this diff:

-func (p *Postgres) getMaxReferencedXID(ctx context.Context) (uint64, error) {
-    query := "SELECT COALESCE(MAX(id)::text, '0')::bigint FROM transactions"
+func (p *Postgres) getMaxReferencedXID(ctx context.Context) (uint64, error) {
+    // Use the highest XID referenced by data, not the surrogate key.
+    query := `
+        SELECT COALESCE(
+                 GREATEST(
+                   COALESCE(MAX(commit_tx_id), 0),
+                   COALESCE(MAX(created_tx_id), 0)
+                 )::text,
+                 '0'
+               )::bigint
+        FROM transactions
+    `
     var x int64
     if err := p.ReadPool.QueryRow(ctx, query).Scan(&x); err != nil {
         return 0, err
     }
     return uint64(x), nil
 }
🧹 Nitpick comments (11)
pkg/testinstance/postgres.go (3)

44-48: Use the existing ctx for Stop/Start

Minor clean-up for consistency.

-err = postgres.Stop(context.Background(), &stopTimeout)
+err = postgres.Stop(ctx, &stopTimeout)

-err = postgres.Start(context.Background())
+err = postgres.Start(ctx)

58-59: MappedPort should include protocol

Be explicit with “/tcp” to match the exposed port and avoid surprises on different testcontainers/go versions.

-port, err := postgres.MappedPort(ctx, "5432")
+port, err := postgres.MappedPort(ctx, "5432/tcp")

25-37: Add test-scoped container cleanup

Terminate the container via Ginkgo’s DeferCleanup to prevent orphaned containers when tests fail or panic.

 import (
+   ginkgo "github.com/onsi/ginkgo/v2"
 )

 postgres, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
   ...
 })
 Expect(err).ShouldNot(HaveOccurred())
+
+// Ensure container is torn down after the test scope
+ginkgo.DeferCleanup(func() {
+  _ = postgres.Terminate(ctx)
+})
internal/storage/postgres/watch_test.go (1)

59-61: Reduce flakiness by replacing sleeps with Eventually/Receive

Sleeping before writes and using long timeouts can cause flaky tests. Prefer Eventually on channels.

- time.Sleep(100 * time.Millisecond)
+ // Wait until watcher emits a change or error
+ Eventually(changes, 10*time.Second, 100*time.Millisecond).Should(Receive())

Also applies to: 94-100

internal/storage/postgres/dataReader_test.go (1)

73-79: Avoid shadowing the imported package name and reduce timestamp flakiness.

  • The local var token shadows the imported package token; rename for clarity.
  • 2ms sleep is tight and may flake under load; bump slightly.

Apply this diff:

-            token, err := dataWriter.Write(ctx, "t1", tuples, attributes)
+            snap, err := dataWriter.Write(ctx, "t1", tuples, attributes)
             Expect(err).ShouldNot(HaveOccurred())

-            mostRecentSnapshot = token
+            mostRecentSnapshot = snap

-            time.Sleep(time.Millisecond * 2)
+            time.Sleep(10 * time.Millisecond)
internal/storage/postgres/utils/version_test.go (1)

92-101: Consider limiting multi-version spins to keep CI fast.

Spinning 13/14/15/16 sequentially can be heavy. Optionally gate via a Ginkgo label or sample a single version when POSTGRES_VERSION is unset.

pkg/database/postgres/repair.go (2)

46-63: Populate result.Duration to reflect actual run time.

You already expose Duration; set it.

Apply this diff (and add time import):

 import (
     "context"
     "fmt"
     "log/slog"
+    "time"
 )
@@
 func (p *Postgres) Repair(ctx context.Context, config *RepairConfig) (*RepairResult, error) {
+    started := time.Now()
     if config == nil {
         config = DefaultRepairConfig()
     }
@@
-    return result, nil
+    result.Duration = time.Since(started).String()
+    return result, nil
 }

16-23: Config has unused fields (MaxRetries, RetryDelay).

Either wire them into advanceXIDCounterByDelta with simple retry/backoff or drop them to avoid dead config.

internal/storage/postgres/gc/gc_test.go (1)

52-53: Speed up tests by shrinking GC window.

A 5s window + sleeps slows CI. Consider a smaller window and matching sleep.

Example diff:

-        Window(5*time.Second),
+        Window(200*time.Millisecond),
@@
-            time.Sleep(5 * time.Second) // Pause for 5 seconds
+            time.Sleep(250 * time.Millisecond) // window + small buffer
@@
-            time.Sleep(5 * time.Second) // Pause for 5 seconds
+            time.Sleep(250 * time.Millisecond) // window + small buffer

Also applies to: 180-182, 305-307

pkg/database/postgres/repair_test.go (2)

70-78: Make cleanup assertions explicit.

Checking Terminate/Close results helps surface cleanup failures.

Apply this diff:

-        if db != nil {
-            db.Close()
-        }
-        if container != nil {
-            ctx := context.Background()
-            container.Terminate(ctx)
-        }
+        if db != nil {
+            Expect(db.Close()).ShouldNot(HaveOccurred())
+        }
+        if container != nil {
+            ctx := context.Background()
+            Expect(container.Terminate(ctx)).ShouldNot(HaveOccurred())
+        }

181-189: Tiny nit: multi-line VALUES indentation.

Purely stylistic; consider compact formatting to reduce vertical space.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 05355b7 and 03deaf5.

📒 Files selected for processing (15)
  • internal/storage/postgres/bundleReader_test.go (2 hunks)
  • internal/storage/postgres/bundleWriter_test.go (2 hunks)
  • internal/storage/postgres/dataReader_test.go (2 hunks)
  • internal/storage/postgres/dataWriter_test.go (2 hunks)
  • internal/storage/postgres/gc/gc_test.go (3 hunks)
  • internal/storage/postgres/gc/options_test.go (1 hunks)
  • internal/storage/postgres/schemaReader_test.go (2 hunks)
  • internal/storage/postgres/schemaWriter_test.go (2 hunks)
  • internal/storage/postgres/tenantReader_test.go (2 hunks)
  • internal/storage/postgres/tenantWriter_test.go (2 hunks)
  • internal/storage/postgres/utils/version_test.go (4 hunks)
  • internal/storage/postgres/watch_test.go (2 hunks)
  • pkg/database/postgres/repair.go (3 hunks)
  • pkg/database/postgres/repair_test.go (1 hunks)
  • pkg/testinstance/postgres.go (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (14)
internal/storage/postgres/schemaWriter_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/tenantWriter_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/gc/gc_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/bundleWriter_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/schemaReader_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/dataWriter_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/tenantReader_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/watch_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
pkg/database/postgres/repair_test.go (2)
pkg/database/postgres/repair.go (3)
  • DefaultRepairConfig (26-34)
  • RepairConfig (17-23)
  • RepairResult (37-41)
pkg/database/postgres/postgres.go (1)
  • Postgres (22-35)
internal/storage/postgres/gc/options_test.go (2)
pkg/database/postgres/postgres.go (1)
  • Postgres (22-35)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/utils/version_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
internal/storage/postgres/bundleReader_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
pkg/database/postgres/repair.go (1)
pkg/database/postgres/postgres.go (1)
  • Postgres (22-35)
internal/storage/postgres/dataReader_test.go (1)
pkg/testinstance/postgres.go (1)
  • PostgresDB (20-89)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Analyze (go)
  • GitHub Check: Test with Coverage
  • GitHub Check: Test with Coverage
🔇 Additional comments (24)
pkg/testinstance/postgres.go (2)

1-1: Package rename to testinstance — looks good

Consistent with the import changes across tests.


1-1: No lingering instance imports or usages detected
Repo-wide search for internal/storage/postgres/instance and instance.PostgresDB( returned no matches—this concern is cleared.

internal/storage/postgres/watch_test.go (1)

16-16: Switch to testinstance.PostgresDB — OK

Import and initializer changes align with the new helper.

Also applies to: 32-32

internal/storage/postgres/bundleWriter_test.go (1)

14-14: Switched to testinstance — LGTM

Import and DB factory updates look correct.

Also applies to: 29-29

internal/storage/postgres/schemaWriter_test.go (1)

16-16: Switched to testinstance — LGTM

Consistent with the shared test DB helper.

Also applies to: 31-31

internal/storage/postgres/schemaReader_test.go (1)

17-17: Switched to testinstance — LGTM

Setup uses the new helper as intended.

Also applies to: 32-32

internal/storage/postgres/tenantWriter_test.go (1)

13-13: Switched to testinstance — LGTM

No issues spotted with the migration here.

Also applies to: 27-27

internal/storage/postgres/tenantReader_test.go (1)

12-12: Switched to testinstance — LGTM

Import and initializer changes are correct.

Also applies to: 27-27

internal/storage/postgres/gc/options_test.go (1)

10-10: Switched to testinstance — LGTM

Casting to *PQDatabase.Postgres remains valid.

Also applies to: 18-18

internal/storage/postgres/dataReader_test.go (2)

15-16: Switch to shared test DB helper looks good.

Importing github.com/Permify/permify/pkg/testinstance aligns with the refactor.


32-35: DB provisioning via testinstance is correct.

Using testinstance.PostgresDB(version) keeps tests consistent with the new helper.

internal/storage/postgres/dataWriter_test.go (2)

16-16: Import change to testinstance is correct.


34-39: DB initialization via testinstance is correct.

Constructing writers/readers from db.(*PQDatabase.Postgres) remains valid.

internal/storage/postgres/utils/version_test.go (2)

12-12: Import change to testinstance is correct.


27-30: Consistent use of testinstance.PostgresDB across setups.

All call sites updated coherently.

Also applies to: 102-106, 139-142

internal/storage/postgres/bundleReader_test.go (2)

14-15: Import change to testinstance is correct.


29-32: DB initialization via testinstance is correct.

pkg/database/postgres/repair.go (3)

65-66: Rename to getCurrentPostgreXID is fine.

Call site updated consistently.


113-120: Safer NULL handling for current XID looks good.

COALESCE(...)::bigint avoids NULL-related scan errors.


150-156: Built-in min is supported (go 1.24.6)
go.mod specifies go 1.24.6, so the Go 1.21+ min function is available and no change is required.

internal/storage/postgres/gc/gc_test.go (2)

23-25: Import change to testinstance is correct.


49-54: DB initialization via testinstance is correct.

pkg/database/postgres/repair_test.go (2)

81-107: Dry-run behavior assertions look good.

Validates error capture while keeping top-level err nil.


165-233: Good integration coverage for real advancement.

End-to-end flow covers insert → detect → dry-run → actual repair.

@tolgaozen tolgaozen merged commit 83d43ab into master Sep 9, 2025
12 checks passed
@tolgaozen tolgaozen deleted the fix-postgres-schemaReader-test branch September 9, 2025 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants