Implement fsevents #1419

gsoltis · 2022-06-17T21:19:09Z

Still WIP, but testing w/ FSEvents now works.

Remaining:

Clean up naming and package structure. It's possible we want to break out a backend package or something
Verify behavior on remaining platforms (should be unchanged, but needs verification)
Documentation on code structure

vercel · 2022-06-17T21:19:12Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
turbo-site	✅ Ready (Inspect)	Visit Preview	Jul 14, 2022 at 6:25PM (UTC)

nathanhammond

Part one: the stuff that isn't filewatcher.

nathanhammond · 2022-07-13T20:25:29Z

cli/Makefile

@@ -15,7 +15,7 @@ SRC_FILES = $(shell find . -name "*.go" | grep -v "_test.go")
 GENERATED_FILES = internal/turbodprotocol/turbod.pb.go internal/turbodprotocol/turbod_grpc.pb.go

 turbo: $(GENERATED_FILES) $(SRC_FILES) go.mod
-	CGO_ENABLED=0 go build $(GO_FLAGS) ./cmd/turbo
+	CGO_ENABLED=1 go build $(GO_FLAGS) ./cmd/turbo


We need to make this platform-dependent.

nathanhammond

All but the backends.

nathanhammond · 2022-07-13T20:46:14Z

cli/internal/filewatcher/filewatcher.go

@@ -24,16 +19,56 @@ var _ignores = []string{".git", "node_modules"}
 // 1) do not need synchronization
 // 2) should minimize the work they are doing when called, if possible
 type FileWatchClient interface {
-	OnFileWatchEvent(ev fsnotify.Event)
+	OnFileWatchEvent(ev Event)


I like that we're now controlling our own events.

nathanhammond · 2022-07-13T20:50:30Z

cli/internal/filewatcher/filewatcher.go

+
+const (
+	// FileAdded - this is a new file
+	FileAdded FileEvent = iota


https://github.com/uber-go/guide/blob/master/style.md#start-enums-at-one

nathanhammond · 2022-07-13T21:20:57Z

cli/internal/filewatcher/filewatcher.go

 // Start recursively adds all directories from the repo root, redacts the excluded ones,
 // then fires off a goroutine to respond to filesystem events
 func (fw *FileWatcher) Start() error {
-	if err := fw.watchRecursively(fw.repoRoot); err != nil {
+	if err := fw.backend.AddRoot(fw.repoRoot, fw.excludePattern); err != nil {


This is a really nice "understand the thing" ergonomics change. Yes, it was required to do what you did, but now low-level implementation details are hidden in a function call.

nathanhammond · 2022-07-13T22:15:38Z

cli/internal/filewatcher/filewatcher_test.go

@@ -76,13 +58,29 @@ func expectNoFilesystemEvent(t *testing.T, ch <-chan struct{}) {
 		} else {
 			t.Error("filewatching closed unexpectedly")
 		}
-	case <-time.After(100 * time.Millisecond):
+	case <-time.After(500 * time.Millisecond):


Glad that this is single-invocation.

nathanhammond · 2022-07-13T22:16:21Z

cli/internal/filewatcher/filewatcher_test.go

@@ -102,64 +100,54 @@ func TestFileWatching(t *testing.T) {
 	//     child/
 	//     sibling/

-	watcher, err := fsnotify.NewWatcher()
+	watcher, err := GetPlatformSpecificBackend(logger)
 	assert.NilError(t, err, "NewWatcher")


Some renaming in this area.

nathanhammond · 2022-07-13T22:20:48Z

cli/internal/filewatcher/filewatcher_test.go

-	got := c.createEvents[len(c.createEvents)-1]
-	c.mu.Unlock()
-	assert.DeepEqual(t, got, expectedEvent)
-	// Windows doesn't watch individual files, only directories


This is a nice smoothing-over of the API.

nathanhammond · 2022-07-13T22:27:41Z

cli/internal/filewatcher/backend.go

+type watchAddMode int
+
+const (
+	dontSynthesizeEvents watchAddMode = iota


Not going to iota + 1 since the values here map well to true/false.

gsoltis force-pushed the gsoltis/cookies branch from 5fe6d41 to 4b8d071 Compare June 23, 2022 17:29

gsoltis force-pushed the gsoltis/fsevents branch from 22254fd to 1f66783 Compare June 23, 2022 17:31

gsoltis force-pushed the gsoltis/cookies branch from 4b8d071 to 2a58625 Compare June 27, 2022 15:48

gsoltis force-pushed the gsoltis/fsevents branch from 4c5cf91 to f9afcc8 Compare June 27, 2022 15:49

gsoltis force-pushed the gsoltis/cookies branch from 2a58625 to 9e1b1e1 Compare June 27, 2022 16:43

gsoltis force-pushed the gsoltis/fsevents branch from f0014f9 to a77b176 Compare June 27, 2022 16:43

gsoltis force-pushed the gsoltis/cookies branch from 9e1b1e1 to 61cd370 Compare June 27, 2022 19:09

gsoltis force-pushed the gsoltis/fsevents branch from e54ba2b to 6f09f04 Compare June 27, 2022 19:14

Base automatically changed from gsoltis/cookies to main July 6, 2022 21:54

gsoltis force-pushed the gsoltis/fsevents branch from 6f09f04 to fb0f532 Compare July 6, 2022 21:55

gsoltis marked this pull request as ready for review July 6, 2022 21:55

gsoltis requested review from jaredpalmer, gaspar09, nathanhammond and tknickman as code owners July 6, 2022 21:55

Greg Soltis added 7 commits July 13, 2022 16:24

Implement fsevents

41a95c5

Start non-darwin platform debugging

4cfe5e0

Comments on AddRoot

5dd110f

Add CGO back, synthesize events in recursive watch

cdfd7a9

Delete unused helper interface

cb70d0c

Renaming and comments

1ac8af0

Lint fix

2d2f3d1

nathanhammond force-pushed the gsoltis/fsevents branch from fb0f532 to 2d2f3d1 Compare July 13, 2022 20:24

nathanhammond reviewed Jul 13, 2022

View reviewed changes

make turbo works on all platforms

ca8f873

nathanhammond reviewed Jul 13, 2022

View reviewed changes

nathanhammond added 3 commits July 13, 2022 18:35

Some cleanup.

b04b0e5

Rework manual locking to use immediate deferral pattern.

5521c09

Document and explain the path processing.

48d3aa7

Merge branch 'main' into gsoltis/fsevents

c5b2438

vercel bot deployed to Preview July 14, 2022 18:25 View deployment

tknickman approved these changes Jul 14, 2022

View reviewed changes

kodiakhq bot merged commit 27ae70b into main Jul 14, 2022

kodiakhq bot deleted the gsoltis/fsevents branch July 14, 2022 18:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implement fsevents #1419

Implement fsevents #1419

Uh oh!

gsoltis commented Jun 17, 2022 •

edited

Loading

Uh oh!

vercel bot commented Jun 17, 2022 •

edited

Loading

Uh oh!

nathanhammond left a comment

Uh oh!

nathanhammond Jul 13, 2022

Uh oh!

nathanhammond left a comment

Uh oh!

nathanhammond Jul 13, 2022

Uh oh!

nathanhammond Jul 13, 2022

Uh oh!

nathanhammond Jul 13, 2022

Uh oh!

nathanhammond Jul 13, 2022

Uh oh!

nathanhammond Jul 13, 2022

Uh oh!

nathanhammond Jul 13, 2022

Uh oh!

nathanhammond Jul 13, 2022

Uh oh!

Uh oh!

Implement fsevents #1419

Implement fsevents #1419

Uh oh!

Conversation

gsoltis commented Jun 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Jun 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nathanhammond left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nathanhammond left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gsoltis commented Jun 17, 2022 •

edited

Loading

vercel bot commented Jun 17, 2022 •

edited

Loading