-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Description
Is your proposal related to a problem?
Yes. We have ~10k active streaming subscriptions in our system at any given time. The vast majority of the polling requests do not have any new data past the current cursor. New events arrive maybe once every 10 mins or so on average. Because Hasura returns 1 row no matter what per unique stream, Postgres is essentially sending back ~10k rows of empty data per second which eats CPU/Network IO.
Describe the solution you'd like
Rather than returning "empty" rows like this in the streaming subscription query:
"result_id","result","cursor"
"893640e9-2088-4d30-a8ec-5eaf1aee6105","{""hasura_demo_events_stream"" : []}","{""id"" : null}"
If these were simply filtered out in the DB query and Hasura treated the missing row as effectively no data/cursor update which could save TONS of IO/CPU. In our case we'd go from sending back ~10k rows/s to likely only 1-5 rows/s max. I suspect many streaming subscriptions may have a similar access pattern.
Describe alternatives you've considered
Not aware of any alternatives.