这是indexloc提供的服务,不要输入任何密码
Skip to content

[Perf Improvement] Skip returning rows on streaming subscription queries when there is no data #9994

@vincentjames501

Description

@vincentjames501

Is your proposal related to a problem?

Yes. We have ~10k active streaming subscriptions in our system at any given time. The vast majority of the polling requests do not have any new data past the current cursor. New events arrive maybe once every 10 mins or so on average. Because Hasura returns 1 row no matter what per unique stream, Postgres is essentially sending back ~10k rows of empty data per second which eats CPU/Network IO.

Describe the solution you'd like

Rather than returning "empty" rows like this in the streaming subscription query:

"result_id","result","cursor"
"893640e9-2088-4d30-a8ec-5eaf1aee6105","{""hasura_demo_events_stream"" : []}","{""id"" : null}"

If these were simply filtered out in the DB query and Hasura treated the missing row as effectively no data/cursor update which could save TONS of IO/CPU. In our case we'd go from sending back ~10k rows/s to likely only 1-5 rows/s max. I suspect many streaming subscriptions may have a similar access pattern.

Describe alternatives you've considered

Not aware of any alternatives.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions