-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Description
We're having this issue where Hasura cannot start at all, just because some remote schemas are not immediately available. In a distributed environment (or even Kubernetes) we can't always guarantee every "child" process to be running/ready, but that shouldn't prevent the "parent" process from running.
We would prefer Hasura to start anyway, and dynamically register remote schemas as they become available.
Very specifically, this is the error we are seeing before Hasura exits with an error:
time="2020-06-16T14:15:33Z" level=fatal msg="failed to apply metadata: cannot apply metadata on the database: failed parsing json: json: cannot unmarshal array into Go struct field HasuraError.internal of type hasuradb.SQLInternalError; response from API: {\"internal\":[{\"definition\":{\"definition\":{\"timeout_seconds\":60,\"url\":\"http://hasura-child:8081/v1/graphql\",\"forward_client_headers\":false},\"name\":\"online-test\",\"comment\":null},\"reason\":\"HTTP exception occurred while sending the request to http://hasura-child:8081/v1/graphql\",\"type\":\"remote_schema\"}],\"path\":\"$.args[1].args\",\"error\":\"cannot continue due to inconsistent metadata\",\"code\":\"unexpected\"}"
Replication is as simple as having a migrations image attempting to load metadata which contains an offline remote schema. Here's an example with a parent trying to load remote schemas from child1 and child2. child1 runs immediately, but child2 has a pg_sleep(121) to simulate a much longer bringup time.