UpdateAll.py: improve robustness in case of service failure. #66
Description
We have noted that the script UpdateAll.py does not handle failures of the service endpoint localconfig particularly well and fails due to invalid data. We suggest that that this script needs some revision in order to improve resilience to service problems of single nodes.
The original cause seems to be the following problem:
What we then observe on our side is the following. This is what we see in the logs of the script:
DEBUG:addRemote:.
DEBUG:addRemote:.
DEBUG:addRemote:Successfully removed ./routing-NOA.xml.bck
DEBUG:addRemote:Successfully renamed ./routing-NOA.xml to ./routing-NOA.xml.bck
DEBUG:addRemote:Successfully renamed ./routing-NOA.xml.download to ./routing-NOA.xml
DEBUG:addRoutes:Entering addRoutes(./routing-NOA.xml)
INFO:addRoutes:Overlaps between routes will NOT be allowed
Traceback (most recent call last):
File "./updateAll.py", line 487, in <module>
main()
File "./updateAll.py", line 483, in main
mergeRoutes('routing.xml', synchroList)
File "./updateAll.py", line 388, in mergeRoutes
routingTable=ptRT, allowOverlaps=allowOverlaps)
File "../routeutils/utils.py", line 106, in addRoutes
event, root = context.next()
File "<string>", line 91, in next
cElementTree.ParseError: syntax error: line 1, column 0
Apparently the received file is not a valid XML document:
sysop@eida:/srv/routing/1/data$ xmllint routing-NOA.xml
routing-NOA.xml:1: parser error : Start tag expected, '<' not found
Trying to connect to eida.gein.noa.gr ...
^
sysop@eida:/srv/routing/1/data$ head routing-NOA.xml
Trying to connect to eida.gein.noa.gr ...
Trying to connect to eida.gein.noa.gr ...
Trying to connect to eida.gein.noa.gr ...
Trying to connect to eida.gein.noa.gr ...
Trying to connect to eida.gein.noa.gr ...
Trying to connect to eida.gein.noa.gr ...
<?xml version="1.0" encoding="utf-8"?>
<ns0:routing xmlns:ns0="http://geofon.gfz-potsdam.de/ns/Routing/1.0/">
<ns0:route networkCode="CQ" stationCode="*" locationCode="*" streamCode="*">
<ns0:station address="http://eida.gein.noa.gr/fdsnws/station/1/query" priority="1" start="2013-01-01T00:00:00" end=""/>