这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@ZIMkaRU
Copy link
Member

@ZIMkaRU ZIMkaRU commented Dec 1, 2022

This PR provides a new sync core to use temp tables during sync processing. The main idea is to put data fetched from the BFX api_v2 to temp tables to have consistent data in the main tables and allow users to request reports from all sections any time even sync is going on, except the first sync when the DB is empty. When the sync is finished and temp tables are filled all data will be moved to the main tables in one atomic transaction thus users don't have to experience transitional processes. Basic changes:

  • Implements new sync core to use temp tables
  • Improves transaction flow to fix the database is locked issue
  • Improves handling ENET errors from the api_v2, adds fetching data retries
  • Adds DB migration v31
  • Adds create and update timestamps to service tables for easier debugging
  • Significantly reduces the number of requests to the api_v2, which increases the speed of the sync
  • Fixes WebSocket timeout connection issue
  • Reduces pragma analysis_limit to 400 for performance
  • Uses inner max request limit to api_v2 for syncing data to speed up
  • Refactors the code, makes some optimizations, removes redundant methods

Copy link
Contributor

@ezewer ezewer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants