CN112015534B - Configurable platform scheduling method, system and storage medium - Google Patents
Configurable platform scheduling method, system and storage medium Download PDFInfo
- Publication number
- CN112015534B CN112015534B CN202010879341.6A CN202010879341A CN112015534B CN 112015534 B CN112015534 B CN 112015534B CN 202010879341 A CN202010879341 A CN 202010879341A CN 112015534 B CN112015534 B CN 112015534B
- Authority
- CN
- China
- Prior art keywords
- task
- scheduling
- dispatching
- annotation
- tasks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明涉及大数据处理技术，揭露了一种配置化平台调度方法，方法包括对配置化平台的调度任务进行注解，并将调度任务和相应的注解信息关联存储于数据库sqlite中，并按照注解信息设定调度任务之间的依赖关系；对带有注解的调度任务进行微服务扫描并进行注册；对调度任务进行分配分片，确定服务器与分配分片后的调度任务间的关联关系；根据所述关联关系以及调度任务之间的依赖关系，确定调度任务的执行服务器及执行方式；根据调度任务的执行服务器及执行方式执行调度任务。本发明还涉及区块链技术，数据存储于区块链中，本发明使得配置化平台的调度系统将在性能、稳定性、易用性、扩展性上有很大的优势。
The present invention relates to big data processing technology, and discloses a configuration platform scheduling method, which includes annotating the scheduling tasks of the configuration platform, and associating and storing the scheduling tasks and corresponding annotation information in the database sqlite, and setting the dependency relationship between the scheduling tasks according to the annotation information; performing microservice scanning and registration on the scheduling tasks with annotations; allocating shards to the scheduling tasks, and determining the association relationship between the server and the scheduling tasks after the shards are allocated; determining the execution server and execution method of the scheduling tasks according to the association relationship and the dependency relationship between the scheduling tasks; and executing the scheduling tasks according to the execution server and execution method of the scheduling tasks. The present invention also relates to blockchain technology, and data is stored in the blockchain. The present invention enables the scheduling system of the configuration platform to have great advantages in performance, stability, ease of use, and scalability.
Description
Technical Field
The present invention relates to big data processing, and in particular, to a method, a system, and a storage medium for scheduling a configured platform.
Background
The scheduling system is a core infrastructure in a big data platform, and the scheduling mode adopted by the current scheduling system is a mode which simply depends on time scheduling, such as crontab. However, the data processing flow has a long dependence chain, so that the problems of unclear dependence among tasks and difficult searching of scheduling errors exist.
In the prior art, by adopting an elastic-job mode, the problem of unclear dependence among tasks is improved, the problem of difficulty in finding errors is solved, and the defects still exist are as follows:
1) The dependency support among tasks is insufficient;
2) The graphical management interface is not fully functional.
Therefore, there is a need for a configurable platform scheduling method with rich scheduling functions.
Disclosure of Invention
The invention provides a dispatching method, a dispatching system and a computer readable storage medium for a configuration platform, which mainly solve the problem of insufficient dependence support between dispatching tasks of the configuration platform.
The method comprises annotating dispatching tasks of a configuration platform, storing the dispatching tasks and corresponding annotation information in a lightweight database sqlite in an associated mode, setting dependency relations among the dispatching tasks according to the annotation information, carrying out micro-service scanning on the dispatching tasks with the annotations, registering the dispatching tasks in a registration center of the configuration platform according to the types of the dispatching tasks, distributing fragments to the dispatching tasks according to an elastic-job frame according to the number of servers in the registration center, determining association relations between the servers and the dispatching tasks after the fragments are distributed, determining an execution server and an execution mode of the dispatching tasks according to the association relations and the dependency relations among the dispatching tasks, and executing the dispatching tasks according to the execution server and the execution mode of the dispatching tasks.
Further, preferably, the annotation of the scheduling task comprises task node classification data, task state attribute data and task version attribute data, wherein the task state attribute is classified into running, waiting to run, suspending, completing or abnormal, and the task node classification comprises a father node task and a child node task.
Further, preferably, the step of storing the scheduling task and the corresponding annotation information in association with each other in the lightweight database sqlite further comprises adding a plurality of data burial points in a plurality of states on the original data burial points by using a database monitor of the configuration platform, and synchronizing all the data burial points to the database sqlite. .
Further, preferably, the scheduling task and the corresponding annotation information are stored in a lightweight database sqlite in an associated mode, corresponding log files are dynamically added to the scheduling task according to the task, the scheduling task is stored in a grading mode according to error, warn, info, debug states according to the log files, and the log files are rolled according to two dimensions of time and size.
Further, preferably, the data of the scheduled task is stored in a blockchain, and the scheduled task data stored in each lightweight database sqlite is synchronized to the database of the configuration platform so as to realize task state query of the scheduled task.
The invention further provides a configuration platform dispatching system which comprises a dispatching task annotating unit, a dispatching task registering unit, a dispatching task associating unit and a dispatching task executing unit, wherein the dispatching task annotating unit is used for annotating dispatching tasks of the configuration platform, associating and storing the dispatching tasks and corresponding annotation information in a lightweight database sqlite, setting dependency relations among the dispatching tasks according to the annotation information, the dispatching task registering unit is used for carrying out micro-service scanning on the dispatching tasks with the annotations and registering the dispatching tasks in a registration center of the configuration platform according to the types of the annotations of the dispatching tasks, the dispatching task associating unit is used for distributing fragments to the dispatching tasks according to an elastic-job frame, determining association relations between the servers and the dispatching tasks after the fragments are distributed, and the dispatching task executing unit is used for determining execution servers and execution modes of the dispatching tasks according to the association relations and the dependency relations among the dispatching tasks, and executing modes of the dispatching tasks.
Further, preferably, the data of the scheduling task is stored in a block chain, the scheduling task annotating unit comprises a scheduling task annotating module and a scheduling task storage module with annotation, the scheduling task annotating module is used for annotating the scheduling task of the configuration platform, the scheduling task annotating module comprises task node classification, task state attribute and task version attribute, the task state attribute is divided into running, to-be-run, pause, completion or abnormality, the task node classification comprises father node tasks and child node tasks, the scheduling task storage module with annotation is used for storing the scheduling task and corresponding annotation information in a lightweight database sqlite, and dependency relationship among the scheduling tasks is set according to the annotation information.
Further, preferably, the scheduling task storage module with annotation comprises a storage sub-module, a dependency relationship setting sub-module and a data embedding point synchronization sub-module, wherein the storage sub-module is used for storing the scheduling tasks and corresponding annotation information in a lightweight database sqlite, the dependency relationship setting sub-module is used for setting the dependency relationship among the scheduling tasks according to the annotation information, and the data embedding point synchronization sub-module is used for adding a plurality of data embedding points in a plurality of states on the original data embedding points by utilizing a database monitor of a configuration platform and synchronizing all the data embedding points to the database sqlite.
To achieve the above object, the present invention further provides an electronic device, which includes at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores a program executable by the at least one processor, and the program is executed by the at least one processor, so that the at least one processor can perform the configured platform scheduling method as described above.
In addition, to achieve the above object, the present invention further provides a computer readable storage medium storing a computer program, where the computer program, when executed by a processor, implements the steps of the above-mentioned configured platform scheduling method.
The dispatching method, the dispatching system, the electronic device and the computer readable storage medium for the configuration platform are characterized by annotating dispatching tasks of the configuration platform, storing the dispatching tasks and corresponding annotation information in a lightweight database sqlite in an associated mode, setting dependency relations among the dispatching tasks according to the annotation information, carrying out micro-service scanning on the dispatching tasks with the annotations, registering the dispatching tasks in a registration center of the configuration platform according to the types of the annotations of the dispatching tasks, distributing fragments to the dispatching tasks according to an elastic-job framework according to the number of servers in the registration center, determining association relations between the servers and the dispatching tasks after the fragments are distributed, judging an execution server and an execution mode of the dispatching tasks according to the association relations and the dependency relations among the dispatching tasks, and executing the dispatching tasks according to the execution server and the execution mode of the dispatching tasks. Enriches the functions of the collocation platform, and has the following beneficial effects:
1) The management interface provides starting, pausing, rerun and ending the existing tasks, and simultaneously provides dynamic addition of new tasks, so that usability is greatly improved;
2) The frame after encapsulation can complete the establishment of the task in an annotation mode, and only the data is concerned, so that the method is greatly helpful for the expansibility of the scheduling task;
3) By combining the advantages of the distributed data scheduling of the elastic-job, the scheduling system of the configuration platform has great advantages in performance, stability, usability and expansibility.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of a configurable platform scheduling method of the present invention;
FIG. 2 is a schematic diagram of a configuration platform scheduling system according to a preferred embodiment of the present invention;
FIG. 3 is a schematic diagram of an electronic device according to a preferred embodiment of the present invention;
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In order to improve the user coding efficiency, the invention provides a configuration platform scheduling method. FIG. 1 shows a flow chart of a preferred embodiment of the configurable platform scheduling method of the present invention. Referring to fig. 1, the method may be performed by an apparatus, which may be implemented in software and/or hardware.
It should be noted that, in the method for scheduling a configured platform according to the present invention, specifically, the method for scheduling a configured platform includes steps S110 to S150.
S110, annotating the dispatching tasks of the configuration platform, storing the dispatching tasks and the corresponding annotation information in a lightweight database sqlite in an associated mode, and setting the dependency relationship among the dispatching tasks according to the annotation information.
In a specific embodiment, the annotation of the scheduling task comprises task node classification data, task state attribute data and task version attribute data, wherein the task state attribute is classified into running, waiting to run, suspending, completing or abnormal, and the task node classification comprises a father node task and a child node task.
That is, each scheduling task is added with an annotation comprising node classification, task state and version number, and the task node classification is to add parent node task annotation and child node task annotation to increase the association (dependency) between tasks. The task state attribute data and the task version attribute data are historical states and versions of scheduled tasks stored through a lightweight database Sqlite, and task state information and version information are initialized, wherein in the specific implementation process, the Sqlite adopts a file mode to establish a new table for storing the version information.
It should be noted that, when the task server is online, the server information is automatically registered in the registry, and when the task server is offline, the server state is automatically updated. That is, adding two attributes of task state and task version to the task provides a basis for adding task dependency functions to the distributed task.
It should be noted that, the initial version number is 0, and the version number is incremented by 1 after execution. The state of the task is divided into running, waiting to run, suspending, finishing or abnormal.
The method comprises the following specific implementation scenes of RUNNING (1), RUNNING (2), STOP (3), RUNNING (4), ERROR (5) and exception, wherein the state of a task in the initial registration is READY, RUNNING is carried out in the execution process, the execution completion is OVER, the state and the version number of each task are registered in a zookeeper in the state change process, the initial version number is 0, and 1 is added once.
In a specific implementation process, the step of storing the scheduling task and the corresponding annotation information in the lightweight database sqlite in an associated manner further comprises adding a plurality of data embedded points in a plurality of states on the original data embedded points by using a database monitor of the configuration platform, and synchronizing all the data embedded points to the database sqlite.
The embedded point is a method for collecting data of application programs such as websites, apps or background. Through burying the point, the generation behavior of the user in the application can be collected, so that the user can be used for analyzing and optimizing the subsequent experience of the product, and data support can be provided for the operation of the product. The buried point corresponds to a task listener, whereas for Elastic-job, the listener is divided into a local listener and a distributed listener. The local monitor is only scheduled when the node executes own slicing, and each slicing task is scheduled. The local listener is defined by the ElasticJobListener interface. The local listener will execute when the job executes the local slicing task, if the job is split into 6 slices, the listener task will execute 6 times. The distributed monitor is executed once when the total task starts to execute and executed once when the total task ends to execute. Distributed listeners are also implemented on the basis of normal listeners.
For each task, the original data burial point is based on the original database listener of the elastic-job. The two attributes of the task state and the task version are added, so that a monitor and a buried point corresponding to the monitor are needed to be added, and further, the monitor can correspondingly control each state of the task.
The specific implementation process is that the related class is rewritten, and the process state data is added on the original data embedded point. Not only the original two states of start and end are included, but also the state of RUNNING (1, "in operation"), READY (2, "to be operated"), STOP (3, "pause"), OVER (4, "complete"), ERROR (5, "abnormal"), 5 states are changed.
In a specific embodiment, a scheduling task and corresponding annotation information are stored in a sqlite database in an associated mode, corresponding log files are dynamically added to the scheduling task according to the task, the scheduling task is stored in a grading mode according to error, warn, info, debug states according to the log files, and the log files are rolled according to two dimensions of time and size.
That is, in the implementation process, if A, B tasks exist, a registers the state and version of each node in the zookeeper during the running process. And the version information of the A task in the zookeeper is compared with the version information of the A task in the sqlite database before the B task is executed, the version information of the A task in the zookeeper is consistent with the version information of the B task stored in the sqlite database, and the B task can be normally executed only when the state information of the A task in the zookeeper is displayed as the OVER, and if not, the B task is suspended.
S120, performing microservice scanning on the scheduling task with the annotation, and registering in a registration center of the configuration platform according to the type of the annotation of the scheduling task.
Registration, i.e. registration of tasks with special notes, is described herein.
Specifically, the microservice scans all classes with special notes, completing the registration. The method comprises the steps of initializing attribute task names, the number of fragments, the fragment parameters, the execution plans and the dependency relations of notes during registration. The attribute tasks are selected and filled according to actual needs.
The development difficulty of the developer is reduced, so that the developer only needs to pay attention to the service, but does not need to pay attention to the framework.
The implementation scenario is exemplified as follows:
When a developer newly builds a scheduling task, the @ Job annotation is added, and meanwhile, the relevant attribute of the annotation is initialized. Job (jobName = "testJob", cron= "0/10
S130, distributing and slicing the scheduling tasks according to the number of servers in the registry and an elastic-job frame, and determining the association relationship between the servers and the scheduling tasks after distributing and slicing.
The task slicing mode is adopted. Splitting a task into n independent task items, and executing the distributed fragmented items in parallel by the distributed server.
In other words, the slicing condition of the task is determined according to the number of servers existing in the registration space, and the task is run in parallel or in series according to the plan. Namely, the situation of task allocation fragmentation is determined according to the number of instances in the naspace. This strategy relies on an elastic-job framework.
When the timing task is triggered, if the segmentation is needed again, the task can be executed only after the segmentation is finished by blocking the segmentation process of the main server. If the main server is off line in the slicing process, the main server is elected first, and slicing is performed. In order to maintain the stability of the operation, only the state of the score sheet is marked in the operation process, and the score sheet cannot be re-segmented. The fragmentation may only occur until the next task trigger. Every time the slicing is performed, the server IP sequence is performed, and the slicing result is ensured not to generate larger fluctuation. And the failover function is realized, unallocated fragments are actively captured after the execution of a certain server is finished, and available servers are actively searched for to execute tasks after the certain server is offline.
And S140, judging an execution server and an execution mode of the scheduling task according to the association relation and the dependency relation among the scheduling tasks.
Specifically, in step S130, one task is split into n independent task items, and according to the association relationship between the server and the scheduled task after the allocation of the shards, it is determined which server each independent task item is specifically executed by, so that the execution server of the scheduled task after the allocation of the shards can be determined through the association relationship between the server and the scheduled task after the allocation of the shards.
In addition, according to the dependency relationship between the scheduled tasks, that is, the dependency relationship between the scheduled tasks has been determined according to the task node classification (parent node task annotation and child node task annotation), that is, the dependency relationship between the scheduled tasks is clear, the association relationship between the scheduled tasks is also determined. Therefore, the execution server and the execution mode of the scheduled task are confirmed.
S150, executing the scheduling task according to the execution server and the execution mode of the scheduling task.
In a specific embodiment, the data of the scheduling task is stored in a blockchain, and the scheduling task data stored in each lightweight database sqlite is synchronized to the database of the configuration platform, so that the task state query of the scheduling task is realized.
Specifically, a log rolling function is added through logback, and the log rolling is a log update. The system comprises a logback-core, a logback-classification, a storage module and a logback-access, wherein the logback-core is a basic module of other two modules, the logback-classification is an improved version of log4j, meanwhile, the slf4j API is completely realized, so that you can conveniently replace the slf4j API with other log systems such as log4j or JDK14 Logging, and the access module and a Servlet container are integrated to provide a function of accessing a log through an Http. By adopting logback to increase the log rolling function, the performance is stable, and the running speed is faster.
The original task comprises an original starting state and an original ending state, two attributes of a task state and a task version are added, and the task state and the task version are changed into RUNNING (1, in operation), READY (2, in operation), STOP (3, pause), OVER (4, finish), ERROR (5, abnormal) and 5 states, so that the update mode of the log also needs to be changed.
Specifically, the corresponding log files are dynamically added to the tasks according to the tasks, the logs are rolled according to two dimensions of time and size, and the logs of each task are stored in a grading mode according to error, warn, info, debug.
According to the dependency relationship, registering and storing task states and versions to the sqlite database, and synchronizing to the sqlite of other servers, and simultaneously synchronizing the states and versions to the database. And creating a node through curator clients, setting content, and acquiring state information and version information in the node by other servers.
The whole process can be added, deleted, paused and re-run tasks in an interface mode. Each operation of the interface will trigger the above procedure (according to the dependency, register store task state and version to the sqlite database and synchronize to the sqlite of other servers while synchronizing state and version to the database. Create node through curator client, set content, other servers get state information and version information in node).
In a specific embodiment, there are two tasks already present in the system A, B. The C task is not provided with @ Job annotation when the system is started, and can be activated at a specific time, at the moment, parameters can be transmitted to a server through a page, the C task is found according to the parameters, other parameters are provided, the C task is activated, and the C task is executed according to an execution plan in the parameters. When the task C is activated, the task is registered according to parameters, a dependency relationship is established, and states and versions are managed in the running process.
In summary, the zookeeper is used as a registry, and the states of the nodes are synchronized to the registry, so that the child node can monitor the task state of the relevant father node through the monitor, and judge whether the current child node performs data processing or not through the task state, thereby realizing the dependence of the task and the monitoring of the graphical interface.
And finishing the operation of the zookeeper by connecting the zookeeper with the pool. The method comprises the steps of adding nodes, writing state data in the nodes, inquiring the state number of a certain node and the like. The executor synchronously refreshes the information in the zookeeper into the database to provide inquiry for the historical data. And developing a new management platform in an http request restful mode, and supporting starting, suspending, rerun and terminating the existing tasks.
FIG. 2 is a diagram of a configuration of a configurable platform scheduling system of the present invention, and with reference to FIG. 2,
The configured platform scheduling system 200 includes a scheduled task annotation unit 210, a scheduled task registration unit 220, a scheduled task association unit 230, and a scheduled task execution unit 240.
The scheduling task annotating unit 210 is configured to annotate the scheduling task of the configuration platform, store the scheduling task and the corresponding annotation information in the lightweight database sqlite in an associated manner, and set a dependency relationship between the scheduling tasks according to the annotation information. And the scheduling task registration unit 220 is used for performing micro-service scanning on the scheduling task with the annotation, and registering in a registration center of the configuration platform according to the type of the annotation of the scheduling task. The scheduling task association unit 230 is configured to allocate fragments to the scheduling task according to the number of servers in the registry, and determine an association relationship between the servers and the scheduling task after allocating fragments. The scheduled task execution unit 240 is configured to determine an execution server and an execution mode of the scheduled task according to the association relationship and the dependency relationship between the scheduled tasks, and execute the scheduled task according to the execution server and the execution mode of the scheduled task.
The scheduling task annotating unit 210 comprises a scheduling task annotating module 211 and a scheduling task storage module 212 with notes, wherein the scheduling task annotating module 211 is used for annotating the scheduling task of the configuration platform, the scheduling task annotating comprises task node classification, task state attribute and task version attribute, the task state attribute is divided into running, to-be-run, pause, completion or abnormality, the task node classification comprises father node tasks and child node tasks, the scheduling task storage module 212 with notes is used for storing the scheduling task and corresponding annotation information in a lightweight database sqlite in an associated mode, and the dependency relationship among the scheduling tasks is set according to the annotation information.
In a specific embodiment, it is emphasized that the annotated scheduled task storage module 212 includes a storage sub-module 2111, a dependency setting sub-module 2112, and a data burial point synchronization sub-module 2113.
The system comprises a storage submodule 2111 for storing the dispatching tasks and the corresponding annotation information in a lightweight database sqlite in an associated mode, a dependency relation setting submodule 2112 for setting the dependency relation among the dispatching tasks according to the annotation information, and a data embedding point synchronization submodule 2113 for adding a plurality of data embedding points in a plurality of states on the original data embedding points by utilizing a database monitor of a configuration platform and synchronizing all the data embedding points to the database sqlite.
Taking an application scene of an on-board positioning system as an example, the image page-based configuration platform scheduling method is utilized to build a server environment. The vehicle-mounted positioning software of each vehicle starts a timing task, the positioning information is uploaded to the server every 5s, the server management interface can operate the timing task of each vehicle, the system server starts the timing task, and the uploading position information of all vehicles is analyzed every 30 s:
That is, the scheduling task here is specifically a positioning task. Firstly annotating a positioning task, storing the positioning task and corresponding annotation information in a lightweight database sqlite in an associated mode, setting a dependency relationship between the positioning task and a history task of a main server according to the annotation information, wherein the history task in the main server is used as a father node task and the positioning task is used as a child node task.
The method comprises the steps of carrying out micro-service scanning on a dispatching task with comments, registering in a registration center of a server according to the type of the comments of the dispatching task, determining the association relation between each server and a positioning task according to the number of the servers in the registration center, namely selecting the server for executing the positioning task, determining the execution server of the positioning task and the history task of an associated main server according to the association relation and the dependency relation between the dispatching tasks, determining the execution server and the father node task, determining the execution mode of the positioning task, executing the positioning task according to the execution server and the execution mode of the positioning task, and then carrying out corresponding early warning according to the positioning task by a vehicle-mounted positioning system according to a specified rule.
The invention provides a dispatching method of a configuration platform, which is applied to an electronic device 3.
Fig. 3 shows an application environment of a preferred embodiment of the deployment method of the deployment platform according to the present invention.
Referring to fig. 3, in the present embodiment, the electronic device 4 may be a terminal device having an operation function, such as a server, a smart phone, a tablet computer, a portable computer, a desktop computer, or the like.
The electronic device 3 comprises a processor 32, a memory 31, a communication bus 33 and a network interface 35.
The memory 31 includes at least one type of readable storage medium. The at least one type of readable storage medium may be a non-volatile storage medium such as flash memory, a hard disk, a multimedia card, a card memory 31, etc. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 3, such as a hard disk of the electronic device 3. In other embodiments, the readable storage medium may also be an external memory 31 of the electronic device 3, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the electronic device 3.
In the present embodiment, the readable storage medium of the memory 31 is generally used for storing a configurable platform scheduler 30 or the like installed in the electronic device 3. The memory 31 may also be used for temporarily storing data that has been output or is to be output.
The processor 32 may in some embodiments be a central processing unit (Central Processing Unit, CPU), microprocessor or other data processing chip for executing program code or processing data stored in the memory 31, such as executing the configurable platform scheduler 30, etc.
The communication bus 33 is used to enable connection communication between these components.
The network interface 34 may optionally comprise a standard wired interface, a wireless interface (e.g. WI-FI interface), typically used to establish a communication connection between the electronic apparatus 3 and other electronic equipment.
Fig. 3 shows only the electronic device 3 with components 31-34, but it is understood that not all shown components are required to be implemented, and that more or fewer components may alternatively be implemented.
Optionally, the electronic device 3 may further comprise a user interface, which may comprise an input unit such as a Keyboard (Keyboard), a voice input device such as a microphone or the like with voice recognition function, a voice output device such as a sound box, a headset or the like, and optionally a standard wired interface, a wireless interface.
Optionally, the electronic device 3 may also comprise a display, which may also be referred to as a display screen or display unit. In some embodiments, the display may be an LED display, a liquid crystal display, a touch-control liquid crystal display, an Organic Light-Emitting Diode (OLED) touch device, or the like. The display is used for displaying information processed in the electronic device 3 and for displaying a visualized user interface.
Optionally, the electronic device 3 may further include a Radio Frequency (RF) circuit, a sensor, an audio circuit, etc., which are not described herein.
In the embodiment of the device shown in fig. 3, the memory 31 as a computer storage medium may include an operating system and a deployment platform scheduler 30, where when executing the deployment platform scheduler 30 stored in the memory 31, the processor 32 implements the following steps of annotating a deployment task of a deployment platform, storing the deployment task and corresponding annotation information in a lightweight database sqlite, setting a dependency relationship between the deployment tasks according to the annotation information, performing micro-service scanning on the deployment task with the annotation, registering the deployment task with a registration center of the deployment platform according to a type to which the annotation of the deployment task belongs, distributing fragments to the deployment task according to an elastic-job frame according to the number of servers existing in the registration center, determining an association relationship between the server and the deployment task after the allocation fragments, determining an execution server and an execution mode of the deployment task according to the association relationship and the dependency relationship between the deployment tasks, and executing the deployment task according to the execution server and the execution mode of the deployment task. .
In other embodiments, the configurable platform scheduler 30 may also be partitioned into one or more modules, one or more modules stored in the memory 31 and executed by the processor 32 to complete the present invention. As used herein, a module refers to a series of computer program segments capable of performing a specified function. The configurated platform scheduler 30 may be divided into a unit for annotating scheduled tasks 210, a unit for registering scheduled tasks 220, a unit for associating scheduled tasks 230, and a unit for executing scheduled tasks 240.
In addition, the invention also provides a computer readable storage medium mainly comprising a storage data area and a storage program area, wherein the storage data area can store data created according to the use of the blockchain node and the like, the storage program area can store an operating system and at least one application program required by functions, and the computer readable storage medium comprises a configuration platform scheduling program which realizes the operation of a configuration platform scheduling method when being executed by a processor.
The embodiments of the computer readable storage medium of the present invention are substantially the same as the embodiments of the above-mentioned configuration platform scheduling method, system and electronic device, and will not be described herein.
In general, the dispatching method, the system, the electronic device and the computer readable storage medium for the configuration platform carry out annotation on dispatching tasks of the configuration platform, store the dispatching tasks with the annotation in a lightweight database sqlite, set dependency relations among the dispatching tasks, carry out micro-service scanning on the dispatching tasks with the annotation, register the dispatching tasks in a registration center of the configuration platform according to the type of the annotation of the dispatching tasks, distribute fragments of the dispatching tasks according to the number of servers existing in the registration center, determine the association relations between the servers and the dispatching tasks after the allocation of fragments, judge the execution servers and execution modes of the dispatching tasks according to the association relations between the servers and the dispatching tasks after the allocation of fragments, enrich the functions of the configuration platform, provide the functions of starting, suspending, rerun and stopping existing tasks, simultaneously provide dynamic addition new tasks, greatly promote usability, and enable the dispatching tasks to have great advantages by the aid of the configuration of the system according to the number of the servers existing in the registration center, and the stability of the configuration of the dispatching tasks.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The blockchain (Blockchain), essentially a de-centralized database, is a string of data blocks that are generated in association using cryptographic methods, each of which contains information from a batch of network transactions for verifying the validity (anti-counterfeit) of its information and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, apparatus, article, or method that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments. From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, including several programs for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (6)
1. A method for dispatching a configured platform, which is applied to an electronic device, the method comprising:
Annotating a scheduling task of a configuration platform, storing the scheduling task and corresponding annotation information in a lightweight database sqlite in an associated mode, and setting a dependency relationship between the scheduling tasks according to the annotation information, wherein a database monitor of the configuration platform is utilized to add a plurality of data embedding points in a plurality of states on an original data embedding point, and all the data embedding points are synchronized to the database sqlite, and the states of the data embedding points comprise RUNNING, READY, STOP, OVER and ERROR;
Performing micro-service scanning on a scheduling task with annotation, and registering in a registration center of the configuration platform according to the type of the annotation of the scheduling task, wherein the annotation of the scheduling task comprises task node classification data, task state attribute data and task version attribute data, and the task state attribute is classified into running, waiting to run, suspending, completing or abnormal;
distributing and slicing the scheduling task according to the number of servers in the registry and an elastic-job frame to determine the association relationship between the servers and the scheduling task after the slicing is distributed, wherein the slicing condition of the task is determined according to the number of servers in the registry and the task is operated in parallel or in series according to a plan;
Determining an execution server and an execution mode of the scheduling task according to the association relation and the dependency relation among the scheduling tasks;
and executing the scheduling task according to the execution server and the execution mode of the scheduling task.
2. The method for dispatching the configuration platform according to claim 1, wherein the dispatching task and the corresponding annotation information are stored in a lightweight database sqlite in an associated mode, corresponding log files are dynamically added to the dispatching task according to the task, the dispatching task is stored in a grading mode according to the log files in error, warn, info, debug states, and the log files are rolled according to two dimensions of time and size.
3. The method for dispatching the configuration platform according to claim 1, wherein the data of the dispatching task is stored in a blockchain, and the dispatching task data stored in each lightweight database sqlite is synchronized into the database of the configuration platform to realize task state query of the dispatching task.
4. A dispatching system of a configured platform is characterized by comprising a dispatching task annotating unit, a dispatching task registering unit, a dispatching task associating unit and a dispatching task executing unit, wherein,
The scheduling task annotating unit is used for annotating scheduling tasks of the configuration platform, storing the scheduling tasks and corresponding annotation information in a lightweight database sqlite in an associated mode, and setting dependency relations among the scheduling tasks according to the annotation information;
The system comprises a dispatch task registration unit, a dispatch task annotation unit and a dispatch task storage module, wherein the dispatch task registration unit is used for carrying out micro-service scanning on a dispatch task with annotation and registering in a registration center of a configuration platform according to the type of the annotation of the dispatch task, wherein the data of the dispatch task is stored in a block chain, the dispatch task annotation unit comprises a dispatch task annotation module and a dispatch task storage module with annotation, the dispatch task annotation module is used for annotating the dispatch task of the configuration platform, the annotation of the dispatch task comprises task node classification, task state attribute and task version attribute, the task state attribute is divided into running, waiting, suspending, finishing or abnormality, the task node classification comprises father node tasks and child node tasks, and the dispatch task storage module with annotation is used for storing the dispatch task and corresponding annotation information in a lightweight database sqlite and setting the dependency relationship among the dispatch tasks according to the annotation information;
The system comprises a registration center, a scheduling task association unit, a task execution unit and a task execution unit, wherein the registration center is used for registering a plurality of servers in the registration center, the number of servers in the registration center is used for carrying out distribution segmentation on the scheduling task according to an elastic-job frame, and determining the association relation between the servers and the scheduling task after distribution segmentation;
The system comprises a scheduling task storage module, a scheduling task execution unit, a scheduling task embedding synchronization sub-module and a scheduling task processing unit, wherein the scheduling task storage module is used for storing a scheduling task in a storage sub-module, and is used for storing the scheduling task in a storage sub-module;
The storage sub-module is used for storing the scheduling task and the corresponding annotation information in a lightweight database sqlite;
A dependency setting sub-module for setting a dependency between scheduling tasks according to the annotation information;
The data embedding point synchronization sub-module is used for adding a plurality of data embedding points in a plurality of states on the original data embedding points by utilizing a database monitor of the configuration platform and synchronizing all the data embedding points to a database sqlite, wherein the states of the data embedding points comprise RUNNING, READY, STOP, OVER and ERROR.
5. An electronic device is characterized by comprising at least one processor; and
A memory communicatively coupled to the at least one processor, wherein,
The memory stores a program executable by the at least one processor to enable the at least one processor to perform the configurable platform scheduling method of any one of claims 1 to 3.
6. A computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the configurable platform scheduling method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010879341.6A CN112015534B (en) | 2020-08-27 | 2020-08-27 | Configurable platform scheduling method, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010879341.6A CN112015534B (en) | 2020-08-27 | 2020-08-27 | Configurable platform scheduling method, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112015534A CN112015534A (en) | 2020-12-01 |
| CN112015534B true CN112015534B (en) | 2025-05-27 |
Family
ID=73503764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010879341.6A Active CN112015534B (en) | 2020-08-27 | 2020-08-27 | Configurable platform scheduling method, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112015534B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114968504A (en) * | 2021-02-26 | 2022-08-30 | 中国联合网络通信集团有限公司 | Distributed task scheduling method, device and storage medium |
| CN113485711A (en) * | 2021-06-30 | 2021-10-08 | 平安证券股份有限公司 | Task triggering method and device based on dependency relationship, electronic equipment and medium |
| CN114090198A (en) * | 2021-10-27 | 2022-02-25 | 青岛海尔科技有限公司 | Distributed task scheduling method, device, electronic device and storage medium |
| CN114327817B (en) * | 2021-12-22 | 2025-03-21 | 马上消费金融股份有限公司 | Task slicing method, device and electronic device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110096342A (en) * | 2019-05-08 | 2019-08-06 | 深圳乐信软件技术有限公司 | Task processing method, device, server and storage medium |
| CN110532084A (en) * | 2019-09-04 | 2019-12-03 | 深圳前海微众银行股份有限公司 | Dispatching method, device, equipment and the storage medium of platform task |
-
2020
- 2020-08-27 CN CN202010879341.6A patent/CN112015534B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110096342A (en) * | 2019-05-08 | 2019-08-06 | 深圳乐信软件技术有限公司 | Task processing method, device, server and storage medium |
| CN110532084A (en) * | 2019-09-04 | 2019-12-03 | 深圳前海微众银行股份有限公司 | Dispatching method, device, equipment and the storage medium of platform task |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112015534A (en) | 2020-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112015534B (en) | Configurable platform scheduling method, system and storage medium | |
| CN111279314B (en) | Providing tenant isolation in a multi-tenant API gateway using micro-service containers | |
| US10503493B2 (en) | Distributed versioning of applications using cloud-based systems | |
| CN108513657B (en) | Data conversion method and backup server | |
| US10185549B2 (en) | Updating live system with static changes | |
| US9600342B2 (en) | Managing parallel processes for application-level partitions | |
| CN106980625A (en) | A kind of method of data synchronization, apparatus and system | |
| EP3398063B1 (en) | Controlled deployment of application feature | |
| US20210184859A1 (en) | Validation and certification of digital assets using a blockchain | |
| CN112463304B (en) | Container image rollback method, device, computer equipment and storage medium | |
| CN113626512A (en) | Data processing method, device, equipment and readable storage medium | |
| US20180046635A1 (en) | Discovery of calling application for control of file hydration behavior | |
| CN111158654A (en) | Algorithm calling method, device, server and storage medium | |
| CN113722228A (en) | Software system testing method and device | |
| US10162626B2 (en) | Ordered cache tiering for program build files | |
| CN115454827A (en) | Compatibility detection method, system, device and medium | |
| US11366796B2 (en) | Systems and methods for compressing keys in hierarchical data structures | |
| CN116991414A (en) | File processing method, device, equipment and storage medium | |
| CN103729451B (en) | A kind of information input method of database, apparatus and system | |
| CN108363671B (en) | Interface switching method, terminal equipment and storage medium | |
| US12222825B2 (en) | Systems and methods for disaster recovery for edge devices | |
| CN115080114A (en) | Application program transplanting processing method, device and medium | |
| CN106357735A (en) | Method and device for operating infrastructure layer of cloud computing architecture | |
| CN114253906A (en) | Configuration file management method, device, configuration distribution system and storage medium | |
| TW201403362A (en) | Managing large data sets through page based information tracking in multi-master environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |