CN109951553B

CN109951553B - Data processing method, system, electronic device and computer readable storage medium

Info

Publication number: CN109951553B
Application number: CN201910217152.XA
Authority: CN
Inventors: 祝剑锋; 冯常健; 尧飘海; 陈谔
Original assignee: Hangzhou Langhe Technology Co Ltd
Current assignee: Hangzhou Netease Shuzhifan Technology Co ltd
Priority date: 2019-03-21
Filing date: 2019-03-21
Publication date: 2022-03-08
Anticipated expiration: 2039-03-21
Also published as: CN109951553A

Abstract

The embodiment of the invention provides a data processing method, a data processing system, electronic equipment and a computer readable storage medium, and relates to the technical field of computers. The data processing method comprises the following steps: the method comprises the steps that a first server receives a first data writing request sent by a first gateway, and data corresponding to the first data writing request are stored in a key value type database; the first server receives a third data writing request sent by the second server, wherein the third data writing request is obtained by performing logic conversion processing on a second data writing request sent by the second gateway through the second server; and the first server stores the data corresponding to the third data writing request to the key-value type database. Therefore, the embodiment of the invention can provide two interfaces (namely, the interfaces corresponding to the first gateway and the second gateway respectively) for the user, so as to reduce the learning cost of the user.

Description

Data processing method, system, electronic device and computer readable storage medium

Technical Field

Embodiments of the present invention relate to the field of computer technologies, and in particular, to a data processing method, a data processing system, an electronic device, and a computer-readable storage medium.

Background

A Container (Container) is a more lightweight, flexible way of virtualization that can package together the code needed by an application and the operating system to enable the application to run anywhere. Furthermore, Kubernetes, K8S for short, is an open source system for automated deployment, scaling, and managing containerized applications.

Generally, a container cloud platform opens a use interface for users, such as a k8s interface or a self-research interface, but because there is a difference in basic knowledge held by different users, there is a certain difficulty in using the platform, for example, some users may use the self-research interface instead of using the k8s interface, and if the users want to use the k8s interface, they need to learn for a while before they can use the platform; and, some users will use the k8s interface and not the self-research interface, and if the user wants to use the self-research interface, the user needs to learn for a while before the user can use the interface, which increases the learning cost of the user.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present invention and therefore may include information that does not constitute prior art known to a person of ordinary skill in the art.

Disclosure of Invention

Typically, a container cloud platform exposes only one usage interface to the user, such as the k8s interface or a self-developed interface, without compromising both interfaces, which increases the learning cost for users who only use a single interface.

For this reason, an improved data processing method is highly needed to reduce the learning cost of the user and to overcome the problem of the limited performance of query filtering of k8s storage that affects the read-write performance of the platform.

In this context, embodiments of the present invention provide a data processing method, a data processing system, an electronic device, and a computer-readable storage medium.

According to a first aspect of an embodiment of the present invention, a data processing method is disclosed, which includes: the method comprises the steps that a first server receives a first data writing request sent by a first gateway, and data corresponding to the first data writing request are stored in a key value type database; the first server receives a third data writing request sent by the second server, wherein the third data writing request is obtained by performing logic conversion processing on a second data writing request sent by the second gateway through the second server; and the first server stores the data corresponding to the third data writing request to the key-value type database.

In one embodiment, based on the foregoing scheme, the first write data request is transmitted to the first gateway through the first interface, and the second write data request is transmitted to the second gateway through the second interface.

In an embodiment, based on the foregoing solution, after the first server receives the third write data request sent by the second server, the data processing method further includes: the third server detects resource change information in the first server; and the third server stores the resource change information to the relational database.

In one embodiment, based on the foregoing solution, after the third server detects the resource change information in the first server, the data processing method further includes: and the fourth server determines target information corresponding to the resource change information and stores the target information in the relational database, wherein the target information comprises at least one of a resource name space, a resource name and a resource specification.

In one embodiment, based on the foregoing scheme, the data processing method further includes: the method comprises the steps that a first server receives a first read data request sent by a first gateway, wherein the first read data request is transmitted to the first gateway through a first interface; the first server obtains data corresponding to the first read data request from the key-value database.

In one embodiment, based on the foregoing scheme, the data processing method further includes: the second server receives a second data reading request sent by a second gateway, wherein the second data reading request is transmitted to the second gateway through a second interface; the second server obtains data corresponding to the second data reading request from the relational database.

In one embodiment, based on the foregoing scheme, the data processing method further includes: and the fifth server synchronizes the data in the relational database to the key-value database or synchronizes the data in the key-value database to the relational database, so that the data in the relational database is the same as the data in the key-value database.

According to a second aspect of the embodiments of the present invention, there is disclosed a data processing system comprising: the first server is used for receiving a first data writing request sent by a first gateway and storing data corresponding to the first data writing request to a key value type database; the second server is used for carrying out logic conversion processing on the second data writing request to obtain a third data writing request and sending the third data writing request to the first server; the first server is further configured to receive a third write data request and store the third write data request to the key-value type database.

In one embodiment, based on the foregoing solution, the data processing system further includes: and the third server is used for detecting the resource change information in the first server after the first server receives the third write data request sent by the second server, and storing the resource change information into the relational database.

In one embodiment, based on the foregoing solution, the data processing system further includes: and the fourth server is used for determining target information corresponding to the resource change information and storing the target information in the relational database after the third server detects the resource change information in the first server, wherein the target information comprises at least one of a resource naming space, a resource name and a resource specification.

In an embodiment, based on the foregoing scheme, the first server is further configured to receive a first read data request sent through the first gateway, where the first read data request is transmitted to the first gateway through the first interface; and acquiring data corresponding to the first read data request from the key-value type database.

In an embodiment, based on the foregoing scheme, the second server is further configured to receive a second read data request sent by the second gateway, where the second read data request is transmitted to the second gateway through the second interface; and acquiring data corresponding to the second data reading request from the relational database.

In one embodiment, based on the foregoing solution, the data processing system further includes: and the fifth server is used for synchronizing the data in the relational database to the key-value database or synchronizing the data in the key-value database to the relational database so as to enable the data in the relational database to be the same as the data in the key-value database.

According to a third aspect of the embodiments of the present invention, there is disclosed an electronic apparatus, comprising: a processor; and a memory having computer readable instructions stored thereon which, when executed by the processor, implement the data processing method as disclosed in the first aspect.

According to a fourth aspect of embodiments of the present invention, a computer program medium is disclosed, having computer readable instructions stored thereon, which, when executed by a processor of a computer, cause the computer to perform the data processing method disclosed according to the first aspect of the present invention.

The method is different from the container cloud platform which only opens one interface (such as a k8s interface or a self-developed interface) in the prior art. The embodiment of the invention can provide two interfaces for a user, and the user can perform writing operation through the first gateway (such as the gateway corresponding to the k8s interface) and the second gateway (such as the gateway corresponding to the self-research interface) corresponding to the two interfaces respectively.

Additional features and advantages of the invention will be set forth in the detailed description which follows, or may be learned by practice of the invention.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:

FIG. 1 is a schematic flow diagram illustrating a data processing method according to an example embodiment of the invention;

FIG. 2 is a block diagram illustrating an architecture of a data processing method according to an exemplary embodiment of the present invention;

FIG. 3 illustrates a sequence diagram for processing a write data request by a container service in accordance with an example embodiment of the present invention;

FIG. 4 illustrates a sequence diagram for processing read data requests by a container service in accordance with an exemplary embodiment of the present invention;

FIG. 5 illustrates a sequence diagram for processing a write data request through a k8s service in accordance with an example embodiment of the present invention;

FIG. 6 illustrates a sequence diagram for processing read data requests through a k8s service in accordance with an exemplary embodiment of the present invention;

FIG. 7 is a block diagram of a data processing system in accordance with an exemplary embodiment of the present invention;

FIG. 8 is a block diagram of a data processing system according to another alternative example embodiment of the present invention.

In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

Detailed Description

The principles and spirit of the present invention will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Thus, the present invention may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

According to an embodiment of the present invention, a data processing method, a data processing system, an electronic device, and a computer-readable storage medium are provided.

Any number of elements in the drawings are by way of example and not by way of limitation, and any nomenclature is used solely for differentiation and not by way of limitation.

The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments of the invention.

Summary of The Invention

The inventor finds that, since the container cloud platform opens a use interface to users, such as a k8s interface or a self-research interface, but there is a difference in basic knowledge grasped by different users, there is a certain difficulty in using the platform, for example, some users may use the self-research interface instead of using the k8s interface, and if the users want to use the k8s interface, they need to learn for a while before they can use the platform; and, some users will use the k8s interface and not the self-research interface, and if the user wants to use the self-research interface, the user needs to learn for a while before the user can use the interface, which increases the learning cost of the user.

Accordingly, an embodiment of the present invention provides a data processing method, including: the method comprises the steps that a first server receives a first data writing request sent by a first gateway, and data corresponding to the first data writing request are stored in a key value type database; the first server receives a third data writing request sent by the second server, wherein the third data writing request is obtained by performing logic conversion processing on a second data writing request sent by the second gateway through the second server; and the first server stores the data corresponding to the third data writing request to the key-value type database. The first gateway corresponds to a first interface, such as a k8s interface, and the second gateway corresponds to a second interface, such as a self-research interface, and the user may send the first write data request through the first interface or send the second write data request through the second interface. Therefore, the embodiment of the invention can open two interfaces to the user to facilitate the writing operation of the user, so as to reduce the learning cost of the user.

Having described the general principles of the invention, various non-limiting embodiments of the invention are described in detail below.

Application scene overview

It should be noted that the following application scenarios are merely illustrated to facilitate understanding of the spirit and principles of the present invention, and the embodiments of the present invention are not limited in this respect. Rather, embodiments of the present invention may be applied to any scenario where applicable.

The embodiment of the invention can be applied to the container cloud platform, so that the container cloud platform can open two interfaces to a user, and the learning cost of the user is reduced; the data processing method disclosed by the embodiment of the invention provides two interfaces for a user, so that the user can perform read-write operation through any one of the interfaces. When a user performs a write operation through the first interface, a first server (e.g., a server running the k8s service) may receive a write data request corresponding to the write operation sent through a first gateway (e.g., a k8s gateway), and store data corresponding to the write data request (which may also be understood as data written by the user) in a key-value type database (e.g., an Etcd database); when a user performs a write operation through the second interface, the second server (e.g., a server running a container service) may perform a logical conversion on a write data request corresponding to the write operation and then send the write data request to the first server, so that the first server stores data corresponding to the write data request in the key-value type database; the Etcd database is a distributed key value storage system for shared configuration and service discovery. In addition, it should be noted that the first server and the second server may be the same general server. If the first server and the second server are the same total server, the total server contains the code corresponding to the k8s service and the code corresponding to the container service.

Exemplary method

In connection with the above application scenarios, a data processing method according to an exemplary embodiment of the present invention is described below with reference to fig. 1 to 6.

Referring to fig. 1, fig. 1 is a flowchart illustrating a data processing method according to an exemplary embodiment of the present invention, where the data processing method may be implemented by a server or a terminal device.

As shown in fig. 1, a data processing method according to an embodiment of the present invention includes:

step S110: the first server receives a first data writing request sent by a first gateway, and stores data corresponding to the first data writing request to a key-value type database.

Step S120: and the first server receives a third data writing request sent by the second server, wherein the third data writing request is obtained by performing logic conversion processing on a second data writing request sent by the second gateway through the second server.

Step S130: and the first server stores the data corresponding to the third data writing request to the key-value type database.

These steps are described in detail below.

In step S110, the first server receives a first write data request sent through the first gateway, and stores data corresponding to the first write data request in the key-value type database.

In the embodiment of the present invention, the first server may run a k8s (kubernets) service, which is used to perform k8s native logic processing on a write data request sent by a user through a k8s gateway, and it may also be understood that the first server may split the write data request sent by the user into a plurality of logic units for storage and invocation. It should be noted that k8s is a container arrangement tool for managing applications running in a container, and its working principle is to split an application program into multiple logical units by grouping containers, so as to facilitate management and discovery.

In this embodiment of the present invention, the first write data request may be a write data request sent by a user through a first gateway, and the first gateway may be a k8s gateway. In addition, the data corresponding to the first write data request may be data written by a user through a write operation. The key value type database can be an etcd database; the etcd database is a non-relational database (NoSQL) model, data of the ETcd database can be organized, indexed and stored according to a key value pair mode, and the reading and writing times of a disk can be reduced through key value storage.

In this embodiment of the present invention, optionally, before the first server receives the first write data request sent through the first gateway, the method may further include: when detecting that a user calls a first interface (e.g., a k8s interface, where the k8s interface corresponds to a k8s gateway) to send a first write data request, the terminal device or the server may send the first write data request to the first gateway (e.g., a k8s gateway) through the first interface, so as to send the first write data request to the first server through the first gateway after the first gateway successfully verifies the user corresponding to the first write data request. The method for detecting that the user calls the first interface may specifically be: detecting that a user operates the kubecect or calls a program in a k8s interface; wherein kubecect may be a command line tool for processing the k8s resource corresponding to the first write data request.

In step S120, the first server receives a third write data request sent by the second server, where the third write data request is obtained by performing logic conversion processing on a second write data request sent by the second gateway through the second server.

In this embodiment of the present invention, the second server may run a container service, and is configured to convert the second write data request into a third write data request after logic processing, and the server running the k8s service may process the third write data request but cannot process the second write data request, that is, it is understood that the server running the container service converts the encoding language of the second write data request into the encoding language that the server running the k8s service can process, that is, the third write data request is obtained, so that the server running the k8s service performs corresponding processing on the third write data request; the number of the third write data requests may be multiple, that is, it may be understood that the second write data request may correspond to multiple k8s resources, and multiple third write data requests may be obtained after the server running the container service performs logic processing on the second write data request. It should be noted that the above logic processing may include parameter checking, data warehousing, and checking logs.

In the embodiment of the present invention, the second data writing request may be a data writing request sent by a user through a second gateway, the data corresponding to the second data writing request may be data written by the user through a writing operation, and the second gateway may be a self-research api gateway; the interface corresponding to the self-research api gateway and opened to the user may be a self-research interface, which may be understood as an interface independently developed by a research and development staff, and the self-research interface may be customized by the research and development staff, which is not limited in the embodiment of the present invention. In addition, the self-research api gateway in the embodiment of the present invention is an entry for a client to access the microservice, and the self-research api gateway provides a self-research interface for each client, and the self-research interfaces of each client may be of the same type or different types. It should be noted that api (application Programming interface) in the self-developed api gateway is an application Programming interface. The self-research api gateway is a gateway corresponding to the self-research interface.

In this embodiment of the present invention, optionally, before the second server sends the third write data request to the first server, the method may further include: when detecting that the user calls the second interface (e.g., a self-research interface corresponding to the self-research api gateway) to send the second write data request, the terminal device or the server may send the second write data request to the second gateway (e.g., the self-research api gateway) through the second interface, so as to send the second write data request to the second server through the second gateway after the user corresponding to the second write data request is successfully verified by the second gateway. The method for detecting that the user calls the second interface may specifically be: the page click operation of the user is detected, and may also be a detection of a touch write operation of the user, and may also be a detection of a non-touch write operation of the user, which is not limited in the embodiment of the present invention.

In step S130, the first server stores the data corresponding to the third write data request in the key-value type database.

In the embodiment of the present invention, optionally, the first write data request is transmitted to the first gateway through the first interface, and the second write data request is transmitted to the second gateway through the second interface; the first interface may be a k8s interface, and the second interface may be a self-developed interface.

Therefore, by implementing the data processing method shown in fig. 1, a container cloud platform with two open interfaces can be provided for a user, and compared with a traditional container cloud platform with one interface, the embodiment of the invention can provide more operation options and more functions for the user, thereby improving the user experience and reducing the learning cost of the user.

As an optional implementation manner, after the first server receives the third write data request sent by the second server, the data processing method may further include the following steps: the third server detects resource change information in the first server; and the third server stores the resource change information to the relational database.

In the embodiment of the present invention, the third server may run a sync service, where the sync service is a service for detecting resource change information in the first server and storing the resource change information in the relational database, and it may be understood that data written by the user is stored in the relational database after detecting a write operation of the user in the first server and the data written by the user. The resource change information may include data written by a user, or may include other data that is changed according to the data written by the user, and the embodiment of the present invention is not limited thereto. The relational database (e.g., SQL database) is a database based on a relational model, and processes data in the database by collecting mathematical concepts and methods such as algebra.

In this embodiment of the present invention, optionally, after the third server stores the resource change information in the relational database, the method may further include: and the third server informs the fourth server of processing the resource change event so as to trigger the fourth server to determine target information corresponding to the resource change information and store the target information in the relational database.

Therefore, by implementing the embodiment of the invention, the resource change information in the first server can be detected and stored, so that the data synchronization of the relational database and the key-value database is facilitated, and the data consistency is maintained to a certain extent.

As another optional implementation, after the third server detects the resource change information in the first server, the data processing method may further include the following steps: and the fourth server determines target information corresponding to the resource change information and stores the target information in the relational database, wherein the target information comprises at least one of a resource name space, a resource name and a resource specification.

In the embodiment of the present invention, the fourth server may run a pages service, where the pages service may be a service for determining target information corresponding to the resource change information, and may be a service for storing the target information in a relational database.

In the embodiment of the present invention, optionally, the fourth server may further process a pod-based charging transaction, for example, charging according to the write-in duration of the user, which is not limited in the embodiment of the present invention; the pod is proposed for deploying, managing and arranging containerized applications, and is a minimum deployment unit in the kubernets. A pod is a group of containers consisting of a group of tightly coupled containers that can serve as carriers for one or more docker containers. In addition, docker is an open-source application container engine, and developers can package applications and dependency packages into portable containers, and then can publish the applications and dependency packages to Linux equipment, and can also implement virtualization.

It can be seen that, by implementing this alternative embodiment, data synchronization between the relational database and the key-value database can be implemented by determining resource change information in the key-value database and storing the resource change information in the relational database, so as to maintain data consistency to some extent.

As another optional implementation, the data processing method may further include the steps of: the method comprises the steps that a first server receives a first read data request sent by a first gateway, wherein the first read data request is transmitted to the first gateway through a first interface; the first server obtains data corresponding to the first read data request from the key-value database.

In this embodiment of the present invention, the first read data request may be a read data request sent by a user through the first gateway. In addition, optionally, after the first read data request is transmitted to the first gateway through the first interface, the first gateway may verify a user corresponding to the first read data request, and send the user to the first server after the verification is successful, so as to trigger execution of an operation of the first server to acquire data corresponding to the first read data request from the key-value type database.

Therefore, by implementing the embodiment of the invention, the query filtering performance of the container cloud platform can be improved and the data reading and writing efficiency can be improved by providing two data storage modes (such as key value type data storage and relational data storage).

As another optional implementation, the data processing method may further include the steps of: the second server receives a second data reading request sent by a second gateway, wherein the second data reading request is transmitted to the second gateway through a second interface; the second server obtains data corresponding to the second data reading request from the relational database.

In this embodiment of the present invention, the second read data request may be a read data request sent by the user through the second gateway. In addition, optionally, after the second data reading request is transmitted to the second gateway through the second interface, the second gateway may verify a user corresponding to the second data reading request, and send the user to the second server after the user is successfully verified, so as to trigger the second server to perform an operation of acquiring data corresponding to the second data reading request from the relational database.

As another optional implementation, the data processing method may further include the steps of: and the fifth server synchronizes the data in the relational database to the key-value database or synchronizes the data in the key-value database to the relational database, so that the data in the relational database is the same as the data in the key-value database.

Therefore, the implementation of the alternative embodiment can keep the consistency of the data to a certain extent by synchronizing the data of the relational database and the data of the key-value database.

It should be noted that the first server, the second server, the third server, the fourth server, and the fifth server according to the embodiment of the present invention may be the same general server, and the general server may execute an operation that can be executed by any one of the first server, the second server, the third server, the fourth server, and the fifth server.

Referring to fig. 2, fig. 2 is a schematic diagram of an architecture of a data processing method according to an exemplary embodiment of the present invention, and as shown in fig. 2, the schematic diagram of the architecture of the data processing method includes a self-research api gateway 201, a k8s gateway 202, a container service 203, a k8s service 204, a relational database 205, a key-value database 206, a sync service 207, and a works service 208.

Specifically, the user may send a user request through the self-research api gateway 201 or the k8s gateway 202, on one hand, if the user sends the user request to the container service 203 through the self-research api gateway 201 and the user request is a write data request, the container service 203 may send the write data request to the k8s service 204 again, and if the user request is a read data request, the container service 203 reads data corresponding to the user request from the relational database 205, encapsulates the data and returns the data to the user; if the user request is a write data request, the k8s service 204 may store data corresponding to the write data request in the key-value type database 206, and the sync service 207 may detect resource change information in the k8s service 204 and write the resource change information in the relational database 205, and then send the resource change information to the hermes service 208, so that the hermes service 208 determines target information corresponding to the resource change information and stores the target information in the relational database 205, where the target information includes at least one of a resource name space, a resource name, and a resource specification. On the other hand, if the user sends a user request to the k8s service 204 through the k8s gateway 202, and the user request is a read data request, the k8s service 204 reads the data corresponding to the user request from the key-value database 206, encapsulates the data, and returns the data to the user; if the user request is a write data request, the k8s service 204 may store data corresponding to the write data request in the key-value type database 206, and the sync service 207 may detect resource change information in the k8s service 204 and write the resource change information in the relational database 205, and then send the resource change information to the hermes service 208, so that the hermes service 208 determines target information corresponding to the resource change information and stores the target information in the relational database 205. In addition, data synchronization can be implemented between the relational database 205 and the key-value database 206 to ensure data uniformity to some extent.

It can be seen that, by implementing the architectural diagram of the data processing method shown in fig. 2, a container cloud platform with two open interfaces can be provided for a user, and compared with a traditional container cloud platform with one interface, the embodiment of the present invention can provide more operation options and more functions for the user, improve the user experience, and reduce the learning cost of the user. In addition, the embodiment of the invention can improve the query filtering performance of the container cloud platform and improve the read-write efficiency of data by providing two data storage modes (such as key value type data storage and relational data storage).

Referring to fig. 3, fig. 3 is a sequence diagram illustrating processing of a write data request by a container service according to an example embodiment of the present invention. As shown in fig. 3, after a user initiates a write data request (i.e., a second write data request), the server or the terminal device may verify the user through the self-research api gateway (i.e., a second gateway); the check may include an authentication check and/or a flow control check. After the verification is completed, sending the write data request initiated by the user to a server (i.e., a second server) running the container service to trigger the server running the container service to perform logic processing on the write data request initiated by the user (for example, referring to verification, data warehousing or an operation log, etc.), wherein the write data request after the logic processing can be further processed by the k8s service conveniently; further, the logically processed write data request is sent to the k8s service again to return the result to the container service after the k8s service processes the write data request, and to output information indicating that the operation was successful to the user. In the process of performing the k8s native logic processing, the k8s service may store data corresponding to the write data request in a key-value type database, the sync service (i.e., the third server) may detect resource change information, write the detected resource change information in a relational database (e.g., an SQL database), and notify a hermes service (i.e., the fourth server) to process a resource change event, so that the hermes service determines target information corresponding to the resource change information and stores the target information in the relational database, where the target information includes at least one of a resource name space, a resource name, and a resource specification.

It can be seen that, by implementing the sequence diagram for processing the write data request through the container service shown in fig. 3, a user familiar with the container service can conveniently perform a write data operation through the self-research api gateway corresponding to the self-research interface, and can also detect and store data corresponding to the write data request initiated by the user, thereby realizing data synchronization of the relational database and the key-value database, so as to ensure data consistency of the relational database and the key-value database to a certain extent, and improve query filtering performance of the container cloud platform.

Referring to fig. 4, fig. 4 is a sequence diagram illustrating processing of read data requests by a container service according to an example embodiment of the present invention. As shown in fig. 4, after a user initiates a read data request (i.e., a second read data request), the server or the terminal device may verify the user through the self-research api gateway (i.e., a second gateway); the check may include an authentication check and/or a flow control check. After the verification is completed, the read data request initiated by the user is sent to the server (i.e., the second server) running the container service, so as to trigger the server running the container service to query and filter relevant data corresponding to the read data request from a relational database (e.g., an SQL database) and return the query and filter result to the self-research api gateway, and then the query and filter result is packaged by the self-research api gateway and returned to the user.

It can be seen that, by implementing the sequence diagram for processing a read data request through a container service shown in fig. 4, a user familiar with the container service can conveniently perform a read data operation through a self-research api gateway corresponding to a container interface.

Referring to FIG. 5, FIG. 5 is a sequence diagram illustrating the processing of a write data request by a k8s service according to an example embodiment of the invention. As shown in fig. 5, after a user initiates a write data request (i.e., a first write data request), the server or the terminal device may check the user through the k8s gateway (i.e., a first gateway), where the check may include an authentication check and/or a flow control check. After the verification is completed, the user-initiated write data request is sent to the k8s service, so as to trigger the k8s service to perform k8s native logic processing on the user-initiated write data request, and return a processing result (such as information indicating successful processing) to the k8s gateway, and then output information indicating successful operation to the user through the k8s gateway, that is, return successful operation to the user. In the process of performing the k8s native logic processing, the k8s service may store data corresponding to the write data request in a key-value type database, the sync service (i.e., the third server) may detect resource change information, write the detected resource change information in a relational database (e.g., an SQL database), and notify a hermes service (i.e., the fourth server) to process a resource change event, so that the hermes service determines target information corresponding to the resource change information and stores the target information in the relational database, where the target information includes at least one of a resource name space, a resource name, and a resource specification.

It can be seen that, by implementing the sequence diagram for processing the write data request through the k8s service shown in fig. 5, a user familiar with k8s can conveniently perform a write data operation through the k8s gateway corresponding to the k8s interface, and data corresponding to the write data request initiated by the user can be detected and stored, so that data synchronization between the relational database and the key-value database is realized, data consistency between the relational database and the key-value database is ensured to a certain extent, and query filtering performance of the container cloud platform is improved.

Referring to fig. 6, fig. 6 is a sequence diagram illustrating the processing of a read data request by a k8s service according to an example embodiment of the present invention. As shown in fig. 6, after a user initiates a read data request (i.e., a first read data request), the server or the terminal device may verify the user through the k8s gateway (i.e., a first gateway); the check may include an authentication check and/or a flow control check. After the verification is completed, the read data request initiated by the user is sent to the k8s service, so as to trigger the k8s service to query and filter relevant data corresponding to the read data request from the key-value type database (such as an etcd database) and return the query and filter result to the k8s gateway, and then the query and filter result is encapsulated by the k8s gateway and returned to the user.

It can be seen that, by implementing the sequence diagram for processing a read data request through the k8s service shown in fig. 6, a user familiar with k8s can conveniently perform a read data operation through the k8s gateway corresponding to the k8s interface, and because the embodiment of the present invention supports dual interfaces and introduces the relational database, the data synchronization between the relational database and the key-value database is performed, so that the data consistency between the relational database and the key-value database is ensured to a certain extent, and the query filtering performance of the container cloud platform is improved.

Exemplary Medium

Having described the method of the exemplary embodiments of the present invention, the media of the exemplary embodiments of the present invention will be described next.

In some possible embodiments, aspects of the present invention may also be implemented as a medium having stored thereon program code for implementing steps in a data processing method according to various exemplary embodiments of the present invention described in the above-mentioned "exemplary methods" section of this specification when the program code is executed by a processor of a device.

Specifically, the processor of the device, when executing the program code, is configured to implement the following steps: the method comprises the steps that a first server receives a first data writing request sent by a first gateway, and data corresponding to the first data writing request are stored in a key value type database; the first server receives a third data writing request sent by the second server, wherein the third data writing request is obtained by performing logic conversion processing on a second data writing request sent by the second gateway through the second server; the first server stores data corresponding to the third data writing request to the key-value type database; the first data writing request is transmitted to the first gateway through the first interface, and the second data writing request is transmitted to the second gateway through the second interface.

In some embodiments of the invention, the program code is further configured to, when executed by the processor of the device, perform the following steps: after the first server receives the third write data request sent by the second server, the data processing method further includes: the third server detects resource change information in the first server; and the third server stores the resource change information to the relational database.

In some embodiments of the invention, the program code is further configured to, when executed by the processor of the device, perform the following steps: after the third server detects the resource change information in the first server, the data processing method further includes: and the fourth server determines target information corresponding to the resource change information and stores the target information in the relational database, wherein the target information comprises at least one of a resource name space, a resource name and a resource specification.

In some embodiments of the invention, the program code is further configured to, when executed by the processor of the device, perform the following steps: the method comprises the steps that a first server receives a first read data request sent by a first gateway, wherein the first read data request is transmitted to the first gateway through a first interface; the first server obtains data corresponding to the first read data request from the key-value database.

In some embodiments of the invention, the program code is executable by a processor of the device to perform the steps of: the second server receives a second data reading request sent by a second gateway, wherein the second data reading request is transmitted to the second gateway through a second interface; the second server obtains data corresponding to the second data reading request from the relational database.

In some embodiments of the invention, the program code is further configured to, when executed by the processor of the device, perform the following steps: and the fifth server synchronizes the data in the relational database to the key-value database or synchronizes the data in the key-value database to the relational database, so that the data in the relational database is the same as the data in the key-value database.

It should be noted that: the above-mentioned medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take a variety of forms, including, but not limited to: an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device over any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., over the internet using an internet service provider).

Exemplary System

Having described the media of an exemplary embodiment of the present invention, a data processing system of an exemplary embodiment of the present invention is next described with reference to FIG. 7.

Referring to FIG. 7, FIG. 7 is a block diagram illustrating a data processing system according to an exemplary embodiment of the present invention. As shown in fig. 7, the data processing system of an exemplary embodiment of the present invention includes: the first server 701 and the second server 702 specifically:

the first server 701 is configured to receive a first write data request sent by a first gateway, and store data corresponding to the first write data request in a key-value type database; the second server 702 is configured to perform logic conversion processing on the second write data request to obtain a third write data request, and send the third write data request to the first server 701; the first server 701 is further configured to receive a third write data request, and store the third write data request to the key-value type database.

In the embodiment of the present invention, optionally, the first write data request is transmitted to the first gateway through the first interface, and the second write data request is transmitted to the second gateway through the second interface.

Therefore, the data processing system shown in fig. 7 can provide a container cloud platform with two open interfaces for a user, and compared with the conventional container cloud platform with one interface, the embodiment of the invention can provide more operation options and more functions for the user, thereby improving the user experience and reducing the learning cost of the user.

As an alternative embodiment, the data processing system may further comprise a third server (not shown), wherein:

and a third server, configured to detect resource change information in the first server 701 after the first server 701 receives the third write data request sent through the second server 702, and store the resource change information in the relational database.

Therefore, by implementing the optional implementation mode, the resource change information in the first server can be detected and stored, so that the data synchronization between the relational database and the key-value database is facilitated, and the data consistency is maintained to a certain extent.

As an alternative embodiment, the data processing system may further comprise a fourth server (not shown), wherein:

and the fourth server is configured to determine target information corresponding to the resource change information after the third server detects the resource change information in the first server 701, and store the target information in the relational database, where the target information includes at least one of a resource name space, a resource name, and a resource specification.

As an optional implementation manner, the first server 701 is further configured to receive a first read data request sent by a first gateway, where the first read data request is transmitted to the first gateway through a first interface; and acquiring data corresponding to the first read data request from the key-value type database.

Therefore, by implementing the optional implementation mode, two data storage modes (such as key value type data storage and relational data storage) are provided, so that the query filtering performance of the container cloud platform can be improved, and the data reading and writing efficiency can be improved.

As an optional implementation manner, the second server 702 is further configured to receive a second read data request sent by a second gateway, where the second read data request is transmitted to the second gateway through a second interface; and acquiring data corresponding to the second data reading request from the relational database.

As an alternative embodiment, the data processing system may further comprise a fifth server (not shown), wherein:

and the fifth server is used for synchronizing the data in the relational database to the key-value database or synchronizing the data in the key-value database to the relational database so as to ensure that the data in the relational database is the same as the data in the key-value database.

Therefore, the optional embodiment can be implemented, and the data consistency can be maintained to a certain extent through the data synchronization of the relational database and the key value type database.

For details which are not disclosed in the embodiments of the apparatus of the present invention, reference is made to the above-described embodiments of the data processing method of the present invention for the respective functional modules of the data processing apparatus of the exemplary embodiment of the present invention correspond to the steps of the above-described exemplary embodiment of the data processing method.

A data processing system 800 in accordance with yet another alternative exemplary embodiment of the present invention is described below with reference to fig. 8. The data processing system 800 shown in FIG. 8 is only an example and should not be used to limit the scope of use or functionality of embodiments of the present invention.

As shown in FIG. 8, data processing system 800 is a general purpose computing device. The components of data processing system 800 may include, but are not limited to: the at least one processing unit 810, the at least one memory unit 820, and a bus 830 that couples the various system components including the memory unit 820 and the processing unit 810.

Wherein the storage unit stores program code that can be executed by the processing unit 810, such that the processing unit 810 performs the steps according to various exemplary embodiments of the present invention described in the description part of the above exemplary methods of the present specification. For example, the processing unit 810 may perform various steps as shown in fig. 1.

The storage unit 820 may include readable media in the form of volatile memory units such as a random access memory unit (RAM)8201 and/or a cache memory unit 8202, and may further include a read only memory unit (ROM) 8203.

The storage unit 820 may also include a program/utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 830 may be any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The data processing system 800 also may communicate with one or more external devices 900 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the data processing system 800, and/or with any devices (e.g., router, modem, etc.) that enable the data processing system 800 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Moreover, data processing system 800 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network such as the Internet) via network adapter 860. As shown in FIG. 8, network adapter 860 communicates with the other modules of data processing system 800 via bus 830. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with data processing system 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiment of the present invention.

In an exemplary embodiment of the present invention, there is also provided a computer program medium having stored thereon computer readable instructions which, when executed by a processor of a computer, cause the computer to perform the method described in the above method embodiment.

According to an embodiment of the present invention, there is also provided a program product for implementing the method in the above method embodiment, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Moreover, although the steps of the methods of the present invention are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiment of the present invention.

Exemplary electronic device

Having described the method, medium, and apparatus of exemplary embodiments of the present invention, an electronic device according to another exemplary embodiment of the present invention is described next.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

In some possible implementations, a computing device according to an embodiment of the invention may include at least one processor, and at least one memory. Wherein the memory stores program code which, when executed by the processor, causes the processor to perform the steps in the data processing method according to various exemplary embodiments of the present invention described in the "exemplary methods" section above in this specification. For example, the processor may execute step S110 shown in fig. 1, where the first server receives a first write data request sent through the first gateway, and stores data corresponding to the first write data request to the key-value type database; step S120, the first server receives a third data writing request sent by the second server, wherein the third data writing request is obtained by performing logic conversion processing on a second data writing request sent by the second gateway through the second server; in step S130, the first server stores the data corresponding to the third write data request in the key-value database.

It should be noted that although in the above detailed description several units or sub-units of the article delivery device are mentioned, this division is only exemplary and not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit according to embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Moreover, while the operations of the method of the invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.

While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the particular embodiments disclosed, nor is the division of the aspects, which is for convenience only as the features in these aspects may not be combined to benefit from the present disclosure. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A data processing method, comprising:

the method comprises the steps that a first server receives a first data writing request sent by a first gateway, and data corresponding to the first data writing request are stored in a key value type database;

the first server receives a third data writing request sent by a second server, wherein the third data writing request is obtained by performing logic conversion processing on a second data writing request sent by a second gateway through the second server;

the first server stores data corresponding to the third write data request to the key-value type database;

the first data writing request is transmitted to the first gateway through a first interface, the second data writing request is transmitted to the second gateway through a second interface, and the first interface and the second interface are both container cloud platform interfaces;

after the first server receives the third write data request sent by the second server, the method further includes:

the third server detects resource change information in the first server;

the third server stores the resource change information to a relational database;

wherein, after the third server detects the resource change information in the first server, the method further includes:

the fourth server determines target information corresponding to the resource change information and stores the target information in the relational database, wherein the target information comprises at least one of resource naming and resource specification;

wherein, still include:

and the fifth server synchronizes the data in the relational database to the key-value database, or synchronizes the data in the key-value database to the relational database, so that the data in the relational database is the same as the data in the key-value database.

2. The method of claim 1, further comprising:

the first server receives a first read data request sent by the first gateway, wherein the first read data request is transmitted to the first gateway through the first interface;

the first server obtains data corresponding to the first read data request from the key-value database.

3. The method of claim 1, further comprising:

the second server receives a second read data request sent by the second gateway, wherein the second read data request is transmitted to the second gateway through the second interface;

and the second server acquires data corresponding to the second data reading request from the relational database.

4. A data processing system, comprising:

the system comprises a first server, a second server and a key value type database, wherein the first server is used for receiving a first data writing request sent by a first gateway and storing data corresponding to the first data writing request to the key value type database;

the second server is used for carrying out logic conversion processing on the second data writing request to obtain a third data writing request and sending the third data writing request to the first server;

the first server is further configured to receive the third write data request and store the third write data request to the key-value type database;

the data writing method comprises the steps that a first data writing request is transmitted to a first gateway through a first interface, a second data writing request is transmitted to a second gateway through a second interface, and the first interface and the second interface are both container cloud platform interfaces;

wherein the data processing system further comprises:

the third server is used for detecting the resource change information in the first server after the first server receives a third write data request sent by the second server, and storing the resource change information to the relational database;

wherein the data processing system further comprises:

the fourth server is used for determining target information corresponding to the resource change information and storing the target information in the relational database after the third server detects the resource change information in the first server, wherein the target information comprises at least one of resource naming and resource specification;

wherein the data processing system further comprises:

and the fifth server is used for synchronizing the data in the relational database to the key-value database or synchronizing the data in the key-value database to the relational database so as to enable the data in the relational database to be the same as the data in the key-value database.

5. The system of claim 4, wherein the first server is further configured to receive a first read data request sent through the first gateway, and wherein the first read data request is transmitted to the first gateway through the first interface; and acquiring data corresponding to the first read data request from the key-value type database.

6. The system of claim 4, wherein the second server is further configured to receive a second read data request sent through the second gateway, and the second read data request is transmitted to the second gateway through the second interface; and acquiring data corresponding to the second data reading request from the relational database.

7. An electronic device, comprising:

a processor; and

a memory having stored thereon computer readable instructions which, when executed by the processor, implement a data processing method as claimed in any one of claims 1 to 3.

8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 3.