CN112579282B - Data processing method, device, system, and computer-readable storage medium - Google Patents
Data processing method, device, system, and computer-readable storage medium Download PDFInfo
- Publication number
- CN112579282B CN112579282B CN201910948681.7A CN201910948681A CN112579282B CN 112579282 B CN112579282 B CN 112579282B CN 201910948681 A CN201910948681 A CN 201910948681A CN 112579282 B CN112579282 B CN 112579282B
- Authority
- CN
- China
- Prior art keywords
- target object
- target
- storage unit
- determining
- response mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
本公开提供一种数据处理方法，包括：一种数据处理方法，包括：接收访问请求；基于访问请求，确定访问请求所针对的目标对象；确定目标对象的被访问状态；以及基于被访问状态，确定针对访问请求的响应方式；其中，当确定响应方式为第一响应方式时，从第一存储单元获取目标对象；当确定响应方式为第二响应方式时，至少从第二存储单元获取目标对象，其中，从第一存储单元获取目标对象的方式与从二存储单元获取目标对象的方式不同。本公开还提供了一种数据处理装置、一种数据处理系统以及一种计算机可读存储介质。
The present disclosure provides a data processing method, including: a data processing method, including: receiving an access request; based on the access request, determining the target object to which the access request is directed; determining the accessed state of the target object; and based on the accessed state, determining a response method for the access request; wherein, when the response method is determined to be a first response method, the target object is obtained from a first storage unit; when the response method is determined to be a second response method, the target object is obtained from at least a second storage unit, wherein the method of obtaining the target object from the first storage unit is different from the method of obtaining the target object from the second storage unit. The present disclosure also provides a data processing device, a data processing system, and a computer-readable storage medium.
Description
Technical Field
The present disclosure relates to the field of computer technology, and more particularly, to a data processing method, a data processing apparatus, a data processing system, and a computer readable storage medium.
Background
Cache systems have been widely used in internet applications. Data can be quickly read and written through the cache system, so that the performance of the system can be improved to a certain extent. Current caching systems include redis, memcache, for example. The cache system mostly adopts the fragments to store data, and when the data is read and written, the data can be accessed on the corresponding fragments. In practical applications, particularly during peak periods of data access, there is often a large amount of access to the data, even with frequently accessed hot spot data.
In the process of implementing the disclosed concept, the inventor finds that at least the following problems exist in the prior art, in the related art, when a large amount of data is accessed, huge pressure is generally caused on a background cache server, meanwhile, as the cached data is generally stored in a byte array, most of the data needs to be subjected to reverse ordering processing, a large amount of server computing resources are consumed, and the overall performance and user experience of the system are further affected.
Disclosure of Invention
In view of this, the present disclosure provides an optimized data processing method, data processing apparatus, data processing system, and computer readable storage medium.
One aspect of the present disclosure provides a data processing method including receiving an access request, determining a target object for which the access request is directed based on the access request, determining an accessed state of the target object, and determining a response manner for the access request based on the accessed state. And when the response mode is determined to be a first response mode, acquiring the target object from a first storage unit, and when the response mode is determined to be a second response mode, acquiring the target object from at least a second storage unit, wherein the mode of acquiring the target object from the first storage unit is different from the mode of acquiring the target object from the second storage unit.
According to the embodiment of the disclosure, the target object can be accessed by a plurality of threads, and the determining the accessed state of the target object comprises determining at least one target thread in the plurality of threads, wherein the at least one target thread is the thread currently accessing the target object, and determining the accessed state of the target object, and the accessed state represents the accessed state of the target object by the at least one target thread.
According to the embodiment of the disclosure, when a plurality of target threads are included, the determining the response mode for the access request includes determining that the response mode for a first target thread is the first response mode, the first thread is a target thread which accesses the target object first in the plurality of target threads, determining that the response mode for a second target thread is the second response mode, and the second thread is a target thread except the first target thread in the plurality of target threads.
According to the embodiment of the disclosure, when one target thread is included, the determining the response mode for the access request includes determining the access order of the target thread to access the target object, determining the response mode for a first access order to be the first response mode, wherein the first access order represents that the target thread accesses the target object for the first time, determining the response mode for a second access order to be the second response mode, and the second access order is the access order of the target thread except the first access order.
According to the embodiment of the disclosure, acquiring the target object from at least a second storage unit comprises acquiring initial data from the second storage unit, and performing deserialization on the initial data to obtain the target object.
According to the embodiment of the disclosure, acquiring the target object from at least a second storage unit comprises determining attribute information of the target object, dividing the target object into static data and dynamic data based on the attribute information of the target object, acquiring the static data from the first storage unit, acquiring the dynamic data from the second storage unit, and performing deserialization processing on the dynamic data.
According to an embodiment of the disclosure, the first storage unit includes a memory unit, and the second storage unit includes a cache unit.
Another aspect of the present disclosure provides a data processing apparatus including a receiving module, a first determining module, a second determining module, and a third determining module. The receiving module receives an access request, the first determining module determines a target object aimed by the access request based on the access request, the second determining module determines an accessed state of the target object, and the third determining module determines a response mode aiming at the access request based on the accessed state. And when the response mode is determined to be a first response mode, acquiring the target object from a first storage unit, and when the response mode is determined to be a second response mode, acquiring the target object from at least a second storage unit, wherein the mode of acquiring the target object from the first storage unit is different from the mode of acquiring the target object from the second storage unit.
According to the embodiment of the disclosure, the target object can be accessed by a plurality of threads, and the determining the accessed state of the target object comprises determining at least one target thread in the plurality of threads, wherein the at least one target thread is a thread which is currently prevented from asking the target object, and determining the accessed state of the target object, and the accessed state represents the accessed state of the target object by the at least one target thread.
According to the embodiment of the disclosure, when the plurality of target threads are included, the third determining module comprises a first determining sub-module and a second determining sub-module. The first determining submodule determines a response mode aiming at a first target thread to be the first response mode, wherein the first thread is a target thread which accesses the target object first in the target threads, and the second determining submodule determines a response mode aiming at a second target thread to be the second response mode, and the second thread is a target thread except the first target thread in the target threads.
According to the embodiment of the disclosure, when one target thread is included, the third determining module comprises a third determining sub-module, a fourth determining sub-module and a fifth determining sub-module. The third determining submodule determines the access order of the target thread to the target object, the fourth determining submodule determines the response mode aiming at the first access order to be the first response mode, the first access order represents that the target thread accesses the target object for the first time, and the fifth determining submodule determines the response mode aiming at the second access order to be the second response mode, and the second access order is the access order of the target thread except the first access order.
According to the embodiment of the disclosure, acquiring the target object from at least a second storage unit comprises acquiring initial data from the second storage unit, and performing deserialization on the initial data to obtain the target object.
According to the embodiment of the disclosure, acquiring the target object from at least a second storage unit comprises determining attribute information of the target object, dividing the target object into static data and dynamic data based on the attribute information of the target object, acquiring the static data from the first storage unit, acquiring the dynamic data from the second storage unit, and performing deserialization processing on the dynamic data.
According to an embodiment of the disclosure, the first storage unit includes a memory unit, and the second storage unit includes a cache unit.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, are configured to implement a method as described above.
Another aspect of the present disclosure provides a computer program comprising computer executable instructions which when executed are for implementing a method as described above.
According to the embodiment of the disclosure, the problems that in the related art, when a large amount of data is accessed, huge pressure is generally caused to a background cache server, and meanwhile, as the cached data is generally stored in a byte array, most of the data needs to be subjected to reverse ordering processing, a large amount of server computing resources are consumed, and the overall performance and user experience of a system are further affected are at least partially solved, and therefore the technical effects of relieving the server pressure and improving the data access efficiency can be achieved.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings in which:
FIG. 1 schematically illustrates a system architecture of a data processing method and data processing system according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a data processing method according to an embodiment of the disclosure;
FIG. 3 schematically illustrates a flow chart of a manner of determining a response in accordance with an embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow chart of a manner of determining a response in accordance with another embodiment of the present disclosure;
FIG. 5 schematically illustrates a block diagram of a data processing apparatus according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a third determination module according to an embodiment of the disclosure;
FIG. 7 schematically illustrates a block diagram of a third determination module in accordance with another embodiment of the present disclosure, and
FIG. 8 schematically illustrates a block diagram of a computer system suitable for data processing in accordance with an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a formulation similar to at least one of "A, B or C, etc." is used, in general such a formulation should be interpreted in accordance with the ordinary understanding of one skilled in the art (e.g. "a system with at least one of A, B or C" would include but not be limited to systems with a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
The embodiment of the disclosure provides a data processing method, which comprises the steps of receiving an access request, and determining a target object for the access request based on the access request. Then, the accessed state of the target object is determined. Finally, based on the accessed state, a response mode for the access request is determined. And when the response mode is determined to be the second response mode, acquiring the target object from at least the second storage unit, wherein the mode of acquiring the target object from the first storage unit is different from the mode of acquiring the target object from the second storage unit.
FIG. 1 schematically illustrates a system architecture of a data processing method and data processing system according to an embodiment of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.
As shown in fig. 1, a system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the data processing method provided in the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the data processing apparatus provided by the embodiments of the present disclosure may be generally provided in the server 105. The data processing method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the data processing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
For example, the access request acquired by the embodiment of the present disclosure may be stored in the terminal device 101, 102, 103, the access request is sent to the server 105 by the terminal device 101, 102, 103, the server 105 may determine a target object for which the access request is directed based on the access request and determine an accessed state of the target object, and determine a response manner for the access request based on the accessed state, or the terminal device 101, 102, 103 may determine the target object for which the access request is directed directly based on the access request and determine an accessed state of the target object, and determine a response manner for the access request based on the accessed state. In addition, the access request may also be directly stored in the server 105, where the server 105 determines, directly based on the access request, a target object for which the access request is directed, determines an accessed state of the target object, and determines a response manner for the access request based on the accessed state.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2 schematically illustrates a flow chart of a data processing method according to an embodiment of the present disclosure.
As shown in FIG. 2, the method includes operations S210-S240.
In operation S210, an access request is received.
According to embodiments of the present disclosure, data is typically stored in a storage system. The storage system may be, for example, a cache system or a memory system. After the storage system receives an access request from the outside, the storage system may respond to the access request and provide corresponding data.
In operation S220, a target object for which the access request is directed is determined based on the access request.
In the embodiment of the disclosure, for example, the access request has the object identifier of the target object to be accessed, and the storage system can determine the target object for which the access request is aimed based on the object identifier after receiving the access request.
In operation S230, the accessed state of the target object is determined.
According to embodiments of the present disclosure, for example, a target object can be accessed by multiple threads. First, at least one target thread of the plurality of threads is determined, and second, an accessed state of the target object is determined, the accessed state representing a state in which the target object is accessed by the at least one target thread.
The access state includes, for example, the number of target threads accessing the target object, the access order, and the like. Based on the access status, a manner of response to the access request may be determined.
In operation S240, a response manner to the access request is determined based on the accessed state.
And when the response mode is determined to be the first response mode, acquiring the target object from the first storage unit. And when the response mode is determined to be the second response mode, acquiring the target object from at least the second storage unit. In other words, the second response means may include retrieving the target object from the second storage unit or retrieving the target object from the first storage unit and the second storage unit.
According to an embodiment of the disclosure, the first storage unit includes a memory unit, and the second storage unit includes a cache unit. According to the embodiment of the disclosure, the manner of acquiring the target object from the first storage unit is different from the manner of acquiring the target object from the second storage unit.
For example, the target object may be directly obtained from the memory unit, in other words, the initial data is stored in the memory unit after being subjected to the deserialization process in advance, for example.
For example, the obtaining the target object from at least the second storage unit includes obtaining initial data from the second storage unit, and performing deserialization processing on the initial data to obtain the target object. That is, when the initial data is directly stored in the buffer unit and the target object is obtained from the buffer unit, the initial object in the buffer unit needs to be subjected to deserialization processing to obtain the required target object.
The method and the device for determining the response mode through the accessed state of the target object can directly acquire the target object from the first storage unit when the response mode is the first response mode, data access efficiency is improved without deserializing, calculation pressure of a server can be reduced without deserializing, and system performance of the server is improved. When the response mode is the second response mode, the target object may be acquired from the second storage unit. According to the scheme, the target object is selectively acquired from the first storage unit or the second storage unit through different response modes, so that the pressure of the first storage unit or the second storage unit is relieved.
The above-described operation S240 includes, for example, the following description of fig. 3 and 4, according to an embodiment of the present disclosure.
Fig. 3 schematically illustrates a flow chart of a manner of determining a response in accordance with an embodiment of the present disclosure.
As shown in FIG. 3, when a plurality of target threads are included, the operation S240 includes S241 a-S242 a.
In operation S241a, it is determined that the response mode for the first target thread is the first response mode, and the first target thread is the target thread that accesses the target object first among the plurality of target threads.
According to the embodiment of the disclosure, when a plurality of target threads access a target object, the target thread accessed first is determined to be a first target thread, and the target object is acquired from a memory unit.
In operation S242a, it is determined that the response mode for the second target thread is the second response mode, and the second target thread is a target thread other than the first target thread among the plurality of target threads. For example, a target thread other than the first target thread among the plurality of target threads is determined as the second target thread, and the target object may be acquired from the cache unit. The second target thread acquires the target object from the cache unit, because the first target thread acquires the target object from the memory unit first, and the first target thread may have performed modification operation on the target object in the memory unit, so that the second target thread acquires the target object from the cache unit, and accuracy of the acquired target object is guaranteed.
Specifically, when a plurality of target threads acquire a target object at the same time, the number of times the target object is referenced can be calculated by reference counting, which is mainly embodied on a spatial level (the plurality of target threads access at the same time). For example, when each of the plurality of target threads performs business logic processing, the reference count of the target object is increased by 1, and when execution is completed, the reference count of the target object is decreased by 1. Wherein the object identification and reference count of the target object may be stored in association in the memory unit.
The reference count can reflect the current access condition of the target object by the target thread, so that whether the current target thread is the first accessed thread can be determined through the reference count of the target object, and the response mode of the target thread can be conveniently determined.
Fig. 4 schematically illustrates a flow chart of a manner of determining a response in accordance with another embodiment of the present disclosure.
As shown in FIG. 4, when a target thread is included, the operation S240 may further include S241 b-S242 b.
In operation S241b, an access order in which the target thread accesses the target object is determined.
According to embodiments of the present disclosure, when a target thread accesses a target object, the target thread may access the target object multiple times. Thus, the order of access of the target thread to the target object may be determined first.
In operation S242b, it is determined that the response manner for the first access order is the first response manner, the first access order representing that the target thread accesses the target object for the first time.
For example, when the target thread accesses the target object for the first time, the target object is retrieved from the memory unit.
In operation S243b, it is determined that the response manner for the second access order is the second response manner, and the second access order is the access order of the target thread other than the first access order.
For example, when the target thread is in addition to the first access, the target object may be fetched from the cache unit for subsequent accesses. The reason why the target object is obtained from the cache unit in the subsequent access except the first access is that the target thread may have performed a modification operation on the target object in the memory unit in the first access, so that the target object is obtained from the cache unit in the subsequent access except the first access, so as to ensure the accuracy of the obtained target object.
Specifically, when one target thread accesses the target object a plurality of times, the number of times the target object is referenced can be calculated by reference counting, which is mainly reflected on the time level (one target thread accesses at different times). For example, each time the target thread performs business logic processing, the reference count of the target object is increased by 1, and when the execution is completed, the reference count of the target object is decreased by 1. Wherein the object identification and reference count of the target object may be stored in association in the memory unit.
The reference count can reflect the number of times the target object is currently accessed by the target thread, so that whether the current access of the target thread is the first access or not can be determined through the reference count of the target object, and the response mode can be conveniently determined according to the number of times the target thread is accessed.
According to an embodiment of the present disclosure, acquiring the target object from at least the second storage unit includes acquiring the target object from the second storage unit or acquiring the target object from the first storage unit and the second storage unit. Specifically, the method comprises the following steps (1) - (4):
(1) Attribute information of the target object is determined.
According to an embodiment of the present disclosure, the target object may include, for example, a plurality of sub-objects, each having, for example, corresponding attribute information.
(2) The target object is divided into static data and dynamic data based on attribute information of the target object.
For example, the plurality of sub-objects are divided into static data and dynamic data according to attribute information. For example, the target object includes a sub-object 1, a sub-object 2, and a sub-object 3. The attribute information of the sub-object 1 and the sub-object 2 is first attribute information, which characterizes, for example, the sub-object 1 and the sub-object 2 as style data, wherein the style data is, for example, data that does not need to be modified or updated in real time. The attribute information of the sub-object 3 is, for example, second attribute information, which characterizes the sub-object 3 as data that needs to be modified or updated in real time. Thus, sub-object 1 and sub-object 2 may be partitioned into static data and sub-object 3 into dynamic data. The static data may be stored in a memory unit, for example, and the dynamic data may be stored in a cache unit, for example.
(3) Static data is retrieved from a first storage unit.
(4) And acquiring dynamic data from the second storage unit, and performing deserialization processing on the dynamic data.
According to the embodiments of the present disclosure, the target object read from the memory unit may be reset (reset includes, for example, changing the data into the data before modification), and then the target object is split dynamically and dynamically according to the attribute information, where the main purpose of the splitting is to reduce the deserialization process. Specifically, whether the target object needs to be deserialized or not can be judged based on the reference count of the target object, if the target object needs to be deserialized, dynamic and static splitting can be further performed on the target object to obtain static data and dynamic data, so that the dynamic data can be deserialized in a targeted manner. Static data can directly read the object in the memory unit because no change exists.
In the embodiment of the disclosure, the target object may be preferentially read from the memory unit, and in consideration of possible changes of the target object in the memory unit, the change of the external object may be monitored by using the distributed coordination service, so as to update the data in the memory unit in real time and update the data in the cache unit synchronously. In addition, the data stored in the memory unit may originate from a plurality of system platforms, and the change mechanism may not be uniform, so that the data of each data source type may be managed separately, that is, the memory unit may be managed in a fine granularity according to the data source type.
According to the embodiment of the disclosure, the data in the memory unit is ensured to be accurately updated in real time by preferentially reading the data of the memory unit. By carrying out dynamic and static splitting on the target object, the reverse serialization processing is reduced, and the computing resource is saved.
Fig. 5 schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure.
As shown in fig. 5, the data processing apparatus 500 includes a receiving module 510, a first determining module 520, a second determining module 530, and a third determining module 540.
The receiving module 510 may be configured to receive an access request. The receiving module 510 may, for example, perform operation S210 described above with reference to fig. 2 according to an embodiment of the present disclosure, which is not described herein.
The first determination module 520 may be configured to determine, based on the access request, a target object for which the access request is directed. According to an embodiment of the present disclosure, the first determining module 520 may perform, for example, the operation S220 described above with reference to fig. 2, which is not described herein.
The second determination module 530 may be used to determine the accessed state of the target object.
According to the embodiment of the disclosure, the target object can be accessed by a plurality of threads, and determining the accessed state of the target object comprises determining at least one target thread in the plurality of threads as the thread currently accessing the target object, and determining the accessed state of the target object, wherein the accessed state represents the accessed state of the target object by the at least one target thread.
The second determining module 530 may, for example, perform operation S230 described above with reference to fig. 2, which is not described herein.
The third determination module 540 may be configured to determine a response manner for the access request based on the accessed state.
According to the embodiment of the disclosure, when the response mode is determined to be the first response mode, the target object is acquired from the first storage unit, and when the response mode is determined to be the second response mode, the target object is acquired from at least the second storage unit, wherein the mode of acquiring the target object from the first storage unit is different from the mode of acquiring the target object from the second storage unit.
According to an embodiment of the present disclosure, the third determining module 540 may perform, for example, the operation S240 described above with reference to fig. 2, which is not described herein.
According to the embodiment of the disclosure, acquiring the target object from at least the second storage unit comprises acquiring initial data from the second storage unit, and performing deserialization on the initial data to obtain the target object.
According to the embodiment of the disclosure, acquiring the target object from at least the second storage unit comprises determining attribute information of the target object, dividing the target object into static data and dynamic data based on the attribute information of the target object, acquiring the static data from the first storage unit, acquiring the dynamic data from the second storage unit, and performing deserialization processing on the dynamic data.
According to an embodiment of the disclosure, the first storage unit includes a memory unit, and the second storage unit includes a cache unit.
Fig. 6 schematically illustrates a block diagram of a third determination module according to an embodiment of the disclosure.
As shown in fig. 6, when a plurality of target threads are included, the third determination module 540 may include a first determination submodule 541a and a second determination submodule 542a.
The first determination submodule 541a may be configured to determine that a response manner for a first target thread is a first response manner, where the first thread is a target thread that first accesses a target object among a plurality of target threads. According to an embodiment of the present disclosure, the first determining submodule 541a may perform, for example, the operation S241a described above with reference to fig. 3, which is not described herein.
The second determination submodule 542a may be configured to determine that the response manner for the second target thread is the second response manner, and the second thread is a target thread other than the first target thread among the plurality of target threads. The second determining submodule 542a may, for example, perform operation S242a described above with reference to fig. 3 according to an embodiment of the present disclosure, which is not described here again.
Fig. 7 schematically illustrates a block diagram of a third determination module according to another embodiment of the disclosure.
As shown in fig. 7, when one target thread is included, the third determination module 540 may include a third determination submodule 541b, a fourth determination submodule 542b, and a fifth determination submodule 543b.
The third determination submodule 541b may be used to determine an access order in which the target thread accesses the target object. According to an embodiment of the present disclosure, the third determining submodule 541b may perform, for example, operation S241b described above with reference to fig. 4, which is not described herein.
The fourth determination submodule 542b may be configured to determine that the response manner for the first access order is the first response manner, the first access order indicating that the target thread accessed the target object for the first time. The fourth determination submodule 542b may, for example, perform operation S242b described above with reference to fig. 4 according to an embodiment of the present disclosure, which is not described here again.
The fifth determination submodule 543b may be configured to determine that the response manner for the second access order is the second response manner, and the second access order is an access order of the target thread other than the first access order. The fifth determination submodule 543b may perform, for example, operation S243b described above with reference to fig. 4 according to an embodiment of the present disclosure, which is not described herein.
Any number of modules, sub-modules, units, sub-units, or at least some of the functionality of any number of the sub-units according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented as split into multiple modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or in any other reasonable manner of hardware or firmware that integrates or encapsulates the circuit, or in any one of or a suitable combination of three of software, hardware, and firmware. Or one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be at least partially implemented as computer program modules, which, when executed, may perform the corresponding functions.
For example, any of the receiving module 510, the first determining module 520, the second determining module 530, and the third determining module 540, the first determining sub-module 541a, the second determining sub-module 542a, the third determining sub-module 541b, the fourth determining sub-module 542b, and the fifth determining sub-module 543b may be combined in one module to be implemented, or any one of the modules may be split into a plurality of modules. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules. According to embodiments of the present disclosure, at least one of the receiving module 510, the first determining module 520, the second determining module 530, and the third determining module 540, the first determining sub-module 541a, the second determining sub-module 542a, the third determining sub-module 541b, the fourth determining sub-module 542b, and the fifth determining sub-module 543b may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in any one of or in any suitable combination of three of software, hardware, and firmware implementations. Or at least one of the receiving module 510, the first determination module 520, the second determination module 530 and the third determination module 540, the first determination sub-module 541a, the second determination sub-module 542a, the third determination sub-module 541b, the fourth determination sub-module 542b and the fifth determination sub-module 543b may be at least partially implemented as a computer program module, which may perform the respective functions when being run.
FIG. 8 schematically illustrates a block diagram of a computer system suitable for data processing in accordance with an embodiment of the present disclosure. The computer system illustrated in fig. 8 is merely an example, and should not be construed as limiting the functionality and scope of use of the embodiments of the present disclosure.
As shown in fig. 8, a computer system 800 according to an embodiment of the present disclosure includes a processor 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. The processor 801 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 801 may also include on-board memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the disclosure.
In the RAM 803, various programs and data required for the operation of the system 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 802 and/or the RAM 803. Note that the program may be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, the system 800 may further include an input/output (I/O) interface 805, the input/output (I/O) interface 805 also being connected to the bus 804. The system 800 may also include one or more of an input portion 806 including a keyboard, mouse, etc., an output portion 807 including a display such as a Cathode Ray Tube (CRT), liquid Crystal Display (LCD), etc., and speakers, etc., a storage portion 808 including a hard disk, etc., and a communication portion 809 including a network interface card such as a LAN card, modem, etc., connected to the I/O interface 805. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.
According to embodiments of the present disclosure, the method flow according to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
The present disclosure also provides a computer-readable storage medium that may be included in the apparatus/device/system described in the above embodiments, or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a computer-nonvolatile computer-readable storage medium, which may include, for example, but is not limited to, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 802 and/or RAM 803 and/or one or more memories other than ROM 802 and RAM 803 described above.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.
Claims (9)
1. A data processing method, comprising:
Receiving an access request;
determining a target object aimed by the access request based on the access request;
Determining an accessed state of the target object, the accessed state including a number of target threads accessing the target object and an order of access, and
Determining a response mode for the access request based on the accessed state;
When the response mode is determined to be a first response mode, acquiring the target object from a first storage unit;
when the response mode is determined to be the second response mode, the target object is acquired from at least the second storage unit,
The first storage unit comprises a memory unit, the memory unit stores initial data subjected to reverse serialization processing, the second storage unit comprises a cache unit, the cache unit stores initial data not subjected to reverse serialization processing, and the mode of acquiring the target object from the first storage unit is different from the mode of acquiring the target object from the second storage unit.
2. The method of claim 1, wherein the target object is accessible by a plurality of threads, the determining the accessed state of the target object comprising:
determining at least one target thread of the plurality of threads, and
An accessed state of the target object is determined, the accessed state representing a state of the target object being accessed by the at least one target thread.
3. The method of claim 2, wherein when comprising a plurality of target threads, the determining a manner of response to the access request comprises:
determining a response mode for a first target thread as the first response mode, wherein the first target thread is the target thread which accesses the target object first in the target threads, and
Determining a response mode for a second target thread as the second response mode, wherein the second target thread is a target thread except the first target thread in the target threads.
4. A method according to claim 2 or 3, wherein when a target thread is included, said determining the manner of response to the access request comprises:
determining an access order of the target thread to the target object;
determining a response means for a first access order to be the first response means, the first access order representing the first access of the target thread to the target object, and
Determining a response mode for a second access order to be the second response mode, wherein the second access order is the access order of the target thread except the first access order.
5. The method of claim 1, wherein the retrieving the target object from at least a second storage unit comprises:
acquiring initial data from the second storage unit, and
And performing deserialization processing on the initial data to obtain the target object.
6. The method of claim 1, wherein the retrieving the target object from at least a second storage unit comprises:
determining attribute information of the target object;
Dividing the target object into static data and dynamic data based on attribute information of the target object;
Acquiring the static data from the first storage unit, and
And acquiring the dynamic data from the second storage unit, and performing deserialization processing on the dynamic data.
7. A data processing apparatus comprising:
The receiving module receives the access request;
The first determining module is used for determining a target object aimed by the access request based on the access request;
a second determination module for determining the accessed state of the target object, the accessed state including the number of target threads accessing the target object and the access order, and
A third determining module that determines a response mode for the access request based on the accessed state;
When the response mode is determined to be a first response mode, acquiring the target object from a first storage unit;
when the response mode is determined to be the second response mode, the target object is acquired from at least the second storage unit,
The first storage unit comprises a memory unit, the memory unit stores initial data subjected to reverse serialization processing, the second storage unit comprises a cache unit, the cache unit stores initial data not subjected to reverse serialization processing, and the mode of acquiring the target object from the first storage unit is different from the mode of acquiring the target object from the second storage unit.
8. A data processing system, comprising:
One or more processors;
a memory for storing one or more programs,
Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1 to 6.
9. A computer readable storage medium storing computer executable instructions which, when executed, are adapted to carry out the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910948681.7A CN112579282B (en) | 2019-09-30 | 2019-09-30 | Data processing method, device, system, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910948681.7A CN112579282B (en) | 2019-09-30 | 2019-09-30 | Data processing method, device, system, and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112579282A CN112579282A (en) | 2021-03-30 |
| CN112579282B true CN112579282B (en) | 2025-06-13 |
Family
ID=75117211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910948681.7A Active CN112579282B (en) | 2019-09-30 | 2019-09-30 | Data processing method, device, system, and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112579282B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117093408B (en) * | 2023-10-20 | 2024-01-23 | 恒生电子股份有限公司 | Data processing method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108132958A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | A kind of multi-level buffer data storage, inquiry, scheduling and processing method and processing device |
| CN110249303A (en) * | 2017-02-16 | 2019-09-17 | 华为技术有限公司 | System and method for reducing reference count expense |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014142861A1 (en) * | 2013-03-14 | 2014-09-18 | Intel Corporation | Memory object reference count management with improved scalability |
| US10592681B2 (en) * | 2017-01-10 | 2020-03-17 | Snowflake Inc. | Data sharing in a multi-tenant database system |
| CN110209597B (en) * | 2019-05-24 | 2021-10-15 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing access request |
-
2019
- 2019-09-30 CN CN201910948681.7A patent/CN112579282B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108132958A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | A kind of multi-level buffer data storage, inquiry, scheduling and processing method and processing device |
| CN110249303A (en) * | 2017-02-16 | 2019-09-17 | 华为技术有限公司 | System and method for reducing reference count expense |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112579282A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11573816B1 (en) | Prefetching and managing container images using cluster manifest | |
| US10789085B2 (en) | Selectively providing virtual machine through actual measurement of efficiency of power usage | |
| CN110865888B (en) | Resource loading method and device, server and storage medium | |
| US9606785B2 (en) | Detecting deployment conflicts in heterogeneous environments | |
| CN109150930B (en) | Configuration information loading method and device, service processing method and device | |
| US10331669B2 (en) | Fast query processing in columnar databases with GPUs | |
| US9870171B2 (en) | Affinity-aware parallel zeroing of memory for initialization of large pages in non-uniform memory access (NUMA) servers | |
| CN113688160B (en) | Data processing method, processing device, electronic device and storage medium | |
| US8935516B2 (en) | Enabling portions of programs to be executed on system z integrated information processor (zIIP) without requiring programs to be entirely restructured | |
| CN112988604A (en) | Object testing method, testing system, electronic device and readable storage medium | |
| CN117762898A (en) | Data migration method, device, equipment and storage medium | |
| CN117573359A (en) | A computing framework management system and method based on heterogeneous clusters | |
| CN115630033A (en) | Log information processing method and device, electronic equipment and medium | |
| CN112579282B (en) | Data processing method, device, system, and computer-readable storage medium | |
| CN108062224B (en) | Data reading and writing method, device and computing device based on file handle | |
| CN110286981A (en) | The display methods and display system of the use state of virtual cloud desktop server | |
| CN112114962A (en) | Method and device for allocating memory | |
| Chu et al. | Designing high-performance in-memory key-value operations with persistent gpu kernels and openshmem | |
| CN107291628B (en) | Method and apparatus for accessing data storage device | |
| US11263047B2 (en) | Metadata management for multi-core resource manager | |
| CN111444017A (en) | Multimedia data processing method, device and system, electronic equipment and storage medium | |
| CN117667632A (en) | Cluster performance evaluation methods, devices, equipment and media | |
| WO2017011021A1 (en) | Systems and methods facilitating reduced latency via stashing in systems on chips | |
| CN111859403B (en) | Dependency vulnerability determination method and device, electronic equipment and storage medium | |
| CN114595061A (en) | Resource configuration method and apparatus, electronic device and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |