CN110007869B - Memory data copying method, device, equipment and computer storage medium - Google Patents

Abstract

The invention discloses a method for copying memory data, comprising: receiving a memory data copy instruction for copying data of a source IOB to a destination IOB; and copying the logical page number of a physical page to be copied in the source IOB to The destination IOB, and the reference count of the physical page to be copied in the source IOB is incremented by one. It can be seen that when copying the data of the physical page corresponding to the source IOB to the destination IOB, it is not necessary to copy the real data corresponding to the source IOB to the destination IOB, but only need to copy the logical number of the physical page corresponding to the source IOB to the destination IOB. In this way, the reference relationship between the source IOB and the destination IOB can be established, thereby reducing the number of copies of memory data and improving the IO processing efficiency; the invention also discloses a memory data copying device, equipment and a computer storage medium , the above-mentioned technical effects can also be achieved.

Description

A method, device, device and computer storage medium for copying memory data

technical field

The present invention relates to the technical field of memory copying, and more particularly, to a method, apparatus, device and computer storage medium for copying memory data.

Background technique

In a storage system, memory management is usually a general module. Memory management affects the IO (Input Output, input and output) processing flow of the entire storage system. If the memory management efficiency is not high enough, the performance of the entire system will be affected. At present, when copying memory, it is necessary to copy real data. This memory copy method is copied many times. For systems with hundreds of thousands or millions of IOPS (IO per second, input and output per second), each IO The number of memory copies during processing directly affects the IO processing efficiency.

Therefore, how to reduce the number of copies of memory data and improve the IO processing efficiency is a problem that needs to be solved by those skilled in the art.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method, device, equipment and computer storage medium for copying memory data, so as to reduce the number of times of copying memory data and improve IO processing efficiency.

To achieve the above purpose, the embodiments of the present invention provide the following technical solutions:

A method for copying memory data, including:

Receive the memory data copy instruction for copying the data of the source IOB to the destination IOB;

According to the memory data copy instruction, the logical page number of the physical page to be copied in the source IOB is copied to the destination IOB, and the reference count of the physical page to be copied in the source IOB is incremented by one.

Wherein, according to the memory data copy instruction, copying the logical page number of the physical page to be copied in the source IOB to the destination IOB, including:

Determine whether the destination IOB has the logical page number of the corresponding physical page;

If there is no corresponding logical page number, copy the logical page number of the physical page to be copied in the source IOB to the destination IOB;

If there is a corresponding logical page number, then determine whether the physical page to be copied in the source IOB is the entire physical page; if it is the entire physical page, decrement the reference count of the physical page corresponding to the destination IOB by one, and add all The logical page number of the physical page to be copied in the source IOB is copied to the destination IOB.

Wherein, each IOB includes: a bitmap used to indicate whether each physical page corresponding to this IOB stores data, a logical page number used to indicate the physical page of this IOB that stores data, and a connection used to point to the next IOB symbol.

Wherein, adding one to the reference count of the physical page to be copied in the source IOB includes:

Add one to the reference count recorded in the page description table and the physical page to be copied in the source IOB; wherein, the page description table records the correspondence between the logical page number of each physical page and the page structure struct page relation.

A memory data copy device, comprising:

The instruction receiving module is used to receive the memory data copy instruction for copying the data of the source IOB to the destination IOB;

a data copy module, configured to copy the logical page number of the physical page to be copied in the source IOB to the destination IOB according to the memory data copy instruction;

A reference count update module, configured to increment the reference count of the physical page to be copied in the source IOB by one.

Wherein, the data copying module includes:

a first judging unit for judging whether the destination IOB has the logical page number of the corresponding physical page;

a first copying unit, configured to copy the logical page number of the physical page to be copied in the source IOB to the destination IOB when there is no corresponding logical page number;

a second judging unit, configured to judge whether the physical page to be copied in the source IOB is the entire physical page when there is a corresponding logical page number;

a second copy unit, configured to copy the logical page number of the physical page to be copied in the source IOB to the destination IOB when the physical page to be copied in the source IOB is an entire physical page;

The first updating unit is configured to decrement the reference count of the physical page corresponding to the destination IOB by one when the physical page to be copied in the source IOB is the entire physical page.

Wherein, the data copy module also includes:

a second update unit, configured to decrement the reference count of the physical page corresponding to the destination IOB by one when the physical page to be copied in the source IOB is not the entire physical page;

a physical page allocation unit for allocating a new physical page to the destination IOB;

The third copy unit is configured to copy both the original data in the physical page corresponding to the destination IOB and the data of the physical page to be copied in the source IOB to the new physical page.

A memory data copy device, comprising:

memory for storing computer programs;

The processor is configured to implement the steps of the above-mentioned method for copying memory data when executing the computer program.

A computer-readable storage medium stores a computer program on the computer-readable storage medium, and when the computer program is executed by a processor, implements the steps of the above-mentioned method for copying memory data.

It can be seen from the above solutions that a method for copying memory data provided by an embodiment of the present invention includes: receiving a memory data copying instruction for copying data of a source IOB to a destination IOB; The logical page number of the physical page to be copied is copied to the destination IOB, and the reference count of the physical page to be copied in the source IOB is incremented by one.

It can be seen that, in this embodiment, when copying the data of the physical page corresponding to the source IOB to the destination IOB, it is not necessary to copy the real data corresponding to the source IOB to the destination IOB, but only the logical number of the physical page corresponding to the source IOB needs to be copied It is enough to reach the destination IOB, and add one to the reference count of the physical page to be copied in the source IOB. In this way, the reference relationship between the source IOB and the destination IOB can be established, thereby reducing the number of copies of memory data. Improve IO processing efficiency;

The invention also discloses a memory data copying device, equipment and computer storage medium, which can also achieve the above technical effects.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a schematic flowchart of a method for copying memory data disclosed in an embodiment of the present invention;

2 is a schematic flowchart of another method for copying memory data disclosed in an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a device for copying memory data disclosed in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Embodiments of the present invention disclose a method, apparatus, device and computer storage medium for copying memory data, so as to reduce the number of times of copying memory data and improve IO processing efficiency.

Referring to FIG. 1, a method for copying memory data provided by an embodiment of the present invention includes:

S101, receiving a memory data copy instruction for copying the data of the source IOB to the destination IOB;

Specifically, an IOB (input Output Buffer) is an input and output buffer, and each IOB corresponds to a predetermined number of physical pages; in this embodiment, memory management is specifically performed on the basis of Linux memory management through a kernel module. The kernel module is loaded when the device is started. When allocating physical pages to the IOB, it allocates a specified number of physical pages to the IOB by calling the physical page allocation interface provided by the Linux kernel, and uses the logical page number starting from 0. Coding; the correspondence between the logical page number of each physical page and the Linux struct page structure is maintained through the page description table page_descriptor, and the struct page structure with the logical page number i can be obtained through pd[i].

In this embodiment, the memory data copy instruction is an instruction to copy the data to be copied stored in the physical page corresponding to the source IOB (src_iob) to the physical page corresponding to the destination IOB (dst_iob); In the scheme, each IOB includes: a bitmap used to indicate whether each physical page corresponding to this IOB stores data, a logical page number used to indicate the physical page of this IOB that stores data, and a bitmap used to point to the next IOB. Joiner. Specifically, the IOB structure is as follows:

It can be seen from the IOB structure that the bitmap in the structure is a bitmap, which is used to maintain whether each physical page corresponding to the IOB structure is valid. The bitmap is 8 uint8_t, and each bit corresponds to a sector in pages[8] Whether the sector has data, therefore, the bitmap can reflect whether the physical page corresponding to the IOB stores data; pages[8] in the structure is used to indicate the logical page number of up to 8 pages corresponding to the IOB. It can be seen that an IOB has the most It has 8 pages, that is, 32KB of memory; next is the connector, which is used to point to the next IOB, which is used to achieve more than 32KB of memory.

S102. According to the memory data copy instruction, copy the logical page number of the physical page to be copied in the source IOB to the destination IOB, and increment the reference count of the physical page to be copied in the source IOB by one.

In this embodiment, when the memory data copy instruction copy(dst_iob, src_iob) is received, the source IOB and the destination IOB have been determined. Therefore, when copying the data in the source IOB to the destination IOB, in order to reduce the time of copying the real data In this embodiment, only the logical page number corresponding to the data to be copied is copied to the destination IOB, and the reference relationship between the source IOB and the destination IOB is established in this way, so that when accessing the data corresponding to the destination IOB, the Access the data in the source IOB directly according to the logical page number of the destination IOB; in order to record that the data in the source IOB has been referenced by other IOBs, after copying the logical page number of the physical page to be copied in the source IOB to the destination IOB, it is necessary to The reference count of the physical page to be copied in the source IOB is incremented by one, indicating that the physical page is referenced by other IOBs.

It should be noted that, in this embodiment, the reference count of the physical page is maintained in the page descriptor table, which is several bits of pd[i]. Therefore, the reference count of the physical page to be copied in the source IOB is added First, it may include: adding one to the reference count of the physical page to be copied in the source IOB recorded in the page description table; wherein the logical page number of each physical page and the page structure struct page are recorded in the page description table. Correspondence between.

It can be seen that the page description table not only records the correspondence between the logical page number of each physical page and the page structure structpage, but also records the reference count of each physical page, so that the physical page is referenced or released. After that, it can be achieved directly by changing the reference count.

To sum up, in this embodiment, the physical page allocation function provided by the Linux kernel can be called through the kernel module to allocate physical pages to the IOB, and the corresponding relationship between the physical page of each logical page number and the linuxstruct page can be maintained through the page_descriptor table. In addition, the structure of the IOB can be represented as a piece of memory. The structure uses bitmap, pages, next, etc. to realize page allocation and management. When copying memory, the copy of the real data is replaced by the copy of the logical page number, which realizes the copying of the memory data. Zero copy can effectively reduce the number of memory copies, improve system performance, and improve system IOPS.

Referring to FIG. 2, a method for copying memory data provided by an embodiment of the present invention includes:

S201, receiving a memory data copy instruction for copying the data of the source IOB to the destination IOB;

S202, determine whether the destination IOB has the logical page number of the corresponding physical page; if not, execute S203, and if so, execute S204;

S203, copying the logical page number of the physical page to be copied in the source IOB to the destination IOB;

S204, determine whether the physical page to be copied in the source IOB is the entire physical page; if so, execute S205; if not, execute S206;

S205. Decrease the reference count of the physical page corresponding to the destination IOB by one, and copy the logical page number of the physical page to be copied in the source IOB to the destination IOB.

S206: Decrease the reference count of the physical page corresponding to the destination IOB by one, allocate a new physical page to the destination IOB, and combine the original data in the physical page corresponding to the destination IOB and the data of the physical page to be copied in the source IOB , are copied to the new physical page.

In this embodiment, for memory data copy, a specific copy strategy is provided, for example:

1. If the source IOB (src_iob) has a page, and the destination IOB (dst_iob) has no page, then execute S203 directly; the src_iob has a page representing that the pages[8] corresponding to src_iob has a page number, that is, the data to be copied in the physical page, Because the memory data copy instruction is executed, src_iob is generally in a paged state; dst_iob has no pages, which means that dst_iob has not allocated a physical page, that is, there is no valid page number in pages[8] of dst_iob; in this case, you can Directly copy the logical page number in src_iob.pages to dst_iob.pages, and increment the reference count of the corresponding physical page in src_iob by 1. In this case, only the physical page number is copied, and there is no real memory copy, which achieves zero copy.

2. If src_iob has pages, dst_iob has pages, and the entire page will be covered, execute S205; the dst_iob has pages, which means that dst_iob originally has some data, and these data may refer to the page numbers of other iobs. In this case, if the physical page to be copied in src_iob is the entire physical page, the corresponding page referenced by dst_iob needs to be released, that is, the reference count of the physical page corresponding to dst_iob is decremented by 1, and then the logical page number in src_iob is copied to dst_iob , and add 1 to the reference count of src_iob. In this case, only physical page copies are made, and no real memory data copies are made, and zero copy is achieved.

3. If src_iob has a page, dst_iob has a page, but only part of the content of the page is copied (in the IO process in the storage system, the memory copy is in units of sectors (512 bytes), there may be only a few sectors copied, but operation without copying the entire page), then execute S206;

That is to say, if pages[8] of dst_iob has a page number, that is, this iob originally has some data, and the data may refer to the page number of other iob, for example:

dst_iob.pages[0]----iobx.pages[3]

dst_iob references a page of iobx, and now it is necessary to copy data to dst_iob, and the copied data is not a complete page, but several sectors of a page. At this time, data cannot be copied directly to the pages of dst_iob, because this page It is a shared page of others. Once the data is copied directly, the page of another iob will be modified; therefore, it is necessary to first cancel the reference of dst_iob to this page, and then allocate a new page to dst_iob, which copies the contents of the old page first. , and then copy the data of src_iob into this new page. It can be seen that in this way, the zero copy is invalid, and a real page copy needs to be done.

From the above, it can be seen that in this scheme, according to the two conditions of whether the destination IOB has a logical page number, and whether the physical page to be copied in the source IOB is the entire physical page, a variety of copy strategies are proposed, as long as the source IOB is satisfied. In the condition that the physical page to be copied is the entire physical page, when a memory copy occurs, only the physical page number needs to be copied, and no real memory copy is performed, which improves the efficiency of memory copy.

The following describes the device for copying memory data provided by the embodiments of the present invention. The device for copying memory data described below and the method for copying memory data described above may refer to each other.

Referring to FIG. 3, an apparatus for copying memory data provided by an embodiment of the present invention includes:

The instruction receiving module 100 is used to receive the memory data copy instruction for copying the data of the source IOB to the destination IOB;

A data copy module 200, configured to copy the logical page number of the physical page to be copied in the source IOB to the destination IOB according to the memory data copy instruction;

The reference count updating module 300 is configured to increment the reference count of the physical page to be copied in the source IOB by one.

Wherein, the data copying module includes:

a first judging unit for judging whether the destination IOB has the logical page number of the corresponding physical page;

a first copying unit, configured to copy the logical page number of the physical page to be copied in the source IOB to the destination IOB when there is no corresponding logical page number;

a second judging unit, configured to judge whether the physical page to be copied in the source IOB is the entire physical page when there is a corresponding logical page number;

a second copy unit, configured to copy the logical page number of the physical page to be copied in the source IOB to the destination IOB when the physical page to be copied in the source IOB is an entire physical page;

The first updating unit is configured to decrement the reference count of the physical page corresponding to the destination IOB by one when the physical page to be copied in the source IOB is the entire physical page.

Wherein, the data copy module also includes:

a second update unit, configured to decrement the reference count of the physical page corresponding to the destination IOB by one when the physical page to be copied in the source IOB is not the entire physical page;

a physical page allocation unit for allocating a new physical page to the destination IOB;

The third copy unit is configured to copy both the original data in the physical page corresponding to the destination IOB and the data of the physical page to be copied in the source IOB to the new physical page.

Wherein, each IOB includes: a bitmap used to indicate whether each physical page corresponding to this IOB stores data, a logical page number used to indicate the physical page of this IOB that stores data, and a connection used to point to the next IOB symbol.

Wherein, the reference count update module is specifically configured to add one to the reference count recorded in the page description table and the physical page to be copied in the source IOB; wherein, the page description table records the reference count of each physical page The correspondence between the logical page number and the page structure struct page.

The embodiment of the present invention also discloses a memory data copy device, comprising:

memory for storing computer programs;

The processor is configured to implement the steps of the method for copying memory data as described in the foregoing embodiments when executing the computer program.

An embodiment of the present invention further discloses a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for copying memory data described in the foregoing embodiment is implemented A step of.

Wherein, the storage medium may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other various storage media that can store program codes medium.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (8)

1. A memory data copying method is characterized by comprising the following steps:

receiving a memory data copying instruction for copying the data of the source IOB to the target IOB;

copying the logical page number of the physical page to be copied in the source IOB to the target IOB according to the memory data copying instruction, and adding one to the reference count of the physical page to be copied in the source IOB;

copying a logical page number of a physical page to be copied in the source IOB to the destination IOB according to the memory data copy instruction, including:

judging whether the destination IOB has a logical page number of a corresponding physical page;

if the corresponding logical page number does not exist, copying the logical page number of the physical page to be copied in the source IOB to the destination IOB;

if the corresponding logical page number exists, judging whether the physical page to be copied in the source IOB is the whole physical page or not; and if the physical page is the whole physical page, subtracting one from the reference count of the physical page corresponding to the target IOB, and copying the logical page number of the physical page to be copied in the source IOB to the target IOB.

2. The method for copying memory data according to claim 1, wherein if the physical page to be copied in the source IOB is not an entire physical page, the method for copying memory data further comprises:

and subtracting the reference count of the physical page corresponding to the target IOB by one, allocating a new physical page to the target IOB, and copying the original data in the physical page corresponding to the target IOB and the data of the physical page to be copied in the source IOB to the new physical page.

3. The memory data copying method of claim 1 or 2, wherein each IOB comprises: a bitmap indicating whether or not each physical page corresponding to the IOB stores data, a logical page number indicating the physical page of the IOB storing data, and a connector for pointing to the next IOB.

4. The memory data copying method according to claim 1 or 2, wherein the increasing the reference count of the physical page to be copied in the source IOB by one includes:

adding one to a reference count recorded in a page description table and of a physical page to be copied in the source IOB; and recording the corresponding relation between the logical page number of each physical page and the structure page of the page in the page description table.

5. An apparatus for copying memory data, comprising:

the instruction receiving module is used for receiving a memory data copying instruction for copying the data of the source IOB to the target IOB;

the data copying module is used for copying the logical page number of the physical page to be copied in the source IOB to the destination IOB according to the memory data copying instruction;

a reference count update module, configured to increment a reference count of a physical page to be copied in the source IOB by one;

wherein the data copy module comprises:

a first judging unit, configured to judge whether the destination IOB has a logical page number of a corresponding physical page;

a first copy unit, configured to copy, when there is no corresponding logical page number, a logical page number of a physical page to be copied in the source IOB to the destination IOB;

a second determining unit, configured to determine whether a physical page to be copied in the source IOB is a whole physical page when there is a corresponding logical page number;

a second copy unit, configured to copy, to the destination IOB, a logical page number of a physical page to be copied in the source IOB when the physical page to be copied in the source IOB is an entire physical page;

and the first updating unit is used for reducing the reference count of the physical page corresponding to the target IOB by one when the physical page to be copied in the source IOB is the whole physical page.

6. The memory data copying device according to claim 5, wherein the data copying module further comprises:

a second updating unit, configured to, when a physical page to be copied in the source IOB is not an entire physical page, decrement a reference count of a physical page corresponding to the destination IOB by one;

a physical page allocation unit for allocating a new physical page to the destination IOB;

and the third copy unit is used for copying the original data in the physical page corresponding to the destination IOB and the data of the physical page to be copied in the source IOB to the new physical page.

7. An in-memory data copying apparatus, comprising:

a memory for storing a computer program;

a processor for implementing the steps of the memory data copying method according to any one of claims 1 to 4 when executing the computer program.

8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the memory data copying method according to any one of claims 1 to 4.

Images