US20150169445A1

US20150169445A1 - Virtual grouping of memory

Info

Publication number: US20150169445A1
Application number: US14/104,438
Authority: US
Inventors: Timothy J. Dell; Prasanna Jayaraman; Anil B. Lingambudi; Girisankar Paulraj
Original assignee: International Business Machines Corp
Current assignee: International Business Machines Corp
Priority date: 2013-12-12
Filing date: 2013-12-12
Publication date: 2015-06-18
Also published as: US20150169446A1; US9600187B2; US20160357459A1

Abstract

The present disclosure includes identifying, in a memory system, a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory; determining a memory segment size from the capacities of the memory modules; identifying a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and creating a virtual group that includes the first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.

Description

TECHNICAL FIELD

This disclosure relates to virtual memory grouping. In particular, it relates to virtual memory grouping across memory channels.

BACKGROUND

Main memory in a system can be setup in various physical configurations to optimize power consumption and performance. Memory may be mirrored for redundancy to mitigate DIMM failures. Fully mirrored memory, while being fully redundant, would cut the capacity of the memory in half vs. non-mirrored memory. However, memory may be selectively mirrored so that a portion of the memory is mirrored, such as that containing information critical to program operation, and the remainder of the memory is non-mirrored.

SUMMARY

The present disclosure includes a method, comprising: identifying, in a memory system, a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory; determining a memory segment size from the capacities of the memory modules; identifying a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and creating a virtual group that includes the first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.
The present disclosure further includes a method, comprising identifying a capacity for each of a plurality of memory modules for a first memory channel and a second memory channel in a memory system; determining a first portion of memory modules and a second portion of memory modules from the capacities of the memory modules for each of the first and second memory channels; creating a first virtual group that includes the first portion of the memory modules from the first memory channel and the first portion of memory modules from the second memory channel; and creating a second virtual group that includes the second portion of the memory modules from the first memory channel and the second portion of memory modules from the second memory channel.
The present disclosure further includes a computer system, comprising service processor firmware configured to identify a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory; determine a memory segment size from the capacities of the memory modules; identify a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and create a virtual group that includes first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings included in the present application are incorporated into, and form part of, the specification. They illustrate embodiments of the present invention and, along with the description, serve to explain the principles of the invention. The drawings are only illustrative of typical embodiments of the invention and do not limit the invention.

FIG. 1 is a flowchart of a method for creating virtual groups of memory, according to embodiments of the disclosure.

FIG. 2A is a diagram of a memory system, according to embodiments of the disclosure.

FIG. 2B is a diagram of memory allocation in a memory system, according to embodiments of the disclosure.

FIG. 3 is an example diagram of a memory system, according to embodiments of the disclosure.

FIG. 4 is an example diagram of grouping of memory segments for the system of FIG. 3, according to embodiments of the disclosure.

FIG. 5A is a diagram of memory from memory controller segments in physical groups for physical grouping, according to embodiments of the disclosure.

FIG. 5B is a diagram of memory from memory controller segments in virtual groups for virtual grouping 1, according to embodiments of the disclosure.

FIG. 5C is a diagram of memory from memory controller segments in virtual groups for virtual grouping 2, according to embodiments of the disclosure.

DETAILED DESCRIPTION

Memory modules in a memory system may be organized into groups along memory channels behind memory controllers and memory buffers to match the internal throughput of the system bus of a computer system. For example, a memory controller may have multiple memory buffers, each memory buffer having multiple memory channels leading to one or more DIMMs. Memory modules may be populated into memory channels in groups of similar sizes of memory to enable memory minoring and memory interleaving, while unequal sizes of memory may be populated to form different groups. An operating system or hypervisor may allocate memory for an application as a physical group across multiple memory modules so that those memory modules may be accessed at the same time. The memory map for a particular physical group will likely be a single contiguous space.
Accessing memory according to the physical organization of the memory behind the memory controller may compromise memory throughput and affinity. Access to the memory groups may be limited by the physical characteristics of the memory, as the grouping is performed along physical boundaries. Mirrored data may be allocated to physical memory groups of equal sizes behind a memory buffer, which may reduce the memory flexibility and utilization of the system. Memory grouped according to a particular bandwidth may not be optimized for the way in which the memory is actually utilized by applications.
According to embodiments of the disclosure, memory may be virtually grouped and accessed to improve memory throughput and affinity. Memory segments of an identified memory size may be grouped virtually according to considerations such as the minimum memory module capacity, application priority, criticality, or memory access characteristics, and performance data for various memory module configurations. Virtual groups accessed by multiple memory controllers may scale the throughput high, as a greater number of memory modules may be accessed in parallel. Virtual groups of equal memory segments may be created from the same physical memory group or from physical memory groups of different sizes and behind multiple memory controllers. Memory presented as and allocated through virtual groups may be accessed symmetrically and diffusely through operations such as memory interleaving.
Performance and configuration information collected from memory in the virtual groups may be utilized by systems and users to improve physical memory access and application performance. The performance of memory in various virtual configurations may be used to create improved physical groupings of the memory modules through memory configuration and population rules. Data may be allocated to a particular memory group based on the priority of the application's data or the memory access characteristics of the application. Virtual groups may also support selective memory mirroring by identifying data to be mirrored and configuring virtual memory groups accordingly.

Method

FIG. 1 is a flowchart of a method for creating virtual groups of memory, according to embodiments of the disclosure. The memory capacities of one or more memory modules in the memory system may be determined, as in 100. Memory from the memory modules may be grouped into virtual groups, as in 110. Once the virtual groups are created, memory from the virtual groups may be allocated to applications, as in 120. The memory modules having memory in a virtual memory group may be monitored for performance, power consumption, and other usage statistics, as in 130.
To determine memory capacities of the memory modules, as in 100, the system may perform an initial program load, as in 101. Once the system has been initialized, vital product data for the memory modules may be collected, as in 102. Vital product data may include hardware information, such as serial number and capacity. The vital product data may be determined for the system or collected from a vital product data storage location on the system. The collection of vital product data may be performed by a system such as the host system of the memory system, a remote system, or a field replaceable unit.
Memory from the memory modules may be grouped into virtual memory groups, as in 110. The capacities of the memory modules may be scanned and evaluated from the vital product data of the memory modules, as in 111. A memory segment size (or portion) may be determined from the scanned memory module capacities, as in 112, according to the memory access characteristics of the memory module. In certain embodiments, the memory segment size may be the smallest independently operated memory unit for one or more physical memory groups, such as the smallest memory module size or a rank size in a memory module. The memory segment size may be evaluated for memory modules across two or more memory channels. For example, if a first memory channel has two 1 GB DIMMs, a second memory channel has four 1 GB DIMMs, the memory segment size may be the smallest memory module size in common between the two memory channels. In this example, 1 GB may be the memory segment size; alternately, 2 GB may be the memory segment size, as both groups have at least two 1 GB DIMMs.
Memory segments of the memory segment size may be identified for memory modules in the memory system, as in 113, and used to create a virtual memory group, as in 114. In the example above, the two 1 GB DIMMs from the first channel and two 1 GB DIMMs from the second channel may be grouped into a first group. If a third memory channel has two 1 GB DIMMs, the remaining two 1 GB DIMMs from the second memory channel may be grouped with the two 1 GB DIMMs from the third memory channel.
The virtual groups may contain memory from memory channels that is less than the total memory capacity of the memory channels. In some embodiments, a memory buffer or controller having two memory channels may have memory grouped into two or more virtual groups, each group containing memory from both memory channels. In other embodiments, a virtual group may have memory from memory channels of different memory capacity connected to two different controllers. In other embodiments, a virtual group may have memory from memory channels having different memory capacities.
Creation of virtual groups may be informed by performance information of the memory modules, as in 131. For example, if an executing processor in a distributed shared memory system has greater processor affinity to a certain group of DIMMs, those DIMMs may be used for the virtual group assigned to an application executed on that processor.
Once the virtual groups are created, memory from the virtual groups may be allocated to applications, as in 120. The application's priority, criticality, or memory access characteristics may inform the operating system or hypervisor as to which virtual group may be allocated to the application, as in 121. For example, if an application is known to have fairly linear memory access or is non-critical, a virtual group providing deeper memory access or limited to fewer memory modules may be used, so that memory for applications with more diffuse memory access or requiring greater fault tolerance may use virtual groups having a greater usage of memory modules. The virtual group may be targeted by or allocated to the application, as in 122. The application's performance may be monitored to determine the appropriateness of the virtual group for that application.
The memory modules in a virtual group may be monitored for performance and power consumption, as in 130. Information of the memory modules, such as performance and power consumption, may be collected, as in 131. The information may be used as feedback to determine memory module configuration options, as in 132. These memory module configuration options may be used to physically configure or populate the memory modules so as to be improve memory access, such as by increasing memory affinity or optimizing throughput. These configuration options may be used as DIMM plug rules when the DIMMs are off-line.

System

FIG. 2A is a diagram of a memory system, according to embodiments of the disclosure. A processor 201 may be connected to one or more memory controllers 202. The memory controller 202 may be part of the processor itself, as shown in FIG. 2, or on a separate chip. In this embodiment, the processor contains a memory controller A 202A and a memory controller B 202B.
Each memory controller may support one or more memory buffers 203. The memory buffer 203 may be part of the memory controller 202 or on a separate chip. Shown here, memory controller A 202A is connected to memory buffer A 203A, and memory controller B 202B is connected to memory buffer B 203B. Each memory controller 202 or memory buffer 203 may be connected to one or more memory modules 204 through one or more memory channels 209. Shown here, memory buffer A 203A is connected to memory module A 204A and memory module C 204C through memory channel A 209A, and memory buffer B 203B is connected to DIMM B 204B through memory channel B. The performance of the memory modules 204 and the processor 201 may be monitored by performance monitor logic 207.
The memory system may contain a service processor 205. The service processor 205 may be a hardware or firmware unit configured for virtual memory management. The service processor 205 may be configured to create virtual groups 206 of memory from the memory modules 204 of different memory channels 209. The service processor 205 may receive feedback from the performance monitor logic 207 as to performance of memory modules 204 in the virtual groups 206 and applications assigned to certain virtual groups 206. This performance information may be used by a hypervisor 208 to assign applications to virtual groups 206 or to provide configuration options for physical configuration of the DIMMs 204.
FIG. 2B is a diagram of memory allocation in a memory system, according to embodiments of the disclosure. Service processor firmware 214 may create virtual memory groups from memory 215. The memory 215 may include memory modules from across a memory system, including memory modules from different physical groups behind different processors, memory controllers, and memory buffers. An operating system 212, hypervisor 213, or application 211 may allocate memory 215 from the memory groups to the application 211. The service processor firmware 214 may present the memory 215 to the hypervisor 213 or operating system 212 as virtual groups.

Example

FIG. 3 is an example diagram of a memory system for use in the examples diagrams of FIGS. 4, 5A, 5B, and 5C, according to embodiments of the disclosure. In this example, a processor 300 has eight memory controllers, each connected to multiple DIMMs through one or more memory channels (and optionally one or more memory buffers, not shown). The memory contained behind each memory controller may be referred to as the memory controller segment (MCS) for the memory controller. Memory controller 1 301 controls DIMM 1A 311A and DIMM 1B 311B as MCS 1; memory controller 2 302 controls DIMM 2A 312A and DIMM 2B 312B as MCS 2; memory controller 3 303 controls DIMM 3A 313A and DIMM 3B 313B as MCS 3; memory controller 4 304 controls DIMM 4A 314A and DIMM 4B 314B as MCS 4; memory controller 5 305 controls DIMM 5A 315A, DIMM 5B 315B, DIMM 5D 315C, and DIMM 5D 315D as MCS 5; memory controller 6 306 controls DIMM 6A 316A, DIMM 6B 316B, DIMM 6C 316C, and DIMM 6D 316D as MCS 6; memory controller 7 307 controls DIMM 7A 317A, DIMM 7B 317B, DIMM 7C 317C, DIMM 7D 317D, DIMM 7E 317E, DIMM 7F 317F, DIMM 7G 317G, and DIMM 7H 317H as MCS 7; and memory controller 8 308 controls DIMM 8A 318A, DIMM 8B 318B, DIMM 8C 318C, DIMM 8D 318D, DIMM 8E 318E, DIMM 8F 318F, DIMM 8G 318G, and DIMM 8H 318H as MCS 8. For this example, each DIMM may be equal to 1 gigabyte (GB) of memory for simplicity of explanation.
FIG. 4 is an example diagram of physical and virtual groupings of memory segments for the system of FIG. 3, according to embodiments of the disclosure. For a physical grouping, memory from a memory controller segment (MCS) may be grouped and accessed with memory from other memory controller segments having the same size and configuration to form physical groups for memory access. In this example, three physical groups may be formed. Group 1 may contain 8 GB of total memory, comprising the 2 GB of memory behind each of the four memory controllers 1-4. Group 2 may contain 8 GB of total memory, comprising the 4 GB of memory behind memory controllers 5 and 6. Group 3 may contain 16 GB of total memory behind memory controllers 7 and 8.
When the memory is grouped virtually, memory access may not be constrained to physical memory groups or memory controller segments, and a greater number of memory access configurations may be created. By virtually grouping smaller, equal segments of memory across memory channels, a greater number of memory buffers and controllers may access memory at a time, reducing access time to the memory. In this example, virtual groups having the same size as the physical groups discussed above may be formed, but with different memory access configurations.
For virtual grouping 1, 2 GB of memory from each memory controller 1-8 may be grouped as memory segments into group 3 to form a 16 GB group; 2 GB of memory from each memory controller 5-8 may be grouped into group 2 to form an 8 GB group; and 4 GB of memory from each memory controller 7 and 8 may be grouped into group 1 to form an 8 GB group.
For virtual grouping 2, 2 GB of DIMMs from each memory controller 1-4 may be grouped as memory segments into group 1 to form an 8 GB group; 4 GB of memory from each memory controller 7 and 8 may be grouped into group 2 to form an 8 GB group; and 4 GB of memory from each memory controller 5-8 may be grouped into group 3 to form a 16 GB group.
FIG. 5A, FIG. 5B, and FIG. 5C are diagrams of the physical and virtual groupings of memory from FIG. 4, according to embodiments of the disclosure. FIG. 5A is a diagram of memory from memory controller segments in physical groups for the physical grouping, according to embodiments of the disclosure. As discussed above, each physical group contains memory from memory controller segments of the same size and configuration.
FIG. 5B is a diagram of memory from memory controller segments in virtual groups for the virtual grouping 1, according to embodiments of the disclosure. Memory access to group 3 is spread across 8 memory controllers, which may allow for greater throughput. Processor operations that require accessing multiple segments of information simultaneously, such as parallel processing, or containing critical data may best utilize the memory access characteristics of virtual group 3. Critical process operations requiring similar fast access or high availability but for a smaller 8 GB capacity may best utilize the memory access characteristics of virtual group 2. Non-critical applications or application that may better utilize access to the same DIMM for information, such as deep or linear computing operations, may utilize virtual group 1. The memory segments from each memory controller segment are similarly sized, and may be used for symmetrical processes such as memory mirroring and interleaving.
For memory access administered with a lower overhead or for fewer critical or parallel access operations, a different virtual grouping configuration may be used. FIG. 5C is a diagram of memory from memory controller segments in virtual groups for the virtual grouping 2, according to embodiments of the disclosure. Applications utilizing faster memory access for parallel processes or performing more critical operations may be assigned group 1. Applications involving non-critical storage may be assigned group 2. Operations involving linear memory access or minoring, such as database storage, may be assigned group 3, as access is spread across memory controllers. The 16 GB group capacity of group 3 is divided among four memory controllers for redundancy, two of which may contain original data and two of which may contain copy data.
Although the present disclosure has been described in terms of specific embodiments, it is anticipated that alterations and modifications thereof will become apparent to those skilled in the art. Therefore, it is intended that the following claims be interpreted as covering all such alterations and modifications as fall within the true spirit and scope of the disclosure.

Claims

1. A method, comprising:

identifying, in a memory system, a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory;

determining a memory segment size from the capacities of the memory modules;

identifying a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and

creating a virtual group that includes the first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.

2. The method of claim 1, wherein the first amount of memory and the second amount of memory are not equal.

3. The method of claim 1, wherein the first memory channel is connected to a first memory controller and the second memory channel is connected to a second memory controller.

4. The method of claim 1, wherein the memory segment size is a capacity of one or more memory modules in common for each of the first memory channel and the second memory channel.

5. The method of claim 4, wherein the memory segment size is a size of a smallest memory module in common for each of the first memory channel and the second memory channel.

6. The method of claim 1, wherein identifying the capacity for each of a plurality of memory modules comprises:

collecting vital product data for each of the plurality of memory modules; and

determining the capacity from the vital product data for each of the plurality of memory modules.

7. The method of claim 1, further comprising:

determining memory access characteristics of an application; and

assigning the virtual group to the application based on the memory access characteristics.

8. The method of claim 1, further comprising:

collecting performance information from the memory modules having memory in the virtual group; and

determining memory module population rules from the performance information for the memory modules having memory in the virtual group.

9. The method of claim 8, further comprising configuring memory modules in the memory system according to the memory module population rules.

10. The method of claim 7, wherein at least one memory access characteristic involves memory mirroring.

11. A method, comprising:

identifying a capacity for each of a plurality of memory modules for a first memory channel and a second memory channel in a memory system;

determining a first portion of memory modules and a second portion of memory modules from the capacities of the memory modules for each of the first and second memory channels;

creating a first virtual group that includes the first portion of the memory modules from the first memory channel and the first portion of memory modules from the second memory channel; and

creating a second virtual group that includes the second portion of the memory modules from the first memory channel and the second portion of memory modules from the second memory channel.

12. The method of claim 11, wherein the first memory channel and the second memory channel are connected to a memory buffer.

13. The method of claim 11, wherein identifying the capacity for each of a plurality of memory modules comprises:

collecting vital product data for each of the plurality of memory modules; and

14. The method of claim 11, further comprising:

determining memory access characteristics of an application; and

15. The method of claim 11, further comprising:

collecting performance information from the memory modules in the virtual group; and

determining memory population rules from the performance information for the memory modules in the virtual group.

16. The method of claim 11, further comprising configuring memory modules in the memory system according to the memory population rules.

17-20. (canceled)