CN111367866A - Cabinet model system of data cloud - Google Patents

Abstract

The invention provides a cabinet model system of a data cloud, and a search device sends a request for searching target data to a configuration distribution network. The configuration distribution network obtains the configuration information of the target data through the cabinet administrator. Generate file paths based on configuration information. The cabinet administrator obtains any cloud data server according to the configuration information. If it is unavailable, it will switch to the next cloud data server until it is available. The cloud data server is used to provide the service of retrieving configuration information. The corresponding cabinet when the cloud data server finds the target data storage. The properties, physical location information and file path of the search device are calculated by using preset rules, the file name is generated, a folder is created with the file name, and the target data is stored in the folder. All storage is directly addressable from any cloud data server. Creating video streams and writing video recording applications using the data cloud's cabinet model never fails, increasing reliability by an order of magnitude.

Description

A Cabinet Model System for Data Cloud

technical field

The invention relates to the technical field of file establishment in a system, in particular to a cabinet model system of a data cloud.

Background technique

The components of a data cloud are defined by numerology, with the underlying storage topology of a disk database. Numerology provides an efficient SQL (Structured Query Language) binding for AI (Artificial Intelligence)/BI (Business Intelligence) applications. However, functions such as network video recording, GUI (Graphical User Interface) thumbnailing, and video playback usually require services like POSIX (Portable Operating System Interface, Shared Memory, Subsystem, hereinafter referred to as POSIX) that can be accessed from a server or video recording application Program access.

A largely POSIX-compliant binding was also created for this purpose. This binding is named cabinet (meaning cabinet in this application) and is similar to files and folders in other implementations. The minimum required functions are as follows: file creation and writing, file opening and reading, folder creation, folder opening and reading, file size and creation, time, and query.

Existing technology NVR (Network Video Recorder) applications that use the data cloud to create video streams and write video recordings will experience interruptions or failures without a complete network structure or power failure, and the cloud to create video streams and write video records does not. Continuity and reliability are relatively low.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present invention is to provide a data cloud cabinet model system for creating video streams and writing video records.

A cabinet model system for data cloud, characterized in that:

S100, the search device sends a request to search for target data to the configuration distribution network. The search device is a server for sending a request for finding target data, and a distribution network is configured to provide the search device with a search target data service.

S110 The configuration distribution network obtains the configuration information of the target data through the cabinet administrator. The cabinet model system includes the cabinet administrator, and the cabinet administrator is used to retrieve the configuration information of the target data. Configuration information is the storage location of target data in the cabinet model system.

S120 generates a file path according to the configuration information.

S130, the cabinet administrator obtains the current cloud data server according to the configuration information, and if the current cloud data server is unavailable, that is, the current cloud data server cannot retrieve the configuration information, the cabinet administrator automatically The next cloud data server is obtained until the cabinet administrator obtains an available cloud data server, and the cloud data server is used to provide a service for retrieving the configuration information.

When the S140 cloud data server finds the corresponding cabinet for storing the target data, the cabinet is a hard disk slot for storing the target data, and the hard disk slot has unique physical location information. The physical location information is used to provide the exact location of the hard disk slot.

S150 calculates the attributes, physical location information and file path of the search device by using preset rules, generates a file name, creates a folder with the file name, and stores the target data in the folder. The file name includes the attributes, physical location information and file path of the search device, and the preset rule is used to generate a uniquely determined file name according to the attributes, physical location information and file path of the search device.

Compared with the prior art, the beneficial effect of the present invention is that all storage can be directly addressed from any cloud data server. Even if the entire data center (AZ) is offline, NVR (network video recorder) applications that create video streams and write video recordings using the cabinet model of the data cloud will never fail. The reliability is increased from 99.999% to 99.9999%, which is an order of magnitude higher than the reliability of the prior art.

A further optimization is: the search device may be at least one of an Internet of Things device or a network-free search device.

By adopting the above technical solution, the search device can search the target data even in the case of network or power failure, so as to avoid failure.

Further optimization is: configuration information includes target data type, serial number, filter type, GUI filter type, physical location, router, server, and disk slot information.

By adopting the above technical solution, through the configuration information, the search device can accurately find the stored target data.

Further optimization is: the physical location information is the area information of the storage location of the target data, and the physical location information includes available area, area, sector, number of layers, region, longitude and latitude.

With the above technical solution, the storage location of the target data is uniquely determined.

Further optimization is: the cabinet stores several files, the files are composed of sections, and the sections span several files.

With the above technical solution, the file is composed of sections, and the section spans several files, the system structure is completely general, the cost of creating the file is low, and the query is convenient and fast.

Further optimization is that the cabinet provides storage space for several documents, and the cabinet is prefixed with a capital letter in the storage space.

By adopting the above technical solution, it is convenient to identify files.

Further optimization is: all entries of the storage space represent the physical location of the cabinet.

By adopting the above technical solution, the uniqueness of the cabinet is determined.

Further optimization is: files can be stored across 4 data centers in the cabinet group.

With the above technical solution, the cabinet group allows the storage of files in the data center.

It is further optimized as follows: the segment file refers to the invisible segment file, the invisible segment file can be encoded, and the invisible segment file is automatically presented when query access is performed.

By adopting the above technical solution, the automatic file generation is realized, and the invisible segment file is automatically presented.

Further optimization is: the file is connected to any number of configuration distribution networks.

By adopting the above technical solution, the whole program keeps running continuously, no errors are reported, and the reliability is improved by an order of magnitude.

Description of drawings

The schematic flow chart of the embodiment of Fig. 1;

2 is a schematic diagram of a cabinet manager of an embodiment;

3 is a schematic diagram of an example of a cabinet group of an embodiment;

4 is a schematic diagram of file generation of an embodiment;

Fig. 5 is the schematic diagram of data storage and reading;

Figure 6 is a numerology diagram for storing data into a configuration distribution network.

Detailed ways

The purpose of the embodiments of the present invention is to provide a data cloud cabinet model system for creating video streams and writing video records. The specific technical solutions are as follows:

A data cloud cabinet model system for creating video streams and writing video records, as shown in Figure 1. The specific process is:

S100, the search device sends a request to search for target data to the configuration distribution network. The search device is a server for sending a request for finding target data, and a distribution network is configured to provide the search device with a search target data service.

S110 The configuration distribution network obtains the configuration information of the target data through the cabinet administrator. The cabinet model system includes the cabinet administrator, and the cabinet administrator is used to retrieve the configuration information of the target data. Configuration information is the storage location of target data in the cabinet model system.

S120 generates a file path according to the configuration information.

S130, the cabinet administrator obtains the current cloud data server according to the configuration information, and if the current cloud data server is unavailable, that is, the current cloud data server cannot retrieve the configuration information, the cabinet administrator automatically The next cloud data server is obtained until the cabinet administrator obtains an available cloud data server, and the cloud data server is used to provide a service for retrieving the configuration information.

When the S140 cloud data server finds the corresponding cabinet for storing the target data, the cabinet is a hard disk slot for storing the target data, and the hard disk slot has unique physical location information. The physical location information is used to provide the exact location of the hard disk slot.

S150 calculates the attributes, physical location information and file path of the search device by using preset rules, generates a file name, creates a folder with the file name, and stores the target data in the folder. The file name includes the attributes, physical location information and file path of the search device, and the preset rule is used to generate a uniquely determined file name according to the attributes, physical location information and file path of the search device.

All storage is directly addressable from any cloud data server. Even if the entire data center (AZ) is offline, NVR (network video recorder) applications that create video streams and write video recordings using the cabinet model of the data cloud are always continuous and never fail. All data storage and reading are continuous and uninterrupted, and the reliability is increased from 99.999% to 99.9999%, which is an order of magnitude higher than the reliability of the existing technology.

The virtual folder and file mechanism is called a cabinet. It provides a POSIX-compliant namespace for the following: a video stream folder for a camera, a file for each video stream, a file for each video segment, a folder for thumbnail albums, a file for thumbnails, and a file for faces historical access records.

Additionally, applications can be associated with cabinet types, which include: a GPU filter on the write path, custom metadata presentation and storage, auto-generation of related components, and custom GUI presentation.

The storage object key is defined by the combination of A-H columns, as shown in Figure 6, all of which are numeric columns in the CDN, similar to standard product codes, with extensions including attributes in the actual environment, such as latitude and longitude.

This is a standard hierarchy (columns A-D) that defines a disk or mount point, but has no notion of pathnames (like C:ProgamFiles), because the folder hierarchy string is not easily queried via SQL. In the present invention, path names are derived from numerology, where columns E-H correspond to individual folders in a traditional hierarchy.

Since folders are derived and namespaces are virtual, the concept of cabinets can also be extended to include individual segments in a media stream. For example, if 2 million cameras are supported, and video is kept for a month, with an average of 14 seconds per video segment, there are 400 billion unique pathnames, which is at least an order of magnitude larger than the state-of-the-art.

Combined with a widely distributed CDN, the cabinet model system also means that all storage is directly addressable from any cloud server.

Cabinet administrators control cabinets through physical networking equipment, and multiple cabinets form cabinet groups. A cabinet stores several files, a file consists of sections, a section spans several files, a file consists of sections, and a section spans several files. The architecture is completely general, the cost of creating files is low, and the query is convenient and fast.

Cabinets provide storage space for several files, cabinets are prefixed with uppercase letters in storage spaces, and in POSIX namespaces, cabinets are by convention prefixed with uppercase alphanumerics. For example, the camera meta folder has the prefix CM_ and the camera media stream has the prefix CF_.

The pathname of an object varies from object to object, as follows:

CF_<fid>_<cabinet> is a media file

CEF_<extid>_<fid>_<cabinet> is a media fragment file

CER_<extid>_<extid>_<fid>_<cabinet> is a time range file

CM_<cabinet> is the folder

<fid> is an 8-digit hexadecimal number in Linux epoch format. <extid> is a 12-digit hexadecimal number that also contains 1/65536 of the second granularity for uniqueness.

Therefore, all entries in the namespace represent the physical location of the cabinet (including the 3D location), plus a timestamp. Since it is the Internet of Things, all entities are directly represented in space/time. Easy to identify files. All entries in the storage space represent the physical location of the cabinet and determine the uniqueness of the cabinet.

As shown in Figure 5, the process of creating an NVR (Network Video Recorder) file includes the following steps:

NVR is connected to CDN (Configuration Distribution Network), CDN provides cabinet administrator.

2. The cabinet administrator corresponds to the 4 cloud servers in the 4 data centers and is defined as a cabinet group.

3. The media file is named according to its AZ (data center) and time index, eg VF_1a2b3000_0_30_1_1_1.

4. Derive the out-of-the-box instance from the file name.

5. Cabinet folders are automatically generated.

The process of reading an NVR file includes the following steps:

1. Search and narrow down the range of video sources to be searched through the location information and time range information of the camera.

2. The cabinet administrator finds the master cabinet instance in his cabinet group

3. Find the disk by opt|az|reg|seg, where opt is the type, az is the available area, reg is the area, and seg is the sector.

4. Find the folder according to the storage path: area|zn|fl|cam|lat|in|c1|c2|router. Where area is the area, zn| is the area, fl is the maturity, cam is the camera number, lat is the longitude, in is the latitude, c1 is the filter type, c2 is the GUI filter type, and router is the router.

5. The namespace includes the unique identifier of the data center and the time index.

6. If the NVR is unavailable, it will automatically switch to the next available data center defined in the cabinet group, repeat step 4.

Pre-set rules: Cabinet keeper is represented by each column in the table, as shown in Figure 2, and row 3 shows an example cabinet keeper. The camera's latitude/longitude and elevation are specified by columns I, J, and G. GPU and GUI classes are represented by column K and column L. Parent metadisks are represented by columns A-D. Composes the usefulness of the locker manager in the table. The cabinet administrator defines the primary and backup locations for an IoT device (such as a camera) to define the configuration and store the data in the data center. Figure 1 shows an example of a device whose opt (type) is cam (camera), az (available area) is numbered 0, reg (area) is 1, sect (sector) is 1, area (area) is 1, zone (area) is 1, floor (number of layers) is 1, cam (camera number) is 1, lat (longitude) is 34.2540882, lng (dimension) is 108.9423662, c1 (filter type) is g (GPU), c2 ( GUI filter type) is face (face), as well as router (router), nid (server id) and slot (disk slot) information.

Files can be stored across 4 data centers in a cabinet group, and the cabinet group allows file data center storage. As shown in Figure 3, rows 3-6 define a zone, represented as 4 availability zones (column B). The cabinet instance for camera permissions is defined on line 8, with a primary reliable region of 0 (line 8, column B), and alternate regions defined on lines 4-6.

The segment file refers to the invisible segment file. The invisible segment file can be encoded. The file can be composed of segments, and the segment can span different original files of the file. For example "VER_1a2b3000_1ac2s4000_0_30_1_1_1" is a time range that spans 3 video files and spans 2 data centers. Typically, the timeframe selection will be a short period that won't span multiple files, but the architecture is completely generic. When query access is performed, the invisible segment file is automatically presented, so as to realize the automatic generation of the file and the automatic presentation of the invisible segment file.

The cabinet is derived from the path name of the file, and a folder is automatically created when the first media file is created, as shown in Figure 4, the cabinet manager (5.1) corresponds to a maximum of 4 sector-level disks, and a maximum of 4 data centers (5.2). ). Copying the directory structure would be cumbersome and expensive.

The cabinet instance (5.1) can be derived from the file pathname (5.4), as can the cabinet folder name (5.5). Folders are automatically created when the first media file is created. Low management cost and implicit creation.

The file connection can be configured to distribute any number of network services, and the file connection reliability is high.

The configuration distribution network server refers to the physical networked equipment being recorded to provide management teller services, in the event of the loss of the entire data center, to ensure that the recording is continuous and error-free.

The steps of file generation include the following steps: as shown in Figure 4 and Figure 6,

S101 files connect to any number of configuration distribution networks.

The S102 configuration distribution network provides up to 4 cloud disks on the cloud server.

S103 creates a file, and the cloud server fails, and the cloud server switches to another server to create a new file.

S104 continues to execute S101-S103. File generation continues to run, no errors are reported, and reliability improves by an order of magnitude without an entire data center.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims (10)

1. a cabinet model system of data cloud, is characterized in that: S100, the search device sends a request for searching target data to a configuration distribution network; the search device is used to send a server for searching for the target data request, and the configuration distribution network is used to provide the search device with a service for searching the target data; S110, the configuration distribution network obtains the configuration information of the target data through a cabinet administrator, the cabinet model system includes a cabinet administrator, and the cabinet administrator is used to retrieve the configuration information of the target data; the configuration information is the storage location of the target data in the cabinet model system; S120 generates a file path according to the configuration information; S130, the cabinet administrator obtains the current cloud data server according to the configuration information, and if the current cloud data server is unavailable, that is, the current cloud data server cannot retrieve the configuration information, the cabinet administrator automatically Obtain the next cloud data server until the cabinet administrator obtains an available cloud data server, and the cloud data server is used to provide a service for retrieving the configuration information; S140 when the cloud data server finds a cabinet corresponding to the target data storage, the cabinet is a hard disk slot for storing the target data, and the hard disk slot has uniquely determined physical location information; the physical The location information is used to provide the specific location of the hard disk slot; S150 uses preset rules to calculate the properties of the search device, the physical location information and the file path, generate a file name, create a folder with the file name, and store the target data in the folder wherein, the file name includes the property of the search device, the physical location information and the file path, and the preset rule is used to search for the device according to the property, the physical location information and the file Path generates a uniquely determined filename. 2 . The cabinet model system of the data cloud according to claim 1 , wherein the search device can be at least one of an Internet of Things device or a no-network search device. 3 . 3. The cabinet model system of the data cloud according to claim 1, wherein the configuration information includes the type, serial number, filter type, GUI filter type, physical location, router, server of the target data and disk slot information. 4. The cabinet model system of the data cloud according to claim 1, wherein the physical location information is the area information of the storage location of the target data, and the physical location information includes available area, area, fan Zone, Layer, Region, Longitude, and Latitude. 5 . The cabinet model system of the data cloud according to claim 1 , wherein: the cabinet stores a plurality of the files, the files are composed of sections, and a section spans a plurality of the files. 6 . 6 . The cabinet model system of the data cloud according to claim 1 , wherein the cabinet provides storage space for several of the files, and the cabinet is prefixed with a capital letter in the storage space. 7 . 7 . The cabinet model system of the data cloud according to claim 6 , wherein: all items in the storage space represent the physical location of the cabinet. 8 . 8 . The cabinet model system of the data cloud according to claim 1 , wherein the files can be stored in the cabinet group across four data centers. 9 . 9 . The cabinet model system of the data cloud according to claim 1 , wherein the segment file is an invisible segment file, the invisible segment file can be encoded, and when query access is performed, the Invisible section files are rendered automatically. 10 . The cabinet model system of the data cloud according to claim 1 , wherein the file is connected to any number of configuration distribution networks. 11 .

Images