CN114760328B - Data storage method, system, electronic equipment and storage medium - Google Patents
Data storage method, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114760328B CN114760328B CN202210393203.6A CN202210393203A CN114760328B CN 114760328 B CN114760328 B CN 114760328B CN 202210393203 A CN202210393203 A CN 202210393203A CN 114760328 B CN114760328 B CN 114760328B
- Authority
- CN
- China
- Prior art keywords
- data
- layer
- storage
- block chain
- array
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 title claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 30
- 238000003672 processing method Methods 0.000 claims description 17
- 230000005540 biological transmission Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 10
- 238000011084 recovery Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000001172 regenerating effect Effects 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 abstract description 9
- 230000008929 regeneration Effects 0.000 abstract description 4
- 238000011069 regeneration method Methods 0.000 abstract description 4
- 230000002159 abnormal effect Effects 0.000 abstract description 3
- 230000007246 mechanism Effects 0.000 abstract description 3
- 230000006855 networking Effects 0.000 description 7
- 238000011161 development Methods 0.000 description 5
- 230000018109 developmental process Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 238000012795 verification Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 208000016253 exhaustion Diseases 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
- H04L9/3239—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3247—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention belongs to the technical field of data processing and data storage, and relates to a data storage method, a data storage system, electronic equipment and a storage medium. The invention designs a new data array mechanism, which effectively reduces the occupied capacity during data storage; according to the method, a large amount of cloud storage resources are arranged on the server layer, so that storage resources which have no upper limit in theory can be provided for terminal application equipment with limited local storage resources, and the limitation of the local hardware storage capacity of the terminal application equipment is broken; the invention provides data redundancy for the data in the block chain network layer, when the cloud server loses or is abnormal in data packet, the data can be repaired or recovered through the redundancy regeneration code, the regeneration coding technology is adopted for the data in the server layer, the block chain network layer, the data node network center, the local block chain layer and the terminal application equipment, and the original data is subjected to redundancy coding and storage, so that the safety and reliability of data storage are further improved.
Description
Technical Field
The invention belongs to the technical field of data processing and data storage, and relates to a data storage method, a data storage system, electronic equipment and a storage medium.
Background
The Internet of things is a network capable of realizing interconnection and intercommunication of people, machines and things, extends from the Internet, is first proposed by an Ashton professor in 1991, and has a situation of rapid development in the global scope along with the development and development of technologies such as cloud computing, big data and blockchain in recent years, and plays an increasingly important role in the current society. The internet of things technology is widely used in the fields of smart cities, the internet of vehicles, intelligent medical treatment, smart factories and the like, is a third world information technology industry revolution following computers and the internet, and in the future, the application of the internet of things can influence the aspects of our lives. Meanwhile, the current centralized access mode of the internet of things equipment also causes a new problem. Firstly, the equipment of the internet of things is wide in regional distribution, numerous in equipment quantity, different in application scene, poor in calculation and storage capacity, and the traditional centralized network mode is low in efficiency due to huge data processing pressure. Secondly, the internet of things equipment produced by different manufacturers is connected to different internet of things centralized platforms, and the platforms are not compatible, so that the internet of things equipment under different platforms is difficult to cooperate with each other and cannot perform message interaction and cooperative work, and the development of the internet of things is greatly limited. Finally, the existing internet of things centralized platform is easy to suffer from malicious attack, and has high data storage risk and poor privacy. With the development of the internet of things technology, the number of internet of things devices will be rapidly expanded in the future, the application scale is larger, and the security requirement is higher, so that the realization of the distributed identity authentication and the trusted access of the internet of things devices is urgently needed.
The internet of things essentially performs networking operation on a large number of terminal application devices, and each terminal application device includes a plurality of sensors, where the sensors collect and transmit a large amount of data in real time, but the local storage hardware capacity of each terminal application device is limited, so that the data generated by the large number of terminal application devices needs to be processed and transferred and stored in time.
The invention patent application number 2021112263934 discloses a data distributed storage system, a method and a storage medium based on a cloud computing platform, wherein the system comprises the following components: the data encryption module is used for carrying out private encryption on the data which meets the value requirement and is allowed to be stored; the data splitting module is used for randomly splitting the data after the private encryption to obtain data blocks; the data backup module is used for backing up the split data blocks to obtain mirror image data blocks; the data distribution module is used for randomly distributing the data blocks and the mirror image data blocks to each distributed storage node; and the data combination module is used for carrying out public encryption on the data block and the mirror image data block and then combining the data block with the existing data for storage. The technical scheme solves the problem that the security and stability are threatened due to the fact that a plurality of storage platforms need the same management system, and can be suitable for a plurality of cloud computing platforms to realize multi-user management and multi-user benefit. However, the technical scheme still has defects in stability, safety and restorability of lost data in data processing and transmission.
Disclosure of Invention
In order to solve the technical problems, the invention provides a data storage method, a system, electronic equipment and a storage medium, which improve the stability and safety of data processing and data storage and the recoverability of data when the data is lost by optimizing the data array format and optimizing the data transmission link.
In order to achieve the above object, the technical scheme of the present invention is as follows:
The data processing method comprises data storage, data recovery and data transmission, wherein only data transmission is performed in the data recovery process, and data storage is not performed; the data are represented by a data array, wherein the data array is M (alpha, beta), and specifically:
Wherein:
Alpha and beta are numerical array points in a data array,
K is a security parameter in the data array,
B is the code length of the data array,
D is the number of data nodes of the data array.
Further, each data in the data array comprises M1(1k,h)、M2(1k,t)、M3(X,PK)、M4(x,W,acc(X),PK)、M5(X*,ac,acc(X),pk)、M6(X#,ac,acc(X),pk)、M7(ac,X,pk)、M8(Wi,ac,pk) total 8 accumulated data values; wherein:
k is a security parameter in the data array,
H is a random type value for the data array,
Pk is the public key of the data array,
A c is the return of accumulated value aiding information,
X= { X1, X2...xn } is an input element,
Acc (X) is the return accumulated value of element X,
W is a data array authentication factor,
X *＝{X* 1、X* 2......X* n is a new element added,
X #＝{X# 1,X# 2,......X# n is the deleted element,
Wi is the updated authentication factor.
The invention also provides a data processing system, which applies the data processing method to process data, and the data processing system comprises: the system comprises a server layer, a blockchain network layer, a data node network center, a local blockchain layer and terminal application equipment; the server layer is a cloud server layer; the block chain network layer is a distributed block chain network adopting a consensus algorithm; the data node network center is a data communication center; the local block chain layer is used for storing data generated by the terminal application equipment; the terminal application device is a hardware or software device of an application terminal.
Furthermore, the data in the server layer, the block chain network layer, the data node network center, the local block chain layer and the terminal application equipment all adopt a regenerative coding technology, and the original data is subjected to redundancy coding and storage.
Further, the local blockchain layer adopts energy perception coding to store data and restore the data.
Further, the data is configured with double hash values, wherein one hash value is calculated by the blockchain network layer, and the other hash value is calculated by the local blockchain layer.
Further, the double hash values make up a merck tree whose root contains the header of the data in the blockchain network layer.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the data processing method described above when executing the computer program or performing data processing using the data processing system described above.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the data processing method described above or performs data processing using the data processing system described above.
The invention has the following beneficial effects:
(1) The invention designs a new data array mechanism, which effectively reduces the occupied capacity during data storage;
(2) According to the method, a large amount of cloud storage resources are arranged on the server layer, so that storage resources which have no upper limit in theory can be provided for terminal application equipment with limited local storage resources, and the limitation of the local hardware storage capacity of the terminal application equipment is broken;
(3) According to the invention, data redundancy is provided for the data in the block chain network layer, when the cloud server loses or is abnormal in data packet, the data can be repaired or recovered through the redundancy regeneration code, so that the safety and reliability of data storage are improved;
(4) The data in the server layer, the block chain network layer, the data node network center, the local block chain layer and the terminal application equipment all adopt a regenerative coding technology, and the original data is subjected to redundancy coding and storage, so that the safety and reliability of data storage are further improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an overall architecture of a data processing system according to the present invention.
Detailed Description
The invention is further described with reference to the following detailed drawings in order to make the technical means, the creation characteristics, the achievement of the purpose and the effect of the implementation of the invention easy to understand.
The data processing method comprises data storage, data recovery and data transmission, wherein only data transmission is performed in the data recovery process, and data storage is not performed; the data are represented by a data array, wherein the data array is M (alpha, beta), and specifically:
Wherein:
Alpha and beta are numerical array points in a data array,
K is a security parameter in the data array,
B is the code length of the data array,
D is the number of data nodes of the data array.
Further, each data in the data array comprises M1(1k,h)、M2(1k,t)、M3(X,PK)、M4(x,W,acc(X),PK)、M5(X*,ac,acc(X),pk)、M6(X#,ae,acc(X),pk)、M7(ac,X,pk)、M8(Wi,ac,pk) total 8 accumulated data values; wherein:
k is a security parameter in the data array,
H is a random type value for the data array,
Pk is the public key of the data array,
Ac is return accumulated value assistance information, x= { X1, X2...xn } is an input element,
Acc (X) is the return accumulated value of element X,
W is a data array authentication factor,
X *＝{X* 1、X* 2......X* n is a new element added,
X #＝{X# 1,X# 2,......X# n is the element to be deleted,
W i is the updated authentication factor.
Compared with the common data array coding format in the prior art, the data array format with eight accumulated values used in the invention can realize the storage of complete data with less occupied storage capacity. The invention takes the array code length and the number of the data nodes of the data as elements for representing the data, so that the characteristics of each data field are more obvious, the storage and the calling of the data field are more accurate, the redundancy and the error storage of the data field are effectively reduced, and the storage space occupied by the data field during the storage is also reduced to a certain extent.
The invention also provides a data processing system, which applies the data processing method to process data, and the data processing system comprises: the system comprises a server layer, a blockchain network layer, a data node network center, a local blockchain layer and terminal application equipment; the server layer is a cloud server layer; the block chain network layer is a distributed block chain network adopting a consensus algorithm; the data node network center is a data communication center; the local block chain layer is used for storing data generated by the terminal application equipment; the terminal application device is a hardware or software device of an application terminal. The data in the server layer, the block chain network layer, the data node network center, the local block chain layer and the terminal application equipment all adopt a regenerative coding technology, and the original data is subjected to redundancy coding and storage. And the local blockchain layer adopts energy perception coding to store data and recover the data. The data is configured with double hash values, one of which is calculated by the blockchain network layer and the other of which is calculated by the local blockchain layer. The double hash values make up a merck tree whose root contains the header of the data in the blockchain network layer.
The data processing method only carries out data transmission and does not carry out data storage in the process of data recovery, so that the data quantity to be read is the same as the data quantity to be transmitted, and the data processing method is more suitable for being used in a storage system.
The invention also provides a data processing system, which applies the data processing method to process data, and the data processing system comprises: the system comprises a server layer, a blockchain network layer, a data node network center, a local blockchain layer and terminal application equipment; the server layer is a cloud server layer; the block chain network layer is a distributed block chain network adopting a consensus algorithm; the data node network center is a data communication center; the local block chain layer is used for storing data generated by the terminal application equipment; the terminal application device is a hardware or software device of an application terminal.
The server layer is provided with a large number of cloud servers, cloud storage service functions can be provided, and a user of the terminal equipment can access data stored in the server layer anytime and anywhere through any network equipment linked to the server layer. In general, a large amount of cloud storage resources are arranged in the server layer, so that a storage resource without an upper limit in theory can be provided for terminal application equipment with limited local storage resources.
The block chain network layer can realize global networking, the data storage safety is guaranteed in a block chain mode data storage mode, each cloud server in the server layer can communicate with the block chain network layer, each cloud server can store and verify data, and a consensus algorithm is used for generating and updating the data. Meanwhile, in order to improve the reliability and safety of data, the invention provides data redundancy for the data in the block chain network layer, and when a cloud server loses or is abnormal in data packets, the data can be repaired or recovered through the redundancy regeneration code, so that the safety and reliability of data storage are further improved.
The data node network center is a communication transfer layer of a local block chain network and a block chain network layer, a plurality of terminal application devices form a network with each other, the terminal application devices are managed by the data node network center, and all the terminal application devices which are networked can participate in the data node network center no matter the hardware specification of the terminal application devices and the size of the local storage amount. Meanwhile, the data node network center is also responsible for data interaction with an upper block chain network layer, and data generated by terminal application equipment and stored on a local block chain layer are transmitted to the block chain network layer for storage at regular intervals. Furthermore, in practical applications, since any end device entity can be registered as a data source, but the end application devices performing data interaction must be successfully registered in the data node network center, it is avoided that end application devices are added to the data node network center without being registered, and all end application devices participating in networking must have one and the same data node center, otherwise, data management on the end application devices of the networking cannot be effectively performed.
The data node center and the terminal application equipment of the invention are commonly networked to form a local block chain, a plurality of local block chains form a local block chain layer for storing data generated on each terminal application equipment, and the local block chain is a simpler block chain network in practice due to limited storage hardware resources of the terminal application equipment, when the data locally stored in the terminal application equipment reaches the upper limit, the data needs to be hashed, transmitted and stored to the local blockchain, and meanwhile, the local data in the terminal application equipment can be deleted to obtain new local hardware storage resources, and then the local blockchain layer is used for summarizing and transmitting the data to the previous level.
The terminal application device of the invention comprises a large number of sensor nodes, the sampling data of most sensors (such as a temperature sensor, a GPS sensor, a pressure sensor and the like) are numerical, but the sampling values of many sensors are multimedia data (such as traffic camera video data, audio sensor sampling data, remote sensing imaging data and the like), each sensor frequently generates new sampling data, and the system needs to store the sampling data of the latest version. In addition, a plurality of sensors, such as traffic sensors, hydrologic sensors, geological sensors, meteorological sensors, biomedical sensors, etc., may be included in the device network formed by the networking of the plurality of terminal application devices. Each sensor comprises a plurality of specific sensors, and can be further divided into a GPS sensor, a license plate recognition sensor, an electrophotographic recognition sensor, a traffic flow sensor (infrared, coil, optical and video sensors), a road sensor, an intact sensor and the like, wherein the sensors form the data source of the bottommost layer of the invention.
In order to improve the safety and reliability of data, the data in the server layer, the block chain network layer, the data node network center, the local block chain layer and the terminal application equipment all adopt a regenerative coding technology, and the original data is subjected to redundant coding and storage.
The local block chain layer adopts energy perception coding to store data and restore the data. Typically, the power of the sensors deployed in most end-use devices is very limited, and the amount of sensor energy in turn determines the performance of the local end-use network to some extent. Therefore, the invention applies the energy sensing coding to the data storage of the local block chain layer, and when the middle node transmits data in the data information transmission process of the local terminal application equipment, the node with the most residual energy is selected from the adjacent nodes to transmit the data, so that the probability of early exhaustion of the energy of the node is reduced. At the same time, the relative residual energy can be used for selecting one node from a plurality of nodes with enough energy as a network device for data transmission, and the network device is used for recoding or recovering lost data.
The block chain storage scheme comprises a local block chain layer and a block chain network layer, wherein the local block chain layer is established by networking of terminal application equipment, and the local block chain layer stores data generated and collected by the terminal application equipment and hash values thereof. The invention also provides a mechanism for periodically verifying the hash value of the data to ensure the data integrity, namely, configuring double hash values for the data, wherein one hash value is calculated by the blockchain network layer, and the other hash value is calculated by the local blockchain layer. The double hash values make up a merck tree whose root contains the header of the data in the blockchain network layer.
In the invention, a corresponding manager is configured for each terminal application device, each terminal application device connected with the local blockchain performs identity verification by the corresponding manager, legal terminal application devices are allocated with a public and private key pair, and terminal application devices without the corresponding public and private key pair cannot participate in local blockchain networking or upload data to the local blockchain. All terminal application devices participating in the local block chain layer network can collect and send data, when the collected data reach the upper limit of the hardware capacity of the terminal application device, the hash value of the data block is calculated, the distributed private key is used for carrying out signature verification on the hash value of the collected data, the hash value is transmitted to the local block chain layer for verification, and after verification is passed, the hash value is written into the local block chain layer. The hash value can construct a merck tree, the root of the merck tree is contained in the head of the local block chain layer, and in addition, the data is sent to the local block chain layer for storage, so that the local block chain layer can serve as a global node of the whole data system, the terminal application equipment serves as a light node, and after the terminal application equipment clears the stored data, the data can be collected again.
The terminal application devices in practical application are usually a large number of sensors which are used for sensing information, cooperatively working to construct a local network and connecting different monitoring objects to the Internet, but the resources of the large number of sensors and the corresponding monitoring devices in storage and communication are very limited, so that each terminal application device is connected with a local block chain layer to construct the local block chain network, and any terminal application device can transmit data with the local block chain layer, thereby getting rid of the limitation of the storage hardware capacity of the terminal application device. In addition, most of terminal application devices in practical application environments use sensors with very limited power, and the data processing method of the invention applies energy sensing codes to data storage of a local block chain layer, and when a middle node transmits data in the process of data information transmission between the terminal application devices and the local block chain layer, the system is automatically set to select the node with the most residual energy among the neighbor nodes of the intermediate node for sending so as to reduce the probability of early exhaustion of the energy of the node, and simultaneously, the system can also automatically monitor the average residual energy values of all adjacent nodes adjacent to the previous node in the data transmission process, and can select a node with enough energy from a plurality of nodes with enough energy as network equipment for information transmission by comparing the relative residual energy values.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the data processing method described above when executing the computer program or performing data processing using the data processing system described above.
The electronic device may be in the form of a general purpose computing device, which may be a server device, for example. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, a bus connecting different system components, including the memory and the processor. The buses include a data bus, an address bus, and a control bus. The memory may include volatile memory such as Random Access Memory (RAM) and/or cache memory, and may further include Read Only Memory (ROM). The memory may also include program means having a set (at least one) of program modules including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. The processor executes various functional applications and data processing by running computer programs stored in the memory.
In addition, the electronic device may also communicate with one or more external devices (e.g., keyboard, pointing device, etc.). Such communication may be through an input/output (I/O) interface. And, the electronic device may also communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be appreciated that other hardware and/or software modules may be used in connection with an electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, data backup storage systems, and the like.
Although in the above detailed description several units/modules or sub-units/modules of an electronic device are mentioned, such a division is only exemplary and not mandatory. Indeed, the features and functionality of two or more units/modules described above may be embodied in one unit/module in accordance with embodiments of the present application. Conversely, the features and functions of one unit/module described above may be further divided into ones that are embodied by a plurality of units/modules.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the data processing method described above or performs data processing using the data processing system described above.
More specifically, among others, readable storage media may be employed including, but not limited to: portable disk, hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.
The invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the data processing method as defined above, or to carry out data processing using the data processing system as defined above, when said program product is run on the terminal device.
Wherein the program code for carrying out the invention may be written in any combination of one or more programming languages, which program code may execute entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on the remote device or entirely on the remote device.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (8)
1. The data processing method is characterized by comprising data storage, data recovery and data transmission, wherein only data transmission is performed in the data recovery process, and data storage is not performed; the data are represented by a data array, wherein the data array is M (alpha, beta), and specifically:
；
wherein: alpha and beta are numerical array points in the data array, k is a security parameter in the data array, B is a code length of the data array, and d is the number of data nodes of the data array;
Each data in the data array comprises M1(1k,h)、M2(1k,t)、M3(X,PK)、M4(x,W,acc(X),PK)、M5(X*,ac,acc(X),pk)、M6(X#,ac,acc(X),pk)、M7(ac,X,pk)、M8(Wi,ac,pk) total 8 accumulated data values; wherein k is a security parameter in the data array, h is a random type value of the data array, pk is a public key of the data array, a c is return accumulated value auxiliary information, x= { X1, X2 … Xn } is an input element, acc (X) is a return accumulated value of element X, W is a data array authentication factor, X *={ X* 1、X* 2……X* n } is an added new element, X #={ X# 1,X# 2,…… X# n } is a deleted element, and W i is an updated authentication factor.
2. A data processing system for performing data processing using the data processing method according to claim 1, characterized in that: the data processing system includes: the system comprises a server layer, a blockchain network layer, a data node network center, a local blockchain layer and terminal application equipment; the server layer is a cloud server layer; the block chain network layer is a distributed block chain network adopting a consensus algorithm; the data node network center is a data communication center; the local block chain layer is used for storing data generated by the terminal application equipment; the terminal application device is a hardware or software device of an application terminal.
3. The data processing system of claim 2, wherein the data in the server layer, the blockchain network layer, the data node hub, the local blockchain layer, and the end-use devices all employ a regenerative encoding technique, and the original data is redundantly encoded and stored.
4. A data processing system according to claim 3, wherein the local blockchain layer uses energy-aware encoding for data storage and data retrieval.
5. The data processing system of claim 4, wherein the data is configured with double hash values, one hash value calculated by a blockchain network layer and the other hash value calculated by a local blockchain layer.
6. The data processing system of claim 5, wherein the double hash values form a merck tree, a root of the merck tree containing a header of data in a blockchain network layer.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the data processing method of claim 1 when executing the computer program.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the data processing method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210393203.6A CN114760328B (en) | 2022-04-14 | 2022-04-14 | Data storage method, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210393203.6A CN114760328B (en) | 2022-04-14 | 2022-04-14 | Data storage method, system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114760328A CN114760328A (en) | 2022-07-15 |
| CN114760328B true CN114760328B (en) | 2024-08-13 |
Family
ID=82331276
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210393203.6A Active CN114760328B (en) | 2022-04-14 | 2022-04-14 | Data storage method, system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114760328B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115292340B (en) * | 2022-09-27 | 2022-12-02 | 国网数字科技控股有限公司 | Block chain storage optimization method and device based on distributed network coding |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109409510A (en) * | 2018-09-14 | 2019-03-01 | 中国科学院深圳先进技术研究院 | Neuron circuit, chip, system and method, storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8625635B2 (en) * | 2010-04-26 | 2014-01-07 | Cleversafe, Inc. | Dispersed storage network frame protocol header |
| US10140574B2 (en) * | 2016-12-31 | 2018-11-27 | Via Alliance Semiconductor Co., Ltd | Neural network unit with segmentable array width rotator and re-shapeable weight memory to match segment width to provide common weights to multiple rotator segments |
-
2022
- 2022-04-14 CN CN202210393203.6A patent/CN114760328B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109409510A (en) * | 2018-09-14 | 2019-03-01 | 中国科学院深圳先进技术研究院 | Neuron circuit, chip, system and method, storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114760328A (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11399063B2 (en) | Network authentication for a storage system | |
| US20210216646A1 (en) | Hardware Token Based Management of Recovery Datasets for a Storage System | |
| US12063296B2 (en) | Securely encrypting data using a remote key management service | |
| CN101488104B (en) | System and method for implementing high-efficiency security memory | |
| US20210110055A1 (en) | Data Deletion for a Multi-Tenant Environment | |
| US20210216631A1 (en) | Filesystem Property Based Determination of a Possible Ransomware Attack Against a Storage System | |
| CN111291046B (en) | A computer big data storage control system and method | |
| EP3739493A1 (en) | File verification method, file verification system and file verification server | |
| US20190004727A1 (en) | Using a namespace to augment de-duplication | |
| CN118690396B (en) | Data storage method and system based on block chain | |
| CN110933040B (en) | Block chain based data uplink method, device, equipment and medium | |
| CN114760328B (en) | Data storage method, system, electronic equipment and storage medium | |
| US20250147684A1 (en) | Storage Network Having Metadata Storage Trees | |
| CN116366673A (en) | Data analysis and sharing method oriented to management of power grid information machine room | |
| US20250124144A1 (en) | Remedial Actions for a Storage System Based on Similar Block Detection Process Anomalies | |
| CN110704511A (en) | Data processing method, device, apparatus and medium | |
| CN105516355B (en) | Intelligent electric energy meter error big data safe storage device based on fountain codes and method | |
| Yu et al. | ACRS-Raft: A Raft Consensus Protocol for Adaptive Data Maintenance in the Metaverse Based on Cauchy Reed–Solomon Codes | |
| CN116755640B (en) | Data processing method, device, computer equipment and storage medium of alliance chain | |
| CN113315753A (en) | Block data credibility recovery method based on coding technology | |
| US12411962B2 (en) | Managed run-time environment-based detection of a ransomware attack | |
| CN105610879A (en) | Data processing method and data processing device | |
| CN118018326A (en) | Data security encryption method and system based on distributed storage | |
| CN106201781A (en) | A kind of cloud date storage method based on the right canonical correcting and eleting codes | |
| CN110781163B (en) | Construction of Heterogeneous Partial Repetition Codes Based on Complete Graph and Repair Method of Faulty Nodes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20220715 Assignee: Zhejiang Weigu Information Technology Co.,Ltd. Assignor: SHANGHAI V&G INFORMATION TECHNOLOGY CO.,LTD. Contract record no.: X2024980016641 Denomination of invention: A data storage method, system, electronic device, and storage medium Granted publication date: 20240813 License type: Common License Record date: 20240929 |
|
| EE01 | Entry into force of recordation of patent licensing contract |