+

WO2018157331A1 - Procédé et système de stockage appliqués à des mégadonnées - Google Patents

Procédé et système de stockage appliqués à des mégadonnées Download PDF

Info

Publication number
WO2018157331A1
WO2018157331A1 PCT/CN2017/075331 CN2017075331W WO2018157331A1 WO 2018157331 A1 WO2018157331 A1 WO 2018157331A1 CN 2017075331 W CN2017075331 W CN 2017075331W WO 2018157331 A1 WO2018157331 A1 WO 2018157331A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage
big data
logical address
network logical
server
Prior art date
Application number
PCT/CN2017/075331
Other languages
English (en)
Chinese (zh)
Inventor
马岩
Original Assignee
深圳市博信诺达经贸咨询有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市博信诺达经贸咨询有限公司 filed Critical 深圳市博信诺达经贸咨询有限公司
Priority to PCT/CN2017/075331 priority Critical patent/WO2018157331A1/fr
Publication of WO2018157331A1 publication Critical patent/WO2018157331A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40—Support for services or applications

Definitions

  • the present invention relates to the field of data processing, and in particular, to a storage method and system for applying big data.
  • Big Data The strategic significance of big data technology is not to master huge data information, but to professionalize these meaningful data.
  • big data the key to profitability in this industry is to increase the “processing capability” of the data and “add value” of the data through “processing”.
  • Big data must not be processed by a single computer, and a distributed architecture must be used. It features distributed data mining for massive data. But it must rely on cloud computing for distributed processing, distributed databases and cloud storage, and virtualization technologies. With the advent of the cloud era, big data (Big Data) has also attracted more and more attention.
  • Big Data (Big) Data) is often used to describe a large amount of unstructured data and semi-structured data created by a company that spends too much time and money when downloaded to a relational database for analysis. Big data analytics is often associated with cloud computing because real-time large dataset analysis requires a framework like MapReduce to distribute work to dozens, hundreds, or even thousands of computers.
  • the existing big data storage has low efficiency and low user experience.
  • the application provides a storage method applied to big data. It solves the shortcomings of low user experience of the prior art technical solutions.
  • a storage method for applying big data comprising the following steps: applying to a storage method of big data, the method comprising the following steps:
  • the server receives the big data that needs to be stored
  • the server extracts the storage amount of the big data, and queries, according to the storage quantity, a network logical address that matches the storage quantity from the network logical address;
  • the server sends the big data to the network logical address storage.
  • the method further includes:
  • the server establishes a storage allocation list including a big data identifier and a corresponding network logical address.
  • the method further includes:
  • the server obtains a message indicating that the storage is successful, it determines that the big data storage is successful, otherwise, another network logical address is queried from the network logical address.
  • a storage system for applying to big data comprising:
  • a transceiver unit configured to receive big data that needs to be stored
  • the processing unit is configured to extract a storage amount of the big data, and query, according to the storage quantity, a network logical address that matches the storage quantity from the network logical address, and send the big data to the network logical address storage.
  • system further includes:
  • a processing unit configured to establish, by the server, a storage allocation list, where the storage allocation list includes a big data identifier and a corresponding network logical address.
  • system further includes:
  • the processing unit is configured to determine that the big data storage is successful if the message that the storage is successful is obtained, otherwise, another network logical address is queried from the network logical address.
  • a third aspect provides a server, including: a processor, a wireless transceiver, a memory, and a bus, wherein the processor, the wireless transceiver, and the memory are connected by a bus, and the wireless transceiver is configured to receive big data that needs to be stored. ;
  • the processor is configured to extract a storage quantity of the big data, query, according to the storage quantity, a network logical address that matches the storage quantity from a network logical address, and send the big data to the network logical address storage.
  • the processor is configured to establish, by the server, a storage allocation list, where the storage allocation list includes a big data identifier and a corresponding network logical address.
  • the processor is configured to determine that the big data storage is successful, such as obtaining a message that the storage is successful, and otherwise query another network logical address from the network logical address.
  • the technical solution provided by the present invention is dynamically allocated by the network logical address and the storage amount, so it has the advantages of high storage efficiency and high user experience.
  • FIG. 1 is a flowchart of a method for storing big data according to a first preferred embodiment of the present invention
  • FIG. 2 is a structural diagram of a storage system applied to big data according to a second preferred embodiment of the present invention.
  • FIG. 3 is a hardware structural diagram of a server according to a second preferred embodiment of the present invention.
  • FIG. 1 is a storage method applied to big data according to a first preferred embodiment of the present invention. The method is as shown in FIG. 1 and includes the following steps:
  • Step S101 The server receives big data that needs to be stored.
  • Step S102 The server extracts the storage amount of the big data, and queries, according to the storage quantity, a network logical address that matches the storage quantity from the network logical address.
  • Step S103 The server sends the big data to the network logical address storage.
  • the technical solution provided by the present invention is dynamically allocated by the network logical address and the storage amount, so it has the advantages of high storage efficiency and high user experience.
  • the server establishes a storage allocation list, where the storage allocation list includes a big data identifier and a corresponding network logical address.
  • the server obtains a message that the storage is successful, it determines that the big data storage is successful, otherwise, another network logical address is queried from the network logical address.
  • FIG. 2 is a storage system applied to big data according to a second preferred embodiment of the present invention.
  • the system is as shown in FIG. 2, and includes:
  • the transceiver unit 201 is configured to receive big data that needs to be stored
  • the processing unit 202 is configured to extract a storage amount of the big data, and query, according to the storage quantity, a network logical address that matches the storage quantity from the network logical address, and send the big data to the network logical address storage.
  • the technical solution provided by the present invention is dynamically allocated by the network logical address and the storage amount, so it has the advantages of high storage efficiency and high user experience.
  • the processing unit 202 is configured to establish, by the server, a storage allocation list, where the storage allocation list includes a big data identifier and a corresponding network logical address.
  • the processing unit 202 is configured to determine that the big data storage is successful, such as obtaining a message that the storage is successful, and otherwise query another network logical address from the network logical address.
  • FIG. 3 is a server 30, including: a processor 301, a wireless transceiver 302, a memory 303, and a bus 304.
  • the wireless transceiver 302 is configured to send and receive data with and from an external device.
  • the number of processors 301 can be one or more.
  • processor 301, memory 302, and transceiver 303 may be connected by bus 304 or other means.
  • Server 30 can be used to perform the steps of FIG. For the meaning and examples of the terms involved in the embodiment, reference may be made to the corresponding embodiment of FIG. 1. I will not repeat them here.
  • the wireless transceiver 302 is configured to receive big data that needs to be stored.
  • the program code is stored in the memory 303.
  • the processor 901 is configured to call the program code stored in the memory 903 for performing the following operations:
  • the processor 301 is configured to extract a storage amount of the big data, and query, according to the storage quantity, a network logical address that matches the storage quantity from the network logical address, and send the big data to the network logical address storage.
  • the processor 301 herein may be a processing component or a general term of multiple processing components.
  • the processing element can be a central processor (Central) Processing Unit, CPU), or a specific integrated circuit (Application Specific Integrated) Circuit, ASIC), or one or more integrated circuits configured to implement embodiments of the present application, such as one or more microprocessors (digital singnal Processor, DSP), or one or more Field Programmable Gate Arrays (FPGAs).
  • CPU central processor
  • ASIC Application Specific Integrated Circuit
  • DSP digital singnal Processor
  • FPGAs Field Programmable Gate Arrays
  • the memory 303 may be a storage device or a collective name of a plurality of storage elements, and is used to store executable program code or parameters, data, and the like required for the application running device to operate. And the memory 303 may include random access memory (RAM), and may also include non-volatile memory (non-volatile memory) Memory), such as disk storage, flash (Flash), etc.
  • RAM random access memory
  • non-volatile memory non-volatile memory
  • flash flash
  • Bus 304 can be an industry standard architecture (Industry Standard Architecture, ISA) bus, Peripheral Component (PCI) bus or extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc.
  • the bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in Figure 3, but it does not mean that there is only one bus or one type of bus.
  • the terminal may further include input and output means connected to the bus 304 for connection to other parts such as the processor 301 via the bus.
  • the input/output device can provide an input interface for the operator, so that the operator can select the control item through the input interface, and can also be other interfaces through which other devices can be externally connected.
  • the program may be stored in a computer readable storage medium, and the storage medium may include: Flash drive, read-only memory (English: Read-Only Memory, referred to as: ROM), random accessor (English: Random Access Memory, referred to as: RAM), disk or CD.
  • ROM Read-Only Memory
  • RAM Random Access Memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

L'invention concerne un procédé de stockage appliqué à des mégadonnées, qui comprend les étapes suivantes : un serveur reçoit des mégadonnées devant être stockées (101) ; le serveur extrait la taille de stockage des mégadonnées et recherche, en fonction de celle-ci, une adresse logique de réseau correspondante dans des adresses logiques de réseau (102) ; le serveur envoie les mégadonnées à l'adresse logique de réseau à des fins de stockage (103). La solution technique selon l'invention présente l'avantage de conférer une bonne expérience utilisateur.
PCT/CN2017/075331 2017-03-01 2017-03-01 Procédé et système de stockage appliqués à des mégadonnées WO2018157331A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/075331 WO2018157331A1 (fr) 2017-03-01 2017-03-01 Procédé et système de stockage appliqués à des mégadonnées

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/075331 WO2018157331A1 (fr) 2017-03-01 2017-03-01 Procédé et système de stockage appliqués à des mégadonnées

Publications (1)

Publication Number Publication Date
WO2018157331A1 true WO2018157331A1 (fr) 2018-09-07

Family

ID=63369627

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/075331 WO2018157331A1 (fr) 2017-03-01 2017-03-01 Procédé et système de stockage appliqués à des mégadonnées

Country Status (1)

Country Link
WO (1) WO2018157331A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101561810A (zh) * 2008-04-16 2009-10-21 鸿富锦精密工业（深圳）有限公司 将文档数据库的数据存储到关系数据库的系统及方法
CN106101270A (zh) * 2016-08-05 2016-11-09 成都佳荣科技有限公司 数据分布式存储系统
CN106294010A (zh) * 2016-08-08 2017-01-04 马岩 分布式系统中大数据的存储方法及系统
CN106293527A (zh) * 2016-08-05 2017-01-04 成都佳荣科技有限公司 数据存储系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101561810A (zh) * 2008-04-16 2009-10-21 鸿富锦精密工业（深圳）有限公司 将文档数据库的数据存储到关系数据库的系统及方法
CN106101270A (zh) * 2016-08-05 2016-11-09 成都佳荣科技有限公司 数据分布式存储系统
CN106293527A (zh) * 2016-08-05 2017-01-04 成都佳荣科技有限公司 数据存储系统
CN106294010A (zh) * 2016-08-08 2017-01-04 马岩 分布式系统中大数据的存储方法及系统

Similar Documents

Publication Publication Date Title
CN108549583B (zh) 大数据处理方法、装置、服务器及可读存储介质
CN106302554A (zh) 一种socket通信方法、装置和存储设备
WO2019153702A1 (fr) Procédé de traitement d'interruptions, appareil et serveur
CN103678425A (zh) 多系统的集成分析
WO2020024392A1 (fr) Procédé et appareil de traitement de nœuds, support d'informations et dispositif électronique
WO2018223354A1 (fr) Procédé et système d'enregistrement de présence à base de positionnement
WO2018157330A1 (fr) Procédé et système de partitionnement de mégadonnées
CN107506144A (zh) 一种集群存储系统机箱统一管理方法及系统
WO2018157331A1 (fr) Procédé et système de stockage appliqués à des mégadonnées
WO2018157391A1 (fr) Procédé et système d'évaluation de mégadonnées en entreprise
CN107665241B (zh) 一种实时数据多维度去重方法和装置
WO2018157332A1 (fr) Procédé et système statistiques appliqués à des mégadonnées
WO2018157333A1 (fr) Procédé et système de traitement de mégadonnées
WO2018157392A1 (fr) Procédé et système pour déterminer les entreprises affiliées sur la base de mégadonnées
CN115604191A (zh) 业务流量控制方法、装置、电子设备及可读存储介质
WO2018170887A1 (fr) Procédé et système d'affichage de liste de mégadonnées
CN111444162A (zh) 大数据初始化方法、装置、电子设备及存储介质
WO2018170888A1 (fr) Procédé et système de combinaison et de sélection de sous-commande de liste de mégadonnées
CN105955979B (zh) 文件处理方法和系统
WO2018165839A1 (fr) Procédé et système de mise en œuvre de chenilles distribuées
WO2019061384A1 (fr) Procédé et système de sélection d'un gestionnaire de tâches dans un système de robot web distribué
CN114338402B (zh) Redis集群的搭建方法、装置、电子设备及存储介质
WO2019061385A1 (fr) Procédé et système de distribution de tâches de robots d'indexation distribués
WO2018223375A1 (fr) Procédé et système de contrôle et de rappel de trafic de terminal
WO2018209504A1 (fr) Procédé et système de gestion d'application de terminal sur la base d'un groupe

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17898492

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 29/01/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17898492

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载