Vakilinia et al., 2016 - Google Patents

Analysis and optimization of big-data stream processing

Vakilinia et al., 2016

Document ID: 8483741926048835950
Author: Vakilinia S; Zhang X; Qiu D
Publication year: 2016
Publication venue: 2016 IEEE global communications conference (GLOBECOM)

External Links

Cited by

Snippet

Big data processing is rapidly growing in recent years due to the immediate demanding of many applications. This growth compels industries to leverage scheduling in order to optimally allocate the resources to the big data streams which requires data-driven big data …

Continue reading at www.researchgate.net (PDF) (other versions)

238000005457 optimization 0 title abstract description 17

Classifications

- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for programme control, e.g. control unit
- G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for programme control, e.g. control unit
- G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for programme control, e.g. control unit
- G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Programme initiating; Programme switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for programme control, e.g. control unit
- G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for programme control, e.g. control unit
- G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
- G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
- G06F17/30386—Retrieval requests
- G06F17/30424—Query processing
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/50—Computer-aided design
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N99/00—Subject matter not provided for in other groups of this subclass
- G06N99/005—Learning machines, i.e. computer in which a programme is changed according to experience gained by the machine itself during a complete run
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a programme unit and a register, e.g. for a simultaneous processing of several programmes
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00

Similar Documents

Publication	Publication Date	Title
Lu et al.	2018	IoTDeM: An IoT Big Data-oriented MapReduce performance prediction extended model in multiple edge clouds
Vakilinia et al.	2016	Analysis and optimization of big-data stream processing
Gautam et al.	2015	A survey on job scheduling algorithms in big data processing
Wang et al.	2016	A novel method for tuning configuration parameters of spark based on machine learning
Mohan et al.	2016	Edge-Fog cloud: A distributed cloud for Internet of Things computations
US11228489B2 (en)	2022-01-18	System and methods for auto-tuning big data workloads on cloud platforms
Ranjan	2014	Modeling and simulation in performance optimization of big data processing frameworks
Khan et al.	2017	Optimizing hadoop parameter settings with gene expression programming guided PSO
US20240303127A1 (en)	2024-09-12	Systems and methods for edge system resource capacity performance prediction
Paul et al.	2017	I/o load balancing for big data hpc applications
Gan et al.	2021	Sage: Leveraging ml to diagnose unpredictable performance in cloud microservices
Tang et al.	2014	An effective reliability-driven technique of allocating tasks on heterogeneous cluster systems
Wen et al.	2023	Health monitoring and diagnosis for geo-distributed edge ecosystem in smart city
dos Anjos et al.	2015	Smart: An application framework for real time big data analysis on heterogeneous cloud environments
Mohamed et al.	2016	Hadoop-MapReduce job scheduling algorithms survey
Shen et al.	2018	Performance prediction of parallel computing models to analyze cloud-based big data applications
Vengadeswaran et al.	2024	IDaPS—Improved data-locality aware data placement strategy based on Markov clustering to enhance MapReduce performance on Hadoop
Shabeera et al.	2015	Optimising virtual machine allocation in MapReduce cloud for improved data locality
Kyriazis et al.	2018	BigDataStack: A holistic data-driven stack for big data applications and operations
Oliveira et al.	2015	Optimizing query prices for data-as-a-service
US20240303134A1 (en)	2024-09-12	Systems and methods for edge resource demand load estimation
Nagarajan et al.	2019	Malleable scheduling for flows of jobs and applications to MapReduce
Rehab et al.	2015	Scalable massively parallel learning of multiple linear regression algorithm with MapReduce
Bengre et al.	2021	A learning-based scheduler for high volume processing in data warehouse using graph neural networks
HS	2025	A Synergistic Hybrid SA-PSO-Double Q-Learning Algorithm for Dynamic Cloud Load Balancing.