Online learning type scheduling method based on container layer dependency relationship in edge computing

Abstract

The invention provides an online learning type scheduling method based on a container layer dependency relationship in edge computing. The invention relates to edge computing and a deep reinforcement learning method of resource scheduling and machine learning in a distributed system. According to the technical scheme, firstly, modeling is conducted on edge calculation based on the level of a container layer; the task completion time of the user in the edge calculation is considered, and the task completion time comprises the downloading time of a container required by the user task and the running time of the user task. On the basis, an algorithm based on factorization is provided, the dependency relationship of a container layer in edge computing is extracted, and high-dimensional and low-dimensional sparse dependency features in the dependency relationship are extracted. And finally, on the basis of the extracted dependency relationship and task and node resource characteristics, a learning type task scheduling algorithm based on strategy gradient is designed, and thus verifying the whole process through real data. According to the method provided by the invention, resources in the edge computing can be better planned, and the total overhead of tasks of users in an edge computing system and the overhead required for downloading container mirror image files during container running in the edge computing are reduced.

Description

technical field [0001] The invention belongs to the field of distributed systems in computer networks, the field of edge computing, and the field of machine learning, and relates to resource scheduling in edge computing and distributed systems, and a deep reinforcement learning method for machine learning. Background technique [0002] Traditional algorithms based on container image scheduling cannot take into account the existence of container layers at a finer-grained level, which may lead to a lot of repeated downloads, wasting limited bandwidth and storage resources in edge nodes. Secondly, the existing layer-based scheduling algorithms only consider the size of the upper layer of edge nodes, and cannot well consider the heterogeneity of edge nodes, the dynamics of available resources, and the hidden dependencies between tasks before and after scheduling. In addition, for traditional deep learning techniques, the hidden dependencies between layers cannot be extracted wel...

AI Technical Summary

Problems solved by technology

[0002] The traditional algorithm based on container image scheduling cannot take into account the existence of the container layer from a finer-grained level, which may lead to a lot of repeated downloads, wasting limited bandwidth and storage resources in edge nodes

Secondly, the existing layer-based scheduling algorithm only considers the size of the upper layer of the edge node, and cannot well consider the heterogeneity of edge nodes, the dynamics of available resources, and the hidden dependencies between tasks before and after scheduling.

In addition, for traditional deep learning techniques, the hidden dependencies between layers cannot be extracted well, so they cannot be directly applied to layer-based scheduling problems.

Images