Distributed deep learning system based on Docker container and establishment method and working method thereof
A docker container and deep learning technology, applied in the field of cloud computing virtualization, can solve the problems of resource waste, large number of hosts, and incomplete utilization, etc., and achieve the effect of simple process, cost saving, and time wasting
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0045] A distributed deep learning system based on Docker containers, such as figure 1 As shown, it includes a host machine and multiple Docker containers, Hadoop distributed platform and Spark are installed on the host machine, and the first distributed deep learning platform or the second distributed deep learning platform is also installed on the host machine; each The Hadoop distributed platform and Spark are installed on the Docker container, and the first distributed deep learning platform or the second distributed deep learning platform is also installed on each Docker container.
[0046] The server host is used as the host machine and as the hardware support of the entire platform. The first distributed deep learning platform and the second distributed deep learning platform are two currently available distributed deep learning platforms, both of which are open sourced by Yahoo, and are the current mainstream distributions. deep learning platform.
[0047] The first d...
Embodiment 2
[0051] The method for building the distributed deep learning system based on the Docker container described in Embodiment 1, the specific steps include:
[0052] (1) Prepare the host machine, which is the server host machine; install the Ubuntu 14.04 operating system; Ubuntu 14.04 is a relatively stable version of the Linux operating system that supports Docker, and you can directly install and configure the Docker environment with the command line;
[0053] (2) Create the main folder required by the Docker container in the root directory of the host machine. The main folder includes folders that can be mounted to save the training model, training data set, test data set, code and configuration files;
[0054] (3) Install the Hadoop distributed platform and Spark in the host machine; to support the CaffeOnSpark distributed deep learning platform or the TensorFlowOnSpark distributed deep learning platform; test whether the Hadoop distributed platform and Spark are installed suc...
Embodiment 3
[0064] The working method of the distributed deep learning system based on the Docker container described in embodiment 1, the specific steps include:
[0065] (1) start the Hadoop platform and Spark in the host computer, the host computer is as the master node of the whole distributed deep learning system, and start the Hadoop platform and Spark in several described Docker containers, several described Docker containers Both serve as slave nodes of the entire distributed deep learning system;
[0066] (2) Store the training model, training data set, test data set, code and configuration files required for deep learning training in the folder that can be mounted on the host computer;
[0067] (3) Start the deep learning training through the script, and the master node distributes the deep learning training tasks to each slave node for parallel training.
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com