Log processing method and device based on docker container

Abstract

The embodiment of the invention provides a log processing method and device based on a docker container, and the method comprises the steps: deploying a filebeat at each host machine needing to collect log data through employing a docker container, and enabling the filebeat to be used for log data collection; generating kafka and zookeeper cluster configuration files in a host machine, and performing deployment by adopting docker container deployment so as to perform log data transmission; generating an elk cluster configuration file in a host machine, and adopting docker container deploymentfor log data processing and display; and generating an nginx configuration file in the host machine, and performing deployment by adopting the docker container for log data access control. According to the embodiment of the invention, standalone cluster deployment based on the docker container is realized, so that the investment and expenditure of physical machines are saved, and the whole processing process becomes simple and efficient.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a docker container-based log processing method and device. Background technique [0002] For general log analysis scenarios: directly in the log file (in the Liunx environment) grep, awk, tail, and less can get the information you want. However, in large-scale scenarios, this method is inefficient, and problems include how to archive too many logs, what to do if the text search is too slow, and how to query in multiple dimensions. Centralized log management is required, with log collection and aggregation on all servers. A common solution is to establish a centralized log collection system to collect, manage, and access logs on all nodes in a unified manner. [0003] ELK provides a complete set of solutions, all of which are open source software, which can be used in conjunction with each other, perfectly connected, and efficiently meet the applications of many occasi...

AI Technical Summary

Problems solved by technology

This results in a particularly large amount of logs. In the face of massive log data, issues such as resource limitations, performance, security, and high availability need to be considered. How to quickly, efficiently, and safely deploy a log analysis system is an urgent issue.

Images