Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A method of file collection and monitoring based on distributed

A collection and monitoring, distributed technology, applied in the direction of electrical components, digital transmission systems, data exchange networks, etc., can solve the problems of few developers, few documents and use, etc., and achieve the effect of strong compatibility

Active Publication Date: 2020-09-29
上海新炬网络技术有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0011] Facebook Scribe: 1. Developed in C++, relatively few developers; 2. Not many domestic documents and usage

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method of file collection and monitoring based on distributed
  • A method of file collection and monitoring based on distributed

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0024] figure 1 It is a flowchart of a method for collecting and monitoring files based on distribution in an embodiment of the present invention.

[0025] See figure 1 , the distributed-based method for file collection and monitoring provided by the present invention, wherein, comprises the following steps:

[0026] S1: Use Mysql cluster and Redis cluster to build the server, and deploy WEBAPP on the server;

[0027] S2: Task definition is provided by WEBAPP, each business or each file path is a task;

[0028] S3: Save the task in the server, scan the information of the file and save it in the server;

[0029] S4: The client obtains task information by reading the Redis cluster each time, and records the file read and write information at the same time, and performs task processing;

[0030] S5: When the task starts, the monitoring program checks ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a distributed file acquisition monitoring method, comprising the following steps: S1, constructing a server by using a Mysql cluster and a Redis cluster, and arranging a WEBAPPat the server; S2, providing a task definition by the WEBAPP; S3, saving the task in the server; S4, a client acquiring task information by reading the Redis cluster every time, and processing the task; S5, the task beginning, a monitoring program checking the integrity of the acquisition process, and performing compensation acquisition; S6, the monitoring program recording a ftp login status anda network status; S7, the client acquiring data and storing the state in a remote storage cluster; and S8, the WEBAPP controlling the downloading, uploading or stopping of the acquisition task. The distributed file acquisition monitoring method implements decentralization, and does not have primary nodes or child nodes, so that the acquisition application can be extended at any time; the providedWEBAPP visualized operation can be used intuitively and conveniently; and the Docker can allow the coexistence of acquisition of multiple versions without conflicts and has strong compatibility.

Description

technical field [0001] The invention relates to a file collection method, in particular to a distributed file collection and monitoring method. Background technique [0002] In the period when big data is becoming more and more important, data acquisition is the most indispensable part of big data development / analysis. In the early stage, hardware or network equipment is required for basic data collection, and data files collected by various manufacturers or equipment need a centralized data center for transmission or maintenance. Due to the diversity and complexity of documents, a stable and monitorable document collection and monitoring method needs to be specially designed to liberate labor costs and quickly locate the collection process and the cause of errors. [0003] At the same time, when the amount of data is increasing, it is necessary to ensure the stability and scalability of the method, the horizontal expansion of the acquisition method, and the need for self-t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08H04L12/26H04L12/24
CPCH04L41/06H04L43/12H04L67/025H04L67/06H04L67/10H04L67/1097
Inventor 程永新谢涛梁超
Owner 上海新炬网络技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com