Data synchronization method and device, and server

A data synchronization and server technology, which is applied in the field of big data, can solve problems such as poor data synchronization methods and no solutions, and achieve the effect of poor solutions

Inactive Publication Date: 2020-01-17
车轮互联科技(上海)股份有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] For the problem of poor data synchronization methods in related technologies, no effective solution has been proposed so far

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data synchronization method and device, and server
  • Data synchronization method and device, and server
  • Data synchronization method and device, and server

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the application. Obviously, the described embodiment is only It is an embodiment of a part of the application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0029] It should be noted that the terms "first" and "second" in the description and claims of the present application and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It should be understood that the data so used may be interchanged under appropriate circumstances for...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data synchronization method and a device, and a server. The method comprises the steps of starting at least one process, and reading data from a message queue; editing the read data according to the current time period directory, and writing the edited data into a data file in a distributed file system format; regularly transferring the data file of the previous time period to a new specified directory; and calling a data file transmission interface, and synchronizing the data file to a data table directory corresponding to the distributed file system. The technical problem that a data synchronization method is poor in effect is solved. The file is directly uploaded to Hive in a webHDFS interface mode, the file is simpler and more direct, a complex flume system and various configurations do not need to be built, and the synchronization rate is increased to a certain degree.

Description

technical field [0001] The present application relates to the field of big data, in particular, to a data synchronization method and device, and a server. Background technique [0002] The existing methods of synchronizing real-time data generally use the MQ message queue, and also need to use the Flume log collection system. [0003] The inventor found that if the Flume log collection system is used, it needs a big data developer with a certain java language background to build it, and the learning cost is high. Furthermore, the maintenance cost is also high. [0004] Aiming at the problem of poor effect of the data synchronization method in the related art, no effective solution has been proposed yet. Contents of the invention [0005] The main purpose of this application is to provide a data synchronization method, device, and server to solve the problem. [0006] In order to achieve the above object, according to one aspect of the present application, a data synchro...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/22G06F16/25G06F16/27G06F16/955
CPCG06F16/25G06F16/27G06F16/2282G06F16/955
Inventor 梁涌吴峰郭伟
Owner 车轮互联科技(上海)股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap