Bridging implementation method from MQTT message system to Pulsar stream data platform

A technology of message system and implementation method, which is applied in the field of data transmission and can solve problems such as lost messages, out-of-sync replicas, and long time consumption

Pending Publication Date: 2022-04-05
四川启睿克科技有限公司
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This is not only a waste of storage, but also a expansion will take a long time, and if the copy is out of sync, messages may be lost
The number of servers, topics, partitions and replicas has to be planned and calculated in advance to avoid scaling issues, which is very difficult

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Bridging implementation method from MQTT message system to Pulsar stream data platform
  • Bridging implementation method from MQTT message system to Pulsar stream data platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0025] combined with figure 1 and figure 2 As shown, a bridge implementation method from the MQTT message system to the Pulsar flow data platform, including:

[0026] S1: Construct a topic, which is convenient for MQTT to subscribe and publish with Pulsar.

[0027] S2: Trigger event message publishing, and trigger the rule for data processing when the message is published.

[0028] S3: Firstly, four data types are designed to construct json data structure description rules. The four data types are:

[0029] The rule engine uses SQL statements to process rule conditions. It is necessary to select all the fields in the payload separately and use the format of payload.fieldName to filter. It also needs the topic, qos, and id information of the message context. The example SQL is as follows:

[0030] SELECT payload.id as client_id, payload.speed as speed, payload.tachometer as tachometer, payload.ts as ts, id FROM "message.publish" WHERE topic=~'t / #'

[0031] S4: Use the SQL...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a bridging implementation method from an MQTT message system to a Pulsar stream data platform, and the method comprises the steps: constructing an MQTT + Pulsar theme, and comprises the steps: creating a theme and a rule; business rules based on SQL are described and transmitted into a data processing rule management engine in an API mode, and the business rules are translated into corresponding Source, Function and Sink in Pulsar; according to idle resources of a system, pulsar is bridged, messages are processed in real time by expanding Function of the pulsar, the messages are sent to an external system after being processed, and log messages are stored. By using MQTT and Pulsar, the message service platform realizes the functions of Internet of Things equipment data access, equipment management, data storage, data analysis and the like, and a flexible programming interface is provided.

Description

technical field [0001] The invention relates to the technical field of data transmission, in particular, a method for realizing bridging from an MQTT message system to a Pulsar streaming data platform. Background technique [0002] Since the MQTT protocol has been used as the basic transmission protocol of the Internet of Things, it has experienced different versions of evolution, including (3.0 / 5.0) and so on. In order to support the latest MQTT transmission protocol and to facilitate the connection of devices with different protocol versions, the functions and services of the message center need to be expanded. To develop a general IoT platform, it is necessary to implement functions such as data access, device management, data storage, and data analysis, and provide a programming interface to the upper layer so that developers can use the platform to quickly develop IoT applications that meet specific vertical industries. Since the Internet of Things itself has the chara...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L67/566H04L67/562G06F16/28G06F16/25G06F16/2458G06F16/242
Inventor 余龙海杨震泉张帅安燕
Owner 四川启睿克科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap