Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A kafka-based message processing method, electronic equipment and storage medium

A message processing and message technology, applied in the field of big data processing, can solve the problems of reducing the processing capacity of Kafka cluster and occupying computing resources of Kafka cluster, so as to achieve the effect of improving processing capacity, dispersing processing pressure and realizing unified management.

Active Publication Date: 2022-06-28
CHINA MOBILE SUZHOU SOFTWARE TECH CO LTD +1
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] The message obtained by the consumer through the Kafka (kafka) cluster contains redundant information; in related technologies, the consumer's consumption message is processed in the kafka cluster according to the consumer's demand; including: according to the consumer's demand Filter, analyze, and reorganize the messages, but the above method will occupy the computing resources of the Kafka cluster and reduce the processing capacity of the Kafka cluster. How to disperse the processing pressure of the Kafka cluster and process consumer messages according to the needs of consumers is necessary Technical issues resolved

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A kafka-based message processing method, electronic equipment and storage medium
  • A kafka-based message processing method, electronic equipment and storage medium
  • A kafka-based message processing method, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

[0049] Kafka refers to a distributed message queuing system. Its main target scenario is to build a data processing framework for processing massive system logs, web pages and other user behaviors and website operation statistical indicators. In the scenario of big data message processing requirements, the producer-consumer mode is generally used to process message data asynchronously to improve system stability and throughput.

[0050] For some systems that need to process a large number of messages in real time, such as the network management system of telecom operators, the messages produced by producers contain redundant information and have no unified structure. The unstructured mess...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present application discloses a message processing method based on kafka, comprising: receiving a message request sent by a kafka cluster; obtaining a topic monitoring thread according to the consumer information corresponding to the message request; obtaining the topic monitoring thread according to the topic monitoring thread The metadata corresponding to the message request; the application also discloses a kafka-based message processing device and storage medium; through the embodiment of the application, in the process of the consumer obtaining the message subscribed by the consumer in the kafka cluster, according to the consumer Process the messages subscribed by consumers according to the needs, and disperse the processing pressure of the Kafka cluster.

Description

technical field [0001] The invention relates to the technical field of big data processing, in particular to a Kafka-based message processing method, electronic equipment and storage medium. Background technique [0002] The messages obtained by consumers through the Kafka cluster contain redundant information; in related technologies, the Kafka cluster processes the consumer's consumption messages according to the consumer's demand; including: according to the consumer's demand Filter, analyze and reorganize the message, but the above method will occupy the computing resources of the kafka cluster and reduce the processing capacity of the kafka cluster. technical problems solved. SUMMARY OF THE INVENTION [0003] The embodiments of the present application provide a Kafka-based message processing method, electronic device, and storage medium. The Kafka client jar server processes the message according to the needs of the consumer, disperses the processing pressure of the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L67/10H04L67/1097H04L67/63H04L69/22H04L67/568
CPCH04L67/10H04L67/1097H04L69/22H04L67/63H04L67/568
Inventor 仇明潘宇轩朱刘江
Owner CHINA MOBILE SUZHOU SOFTWARE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com