Controllable high-concurrency distributed message queue push method

Abstract

The invention discloses a controllable high-concurrency distributed message queue push method. An FIFO queuing model is arranged; a message queue entrance gateway and a message distribution module are arranged at the front end of the FIFO queuing model; the message distribution module carries out unified controllable message push controlling; and the message queue entrance gateway carries out message queue entrance management and fault-tolerance processing / recovery. A schedule program module and a consumer message processor are arranged at the tail end of the FIFO queuing model. Compared with the prior art, the provided method has the following beneficial effects: the message consumer access number or request frequency can be controlled actively; the queue service resources are saved; the downtime risk can be avoided to the greatest extent; the message fault-tolerance delayed processing is realized; process blocking, message queue data accumulation, and dramatic service resource consumption are avoided; and with a service-free fault-tolerance processing function, a phenomenon that a message producer fails to send a message when a storage service that the queue relies on is not available can be prevented.

Description

technical field [0001] The invention relates to an Internet technology, in particular to a method for pushing a controllable large concurrent distributed message queue. Background technique [0002] The existing message queue methods are mainly RabbitMQ, ActiveMQ, JMS and so on. The above technologies / products have their own strengths in message data throughput performance, service availability, and functional features, but basically they only have basic queue functions, including: multicast, on-demand, broadcast, etc., and some of them can provide delay after message consumption failure Time retweet function. However, there are no relevant technical solutions or tools for performance / service resource control and support for the business system requirements it is connected to. for example: [0003] 1. Cannot take the initiative to control the access quantity or request frequency of message consumers. This defect can easily lead to excessive consumption of queue service re...

AI Technical Summary

Problems solved by technology

[0003] 1. Cannot actively control the access quantity or request frequency of message consumers. This defect can easily lead to excessive consumption of queue service resources by message consumers. Even if this situation is found, it cannot be solved by its own means

[0004] 2. Most of them do not have the function of message fault tolerance and delayed processing

For example, when a message in the queue cannot be processed by the consumer due to the abnormal data it contains, you can neither choose to discard the message, nor ignore the error, which will cause the subsequent processing process to block, which may lead to the accumulation of message queue data , consumes service resources rapidly and cannot be resolved in a short time

[0005] 3. Most of them have no delay / scheduled message push function, which is required in most business systems

[0006] 4. It is impossible to centrally feed back the reasons for the failure of the consumer, making troubleshooting difficult and inefficient

[0007] 5. No service fault-tolerant processing, when the storage service that the queue depends on is unavailable, the message producer will fail to send the message

[0008] 6. Does not provide system problem warning function

[0009] 7. No direct consumer service load balancing function

Images