Device and method for anomaly detection on an input stream of events

Abstract

The present invention relates to the field of artificial intelligence, machine learning and Big Data, and provides a device (100) and method (300) for anomaly detection, performed in a stream of events. To this end, the device (100) for anomaly detection on an input stream (101) of events (102) is provided, wherein each event (102) comprises a value (103). The device (100) comprises a modeling unit (104), configured to represent each event 102 from the input stream (101) of events (102) into at least one histogram (105) over a predefined tree data structure (106), to obtain a model 107 of the input stream (101); a change detection and adaptation module (108), configured to detect a long term change (109) of the distribution of the events (102) in the input stream 101 based on the model (107) of the input stream (101); and a scoring unit (110), configured to calculate an anomaly score (111), representing the probability of an anomaly in the input stream (101) of events (102), based on the model (107) of the input stream (101), and the detected long term change (109).

Description

technical field [0001] The present invention relates to the fields of artificial intelligence (artificial intelligence, AI), machine learning (machine learning, ML) and big data, and provides a device and method for anomaly detection. Anomaly detection for sequences of events. A sequence of events may also be referred to as streaming data. That is, the present invention also relates to the field of stream processing. Background technique [0002] Currently, machine learning for distributed stream processing of big data is an important research area. A stream is a sequence of events (e.g., tuples containing various types of data) generated in a time-ordered manner by various sources (e.g., sensors, machines, or humans). The stream processing paradigm involves applying analytics or more complex learning functions to events in streams. A typical approach to stream processing assumes that such events are accumulated over a range at a given time, and applies analytical functi...

AI Technical Summary

Problems solved by technology

[0006] Known approaches to stream processing treat computations as window functions, but the approach suffers from two major limitations

The first limitation is that by default each window function keeps all events in a window buffer which grows with the number of elements in the buffer and the number of functions to be evaluated

The second limitation of the current approach is that the entire data needs to be passed for every function update, especially for non-associative functions

Due to its fixed structure, its parsing is limited, and potential expansion will destroy resource and time performance

Images