Method and system for determining black groups based on Spark

Abstract

The embodiment of the invention provides a method and system for determining black grouping based on Spark, and the method comprises the steps: enabling a calculation engine Spark to periodically obtain the group number of a newly-built user group in a previous period from a database, and forming a first elastic data set; according to the group number of each newly-built user group in the first elastic data set, obtaining member information in each newly-built user group and a total integral obtained by all members in each newly-built user group participating in an activity in a previous period, and forming a second elastic data set; and for each newly-built user group in the second elastic data set, judging whether the newly-built user group is a black sub-group or not according to the member information of the newly-built user group and the total integral obtained by all the members in the newly-built user group participating in the activity in the previous period. The black groups are mined by adopting Spark, so that data analysis, statistics and exception authentication for judging whether all newly-built groups are black groups or not can be carried out on a large internet platform within a period of time.

Description

technical field [0001] The invention relates to the field of data analysis, in particular to a method and system for determining black clusters based on Spark. Background technique [0002] On Internet social platforms, there is often a rule of an activity that if one person in the group gets points, everyone will get the same number of points. The idea is to encourage users to socialize. However, some criminals will register small accounts in batches and then create many groups, and then score points for each other or a large account. This behavior not only damages the interests of normal users but also causes great losses to the platform. Such groups are called Black group. The goal now is to find out all these black groups. [0003] In the process of realizing the present invention, the applicant found that at least the following problems existed in the prior art: [0004] If you use ordinary analysis methods to analyze all the groups, the workload is very heavy, and ...

AI Technical Summary

Problems solved by technology

Currently there is no particularly good solution to the problem to analyze all groups and find black groups

Images