Data set creation with crowd-based reinforcement

Abstract

A system and method for creation and expansion of high quality data set collections for training of machine learning algorithms via crowdsourced curation that utilizes a data marketplace which incentivizes data gatherers, publishers, and users to contribute to the creation of a vast resource of reliable data set collections. Data is automatically ingested from disparate sources and autonomously checked for data quality, provenance, and cyber-risks and subsequently given a reputation score. Data stewards curate a queue of low scoring real data as well as synthetically generated data. All reputable data is stored for user consumption and further iterative data generation.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]ApplicationNo.Date FiledTitleCurrentHerewithCYBERSECURITY PROFILING ANDapplicationRATING USING ACTIVE AND PASSIVEEXTERNAL RECONNAISSANCEIs a continuation-in-part of:15 / 931,534May 13, 2020SECURE POLICY-CONTROLLEDPROCESSING AND AUDITING ONREGULATED DATA SETSwhich is a continuation-in-part of:16 / 777,270Jan. 30, 2020CYBERSECURITY PROFILING ANDRATING USING ACTIVE AND PASSIVEEXTERNAL RECONNAISSANCEwhich is a continuation-in-part of:16 / 720,383Dec. 19, 2019RATING ORGANIZATIONCYBERSECURITY USING ACTIVE ANDPASSIVE EXTERNAL RECONNAISSANCEwhich is a continuation of:15 / 823,363Nov. 27, 2017RATING ORGANIZATIONPatentIssue DateCYBERSECURITY USING ACTIVE AND10,560,483Feb. 11, 2020PASSIVE EXTERNAL RECONNAISSANCEwhich is a continuation-in-part of:15 / 725,274Oct. 4, 2017APPLICATION OF ADVANCEDPatentIssue DateCYBERSECURITY THREAT MITIGATION10,609,079Mar. 31, 2020TO ROGUE DEVICES, PRIVILEGEESCALATION, AND RISK-BASEDVULNERABILITY AND PATCHMANAGEMENTwhich is a con...

AI Technical Summary

Benefits of technology

The patent describes a system and method for creating and expanding high-quality data set collections for training machine learning algorithms via crowdsourced curation. The system receives a data set and uses a machine learning algorithm to score data entries based on multiple metrics, with the scores combined to form an overall reputation score. The system also flags any erroneous data entries and assigns them to a human curator for resolution. Additionally, the system can use a generative adversarial network or other machine learning frameworks to generate synthetic data sets based on a reputable data set collection. The technical effects of this system and method include improved data quality, reduced human error, and improved efficiency in training machine learning algorithms.

Problems solved by technology

Additionally, real-world data offers only one historical view and provides no lateral understanding of variances that could have happened in the market.

Images