Systems and methods for forecast alerts with programmable human-machine hybrid ensemble learning

Abstract

A method for computing a human-machine hybrid ensemble prediction includes: receiving an individual forecasting question (IFF); classifying the IFF into one of a plurality of canonical question topics; identifying machine models associated with the canonical question topic; for each of the machine models: receiving, from one of a plurality of human participants: a first task input including a selection of sets of training data; a second task input including selections of portions of the selected sets of training data; and a third task input including model parameters to configure the machine model; training the machine model in accordance with the first, second, and third task inputs; and computing a machine model forecast based on the trained machine model; computing an aggregated forecast from machine model forecasts computed by the machine models; and sending an alert in response to determining that the aggregated forecast satisfies a threshold condition.

Description

[0001] Cross References to Related Applications [0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 824,150, "A Forecast Alert System with Programmable Human-Machine Hybrid Ensemble Learning Methods," filed with the U.S. Patent and Trademark Office on March 26, 2019, the entire disclosure of which is incorporated by reference This article. [0003] Statement Regarding Federally Funded Research or Development [0004] This invention was made with United States Government support under Contract No. 2017-17061500006 issued by the Intelligence Advanced Research Projects Activity. The US Government has certain rights in this invention. technical field [0005] Aspects of embodiments of the invention relate to human-machine hybrid forecasting systems and methods and user interfaces therefor for forecasting events using a mix of human and machine forecasters. Background technique [0006] Predicting the timing or outcome of future events is...

AI Technical Summary

Problems solved by technology

However, machine prediction systems also exhibit the so-called "cold start problem", wherein, when a new problem is introduced, the accuracy of the system is extremely limited or decreases and remains low until the system has accumulated enough data to understand the problem

This is one of the reasons for the rigidity of machine-only forecasting systems, slowing down the rate at which the system can adapt to new problems and subject areas

Images