43 results about "Data heterogeneity" patented technology

The invention discloses a superframe-based low-energy-consumption media access control method in a wireless body area network, mainly solving the problems of high energy consumption and failure in adaption to data heterogeneity of a competitive mechanism of the current protocol in the wireless body area network. The superframe-based low-energy-consumption media access control method comprises the steps of: dividing a superframe into three stages: a WBAN (Wireless Body Area Network) intranet data period, a period of communication between an individual terminal and a public network, and a WBAN reserved alarm period; setting three priority levels including high priority, secondary-high priority and common priority for each node by a coordinator node by means of monitoring abnormal data and calculating cache and time delay of each sensor node in the intranet data period; and allocating different time slot lengths and sequences according to three priorities. Under the condition that that cache and time delay meet the condition that, time delay is fully utilized, and the characteristic of the data heterogeneity in the wireless body area network is better adapted. Due to synchronism of data, without data interaction, all sensor nodes can judge whether to enter into a sleep mode or not just according to the local cache and the emission rate, thus reducing communication loss.

The invention relates to a big data technology, and aims to provide a credit risk control system and method based on a federation mode. The system comprises a heterogeneous data access layer used foraccessing and converting data, a data preprocessing layer used for preprocessing original data, and a sample alignment layer used for keeping training samples of different data providers aligned, anda federated learning layer used for training a local model by utilizing the participant local data and forming a global model after gradient aggregation. The invention provides a unified data access format, data preprocessing and a risk prediction model based on federated learning, and solves the challenge problem brought by data heterogeneity and privacy leakage to risk control. A central serverdoes not need to participate in the model training and learning process, and it can be guaranteed that user privacy is not eavesdropped. Risk control modeling can be carried out by combining a plurality of different participants, the modeling process is standardized, the risk control capability is finally improved, and the cost is reduced for enterprises.

The invention discloses a crowdsourcing task distribution method based on user reliability. The method comprises: S1, inputting a user set W and a task set T; S2, starting task distribution and carrying out initialization on a correlated task and the user set; S3, for each task ti in the to-be-distributed task set T, selecting a user meeting a condition and adding the user into a user set Wr,I of the task ti; S4, starting a first round of user distribution and carrying out ergodic processing on each task in the to-be-distributed task set T; S5, carrying out user distribution form a second round to the first round and carrying out ergodic processing on each task ti in the to-be-distributed task set T; and S6, completing the task distribution and updating a reputation value of the ser according to the quality of task completion by the user in the set Wr. Therefore, reliability and heterogeneity of the user and task heterogeneity can be guaranteed; and the effective yield of the task can increase and thus more user can be simulated to complete tasks with high quality.

The invention relates to an image data expansion method for deep learning model training and learning, and belongs to the technical field of computer medical image calculation. The method comprises the following steps: firstly, judging a data type, and identifying CT or MRI image data; For the image data, judging whether an ROI (Region Of Interest) is defined or not, and selecting a correspondingmethod to complete the construction of an image data set in combination with the size of a tumor region; Training the image data set by adopting a basic image transformation method to obtain a preliminary training data set; And finally, carrying out data expansion on the preliminary training data set, carrying out deep training by adopting a network model, and finally, carrying out probability prediction. Based on artificial intelligence deep learning, a series of data expansion methods are applied to learning of deep model training in the field of medical image processing, the influence of medical image data heterogeneity on abnormal data is solved, computer-aided diagnosis is facilitated, and the diagnosis efficiency and accuracy are improved.

The invention relates to a data management service system based on large data. The data management service system comprises a heterogeneous data normalized-description module, a data semantization module, a data storage performance module, a data logic-management module, a data scenarization and service matching module and a data display module. The data management service system based on scenes solves the problems as follows: first, the data volumes are large at present, the data variety is large, the data is from various data sources, and the data categories and formats are rich; as a result, the problem of difficult storage is formed; second, the description of data heterogeneity: the data with multi-source large data forms data islands; different data structures exit in each data source, and at the same time, different designation systems also exist in each data source; the homogeneous data can also not interoperate; third, the data matching problem: the data matching problem is formed as the structures of data categories are different, the precision ratio and the recall rate are low, and the query cost is high.

The present disclosure relates a system, method, and computer program for detecting anomalous user network activity based on multiple data sources. The system extracts user event data for n days from multiple data sources to create a baseline behavior model that reflects the user's daily volume and type of IT events. In creating the model, the system addresses data heterogeneity in multi-source logs by categorizing raw events into meta events. Thus, baseline behavior model captures the user's daily meta-event pattern and volume of IT meta events over n days. The model is created using a dimension reduction technique. The system detects any anomalous pattern and volume changes in a user's IT behavior on day n by comparing user meta-event activity on day n to the baseline behavior model. A score normalization scheme allows identification of a global threshold to flag current anomalous activity in the user population.

The invention discloses a shale heterogeneity classification and comparative method, which comprises:(1) building the isochronous stratigraphic comparison lattice for the vertical heterogeneity classification of the shale reservoir by the high-frequency sequence classification and comparation; (2) studying the macro heterogeneity in the isochronous stratigraphic comparison lattice and analyzing and establishing the vertical and horizontal change characteristics of the macro heterogeneity of different high-frequency sequences of the shale reservoir in the isochronous stratigraphic comparison lattice; (3) studying the micro heterogeneity in the isochronous stratigraphic comparison lattice and establishing the vertical and horizontal change characteristics of the micro heterogeneity of different high-frequency sequences of the shale reservoir; (4) classifying and comparing small layers, and conducting detailed classification of small layers in the isochronous stratigraphic comparison lattice according to the heterogeneity characteristics of the shale reservoir in research results of the macro heterogeneity and the micro heterogeneity. The method selects the shale reservoir with high heterogeneity both in vertical and lateral directions as stable and continuous output selecting layer and provides geological basis for shale gas' exploiting vertical selecting layer, plane division and development technique policy in the future.

The invention belongs to the technical field of federated learning and big data, and particularly relates to an optimization method for transverse federated learning, which comprises the following steps: S1, enabling a client to acquire data and preprocess the data; s2, processing the data of the client through an SMOTE-Non-IID model; s3, performing SMOTE-Non-IID model training through a client server architecture in transverse federated learning in combination with a data generation environment; s4, obtaining an optimization model of the SMOTE-Non-IID, and obtaining an evaluation result through prediction; according to the invention, each client synthesizes the data by adopting the SMOTE algorithm and receives the synthesized data from other clients at the same time, so that the problem of data heterogeneity in federated learning is effectively solved, and relatively high prediction accuracy can be ensured.

The invention discloses a traffic accident severity prediction method applied to a regional road network. The method comprises the following steps: 1, acquiring and preprocessing regional road networktraffic accident data; 2, based on the regional road network traffic accident data, establishing a potential category analysis model; 3, respectively establishing a CART decision tree model for eachsub-category according to a potential category analysis result; and 4, respectively establishing an accident severity model (considering independent variables and interaction items) based on binary logistic regression for each subcategory, and taking intersection points of sensitivity and specificity curves as model prediction classification thresholds. According to the method, the adverse effectof accident data heterogeneity on an analysis result can be reduced, the problems that a traditional traffic accident severity prediction model ignores interaction items and the comprehensive prediction effect of unbalanced data is poor are solved, and the prediction precision and goodness of fit of the accident severity model are improved.

Disclsoed is a predictive measurement model of connectivity strength between vehicle nodes based on spatio-temporal data analysis of vehicle networking in urban scenarios. Aiming at the connectivity problems caused by spatio-temporal data heterogeneity and topology frequent changes in vehicle networking, a neural network model based on tensor factor aggregation is constructed to predict the connectivity strength between vehicle nodes, thus providing model support for the connectivity research of vehicle networking without infrastructure and with infrastructure.

The invention discloses a suspicious event detection method and system based on URL heterogeneity. The method includes the steps of firstly, capturing network data packets in the user sending direction; secondly, analyzing the network data packets, and extracting URLs in the network data packets; thirdly, judging whether the URLs are highly-suspicious events or not based on the detection rule in a knowledge base, conducting deep detection if the URLs are highly-suspicious events, and otherwise, conducting detection through a preset detection scheme; fourthly, adding or deleting the detection rule as required, judging whether server ports required by the URLs are system reserved ports or not, judging the URLs to be security events if the server ports required by the URLs are the system reserved ports, and judging the URLs to be the highly-suspicious events if the server ports required by the URLs are not the system reserved ports; fifthly, judging whether domain names of the URLs are meaningful words or not, judging the URLs to be security events if the domain names of the URLs are meaningful words, and judging the URLs to be the highly-suspicious events if the domain names of the URLs are not meaningful words. The suspicious event detection method and system solve the problem that a traditional detection method has an effect on known malicious URLs and has no effect on unknown or non-captured URLs.

The invention discloses a CA-NARX water quality prediction method based on meteorological factors, and belongs to the technical field of intelligent water quality prediction data application. The method comprises the following steps: 1, ; data standardization, 2, creating a sample matrix, 3, determining an initial clustering center according to the quantile; (4) initial clustering is carried out according to the Euclidean distance; (5) the mean value of each class is used as a new clustering center; (6) clustering is carried out in batches according to the Mahalanobis distance between each sample and the clustering center; (7) clustering number screening is carried out; (8) the best clustering number is selected; according to the method, the problems of high cost and low prediction accuracy of water quality prediction of small and medium-sized reservoirs are mainly solved. Meanwhile, the problem that a traditional clustering algorithm is inapplicable to data heterogeneity and differentvariances is solved, and the training accuracy of the NARX neural network is improved to a certain extent.

A method and apparatus for rapid identification of column heterogeneity in databases are disclosed. For example, the method receives data associated with a column in a database. The method computes a cluster entropy for the data as a measure of data heterogeneity and then determines whether said data is heterogeneous in accordance with the cluster entropy.

The invention discloses a multi-source information fusion method and application thereof, and the method comprises the steps: standardizing and preprocessing collected data from a sensor data source in environment monitoring; and for the multi-source isomorphic data, introducing a support degree correction iterative fusion idea, comparing the difference between the fused evidence and the originalevidence to evaluate the support degree of the original evidence, and performing multiple iterations until convergence to obtain a final fusion result. For multi-source heterogeneous data, a multi-source fuzzy fusion algorithm is provided, the problem of multi-source perception data heterogeneity is solved, and the mass data fusion efficiency between different unit data formats is improved.

The present disclosure relates a system, method, and computer program for detecting anomalous user network activity based on multiple data sources. The system extracts user event data for n days from multiple data sources to create a baseline behavior model that reflects the user's daily volume and type of IT events. In creating the model, the system addresses data heterogeneity in multi-source logs by categorizing raw events into meta events. Thus, baseline behavior model captures the user's daily meta-event pattern and volume of IT meta events over n days. The model is created using a dimension reduction technique. The system detects any anomalous pattern and volume changes in a user's IT behavior on day n by comparing user meta-event activity on day n to the baseline behavior model. A score normalization scheme allows identification of a global threshold to flag current anomalous activity in the user population.

The invention discloses a semi-supervised heterogeneous software defect prediction algorithm based on GitHub, which comprises the following steps of: firstly, collecting a data set, and establishing adatabase of the data set; preprocessing the collected data; secondly, processing isomerous data, introducing an enhanced typical correlation analysis method which is composed of unified metric representation (UMR) and typical correlation analysis (CCA); finally, adding a cost-sensitive nuclear semi-supervised discrimination method. A semi-supervised heterogeneous software defect prediction algorithm based on GitHub is realized; the method has the advantages that the problem of data heterogeneity in software defect prediction is solved, a cost-sensitive CKSDA (kernel semi-supervised discriminant analysis) technology is put forward for the first time, different error classification costs are solved by utilizing a cost-sensitive learning technology, and a defect prediction effect is realized.

The invention discloses a data integration system for grid state detection, and an implementation method of the data integration system. The method comprises the following steps of: classifying grid state detection systems according to detection objects; establishing a special database for storing original detection data of various state detection systems; establishing a standard database for storing unified grid equipment ledger information; establishing standard data formats of various state detection systems, converting the original detection data into the standard data formats, and associating the standard data formats with the corresponding grid equipment ledger information; and publishing the data. By the invention, the problems of inconvenience in use, difficulty in integration and the like caused by difference in transmission modes of different manufacturers of the grid state detection systems and data heterogeneity are solved. Meanwhile, a unified and integrated data storage and publishing mode is provided, so that a user can acquire all data through a platform. By establishing the unified grid equipment ledger information in the standard database, the equipment ledger maintenance workload of a maintainer is reduced, working efficiency is improved, and labor cost is reduced.

The invention relates to the field of atmospheric pollutant space-time distribution calculation, and discloses a fine particulate matter (PM2.5) comprehensive domain space-time calculation inference method based on multi-source city big data. According to the method, PM2.5 concentration data of a fixed station and a sensor, satellite remote sensing aerosol optical thickness and other environment covariants are collected, an iteration vacancy filling-machine learning model is established, and the problems of data heterogeneity, space-time insufficient matching, sampling deviation and the like existing in multi-source data fusion are effectively solved. According to the method, multi-source data of a fixed station, a sensor, satellite remote sensing and the like are flexibly and efficiently fused, high-resolution spatial-temporal distribution of PM2.5 can be more accurately reconstructed, a high-resolution spatial-temporal distribution result of PM2.5 hourly concentration of a 1km grid is formed, the method is an important technical basis for realizing fine management and control of air quality, high potential pollution sources can be explored and positioned in real time, and finally, pollution discharge can be monitored and controlled in a targeted manner.

The invention relates to the field of federated learning, in particular to a heterogeneous model aggregation method and system based on federated learning, and the method comprises the steps of initializing a neural network model; the method also includes that each client contributes a part of local data and uploads the local data to the server to form a shared data set, and a CGAN model is trained; the client uses a local data set and a data set generated by the CGAN model to train a local model, predicts each data in the shared data set and uploads a prediction score to the server; the server calculates the prediction score deviation degree of each client, takes the reciprocal of a calculation result as a weight, calculates a global prediction score, and uses the global prediction score to perform knowledge distillation on the server model; the client downloads the prediction scores of other client models from the server for cooperative training; and model convergence is performed after multiple iterations. According to the invention, the problem of data heterogeneity of the client side can be solved, the client side model uploads and downloads the prediction score of the shared data set, and the communication traffic between the client side and the server side is reduced.

The invention discloses a photoetching hot area detection method based on federal personalized learning. The method comprises the following steps that global model parameters returned by each node by a central server are aggregated, common characteristics of each node are fused, the global model parameters are updated, and the latest global model parameters to each node are fed back; each node downloads a global model parameter from the central server, and then trains a local model parameter by using local data to find the optimal local model parameter under the current global model parameter so as to overcome model heterogeneity and data heterogeneity of different nodes; and after the local model parameters are finely adjusted, the nodes train all the parameters by using local data to find the optimal current parameters for searching common features of different nodes. According to the method, the problem of model overfitting caused by too little local data is solved; data between chip design manufacturers is protected, and privacy protection is achieved; and the stability and the overall precision of the federal personalized learning model in the heterogeneous environment are improved.

The invention belongs to the technical field of big data, and particularly relates to a research and judgment platform based on big data analysis. The research and judgment platform based on big dataanalysis comprises a big data analysis system, and the big data analysis system comprises a data acquisition and access part, a data storage part, a computing service part, an algorithm library, an analysis model, a system assembly, a business application part, a standard and safety system and an operation and maintenance management platform. The current situations of data heterogeneity, sharing difficulty, data chimney, application island and the like are solved; the problems of fuzzy construction specifications, incapability of intercommunicating systems, different standards, different dataformats, disordered name use and the like are solved; the problems that the information sharing amount and the sharing convenience degree cannot be guaranteed, the information amount is too small, theinformation sharing amount is influenced in turn, and a vicious circle of information sharing is formed are solved. The problems that data applications such as data cleaning, data management and dataanalysis are lacked, and functions such as intelligent analysis, intelligent early warning and intelligent handling of data are lacked are solved.

The present invention discloses systems and methods for measuring intra-tumor heterogeneity based on genetic information of a tumor. Such systems and methods may indentify genetic information of mutation specific to the tumor, determine a mutant-allele fraction for each mutated locus, calculate mutant-allele tumor heterogeneity (MATH), and measure the distribution of mutant-allele fractions among tumor-specific mutated loci of the tumor.

The invention relates to the technical field of document reading, in particular to a heterogeneous legal data-oriented multitask reading system and method, and the system comprises the following modules which are sequentially connected: a data input module which is used for inputting statistical and textual legal data; a data preprocessing module which is used for carrying out data cleaning and data conversion on the law data; a data analysis module which is used for analyzing the preprocessed data; a reading result processing module which is used for integrating the analyzed data to form structured reading result data; and a result pushing module which is used for feeding back the reading result data to the law researcher. Statistical analysis and machine reading understanding technologies are used at the same time, structured data such as statistical yearbook and unstructured data such as judgment documents, file materials and interview text records can be processed at the same time,and the problem of data heterogeneity is solved.

The invention provides a method and system for detecting an overflow loophole based on format heterogeneity and a storage medium. The method includes the steps that a network data packet is acquired and restored into files, the files are distributed to each detection branch according to file types, structure features of the file in each detection branch are extracted, target detection features aredetermined through the calculation of the difference degree, then clustering analysis is conducted on the features of each target detection feature, and the obtained file corresponding to the small part of the features is the file with an abnormal structure. According to the method, through a clustering algorithm of the structure features, a small part of abnormal files in a large number of filescan be quickly distinguished, deep detection is conducted only on the abnormal files, the efficiency of detecting the overflow loophole is effectively improved, and an inspired detection capability for unknown overflow loopholes is achieved.

The invention belongs to the technical field of dual-redundancy servo control systems, andparticularly relates to a real-time redundant communication system based on RS422 and CAN bus heterogeneity, which is used for communication of different intelligent single machines of a servo system for rocket flight control, realizes high-reliability communication with relatively low cost, has self-diagnosis and self-recovery functions, and ensures high reliability of flight control systems such as a rocket. The system comprises an intelligent single machine I and an intelligent single machine II, the intelligent single machine I comprises an RS422 interface and a CAN interface, and the intelligent single machine II comprises an RS422 interface and a CAN interface; An RS422 bus is arranged between the two RS422 interfaces, and a CAN bus is arranged between the two CAN interfaces; And combining the effective data of the RS422 into a message frame, each frame of data comprising a frame header, a frame length, a frame identifier, a data area, a check and a frame tail, the frame header and the frame tail being fixed byte bytes, and the frame length representing the total length of the effectivedata needing to be sent.

The invention relates to the field of credit risk assessment, and aims to provide a financial risk control cold start modeling method based on unsupervised field self-adaption. The method comprises the steps of data input and preprocessing, variational auto-encoder mapping, domain alignment based on adversarial, sample adaptive weighting based on weighted adaptation, pseudo-tag iterative optimization, parameter optimization and result output. The risk control modeling algorithm can be applied to the cold start stage of new service online lack of label samples, is suitable for a cold start scene without label samples, and is higher in precision and better in effect in a financial risk control scene. The problem of negative migration in existing migration learning and the problem of data heterogeneity in the financial risk control field can be effectively solved. The model training and deployment framework is high in generalization, can be effectively applied to other service scenes, and has good adaptability and mobility.

The invention discloses an Agent-based heterogeneous geographical information public service platform operation and maintenance data collection method. The method comprises the steps of establishing operation and maintenance data structure standards and operation and maintenance information classification standards; according to a geographical information public service platform operation and maintenance interface, establishing an exclusive adaptive Agent, standardizing data, and providing operation and maintenance data in a REST service form; calling an adaptive Agent service to obtain the operation and maintenance data by a report Agent, adding data tags for the data, and reporting the data to an aggregate Agent; and receiving the reported operation and maintenance data by the aggregateAgent, determining operation needed to be performed for the operation and maintenance data according to the data tags, collecting the data to a database, and returning a collection result to the report Agent. For the situations of platform heterogeneity, operation and maintenance data heterogeneity, different network environments where a platform is located and the like in geographical informationpublic service platform construction, unified data structure standards are established, and unified collection of the heterogeneous geographical information public service platform operation and maintenance data is realized.

The invention relates to a smart home evidence obtaining method based on an ontology, and belongs to the technical field of the Internet of Things. The method comprises the following steps: S1, acquiring smart home equipment data; s2, analyzing intelligent equipment data in the intelligent home environment; s3, uniformly expressing the evidence based on the ontology; s4, abstracting an evidence obtaining mode; and S5, sharing and applying evidence obtaining investigation. A unified representation method of different data formats generated by different smart home devices is provided based on ontology. The evidence integrity is fused to ensure the hash value, so that evidence sharing, transmission, exchange, management, evidence presentation and the like of case management among evidence obtaining investigation and other related personnel in the same case, the same case and different cases can be more effective and standardized, and the problem of data heterogeneity in smart home evidence obtaining is solved. The evidence obtaining mode based on ontology attributes effectively solves the problem of data sensitivity of smart home evidence obtaining knowledge sharing.

The invention provides a system and a method for drawing a visual data analysis graph based on gplot2, and the system comprises a user instruction module which is used for receiving a user instruction sent by a user; the import module is used for importing input data according to the user instruction; the drawing module is used for drawing a data analysis graph and comprises a drawing unit; the drawing unit comprises a single graph model and a combined model; the single graph model comprises a scatter diagram, a box plot and a violin graph; the combined model is a combination of two single-graph models, and comprises a scatter diagram and a box plot, a scatter diagram and a violin graph, and a box plot and a violin graph; the drawing module takes the received user instruction as a parameter to call each unit for drawing; and the export module is used for exporting the data analysis graph. According to the method, breakthroughs are made in multiple aspects of graph combination, form diversification, color specialization, data heterogeneity and the like, and richer and more professional choices are provided for data analysis visualization.