A method and device for batch submitting data to database

Abstract

The invention discloses a method and device for submitting data to a database in batch. The method comprises the following steps: after a data submitting device detects that the number of data pieces cached in a data-caching queue exceeds a set threshold value, data is completely extracted out and a SQL data insert statement corresponding to each piece of data is generated for each piece of data; then, the SQL data insert statements are written into an affair in sequence to be submitted to a middleware according to the magnitude sequence of the main keys of each piece of data; the middleware converts each SQL data insert statement in the affair into a corresponding database command to be sent to a database, and if an abnormal execution result fed back from the database is received, the abnormal execution result is reported to the data submitting device; the data submitting device determines the main keys of error data according to the reported abnormal execution result; the data of the main keys, behind the main keys of the error data, is stored into the data-caching queue again. Through the application of the invention, the phenomenon that correct data is discarded can be avoided, and the in-batch data storing efficiency is improved.

Description

Technical field [0001] The present invention relates to database storage technology, in particular to a method and device for submitting data to a database in batches. Background technique [0002] At present, large-scale business application systems such as OA (Office Automation), ERP (Enterprise Resource Planning), CRM (Customer Relationship Management, customer relationship management system), and log analysis systems all need to be stored in a database Of massive business data. Generally speaking, data can be submitted one by one in the database, that is, every time a piece of business data is generated at the data source, the middleware submits the data to the database; however, because only one piece of data can be submitted to the database at a time, the system has a large amount of data. Data needs to be stored in the database, which leads to frequent interaction with the database during the submission process, which occupies a lot of database resources, and the speed of...

AI Technical Summary

Problems solved by technology

[0004] However, in the existing method of submitting data to the database in batches, if there is an error in the data submitted in batches, all the data submitted in batches will be rolled back; Wrong data and correct data, so, either discard the wrong data together with the correct data to ensure that the wrong data is eliminated, which will cause part of the correct data to be discarded and cannot be stored in the database; or submit the rolled back data one by one Submit to the database to ensure that the correct data is successfully stored in the database. However, the method of submitting data one by one will lead to low data storage efficiency

Images