Biological big data analysis system and method based on Linux single command line

Abstract

The invention discloses a biological big data analysis system and method based on a Linux single command line. The system comprises an access module which is linked to a Shell mode interface when a user accesses; a Shell mode interface used for inputting a Linux single command line and generating a job number, wherein the Linux single command line at least comprises a program unit name and a to-be-analyzed data file name; a user level acquisition unit; a user resource management module which is used for allocating system sub-resources with corresponding levels to the users according to the levels of the users output by the user level acquisition unit; the analysis module comprises a plurality of program units; and a job scheduling management module which is used for distributing the job number to a job queue corresponding to the user level, scheduling a program unit associated with the job number and executing to obtain an analysis result. Data analysis and job submission can be completed only through a Linux single command line, graded management is conducted on resources and job queues, the resources are optimized, the use efficiency is improved, and the system stability is improved.

Description

technical field [0001] The present invention relates to data analysis in the fields of bioinformatics multi-omics and computational biology, in particular to a biological big data analysis system and method based on a Linux single command line. Background technique [0002] With the new generation of high-throughput data technologies such as next-generation gene sequencing technology and protein profiling technology, more and more applications are used to solve biological problems, especially the research of human diseases. It makes it possible to carry out all-round and multi-level research on diseases from the perspectives of genome level, transcriptome level, proteome level, and metabolome level. With the help of bioinformatics analysis methods, multi-omics integration research can explain diseases more systematically. It provides a powerful tool for the diagnosis and treatment of diseases (such as tumors). [0003] Most of the existing tumor big data analysis systems ar...

AI Technical Summary

Problems solved by technology

In this process, the user needs to compile, debug, and run the program separately, and finally submit the job to the server through the qsub program. The whole process is time-consuming and labor-intensive, and seriously restricts the promotion and use of the analysis platform.

Users are required to have the experience of using Linux commands proficiently, because each operation needs to be completed by inputting Linux commands, so the user's operation is complicated; for the user's data analysis process, it is more necessary for the user to manually write the code to run and debug the program; For customized data analysis, there is no existing code that does not need to be modified, which increases the complexity of use for users without programming foundation and restricts the use of this method

[0005] In addition, regardless of whether it is based on windows or Linux, for multiple users to use the analysis system for data analysis at the same time, it will be limited by the performance of the server, including threads, memory and storage space

Under the Linux server system, there are generally situations where multiple users use it at the same time. Under this system, if the resources (including threads, memory, and storage space) used by each user are not controlled, the performance of the server will be severely restricted. Thereby greatly reducing the utilization efficiency of the server

Images