53 results about "Deep Web" patented technology

A method of enabling anonymous Internet users to publish and manage extensive, non-identifying personal data, including demographic, psychographic, needs, wants, interests, propensities, means to purchase, credibility and other data which in turn, enables a marketplace wherein such users, advertisers, websites, and other third-parties can mutually benefit from the commercial exploitation of such data. Advertisers can directly use the data to segregate the users into highly differentiated anonymous audiences for the purposes of targeting them with individualized marketing campaigns and then monitor user responses in near real-time. Websites can individualize their content to the profiles of visiting users. Users can share surface and deep web links with other users having similar profiles. Consumers participating in good faith are proportionately rewarded via revenue sharing, which they may withdraw from the marketplace or use to purchase and rent digital content offered in the marketplace's micropayment-enabled storefronts by other users and third-party content providers.

Systems, computer implemented methods and computer program products are provided for selectively capturing and / or evaluating information including content and metadata from across a network such as the “wide world web” (WWW), or more generally, the Internet. A deep web mining tool may be utilized to exploit the deep web by understanding forms, search engines and results pages. Moreover, deep web mining tool may be utilized to extract and exploit structured and unstructured content and metadata from web sites and documents, generate queries, capture and re-link web sites, crawl through web sites and non-HTML files and perform other aspects of obtaining and / or evaluating information.

A system that allows location and extraction of information from database driven web sites that are part of the deep web is described herein. The system uses an automatic wrapper generation mechanism that understands the meaning of deep web pages, extend search technologies capabilities, and help users extract information from database driven web sites. A method therefor is also described herein.

A method and apparatus for organizing deep Web services are provided. In one aspect, the method and apparatus obtains a collection of sources and their associated attributes and / or input modes, for instance, using a crawling algorithm. The method and apparatus uses this information to organize the sources into communities. A mining algorithm such as the hyperclique mining algorithm is used to obtain cliques of highly correlated attributes. A clustering algorithm such as the hierarchical agglomerative clustering algorithm is used to further cluster the cliques of attributes into larger cliques, which in the present disclosure is referred to as signatures. The sources that are associated with each signature form a community and a graph representation of the communities is constructed, where the vertices are communities and the edges are the shared attributes.

The invention discloses an RNN-based automatic picture description generation method. A deep web which is well trained in advance is firstly used for image feature extraction; non-noun and non-verb components are removed for words in the sentence; an LSTM network is finally used for joint training on the image features and lexical features; during the sentence generation process, a sentence formed by nouns and verbs is generated through the inputted image and the well-trained LSTM network; and then, through large corpus on the network, the final outputted sentence is generated. Automatic recognition can be realized, a digital image uploaded by the user is understood, and a natural sentence understood by a human being is generated.

The invention relates to a security test of WEB application, and aims to provide a WEB dynamic security flaw detection method based on JAVA. The WEB dynamic security flaw detection method based on JAVA is used for detecting the security flaws of a WEB application system, and comprises the following steps: modifying JAVA middleware; performing fuzzing test and dynamic flaw tracking. Due to the adoption of the WEB dynamic security flaw detection method, more WEB security flaw problems can be found rapidly, the security flaw range of black box test can be better covered, more deep WEB security problems can be found, the problem of high cost in white box test can be solved, the specific position of a flaw code can be determined more accurately, and lower missing report rate and error report rate in a detection process are ensured.

Methods for creating deep web mining plans from dynamic content filters are described. Dynamic content filters allow for the creation of deep web mining plans that are able to be used even when the structure of documents including web pages and PDF files changes or to apply the same filters to different variants of the pages generated in deep web mining. By basing the dynamic filters on ontological and semantic information many common changes in web page structure, terminology and format can be made without preventing the extraction of data from these pages in deep web mining. Dynamic content filters may be created by persons without expertise in the creation of deep web mining data extraction plans.

The invention discloses a data mining device based on Deep Web deep dynamic data and a method thereof. The device comprises a commercial server, a data storage server, a data index server and a file server; device systems based on the device comprise an acquisition simulative theme thesaurus management system, an acquisition task scheduling management system, an acquisition server and an acquisition storage scheduling system. The invention provides a dynamic data acquisition means with large quantity, high data quality, strong real-time property and easy deep analysis, and makes up the defect that the quantity and quality of the conventional search engine are all limited; and the invention has simple and practical operation, rich customization function and good expandability and robustness, and a user can customize, acquire and reestablish a management database according to the specific or strongly-monographic requirements, provide data utilization efficiency to great extent, and expand data source and information resource.

A method for discovering data source used on deep web data integration includes setting up station root chain queue and local chain queue, taking page chain with highest score out from local chain queue and using creepage module to download it, processing downloaded page by table sorter, adding said page into deep web data source if it has table query interface, processing downloaded page by page sorter and returning back to step of taking page chain if subject score is less than threshold, picking up chain address in page then placing it into local chain queue, repeating step of taking page chain to step of picking up chain address for realizing automatic creepage of deep web data source.

The invention discloses an integration method of Deep Web query interface based on tree merging. A pattern tree is used for representing the query interface, and the structural features of the tree are utilized to embody the logical relation implied in the physical layout between query properties. Except for calculating the semantic similarity of attribute in conventional pattern matching, the matching process also introduces the structural similarity of attribute in the pattern tree, puts forward the method for calculating the structural similarity between nodes, thereby improving the accuracy of attribute matching. The integration of query interface is realized based on tree merging, which can not only inherit the structural features of the initial query interface, but also realize the accession of new query interfaces by one merging with good expansibility. Except for generating integration interface, the invention can also conveniently generate the mapping relation of attributes between the original query interface and the integration interface.

The invention discloses a self-adaptive incremental deep web data source discovery method. According to the method, the deep web data source discovery processes comprise a website positioning stage and an in-web searching stage, and in the website positioning stage, a website discovery mechanism is introduced so that website data can be efficiently expanded and the creep effect can be improved; a self-adaptive sorting mechanism is adopted in website and in-web linkage so that a deep web website and a queryable form can be discovered more rapidly. The method achieves automatic incremental efficient deep web data source acquisition, can be applied to deep web data integration and a hidden web crawler, and meanwhile is also suitable for building on-line database catalog websites.

Methods, apparatuses, and articles for receiving a search request associated with a hospitality service from a client device, the search request including a plurality of search criteria, are described herein. Additionally, the methods, apparatuses, and articles further return to the client device an answer page having a plurality of answers potentially associated with the hospitality service, the plurality of answers identifying a plurality of information locations having information potentially associated with at least a one of the plurality of search criteria, where at least one of the answers includes at least one input field of a query answer page for entry of at least one feature of the hospitality service, the query answer page to be dynamically generated by one of the information locations in response to a query.

A method for organizing deep Web services is provided. In one aspect, the method obtains a collection of sources and their associated attributes and / or input modes, for instance, using a crawling algorithm. The method uses this information to organize the sources into communities. A mining algorithm such as the hyperclique mining algorithm is used to obtain cliques of highly correlated attributes. A clustering algorithm such as the hierarchical agglomerative clustering algorithm is used to further cluster the cliques of attributes into larger cliques, which in the present disclosure is referred to as signatures. The sources that are associated with each signature form a community and a graph representation of the communities is constructed, where the vertices are communities and the edges are the shared attributes.

A system and method for searching deep web services are provided. The system and method in one aspect allow organizing communities, sources and schema attributes in a multi-tier containment relationship; searching representative schema attributes in one or more communities; searching representative services in one or more communities; searching for related schema attributes; and searching for related communities.

A system for analyzing images and video that is capable of recognizing, classifying, and processing the context and meaning contained therein in a manner similar to human intuitive understanding of such context and meaning. Images and video are gathered through a crowdsourcing portal, fixed cameras, and other remote sensing devices. Real world data relevant to the images and video is gathered using a deep web extraction engine. The resulting inputs are analyzed for context and meaning using machine learning algorithms, whose outputs and reviewed and adjusted by humans through a crowdsourcing portal.

Methods, apparatuses, and articles for receiving a search request associated with a transportation service from a client device, the search request including a plurality of search criteria, are described herein. Additionally, the methods, apparatuses, and articles further return to the client device an answer page having a plurality of answers potentially associated with the transportation service, the plurality of answers identifying a plurality of information locations having information potentially associated with at least a one of the plurality of search criteria, where at least one of the answers includes at least one input field of a query answer page for entry of at least one feature of the transportation service, the query answer page to be dynamically generated by one of the information locations in response to a query.

The invention discloses a sorting technique of a deep web database only providing a simple query interface. The method comprises the following steps: setting the result model and the result webpage data area content of the deep web database as two sorting characteristics and respectively establishing a sorter based on the result model and a sorter based on the result webpage data area content; sorting based on the result model to obtain probability omega of the simple query interface, which is based on the result model and belongs to the field D; sorting based on the result webpage data area content to obtain probability theta of the simple query interface, which is based on the result webpage data area content and belongs to the field D; and integrating the results of the two sorting techniques and determining the category of the deep web database to be sorted according to weight and sorting threshold value. The method of the invention can realize automatic sorting of the deep web database only providing the simple query interface. Experiments prove that the method of the invention enjoys high degree of accuracy.

A data processing system and a computer implemented method for searching registered websites including multimedia content according to a user query. The data processing system includes a mediator server with a database storing the multimedia content from the registered websites and an application configured to receive and apply the user's query to the database and provide search results at least one resolution. The computer implemented method includes: (i) receiving multimedia content of the registered websites and storing the content in a database, (ii) receiving and applying the user's query, and (iv) providing search results at least one resolution.

The invention relates to a network information acquirement method and system and an enterprise information searching system. The method includes the steps of obtaining webpage information correlativeto assigned information, obtaining an object page of a correlative webpage according to a selected retrieval strategy, and extracting data information in the object page. Through a crawler technique and the targeted retrieval strategy, mining of data in a deep web is completed, users can obtain a great amount of effective data in a short time and have no need to search each independent website oneby one, a one-stop information service is provided for the users, and the efficiency of collecting the data is improved.

Methods, apparatuses, and articles for receiving a search request associated with a hospitality service from a client device, the search request including a plurality of search criteria, are described herein. Additionally, the methods, apparatuses, and articles further return to the client device an answer page having a plurality of answers potentially associated with the hospitality service, the plurality of answers identifying a plurality of information locations having information potentially associated with at least a one of the plurality of search criteria, where at least one of the answers includes at least one input field of a query answer page for entry of at least one feature of the hospitality service, the query answer page to be dynamically generated by one of the information locations in response to a query.

The invention discloses a unique constraint based Deep Web entity identification method. The unique constraint based Deep Web entity identification method includes steps of firstly sorting problems into K-map cluster problems from the aspect of rigid constraint, and providing a clustering algorithm; and expanding the k-map cluster problems to the flexible constraint, sorting entity identification problems into optimization problems, and providing matching algorithm. In the unique constraint based Deep Web entity identification method, recording connection and data integration are integrated to be applied in overall situation, and the k-map cluster problems under rigid constraint are provided and are expanded to the flexible constraint. In the meantime, overall policy is determined on the basis of similarity of attribute values and relevancy among attributes in a same record, incorrect values can be identified and are differentiated from correct values from the beginning, and better identification effect is achieved. Clustering the attribute values can show clustering effect with finer grid.

The embodiment of the invention provides a deep web mobile search method, a server and a system. The method comprises the following steps: obtaining member search engine representing values of deep web member search engines; receiving search requests sent by a client, and obtaining search request information from the search requests; calculating the matching degree of the search requests and the member search engines according to the search request information and the member search engine representing values, and selecting the member search engines from the member search engine set according to the matching degree for carrying out content data search; and sending the searched content data to the client. The invention is used for integrating the deep web member search engines, realizing the representing of the deep web member search engines and automatically selecting the member search engines for search through the deep web member search engine representing values.

The invention discloses an automatic extraction method oriented to data of deep web pages, and belongs to the field of computer data mining. The automatic extraction method includes obtaining two deep web pages of the same website at first, and respectively marking the two deep web pages as a first page and a second page; converting HTML (hypertext markup language) documents of the first page and the second page into XHTML (extensible hypertext markup language) documents; then removing noise of the first page and the second page; eliminating repeated modes of the first page and the second page to generate a webpage data extraction wrapper; removing noise of the page with the data to be extracted at first when the page is extracted; marking the page by the webpage data extraction wrapped after the noise of the webpage is removed, and finally extracting the marked page. By the aid of the automatic extraction method, efficiency of a repeated mode elimination algorithm and efficiency of a matching algorithm are improved, extraction complexity is reduced, the matching algorithm and an extraction algorithm, which are designed according to characteristics of the repeated mode elimination algorithm, in the method are simple and speedy in process, and data extraction accuracy is improved.

The invention discloses a high efficiency data acquisition method and system based on deep web reptiles; the method comprises the following steps: using a known name as the keyword so as to carry out preliminary search sampling, thus obtaining corresponding number information and number rules; grouping all number information according to number rules, and ranking the number information in an ascending order, wherein a data gap is built between every two adjacent number information; traversing and searching the data according to the number information and data interval in a rising sequence, thus obtaining the data complete set. The method and system can grab various industry information data including but not limited to: enterprise business information, book information, goods information, and trial files; the method and system can grab deep web data on industrial websites, can accurately obtain related data complete set, and can finish mass effective data acquisition in a short time.

System and method for prevention of threat are disclosed. The system includes a processing subsystem. The processing subsystem includes a data extraction module configured to extract data from one or more internal sources such as a router, a firewall or a security solution and one or more external sources such as a deep, a dark and a surface web. The processing subsystem also includes an analysis module configured to analyse the data by using at least one threat analysis method for detection of the threat, a rule generation module configured to generate one or more rules to enable prevention of the threat detected, an implementation module configured to implement the one or more generated rules on at least one node associated with the one or more internal sources for prevention of the threat.

System and method for data analysis and detection of threat are provided. The system includes a processing subsystem. The processing subsystem includes a reconnaissance module configured to acquire data from one or more internal sources and one or more external sources. The data from the one or more internal sources includes the data from at least one of a firewall, a router and a security solution. The data from the one or more external sources includes the data from at least one of a deep web, a dark web and a surface web. The processing subsystem also includes an analysis module configured to analyse the data by using at least one threat analysis method for detection of threat and a dissemination module configured to present detected threat in one or more forms. The system also includes a memory configured to store data acquired from the one or more sources.

A method for discovering data source used on deep web data integration includes setting up station root chain queue and local chain queue, taking page chain with highest score out from local chain queue and using creepage module to download it, processing downloaded page by table sorter, adding said page into deep web data source if it has table query interface, processing downloaded page by page sorter and returning back to step of taking page chain if subject score is less than threshold, picking up chain address in page then placing it into local chain queue, repeating step of taking page chain to step of picking up chain address for realizing automatic creepage of deep web data source.