Browser request optimization method and device, terminal and storage medium

An optimization method and a technology for optimizing devices, which are applied in the fields of instrumentation, electrical digital data processing, calculation, etc., can solve the problems of limited application scenarios, limited use scenarios of browser crawlers, and inability to satisfy users, etc., and reduce the data used for rendering , network request data optimization, and the effect of improving request efficiency

Pending Publication Date: 2019-08-13
北京墨云科技有限公司
View PDF6 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, the above solutions have the following defects: For example, based on browser restrictions, all data requests are sent through the browser itself, which is close to transparency to the upper layer caller, and cannot achieve fine-grained control of request data (such as modifying fixed cookies, adding deformities request data), unable to dynamically control the concurrency of network requests, and unable to customize the processing of request middle layer data (such as 301 status, etc.). crawler control requirements, so that it is subject to many restrictions in application scenarios

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Browser request optimization method and device, terminal and storage medium
  • Browser request optimization method and device, terminal and storage medium
  • Browser request optimization method and device, terminal and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings. It should be noted here that the descriptions of these embodiments are used to help understand the present invention, but are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not constitute a conflict with each other.

[0028] Among them, in the description of the embodiments of this application, unless otherwise specified, " / " means or means, for example, A / B can mean A or B; "and / or" in this article is only a description of associated objects The association relationship of indicates that there may be three kinds of relationships, for example, A and / or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. In addition, in the description of the embo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a browser request optimization method. The method comprises the following steps: intercepting network request data of a crawler through a hook function; removing useless request data in the network request data to obtain useful request data; obtaining to-be-rendered data according to the useful request data; and sending the to-be-rendered data to a browser through a hook function, so that the browser performs rendering. According to the browser request optimization method and device, the terminal and the storage medium, the browser crawler request efficiency can be improved.

Description

technical field [0001] The invention relates to the technical field of browser request optimization, in particular to a browser request optimization method, device, terminal and storage medium. Background technique [0002] With the advent of the H5 era, more and more sites load a large amount of js code in the webpage to render the website browsing effect. Many websites will use js to navigate the jump instead of using it directly. Tags; for search engines, these pages that are redirected through js are difficult to obtain. According to this situation, there are several solutions: based on the browser's own plug-in and extension mechanism, integrated into the browser to achieve content collection; there are also some third-party headless browsers, such as PhantomJS, Selenium, etc. to obtain the target Page content; and calling some existing browser kernels to directly access the target page to obtain content. [0003] However, the above solutions have the following defe...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/38
CPCG06F8/38
Inventor 谢鑫
Owner 北京墨云科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap