Web page element searching method and device, and computing equipment

Abstract

The invention discloses a web page element searching method and device, and computing equipment. The web page element searching method comprises the following steps: in response to a request of a user for searching web page elements on a Chrome browser, sending a domain name and a URL of a current site to a server side, and then obtaining a template returned by the server side, wherein a target field and an extraction rule of the target field are recorded in the template; when the extraction rule of the target field is absolute positioning, extracting the target field in the current page by using one of an Xpath selector, a CSS selector and an ID selector; when the extraction rule of the target field is relative positioning, extracting a target container in the current page, and extracting the target field in the target container; and taking the extracted target field as a search result of the web page element. The invention also discloses corresponding computing equipment and device.

Description

technical field [0001] The invention relates to the field of reptiles, in particular to a web page element search method and device and computing equipment. Background technique [0002] Grabbing data from Internet websites is a common requirement. The usual way is to write a program or script to obtain the response from the target website, and then parse out the required element fields from the response according to your own needs. When there are many fields, many target websites, and target elements are sporadic according to data changes, the difficulty and workload of parsing element fields will increase accordingly. [0003] One of the ways to find web page elements is manual parsing. This method requires programmers to customize a set of crawling programs for each target website, obtain the response of the target website through network requests, and then parse them one by one from the response HTML. required fields. Before parsing the fields, the developer needs to a...

AI Technical Summary

Problems solved by technology

For example, in some cases, the XPath collected by XPath Helper contains randomly generated styles, which will change after the website is updated and released, causing the XPath to fail

In addition, the XPath Helper Chrome plug-in also has the following problems: it cannot be persisted on the ground, has no memory function, the selected content is lost after the page is refreshed, multiple fields cannot be selected at the same time, manual copying is required, and "crowded elements" cannot be selected , the element pointed to by the mouse is the topmost element

[0013] In summary, the above-mentioned existing crawler technology has problems such as difficulty in developing web page element plug-ins, and insufficient adaptability to dynamic pages.

Images