Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

An anti-over-crawler method based on sdn

An anti-crawler and crawler technology, which is applied in the field of information security, can solve the problems that the security of sensitive data in the data center cannot be guaranteed, and the illegal crawler has no targeted defense means, so as to achieve the effect of protecting data in sensitive areas.

Active Publication Date: 2021-03-12
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The situation awareness of crawlers cannot be effectively targeted at the sensitive data areas of the data center and specific service requirements, there is no targeted defense against illegal crawlers, and the security of sensitive data in the data center cannot be guaranteed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • An anti-over-crawler method based on sdn
  • An anti-over-crawler method based on sdn

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] A kind of SDN-based anti-excessive crawler method, comprises the following steps:

[0031] a. Build an anti-crawler SDN environment

[0032] Build an anti-crawler SDN environment in the data center;

[0033] b. Deploy the sensitive data area on the SDN controller

[0034] By writing the API, the SDN controller will grab the robots.txt file disclosed by each newly added server, obtain the formulated crawler access rules, and integrate the rules about the same User_Agent, and integrate the Disallow page directory Define it as a sensitive data area, and deploy this information on the SDN controller;

[0035] c. Blacklist level restriction

[0036] Monitor all crawlers that request access to the network through the SDN monitor. If the IP is in the blacklist, the access operation will be restricted; if there is no request to access any sensitive data area, it will be defined as a safe crawler; if there is a crawler When a program tries to access the sensitive data area s...

Embodiment 2

[0039] A kind of SDN-based anti-excessive crawler method, comprises the following steps:

[0040] a. Build an anti-crawler SDN environment

[0041] Build an anti-crawler SDN environment in the data center;

[0042] b. Deploy the sensitive data area on the SDN controller

[0043] By writing the API, the SDN controller will grab the robots.txt file disclosed by each newly added server, obtain the formulated crawler access rules, and integrate the rules about the same User_Agent, and integrate the Disallow page directory Define it as a sensitive data area, and deploy this information on the SDN controller;

[0044] c. Blacklist level restriction

[0045] Monitor all crawlers that request access to the network through the SDN monitor. If the IP is in the blacklist, the access operation will be restricted; if there is no request to access any sensitive data area, it will be defined as a safe crawler; if there is a crawler When a program tries to access the sensitive data area s...

Embodiment 3

[0049] A kind of SDN-based anti-excessive crawler method, comprises the following steps:

[0050] a. Build an anti-crawler SDN environment

[0051] Build an anti-crawler SDN environment in the data center;

[0052] b. Deploy the sensitive data area on the SDN controller

[0053] By writing the API, the SDN controller will grab the robots.txt file disclosed by each newly added server, obtain the formulated crawler access rules, and integrate the rules about the same User_Agent, and integrate the Disallow page directory Define it as a sensitive data area, and deploy this information on the SDN controller;

[0054] c. Blacklist level restriction

[0055] Monitor all crawlers that request access to the network through the SDN monitor. If the IP is in the blacklist, the access operation will be restricted; if there is no request to access any sensitive data area, it will be defined as a safe crawler; if there is a crawler When a program tries to access the sensitive data area s...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an SDN-based anti-excessive crawler method, which belongs to the field of information security, and is characterized in that it comprises the following steps: a. building an anti-crawler SDN environment; Grab the robots.txt file disclosed by each newly added server, obtain the formulated crawler access rules, and integrate the same User_Agent rules, define the integrated Disallow page directory as a sensitive data area, and deploy this information On the SDN controller; c, blacklist level restriction. The present invention deploys a specific SDN network in the data center, aims at the sensitive data area and specific service requirements of the data center, conducts situational awareness to the crawlers, and adopts targeted defense measures against illegal crawlers according to the service requirements, thereby effectively protecting the data center Security of sensitive data.

Description

technical field [0001] The invention relates to the technical field of information security, in particular to an SDN-based anti-excessive crawler method. Background technique [0002] SDN originated from a campus project called clean slate at Stanford University. It is an innovative network architecture, and its core idea is to separate the forwarding plane and the control plane. The centralized controller uses standard interfaces to configure and manage various network devices, making network management more centralized and refined. As the prototype implementation mode of SDN, OpenFlow fully embodies the separation of management and control of SDN. Therefore, people generally regard OpenFlow as the communication standard of SDN, just as the TCP / IP protocol is the communication standard of the Internet. [0003] A web crawler is a program that automatically extracts web pages. It downloads web pages from the World Wide Web for search engines and is an important component ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/06
CPCH04L63/10H04L63/145H04L63/1466
Inventor 杨挺董冰汪文勇
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com