CN108306918B - Automatic website access information acquisition method based on program dynamic analysis - Google Patents
Automatic website access information acquisition method based on program dynamic analysis Download PDFInfo
- Publication number
- CN108306918B CN108306918B CN201710033453.8A CN201710033453A CN108306918B CN 108306918 B CN108306918 B CN 108306918B CN 201710033453 A CN201710033453 A CN 201710033453A CN 108306918 B CN108306918 B CN 108306918B
- Authority
- CN
- China
- Prior art keywords
- user
- webpage
- website
- files
- variable
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- BUGBHKTXTAQXES-UHFFFAOYSA-N Selenium Chemical compound [Se] BUGBHKTXTAQXES-UHFFFAOYSA-N 0.000 claims abstract description 32
- 229910052711 selenium Inorganic materials 0.000 claims abstract description 32
- 239000011669 selenium Substances 0.000 claims abstract description 32
- 238000005516 engineering process Methods 0.000 claims abstract description 7
- 239000003795 chemical substances by application Substances 0.000 claims abstract 5
- 238000012546 transfer Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 2
- 238000003780 insertion Methods 0.000 claims 2
- 230000037431 insertion Effects 0.000 claims 2
- 230000006399 behavior Effects 0.000 description 16
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/535—Tracking the activity of the user
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method for automatically acquiring website access information of a user at different time periods based on program dynamic analysis. The method comprises the steps of firstly acquiring the behavior of a user for accessing a webpage by using a dynamic instrumentation technology, then collecting the behavior and storing the behavior in a variable, meanwhile caching the webpage accessed by the user according to a domain name by using an agent, then automatically accessing the same webpage for multiple times at regular time by using a tool Selenium, and then comparing collected files to find the behavior difference of the user. The method has obvious effect on automatically acquiring the dynamic behavior of the user for many times.
Description
Technical Field
The invention relates to a dynamic analysis framework for simulating a user to automatically visit a website for many times at regular time, recording the behavior of the website and analyzing the behavior, and belongs to the field of internet application.
Background
Along with the development of the internet, a JavaScript (JS for short) scripting language is widely applied to a Web client. Js and the like, so that writing a server-side program by using JavaScript becomes feasible, more and more JavaScript applications appear at the server side, and the power of a JavaScript language is reflected. Research and applications for JavaScript are also becoming more and more popular today.
The Selenium is a tool for Web application client program testing. The Selenium test runs directly in a browser, simulates the real operation of a user based on JavaScript in combination with WebDriver thereof, and supports various browsers (such as Safari, IE, Firefox, Chrome, etc.), which can run on various operating systems.
Jalangi is a framework for dynamically analyzing front-end and back-end JavaScript code, and can monitor each operation of the JavaScript by adding hooks, such as reading/writing of variables, unary/binary operation, function/method call and the like. According to actual needs, JavaScript code can be instrumented by Jalangi to extract relevant information.
Disclosure of Invention
The technical problem is as follows: the invention aims to simulate a user to access the same webpage at different time periods, automatically record the dynamic behavior of the user, store the dynamic behavior into a local file in a serialized manner, and compare the difference of webpage contents at different time periods. The technology overcomes the defects that the prior crawler technology can only obtain the existing content of the webpage in batch and can not record and analyze certain special behaviors of the user in batch.
In order to achieve the purpose, the JavaScript dynamic instrumentation tool Jalangi is used for instrumentation the JavaScript source codes of the whole webpage to acquire the information required by the user (such as a JavaScript file executed by the user and a JavaScript variable containing the information), and then the JavaScript variable in the instrumented webpage is acquired through the Selenium. Since the Selenium supports multiple languages (including Java), we save the acquired information to a local file using a streaming method in Java, and set timing to acquire dynamic behavior of users and access server information in bulk. Furthermore, the Selenium can process the page using the executeScript method to execute a JS script, from which we execute a JS script or return a JS variable.
The method specifically comprises the following steps:
step 1: and performing instrumentation on JavaScript codes executed in the webpage accessed by the user.
And (3) using a dynamic instrumentation tool Jalangi of JavaScript, and instrumenting JavaScript codes executed by a user when a webpage is loaded so as to acquire the dynamic characteristics of the JavaScript codes. Since the JavaScript allows the object to dynamically bind the attribute, a JS variable can be added to the object of the required information, and the JS variable is added and transmitted during the execution of the webpage, so as to record the dynamic behavior of the user.
Step 2: and collecting the dynamic behavior of the user in one visit.
Step 2.1: setting a proxy server by using a Firefox browser, so that a webpage executed by a user passes through the proxy server, calling Jalangi by the proxy server for instrumentation, and returning a page to the user to contain the JS variable added in the step one; acquiring information contained in the variable by using an executeScript method of the Selenium, and serializing the information into a local file by using a stream method of Java for permanent storage; in addition, a piece of JS script can be executed in the Selenium by using exenutscript to acquire information required by us, such as url information containing an src attribute in a script tag and the like.
Step 2.2: and caching the webpage accessed by the user by calling a Python script by using the proxy mitmprroxy, storing the files according to url domain names of the requested Web contents, recording md5 codes of each file (the files are different, and the generated md5 codes are also different), and copying the recorded files to the local.
And step 3: and simulating the user to visit the same website for many times at regular time, and recording the batch information of the website.
The Selenium has the property of automatically turning on and off the Firefox browser, thereby simulating a user's access to a web site. The action can be repeatedly executed for a plurality of times in a timing manner by utilizing the timer so as to fulfill the aim of accessing the same website for a plurality of times in a timing manner by the same user. Because the files cached by the mitmprroxy are continuously recorded, the cached files are deleted after the website is visited and before the website is revisited, so that the result of one visit is ensured to correspond to each record. Therefore, after the webpage is accessed and the browser is closed within the specified time, the file is copied and the cached file is deleted, and the delayed action of the Selenium is utilized to ensure that the browser does not access the webpage within the specified time, and after the cache is deleted, the same webpage is continuously accessed.
And 4, step 4: and analyzing and comparing the acquired dynamic behaviors of the user.
Analysis was performed on batches of files obtained by mitmprroxy: if the files accessed by the user are different, the generated md5 is different, and the difference of the dynamic behaviors of the user accessing the same website can be distinguished by comparing the differences of md5 codes; for multiple accesses of a user, the time of the current access is taken for distinguishing. We compare the information obtained from the first access with the subsequent files by md5 codes, and record the file that existed before but did not exist after the comparison with the domain name as the key.
The JS variable information acquired by the Selenium can be analyzed in the same manner.
Compared with the prior art, the invention has the following beneficial effects:
by using the technologies of Selenium, Jalangi, a proxy server and the like, a user can be simulated to visit the same website for many times at regular time, and the JavaScript code of the webpage is inserted by using Jalangi, so that the dynamic behavior characteristics of the user can be conveniently tracked, detected and acquired.
In addition, a proxy server is arranged in the browser, when a user accesses a webpage, the webpage passes through the proxy server, the proxy server calls Jalangi to perform instrumentation, the returned webpage contains a JS variable of information recorded by the user, the information of the variable can be automatically obtained by using Selenium, and the variable does not exist before the webpage instrumentation, so that the defect that the existing crawler technology can only obtain the static content of the webpage and cannot crawl the same user to access the content for many times can be effectively overcome. And because the proxy server can cache user access information before and after instrumentation according to the domain name, including JS, HTML and the like, the difference that one user accesses the webpage for many times can be compared, and the crawler technology can only crawl the source code of the whole webpage and cannot distinguish different domain name contents.
Drawings
FIG. 1 is a flow chart of the JS variables for the stored information when the user accesses a web page once;
FIG. 2 is a flowchart of obtaining md5 file without domain name when a user accesses a web page once;
FIG. 3 is a flowchart of file acquisition when a user accesses the same web page multiple times;
FIG. 4 is a flowchart of a comparison process for acquired files;
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
Step 1: and performing instrumentation on JavaScript codes executed in the webpage accessed by the user.
Using dynamic instrumentation tool Jalangi of JavaScript, using Selenium to imitate the user to access the web page, and setting up the proxy server before Selenium starts the firefox browser, including setting up the IP address of the proxy server, the port address of the proxy server, and so on. And then starting a browser, using a get method of the browser to transmit a website to be visited by a user, loading a page, and when the page is loaded, performing instrumentation on a JavaScript code executed by the user to obtain the dynamic characteristics of the webpage.
Step 2: and acquiring one-time dynamic behavior of one-time access of the user.
Step 2.1: and setting a proxy server by using a Firefox browser, so that a webpage executed by a user passes through the proxy server, calling Jalangi by the proxy server for instrumentation, returning a webpage to the user to contain the JS variable added in the step one, acquiring information contained in the variable by using a Selenium executeScript method, returning the variable of the object type by using the method, and converting the variable into a type which is convenient to compare, such as a JSON format.
When Selenium is adopted for automatic variable acquisition, the problem of cross-iframe must be noticed: a website may contain multiple iframes, and the dynamic behavior of a user spans multiple iframes; when acquiring a variable in an iframe, the variable must be acquired by transferring to the iframe by using a method of the browser switch to (). frame (); when transferring from one iframe to another, the switchTo (). defaultContent () method must first be used to transfer to the default content and then to the iframe to be entered.
Then, the Java files are serialized into a local file by using a Java streaming method for permanent storage, and a regular expression (? Matching the website accessed by the user to obtain the secondary domain name of the website as the name of the folder, and using the current access time as the name of the file, to distinguish different accesses and facilitate later comparison.
Furthermore, a JS script can be executed in the Selenium by using exenutscript to acquire information required by us, such as url information containing src attribute in script tag and the like. Different folders may be created to store different information we have obtained.
Step 2.2: and calling a Python script cache webpage accessed by the user by using the proxy mitmprroxy, storing the files according to url domain names of the requested Web contents respectively, and automatically recording md5 codes of each file. These recorded files are then copied locally.
The time node at which the file is copied is critical. We set up a server and listen to a port: when monitoring that the port is accessed, starting copying; and after copying is finished, deleting the files cached by the proxy mitmprroxy.
And step 3: and simulating the user to visit the same website for many times at regular time, and recording the batch information of the website.
The Selenium has the property of automatically turning on and off a browser for firefox, thereby simulating a user's access to a web site. The action can be repeatedly executed for a plurality of times in a timing manner by utilizing the timer so as to fulfill the aim of accessing the same website for a plurality of times in a timing manner by the same user.
Since the files cached by the mitmprroxy are continuously recorded, the cached files are deleted after the website is visited and before the website is revisited, so that each record corresponds to the result of one visit. Therefore, after the webpage is accessed, the file needs to be copied and the cached file needs to be deleted within the specified time when the browser is closed, and the browser is guaranteed not to access the webpage within the specified time by using the thread () function, and continues to access the same webpage after the cache is deleted.
After a browser is started, waiting for a certain time is set, then a get () method is called to transfer a url to load a webpage, after a JS variable is obtained and stored by using a Selenium, a url link is sent to a server to serve as a signal for starting copying an md5 file, the current time is used as a url parameter to be transferred, and the server obtains the variable of the parameter to serve as the name of a folder.
And 4, step 4: and analyzing and comparing the acquired dynamic behaviors of the user.
The batch file obtained by mitmprroxy was analyzed. If the files accessed by the user are different, then md5 generated is not the same. Therefore, the dynamic behavior difference of the user accessing the same website can be distinguished by comparing the differences of md5 codes. For multiple accesses of a user, the time of the current access is taken for distinguishing. We compare the md5 code with the information obtained from the first access and the subsequent files, and record the file that was previously but not subsequently as a value, using the domain name as the key. Two identical files are md5 identical, and different files are md5 subsets different from the dynamic behavior of the last user.
The invention is not limited to the above examples, and all technical solutions formed by equivalents or equivalent substitutes are within the scope of the invention as claimed.
Claims (6)
1. A method for automatically acquiring website access information based on program dynamic analysis is characterized in that information of a user accessing a website at different time periods is automatically acquired based on program dynamic analysis, namely: firstly, a JavaScript source code of the whole webpage is inserted by using an insertion technology to obtain a dynamic behavior of a user in the webpage; then collecting the dynamic behavior and storing the dynamic behavior in a variable, and caching the web pages accessed by the user according to the domain name by using an agent; secondly, obtaining JavaScript variables in the instrumented webpage by using the Selenium, and setting the webpage to automatically access the same webpage for multiple times at regular time; finally, comparing the collected files to find the dynamic behavior difference of the user;
caching the webpage accessed by the user by using an agent mitmprroxy to call a Python script, respectively storing the files according to url domain names of the requested Web contents, and recording md5 codes of each file;
analysis was performed on batches of files obtained by mitmprroxy: the dynamic behavior difference of the user accessing the same website is distinguished by comparing the differences of md5 codes; comparing the information obtained by the first access with the subsequent files by using md5 codes respectively, and recording the files which exist at the previous time but do not exist at the next time as values by using the domain name as a key according to the comparison result; two identical files are md5 identical, and different files are different md5 subsets of the dynamic behavior of the most recent user.
2. The method for automatically acquiring website access information based on program dynamic analysis as claimed in claim 1, comprising the steps of:
1) using a dynamic instrumentation tool Jalangi of JavaScript, using a Selenium to imitate a user to visit a webpage, instrumenting a JavaScript code executed in the webpage visited by the user, adding a JS variable on an object needing information, and adding and transmitting the JS variable in the webpage execution process so as to record the dynamic behavior of the user;
2) setting a proxy server by using a Firefox browser, so that a webpage executed by a user passes through the proxy server, and calling Jalangi by the proxy server for pile insertion; acquiring information contained in the variable by utilizing an executeScript method of the Selenium;
3) caching the webpage accessed by the user by using an agent mitmprroxy to call a Python script, respectively storing the files according to url domain names of the requested Web contents, and recording md5 codes of each file;
4) simulating a user to visit the same website for many times at regular time and recording batch information of the website, namely simulating the user to visit the website by using the Selenium and repeatedly executing the action for many times at regular time by using a timer;
5) the batch files obtained through the mitmprroxy are analyzed, because the files accessed by users are different, the generated md5 is different, and the dynamic behavior difference of the users accessing the same website is distinguished by comparing the differences of md5 codes.
3. The method for automatically acquiring website access information based on program dynamic analysis according to claim 2, wherein in step 1), the JavaScript code executed in the webpage accessed by the user is instrumented, that is:
using a dynamic instrumentation tool Jalangi of JavaScript, using a tool Selenium to simulate a user to access a webpage, and setting a proxy server before starting a Firefox browser by the Selenium, wherein the setting comprises setting an IP address of the proxy server and a port address of the proxy server; and then starting a browser, using a get method of the browser to transmit a website to be visited by a user, loading a page, performing instrumentation on a JavaScript code executed by the user when the page is loaded to obtain the dynamic characteristics of the webpage, adding a JS variable on an object of required information, and adding and transmitting the JS variable in the process of executing the page to record the dynamic behavior of the user.
4. The method for automatically acquiring website access information based on program dynamic analysis as claimed in claim 2, wherein in step 2), a dynamic behavior of one access of a user is collected:
setting a proxy server by using a Firefox browser, so that a webpage executed by a user passes through the proxy server, calling Jalangi by the proxy server for instrumentation, returning a webpage to the user to contain the JS variable added in the step 1), acquiring information contained in the variable by using a Selenium executeScript method, returning the variable of an object type by the method, converting the variable into a type which is convenient to compare, wherein the type comprises the following steps: JSON format;
when Selenium is adopted for automatic variable acquisition, the problem of cross-iframe must be noticed: a website may contain multiple iframes, and the dynamic behavior of a user spans multiple iframes; when acquiring a variable in an iframe, the variable must be acquired by transferring to the iframe by using a method of the browser switch to (). frame (); when transferring from one iframe to another, the switch to (). defaultContent () method must be used to transfer to the default content and then to the iframe to be entered;
in addition, executing a JS script in the Selenium by utilizing exenutscript to acquire required information, and establishing different folders to store the acquired different information;
the method for acquiring the required information by executing one JS script in the Selenium by utilizing exenutscript comprises the following steps:
and acquiring url information of the src attribute in the script tag.
5. The method for automatically acquiring website access information based on program dynamic analysis according to claim 2, wherein in step 3), the user's accessed web pages are cached by proxy, that is:
caching the webpage accessed by the user by using an agent mitmprroxy to call a Python script, respectively storing the files according to url domain names of the requested Web contents, and automatically recording md5 codes of each file; then copying the recorded files to the local; the time node for copying the file is very critical, and a server is arranged and a port is monitored: when monitoring that the port is accessed, copying is started, and after copying is finished, the files cached by the proxy mitmprroxy are deleted.
6. The method for automatically acquiring website access information based on program dynamic analysis according to claim 2, wherein in step 4), the user is simulated to access the same website for a plurality of times at regular time, and batch information of the website is recorded, that is:
the Selenium has the characteristic of automatically opening and closing the Firefox browser, so that a user is simulated to access a website, and the action is repeatedly executed for multiple times at fixed time by using a timer, so that the aim of accessing the same website for multiple times by the same user at fixed time is fulfilled;
because the files cached by the mitmprroxy are continuously recorded, the cached files are deleted after the website is visited and before the website is revisited, so that each record corresponds to a result of visiting once, the files are copied and the cached files are deleted within the specified time of closing the browser after the webpage is visited, and the browser is ensured not to visit the webpage within the specified time and continues to visit the same webpage after the cache is deleted by using a delay function thread ();
after the browser is started, setting to wait for a certain time and then calling a get () method to transfer into a url loading webpage, after a JS variable is obtained and stored by using a Selenium, sending a url link to a server as a signal for starting copying an md5 file, transferring the current time as a url parameter, and transferring the variable obtained by the server as the name of a folder.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710033453.8A CN108306918B (en) | 2017-01-13 | 2017-01-13 | Automatic website access information acquisition method based on program dynamic analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710033453.8A CN108306918B (en) | 2017-01-13 | 2017-01-13 | Automatic website access information acquisition method based on program dynamic analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108306918A CN108306918A (en) | 2018-07-20 |
| CN108306918B true CN108306918B (en) | 2021-08-31 |
Family
ID=62872143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710033453.8A Active CN108306918B (en) | 2017-01-13 | 2017-01-13 | Automatic website access information acquisition method based on program dynamic analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108306918B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109460529A (en) * | 2018-10-15 | 2019-03-12 | 杭州安恒信息技术股份有限公司 | A kind of front end micro services module communication means based on iframe |
| CN109828921A (en) * | 2019-01-18 | 2019-05-31 | 上海极链网络科技有限公司 | HTML5 webpage automated function test method, system and electronic equipment |
| CN112965873B (en) * | 2021-03-04 | 2024-09-27 | 中国邮政储蓄银行股份有限公司 | Page processing method, device, storage medium and processor |
| CN117422427B (en) * | 2023-12-19 | 2024-03-15 | 广东省建设工程质量安全检测总站有限公司 | Online batch analysis method and system for low-strain data |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1987862A (en) * | 2005-12-22 | 2007-06-27 | 国际商业机器公司 | Method for analyzing state transition in web page |
| CN101515300A (en) * | 2009-04-02 | 2009-08-26 | 阿里巴巴集团控股有限公司 | Method and system for grabbing Ajax webpage content |
| CN101996196A (en) * | 2009-08-28 | 2011-03-30 | 中国移动通信集团公司 | Dynamic webpage acquisition method and device |
| CN103297469A (en) * | 2012-02-25 | 2013-09-11 | 阿里巴巴集团控股有限公司 | Method and device of collecting website data |
| CN105468779A (en) * | 2015-12-16 | 2016-04-06 | 中国科学院软件研究所 | Browser compatibility detection oriented client Web application capture and playback system and method |
| CN106022132A (en) * | 2016-05-30 | 2016-10-12 | 南京邮电大学 | Real-time webpage Trojan detection method based on dynamic content analysis |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9158656B1 (en) * | 2014-07-15 | 2015-10-13 | American Express Travel Related Services Company, Inc. | Systems and methods for progressively launching websites |
-
2017
- 2017-01-13 CN CN201710033453.8A patent/CN108306918B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1987862A (en) * | 2005-12-22 | 2007-06-27 | 国际商业机器公司 | Method for analyzing state transition in web page |
| CN101515300A (en) * | 2009-04-02 | 2009-08-26 | 阿里巴巴集团控股有限公司 | Method and system for grabbing Ajax webpage content |
| CN101996196A (en) * | 2009-08-28 | 2011-03-30 | 中国移动通信集团公司 | Dynamic webpage acquisition method and device |
| CN103297469A (en) * | 2012-02-25 | 2013-09-11 | 阿里巴巴集团控股有限公司 | Method and device of collecting website data |
| CN105468779A (en) * | 2015-12-16 | 2016-04-06 | 中国科学院软件研究所 | Browser compatibility detection oriented client Web application capture and playback system and method |
| CN106022132A (en) * | 2016-05-30 | 2016-10-12 | 南京邮电大学 | Real-time webpage Trojan detection method based on dynamic content analysis |
Non-Patent Citations (1)
| Title |
|---|
| 基于插桩行为的动态检测技术研究;李超琪;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160315;第2016年卷(第3期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108306918A (en) | 2018-07-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Demystifying page load performance with {WProf} | |
| CN102272757B (en) | Method for server-side logging of client browser state through markup language | |
| US10567407B2 (en) | Method and system for detecting malicious web addresses | |
| CN108306918B (en) | Automatic website access information acquisition method based on program dynamic analysis | |
| US20100325615A1 (en) | Method and system for capturing web-page information through web-browser plugin | |
| US20170199850A1 (en) | Method and system to decrease page load time by leveraging network latency | |
| CN107918575B (en) | Page state monitoring method and device | |
| CN106776318A (en) | A kind of test script method for recording and system | |
| CN106599270B (en) | Network data capturing method and crawler | |
| KR20160024293A (en) | Method and apparatus for reducing page load time in a communication system | |
| CN112637361B (en) | Page proxy method, device, electronic equipment and storage medium | |
| CN110851681A (en) | Crawler processing method and device, server and computer readable storage medium | |
| Shroff et al. | Critical rendering path optimizations to reduce the web page loading time | |
| Kelly et al. | On the change in archivability of websites over time | |
| CN112612943A (en) | Asynchronous processing framework-based data crawling method with automatic testing function | |
| CN114491560A (en) | Vulnerability detection method and device, storage medium and electronic equipment | |
| CN104836779A (en) | XSS vulnerability detection method, system and Web server | |
| Goel et al. | Jawa: Web Archival in the Era of {JavaScript} | |
| CN111324894A (en) | XSS vulnerability detection method and system based on web application security | |
| US20210334113A1 (en) | Method and device for lazy loading of js scripts | |
| DE112016002416T5 (en) | CODE INTERMEDIATE SYSTEM | |
| CN110691005A (en) | Website monitoring system and method | |
| CN110232019A (en) | Page test method and Related product | |
| CN113127788B (en) | Page processing method, object processing method, device and equipment | |
| KR100446209B1 (en) | System for collecting information based on internet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20180720 Assignee: Jiangsu Yanan Information Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2023980047097 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20231117 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20180720 Assignee: Yancheng Nongfu Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2023980049126 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20231203 Application publication date: 20180720 Assignee: Yanmi Technology (Yancheng) Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2023980049119 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20231203 |
|
| EC01 | Cancellation of recordation of patent licensing contract |
Assignee: Yancheng Nongfu Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2023980049126 Date of cancellation: 20241028 Assignee: Yanmi Technology (Yancheng) Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2023980049119 Date of cancellation: 20241029 Assignee: Jiangsu Yanan Information Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2023980047097 Date of cancellation: 20241028 |
|
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20180720 Assignee: Yancheng Nongfu Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2024980021382 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20241030 Application publication date: 20180720 Assignee: Yancheng Hongrui Huicheng Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2024980020857 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20241028 Application publication date: 20180720 Assignee: Shuzhilian (Yancheng) Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2024980020855 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20241028 Application publication date: 20180720 Assignee: Borui Hengchuang (Yancheng) Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2024980020851 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20241028 |
|
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20180720 Assignee: Jiangsu Yanan Information Technology Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2024980022197 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20241101 Application publication date: 20180720 Assignee: Yanmi Technology (Yancheng) Co.,Ltd. Assignor: NUPT INSTITUTE OF BIG DATA RESEARCH AT YANCHENG Contract record no.: X2024980021700 Denomination of invention: A method for automatically obtaining website access information based on program dynamic analysis Granted publication date: 20210831 License type: Common License Record date: 20241031 |