CN112148507A - Internet financial industry data acquisition system - Google Patents
Internet financial industry data acquisition system Download PDFInfo
- Publication number
- CN112148507A CN112148507A CN202011059045.8A CN202011059045A CN112148507A CN 112148507 A CN112148507 A CN 112148507A CN 202011059045 A CN202011059045 A CN 202011059045A CN 112148507 A CN112148507 A CN 112148507A
- Authority
- CN
- China
- Prior art keywords
- acquisition
- data
- module
- information
- data acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/541—Client-server
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention relates to an internet financial industry data acquisition system.A client terminal comprises a service data acquisition unit and an acquisition system acquisition unit; the service data acquisition unit is used for acquiring service data and sending the service data to the collected data using system, the collection system acquisition unit is used for acquiring the collected data and sending the collected data to the data collection system, and the data collection system is in signal connection with the collected data using system; the acquisition unit of the acquisition system comprises an SDK module, a data acquisition module and an acquisition ID generation module are arranged in the SDK module, and the data acquisition module is used for acquiring acquisition information; the acquisition ID generation module is used for returning the acquisition serial number gatherID after the acquisition of the acquisition information is finished, and the acquisition serial number gatherID is associated with the acquisition information; the data acquisition system associates the service with the acquired data through the acquisition flow, and the service system is not coupled with the data acquisition, so that the maintainability of the system is greatly improved.
Description
Technical Field
The invention relates to the technical field of information, in particular to an internet financial industry data acquisition system.
Background
In the field of internet finance, risk prevention is always an important subject, and in order to achieve risk control, data collection is carried out at a client terminal.
The collected data generally includes terminal device information, terminal software environment, terminal network environment information, and the like. Due to the diversification of the terminals, including an android mobile phone client, an IOS mobile phone client, an H5 page embedded in third-party software and the like, each terminal has limited requirements on information acquisition, and the acquired information is inconsistent.
As shown in fig. 2, in the conventional collection method, a special code is written in a terminal service program for a specific environment, so as to achieve the purpose of collecting data. The conventional data acquisition system implementation has the following problems:
firstly, as the number of terminals is too many, a plurality of terminal data acquisition codes need to be developed for realization, and the workload is large;
secondly, once the data acquisition requirement changes, a plurality of terminal data acquisition codes need to be modified simultaneously, and the maintenance is difficult;
thirdly, the data collected by each terminal is collected and transmitted by the corresponding terminal foreground system, so that the data collection function has invasion to the terminal foreground system, the coupling degree is high, and the expandability is poor.
In order to reduce the change of data acquisition requirements on the business system, meet the requirement of expandability of the business system, reduce the difficulty of system maintenance and solve the problem of data acquisition, the method is imperative.
Disclosure of Invention
The technical problem to be solved by the invention is to provide an internet financial industry data acquisition system, which can solve the problems of poor expandability and high system maintenance difficulty of the data acquisition system in a multi-system environment.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
the utility model provides an internet financial industry data acquisition system which characterized in that: the client terminal comprises a service data acquisition unit and an acquisition system acquisition unit; the system comprises a service data acquisition unit, an acquisition system acquisition unit, a data acquisition system and a data transmission unit, wherein the service data acquisition unit is used for acquiring service data and transmitting the service data to the acquisition data use system;
the acquisition unit of the acquisition system comprises an SDK module, wherein a data acquisition module and an acquisition ID generation module are arranged in the SDK module, and the data acquisition module is used for acquiring acquisition information; the acquisition ID generation module is used for returning an acquisition serial number gatherID after the acquisition of the acquisition information is finished, and the acquisition serial number gatherID is associated with the acquisition information;
the collection serial number gatherID is generated according to a set algorithm rule.
The SDK module includes but is not limited to an android SDK module, an IOS SDK module and an H5 SDK module according to the SDK module.
The SDK module also comprises an information encryption module and an information transmission module; the information transmission module is used for outputting acquisition information and acquisition serial number gatherID; the information encryption module is used for encrypting the information output by the information transmission module.
The data acquisition module comprises a plurality of groups of API modules for acquiring set target information, and the acquisition action of the API modules for acquiring the set target information comprises but is not limited to acquiring terminal equipment information, terminal software environment information and terminal network environment information.
The generation formula of the collection serial number gatherID is as follows:
GatherID=MD5[STR(current_time)+UNIQUE(user_tag)+STR(device_tag)+STR(random_digital)];
wherein, STR (current _ time) is the current time character string; user _ tag is user identification; STR (device _ tag) is equipment identification; STR (random _ digital) is a string of random digits.
The acquisition serial number gatherID is associated with the service data output by the service data acquisition unit. The service data output by the service data acquisition unit is in signal connection with the collected data using system through a foreground portal system.
The steps of the acquisition system for acquiring the acquisition information by the SDK module in the acquisition unit are as follows:
step 1.1, the acquisition ID generation module generates an acquisition serial number gatherID and quickly returns the acquisition serial number gatherID to the calling terminal;
and step 1.2, the data acquisition module acquires the acquired information, and the acquired information is locally encrypted by the information encryption module and then sent to a back-end data acquisition system by the information transmission module.
The collected serial number gatherID and the collected information are sent to the data collection system and then stored by the data collection system, and the collected data use system inquires the collected data from the data collection system through the collected serial number gatherID.
The system comprises a data acquisition system, a data acquisition system and a data acquisition system, wherein the data acquisition system is used for actively pushing data to the data acquisition system, the data acquisition system judges whether message pushing is needed according to data types, and if the message pushing is needed, the message pushing is carried out according to a pushing protocol and an address.
The beneficial effect that this kind of internet financial industry data acquisition system can produce does: firstly, the SDK module is used for carrying out acquisition API maintenance, terminal codes do not need to be modified, and only the acquisition SDK module needs to be upgraded, so that the problem of coupling of acquisition services and functional service codes is solved, and a program has expansion capability;
secondly, the change of the acquired service only causes the change of the SDK, the research and development and test range is reduced, and the research and development and test efficiency is correspondingly improved;
thirdly, the service end associates the service with the collected data by collecting serial number gatherID, and the service system is not coupled with the data collection, so that the maintainability of the system is greatly improved.
Drawings
FIG. 1 is a system interaction diagram of an Internet financial industry data acquisition system according to the present invention.
FIG. 2 is a schematic diagram of system interaction in the prior art.
FIG. 3 is a system flowchart of the data collection system of the Internet financial industry according to the present invention.
Fig. 4 is a schematic structural diagram of an SDK module in the data acquisition system of the internet financial industry according to the present invention.
Fig. 5 is a schematic diagram of a data collection structure in an embodiment of the data collection system for the internet financial industry according to the present invention.
Fig. 6 is a schematic diagram of a format of a query interface for collected data in an embodiment of the data collection system for the internet financial industry according to the present invention.
Detailed Description
The invention is further described below with reference to the drawings and specific preferred embodiments.
As shown in fig. 1, an internet financial industry data acquisition system is characterized in that: the client terminal comprises a service data acquisition unit and an acquisition system acquisition unit; the system comprises a service data acquisition unit, an acquisition system acquisition unit, a data acquisition system and a data transmission unit, wherein the service data acquisition unit is used for acquiring service data and transmitting the service data to the acquisition data use system;
the acquisition unit of the acquisition system comprises an SDK module, wherein a data acquisition module and an acquisition ID generation module are arranged in the SDK module, and the data acquisition module is used for acquiring acquisition information; the acquisition ID generation module is used for returning an acquisition serial number gatherID after the acquisition of the acquisition information is finished, and the acquisition serial number gatherID is associated with the acquisition information;
the collection serial number gatherID is generated according to a set algorithm rule.
In this embodiment, the SDK module includes, but is not limited to, an android SDK module, an IOS SDK module, and an H5 SDK module according to the SDK module. The android SDK module is suitable for APP in an android system environment, the IOS SDK module is suitable for APP in the IOS system environment, and the H5 SDK module is suitable for an H5 end and an embedded third-party application terminal.
In this embodiment, the SDK module further includes an information encryption module and an information transmission module; the information transmission module is used for outputting acquisition information and acquisition serial number gatherID; the information encryption module is used for encrypting the information output by the information transmission module.
In this embodiment, the data acquisition module includes a plurality of groups of API modules for acquiring the set target information, and the acquisition action of the API module for acquiring the set target information includes, but is not limited to, acquiring terminal device information, terminal software environment information, and terminal network environment information.
In this embodiment, in consideration of the influence on the user experience, the performance of the acquisition process must be high, the generation of the acquisition serial number GatherID is put in the SDK, and meanwhile, in order to ensure that the acquisition serial number is as unique as possible, a certain algorithm rule needs to be followed, and the generation formula of the acquisition serial number GatherID is as follows:
GatherID=MD5[STR(current_time)+UNIQUE(user_tag)+STR(device_tag)+STR(random_digital)];
wherein, STR (current _ time) is the current time character string; the format is as follows: yyyyMMddHHmmssSSS, where yyyy represents 4-digit year, MM represents 2-digit month, dd represents 2-digit date of the month, HH represents 2-digit hour in 24-hour format, MM represents minute, ss represents second, SSS represents millisecond, and STR (current _ time) may take the values: 20200717185408788, respectively;
user _ tag is user identification; this field is passed in by the API, typically using a value that is uniquely identified in the system by the client;
STR (device _ tag) is equipment identification; the values that may be taken include: ANDROID terminal, "IOS", IOS terminal, "H5" of H5;
STR (random _ digital) is a string of random digits; random strings of 6-bit length are typically generated randomly.
Splicing character strings of STR (current _ time), UNIQUE (user _ tag), STR (device _ tag) and STR (random _ digital), and calculating a character string with the length of 32 bits by using an MD5 algorithm, wherein the character string is an acquisition serial number. The serial number generation comprises information such as acquisition occurrence time, users, equipment, random numbers and the like, and the serial numbers are different.
In this embodiment, the collected serial number GatherID is associated with the service data output by the service data acquisition unit. The service data output by the service data acquisition unit is in signal connection with the collected data using system through a foreground portal system. When the parameters of the service system are transmitted, the acquisition information can be associated only by attaching the acquisition serial number gatherID returned by the acquisition API, so that no matter how the acquisition is changed subsequently, the service logic does not need to be modified, and the functional service and the acquisition service are thoroughly separated.
In this embodiment, the step of acquiring the acquisition information by the SDK module in the acquisition unit of the acquisition system is as follows:
step 1.1, the acquisition ID generation module generates an acquisition serial number gatherID and quickly returns the acquisition serial number gatherID to the calling terminal;
and step 1.2, the data acquisition module acquires the acquired information, and the acquired information is locally encrypted by the information encryption module and then sent to a back-end data acquisition system by the information transmission module.
In this embodiment, the collected serial number GatherID and the collected information are stored by the data collection system after being sent to the data collection system, the data collection system provides data services to the outside, and the service modes are divided into two types:
first, the data acquisition system provides an inquiry interface, and the acquired data using system inquires the acquired data from the data acquisition system by acquiring the serial number GatherID.
Secondly, the data acquisition system actively pushes the acquired data; the data acquisition system back pipe has the collection propelling movement configuration function, and the propelling movement configuration item includes: the protocol, the acquisition address and the acquisition data type of data pushing are determined, the protocol type comprises Http, AMQP and the like, if the data acquisition system receives the acquisition data, whether the message pushing is needed is determined according to the data type, and if the message pushing is needed, the message pushing is performed according to the pushing protocol and the address.
Furthermore, the acquisition system can receive and store the acquisition data transmitted by the Internet, and the acquisition system is deployed in a cluster mode to prevent single-point faults; the structure of the acquired acquisition data is shown in fig. 5. The data acquisition system can provide an acquired data query interface, which may be in the format shown in fig. 6.
The data acquisition system for the internet financial industry performs business and data acquisition association by acquiring serial number gatherID, and the business system is not coupled with the data acquisition, so that the maintainability of the system is greatly improved. Meanwhile, the acquisition service is not associated with the functional service code, so that the system can be upgraded and maintained aiming at different system environments.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.
Claims (9)
1. The utility model provides an internet financial industry data acquisition system which characterized in that: the client terminal comprises a service data acquisition unit and an acquisition system acquisition unit; the system comprises a service data acquisition unit, an acquisition system acquisition unit, a data acquisition system and a data transmission unit, wherein the service data acquisition unit is used for acquiring service data and transmitting the service data to the acquisition data use system;
the acquisition unit of the acquisition system comprises an SDK module, wherein a data acquisition module and an acquisition ID generation module are arranged in the SDK module, and the data acquisition module is used for acquiring acquisition information; the acquisition ID generation module is used for returning an acquisition serial number gatherID after the acquisition of the acquisition information is finished, and the acquisition serial number gatherID is associated with the acquisition information;
the collection serial number gatherID is generated according to a set algorithm rule.
2. The internet financial industry data acquisition system of claim 1 wherein: the SDK module includes but is not limited to an android SDK module, an IOS SDK module and an H5 SDK module according to the SDK module.
3. The internet financial industry data acquisition system of claim 1 wherein: the SDK module also comprises an information encryption module and an information transmission module; the information transmission module is used for outputting acquisition information and acquisition serial number gatherID; the information encryption module is used for encrypting the information output by the information transmission module.
4. The internet financial industry data acquisition system of claim 3 wherein: the data acquisition module comprises a plurality of groups of API modules for acquiring set target information, and the acquisition action of the API modules for acquiring the set target information comprises but is not limited to acquiring terminal equipment information, terminal software environment information and terminal network environment information.
5. The internet financial industry data acquisition system of claim 4 wherein: the generation formula of the collection serial number gatherID is as follows:
GatherID=MD5[STR(current_time)+UNIQUE(user_tag)+STR(device_tag)+STR(random_digital)];
wherein, STR (current _ time) is the current time character string; user _ tag is user identification; STR (device _ tag) is equipment identification; STR (random _ digital) is a string of random digits.
6. The internet financial industry data acquisition system of claim 4 wherein: the acquisition serial number gatherID is associated with the service data output by the service data acquisition unit.
7. The internet financial industry data acquisition system of claim 4 wherein: the steps of the acquisition system for acquiring the acquisition information by the SDK module in the acquisition unit are as follows:
step 1.1, the acquisition ID generation module generates an acquisition serial number gatherID and quickly returns the acquisition serial number gatherID to the calling terminal;
and step 1.2, the data acquisition module acquires the acquired information, and the acquired information is locally encrypted by the information encryption module and then sent to a back-end data acquisition system by the information transmission module.
8. The internet financial industry data acquisition system as claimed in claim 7, wherein: the collected serial number gatherID and the collected information are sent to the data collection system and then stored by the data collection system, and the collected data use system inquires the collected data from the data collection system through the collected serial number gatherID.
9. The internet financial industry data acquisition system as claimed in claim 7, wherein: the system comprises a data acquisition system, a data acquisition system and a data acquisition system, wherein the data acquisition system is used for actively pushing data to the data acquisition system, the data acquisition system judges whether message pushing is needed according to data types, and if the message pushing is needed, the message pushing is carried out according to a pushing protocol and an address.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011059045.8A CN112148507A (en) | 2020-09-30 | 2020-09-30 | Internet financial industry data acquisition system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011059045.8A CN112148507A (en) | 2020-09-30 | 2020-09-30 | Internet financial industry data acquisition system |
Publications (1)
Publication Number | Publication Date |
---|---|
CN112148507A true CN112148507A (en) | 2020-12-29 |
Family
ID=73894357
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011059045.8A Withdrawn CN112148507A (en) | 2020-09-30 | 2020-09-30 | Internet financial industry data acquisition system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112148507A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114969059A (en) * | 2022-07-28 | 2022-08-30 | 建信金融科技有限责任公司 | Method and device for generating order information, electronic equipment and storage medium |
-
2020
- 2020-09-30 CN CN202011059045.8A patent/CN112148507A/en not_active Withdrawn
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114969059A (en) * | 2022-07-28 | 2022-08-30 | 建信金融科技有限责任公司 | Method and device for generating order information, electronic equipment and storage medium |
CN114969059B (en) * | 2022-07-28 | 2022-10-25 | 建信金融科技有限责任公司 | Method and device for generating order information, electronic equipment and storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111666490B (en) | Information pushing method, device, equipment and storage medium based on kafka | |
CN107592351B (en) | Multi-user publishing and subscribing method and system based on Redis | |
US8825749B2 (en) | Method of tracking offline user interaction in a rendered document on a mobile device | |
CN107122379A (en) | Collect the Transaction Information associated with the data file being locally stored | |
US20210141922A1 (en) | Privacy data reporting method and apparatus, and storage medium | |
US10635957B2 (en) | System and method for collecting and analyzing multi-fields two-dimensional code | |
CN101695157A (en) | Self-adapting method of display of mobile terminal screen in network application and system thereof | |
CN113553310B (en) | Data acquisition method and device, storage medium and electronic equipment | |
CN107145556B (en) | Universal distributed acquisition system | |
CN102882974A (en) | Method for saving website access resource by website identification version number | |
CN110109660A (en) | A kind of monitoring short message touching reaches the system and method for effect | |
CN101494832A (en) | Point-to-point identification method and system for embedded chaining short message | |
CN107438084A (en) | Multi-client data synchronization method and apparatus | |
CN105468472A (en) | iOS operation system based data backup and recovery method and apparatus | |
CN112148507A (en) | Internet financial industry data acquisition system | |
CN112035205A (en) | Data processing method, device, equipment and storage medium | |
CN114741400A (en) | Data synchronization and analysis method, device and terminal equipment | |
CN108011934B (en) | Method and device for processing resource data | |
CN111427710B (en) | Communication method, device, equipment and storage medium of components in application program | |
CN100452704C (en) | Method and method for issuing blog articles | |
CN101953184A (en) | Data communication of fixed size data packages | |
CN103532831A (en) | Method of tracking electronic message, mobile equipment and server | |
CN108234584A (en) | The method and apparatus that structure moving advertising is subscribed to based on change | |
CN105099764A (en) | Log processing method and log processing device | |
CN115756538A (en) | Method for software online upgrade |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201229 |
|
WW01 | Invention patent application withdrawn after publication |