CN113486106B

CN113486106B - Python method for acquiring SIS or supervisory system data and analyzing big data

Info

Publication number: CN113486106B
Application number: CN202110875800.8A
Authority: CN
Inventors: 赵鹏东; 杜保华; 曹旭
Original assignee: Xian TPRI Power Station Information Technology Co Ltd
Current assignee: Xian TPRI Power Station Information Technology Co Ltd
Priority date: 2021-07-30
Filing date: 2021-07-30
Publication date: 2023-03-14
Anticipated expiration: 2041-07-30
Also published as: CN113486106A

Abstract

The invention discloses a method for Python to acquire SIS or supervisory system data and analyze big data, which comprises the following steps: 1) Redefining the data type of the API dynamic library of the RDBPproxy database agent; 2) Various methods for packaging the API are Python formats, and data are obtained; 3) And analyzing the big data by using the acquired data. The invention solves the problem that a set of uniform API function method can simultaneously acquire a plurality of real-time libraries, relational libraries and memory data from an SIS or a supervisory system, simultaneously reduces the work of developing a plurality of sets of different data interfaces, is convenient for the uniform maintenance of program versions, also reduces the links of data transmission and conversion, and enhances the reliability of data. The acquired data can be directly subjected to data interaction with a big data analysis system.

Description

Python method for acquiring SIS or supervisory system data and analyzing big data

Technical Field

The invention belongs to the technical field of data acquisition by Python in an API mode, and particularly relates to a method for acquiring SIS or supervisory system data and analyzing big data by Python.

Background

With the rapid development of big data analysis, artificial intelligence and Internet plus technology in recent years, smart power plants also become a new point of innovation for fusion of big data and industrial production. However, the data of the power plant has strong real-time performance, mass performance, diversity and low value density, and only by speaking the data, the intelligent power generation and future can be achieved.

At present, a thermal power plant level monitoring information system (SIS) is generally built in a domestic thermal power plant, most of the system is developed by Java or C #, and the two languages are not good for big data analysis. The API for providing data access by the SIS system is also encapsulated by Java or C #, if Python is needed to perform related big data analysis, a corresponding data driving interface needs to be developed, and currently, domestic common real-time libraries have more types and different versions, so that great troubles are brought to the development of data interface driving.

In addition, the data format provided by Java or C # is different from the data structure of a big data analysis system, and redundant links are needed to reorganize data, so that the problem of data transmission and conversion is increased.

Disclosure of Invention

The invention aims to provide a method for Python to acquire SIS or supervisory system data and analyze big data. The method provides a unified set of API access method, so that data of real-time databases such as PI, EDNA, rython, openplant and the like can be obtained from the RDBPproxy database, and data and memory data in relational databases such as SQL, oracle and the like can also be obtained, thereby providing a basic data source for big data analysis and data mining by using Python.

The invention is realized by adopting the following technical scheme:

a method for Python to acquire SIS or supervisory system data and analyze big data comprises the following steps:

1) Redefining the data type of an API dynamic library of the RDBProxy database proxy;

2) Various methods for packaging the API are Python formats, and data are obtained;

3) And analyzing the big data by using the acquired data.

A further development of the invention is that step 1) comprises the following steps:

11 Python) import external function library-ctypes;

12 Redefine the data type of the API as that of Python;

13 Redefines the enumerated classes.

A further improvement of the invention is that in step 11) a data type compatible with C is provided and functions in the DLL or shared libraries are allowed to be called, which are used to encapsulate the libraries in pure Python form.

A further refinement of the invention provides that in step 12) the data type comprises tag information, constants, proxy tags, error codes and server connection parameters.

The invention is further improved in that, in step 13), the enumeration class includes data quality, value mode and data structure.

A further development of the invention is that step 2) comprises the following steps:

21 Initializing basic parameters of the API first;

22 When the connection is successfully established, calling DBP _ QueryTag to inquire a measuring point, and calling DBP _ GetTagAttr to acquire label information;

23 After the label information is obtained, calling DBP _ GetSnapshot to obtain a real-time snapshot, calling DBP _ GetHisVal to obtain historical data of a certain label, calling DBP _ GetMultiPointHisVal to obtain multi-label historical section data, calling DBP _ WriteReal to write in real-time basic data type data, and calling DBP _ WriteHis to write in historical basic data type data;

24 Obtaining required data, converting the data into a list type, a ditt type or a tuple type of Python, and organizing the data into a data structure required by a big data analysis system;

25 Returning the data structure converted in the step 24) to the caller of the function;

26 Finally call DBP _ Disconnect to DisConnect and call DBP _ Close to release system resources.

A further improvement of the invention is that in step 21) the basic parameters for initializing the API include loading the DLL, establishing a TCP connection with the RDBProxy database proxy.

A further development of the invention is that step 3) comprises the following steps:

31 A common big data analytics library loaded with Python;

32 Organizing the data obtained in the step 2) according to the data format of a big data analysis system;

33 Call the related function of the big data analysis system, and transmit data to obtain an analysis result;

34 Communicate the results of the analysis to an associated presentation system;

35 Data results obtained by big data analysis are utilized, and potential data value is further mined.

Compared with the prior art, the invention has the following advantages:

1. the data are directly acquired and used for a big data analysis system, so that the reliability and the interactivity of the data are enhanced

Compared with the API for providing data access packaged by Java or C #, the method directly uses the Python packaging API mode to obtain data from the RDBProxy database proxy, reduces links of data conversion and enhances the reliability of the data. The acquired data is directly converted into a data format corresponding to the big data analysis system, and data interaction with the big data analysis system is very convenient.

2. The unified API can perform data interaction with a multi-type database

The invention provides a set of API function method, which can simultaneously carry out data interaction with a plurality of real-time libraries, relational libraries and memory data, reduces the work of developing a plurality of sets of different data interfaces and is convenient for the unified maintenance of program versions. And after the analysis result of the big data analysis system is obtained, the relevant function of the program is directly returned, and the analysis result is written into the relevant database by the program in real time, so that the timeliness of the data is ensured.

Drawings

FIG. 1 is a flow chart of the operation of the present invention.

Fig. 2 is a schematic diagram of a data interaction method.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.

The data source of the invention can adopt a real-time database of a power plant SIS system, a regional production real-time monitoring system or a group production real-time monitoring system. Taking data collected from a power plant SIS system as an example, an embodiment of the method is described as follows:

1) Investigating the type, IP address and network architecture of a real-time database, determining the installation position and network access of a system server and configuring a corresponding data reading interface;

2) Installing and configuring an RDBProxy database proxy server to ensure normal data connection;

3) Installing a Python working environment, wherein a default installation mode is generally adopted;

4) Redefining the data format according to the corresponding relation of the table 1;

5) Repackaging each function according to a language format of Python according to a development manual of the API;

6) Testing the correctness of each function, testing the running correctness of the program integrally, and packaging program codes;

7) And embedding the packaged program code into a big data analysis module, configuring related parameters and ensuring the normal operation of the program.

8) And the big data analysis system performs related analysis according to rules by using the acquired data to obtain an analysis result, and transmits the result to a related system for displaying.

And providing a corresponding data access interface for further data mining.

Table 1 is a table corresponding to the basic data types compatible with ctypes and C.

Type ctypes	Type C	Python data type
			c_bool	_Bool	bool(1)
c_char	char	One character byte string object
			c_wchar	wchar_t	Single character string
c_byte	char	int
			c_ubyte	unsigned char	int
c_short	short	int
			c_ushort	unsigned short	int
c_int	int	int
			c_uint	unsigned int	int
c_long	long	int
			c_ulong	unsigned long	int
c_longlong	\\ int64 or Long	int
			c_ulonglong	unidimensional _int64 or unidimensional long	int
c_size_t	size_t	int
			c_ssize_t	ssize _ t or Py _ ssize _ t	int
c_float	float	float
			c_double	double	float
c_longdouble	long double	float
			c_char_p	char*(NUL terminated)	Byte string objects or None
c_wchar_p	wchar_t*(NUL terminated)	Character strings or None
			c_void_p	void*	int or None

Although the invention has been described in detail hereinabove with respect to a general description and specific embodiments thereof, it will be apparent to those skilled in the art that modifications or improvements may be made thereto based on the invention. Accordingly, such modifications and improvements are intended to be within the scope of the invention as claimed.

Claims

1. A method for Python to acquire SIS or supervisory system data and analyze big data is characterized by comprising the following steps:

1) Redefining the data type of the API dynamic library of the RDBPproxy database agent; the method comprises the following steps:

11 Python) import external function library-ctypes;

12 Redefine the data type of the API as that of Python;

13 Redefines enumerated classes;

2) Various methods for packaging the API are Python formats, and data are obtained; the method comprises the following steps:

21 Initializing basic parameters of the API first;

26 Finally calling DBP _ Disconnect to DisConnect, calling DBP _ Close to release system resources;

3) Analyzing big data by using the acquired data; the method comprises the following steps:

31 A common big data analytics library loaded with Python;

34 Communicate the analysis results to the associated presentation system;

2. A method of Python for data acquisition SIS or supervisory systems and big data analytics as claimed in claim 1, characterized in that in step 11) C-compatible data types are provided and functions in DLLs or shared libraries are allowed to be called, which are used to encapsulate the libraries in pure Python form.

3. The method of claim 1, wherein in step 12), the data types include tag information, constants, proxy tags, error codes, and server connection parameters.

4. The method according to claim 1, wherein in step 13), the enumerated classes include data quality, value patterns, and data structures.

5. The method for Python to obtain SIS or supervisory system data and big data analysis according to claim 1, wherein, in step 21), initializing the basic parameters of API includes loading DLL, establishing TCP connection with RDBProxy database agent.