GB2390916A - An interface between windows-based applications - Google Patents

Abstract

A computer program providing an interface between first and second applications on a windows-based operating system, wherein the first application communicates with a user via a user interface, the program comprising a monitoring software module for monitoring messages generated by the operating system for messages associated with a window control belonging to the first application and, in response to such a message being detected, using the operating system to obtain from the user interface the value of the control for use by the second application. Also disclosed are a monitoring software module that monitors a windows-based operating system for messages fulfilling certain criteria, stores associated values in a storage means, and updates these values on receipt of messages indicating that these values have changed; and training software which is user operable to select a window control associated with an application and to identify the selected window control.

Description

Monitoring.Software and Method I.'ield of tle invclltion I'le inventing' r

elates; to tle fieltl of monitoring input by a user- and output from a cnnluter. In particular it relates to software and a netlod tor extracting or acting on inf'rnation input by a u.scr, or displayed to a user, with divcrsc applications.

Backgrou,t,_o the invention common computing problem is cnahling systems using diverse.sollware and iardware inpicnenttions to work tocther. For cxanple, at present time, different elencnts of tlc llritisl National Health Service use a variety of dil'l'crcnt software packages, and dit'fercnt hardware installations to store inl'ormation about patients. Individual general practitioners, consultants and ttepartncuts have their Own legacy systems. As with many computer systems, it is desirable to find a way to enable these systems to be used together enahliup,, for example, the provision of uniform access to patient records.

In the s.'t'twarc industry, there arc a number of nethods of dealing faith diverse sot'twarc and hardware which are in common use. I;,r instance, most modern computer progran.s are written with an application programming interface (Awl), that enables other applications to retrieve information from then and/or write information to them.

Unl'ortunately, this facility is not present in many legacy systems.

Where an application does not have an appropriate Al'l, a computer programmer will usually consider whether they could directly access a database used by that application.

Ellis is typically carried using a languapc such as titructurcl Query Language (SQI.).

Ilocvcr, it is not always pos.sihic to access an underlying database. Ior example, the

underlying databasc ntay bc rcntotcly located, encrypted, or not contain all the information titat is necticd.

A further known technology used wlicn there is no Al'l, and direct access t' an underlying database is not practical, is sir called screen-scraping, techn:,kgy. Scteen scrapers are used in networks and undulate clients and scud instructions to a server, ntcnitoring the data that is retunted tat thent, extracting relevant data l'rr.'rtt, for exanlile, ASC'II text intended tier screen presentation.

Il-'wevcr, screen-scraping technology is also of only limited applicability. Screen scrapitty

i s typically used in a latch node and is n't,encrally suitable liar real tinte use; furthermore, one can only retrieve infonnati'tt in tle fornt in which the server transmits it.

Yet further, screen-scrapiTl, is difficult to huplenent with applications running in a modern windrwig operating system, such as Microsoh Windows00 N1', XWindows, MacOS09 or PalmO5;04.

()ste ntethcd of retrieving information throats a computer system is to use a so-called kcylogger program. Keykgger programs store all information typed in to a computer by a user. It is possible to use suClt prograTtts to establish information which has been input by a user into as application. lJsf'ortunately, such programs are impractical to use in a windowing operating system where the user (or an automatic action) may switch input frosts one window to anot}tcr. 'I'he resulting stream of keyprcsses is us' littic practical value. Summary of the Ilivention

According to the first aspect ot' the prcsestt invention, there is prr'vidcd a contputcr program for prrvidirtg an intcrf'acc between a first application, w}tich comsTtunicates with a user via a user interface. and a second applicatic'Tt on a windowing computer staving an operating system, the prograsn corttprising a monitoring software module for monitoring, sessapcs gcscratcd by t}e opcratirtg system for messages associated with a window

control for said first application and, in rcspnse to sully a messac being letcctcci, using tle 'perating syslen t, oltain fr'n, the user interf;acc the value of ttc control for use fly the second application.

Thus the software module can obtain the risers input and coumands in respect of the first application anti relays tiles for the second application to use. Il'wcver only inputs or comnancis as!j<ciated with window cc:'ntr,ls are obtained. I;or exurnpic. the software module couitl obtains text input into a inl'guc h>x only when the conrputcr generates a message indicating their tire user is no longer ipuLtirrg or editing text iT1 that inbox. For cxanple, the user nay have pressed the tah-key lo move the text input/eclit cursor to a different Acid or Ray have clicked Ott tire "()K" button displayed -,n a dialogue t,.x.

I'hus, by selecting which types of messapc are to trigger interrogation of a user interface, the invention can he configured to avoid unnecessary processing of user inputs.

According lo a second aspect ol the present invention there is provicicd a ccnputer program comprising a rnonit->rhg software module operable to monitor ncssapcs generated by a win.l,wing computer operating system for messages fulfilling one or nature prcletcrmined criteria, the criteria including al least that the message is associated with a particular window control, the program preferably also comprising queryalle storage means for storing a value associated well to window control, ante updating means associated with the monitoring software nodule for updating the value stored try the qucryabic storage means upon receipt ol a rmcssage which is associated with a change in the value associated with the window control.

Advantageously, this Tneans that not only can user input be monitored, but that it can he given a context. Messages preferatly include messages relating to user input events such as key presses ancl Tn,use operations. An application can ther-ef're, by yucryiTg the queryabic storage means, find out what information has been input sciccted or retrieved by a user. In contrast to a l;cylogger, the stored value is associated with a known window control. I'rctcrahly, the queryabic storac Scans can persist, even OTICC the window

control has been deleted. This enables an application to access information recently input by a user even if that information is no longer stored anywhere else on that computer.

Window controls are controls which relate to elements of a user interface in a windowing environment and have properties. Example window controls include text boxes, scroll bars, list boxes, progress bar controls etc. which are found in various windowing operating systems. Example sets of window controls include the Windows 20002 common controls and Windows. user controls available from Microsoft (Redmond, WA). Window controls are often referred to in the art as widgets".

The value associated with a window control depends on the type of window control and may be a property of the window control. For example, it may be a text string equal to the current value of the text property of a text box control. Furthermore, the value associated with the window control may be a value which is not a property of the window control itself but requires deduction. For example, a queryable storage means might record the time at which a window control was created. Window controls do not typically include as a property the time at which they were created but, by storing this information in a queryable storage means, an application such as a logger of time spent working on an application may be readily built. Other examples values associated with window controls include: the application which created the control and the value of a check box (established by monitoring check and uncheck messages) It is to be understood that in this context, the term 'value' is a qualitative or quantitative reference to a property. Thus a character string will have a value which is unique to that string and from which the string can therefore be deduced.

A queryable storage means may be associated with a plurality of window controls, in which case it may be adapted to store a value associated with the last of the plurality of window controls to change. The plurality of window controls need not be in existence at the same time. For example, the plurality of window controls may be "Save" dialogue boxes generated in a plurality of different circumstances by one or more applications.

d In the preferred embodiment, the monitoring software module is also adapted to generate one or more events upon detecting a message fulfilling one or more predetermined criteria associated with that message, the criteria including at least that the message is associated with one or more particular window controls. Preferably, these one or more events are generated in a monitor application.

Thus, the monitor application receives events enabling it to respond to specific user events in specific contexts. The monitor application may respond by querying a storage means.

For example, an event may be generated every time a windows message BUTTONUP is generated on a control of class CBTN (button), with the caption: OK, having been created by a specified application, whereupon the monitor application can carry out a intervention, for example displaying a dialogue box stating: "are you sure?" Preferably, the monitoring software module is embedded within an Active control embedded within the monitor application, thereby enabling an event driven monitor application which is COM+ aware, such as one written in the Visual Basic. development system, Delphi., Microsoft. Access database environment or Microsoft. Word for Windows@, to receive events generated by monitoring software module, and also to query, through an ActiveX. interface, the queryable storage means.

An event may also be generated by the monitor application itself. For example, the monitor application may generate an event every time it is selected from a toolbar. An example application of this would be a time logging program, which, when selected from a tool bar, queries a queryable storage means which has monitored the time that each application on the system has been opened and then queries a second queryable storage means which has recorded the most recently opened application, and can therefore deduce the amount of time the user has spent on the application which they previously had activated.

Preferably, the queryable storage means are also writable to by either or both applications and controls. Preferably also, the updating means includes computer software to set a property of a window control associated with a specific queryable storage means.

For example, in response to an event that the user has brought up a specified dialogue box, the monitor application may retrieve some information, or make a calculation, and then write to the observer, the value of text or numeric data which should be input into a particular text box control. The updating means will then amend the value of the window control to correspond to this new value. This enables a monitor application to be written which can retrieve information or make calculations and automatically set the input which should be made by the user.

In a more detailed example, the user of a hospital administration computer program brings up a dialogue box asking for the user to input the name of a patient who is checking in to a hospital, the consultant they are to see, the type of operation for which they are being admitted and the length of time they are likely to stay. The monitoring software module determines that the user has input each of the above data items except the length of time they are likely to stay and then generates an event in a monitor application. The monitor application then calculates the expected length of time that the patient will stay and writes that to the queryable storage means, whereupon the updating means writes the length of time to the control which specifies the dialogue box. Thus it is possible to automate one aspect of data entry without having to customise the hospital administration computer program. The monitoring software module may store data retrieved from queryable storage means.

For example, it may monitor a form completed by a user, and store that data in a remote database. Preferably, the queryable storage means also stores properties of window controls which were generated by an application. Therefore, it is possible to store both input and output of third party applications, providing another method to record data from third party or legacy systems.

Preferably, the monitoring software module monitors a messaging queue. Preferably, the operating system is Microsoft09 Windows0 Preferably, the monitoring software module uses a hook to retrieve messages from the windows messaging queue. Preferably also the monitoring software module then returns the messages to the windows messaging queue.

Preferably, the COT (computer based training) hook in Microsoft Windows49 is used.

Preferably also, the monitoring software module is operable to monitor messages on the computer for messages relating to the creation of new windows controls. This enables the monitoring software module to learn the identifier of a windows control. Preferably, the one or more predetermined criteria include data which can be used to identify the window control when it is created, and, by monitoring the creation of new windows controls, the monitoring software module can determine the identifier of a window control for use with those predetermined criteria. An example criterion is the text displayed by a window control. Preferably, however, the criteria include the window control's parent window control and, optionally, its parent window control and so forth, all the way back to the desktop. For example, in MicrosoftX Windows., each window control is allocated an identifier property, hWnd, when it is created. hWnd is different each time. Thus, a window control cannot be specified in advance by its hWnd property. However, if a new window control having a specific parent window control is created, the hWnd property of the new window can be stored and the monitoring software module can thereafter identify messages relevant to the window control with that hWnd property. The predetermined criteria may include information about other properties of the window control as its parent window control may not be enough to identify it uniquely.

Importantly, a specific event generated by the monitoring software module may be associated with more than one set of predetermined criteria. For example, a single type of event (intended to indicate the user has started work on a document) may be generated by the monitoring software module in response to the user selecting NEW from the FILE menu or OPEN from the FILE menu in MicrosoR. Word for Windows@.

Preferably, the software module is operable to intercept all messages placed to be acted upon by said first application, those messages not associated with window controls relevant to the second application being returned to the queue for onward processing, whilst messages which are relevant to the data to be captured for the second application cause the module to capture said data before being returned to the queue.

Preferably, the monitoring software module resides in the operating system kernel, where it can operate quickly with minimal effect on the apparent speed of the computer on which it is executed.

According to a further aspect of the present invention, there is provided computer software comprising training software means, which, when executed on a computer that is further executing a first software application which has window controls associated therewith, is user activatable to select a window control associated with the first software application, and to establish the identity of the selected window control.

Preferably, the computer program further comprises user interface means through which a user can associate selected window control with one or more actions. Preferably, the user interface means allows the user to relate a window control to one or more events to be generated in response to specific events related to the said window control.

Thus, the training software means can be used to identify the window controls that are generated during the execution of the first application, and to generate events in the computer programs of the first or second aspect, in response to specific events generated in relation to the identified window control, when the first application is executed.

Preferably, a window control is identified by specifying a hierarchy of window parameters that define a hierarchy of window controls. A window control within a hierarchy of window controls is associated with a parent or owner window control (or the desktop).

Windows within the hierarchy are identified by their parameters, so that the computer

program according to the first or second aspect of the present invention can identify when the said window control, or a window control equivalent thereto is created during execution of the first application, even though a particular window control will generally have a different window handle every time it is created.

Detailed description of an example embodiment

An example embodiment of the present invention will now be illustrated with reference to the following Figures in which: Figure 1 is a schematic diagram of a workstation; and Figure 2 is a schematic diagram of software running on a computer.

Figure I is a schematic diagram of a workstation I comprising a computer 2 and a screen 4 showing the desktop 6 of a Microsoft Windows. operating system 7 installed on the computer 2. MicrosoftX Windows0 displays are made up of various window controls, known in the art as "widgets", for example a window 8 and various user interface elements 10, such as text boxes, scroll bars, menus, buttons and the like. The user may interact with the computer 2 using user input peripherals including a keyboard 12 and mouse 14.

Figure 2 is a schematic diagram of software running on the computer 2. A monitor application 20 and a third party application 22 are both able to interact with software and hardware devices through the operating system 7.

The operating system 7 maintains information about window controls 26. The operating maintains a message queue 28 which stores messages 30 relevant to window controls 26.

Windows controls 26 capture interface actions and post corresponding messages to the message queue 28 until they reach the head of the queue whereupon they are processed by the third party application 22.

A monitoring software module 100 is installed in the operating kernel. The monitoring software module 100 is written in a low-level language such as C++ and makes data available to an Actively control 24 which is embedded within the monitor application 20.

The system is adapted to enable the monitoring software module 100 to monitor messages 30. This is achieved by use of Windows hooks? which are a mechanism by which a function can intercept events before they reach the application 22. The monitoring software module 100 is attached to the COT (Computer Based Training) Hook which was chosen as it provides a broad range of messages. Thereafter, the monitoring software module 100 is called in response to all CBT hook messages. The response of the monitoring software module 100 is discussed below.

The module 100 stores a list of observers and triggers against which each intercepted message in turn is tested to determine whether the value of the message is to be stored in queryable storage means (also part of the module 100) or whether an intervention or data capture operation is to be performed before the message is returned to the queue 28, and hence before the application 22 can act on the message.

Each observer stored in the module 100 contains class defining data that includes the "widget route" from the desktop to the windows widget being monitored. The widget route is a list of a set of window parameters that uniquely and recursively identifies a window and its parent in a route that leads from the desktop to each child window or widget until the route reaches the destination window or widget.

For example, the following table shows the data that is always available when a window is created in Microsoft. Windows.. Any or all of this data, together with the data contained in the structures may be used to identify Windows on a Widget Route.

LPCTSTR IpClassNamc pointer to registered class name LPCTSTR IpWindowNarne pointer to window name _. DWORD dwStyle window style int x horizontal position of window int y vertical position of window int nWidth window width int nHeight window height HWND hWndParent handle to parent or owner window HMENU hMenu handle so menu or child-window identifier Handle hlostance handle to application instance LPVOID lpParam pointer to window-creation data To optimise recognition of the creation of windows that are on a widget route, the module 100 maintains a record of the windows handles (hWnd) of the relevant windows as they created along a path. A windows handle is a number created by the operating system at the time of creation of a window that uniquely identifies that particular window.

The observer also includes the windows handle for a given widget, a list of messages that trigger events, the events they trigger and a list of dead events and the name of the observer. For ease of reference, the last window in a windows route, i.e. the window for which the observer is created, will be referred to as a destination window. The information enabling the observers to be set up in the module 100 are stored on a disk file.

When the module 100 is activated, it reads the disk file into its memory and checks the " widget route" of every new window created. If the window being created is a destination

window then a corresponding observer is activated and the windows handle (created at window creation time) captured and applied to the observer. When the window is subsequently destroyed, the observer is destroyed too.

Multiple incidences of windows created (e.g. two copies of notepad running simultaneously) cause the software module to initiate multiple incidences of the observer.

The observers are used as "data sources" and each watches the message queue 28 for messages that affect the current display value of the underlying widget (e.g. the text in a text box).

The triggers stored in the module are set up in a similar fashion, and cause the module to perform a pre-set function when specific messages are posted from associated widgets to the queue 28. The module checks all the messages that originate from the widgets which might provide a trigger message and, on detecting such a message, performs the tasks trained by the user as explained below. Examples of trigger messages are the user clicking the "OK" button on a window or closing the window. The events are implemented through the ActiveX control 24. The ActiveX control has properties that also allow the monitor application 20 to query or set the value for each observer.

In use, messages read from the queue 28 and are checked against the observers/triggers 32 to determine what actions if any needs to be taken.

Messages can result in the creation or destruction of observers, the updating of observer value information, an event or multiple events being fired (i.e. initiated) and, optionally, the return of a message to the queue. The return of a message is optional because the firing of an event may cause the message to be replaced by an alternative message in the other queue. Thus, for example, a user inputting a "minimised window" command into the application 22 could find that the application 20 overrides this command and, for example, provides the user with another prompt or a message.

All messages in the queue 28 are checked against the following criteria.

( 13 1) Is a message a "window request"? If it is, is the parent (identified by each hWnd) part of a widget route. If so, is the new window still on the widget route? If the new window is the end of the path a module creates an observer and labels it with the hWnd of the created window. If it is not the end of a path, then the module 100 makes a note of the hWnd as a marker on the widget route to simplify the above mentioned test on the parent.

2) Does the hWnd match one of the existing observers 32? If not, the module returns the message to the queue 28.

3) Is the message a "destroy window" request? Is so, the module 100 destroys the associated observer. Otherwise the module ascertains whether the hWnd marker is on a widget route. If it is, the module replaces the marker with its parent.

4) The module 100 checks whether the message is a "data modification" message. If it is, the module then obtains a new "value" of the window and updates the observer 32.

5) The module also checks whether the message is in the list of messages that have already been defined (in training the module) as an event for this observer. If it is, then the event is initiated.

6) The module also checks whether the message is in the list of messages that have already been defined as a "dead end message" from any given observer. If it is not, then the message is returned to the queue.

Reference 2 denotes the "updating means" which is used to update a stored value relating to an observer when the value is changed by the receipt of an associated message in the queue 28.

The program may also include a training application which is used to teach the module 100 which controls needs to be monitored and what events (if any) need to be performed in response to a given input via the application 22. When setting up the module in this way, the user, first starts the training application and then minimise it. The user then starts the application that the data agent needs to learn about and navigates through the application to a screen containing widgets of interest. When the computer screen is showing those widgets, the user activates the training application which changes the cursor shown on the computer screen to a cross hare. The user then clicks the cross-hair cursor on each widget of interest (the module then highlighting the widget to show which widget is working with) and the user completes a dialogue box describing what the user wants to do with that widget. The dialogue box can produce a list of possible events to be associated with a given input.

One of the new messages which can be inserted into the message queue by module 100 is a command for the operating system to read the text typed into a input window by the user and pass that data to the monitor application 20 (in a format suitable for the monitor applications).

Claims (1)

1. A computer program for providing an interface between a first application, which communicates with a user via a user interface, and a second application on a windowing computer having an operating system, the program comprising a monitoring software module for monitoring messages generated by the operating system for messages associated with a window control for said first application and, in response to such a message being detected, using the operating system to obtain from the user interface the value of the control for use by the second application.

2. A computer program comprising a monitoring software module operable to monitor messages generated by a windowing computer operating system for messages fulfilling one or more pre-determined criteria, the criteria including at least that the message is associated with a particular window control, the program further comprising queryable storage means for storing a value associated with the window control, and updating means associated with the monitoring software module for updating the value stored by the queryable storage means upon receipt of a message which is associated with a change in the value associated with the window control.

A computer program according to Claim 2, wherein the queryable storage means can

persist once the window control whose associated value it stores, has been deleted.

4. A computer program according to Claim 2 or Claim 3, wherein the value associated with the window control is not a property of the window control, but is deduced by the computer program in relation to the window control.

5. A computer program according to any one of Claims 2 to 4, wherein the queryable storage means is associatable with a plurality of window controls.

( 16 6. A computer program according to Claim 5, wherein the queryable storage means is adapted to store a value associated with the window control of the plurality of window controls which most recently changed 7. A computer program according to any one of Claims 2 to 6, wherein the queryable i storage means are writable to by either or both applications and controls.

8. A computer program according to any one of Claims 2 to 7, wherein the updating means includes computer software to set a property of a window control associated with a specific queryable storage means.

9. A computer program according to any one of Claims 2 to 8, wherein the monitoring software module stores data retrieved from queryable storage means.

10. A computer program according to any one of Claims 2 to 9, wherein the monitoring software module is operable to monitor messages on the computer for messages relating to the creation of new windows controls.

11. A computer program according to any one preceding Claim, wherein the monitoring software module is adapted to generate one or more events upon detecting a message fulfilling one or more predetermined criteria associated with that message, the criteria including at least that the message is associated with one or more particular window controls. 12. A computer program according to Claim 11$ wherein the one or more events generated upon detecting a message for filling one or more pre-determined criteria are generated in a monitor application.

13. A computer program according to Claim 12, when dependent on Claim 2, wherein the I monitor application is operable to respond to one or more events by querying a storage means.

i 14.A computer program according to Claim 12 or Claim 13, wherein an event is generated by the monitor application.

15. A computer program according to any one of Claims 11 to 14 wherein the one or more pre-determined criteria include data which can be used to identify the window control that is created, and, the monitoring software module monitors the creation of new window controls and thereby determines the identifier of a window control for use with those pre- determined criteria.

16. A computer program according to any one of Claims 1 I to 15, wherein a particular event generated by the monitoring software module may be associated with more than one set of pre-deterrnined criteria.

17. A computer program according to any one preceding Claim, wherein the monitoring software module monitors messages in a queue.

18. A computer program according to any one preceding Claim, wherein the software module is operable to intercept all messages placed in a queue to be acted upon by said first application, those messages not associated with windows controls relevant to the second application being returned to the queue for normal processing, whilst messages which are relevant to the data to be captured for the second application cause the module to capture said data before being returned to the queue.

19. A computer program according to any one preceding Claim wherein the monitoring software module resides in the operating system kernel of the computer.

20. Computer software comprising training software means, which, when executed on a computer that is further executing a first software application which has window controls associated therewith, is user activatable to select a window control associated I with the first software application, and to establish the identity of the selected window control.

21. The computer program of Claim 20 further comprising user interface means through which a user can associate a selected window control with one or more actions.

22. A computer program according to Claim 21 wherein the user interface means allows the user to relate a window control to one or more events to be generated in response to specific events related to the said window control.

23. A computer program according to any one of Claims 20 to 22, wherein a window control is identified by a hierarchy of window parameters defining a hierarchy of window controls.