DE20221624U1 - Virtual microscope system for viewing large sections of a biological preparation or other image, whereby captured mircoscopical images are combined and stored in a server that can be accessed online - Google Patents

Abstract

Virtual microscope system comprises digital image data (31) that has been scanned by an image capture system with predefined magnification; a server system (2) with software (231) for generating a virtual section of the preparation from the digital image data; image analysis software (232); and at least one client system (1) with application software for the representation of data of a selectable virtual section from the case database (222). The selected data is used by the server to generated a virtual section with the highest available, or a selected, magnification over the largest available, or selected part, of a preparation. The invention also relates to a corresponding method for analysis of digital microscope data.

Description

The The invention relates to a virtual microscope system with a Image recording system in a predefinable magnification and in a predefinable Extract scanned or otherwise imported digital image data a preparation, a server system with software for generating a virtual Section of the preparation from those provided by the imaging system or otherwise imported digital image data of the preparation, a software for Image editing of the virtual cut and a storage device for storing the virtual cut in a case database, and at least one client system with user software for Representation of user-selectable data of a selectable one Virtual section of the case database.

For the inspection of objects, such as histological specimens, circuits on processors, or other medical preparations or technical components under high magnification usually light microscopes are used. Light microscopes can be further subdivided according to the type of specimen illumination (reflected light or transmitted light microscopes), the wavelength of the illumination light (simple light or fluorescence microscope), the direction of oscillation of the illumination light (polarization or interference contrast microscope) as well as the phase angle of the detected Light (phase contrast or dark field microscope). All light microscopes are limited in resolution by the wavelength of the light, and therefore have a minimum distinguishable dot pitch of about 0.2 microns (0.2 x 10 ^-6 m). This corresponds to a maximum possible magnification of about 1,000 times, taking into account the minimum distance of about 0.2 millimeters to be recognized by the human eye. Higher resolutions are achieved with electron microscopes. In these, instead of light waves, the specimen is bombarded with the much shorter-wave electrons, and their deflection is detected as they fly through the specimen. As a result, theoretically a resolution corresponding to a dot spacing of about 2 angstroms (2 × 10 ^-10 m) can be achieved. Due to a limited achievable lens quality is actually only a visible magnification of about 300,000 times possible. Electron microscopes are differentiated according to the type of beam path (transmission or scanning electron microscope).

• – Es kann immer nur ein Ausschnitt betrachtet werden.
• – Es stehen nur wenige, bestimmte objektivabhängige Vergrößerungsstufen zur Verfügung.
• – Es kann immer nur ein Präparat zur gleichen Zeit betrachtet werden.
• – Es gibt keine Möglichkeit der Markierung auf lokaler mikroskopischer Ebene.
• – Es gibt keine Möglichkeit einer inhaltsbezogenen Präsentation.
• – Es kann nur einen lokalen Betrachter zur gleichen Zeit geben.
• – Ein Präparat ist immer nur einmal vorhanden und kann nicht vervielfältigt werden.

All of these devices are subject to the following restrictions:

• - Only one section can be viewed at a time.
• - There are only a few, certain lens-dependent magnification levels available.
• - Only one product at a time can be considered.
• - There is no possibility of marking at the local microscopic level.
• - There is no possibility of a content-related presentation.
• - There can only be one local viewer at a time.
• - A preparation is only once available and can not be duplicated.

moreover is the possibility the commenting of a case severely limited. Notes or comments to the picture are usually written recorded or in a dictaphone spoken. The link of clues with specific places on the specimen is without aids (for example, scanning table and software) are not clearly possible. The places of interest the user must remember in general. The appraisal of an object at a distance from the microscope is not without special aids (such as a TV camera) possible, as well sending a case in digital form.

to Improvement of functionality Such microscopes have been various additional components in the past developed, each with its own technical limitations of a microscope. So allows a so-called "multiple viewing device" several persons the simultaneous viewing of a preparation by a microscope to be attached beam splitting optics. The narrow field of view can be replaced by a special Eyepiece optics (large field optics) be extended. Video cameras with monitor allow several people the simultaneous consideration of the preparation, albeit in strong limited Quality. The generation of digital images of a single section and their duplication is also realized by means of video camera and computer connection Service. And finally telepathology systems have been increasingly in existence for a few years, with which the live broadcast of the microscope image to a remote user as well as the remote control of the microscope over allows this remote user has been.

Recently, digital methods have also been developed to support microscopes. Thus, for example, the generation of digitized microscope images, so-called "virtual sections" (VS), via computer-controlled microscopes is known from WO 98/39728 in which an overview image composed of a multiplicity of partial images is recorded and the partial images are combined with a common, The user can subsequently select areas of interest in which the data is measured in a particular higher resolution or magnification level and also be stored coordinate-bound. The data structure described in this document does not provide for the provision of an overall image of the preparation in higher resolution. Likewise, the possibility of creating multilayer images as well as the application of a drawing plane or stepless zooming by the user is not realized.

Out WO 01/54052 is known, the data of a virtual cut on a server and on the client side using a web browser in the form of an overview picture low resolution and retrievable fields of higher resolution to visualize, with a back and forth Switching between the different resolutions is provided. Here too will not be a total picture in highest Resolution provided. Moreover, not the possibility the use of multilayer images and a drawing plane provided. Also, there is no continuous zooming, and no administration the user data and case data.

The WO 99/47964 describes specific techniques for scanning a preparation.

Finally is from WO 01/26541, the virtual sections as a data structure via inter- and Intranet.

Out Telepathology is also the possibility of live discussion a case known by several users via the Internet.

In summary it should be noted that no system is yet available, which (apart from the image acquisition, the scanning) a microscope indeed in it's entirety could replace. Such a system would be desirable, because there are all advantages in a solution would be cheaper and more convenient, and also enable functionalities that would be urgently desired by microscope users. Such functionalities include, for example the application of multilayer images, the use of a marking layer, the automatic management of cases to automate a Processing workflows, the automatic image processing the pictures in support and Accelerating the work of a microscope user as well as the stepless Back in.

Of the Invention is therefore the object of a so-called virtual Microscope available to put, with the exception of the actual image acquisition all Functionalities a conventional one mechanical light or electron microscope completely replaced and some extra, from the mechanical solution previously unknown functionalities offers. Furthermore, the virtual microscope is said to be the limitations of today overcome digital microscopes and in particular a remote transmission enable image data in real time.

The object is achieved by a virtual microscope with the features of claim 1. The virtual microscope according to the invention is characterized in that the digital image data of the specimen imported from the image acquisition system, for example a light or electron microscope, or otherwise imported (for example from a PACS) and the virtual section produced by the latter on the server side, form an image of the specimen in FIG the highest available or desired magnification and the highest available or desired section of the preparation, in particular an overall view of the preparation, all data of each preparation being represented as a virtual section ( 212 ) are stored in a file and that, on the client side, this virtual cut file ( 212 ) of the highest available or desired magnification and the highest available or desired cutout.

Therefore the database according to the invention contains only one digital each Microscope image of a preparation in a single, namely the maximum magnification. Here, high or low magnification means a representation by the User in high or low resolution, as shown digital images on monitors usual. In this way, a user at a client station can always be immediate to access the virtual sections without, for example re-microscopic measurements in the area of interest with the desired Magnification be performed have to.

The basic principle of the invention is thus the scanning of a whole specimen under the highest available or necessary magnification into an overall image, its presentation in an Internet- or Intranet-based or independent surface and the processing of the case using new digital tools in a client server environment. The complete slide is digitized at maximum magnification and stored as an electronic file (virtual cut) on a computer (server). The image may be from a digital camera installed on a microscope, other imaging system, or PACS (Picture Archiving and Communications System). The image data are preferably present in compressed form, since very large amounts of data have to be handled. The image can be loaded into the system via inter- or intranet protocols (such as TCP / IP) or via fixed media (such as a CD-ROM). To examine the virtual sections (VS) now serves the Virtual Microscope System, which exactly on the desired preparation section or the entire preparation on visualized a computer monitor of a client system. Control is via an interactive input medium (eg mouse, joystick, touchpad, ...) by the user.

The Implementation of the appraisal of objects under high magnification in digital form brings significant temporal and functional improvements with himself. Is an object once by means of a suitable image acquisition system digitized (for example, with a conventional microscope with scanning table and digital camera), it can, by means of the system according to or internet any desired User made accessible be, even several at the same time. The digital picture can with a additional Layer (marking level) superimposed be attached to the comments or notes, or interesting places can be marked. In addition allows the digital form stepless zooming and moving as well as diverse Methods of image processing.

To a particularly advantageous embodiment of the invention the client software of the client system has a viewer in which the Overall view of the drug as an overview picture with low resolution and / or at least one image area currently selected by the user as a detail view in higher resolution is pictured. Preferably, in addition to the representation of the selected preparation section constantly the overview picture represented, which the entire preparation and preferably on it marks the area of the selected one Section indicates. additionally can also several sections are considered simultaneously. Both in the overview picture as well as in all detail views, the resolution can be released by the user be set.

It Advantageously, means are also provided within the client system, with exactly one image area of the selected virtual selected by the user Intersection of the case database of the server system currently up to date requested and displayed on the client side in the viewer. Optional can the data too mobile data carriers (for example, CD-ROM) are loaded in the client-server system. The generated by the software for creating virtual slices highly compressed image data, preferably in the so-called Enhanced Compressed Wavelet format (ECW, format of the company Earth Resource Mapping) while the transmission and presentation of the selected Area almost in real time. In each case only the Data of the image area of interest in the compression stage the resolution of interest transmitted from the server system to the client system, client side means a decompression software, preferably in the form of a compression format corresponding plug-ins of the company "Earth Resource Mapping", decompressed and in the viewer in the desired Size shown.

There according to the invention, the overall picture drug in highest resolution in the form of the virtual cut, becomes a stepless digital zoom function realizes a detail view within any place within the overall view of the virtual Cut to the highest Magnification (resp. Resolution) allowed. Therefore For example, the user manually sets one in the overall view or the detailed view provided tracker to the position and Extension of the desired Subset, whereupon the client system makes the necessary Request image data from the server system and as a new detail view represents. It corresponds - how in digital zooming usual - the presentation a section or image in low magnification easy the representation of the image in low resolution. This allows a preparation stepless in any magnification level are displayed. The zooming is done in real time, so that the user can interactively set the optimum magnification level. The tracker can also be used for manual scanning of the virtual cut.

A particularly advantageous embodiment of the invention provides the opportunity the use of multilayer images consisting of several overlapping ones Overall pictures of the same preparation - measured using various dyeings or recording techniques. Multilayer images are known in principle. It is an image data set, within a file contains multiple images of the same object, the geometrically matched to each other. This allows the VM user the indeed simultaneous viewing and thus the direct and fast visual Comparison of two or more images of an object, as with Today's microscopes and also referred to the previous digital solutions on the overall picture of the drug was not possible. The simultaneous visualization of the images is done by division Detail view, quick and easy switching of the view or over a semi-transparent view of several preparations with the simultaneous possibility the setting of transparency. The geometric adaptation of Pictures (matching) take place within an image processing module on the server.

One Another decisive advantage of the invention is that the image or multilayer image is superimposed with a marking plane on which by the User Image-coordinate-related comments, markings and / or the like can be accommodated.

This allows the marking of places or areas of interest, the entry image-coordinate-related texts, symbols, graphics, drawings, Tags, hyperlinks, references to other data (e.g., audio data) and drawing, also freehand, in the picture, as well as the linkability of marked locations and other preparation information as needed. Further, the marking plane may be marks during the Image analysis considered image areas and / or a so-called Include inspection paths that one inspected by the user Way through the drug represents. The marking level can be optionally displayed or invisible provided. It is also possible to use the marking plane permanently together with the virtual cut in the case database to save.

According to one preferred embodiment of the invention comprises the software for Create virtual slices installed on the server system is, an import function that imports the from the recording system provided image data in any image data formats allowed. The image data formats are then compressed by the software Image format (especially in ECW format) converts to a color depth can be set variably. In this form, the image data saved a virtual cut. In addition to the actual image data contain the virtual snippets stored in the case database further information, in particular status data of the case, preparation and recording parameters and / or the like, these textual data preferably structured by SQL (Structured Query Language, a versatile query format for databases) available. The text data can from an operator of the recording system and / or client side of a user with usual Input means are entered.

• – Kontrastverbesserung
• – Normierung der Helligkeitsverteilung
• – Weißabgleich
• – Kantenextraktion
• – Matching auf bekannte Strukturen zur Vorauswahl interessierender Bereiche
• – Messen und Zählen von Strukturen

und/oder weitere Funktionen umfasst. Die Bildbearbeitungsschritte können entweder von einem Anwender aufgerufen werden, oder entsprechend bestimmter Falltypen automatisch gestartet werden. Es kann eine Liste von Bildbearbeitungsschritten für relevante Falltypen angelegt werden, so dass das System entsprechend dem aktuellen Falltyp automatisch festgelegte Schritte startet. Die Bildbearbeitung dient dazu, automatisch Strukturen hervorzuheben, die den Anwender bei der späteren Beurteilung des Bildes unterstützen bzw. ihn gezielt auf relevante Strukturen (z.B. Mitosen in Tumorpräparaten) hinweisen.A further particularly advantageous embodiment of the invention provides the server side, the provision of software or software package for image processing of the virtual sections, which can be started automatically by the server system after generating the virtual cut - according to the case type - or individually by a user, the Image editing one or more of the steps

• - Contrast improvement
• - Normalization of the brightness distribution
• - white balance
• - Edge extraction
• - Matching to known structures for pre-selection of areas of interest
• - Measuring and counting structures

and / or other functions. The image processing steps can either be called by a user or automatically started according to specific case types. A list of image processing steps can be created for relevant case types, so that the system automatically starts predetermined steps according to the current case type. Imaging is used to automatically emphasize structures that assist the user in the subsequent assessment of the image or specifically point out relevant structures (eg, mitoses in tumor preparations).

To A further advantageous embodiment of the invention are means provided for the management of case, image and user data. There exists in addition to the case database, in which all case data, image data and status data a case structured (for example, using SQL) filed are, a user database, in the access data, access rights and assignments to cases are stored. The server system where these databases are stored are taking care of it a secure, redundant storage of the data (for example, by means of a RAID system). Only the careful Design of these two databases makes it possible with the virtual microscope system a conventional one Microscope in its entirety to replace. This includes also a detailed logging of the workflow and the work of the user.

• a. Scannen oder anderweitiger Import digitaler Bilddaten eines Präparates in einer höchsten verfügbaren oder gewünschten Vergrößerung und einem höchsten verfügbaren oder gewünschten Ausschnitt des Präparates durch ein Bildaufnahmesystem,
• b. Erzeugung eines Virtuellen Schnittes in der höchsten verfügbaren oder gewünschten Vergrößerung und dem höchsten verfügbaren oder gewünschten Ausschnitt des Präparates aus den von dem Bildaufnahmesystem bereitgestellten oder anderweitig importierten digitalen Bilddaten des Präparates mittels einer ersten Software eines Serversystems,
• c. Veränderung des Virtuellen Schnittes mittels Bildbearbeitungsfunktionen einer zweiten Software des Serversystems,
• d. Speichern des Virtuellen Schnittes in einer Falldatenbank einer Speichereinrichtung und
• e. Darstellung von durch einen Anwender mindestens eines Clientsystems ausgewählten Daten eines ausgewählten Virtuellen Schnittes der Falldatenbank mittels einer Anwendersoftware.

A method of processing digital microscope data comprising the steps of:

• a. Scanning or otherwise importing digital image data of a preparation in a highest available or desired magnification and a highest available or desired section of the preparation through an image acquisition system,
• b. Generating a virtual cut in the highest available or desired magnification and the highest available or desired portion of the specimen from the digital image data of the specimen provided by the image acquisition system or otherwise imported by means of a first software of a server system,
• c. Modification of the virtual cut by means of image processing functions of a second software of the server system,
• d. Storing the virtual cut in a case database of a storage device and
• e. Representation of selected by a user of at least one client system data of a selected virtual section of the case database by means of a user software.

Further advantageous embodiments of the invention are the subject of the remaining dependent claims.

The invention will be described below with reference to associated drawings explained. Show it:

1 schematically the data flow of the entire system of a virtual microscope;

2 schematically the operation of the server system within the overall system, and

3 schematically the operation of a client system within the overall system.

According to 1 The virtual microscope according to the invention is a client-server-based system. It comprises the two main components, consisting of at least one client system 1 and a server system 2 , although also a client 1 together with the server system 2 can be installed on the same computer.

The server system 2 includes a storage system 21 ( 2 ), on which all virtual sections 212 a case are filed. Next to it is the server 2 also server software 23 , especially software 231 for creating virtual cuts 212 and a software 232 for image processing of virtual sections 212 ready.

As a client system 1 All of them apply via the Internet or Intranet to the server system 2 communicating computer stations by users 13 ( 3 ) of the system. There can be any number of client systems 1 be present at the same time on the server 2 can access. Each client system has communication and data transmission 1 via an interface 15 and the server system 2 via an interface 25 which are preferably TCP / IP interfaces. Client side becomes the request, presentation and assessment of a virtual cut 212 an application software 11 needed on every client system 1 can run. The application software 11 becomes according to the illustrated example of the server system 2 within a web browser 111 provided. Optionally, the application software 11 but also as a separate program interface or within a surface of another software in the client system 1 to be provided.

Nchfolgend is discussed in detail on some individual components of server and client system.

server

The server system 2 consists of the storage system 21 which is in particular a read-only memory, and which is a database module 22 includes, and software 23 and the interface 25 ,

• – Speicherung der von einem Bildaufnahmesystem aufgenommenen digitalen Bilder in Form von Virtuellen Schnitten 212 in einer Falldatenbank 222,
• – Betrieb der Serversoftware 23, mit der die Bilder und Daten betrachtet und verändert werden können,
• – Verwaltung der Bild- und Textdaten in den Datenbanken 22,
• – Zugangssteuerung über eine Benutzer-Zugangs-Datenbank 221, welche die angemeldeten Benutzer 13 und deren Zugriffsrechte verwaltet.

The database engine 22 has the following tasks:

• Storage of the digital images recorded by an image acquisition system in the form of virtual sections 212 in a case database 222 .
• - Operation of the server software 23 with which the images and data can be viewed and changed,
• - Management of image and text data in the databases 22 .
• - Access control via a user access database 221 which the logged in users 13 and manage their access rights.

The image data is stored in a special image data format which enables high compression (preferably in ECW format). The descriptive data and case data ( 212 ) are managed based on SQL. This allows you to find specific cases based on all the parameters you have collected.

• 1. Die Anwendersoftware 11, die an den Webbrowser 111 oder als eigenständiges Programm eines Anwenders 13 übertragen werden kann. Diese Anwendersoftware 11 dient als Anwenderoberfläche. Sie beinhaltet ein erstes Anzeigefenster, in dem ein Übersichtsbild des Virtuellen Schnitts 212 dargestellt wird und über den im Virtuellen Schnitt navigiert werden kann, ein weiteres Anzeigefenster, das einen aktuell ausgewählten Bildausschnitt anzeigt und mehrere Steuerungstools, die zur Navigation im Bild, zur Bearbeitung der Falldaten und zur Datenverwaltung dienen. Diese Anwendersoftware kann clientseitig alternativ auch von einem festen Datenträger (beispielsweise einer CD-ROM) geladen werden.
• 2. Ein Programm 231 zur Erzeugung der Virtuellen Schnitte 212. Das Programm ist nötig, um die digitalen Bilddaten 31 eines Bildaufnahmesystems in einen Virtuellen Schnitt 212 zu formatieren, der dann in der Falldatenbank 222 abgelegt wird. Initial besteht ein Virtueller Schnitt 212 aus den Bilddaten, wobei auch Multilayer-Daten möglich sind, und einigen Daten 32 wie zum Beispiel Fallnummern, Aufnahmedatum und ähnliches, die vom Anwender 13 interaktiv eingegeben bzw. über eine Schnittstelle aus einem anderen Datenbanksystem (z.B. Krankenhausinformationssystem oder PACS) importiert werden. Die Daten eines solchen Virtuellen Schnittes 212 wachsen aber bei der Bearbeitung des Falles an, da nachfolgend diverse Anmerkungen, Marker und ähnliches hinzukommen. Das Programm 231 zur Erzeugung der Virtuellen Schnitte 212 hat also die Aufgabe, die übertragenden Bilddaten in das spezielle Bildformat eines Virtuellen Schnittes 212 umzuformatieren, falls nötig Teilbilder zu einem Gesamtbild zusammenzufügen oder mehrere Bildlayer so zusammen zu matchen, dass sie einen gemeinsamen geometrischen, beispielsweise koordinatenbezogenen Bezug bekommen. Außerdem müssen die vom Anwender 13 angegebenen Daten in das Format des Virtuellen Schnittes 212 eingetragen werden. Der Ablauf der Virtuellen-Schnitt-Erzeugung ist wie folgt:
• – Zunächst werden digitale Bilddaten 31, die von dem Bildaufnahmesystem in raw, tiff, jpg oder einem anderem Format bereitgestellt werden, geladen.
• – Entsprechend dem aktuellen Falltyp (präparatabhängig) werden diverse Bildbearbeitungsschritte in einer festgelegten Sequenz angewandt.
• – Dabei erfolgt eine Farbanpassung auf eine festgelegte Farbtiefe (z.B. 16 Bit = 2 × 5 Bit + 1 × 6 Bit).
• – Nachfolgend wird in ein komprimiertes Bildformat (z.B. ECW) umformatiert.
• – Bei Vorhandensein erfolgt eine Überlagerung und ein geometrisches Matching zusätzlicher Bildlayer zur einem Multilayer-Bild.
• – Zusatzinformationen durch Interaktion mit dem Anwender 13 werden angebracht.
• – Fallnummer, Bearbeiter, Status, Falltyp werden erstellt.
• – Alle Bild- und Textdaten werden als ein Virtueller Schnitt 212 in der Falldatenbank 222 abgelegt.
• 1. Eine Software 232 zur Bildbearbeitung Virtueller Schnitte 212. Diese kann alle Methoden der Bildverarbeitung umfassen, die an einem Digitalbild anwendbar sind, um automatisch Strukturen hervorzuheben, die den Anwender 13 bei der späteren Beurteilung des Bildes unterstützen. Solche Methoden sind zum Beispiel:
• – Kontrastverbesserung,
• – Normierung der Helligkeitsverteilung,
• – Weißabgleich,
• – Kantenextraktion,
• – Matching auf bekannte Strukturen zur Vorauswahl interessanter Bereiche,
• – Erkennen, Messen und Zählen von Objekten und Strukturen,
• – diverse weitere.

The server system 2 also includes some software components 23 , These are:

• 1. The application software 11 pointing to the web browser 111 or as a stand-alone program of a user 13 can be transferred. This application software 11 serves as user interface. It contains a first display window in which an overview image of the virtual section 212 is displayed and can be navigated in the virtual cut, another display window that displays a currently selected image section and several control tools that are used for navigation in the image, for processing the case data and for data management. Alternatively, this application software can also be loaded on the client side from a fixed data carrier (for example a CD-ROM).
• 2. A program 231 for generating the virtual cuts 212 , The program is needed to view the digital image data 31 of an image acquisition system into a virtual section 212 to format that then in the case database 222 is filed. Initially there is a virtual cut 212 from the image data, which also multilayer data are possible, and some data 32 such as case numbers, shooting date and the like, by the user 13 entered interactively or imported via an interface from another database system (eg hospital information system or PACS). The data of such a virtual cut 212 but grow in the processing of the case, as in the following various comments, markers and the like added. The program 231 for generating the virtual cuts 212 has the task, the transmitted image data in the special image format of a virtual cut 212 reformat, if necessary, to merge partial images into an overall image or to match several image layers together in such a way that they receive a common geometric, for example coordinate-related, reference. In addition, the user must 13 specified data in the format of the virtual cut 212 be registered. The procedure of the virtual cut generation is as follows:
• - First, digital image data 31 loaded from the image acquisition system in raw, tiff, jpg, or any other format.
• - According to the current case type (preparation-dependent) various image processing steps are applied in a fixed sequence.
• - There is a color adjustment to a specified color depth (eg 16 bits = 2 × 5 bits + 1 × 6 bits).
• - Subsequently, it is reformatted to a compressed image format (eg ECW).
• - If present, an overlay and geometric matching of additional image layers to a multilayer image occurs.
• - Additional information through interaction with the user 13 be installed.
• - Case number, engineer, status, case type are created.
• - All image and text data is considered a virtual section 212 in the case database 222 stored.
• 1. A software 232 for image editing of virtual sections 212 , This can include all image processing methods that are applicable to a digital image to automatically highlight structures that are the user 13 assist in the later assessment of the picture. Such methods are for example:
• - contrast enhancement,
• - normalization of the brightness distribution,
• - white balance,
• Edge extraction,
• - Matching to known structures for pre-selection of interesting areas,
• - detecting, measuring and counting objects and structures,
• - various others.

The image processing steps can be either by a user 13 be called or started automatically according to certain case types. A list of image processing steps can be created for each case type used, so that the system automatically starts predetermined steps according to the current case type.

Finally belongs to the server system 2 also an interface 25 with which the server 2 with the client systems 1 can be connected. This is preferably realized via TCP / IP.

client

On the user side 1 is the application software 11 via a web browser 111 controllable or runs as an independent surface and is therefore system-independent. The viewer for large image files is displayed on the user-side web browser 111 or the independent program interface. The image data are all stored on the server side. In the process, only the image data of interest is transferred from the server to the client system and decompressed. The special format for the image data ensures a high compression without visible loss of quality with fast access to individual areas within the image file. A high compression ratio is particularly important given the enormous size of the image data. The fast access to the image data allows the zooming and browsing of the images almost in real time. The software is accessed via a login to control access rights and to protect the data. This grants each user access to only certain data and receives specific administrative rights (such as create, read, modify, delete) for each type of data.

• 1. Die Sildbewegung kann mit variablen Eingabegeräten ermöglicht werden, wobei folgende Funktionen vorgesehen sind:
• a. stufenloses Bewegen des Bildes (ineteraktiv),
• b. stufenloses des Bildes (interaktiv),
• c. Verschieben des Bildes um einen bestimmten Betrag (halbautomatisch),
• d. Einstellen einer bestimmten Vergrößerungsstufe (halbautomatisch),
• e. Auswahl eines Bildausschnitts über Auswahlrechteck im Übersichtsbild (halbautomatisch),
• f. Auswahl eines Bildausschnitts über Auswahlpunkt im Übersichtsbild (halbautomatisch) und/oder
• g. Auswahl einer Bildmarke (s. u.) (automatisch).
• 2. Eine Bilddokumentation über eine Markierungsebene umfasst die Optionen:
• h. Einfügen von Bildmarken (interaktiv),
• i. Einfügen von Text oder Symbolen (interaktiv),
• j. Einfügen von Grafiken oder freihändiges Einzeichnen (interaktiv) und/oder
• k. Einfügen von Verweisen oder Links (interaktiv).
• 3. Eine Intelligente Suche von Bildern oder Bildbereichen umfasst:
• l. Suche nach Markern (automatisch),
• m. Suche nach Textinhalten der Markierungsebene (automatisch),
• n. Suche nach Falldaten (automatisch) und/oder
• o. Suche nach Bildinhalten (Strukturen, Texturen, Formen, Farben) (halbautomatisch).
• 4. Daneben können folgende Messfunktionen vorgesehen sein:
• p. Automatisches Zählen von Strukturen (automatisch),
• q. Streckenberechnung mit Angabe der Strecke im reellen Maßstab (automatisch),
• r. Flächenberechnung mit Angabe der Fläche im reellen Maßstab (automatisch) und/oder
• s. Klassifikation von Texturen oder Farben (halbautomatisch).
• 5. Weitere Bildverarbeitungsfunktionen können sein:
• t. Matching mehrerer Bildlayer (geometrische Anpassung von Bildern des gleichen Objektes in unterschiedlicher Färbung oder Aufnahmetechnik) (automatisch) und/oder
• u. Vergleich mehrerer Bildlayer (Differenzbilder) (automatisch).
• 6. Weitere Funktionen können sein:
• v. Snapshot (Speichern von Teilbildern) (automatisch) und/oder
• w. Trackhistory (Registrierung des Betrachtungspfades) (automatisch).
• 7. Als Verwaltungsfunktionen können vorgesehen sein:
• x. Eingabe einer Beurteilung (interaktiv),
• y. Benutzerverwaltung (halbautomatisch),
• z. Falldatenverwaltung in der Datenbank (halbautomatisch) und/oder
• aa. Tracking eines Falles entlang des gesamten Workflows (automatisch).

The user program 11 of the client system 1 may include a number of the following functionalities, particularly preferred basic functionalities being bold, advantageous meaningful main functionalities being normal and optional functionalities having only additional function being italicized:

• 1. The Sildbewegung can be made possible with variable input devices, whereby the following functions are provided:
• a. stepless movement of the image (inactive),
• b. continuous image (interactive),
• c. Moving the image by a certain amount (semi-automatic),
• d. Setting a certain magnification level (semi-automatic),
• e. Selection of an image section via selection rectangle in the overview image (semi-automatic),
• f. Selection of an image section via selection point in the overview image (semi-automatic) and / or
• G. Selection of a picture mark (see below) (automatic).
• 2. An image documentation over a marking level includes the options:
• H. Insertion of figurative marks (interactive),
• i. Insert text or symbols (interactive),
• j. Insert graphics or freehand drawing (interactive) and / or
• k. Insert links or links (interactive).
• 3. An intelligent search of images or image areas includes:
• l. Search for markers (automatically),
• m. Search for text contents of the marking level (automatic),
• n. Search for case data (automatic) and / or
• o. Search for image content (structures, textures, shapes, colors) (semi-automatic).
• 4. In addition, the following measuring functions can be provided:
• p. Automatic counting of structures (automatic),
• q. Route calculation with specification of the route on a real scale (automatic),
• r. Area calculation with indication of the area in real scale (automatic) and / or
• s. Classification of textures or colors (semi-automatic).
• 5. Other image processing functions can be:
• t. Matching multiple image layers (geometric adjustment of images of the same object in different colors or recording technique) (automatic) and / or
• u. Comparison of several image layers (difference images) (automatic).
• 6. Other functions can be:
• v. Snapshot (saving frames) (automatically) and / or
• w. Trackhistory (registration of the viewing path) (automatic).
• 7. As administrative functions can be provided:
• x. Input of a judgment (interactive),
• y. User management (semi-automatic),
• z. Case data management in the database (semi-automatic) and / or
• aa. Tracking a case along the entire workflow (automatically).

surface

• 1. Menüleiste Die Menüleiste enthält Steuerungen für alle Funktionen, die das Virtuelle-Mikroskop-System bietet. Zudem können über die Menüleiste die Verwaltung eines Falles gesteuert sowie alle Optionen und Systemattribute gesetzt und verändert werden.
• 2. Übersichtsbild Das Übersichtsbild zeigt anfangs das gesamte digitale Präparat (Virtueller Schnitt). Im Verlauf der Begutachtung dient es zur Auswahl einer neuen Bildansicht und zur Darstellung des aktuell angezeigten Bildbereiches. Hierzu erscheint im Übersichtsbild ein „Tracker", also ein Rechteck, dessen Begrenzungen mit den Rändern des Bildes der Bildansicht übereinstimmen. Mittels verschiedener Funktionen kann über den Tracker eine neue Bildansicht gewählt werden. Das Übersichtsbild selbst kann jedoch auch zur Anzeige einer Teilansicht veranlasst werden, wenn dies gewünscht ist.
• 3. Steuerungselemente Die Steuerungselemente umfassen verschiedene grafische Steuerungselemente, die die Navigation im Bild erleichtern sollen. Die Steuerung geschieht hier über die Maus. Alternativ kann auch über andere Eingabegeräte (Joystick, Touchpad, ...) oder über die Menüleiste navigiert werden.
• 4. Bildansicht Die Bildansicht zeigt das über das Übersichtsbild oder anderweitig ausgewählte aktuelle Teilbild. Anhand dieses Bildes kann der Anwender 13 ein Präparat begutachten. Das Übersichtsbild ist auf verschiedene Weisen navigierbar. Das kann über die Maus, über die Steuerungselemente, über das Übersichtsbild oder über andere Eingabegeräte geschehen. Zusätzlich können in der Bildansicht andere Bildlayer oder die Markierungsebene angezeigt werden, wahlweise auch in halbtransparenter Darstellung. Über die Maus ist ein Kontextmenü verfügbar, mit dem ortsgebundene Funktionen (beispielsweise das Setzen eines Markers) aktiviert werden können.
• 5. Statuszeile Die Statuszeile zeigt in einer Zeile Programmhinweise für den Anwender wie Progressbars, den Programmstatus etc.
• 6. Bildliste Die Bildliste ist eine Tabelle von kleinen Teilbildern, die entscheidende Inhalte des VS zusammenstellen, oder auch eine Tabelle von Übersichtsbildern aller VS eines Falles. Über Steuerungselemente ist es möglich, ein Bild auszuwählen, und in die Bildansicht umzuschalten.

The interface runs within a web browser 111 or an independent program. The program window (viewer) within the browser or program window is preferably divided into six areas. These six areas do not always have to appear simultaneously. Rather, the user can activate, deactivate and assemble the user-defined interface elements by himself.

• 1. Menu bar The menu bar contains controls for all the functions offered by the virtual microscope system. In addition, the administration of a case can be controlled via the menu bar and all options and system attributes can be set and changed.
• 2. Overview image The overview image initially shows the entire digital preparation (virtual section). In the course of the appraisal, it is used to select a new image view and to display the currently displayed image area. For this purpose, a "tracker" appears in the overview image, ie a rectangle whose borders correspond to the edges of the image in the image view.With various functions, a new image view can be selected via the tracker. if desired.
• 3. Controls The controls include various graphical controls designed to facilitate navigation in the image. The control happens here over the mouse. Alternatively, you can also navigate using other input devices (joystick, touchpad, ...) or via the menu bar.
• 4. Image view The image view shows the current partial image selected via the overview image or otherwise. Based on this image, the user can 13 to examine a preparation. The overview screen can be navigated in various ways. This can be done via the mouse, via the control elements, via the overview screen or via other input devices. In addition, other image layers or the marking level can be displayed in the image view, optionally also in semitransparent representation. A context menu is available via the mouse with which local functions (for example the setting of a marker) can be activated.
• 5. Status bar The status bar shows program hints for the user in one line, such as progress bars, the program status etc.
• 6. Image list The image list is a table of small sub-images that compile crucial contents of the VS, or a table of overview images of all VS of a case. Via controls it is possible to select an image and switch to the image view.

input devices

• 1. Bildaufnahmesystem Als System zur Digitalisierung der Bilddaten
• 2. Tastatur Zur Dateneingabe und Steuerung
• 3. 5-Tasten-Scrollmaus
• 4. Interface (z.B. HL7 oder DICOM-Scnittstelle) Zum Import von Daten aus anderen Datenbanksystemen
• 5. Joystick Werkzeug zur präzisen Bewegung eines Bildes. Dient auch zur Auswahl von Menüpunkten oder Ansteuerung von Buttons und ähnlichen Steuerungselementen.
• 6. Mikrofon Zur Dateneingabe (Aufsprechen hörbarer „Anmerkungen", In Verbindung mit einer Spracherkennungssoftware zur Eingabe von Textdaten) und Steuerung (Direkte Befehlseingabe mittels Spracherkennung, z.B. „Rauszoomen!"; Menü- und Oberflächensteuerung mittels Spracherkennung, z.B. „Datei-Öffnen-Laufwerk A!").
• 7. Datenhandschuh (Handschuh mit Bewegungssensoren zur direkten Steuerung eines Mauszeigers/Bildes auf dem Bildschirm). Als Steuerungswerkzeug zum Bewegen von Bildern auf dem Bildschirm, Ansteuern von Menüs oder Buttons und ähnlicher Steuerungselemente oder auch für Bewegung in 3D-Objekten.
• 8. Gestenerkennung (Kamera mit Software zur Erkennung von Gesten und Handbewegungen. Denkbar auch Erkennung der Blickrichtung). Als Steuerungswerkzeug zum Bewegen von Bildern auf dem Bildschirm durch Bewegen der Hand oder bloßes Hinsehen, Ansteuern von Menüs oder Buttons und ähnlicher Steuerungselemente auf dem Bildschirm durch Zeigen oder bloßes Hinsehen.

Input devices are devices with which a user can interact with the PC or enter data. This input is required for user interaction (operation of control tools) or data entry. The possible input devices are summarized in the following table, again particularly preferred basic functionalities being bold, advantageous meaningful main functionalities being normalized and optional functionalities having only additional functions being italicized:

• 1. Image recording system As a system for digitizing the image data
• 2. Keyboard For data entry and control
• 3. 5-key scroll mouse
• 4. Interface (eg HL7 or DICOM interface) For importing data from another database systems
• 5. Joystick tool for precise movement of an image. Also serves to select menu items or control buttons and similar controls.
• 6. Microphone For data input (recording audible "annotations", in conjunction with a speech recognition software for inputting text data) and control (direct command input via speech recognition, eg "Zoom out!"; Menu and surface control via speech recognition, eg "File Open Drive A! ").
• 7. Data glove (glove with motion sensors for direct control of a mouse pointer / image on the screen). As a control tool for moving images on the screen, driving menus or buttons and similar controls or for moving in 3D objects.
• 8. Gesture recognition (camera with software for recognition of gestures and hand movements, also conceivable detection of the viewing direction). As a control tool for moving images on the screen by moving the hand or just looking, driving menus or buttons and similar controls on the screen by pointing or looking.

Further usable, optional input devices are trackball, touch screen and / or touchpad:

output devices

• 1. Monitor (auch Touchscreen).
• 2. Modem bzw. Netzwerkkarte Zum Versenden eines Falles oder Teilen davon (z.B. einzelne Bilder) an entfernte Anwender durch Fax, E-Mail oder Direktverbindung (TCP/IP).
• 3. Interface (z.B. HL7 oder DICOM-Schnittstelle) Zum Export von Daten in andere Datenbanksysteme.
• 4. Lautsprecher Zur Ausgabe aufgesprochener Bemerkungen und zur Kommunikation bei Live-Diskussionen.
• 5. Drucker.
• 6. Beamer & Projektor.

Output devices are devices with which a user can access data from the computer. This includes devices with which a remote third party can perceive the data. The possible output devices are summarized in the following table, whereby particularly preferred basic functionalities are highlighted in bold, advantageous meaningful main functionalities are normal and optional functionalities with only additional function are shown in italics:

• 1st monitor (also touchscreen).
• 2. Modem or network card To send a case or parts thereof (eg individual pictures) to remote users by fax, e-mail or direct connection (TCP / IP).
• 3. Interface (eg HL7 or DICOM interface) For exporting data to other database systems.
• 4. Speakers For issuing recorded remarks and communicating during live discussions.
• 5. Printer.
• 6. Projector & Projector.

1

client system

11

application software

111

Web browser or independent Software interface

12

storage system

121

access data

13

user

14

access software on external databases

15

TCP / IP interface

2

server system

21

storage system

211

access data

212

virtual cut

22

databases

221

Senutzerdatenbank

222

Databank

23

server software

231

software for creating virtual cuts

232

software for image processing of virtual sections

24

external software

25

TCP / IP interface

3

input data

31

digital Image data of a preparation

32

case data of a preparation

4

external databases

Claims (27)

Virtual microscope system with a. digital image data provided by an image capture system or otherwise imported ( 31 ) of a preparation in the highest available or desired magnification and the highest available or desired section of the preparation, b. a server system ( 2 ) with - a software ( 231 ) for generating a virtual cut ( 212 ) in the highest available or desired magnification and the highest available or desired portion of the preparation from the digital image data ( 31 ) of the preparation, - a software ( 232 ) for image processing of the virtual cut ( 212 ) and - a memory device ( 21 ) for storing the virtual cut ( 212 ) in a case database ( 222 ), and c. at least one client system ( 1 ) with an application software ( 11 ) for presentation by a user ( 13 ) selectable data of a selectable virtual cut ( 212 ), whereby one by the user ( 13 ) selected image area of the selected virtual section ( 212 ) is currently requested and displayed, characterized in that all the data of each preparation as a virtual section ( 212 ) are stored in a file and that on the client side always on this file of the virtual Cut ( 212 ) of the highest available or desired magnification and the highest available or desired cutout. Virtual microscope system according to claim 1, characterized characterized in that the highest available or desired Section of the preparation one Overall view of the drug is. Virtual microscope system according to claim 1 or 2, characterized in that the software ( 231 ) for creating the virtual sections ( 212 ) a function of reformatting the digital image data provided by the image pickup system ( 31 ) into a compressed image format and the virtual sections ( 212 ) in the compressed image format in the case database ( 222 ), in particular in Enhanced Compressed Wavelet Format (ECW). Virtual microscope system according to one of the preceding claims, characterized in that the user software ( 11 ) of the at least one client system ( 1 ) Includes means by which the user ( 13 ) selected image area of the selected virtual section ( 212 ) in a compression level corresponding to a selected resolution from the case database ( 222 ) of the server system ( 2 ) or loaded and displayed by a mobile data carrier. Virtual microscope system according to one of the preceding claims, characterized in that the user software ( 11 ) comprises a viewer in which the overall view of the preparation and / or by the user ( 13 ) currently selected image area can be displayed as a detail view. Virtual microscope system according to one of the preceding claims, characterized in that the user software ( 11 ) comprises a continuous digital zooming function with which a detail view of any location selected by the user within the overall view of the virtual section (FIG. 212 ) is displayed to the highest resolution and wherein the zooming by interactive selection of the image area and determination of a representation size by the user ( 13 ) is infinitely feasible. Virtual microscope system according to claim 5 or 6, characterized in that the user software ( 11 ) comprises a tracker shown in the overall view of the specimen, which illustrates the position and extent of the illustrated detailed view and which is selected by the user ( 13 ) for selecting the image area and for manually scanning the virtual section ( 212 ) can be set manually, displaced and dimensioned, whereby a new detail view is selected. Virtual microscope system according to one of the preceding claims, characterized in that the digital image data transmitted by the recording system ( 31 ) in any image data format and by the software ( 231 ) for creating the virtual sections ( 212 ) are importable, whereby the color depth can be specified variably. Virtual microscope system according to one of the preceding claims, characterized in that in the case database ( 222 stored virtual sections ( 212 ) next to the image data further, by a user ( 13 ) of the recording system and / or a client system ( 1 ) input data ( 32 ), in particular status data of the case, acquisition parameters, preparation data, and / or the like, all data being in particular structured in SQL. Virtual microscope system according to one of the preceding claims, characterized in that the software ( 232 ) for editing the virtual sections ( 212 ) Image processing functions, in particular contrast enhancement; Normalization of the brightness distribution; White balance; Edge extraction; Matching to known structures for pre-selection of areas of interest; Detecting, measuring and counting objects and structures and / or other functions includes. Virtual microscope system according to one of the preceding claims, characterized in that one or more image processing functions of the software ( 232 ) after generation of the virtual cut ( 212 ) automatically depending on a case type or individually by a user ( 13 ) are executable. Virtual microscope system according to one of the preceding claims, characterized in that the server system ( 2 ) Medium ( 211 . 221 ) for the access control of users ( 13 ) on the server system ( 2 ). Virtual microscope system according to claim 12, characterized in that the means ( 211 . 221 ) for access control a user database ( 221 ) with user data stored therein, access rights and assignments to cases and the like as well as access data ( 211 ). Virtual microscope system according to one of the preceding claims, characterized in that the at least one client system ( 1 ) a client interface ( 15 ) and the server system ( 2 ) a server interface ( 25 ), which are in particular ICP / IP interfaces, and communication between client system ( 1 ) and server system ( 2 ) via the interfaces ( 15 . 25 ) via Intranet or Internet is feasible, with several clients systems ( 1 ) at the same time to the server system ( 2 ). Virtual microscope according to one of the preceding claims, characterized in that a client system ( 1 ) with the server system ( 2 ) is installed together on a computer station. Virtual microscope system according to one of claims 1 to 15, characterized in that the user software ( 11 ) within a web browser ( 111 ) of the clienis system ( 1 ) through the server system ( 2 ) provided. Virtual microscope system according to one of claims 1 to 15, characterized in that the user software ( 11 ) as an independent program interface in the client system ( 1 ) or within a program interface of another software in the client system ( 1 ) provided. Virtual microscope system according to one of the preceding claims, characterized in that in the server system ( 2 ) Means for Assignment of Virtual Section Data ( 212 ) to users ( 13 ), Case Status, Case Data, and Virtual Section Data ( 212 ) are provided with each other. Virtual microscope system according to one of the preceding claims, characterized in that the server system ( 2 ) and / or the at least one client system ( 1 ) Means for generating multilayer images from multiple overlaying virtual slices ( 212 ) of the same preparation, measured using different staining and / or imaging techniques. Virtual microscope system according to claim 19, characterized characterized in that the generation of multilayer images as well the composition of several partial images to form a complete picture automatic geometric matching is supported. A virtual microscope system according to claim 19 or 20, characterized in that by division of the detail view, Switching the detail view or by semitransparent display, being the measure of Transparency is adjustable, several bearings can be displayed simultaneously are. Virtual microscope system according to one of the preceding claims, characterized in that the server system ( 2 ) and / or the at least one client system ( 1 ) Means for generating a with the image data of the virtual cut ( 212 ) superimposed marking level, on which by the user ( 13 ) Image-coordinate-related comments, markings and / or the like attachable and integrated with the virtual section ( 212 ) can be stored in the case database. Virtual microscope system according to claim 22, characterized in that the marking plane is a marking during a virtual section analysis ( 212 ) by a user ( 13 ) and / or an inspection path considered by the user ( 13 ) inspected path through the preparation represents. Virtual microscope system according to one of the preceding claims, characterized in that the user software ( 11 ) Image processing functions, in particular contrast enhancement, normalization of the brightness distribution, white balance, edge extraction, matching to known structures for pre-selection of areas of interest, measuring and counting of structures and / or other functions includes. Virtual microscope system according to one of the preceding claims, characterized in that a virtual section ( 212 ) includes retrievable tracking data representing the virtual section ( 212 ) and constantly logging processes and information along the entire workflow, in particular the current status, users ( 13 ), Process and the like. Virtual microscope system according to one of the preceding claims, characterized in that the digital image data provided by the image acquisition system ( 31 ) of the preparation and the server-side generated virtual section ( 212 ) is a composite of partial areas image of the preparation. Virtual microscope system according to one of the preceding claims, characterized in that the virtual section ( 212 ) of a preparation comprises a multilayer image of a plurality of overlapping image layers of the same preparation, which is in particular structured in a SQL-structured manner.