TWI717141B - Gesture recognition method and mobile device - Google Patents

Abstract

The invention provides a gesture recognition method and a mobile device. The method includes: obtaining a first image, and determining a first ambient light intensity based on the first image; and finding a first brightness level of the first ambient light intensity; selecting a first gesture recognition model corresponding to the first brightness level; identifying a first gesture appearing in the first image with the first gesture recognition model.

Description

Gesture recognition method and mobile device

The present invention relates to an image recognition technology, and particularly relates to a gesture recognition method and mobile device.

With the development of technology, there are many mechanisms that allow users to input gestures to control electronic devices. However, since most of the existing gesture recognition mechanisms do not select different gesture recognition mechanisms in response to changes in the intensity of the ambient light, the effect of gesture recognition is poor.

In view of this, the present invention provides a gesture recognition method and mobile device, which can be used to solve the above technical problems.

The present invention provides a gesture recognition method suitable for a mobile device, including: obtaining a first image, and determining a first ambient light intensity based on the first image; finding the first ambient light intensity from a plurality of preset brightness levels A first brightness level in which the aforementioned preset brightness levels respectively correspond to a plurality of gesture recognition models; select a first gesture recognition model corresponding to the first brightness level from the aforementioned gesture recognition models; use the first gesture recognition The model recognizes a first gesture appearing in the first image.

The invention provides a mobile device, which includes an image capturing circuit and a processor. The processor is coupled to the imaging circuit and is configured to: control the imaging circuit to obtain a first image, and determine a first ambient light intensity based on the first image; find the first ambient light intensity from a plurality of preset brightness levels A first brightness level in which the aforementioned preset brightness levels respectively correspond to a plurality of gesture recognition models; select a first gesture recognition model corresponding to the first brightness level from the aforementioned gesture recognition models; use the first gesture recognition The model recognizes a first gesture appearing in the first image.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

In summary, the mobile device of the present invention can store multiple gesture recognition models corresponding to different preset brightness levels, so that after determining the ambient light intensity in the image, the corresponding gesture recognition model is selected for gesture recognition, thereby improving The details of the recognition accuracy are detailed below.

Please refer to FIG. 1, which is a schematic diagram of a mobile device according to an embodiment of the present invention. In the embodiment of the present invention, the mobile device 100 is, for example, smart glasses, a smart phone, a tablet computer or other similar smart devices, but it may not be limited thereto. As shown in FIG. 1, the mobile device 100 may include an image capturing circuit 102 and a processor 104. The image capturing circuit 102 is, for example, a charge coupled device (CCD) lens, a complementary metal oxide semiconductor transistors (CMOS) lens or any other circuit that can be used to capture/capture images, but it may not Limited to this.

The processor 104 is coupled to the image capturing circuit 102, and may be a general purpose processor, a special purpose processor, a traditional processor, a digital signal processor, multiple microprocessors, one or more combined digital signals Processor core microprocessors, controllers, microcontrollers, Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), any other types of integrated circuits Circuits, state machines, processors based on Advanced RISC Machine (ARM) and similar products.

In the embodiment of the present invention, the processor 104 can implement the gesture recognition method proposed by the present invention by accessing/executing specific modules and program codes. The details of the method are as follows.

Please refer to FIG. 2, which is a flowchart of a gesture recognition method according to an embodiment of the present invention. The method of this embodiment can be executed by the mobile device 100 in FIG. 1. The details of each step in FIG. 2 are described below with the components shown in FIG. 1.

First, in step S210, the processor 104 may control the image capturing circuit 102 to obtain a first image, and determine the first ambient light intensity based on the first image. In one embodiment, the processor 104 may call an appropriate light intensity recognition application programming interface (API) (for example, the API of ARCore LightEstimate getPixelIntensity() in the Android system) to determine the first image corresponding to the first image. An ambient light intensity, but not limited to this.

After that, in step S220, the processor 104 may find the first brightness level to which the first ambient light intensity belongs from a plurality of preset brightness levels, where the preset brightness levels individually correspond to a plurality of gesture recognition models.

In one embodiment, the aforementioned gesture recognition model can be generated by other devices (such as a computer device) after a certain training process, and provided to the mobile device 100 for use. For ease of understanding, it will be assumed below that the preset brightness levels considered in the present invention include a bright level, a normal level, and a dim level, and each of them corresponds to a brightness range. But it is not used to limit the possible implementation of the present invention.

表示），並據以將各訓練影像資料分為對應於上述預設亮度級別的多個訓練資料群組。例如，電腦裝置可將

高於0.7的訓練影像資料歸類至對應於明亮級別的第一訓練資料群組，以及將

介於0.4及0.7之間的訓練影像資料歸類至對應於一般級別的第二訓練資料群組。針對

介於0.15及0.4之間的訓練影像資料，電腦裝置可將其歸類至對應於昏暗級別的第三訓練資料群組，但可不限於此。 Specifically, in the above training process, the computer device may first obtain a batch of training image data, which may include various gesture images corresponding to different ambient light intensities. After that, for each training image data, the computer device can call the appropriate light intensity recognition API (such as the above LightEstimate getPixelIntensity() API) to obtain the ambient light intensity corresponding to each training image data (in

Indicates), and each training image data is divided into multiple training data groups corresponding to the above-mentioned preset brightness level. For example, a computer device can

The training image data higher than 0.7 are classified into the first training data group corresponding to the brightness level, and the

The training image data between 0.4 and 0.7 are classified into the second training data group corresponding to the general level. against

The training image data between 0.15 and 0.4 can be classified by the computer device into the third training data group corresponding to the dim level, but it is not limited to this.

After obtaining the first, second, and third training data groups corresponding to preset brightness levels such as the bright level, the normal level, and the dim level, the computer device can train a corresponding gesture recognition model based on each training data group. For example, the computer device can use the first training data group to train a first convolutional network model to generate a gesture recognition model corresponding to the brightness level. In addition, the computer device can use the second training data group to train a second convolutional network model to generate a gesture recognition model corresponding to the brightness level. Furthermore, the computer device can use the third training data group to train a third convolutional network model to generate a gesture recognition model corresponding to the brightness level.

In one embodiment, for each training data group, the computer device can use TensorFlow Lite to create a lightweight static gesture multi-layer convolutional network model, which includes, for example, N convolutional layers as feature extraction layers, and each convolutional layer represents A convolution (Convolution2D), a batch normalization (Batch Normalization) and a linear rectification function (ReLU) are performed once to increase the nonlinear characteristics. In one embodiment, the output layer of the convolutional layer can return 176 values, which represent eight preset anchor boxes, and the pixel values of the length and width are (2.26, 3.78) and (2.89, 5.77). , (3.25, 4.01), (3.63, 5.38), (3.85, 6.79), (4.56, 4.38), (4.81, 7.65), (5.29, 5.36) and their corresponding coordinate values, object confidence values and the difference between each category The confidence value of to give it the ability to distinguish multiple gestures (such as the 17 gestures illustrated in Figure 3 (ie, gestures 0 to 16)).

In one embodiment, the computer device can also use the imgaug package and write related automated programs to process each of the collected training image data, and related processing items include, for example: Fliplr, translate, zoom (scale), blur (GaussianBlur, averageBlur, MedianBlur), noise (AdditiveGaussianNoise), knockout (Dropout), brightness (Add, Multiply), increase contrast (ContrastNormalization), but not limited to this.

In other words, the aforementioned gesture recognition model corresponding to each preset brightness level can be used to recognize various gestures (for example, the 17 gestures illustrated in FIG. 3), but the corresponding ambient light intensity is different. Moreover, after completing the training of the aforementioned gesture recognition model, the computer device can compress each gesture recognition model with a TensorFlow Lite converter, and provide the compressed gesture recognition model to the mobile device 100 so that the mobile device 100 can proceed accordingly. Gesture recognition operation, but not limited to this.

In an embodiment, the aforementioned gesture recognition model may individually include N convolutional layers and one output layer, and N is, for example, a positive integer less than or equal to 15. In a preferred embodiment, N may be less than or equal to 12, for example. In an embodiment, N may be equal to 10, and its corresponding configuration may be as shown in Table 1 below. Types of Number of neurons Stride Output dimension Convolutional layer 1 32 2 112x112 Convolutional layer 2 32 1 112x112 Convolutional layer 3 64 2 56x56 Convolutional layer 4 32 1 56x56 Convolutional layer 5 128 2 28x28 Convolutional layer 6 61 1 28x28 Convolutional layer 7 256 2 14x14 Convolutional layer 8 128 1 14x14 Convolutional layer 9 256 2 7x7 Convolutional layer 10 256 1 7x7 Output layer 8(4+1+17)=176 1 7x7 Table 1 In the embodiment of the present invention, since the number of convolutional layers used is small, the mobile device 100 can perform subsequent gesture recognition operations with better efficiency.

Therefore, after obtaining the first ambient light intensity corresponding to the first image, the processor 104 may, for example, determine which of the preset brightness levels is within the brightness range to determine the first brightness level corresponding to the first ambient light intensity. For example, if the first ambient light intensity is higher than 0.7, the first brightness level corresponding to the first ambient light intensity is, for example, a bright level. If the first ambient light intensity is between 0.4 and 0.7, the first brightness level corresponding to the first ambient light intensity is, for example, a normal level. If the first ambient light intensity is 0.15 and 0.4, the first brightness level corresponding to the first ambient light intensity is, for example, a dim level, but it is not limited to this.

Next, in step S230, the processor 104 may select a first gesture recognition model corresponding to the first brightness level from the aforementioned gesture recognition models. That is, the processor 104 may extract the gesture recognition model corresponding to the first brightness level from the gesture recognition models corresponding to the bright level, the normal level, and the dim level as the first gesture recognition model.

After that, in step S240, the processor 104 may recognize the first gesture appearing in the first image by the first gesture recognition model.

It can be seen from the above that the mobile device 100 of the present invention can select the corresponding gesture recognition model according to the ambient light intensity corresponding to the image, so as to achieve a better gesture recognition effect.

In addition, in some embodiments, since the ambient light intensity around the mobile device 100 may change over time, the present invention further proposes a mechanism for replacing the adopted gesture recognition model to adaptively select the model based on the current ambient light intensity. For a suitable gesture recognition model, the relevant details will be illustrated in Figure 4.

Please refer to FIG. 4, which is a flowchart of an adaptive replacement gesture recognition model according to an embodiment of the present invention. In this embodiment, after selecting the first gesture recognition model to perform the gesture recognition operation, the processor 104 can then control the imaging circuit 102 to obtain a plurality of second images in step S401, and determine the plurality of second images corresponding to the aforementioned second images. The second ambient light intensity. After that, in step S402, the processor 104 may recognize a plurality of second gestures appearing in the aforementioned second image using the first gesture recognition model, wherein the aforementioned second gestures individually correspond to multiple recognition confidence values (ie, the first The degree of confidence in the recognition of each second gesture by the gesture recognition model).

Next, in step S403, the processor 104 may execute the first K-means algorithm on the aforementioned recognition confidence value to classify the aforementioned recognition confidence value into multiple confidence value groups. In one embodiment, the processor 104 may plot the above-mentioned recognition confidence value according to the corresponding time as a graph. The horizontal axis of the graph is time, for example, and the vertical axis is the recognition confidence value, for example. After that, the processor 104 may execute the first K-means algorithm based on the content of the schema to classify the recognition confidence values corresponding to different times into multiple (for example, K1) confidence value groups.

Then, the processor 104 may obtain the confidence reference value of each confidence value group (for example, the centroid of each confidence value group, but not limited to this) in step S404, and estimate the corresponding confidence value group Confidence average. That is, the processor 104 may calculate the average value of the confidence reference values (K1 in total) in each confidence value group as the above-mentioned confidence average value.

After that, the processor 104 may determine whether the confidence average value is higher than a confidence threshold (for example, 0.65) in step S405. In one embodiment, in response to the processor 104 determining that the confidence average value is higher than a confidence threshold, which represents that the recognition effect of the first gesture recognition model is still at a certain level, the processor 104 may perform step S406 to determine that no replacement is required The first gesture recognition model. However, in another embodiment, if the processor 104 determines that the confidence average value is not higher than the confidence threshold, this means that the recognition effect of the first gesture recognition model has gradually decreased. Therefore, the processor 104 may further execute the following methods to determine whether Replace the first gesture recognition model.

Specifically, the processor 104 may perform a second K-means algorithm on the second ambient light intensity corresponding to the second image in step S407, so as to divide the second ambient light intensity into multiple light intensity groups . In an embodiment, the processor 104 may plot the above-mentioned second ambient light intensity according to the corresponding time as a graph, where the horizontal axis of the graph is, for example, time, and the vertical axis is, for example, the ambient light intensity. After that, the processor 104 may execute a second K-means algorithm based on the content of this pattern to classify the second ambient light intensity corresponding to different times into multiple (for example, K2) light intensity groups.

After that, the processor 104 may obtain the light intensity reference value in each light intensity group (for example, the centroid of each light intensity group, but not limited to this) in step S408, and estimate the corresponding light intensity group Average light intensity. That is, the processor 104 may calculate the average value of the light intensity reference values (K2 in total) in each light intensity group as the above-mentioned average light intensity.

Next, the processor 104 may determine whether the average light intensity is lower than a light intensity threshold (for example, 0.15) in step S409. If so, this means that the current ambient light intensity may be too low to be suitable for subsequent gesture recognition operations, so the processor 104 may determine to stop performing the gesture recognition operations in step S410.

In addition, if the processor 104 determines that the average light intensity is not lower than the light intensity threshold, the processor 104 may determine whether the average light intensity belongs to the first brightness level in step S411. If so, it means that the first gesture recognition model is still applicable to the current environment, so the processor 104 may execute step S406 to determine that the first gesture recognition model does not need to be changed. In one embodiment, the processor 104 may further capture and send the considered second image to the back-end image database server to further analyze the reason for the low recognition confidence value, but it is not limited to this.

On the other hand, if the processor 104 determines that the average light intensity does not belong to the first brightness level, it means that the first gesture recognition model is no longer suitable for the current environment. Therefore, the processor 104 may perform step S412 to find the second brightness level to which the average light intensity belongs from the aforementioned preset brightness levels, and select the second gesture recognition corresponding to the second brightness level from the aforementioned gesture recognition model. Model, and the method of this step is similar to step S220, so it will not be repeated here. After that, the processor 104 may replace the first gesture recognition model with the second gesture recognition model in step S413, and continue to perform the gesture recognition operation.

It can be seen from the above that the present invention can also adaptively change the selected gesture recognition model according to the current ambient light changes, so that it has a better recognition effect in various environments.

Please refer to FIG. 5, which is an application scenario diagram drawn according to an embodiment of the present invention. In this embodiment, it is assumed that visitors wearing the mobile device 100 (such as AR glasses) proposed by the present invention move in the exhibition hall. When the visitors move to a dark environment, the mobile device 100 of the present invention can automatically select a corresponding gesture recognition model (for example, a model corresponding to a dark level) to perform gesture recognition operations. Later, when the visitors move from a dark environment to a bright environment, the mobile device 100 of the present invention can automatically select other gesture recognition models (for example, models corresponding to the brightness level) to perform gesture recognition operations. In this way, the present invention can have a better recognition effect in various environments with different brightness.

In other embodiments, the present invention can also adjust the way of controlling the mobile device 100 according to the detected gesture, which will be further described below.

In summary, the gesture recognition operation performed by the mobile device 100 of the present invention can roughly have the following three modes: a static gesture mode, a dynamic gesture mode, and a mouse gesture mode. In the static gesture, the mobile device 100 can perform a corresponding single specific operation according to the detected gesture. In one embodiment, it is assumed that the mobile device 100 detects that the user inputs a first preset gesture (such as gesture 1 in FIG. 3) in the static gesture mode, and moves the first preset in a specific manner (such as a large horizontal movement) Gesture, the mobile device 100 can switch from the static gesture mode to the mouse gesture mode.

In the mouse gesture mode, the mobile device 100 can capture the movement track of the finger in the user's gesture, and accordingly display the movement track on the display for the user's reference. In one embodiment, assuming that the mobile device 100 detects a second preset gesture input by the user in the mouse gesture mode (for example, gesture 7 in FIG. 3), the mobile device 100 can switch from the mouse gesture mode to a static gesture mode.

In addition, assuming that the mobile device 100 detects a third preset gesture (such as gesture 5 in FIG. 3) that does not move significantly in the static gesture mode, the mobile device 100 can switch from the static gesture mode to the dynamic gesture mode. In the dynamic gesture mode, the mobile device 100 can switch the page displayed by the mobile device 100 according to the swiping direction of the user's gesture, but it is not limited to this.

The relevant details of the above gesture modes are further explained as follows. For ease of description, it is assumed that the mobile device 100 is preset to be in the static gesture mode, but the present invention is not limited to this. In the static gesture mode, the processor 104 can control the imaging circuit 102 to obtain a plurality of third images, and recognize the plurality of third gestures appearing in the aforementioned third image and the position of each third gesture in the corresponding third image. Hand position.

In one embodiment, it is reflected in determining that the aforementioned third gesture corresponds to the first preset gesture (such as gesture 1 in FIG. 3), and the movement of the hand position (such as the fingertip position) in each third image When the first preset condition is met, the processor 104 can switch the mobile device 100 from the static gesture mode to the mouse gesture mode. In one embodiment, the processor 104 may obtain the positions of the fingertips of the first preset gesture in 10 consecutive third images, and obtain the displacement values between these positions (9 in total) according to time. If five of these displacement values exceed a threshold, the processor 104 can determine that the movement of the hand position in each third image meets the first preset condition, and then switch the mobile device 100 from the static gesture mode to the sliding mode. Mouse gesture mode.

In the mouse gesture mode, the processor 104 can control the imaging circuit 102 to obtain multiple fourth images, and recognize multiple fourth gestures appearing in the aforementioned fourth image and each fourth gesture in the corresponding fourth image Position of the hand. After that, the processor 104 may present a movement track on the mobile device according to the hand position (for example, the fingertip position) of each fourth gesture in the corresponding fourth image.

In addition, in the mouse gesture mode, if the processor 104 determines that a fourth gesture corresponding to a second preset gesture (such as gesture 7 in FIG. 3) is detected, the processor 104 can remove the mobile device 100 from the mouse The gesture mode is switched to static gesture mode.

In one embodiment, in the static gesture mode, the response is that the processor 104 determines that the consecutive multiple gestures in the third gesture correspond to a third preset gesture (such as gesture 5 in FIG. 3), and the gesture corresponds to The movement situation presented by the hand position meets a second preset condition, which switches the mobile device 100 from a static gesture mode to a dynamic gesture mode. In one embodiment, if the processor 104 detects 10 consecutive images corresponding to gesture 5, and the position of gesture 5 does not change significantly in these 10 images, the processor 104 can determine the hand corresponding to the gesture The movement situation presented by the location meets the second preset condition, and the mobile device 100 is switched from the static gesture mode to the dynamic gesture mode.

In the dynamic gesture mode, the processor 104 can control the imaging circuit 102 to obtain a plurality of fifth images, and recognize the plurality of fifth gestures appearing in the aforementioned fifth image and the position of each fifth gesture in the corresponding fifth image. Hand position. After that, the processor 104 may switch the display page of the mobile device according to the movement direction of the hand position in the corresponding fifth image of each fifth gesture.

In one embodiment, in the dynamic gesture mode, the processor 104 may return data of four directions including down, up, left, and right after determining the direction of the user's gesture. Moreover, when the gesture appears to move significantly in the consecutive fifth images, the processor 104 can switch the page displayed by the mobile device 100 accordingly. In one embodiment, when the movement range of the gesture reaches one third of the length and width of the screen, it can be determined that the gesture has moved significantly, but it is not limited to this. In this case, assuming the screen is 300 long and 600 wide, when the gesture is swiped to the left or right over 200 on the screen, the left or right swipe can be triggered, and the up or down swipe can be triggered when the gesture exceeds 100. .

In addition, in order to achieve continuous right swing instead of always producing left and right swings, we will stop the detection in M frames every time a swing is triggered, so that we can complete continuous swings in the same direction like turning a book.

Please refer to FIG. 6, which is a schematic diagram of an application scenario of a dynamic gesture mode according to an embodiment of the present invention. In this embodiment, it is assumed that images 611 to 616 are detected at time points T1 to T5, respectively. As shown in FIG. 6, at time T4, from the position of the gesture in the image 614 (ie, the coordinates (100, 150)) and the position of the gesture in the image 611 (ie, the coordinates (100, 50)), the gesture in the image 614 The corresponding change is (0, 100). Assuming that the width of the screen is 300, it can be known that the gesture at time T4 has been moved significantly (to the right), so the processor 104 can determine that the user has input a right swipe gesture, and can perform operations such as page changing accordingly, but not Limited to this. In addition, at the time points T5 and T6, the processor 104 may stop recognizing gestures to achieve the effect of continuous waving.

In addition, in an embodiment, if the processor 104 determines that one of the fifth gestures corresponds to the second preset gesture, the processor 104 can switch the mobile device 104 from the dynamic gesture mode to the static gesture mode, but it may not be limited to this .

Please refer to FIG. 7, which is a schematic diagram of a static/dynamic gesture mode according to an embodiment of the present invention. In this embodiment, the user can perform a desired operation by inputting one of the three gestures in the lower left corner of FIG. 7, for example. For example, the user can input the gesture 9 of FIG. 3 to perform the function of "selecting a device", or input the gesture 12 of FIG. 3 to perform the function of "calling an expert". In addition, the user can also input gesture 5 (ie, the third preset gesture) to enable the processor 104 to switch the mobile device 100 to the dynamic gesture mode accordingly. In this case, the user can switch the device page displayed by the mobile device 100 by swiping left and right gestures.

Please refer to FIG. 8, which is a schematic diagram illustrating a mouse gesture mode according to an embodiment of the present invention. As shown in FIG. 8, in the mouse gesture mode, the mobile device 100 can display a corresponding movement track 810 on the displayed screen 800 according to the movement of the user's gesture.

In summary, the present invention develops a method for achieving static and dynamic gesture control by adapting to various light source environments through the multilayer perceptron neural network and light source intensity detection technology, and can improve the accuracy and success rate of recognition gesture control. Using hand and finger gestures and tracking finger positions to interact with virtual devices to replace computer mouse control, human-computer interaction applications are more intuitive and convenient, replacing traditional control devices such as physical mice and keyboards.

In addition, the present invention has at least the following features: (1) It allows users to switch and interact between different devices without the need for a physical device to use gestures; (2) Using a multilayer perceptron neural network, through a deep convolutional neural model Automatically learn the image characteristics of the hand, detect the hand and hand posture in the image, identify and track its position and displacement direction, and use the technology of tracking fingers and recognizing gestures to interact with the virtual device; (3) Through the intensity of the ambient light source Detect and load a suitable gesture detection model to effectively improve the accuracy of gesture recognition and the success rate of gesture control.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be determined by the scope of the attached patent application.

100: mobile device 102: Acquisition circuit 104: processor 611~616: Video 800: screen 810: Movement Track S210~S240, S411~S413: steps T1~T6: time point

FIG. 1 is a schematic diagram of a mobile device according to an embodiment of the present invention. Fig. 2 is a flowchart of a gesture recognition method according to an embodiment of the present invention. FIG. 3 is a schematic diagram illustrating various gestures according to an embodiment of the present invention. 4 is a flowchart of an adaptive replacement gesture recognition model according to an embodiment of the present invention. Fig. 5 is an application scenario diagram drawn according to an embodiment of the present invention. Fig. 6 is a schematic diagram of an application scenario of a dynamic gesture mode according to an embodiment of the present invention. FIG. 7 is a schematic diagram of a static/dynamic gesture mode according to an embodiment of the invention. FIG. 8 is a schematic diagram illustrating a mouse gesture mode according to an embodiment of the invention.

S210~S240: steps

Claims (15)

A gesture recognition method, suitable for a mobile device, includes: Obtain a first image, and determine a first ambient light intensity based on the first image; Finding a first brightness level to which the first ambient light intensity belongs from a plurality of preset brightness levels, wherein the preset brightness levels respectively correspond to a plurality of gesture recognition models; Selecting a first gesture recognition model corresponding to the first brightness level from the gesture recognition models; The first gesture recognition model is used to recognize a first gesture appearing in the first image. For the method described in item 1 of the scope of patent application, each of the gesture recognition models is a convolutional network model, which includes N convolutional layers and 1 output layer, where N is a positive integer less than or equal to 15. The method described in item 2 of the scope of patent application, wherein N is less than or equal to 12. The method according to item 1 of the scope of patent application, wherein each of the preset brightness levels corresponds to a brightness range, and the brightness ranges corresponding to each of the preset brightness levels do not overlap with each other, and reflect the determination of the first ambient light intensity If it falls within the brightness range corresponding to the first brightness level, it is determined that the first ambient light intensity belongs to the first brightness level. The method described in item 1 of the scope of patent application further includes: Acquiring a plurality of second images, and determining a plurality of second ambient light intensities corresponding to the second images; Using the first gesture recognition model to recognize a plurality of second gestures appearing in the second images, wherein the second gestures respectively correspond to a plurality of recognition confidence values; Performing a first K-means algorithm on the identification confidence values to classify the identification confidence values into multiple confidence value groups; Obtain a confidence reference value in each confidence value group, and estimate a confidence average corresponding to the confidence value group accordingly; In response to determining that the confidence average value is higher than a confidence threshold, it is determined that the first gesture recognition model does not need to be replaced. For the method described in item 5 of the scope of patent application, wherein the second images respectively correspond to the second ambient light intensities, and are reflected in determining that the confidence average value is not higher than the confidence threshold, the method further includes : Performing a second K-means algorithm on the second ambient light intensities corresponding to the second images, so as to divide the second ambient light intensities into multiple light intensity groups; Obtain a light intensity reference value in each light intensity group, and estimate an average light intensity corresponding to the light intensity groups accordingly; In response to determining that the average light intensity is lower than a light intensity threshold, it is determined to stop performing a gesture recognition operation. The method described in item 6 of the scope of patent application, wherein in response to determining that the average light intensity is not lower than the light intensity threshold, the method further includes: Judging whether the average light intensity belongs to the first brightness level; In response to determining that the average light intensity belongs to the first brightness level, it is determined that the first gesture recognition model does not need to be replaced. For the method described in item 7 of the scope of patent application, wherein in response to determining that the average light intensity does not belong to the first brightness level, the method further includes: Find a second brightness level to which the average light intensity belongs from the preset brightness levels, and select a second gesture recognition model corresponding to the second brightness level from the gesture recognition models; Replace the first gesture recognition model with the second gesture recognition model, and continue to perform the gesture recognition operation. The method described in claim 1, wherein the mobile device operates in a static gesture mode, and the method further includes: Acquiring a plurality of third images, and identifying a plurality of third gestures appearing in the third images and the hand position of each third gesture in the corresponding third image; In response to determining that the third gestures correspond to a first preset gesture, and the movement of the hand position in each of the third images meets a first preset condition, the mobile device is changed from the static gesture The mode is switched to a mouse gesture mode. The method described in item 9 of the scope of patent application includes: Acquiring a plurality of fourth images, and identifying a plurality of fourth gestures appearing in the fourth images and the hand position of each fourth gesture in the corresponding fourth image; A movement track is presented on the mobile device according to the hand position of each fourth gesture in the corresponding fourth image. The method described in item 10 of the scope of patent application further includes: In response to determining that one of the fourth gestures corresponds to a second preset gesture, the mobile device is switched from the mouse gesture mode to the static gesture mode. The method described in item 9 of the scope of patent application includes: In response to determining that the consecutive multiple gestures in the third gestures all correspond to a third preset gesture, and the movement of the hand positions corresponding to the gestures meets a second preset condition, the mobile device Switch from the static gesture mode to a dynamic gesture mode. The method described in item 12 of the scope of patent application further includes: Acquiring a plurality of fifth images, and identifying a plurality of fifth gestures appearing in the fifth images and the hand position of each fifth gesture in the corresponding fifth image; Switching a display page of the mobile device according to the movement direction of the hand position in the corresponding fifth image of each of the fifth gestures. The method described in item 13 of the scope of patent application further includes: In response to determining that one of the fifth gestures corresponds to a second preset gesture, the mobile device is switched from the dynamic gesture mode to the static gesture mode. A mobile device including: An imaging circuit; and A processor coupled to the image capturing circuit and configured to: Controlling the image capturing circuit to obtain a first image, and determining a first ambient light intensity based on the first image; Finding a first brightness level to which the first ambient light intensity belongs from a plurality of preset brightness levels, wherein the preset brightness levels respectively correspond to a plurality of gesture recognition models; Selecting a first gesture recognition model corresponding to the first brightness level from the gesture recognition models; The first gesture recognition model is used to recognize a first gesture appearing in the first image.

Images