CN117850918A - Method, device and equipment for realizing applet image recognition based on Taro framework - Google Patents

Abstract

The application provides a Taro frame-based applet image recognition realization method, a Taro frame-based applet image recognition realization device and equipment, wherein the method creates an object global variable and a monitoring object variable in the Taro frame, automatically realizes the call of a camera component in an applet source code module, uniformly formats images before target recognition, improves the efficiency of recognizing the images, and enables a rear end interface to be connected with different image recognition algorithms so as to recognize images which are not limited to code types; according to the further embodiment of the application, a mutual exclusion locking mechanism is further introduced, so that only one function is executed when the camera component is called, race conditions are avoided, stability and reliability of the system are improved, and user experience is improved.

Description

Method, device and equipment for realizing applet image recognition based on Taro framework

Technical Field

The invention belongs to the image recognition technology, and particularly relates to a Taro frame-based applet image recognition implementation method, device and equipment.

Background

The applet is an emerging application in recent years, and can be used by a user without installation, so that the applet is a lightweight, convenient and easy-to-use application program, and has wide application prospects in various scenes. The application of image recognition in small programs is attracting more and more attention, the related functions of image recognition are realized mainly by utilizing computer vision and deep learning technology, and the small programs can be used for face recognition, object recognition, text recognition and the like, so that convenient image processing services are provided for users.

At present, most small programs use an API interface to call a camera of a terminal device to scan or select images in a picture storage area to identify, the identification content is limited to one-dimensional codes, two-dimensional codes and Data Matrix codes, so that service scenes are limited, and when images outside identification code types are needed, the API interface provided by authorities cannot be realized. In addition, when images are identified in the existing applet, the conflict of identification results can occur in the scanning identification and the album selection identification.

Disclosure of Invention

Based on the method, the device and the equipment for realizing the recognition of the applet image based on the Taro framework are provided, so that the content of the image recognition in the applet is not single any more, and cross-platform development is supported.

In a first aspect, the present invention provides a Taro framework-based applet image recognition implementation method, including:

creating an object global variable under a Taro framework to call a camera component to acquire an image;

creating a monitoring object variable to monitor an image to be identified obtained when the camera component runs;

analyzing the image to be identified into a preset format to obtain a formatted image;

transmitting the formatted image to the rear end through an interface for target identification;

and acquiring the recognition result returned by the rear end and displaying the recognition result at the front end.

Further, the method further comprises the steps of:

the camera component is called and acquires a mutual exclusive lock when a scanning function is triggered, so that the camera component is kept in the current scanning process;

and acquiring the state lock, and releasing the state lock after the current scanning process is completed.

Further, before parsing the image to be identified into the preset format, the method further includes:

and rendering the image to be identified by utilizing the Taro component library to obtain a rendered image.

Further, the method comprises the steps of:

rendering the image to be identified to a Canvas engine by utilizing a Taro component library to obtain a rendered image;

converting the Canvas engine into a temporary file storage path;

the rendered image is stored to the Canvas engine.

Further, the preset format is a base64 format.

In a second aspect, the present invention provides an applet image recognition implementation apparatus, comprising:

the calling module is configured to create an object global variable under the Taro framework so as to call the camera component to acquire an image;

the component monitoring module is configured to create a monitoring object variable so as to monitor an image to be identified, which is acquired when the camera component runs;

the format conversion module is configured to analyze the image to be identified into a preset format to obtain a formatted image;

the image transmission module is configured to transmit the formatted image to the rear end through the interface for target identification;

the result display module is configured to acquire the recognition result returned by the back end and display the recognition result at the front end.

Further, the apparatus further includes:

and the image rendering module is configured to render the image to be identified by utilizing the Taro component library to obtain a rendered image.

In a third aspect, the present invention provides an applet image recognition implementation device comprising a memory storing computer executable instructions and a processor, which when executed by the processor causes the device to perform the steps of the Taro frame based applet image recognition implementation method provided in the first aspect.

In a fourth aspect, the present invention provides a readable storage medium storing a computer-executable program which, when executed, implements the steps of the Taro frame-based applet image recognition implementation method provided in the first aspect.

The invention also provides an applet application comprising a computer program which when run implements the Taro framework-based applet image recognition implementation method of the first aspect.

From the above technical scheme, the invention has the following beneficial effects:

the invention provides a Taro frame-based applet image recognition realization method, a Taro frame-based applet image recognition realization device and a Taro frame-based applet image recognition device, wherein the method is characterized in that an object global variable and a monitoring object variable are created in the Taro frame, the call of a camera component is automatically realized in an applet source code module, the uniform formatting is carried out on an image before target recognition, the efficiency of recognizing the image is improved, a rear end interface can be connected with different image recognition algorithms, and images which are not limited to code types can be recognized; according to the further embodiment of the application, a mutual exclusion locking mechanism is further introduced, so that only one function is executed when the camera component is called, race conditions are avoided, stability and reliability of the system are improved, and user experience is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of implementation of a Taro framework-based applet image recognition implementation method according to an embodiment of the present application;

FIG. 2 is a flow chart of implementing the exclusive lock execution logic provided in the embodiment of the present application;

fig. 3 is a schematic structural diagram of an applet image recognition implementation device according to an embodiment of the present application;

fig. 4 is a schematic diagram of an electronic device according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a computer-readable storage medium provided in an embodiment of the present application;

fig. 6 is a schematic diagram of an applet application loading page according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

At present, the scanning function of most small program applications only can identify code class images, such as one-dimensional codes, two-dimensional codes, data Matrix codes and PDF417 bar codes, and the API interface provided by the official application does not support images beyond the identification code class, so that the application of business scenes is limited; in addition, the applet application cannot be compatible with the scanning function and the album selection recognition function at the same time when recognizing the diversified image contents, and if the recognition is performed a plurality of times, a recognition result collision may also occur.

The following embodiments of the present application provide a Taro framework-based applet image recognition implementation method, apparatus and device, which are used for solving the technical problems of single recognition image type and process resource competition conflict when the existing applet application calls a camera component.

According to the technical scheme, the Taro framework is utilized to compile the applet code module, so that the identification of the multi-type image content is realized.

Referring to fig. 1, an embodiment of the present application provides a Taro framework-based applet image recognition implementation method, including the following steps:

and S110, creating an object global variable under the Taro framework to call the camera component to acquire an image.

In this step, the official APP carrying the applet application provides some built-in scanning function functions, such as taro.

And S120, creating a monitoring object variable to monitor an image to be identified obtained when the camera component runs.

Specifically, in order to ensure the normal operation of the Camera component and facilitate the call in the page, firstly, two object global variables ctx and a listening object variable listener are required to be created, wherein ctx is a Camera context object created by a taro.createCamera context () function, the function is used for executing logic such as an applet starting Camera component, photographing and the like, and setting the function as the global variable ctx can facilitate the operation of the Camera component of the terminal in applet application; listener is a listener object created by executing the ctx.onCameraframe ((frame) = > { }) function to listen to the data of each frame image for subsequent processing.

Illustratively, when the code module is compiled, the onLoad function is invoked to load the applet application page, and then the logic for starting the camera component is executed, that is, starting ctx=taro.createcamerac-ontext (), this function is started to initialize and create the context object ctx of the camera component so that it can be invoked in a subsequent step, and at a stage after the onLoad function is started, the Listener=ctx.onsamera frame ((frame) = > { }) is executed, this step is to create a listener for listening to the data such as the length and width of the image recognized by the camera component at intervals.

In some embodiments, the application sets the scan function to a default state after the applet application page loads, i.e., triggers the scan execution logic when the applet application page loads, and may call the startScan function after calling the onLoad function so that the applet application initiates the scan once the page loads.

S130, analyzing the image to be identified into a preset format to obtain a formatted image.

In this step, the parsing of the image to be identified into the preset format is to improve the efficiency of the target identification of the back end, that is, before the image to be identified is transmitted to the back end interface, the image to be identified is parsed into a uniform format, and then the back end performs backtracking on the formatted image to obtain the original image for identification.

Specifically, the taro.getfilesystemmanager () function is started under the Taro framework, whose execution logic is to retrieve the file storage path, and then format the image into a preset format, typically into a base64 format.

In some embodiments, the image to be identified is rendered before the image is formatted, specifically, a taro.canvas putimagedata () function may be started to render the image data to be identified into a Canvas engine, and then a taro.canvas totempfile path () function is started, the logic of the function is to convert the Canvas engine into a temporary file storage path, and the rendered image is stored in the path.

And S140, transmitting the formatted image to the rear end through an interface for target identification.

And S150, acquiring the identification result returned by the rear end and displaying the identification result at the front end.

In some embodiments, the applet application front end may be a process of presenting processing related data, or performing data processing directly at the front end, in addition to presenting the recognition result of the image.

The method provided by the embodiment can realize image recognition by utilizing the camera component of the applet application starting terminal built by the Taro framework, and transmits the acquired image to the target recognition algorithm at the rear end through the rear end interface for processing, so that the recognition of multi-type image content is realized.

The scan identification and album selection identification are two concurrently executed functions that a user may attempt to use at the same time, as the two operations may access shared resources or execute similar processing logic at the same time, resulting in conflicting identification results, poor user experience, or inaccurate results. Therefore, in some embodiments, to resolve this conflict, the present application introduces a mutual exclusion lock mechanism, ensuring that the camera component performs only one function at any time, avoiding competing resources between the execution logic of different functions.

As shown in FIG. 2, in one embodiment, the logic for initiating a mutex lock of the present application has the steps of:

s210, acquiring a mutual exclusion lock when the camera component is called and a scanning function is triggered, so that the camera component is kept in the current scanning process.

In this step, the scan logic parameter of the camera component and the album selection map identification logic parameter may be defined respectively, for example, the former is defined as scan, the latter is defined as zoom, and since the camera component is started and then monitors the running state through the execution logic of the listner function, the monitoring logic of listner under the scan logic may be set to be turned on by default, and when the user triggers the album selection map identification, the monitoring logic of listner is turned off.

Specifically, the function judging logic of the camera component is started, that is, an if (type= = 'scan') function is executed, the logic of the function is to judge whether the camera component is in the scanning logic, if yes, a listener. Start () function is started, otherwise, an else (type= 'zoom') function is executed, and a listener. Stop () function is started.

The execution logic can judge whether the current scanning identification or album selection identification is performed by maintaining the state of a lock, when a function is triggered, firstly, the lock is tried to be acquired, and if the lock is successfully acquired, the corresponding identification logic is executed; if the lock is already held by another function, then the appropriate processing policy is waited or enforced.

The advantage of utilizing locks for mutual exclusion limiting is that it ensures that only one function can modify the shared resource or perform sensitive operations at any time. This helps to avoid race conditions and improves the stability and reliability of the system.

And S220, acquiring a state lock, and releasing the state lock after the current scanning process is completed.

In the step, a scanning state lock is added on the basis of the mutual exclusion lock, so that only one scanning exists in the running process.

Specifically, another state may be introduced to perform scan control, a scan lock=false function is started, where the execution logic of the function is that the scan function of the camera component may be awakened when the scan variable is false, and when the scan starts, the scan lock=true function is executed, where the logic of the function is that the scan function of the camera component is locked, and the second scan cannot be performed, and it is necessary to wait until all the current scan process completes, including the scan image, the image conversion result, the image recognition result, and the front end completes all the processing of the above results, so that the scan variable is set to false, and the next scan can be performed.

By introducing a lock mechanism at the code level, orderly operation between scan identification and album selection identification can be ensured, so that user experience is improved, and users do not need to worry about conflicts and uncertainties possibly caused by simultaneous operation of two identification operations when using applet applications.

According to the Taro frame-based applet image recognition implementation method provided by the embodiments, the object global variable and the monitoring object variable are created in the Taro frame, the call to the camera component is automatically realized in the applet source code module, the image is uniformly formatted before target recognition, the image recognition efficiency is improved, the rear end interface can be connected with different image recognition algorithms, and images which are not limited to code types can be recognized; according to the further embodiment of the application, a mutual exclusion locking mechanism is further introduced, so that only one function is executed when the camera component is called, race conditions are avoided, stability and reliability of the system are improved, and user experience is improved.

When referring to the camera assembly, the terminal carrying the assembly may be an intelligent terminal having an image processing function including a smart phone, a tablet computer, a notebook computer, a desktop computer, a wearable device, a smart home, a head-mounted device, and the like.

The embodiment of the invention discloses an implementation method for identifying the small program image based on the Taro framework, which can be implemented by adopting various types of equipment, so that the invention also discloses an implementation device for identifying the small program image corresponding to the method, and specific embodiments are given below for detail.

As shown in fig. 3, one embodiment of the present application provides an applet image recognition implementation apparatus, which includes a calling module 310, a component listening module 320, a format conversion module 330, an image transmission module 340, and a result presentation module 350. The method specifically comprises the following steps:

a calling module 310 configured to create an object global variable under the Taro framework to call the camera component for image acquisition;

the component monitoring module 320 is configured to create a monitoring object variable to monitor an image to be identified acquired by the camera component during operation;

the format conversion module 330 is configured to parse the image to be identified into a preset format to obtain a formatted image;

an image transfer module 340 configured to transfer the formatted image to the backend for object recognition via the interface;

the result display module 350 is configured to obtain the identification result returned by the back end and display the identification result at the front end.

In a further embodiment, the apparatus further includes an image rendering module configured to render the image to be identified using the Taro component library to obtain a rendered image.

The device provided in the foregoing embodiments of the present application has the same implementation principle and technical effects as those of the foregoing method embodiments, and for a brief description, reference may be made to corresponding matters in the foregoing method embodiments for the part of the device embodiment that is not mentioned.

The following embodiments will be described by taking the application of the method to a computer device as an example, and it is to be understood that the computer device may be any device having a calculation and processing function, and may be, but not limited to, a server, a personal notebook, or the like. In one embodiment, the computer device may be an application server, which may be a server for running an application program under test.

Referring to FIG. 4, a block diagram of the hardware architecture of an electronic device, intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers, is shown. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.

As shown in fig. 4, the electronic device includes: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

in the embodiment of the application, the number of the processor 1, the communication interface 2, the memory 3 and the communication bus 4 is at least one, and the processor 1, the communication interface 2 and the memory 3 complete communication with each other through the communication bus 4;

processor 1 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention, etc.;

the memory 3 may comprise a high-speed RAM memory, and may further comprise a non-volatile memory (non-volatile memory) or the like, such as at least one magnetic disk memory;

wherein the memory stores a program, the processor is operable to invoke the program stored in the memory, the program operable to: and realizing the processing flows of the applet image identification implementation scheme based on the Taro framework.

Referring to fig. 5, an embodiment of the present invention further provides a readable storage medium 50, on which a computer program is stored, which when executed by a processor, implements the above-described embodiments and/or the respective process flows of the Taro frame-based applet image identification implementation provided in connection with any one of the possible implementations of the embodiments.

The computer readable storage medium 50 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 50 comprises a non-transitory computer readable medium (non-transitory computer-readable storage medium). The computer readable storage medium 50 has storage space for program code 52 that performs any of the method steps described above. The program code can be read from or written to one or more computer program products. Program code 52 may be compressed, for example, in a suitable form.

As shown in fig. 6, an embodiment of the present application further provides an applet application, including a computer program, which when run implements the Taro framework-based applet image recognition implementation method of the first aspect. Fig. 6 illustrates a loading page of the applet product running with the page open, i.e. with the scanning function of the camera assembly as default.

The methods and related devices of the embodiments described above are described with reference to the method flowcharts and/or structure diagrams provided in the embodiments of the present application, and each flowchart and/or block of the method flowcharts and/or structure diagrams may be implemented by computer program instructions, and combinations of flowcharts and/or blocks in the flowchart and/or block diagrams. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or structural diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or structures.

The embodiments described above have described the invention in particular detail with respect to possible scenarios, and those skilled in the art will recognize that the invention can be practiced with other embodiments. No particular naming of the components, case of terminology, attribute, data structure, or any other programming or structural aspect is mandatory or significant, and the mechanisms that implement the invention or its features may have different names, forms, or procedures. The system may be implemented by a combination of hardware and software (as described), entirely by hardware elements or entirely by software elements. The specific division of functionality between the various system components described herein is exemplary only and not mandatory; in contrast, functions performed by a single system component may be performed by multiple components, or functions performed by multiple components may be performed by a single component.

It will be appreciated by those skilled in the art that the various steps of the disclosed methods may be implemented by general purpose computing devices, they may be concentrated on a single computing device, or distributed across a network of computing devices, or they may alternatively be implemented in program code executable by computing devices, such that they may be stored in storage devices for execution by computing devices, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module. Thus, the present disclosure is not limited to any specific combination of hardware and software.

The programs (also referred to as programs, software applications, or code) executable by these computing devices include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

Certain aspects of the invention include the process steps and instructions described herein in the form of algorithms. It should be noted that the process steps and instructions of the present invention may be embodied in software, firmware, and/or hardware, which when implemented in software, can be downloaded to reside on and be operated from different platforms used by a variety of operating systems.

It will be appreciated by persons skilled in the art that the structures shown in the various figures are block diagrams of only some of the structures associated with the aspects of the present application and are not intended to limit the terminal device to which the aspects of the present application may be applied, and that a particular terminal device may include more or less components than those shown, or may combine some of the components, or have a different arrangement of components.

In the description of the present specification, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "possible designs," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that: the technical scheme described in the foregoing embodiments can be modified or some of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. The method for realizing the recognition of the applet image based on the Taro framework is characterized by comprising the following steps of:

creating an object global variable under a Taro framework to call a camera component to acquire an image;

creating a monitoring object variable to monitor an image to be identified obtained when the camera component runs;

analyzing the image to be identified into a preset format to obtain a formatted image;

transmitting the formatted image to the rear end through an interface for target identification;

and acquiring the recognition result returned by the rear end and displaying the recognition result at the front end.

2. The method according to claim 1, wherein the method further comprises:

the camera component is called and acquires a mutual exclusive lock when a scanning function is triggered, so that the camera component is kept in the current scanning process;

and acquiring the state lock, and releasing the state lock after the current scanning process is completed.

3. The method of claim 1, wherein prior to parsing the image to be identified into the preset format, the method further comprises:

and rendering the image to be identified by utilizing the Taro component library to obtain a rendered image.

4. A method according to claim 3, characterized in that the method comprises:

rendering the image to be identified to a Canvas engine by utilizing a Taro component library to obtain a rendered image;

converting the Canvas engine into a temporary file storage path;

the rendered image is stored to the Canvas engine.

5. The method of claim 1, wherein the predetermined format is a base64 format.

6. An applet image recognition implementation apparatus, comprising:

the calling module is configured to create an object global variable under the Taro framework so as to call the camera component to acquire an image;

the component monitoring module is configured to create a monitoring object variable so as to monitor an image to be identified, which is acquired when the camera component runs;

the format conversion module is configured to analyze the image to be identified into a preset format to obtain a formatted image;

the image transmission module is configured to transmit the formatted image to the rear end through the interface for target identification;

the result display module is configured to acquire the recognition result returned by the back end and display the recognition result at the front end.

7. The apparatus of claim 6, wherein the apparatus further comprises:

and the image rendering module is configured to render the image to be identified by utilizing the Taro component library to obtain a rendered image.

8. An applet image recognition implementation device comprising a memory storing computer executable instructions and a processor, which when executed by the processor causes the device to perform the Taro framework based applet image recognition implementation method according to any one of claims 1-5.

9. A readable storage medium storing a computer-executable program which, when executed, implements the Taro framework-based applet image recognition implementation method according to any one of claims 1 to 5.

10. An applet application, characterized in that it comprises a computer program which, when run, implements the Taro framework-based applet image recognition implementation method according to any one of claims 1 to 5.