KR20210105679A - Artificial intelligence mobile interface system and method - Google Patents

Abstract

The present invention relates to an artificial intelligence mobile interface system and a method thereof. The artificial intelligence mobile interface method according to the present invention comprises the steps of: after executing an AI mouse by a user terminal, clicking a button for using a handlight on an initial screen of the user terminal; generating a UI capable of drawing a picture by the handlight to provide an environment for drawing a picture on a smartphone screen; drawing a specific picture by the handlight through the provided UI screen, to input and store new picture information; searching for handlight information pre-stored in a memory based on the input new picture information, and determining whether the newly input picture information matches the pre-stored picture information; when the newly input picture information and the pre-stored picture information match, transmitting a corresponding message to a server; receiving, by the server, the message from the user terminal through a cloud computer; and outputting and executing an event according to the message received by the server. According to the present invention, by recognizing characters, symbols, or icons input through a handwriting recognition screen in a mobile device, and performing a macro function for the Internet of things including a PC and a smart car, a variety of convenience can be provided to a user under the Internet of things environment.

Description

Artificial intelligence mobile interface system and method

The present invention relates to an artificial intelligence mobile interface system and method, and more particularly, by recognizing characters, symbols, or icons input through a handwriting recognition view on a mobile device, including a PC and a smart car. It relates to an artificial intelligence mobile interface system and method for performing macro functions for the Internet of Things.

The Internet connects various computing devices (eg, various servers, personal PCs, mobile devices, etc.) to enable the exchange of information between such devices.

In recent years, attempts have been made to exchange various information or data by connecting general computing devices as well as things that do not have a computing function, for example, sensors, electrical appliances, and meters to the Internet. This mechanism for connecting things to the Internet is commonly referred to as the Internet of things (IoT). In relation to the Internet of Things, a lot of research and development is being conducted on the establishment of protocols for communication between various devices, communication between devices, and data collection. The Internet of Things is expected to contribute to technological development through the Internet by being applied to various technological fields.

On the other hand, Korean Patent Application Laid-Open No. 10-2008-0107383 (Patent Document 1) discloses "a system that facilitates intuitive interaction between humans and machines". an authoring/analysis component facilitating generation of a plurality of assets having a plurality of parameters mapped to, and statistically analyzing the input criteria and rendering the assets based at least in part on the input criteria. a reasoning component, wherein the authoring/analysis component collects feedback based at least in part on the input and the rendered asset, learns from the feedback and based at least in part on the user intent infer, and wherein the inference component selects the asset based at least in part on the analysis of the user intent.

The system that facilitates the intuitive interaction between a person and a machine of Patent Document 1 as described above can discover one asset or a group of assets based on inference, and one or more user input(s), operation(s) , and/or state(s) that may have the advantage of providing a natural language interface that learns and/or adapts based on state(s), but It recognizes characters, symbols, or icons to perform macro functions for the Internet of Things, including PCs and smart cars, and analyzes the user's process of using the Internet of Things through a specific mobile interface and automatically recommends it to the mobile interface. We cannot provide the same service as

The present invention has been created in consideration of the above, and recognizes characters, symbols, or icons input through a handwriting recognition view on a mobile device, and for the Internet of Things including PCs and smart cars. To provide an artificial intelligence mobile interface system and method that performs macro functions and enables users to set and use their own characters, symbols, and icons input through the handwriting recognition screen when using the Internet of Things for security. There is a purpose.

In order to achieve the above object, an artificial intelligence mobile interface system according to the present invention,

An artificial intelligence mobile mouse (hereinafter referred to as 'AI mouse') application that can perform an artificial intelligence mobile interface function is loaded, and using the AI mouse, a message for performing the operation of electronic devices constituting the Internet of Things is displayed. transmitting user terminal;

a cloud computer that transmits and receives signals through wireless communication with the user terminal, receives a message from the user terminal, and delivers the message to a corresponding electronic device constituting the Internet of Things; and

The feature is that it includes a server program that is downloaded and executed on a PC or driven on the Internet of Things, receives a message from the user terminal via the cloud computer, and outputs and performs an event according to the received message. have.

Here, the AI mouse app mounted on the user terminal may include a handlight recognizer for recognizing a picture drawn by handwriting.

In this case, the handlight recognizer may be connected to an artificial intelligence macro function.

According to the present invention, by recognizing characters, symbols, or icons input through a handwriting recognition view on a mobile device, and performing a macro function for the Internet of Things including a PC and a smart car, Various conveniences can be provided to users in the Internet of Things environment.

In addition, when using the Internet of Things for security, users can set their own characters, symbols, and icons input through the handwriting recognition screen and use them, further solidifying the security in the use of the Internet of Things for security. have.

The terms or words used in the present specification and claims should not be construed as being limited to their ordinary or dictionary meanings, and the inventor may properly define the concept of the term in order to best describe his invention. Based on the principle, it should be interpreted as meaning and concept consistent with the technical idea of the present invention.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as “…unit”, “…group”, “module”, and “device” described in the specification mean a unit that processes at least one function or operation, which is hardware or software or hardware and software. It can be implemented as a combination.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The term "server" used in this embodiment is a concept including both a hardware meaning and a software meaning. That is, when expressed as "server", it may mean a server as hardware or a server program as software.

1 is a diagram schematically showing the configuration of an artificial intelligence mobile interface system according to an embodiment of the present invention.

Referring to FIG. 1 , the artificial intelligence mobile interface system 100 according to the present invention may be configured to include a user terminal 110 , a cloud computer 120 and a server program.

The user terminal 110 is equipped with an artificial intelligence mobile mouse (hereinafter referred to as 'AI mouse') application that can perform an artificial intelligence mobile interface function therein, and configures the Internet of Things using the AI mouse. It transmits a message for performing the operation of the electronic device. A smartphone may be used as such a user terminal 110 . However, it is not limited to a smartphone, and any mobile device having a display window and capable of wireless communication (eg, PDA, tablet PC, netbook PC, notebook PC, etc.) can be used. Here, the AI mouse app mounted on the user terminal 110 is also a handlight recognizer that recognizes a picture drawn by handwriting (one that recognizes a picture, character, symbol, etc. drawn by handwriting) software program). In this case, such a handlight recognizer may be connected to an artificial intelligence macro function. And this AI macro function can be linked with various handwritten characters, symbols, and icons.

In addition, preferably, the AI mouse app monitors the sensor value of the gyro sensor of the user terminal (smartphone), so that the user terminal (smartphone) does not move for a predetermined time (eg, for 3 minutes or 5 minutes) When placed in a state, it may have a notification function to remind whether or not to use the AI mouse (eg, a notification such as "Do you want to use the AI mouse?").

Also, preferably, the AI mouse app may have a function of setting an artificial intelligence macro function selected by the user so that the user can use the desired artificial intelligence macro function through login.

In addition, the AI mouse app may have a function of connecting a communication channel to a PC and the Internet of Things through login.

In addition, the AI mouse app may have a function of transmitting data from the PC to the mobile through a mouse click, or transmitting data from the mobile to the mobile or from the mobile to the PC through an artificial intelligence macro. That is, AI mouse (app) transfers data including multimedia from PC or Internet of Things from PC to mobile phone, mobile phone to mobile phone, and mobile phone to PC through mouse clicks or artificial intelligence macro input (handwriting input). can be moved Here, the data movement speed between PC and PC, PC and mobile, and mobile and mobile can be increased by using image downsampling and super-resolution techniques for moving and storing such multimedia.

In addition, the AI mouse app hand-recognizes a pattern of a PC or Internet of Things (for example, a repetitive situation in which an important e-mail must be delivered to a company representative or a boss once a day among mails confirmed by executing Word) It can be equipped with an artificial intelligence macro function that is automatically executed for a certain period of time by giving a command to (eg, Hi!). In particular, the artificial intelligence macro function using handwriting recognition as described above may be preferentially provided.

The cloud computer 120 transmits and receives signals through wireless communication with the user terminal 110, receives a message from the user terminal 110, and delivers the message to the corresponding electronic device constituting the Internet of Things. Here, the cloud computer 120 refers to a computer under a cloud computing environment, and the cloud computing environment is a computing environment in which IT-related services such as data storage, network, and content use can be simultaneously used through a server on the Internet. say

The cloud computer 120 as described above may serve to store (manage) login information of a user who uses a smartphone. In addition, the cloud computer 120 may have a function that allows the user to set an event to be performed after the user draws his or her own picture (icon, symbol, character, etc.) with a handlight. That is, the cloud computer 120 stores information about a picture (icon, symbol, alphabet, etc.) drawn by a user to set an event, and even stores event information set for the drawn picture. In this way, since the set event is stored in the cloud computer 120, when the logged-in user draws his/her own icon, the server program can perform the corresponding event.

In addition, the cloud computer 120 stores information that allows the user to continue using the user's own icon set by the user when the user logs in.

In addition, the cloud computer 120 maintains the AI mouse and the PC or Internet of Things channel of the user terminal 110 in an always open state through login, so that the user can always use the AI mouse. In addition, the cloud computer 120 sets and stores the handwriting input method artificial intelligence macro function desired by the user as a function desired by the user.

The server program is downloaded and executed on a PC or runs on the Internet of Things, receives a message from the user terminal 110 via the cloud computer 120, and outputs and performs an event according to the received message. do. That is, the server program is executed in a place that can use Wi-Fi or Bluetooth, such as a PC or the Internet of Things, and the present invention assumes that such a server program is being executed.

In FIG. 1, reference numerals 130a to 130e denote things (electronic devices) in an Internet of Things environment to which the artificial intelligence mobile interface system 100 according to the present invention is applied, 130a denotes a smart vehicle, 130b denotes a personal PC, and 130c denotes a personal computer. A smart TV, 130d, a refrigerator, and 130e, a light/stand, respectively.

Hereinafter, an artificial intelligence mobile interface method according to the present invention will be described based on the artificial intelligence mobile interface system according to the present invention having the above configuration.

2 is a flowchart illustrating an execution process of an artificial intelligence mobile interface method according to an embodiment of the present invention.

1 and 2, the artificial intelligence mobile interface method according to the present invention is an artificial intelligence having a user terminal (smartphone) 110, a cloud computer 120 and a server (program) as described above. It is based on a mobile interface system, and an artificial intelligence mobile mouse (hereinafter referred to as 'AI mouse') application that can perform an artificial intelligence mobile interface function on a user terminal is downloaded and installed. As a method of executing the interface, first, the AI mouse (AI mouse app) is executed by the user terminal 110 (here, a case where a smartphone is used as the user terminal as an example) is executed, and then as shown in FIG. The button 113 for using a handwrite is clicked on the initial screen displayed on the display window of the same user terminal 110 (step S201).

Then, the AI mouse (AI mouse app) creates a UI (user interface) that can draw a picture by handlight, and displays pictures (pictures, characters, symbols, icons, etc.) on the smartphone screen as shown in FIGS. 5 to 7 . An environment for drawing is provided (step S202).

If an environment (UI environment) for drawing pictures (pictures, characters, symbols, icons, etc.) is provided on the smartphone screen by the above, a specific picture (picture, text, etc.) is provided by the user through the UI screen provided by the user. , symbol, icon, etc.) as a picture (step S203), and new picture information is input and stored (step S205).

Here, in the step S203, when drawing is temporarily stopped while a specific picture is drawn by the handlight through the UI screen, determining whether a predetermined time (for example, 3 seconds) has elapsed after stopping drawing ( S204) may be further included. At this time, if the predetermined time (3 seconds) has not elapsed in the determination of the step S204, it is determined that the picture is still being drawn and the process returns to the step S203. If the predetermined time (3 seconds) has elapsed, new picture information is input and to store (ie, proceed to step S205 above).

Here, before executing step S207 to be described later after step S205, the method may further include a step (S206) of resetting the handlight UI screen on which the picture is drawn. That is, if the information about the picture is stored by the step S205, the picture (characters, symbols, etc.) drawn on the macro screen 110m (see FIG. 5) that can be drawn by the handlight is removed and the screen is clean. It further includes a reset step to return to (the state before the picture was drawn).

Then, based on the new picture (picture, text, symbol, icon, etc.) information input in step S205, the handlight information stored in memory is searched (step S207), and the newly input picture (picture, text, It is determined whether there is a match between the information (symbol, icon, etc.) information and the pre-stored picture (picture, text, symbol, icon, etc.) information (step S208).

In the determination of step S208, if the newly input picture information and the pre-stored picture information match, a corresponding message is transmitted to the server (server program) (step S209). Here, as described above, such a server program is executed in a place that can use Wi-Fi or Bluetooth, such as a PC or the Internet of Things, and in the present invention, on the premise that such a server program is running do.

In this way, when a message is transmitted from the user terminal (smartphone) 110, the message is received from the user terminal 110 through the cloud computer 120 by the server (program) (step S210).

Then, an event according to the message received by the server (program) is output and performed (steps S211 and S212).

Here, in relation to the above, with reference to FIGS. 3 to 7 will be described in more detail.

3 is a view showing an initial screen after login when executing an AI mouse in the user terminal of the artificial intelligence mobile interface system according to the present invention.

Referring to FIG. 3 , this is an initial screen displayed when the user logs in after running the AI mouse app mounted on the user terminal 110 . In FIG. 3 , reference number 111 is an AI (artificial intelligence) mouse. Point cursor performing a kind of cursor function, 112 is a keyboard button to display the keyboard pad 110k as in FIG. 4 on the screen, 113 is an artificial intelligence (AI) macro for using a handwrite (handwrite) Buttons 114 and 115 denote enlargement and reduction buttons, 116 and 117 denote left and right buttons in a normal mouse, and 118 and 119 denote scroll buttons having an up and down scroll wheel function in a normal mouse, respectively.

Referring to FIG. 5 , this shows the AI macro screen 110m that appears when the artificial intelligence macro button 113 is touched on the initial screen of the user terminal 110 described above. As shown on this AI macro screen 110m, when the English alphabet "P" is written by handwriting, it is recognized by the handlight recognizer and the user terminal 110 is the personal PC 130b in FIG. It sends an event message that can be shut down. At this time, of course, it is assumed that there is a server program in the PC and the server program is running.

6 shows a case in which the letter "L" of the English alphabet is written by handwriting. At this time, the handlight recognizer recognizes this and the user terminal 110 transmits a logoff event message of the PC.

7 shows a case in which the letter "R" of the English alphabet is written by handwriting. At this time, the handlight recognizer recognizes this and the user terminal 110 transmits a PC restart event message.

As described above, when a message related to performing each event is transmitted from the user terminal 110, the server program installed in the corresponding electronic device constituting the Internet of Things receives it through the cloud computer 120, and Outputs and executes events according to messages.

Here, the cloud computer 120 stores information on pictures (icons, symbols, alphabets, etc.) drawn by the user to set events as described above, and even stores event information set for the drawn pictures. In this case, when the logged-in user draws his/her own icon, the server program can perform a corresponding (corresponding) event. For example, as described above, "P" is set to shut down the PC, but this can be set so that the PC can be shut down by drawing "W".

As described above, the artificial intelligence mobile interface system and method according to the present invention recognizes characters, symbols, or icons input through a handwriting recognition view on a mobile device, and objects including a PC and a smart car. It performs a macro function for the Internet, analyzes the user's process of using the Internet of Things through a specific mobile interface, and automatically recommends it to the mobile interface, which has the advantage of providing various conveniences to users in the Internet of Things environment.

In addition, when using the Internet of Things for security, users can set their own characters, symbols, and icons input through the handwriting recognition screen and use them, further solidifying the security in the use of the Internet of Things for security. There is an effect.

As described above, the present invention has been described in detail through preferred embodiments, but the present invention is not limited thereto, and it is common knowledge in the art that various changes and applications can be made without departing from the technical spirit of the present invention. self-explanatory to the technician. Accordingly, the true protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (4)

An artificial intelligence mobile mouse (hereinafter referred to as 'AI mouse') application that can perform an artificial intelligence mobile interface function is loaded, and using the AI mouse, a message for performing the operation of electronic devices constituting the Internet of Things is displayed. transmitting user terminal;

a cloud computer that transmits and receives signals through wireless communication with the user terminal, receives a message from the user terminal, and delivers the message to a corresponding electronic device constituting the Internet of Things; and

An artificial intelligence mobile interface including a server program that is downloaded and executed on a PC or driven on the Internet of Things, receives a message from the user terminal via the cloud computer, and outputs and performs an event according to the received message system.
According to claim 1,

The AI mouse app mounted on the user terminal is an artificial intelligence mobile interface system, characterized in that it has a handwriter recognizer for recognizing a picture drawn by handwriting (handwrite).
3. The method of claim 2,

The handlight recognizer is an artificial intelligence mobile interface system, characterized in that connected to the artificial intelligence macro (macro) function.
4. The method of claim 3,

The artificial intelligence macro function is an artificial intelligence mobile interface system, characterized in that it connects with various handwritten characters, symbols, and icons.