+

WO2018033953A1 - SYSTÈME D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO, PROCÉDÉ D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO ET PROGRAMME - Google Patents

SYSTÈME D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO, PROCÉDÉ D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO ET PROGRAMME Download PDF

Info

Publication number
WO2018033953A1
WO2018033953A1 PCT/JP2016/073856 JP2016073856W WO2018033953A1 WO 2018033953 A1 WO2018033953 A1 WO 2018033953A1 JP 2016073856 W JP2016073856 W JP 2016073856W WO 2018033953 A1 WO2018033953 A1 WO 2018033953A1
Authority
WO
WIPO (PCT)
Prior art keywords
user interface
iot device
user
user terminal
icon
Prior art date
Application number
PCT/JP2016/073856
Other languages
English (en)
Japanese (ja)
Inventor
俊二 菅谷
Original Assignee
株式会社オプティム
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社オプティム filed Critical 株式会社オプティム
Priority to PCT/JP2016/073856 priority Critical patent/WO2018033953A1/fr
Publication of WO2018033953A1 publication Critical patent/WO2018033953A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04Q—SELECTING
    • H04Q9/00—Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Definitions

  • the present invention relates to an IoT device user interface display system, an IoT device user interface display method, and a program for displaying a user interface of an IoT (Internet of Things) device on a user terminal.
  • IoT Internet of Things
  • IoT device In recent years, by connecting various devices to a public network such as the Internet, it has been possible to remotely operate the devices. Such a device is called an IoT device.
  • IoT device the user performs an operation desired by the user through a user terminal or a dedicated terminal device owned by the user.
  • An object of the present invention is to provide an IoT device user interface display system, an IoT device user interface display method, and a program capable of displaying a user interface of an IoT device on a user terminal.
  • the present invention provides the following solutions.
  • the invention according to the first aspect is an IoT device user interface display system for displaying a user interface of an IoT device on a user terminal, Captured image acquisition means for acquiring a captured image of an IoT device imaged by a camera; Image analysis means for image analysis of the acquired captured image; IoT device specifying means for specifying the IoT device based on the image analysis result; User interface specifying means for specifying a user interface of the specified IoT device; User interface display means for displaying the identified user interface on the user terminal; An IoT device user interface display system is provided.
  • an IoT device user interface display system that displays a user interface of an IoT device on a user terminal acquires a captured image of the IoT device captured by a camera, and the acquired captured image The IoT device is identified based on the result of the image analysis, the user interface of the identified IoT device is identified, and the identified user interface is displayed on the user terminal.
  • the invention according to the first feature is a category of the IoT device user interface display system, but in other categories such as a method or a program, the same actions and effects according to the category are exhibited.
  • the invention according to the second feature is an input receiving means for receiving an input to the displayed user interface; Information transmitting means for transmitting the input information to the specified IoT device; An IoT device user interface display system, which is an invention according to a first feature, is provided.
  • the IoT device user interface display system receives an input to the displayed user interface, and the input information is determined by the specified information. Send to IoT device.
  • the information transmission unit transmits the input information from the user terminal to the specified IoT device using short-range radio.
  • An IoT device user interface display system which is an invention according to the second feature is provided.
  • the IoT device user interface display system uses the short-range radio to the specified IoT device from the user terminal, and the input Sent information.
  • the user interface specifying unit when the user interface specifying unit cannot specify the user interface of the specified IoT device, it specifies a general-purpose user interface according to the type of the IoT device,
  • the user interface display means displays the specified universal user interface on the user terminal;
  • An IoT device user interface display system which is an invention according to the first feature is provided.
  • the IoT device user interface display system when the IoT device user interface display system according to the first feature of the invention is unable to specify the user interface of the specified IoT device, the IoT device user interface display system sets the type of the IoT device. A corresponding general-purpose user interface is specified, and the specified general-purpose user interface is displayed on the user terminal.
  • An invention according to a fifth aspect is a IoT device user interface display method for displaying a user interface of an IoT device on a user terminal, Obtaining a captured image of the IoT device captured by the camera; Analyzing the acquired captured image; and Identifying the IoT device based on the image analysis result; Identifying a user interface of the identified IoT device; Displaying the identified user interface on the user terminal; An IoT device user interface display method is provided.
  • the invention according to a sixth aspect provides an IoT device user interface display system for displaying a user interface of an IoT device on a user terminal. Obtaining a captured image of an IoT device captured by a camera; Image analysis of the acquired captured image; Identifying the IoT device based on the image analysis result; Identifying a user interface of the identified IoT device; Displaying the identified user interface on the user terminal; Provide a program to execute.
  • an IoT device user interface display system an IoT device user interface display method, and a program capable of displaying a user interface of an IoT device on a user terminal.
  • FIG. 1 is a diagram showing an overview of an IoT device user interface display system 1.
  • FIG. 2 is an overall configuration diagram of the IoT device user interface display system 1.
  • FIG. 3 is a functional block diagram of the computer 10, the user terminal 100, and the IoT device 200.
  • FIG. 4 is a diagram illustrating an IoT device user interface display process executed by the computer 10, the user terminal 100, and the IoT device 200.
  • FIG. 5 is a diagram illustrating an IoT device user interface display process executed by the computer 10, the user terminal 100, and the IoT device 200.
  • FIG. 6 is a diagram illustrating an example of a user interface corresponding to the IoT device 200 displayed on the user terminal 100.
  • FIG. 1 is a diagram showing an overview of an IoT device user interface display system 1.
  • FIG. 2 is an overall configuration diagram of the IoT device user interface display system 1.
  • FIG. 3 is a functional block diagram of the computer 10, the user terminal 100, and the IoT device 200.
  • FIG. 7 is a diagram showing an example of a user interface showing a state where the user has selected the remote control selection icon 450 displayed on the user terminal 100.
  • FIG. 8 is a diagram illustrating an example of a general-purpose user interface corresponding to the type of the IoT device 200 displayed on the user terminal 100.
  • FIG. 9 is a diagram illustrating an example of a general-purpose user interface showing a state where the user has selected the remote control selection icon 640 displayed on the user terminal 100.
  • FIG. 1 is a diagram for explaining an overview of an IoT device user interface display system 1 according to a preferred embodiment of the present invention.
  • the IoT device user interface display system 1 includes a computer 10, a user terminal 100, and an IoT device 200.
  • the computer 10, the user terminal 100, and the IoT device 200 are not limited to one and may be plural.
  • the computer 10, the user terminal 100, and the IoT device 200 are not limited to real devices and may be virtual devices.
  • each process mentioned later may be implement
  • the computer 10 is a computer device capable of data communication with the user terminal 100 and the IoT device 200.
  • the user terminal 100 is a terminal device operated by a user capable of data communication with the computer 10 and the IoT device 200.
  • the user terminal 100 is, for example, a cellular phone, a portable information terminal, a tablet terminal, a personal computer, an electric appliance such as a netbook terminal, a slate terminal, an electronic book terminal, a portable music player, a smart glass, a head mounted display, or the like Wearable terminals and other items.
  • the IoT device 200 is a device capable of data communication with the computer 10 and the user terminal 100.
  • the IoT device 200 is, for example, an electric appliance such as an air conditioner, a television, and a refrigerator, an electronic device such as a monitor, a tablet, a printer, and a camera, and various items such as clothing and daily necessities.
  • the user terminal 100 acquires a captured image of the IoT device 200 (step S01).
  • the user terminal 100 acquires a captured image of the IoT device 200 by, for example, capturing all or a part of the IoT device 200 with an imaging device such as a camera that the user terminal 100 has.
  • the user terminal 100 connects an imaging device such as a camera by wire or wirelessly, the connected imaging device images the IoT device 200, and acquires a captured image of the captured IoT device 200.
  • the user terminal 100 transmits captured image data that is acquired captured image data to the computer 10 (step S02).
  • an imaging device connected to the computer 10 via a public network or the like may transmit the captured image data to the computer 10.
  • the computer 10 receives captured image data.
  • the computer 10 analyzes the captured image based on the captured image data (step S03).
  • the computer 10 extracts feature quantities of the IoT device 200 included in the captured image, for example, by executing pattern recognition and edge extraction.
  • the computer 10 extracts, for example, the shape, the model, the position of the structure, and the like as the feature amount of the IoT device 200.
  • step S03 may be executed by the user terminal 100 instead of the computer 10.
  • the user terminal 100 may extract the feature amount of the IoT device 200.
  • step S04 may be executed by the user terminal 100 instead of the computer 10.
  • the user terminal 100 may store the device database described above and specify device data.
  • step S05 may be executed by the user terminal 100 instead of the computer 10.
  • the user terminal 100 stores the above-described user interface database and the like, and specifies the user interface.
  • the computer 10 creates a URL (Uniform Resource Locator) for accessing the specified user interface (step S06).
  • the computer 10 provides a user interface to the user terminal 100 as a Web page.
  • the computer 10 may create a URL for accessing a part of a preset user interface, or may create a URL for accessing all user interfaces. Note that the computer 10 may create a URL in the same manner even when the above-described general-purpose user interface is specified.
  • the computer 10 transmits the created URL to the user terminal 100 (step S07).
  • the user terminal 100 receives the URL and displays the user interface of the IoT device 200 specified by the computer 10 based on the URL (step S08). Note that the user terminal 100 may display all user interfaces or may display some preset user interfaces.
  • the user terminal 100 may receive an input to the displayed user interface, and the computer 10 may transmit the input information to the specified IoT device 200 via a public line network or the like. Further, the user terminal 100 receives an input to the displayed user interface, the computer 10 transmits the input information to the user terminal 100, and the user terminal 100 uses the short-range radio to transmit this information to the IoT. You may transmit to the apparatus 200.
  • FIG. 1 A block diagram illustrating an a public line network or the like.
  • the computer 10 is the above-described computer device having the functions described below.
  • the user terminal 100 is the above-described terminal device having the functions described later.
  • the IoT device 200 is the above-described device having the functions described later.
  • FIG. 3 is a functional block diagram of the computer 10, the user terminal 100, and the IoT device 200.
  • the computer 10 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc. as the control unit 11, and a device for enabling communication with other devices as the communication unit 12.
  • a WiFi (Wireless Fidelity) compatible device compliant with IEEE 802.11 is provided.
  • the computer 10 also includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card as the storage unit 13.
  • the storage unit 13 stores a device database and a user interface database, which will be described later.
  • the computer 10 includes various image analysis devices as the analysis unit 14.
  • control unit 11 reads a predetermined program, thereby realizing the data transmission / reception module 20 in cooperation with the communication unit 12.
  • control unit 11 reads a predetermined program, thereby realizing the data storage module 30 in cooperation with the storage unit 13.
  • control unit 11 reads a predetermined program, thereby realizing an analysis module 40, a device identification module 41, a user interface identification module 42, and a URL creation module 43 in cooperation with the analysis unit 14.
  • the user terminal 100 includes a CPU, RAM, ROM, and the like as the control unit 110, and a WiFi compatible device and the like for enabling communication with other devices as the communication unit 120.
  • the user terminal 100 includes, as the input / output unit 140, a display unit that outputs and displays data and images controlled by the control unit 110, an input unit such as a touch panel, a keyboard, and a mouse that receives input from the user, An imaging device such as a lens is provided.
  • the control unit 110 when the control unit 110 reads a predetermined program, the data transmission / reception module 150 is realized in cooperation with the communication unit 120. In the user terminal 100, the control unit 110 reads a predetermined program, thereby realizing the imaging module 170, the display module 171, and the input receiving module 172 in cooperation with the input / output unit 140.
  • the IoT device 200 includes a CPU, RAM, ROM, and the like as the control unit 210, and a WiFi compatible device and the like for enabling communication with other devices as the communication unit 220.
  • the IoT device 200 includes various devices that perform various functions included in the IoT device 200 as the driving unit 240.
  • IoT device user interface display processing The IoT device user interface display process executed by the IoT device user interface display system 1 will be described with reference to FIGS. 4 and 5.
  • 4 and 5 are flowcharts of IoT device user interface display processing executed by the computer 10, the user terminal 100, and the IoT device 200.
  • FIG. The processing executed by the modules of each device described above will be described together with this processing.
  • the imaging module 170 images the IoT device 200 (step S10).
  • step S10 for example, the imaging module 170 images all or a part of the IoT device 200.
  • the imaging module 170 captures a captured image such as a moving image or a still image that can specify device data such as the shape, type, and type of the IoT device 200, for example.
  • the user terminal 100 acquires a captured image of the IoT device 200.
  • step S10 instead of the imaging module 170, an external device such as a camera connected to the user terminal 100 by short-range wireless or wired, or an external device connected to the computer 10 via the public network 5 or the like.
  • the IoT device 200 may be imaged.
  • the external device transmits the captured image to the user terminal 100 or the computer 10, whereby the user terminal 100 or the computer 10 acquires the captured image of the IoT device 200.
  • the data transmission / reception module 150 transmits captured image data, which is acquired captured image data, to the computer 10 (step S11).
  • the data transmission / reception module 20 receives captured image data.
  • the analysis module 40 performs image analysis of the captured image based on the captured image data (step S12).
  • the analysis module 40 performs image analysis of the captured image by executing various types of analysis, recognition, and processing such as pattern recognition, color tone recognition, dimension measurement, and edge extraction, for example.
  • the analysis module 40 extracts, for example, the shape, model, position of the structure, etc. of the IoT device 200 as the feature amount.
  • step S12 may be executed by the user terminal 100 instead of the computer 10.
  • the user terminal 100 may analyze the captured image and extract the above-described feature amount.
  • the device identification module 41 identifies the IoT device 200 based on the extracted feature amount (step S13). In step S13, the device identification module 41 compares the extracted feature value with the feature value of the IoT device 200 stored in the device database stored in the data storage module 30 based on the extracted feature value. The IoT device 200 to be identified is specified.
  • step S13 may be executed by the user terminal 100 instead of the computer 10.
  • the user terminal 100 may specify the IoT device 200 corresponding to the extracted feature amount by storing the device database or acquiring information corresponding to the device database from the computer 10 or the like.
  • the device database is a database in which the feature amount of the IoT device 200 is associated with device data such as the name, manufacturer, and model of the IoT device 200.
  • the device database stores, for example, the shape, color, and shape of the structure of the IoT device 200 as feature amounts.
  • the device specifying module 41 compares the extracted feature amount of the IoT device 200 with the feature amount stored in the device database, and specifies device data associated with the matching feature amount of the IoT device 200. Thus, the IoT device 200 is specified.
  • the user interface specifying module 42 specifies the user interface of the specified IoT device 200 (step S14).
  • the user interface is a screen configuration that accepts an instruction input to a position such as an icon or a character on the screen.
  • the user interface specifying module 42 compares the device data of the IoT device 200 stored in the user interface database stored in the data storage module 30 based on the device data of the specified IoT device 200, A user interface corresponding to the device data of the specified IoT device 200 is specified.
  • the user interface is, for example, an operation screen, a setting screen, a manual, a device state, or the like.
  • the user interface database is a database in which device data of the IoT device 200 is associated with a user interface of the IoT device 200.
  • the user interface database stores, for example, a manufacturer and model as device data.
  • the user interface specifying module 42 compares the device data of the specified IoT device 200 with the device data stored in the user interface database, and specifies the user interface associated with the matching device data.
  • the user interface of the IoT device 200 is specified.
  • the user interface specifying module 42 determines whether the user interface has been specified (step S15). In step S15, the user interface specifying module 42 determines whether or not the user interface of the specified IoT device 200 has been specified in the process of step S14 described above.
  • the user interface identification module 42 can identify the device interface based on, for example, whether the device data can be identified, whether the user interface corresponding to the device data has been identified, whether the device data has not been identified, or the like. It is judged whether it was able to be done.
  • step S15 when the user interface specifying module 42 determines that the user interface could not be specified (NO in step S15), the user interface specifying module 42 responds to the type of the IoT device 200 based on the device data specified in step S13 described above.
  • a general-purpose user interface that is a general-purpose user interface is acquired (step S16).
  • step S16 the user interface specifying module 42 specifies and acquires a general-purpose user interface based on the type and device name of the IoT device 200 in the device data.
  • step S15 if it is determined in step S15 that the user interface identification module 42 has identified the user interface (YES in step S15), the user interface identification module 42 acquires a user interface corresponding to the identified IoT device 200 (step S17).
  • step S14 to step S17 may be executed by the user terminal 100 instead of the computer 10.
  • the user terminal 100 stores the user interface database or acquires information corresponding to the user interface database from the computer 10 or the like, thereby specifying and acquiring the user interface or general-purpose user interface corresponding to the specified device data. do it.
  • the URL creation module 43 creates a URL for the user terminal 100 to access the identified user interface (step S18).
  • the URL creation module 43 creates a URL for accessing part or all of the acquired user interface.
  • the URL creation module 43 may create a URL for accessing a part of the acquired user interface based on a part of the preset user interface, a part of the user interface specified by the user terminal 100, or the like. Alternatively, a URL for accessing the entire acquired user interface may be created. By doing so, the computer 10 can provide the user interface 100 to the user terminal 100 as a Web page.
  • the URL is created regardless of whether it is a user interface or a general-purpose user interface.
  • the computer 10 may transmit the acquired user interface data as user interface data to the user terminal 100 instead of creating a URL.
  • the data transmission / reception module 20 transmits the created URL as URL data to the user terminal 100 (step S19).
  • the data transmission / reception module 150 receives URL data.
  • the display module 171 displays the URL based on the URL data (step S20).
  • the input receiving module 172 receives a selection input to the URL from the user, accesses the computer 10, and displays the user interface as a Web page (step S21).
  • FIG. 6 is a diagram illustrating an example of a user interface corresponding to the IoT device 200 displayed by the display module 171.
  • the IoT device 200 will be described as an air conditioner.
  • the display module 171 displays a model name display icon 410, a manual selection icon 420, a setting selection icon 430, an operation status selection icon 440, and a remote control selection icon 450 as the user interface screen 400.
  • the model name display icon 410 displays the type and model of the IoT device 200.
  • the instruction selection icon 420 receives a selection input from the user and displays an instruction of the IoT device 200.
  • the setting selection icon 430 receives a selection input from the user and displays a setting screen of the IoT device 200.
  • the driving status selection icon 440 displays a voice input icon 460, accepts a selection input or voice input from the user, and displays a driving status screen of the IoT device 200. By selecting the voice input icon 460, the user can execute the operation of the driving situation selection icon 440 by voice.
  • the remote control selection icon 450 displays a voice input icon 460, accepts a selection input or voice input from the user, and displays a remote control screen of the IoT device 200. By selecting the voice input icon 460, the user can perform an operation of the remote control selection icon 450 by voice.
  • the selection input to each icon mentioned above may accept not only the selection input but also the voice input.
  • only a part of the icons described above may be displayed by the user setting in advance.
  • you may display icons and description other than each icon mentioned above.
  • the display mode, display position, and the like of each icon described above can be changed as appropriate, and may be anything corresponding to the device data of the IoT device 200.
  • FIG. 7 is a diagram showing an example of a user interface showing a state where the user has selected the remote control selection icon 450 displayed by the display module 171.
  • the display module 171 displays a model name display icon 510, an operation content display icon 520, a cooling icon 530, a dehumidification icon 540, a heating icon 550, a transmission icon 560, and a stop icon 570 as the user interface screen 500.
  • the model name display icon 510 displays the type, model, and current screen content of the IoT device 200.
  • the operation content display icon 520 displays the type of operation, the set temperature, the up / down direction of the wind direction, the action of the wind direction, and the air volume.
  • the operation content display icon 520 receives a selection input from the user and changes the operation content. For example, a selection input for a change in set temperature, a change in wind direction, a change in air volume, or the like is received, and the operation content is changed.
  • the cooling icon 530, the dehumidification icon 540, and the heating icon 550 accept selection input from the user, and change the operation content according to the selected icon.
  • the transmission icon 560 receives a selection input from the user, and the computer 10 transmits an instruction based on information input to the user terminal 100 or the IoT device 200.
  • the stop icon 570 receives a selection input from the user, and the computer 10 transmits an instruction to stop the operation of the user terminal 100 or the IoT device 200.
  • the user interface screen 500 demonstrated in FIG. 7 is not restricted to the screen mentioned above, It can change suitably.
  • the selection input to each icon mentioned above may accept not only the selection input but also the voice input.
  • only a part of the icons described above may be displayed by the user setting in advance.
  • the display mode, display position, and the like of each icon to be displayed can be changed as appropriate, and any icon corresponding to the IoT device 200 may be used.
  • FIG. 8 is a diagram illustrating an example of a general-purpose user interface corresponding to the type of the IoT device 200 displayed by the display module 171.
  • the IoT device 200 will be described as an air conditioner.
  • the display module 171 displays a type display icon 610, a setting selection icon 620, an operation status selection icon 630, a remote control selection icon 640, and a model designation icon 650 as a general-purpose user interface screen 600.
  • the type display icon 610 displays the type of the IoT device 200 and that the model name is unknown.
  • the setting selection icon 620 receives a selection input from the user and displays a general-purpose setting screen corresponding to the type of the IoT device 200.
  • the driving status selection icon 630 displays a voice input icon 660, accepts a selection input or voice input from the user, and displays a general-purpose driving status screen corresponding to the type of the IoT device 200.
  • the remote control selection icon 640 displays a voice input icon 660, accepts a selection input or voice input from the user, and displays a general-purpose remote control screen corresponding to the type of the IoT device 200.
  • the model designation icon 650 receives a selection input or voice input from the user, and displays a screen for accepting input of specific information that can uniquely identify the IoT device 200 such as a model name of the IoT device 200 as a model designation screen.
  • the user inputs specific information on the model designation screen, and the computer 10 specifies the IoT device 200 based on the specific information, specifies the user interface of the specified IoT device 200, and uses this user interface. Based on this, the processing from step S17 onward is executed. That is, on the general-purpose user interface screen 600, the user can acquire a desired user interface of the IoT device 200, and the operation of the IoT device 200 becomes easy.
  • the selection input to each icon mentioned above may accept not only the selection input but also the voice input.
  • only a part of the icons described above may be displayed by the user setting in advance.
  • you may display icons and description other than each icon mentioned above.
  • the display mode, display position, and the like of each icon described above can be changed as appropriate, and may be anything that corresponds to the type of the IoT device 200.
  • FIG. 9 is a diagram showing an example of a general-purpose user interface showing a state where the user has selected the remote control selection icon 640 displayed by the display module 171.
  • a general-purpose user interface corresponding to the selected icon is displayed.
  • the display module 171 displays a type display icon 710, an operation content display icon 720, a cooling icon 730, a dehumidification icon 740, a heating icon 750, a transmission icon 760, and a stop icon 770 as a general-purpose user interface screen 700.
  • the type display icon 710 displays the type of the IoT device 200 and the current screen content.
  • the operation content display icon 720 displays the current operation content.
  • the operation content display icon 720 displays the type of operation and the set temperature.
  • the operation content display icon 720 accepts a selection input from the user and changes the operation content. For example, a selection input for changing the set temperature or the like is received, and the operation content is changed.
  • the cooling icon 730, the dehumidification icon 740, and the heating icon 750 receive selection input from the user, and change the operation content according to the selected icon.
  • the transmission icon 760 receives a selection input from the user, and the computer 10 transmits an instruction based on information input to the user terminal 100 or the IoT device 200.
  • the stop icon 770 receives a selection input from the user, and the computer 10 transmits an instruction to stop the operation of the user terminal 100 or the IoT device 200.
  • the general-purpose user interface screen 700 described in FIG. 9 is not limited to the screen described above, and can be changed as appropriate.
  • the selection input to each icon mentioned above may accept not only the selection input but also the voice input.
  • only a part of the icons described above may be displayed by the user setting in advance.
  • the display mode, display position, and the like of each icon to be displayed can be changed as appropriate, and may be anything corresponding to the type of the IoT device 200.
  • the input receiving module 172 determines whether or not an input to the displayed user interface has been received (step S22). In step S22, the input reception module 172 makes a determination based on whether or not the input of the transmission icons 560 and 760 or the stop icons 570 and 770 described above has been received. Note that the input reception module 172 may determine whether other input has been received.
  • step S22 when the input reception module 172 determines that the input is not received (NO in step S22), the process is repeated. In step S22, when the input reception module 172 receives an input for ending display of the user interface, the process ends.
  • step S22 when the input reception module 172 determines that the input has been received (step S22: YES), the data transmission / reception module 20 transmits an instruction to the user terminal 100 based on the input information (step S23).
  • step S23 the data transmission / reception module 20 may directly transmit an instruction to the IoT device 200.
  • the data transmission / reception module 20 transmits an instruction to the IoT device 200 via the public line network 5 or the like.
  • the data transmission / reception module 150 receives the instruction transmitted by the computer 10.
  • the data transmission / reception module 150 transmits this instruction to the IoT device 200 (step S24).
  • step S ⁇ b> 24 the data transmission / reception module 150 transmits this instruction to the IoT device 200 using the short-range wireless.
  • the data transmission / reception module 150 may transmit the instruction to the IoT device 200 by another communication method such as a wired connection.
  • the data transmission / reception module 250 receives the instruction.
  • the drive module 270 performs a behavior based on the received instruction (step S25).
  • the drive module 270 executes various behaviors such as, for example, setting the temperature of the air conditioner, adjusting the wind direction and the air volume, and stopping the operation of the air conditioner.
  • the drive module 270 is not limited to the example described above, and may execute other behaviors.
  • the above is the IoT device user interface display processing.
  • the means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program.
  • the program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.).
  • the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it.
  • the program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.
  • IoT device user interface display system 10 computers, 100 user terminals, 200 IoT devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Le problème décrit par la présente invention est de fournir un système d'affichage d'interface utilisateur de dispositif IdO capable d'afficher l'interface utilisateur d'un dispositif IdO sur un terminal utilisateur, un procédé d'affichage d'interface utilisateur de dispositif IdO, et un programme. [Solution] Un système d'affichage d'interface utilisateur de dispositif IdO (1) pour afficher l'interface utilisateur d'un dispositif IdO (200) sur un terminal utilisateur (100) obtient une image capturée du dispositif IdO (200), laquelle a été capturée par une caméra, analyse l'image capturée obtenue, et sur la base des résultats d'analyse d'image, identifie le dispositif IdO (200), identifie l'interface utilisateur du dispositif IdO (200) identifié, et affiche l'interface utilisateur identifiée sur le terminal utilisateur (100).
PCT/JP2016/073856 2016-08-15 2016-08-15 SYSTÈME D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO, PROCÉDÉ D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO ET PROGRAMME WO2018033953A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/073856 WO2018033953A1 (fr) 2016-08-15 2016-08-15 SYSTÈME D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO, PROCÉDÉ D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO ET PROGRAMME

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/073856 WO2018033953A1 (fr) 2016-08-15 2016-08-15 SYSTÈME D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO, PROCÉDÉ D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO ET PROGRAMME

Publications (1)

Publication Number Publication Date
WO2018033953A1 true WO2018033953A1 (fr) 2018-02-22

Family

ID=61197178

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/073856 WO2018033953A1 (fr) 2016-08-15 2016-08-15 SYSTÈME D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO, PROCÉDÉ D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO ET PROGRAMME

Country Status (1)

Country Link
WO (1) WO2018033953A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019184779A (ja) * 2018-04-09 2019-10-24 シャープ株式会社 端末装置、情報処理装置、表示システム、情報表示方法、情報処理方法、制御プログラム、及び電気機器
WO2020085078A1 (fr) * 2018-10-24 2020-04-30 オムロン株式会社 Dispositif de commande
US20220067669A1 (en) * 2020-09-01 2022-03-03 International Business Machines Corporation Predictive device maintenance

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005065118A (ja) * 2003-08-19 2005-03-10 Hitachi Ltd リモコン機能付き携帯端末及びリモコンサーバ
JP2007318380A (ja) * 2006-05-25 2007-12-06 Pioneer Electronic Corp リモートコントローラ
WO2011086956A1 (fr) * 2010-01-18 2011-07-21 日本電気株式会社 Dispositif de terminal d'informations, procédé de commande mettant en œuvre ledit procédé et programme
JP2016144065A (ja) * 2015-02-03 2016-08-08 シャープ株式会社 制御端末装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005065118A (ja) * 2003-08-19 2005-03-10 Hitachi Ltd リモコン機能付き携帯端末及びリモコンサーバ
JP2007318380A (ja) * 2006-05-25 2007-12-06 Pioneer Electronic Corp リモートコントローラ
WO2011086956A1 (fr) * 2010-01-18 2011-07-21 日本電気株式会社 Dispositif de terminal d'informations, procédé de commande mettant en œuvre ledit procédé et programme
JP2016144065A (ja) * 2015-02-03 2016-08-08 シャープ株式会社 制御端末装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019184779A (ja) * 2018-04-09 2019-10-24 シャープ株式会社 端末装置、情報処理装置、表示システム、情報表示方法、情報処理方法、制御プログラム、及び電気機器
WO2020085078A1 (fr) * 2018-10-24 2020-04-30 オムロン株式会社 Dispositif de commande
US20220067669A1 (en) * 2020-09-01 2022-03-03 International Business Machines Corporation Predictive device maintenance

Similar Documents

Publication Publication Date Title
EP3345379B1 (fr) Procèdè pour la commande d'un objet par un dispositif èlectronique et dispositif èlectronique
JP6267838B1 (ja) コンピュータシステム、エッジデバイス制御方法及びプログラム
WO2011086956A1 (fr) Dispositif de terminal d'informations, procédé de commande mettant en œuvre ledit procédé et programme
CN110800314B (zh) 计算机系统、远程操作通知方法以及记录介质
KR20120019699A (ko) 통신 연결 방법 및 장치
US9196154B2 (en) Method and electronic device for controlling display device using watermark
US11132398B2 (en) Electronic device for generating video comprising character and method thereof
WO2018033953A1 (fr) SYSTÈME D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO, PROCÉDÉ D'AFFICHAGE D'INTERFACE UTILISATEUR DE DISPOSITIF IdO ET PROGRAMME
CN104239842B (zh) 一种实现视觉识别的方法、装置和系统
CN105683959A (zh) 信息处理设备、信息处理方法及信息处理系统
JP2014068206A (ja) 情報処理端末、情報処理方法、及び機器制御システム
JP6267840B1 (ja) コンピュータシステム、エッジデバイス制御方法及びプログラム
JP5997824B1 (ja) 遠隔端末、遠隔指示方法及び遠隔端末用プログラム
WO2017208356A1 (fr) Système de commande d'iot, procédé de commande d'iot, et programme
KR102172588B1 (ko) 전시관 관람객을 위한 동영상 앨범 제작 방법 및 시스템
JP6404526B2 (ja) 撮像画像共有システム、撮像画像共有方法及びプログラム
JP6274742B2 (ja) 電子機器設定システム、電子機器設定方法、電子機器設定システム用プログラム
JP6195960B2 (ja) 遠隔指示方法及び遠隔端末用プログラム
JP2017130079A (ja) 被撮像物情報提供システム、被撮像物情報提供方法、および被撮像物情報提供プログラム
JP6404536B1 (ja) コンピュータシステム、設定方法及びプログラム
JP6668519B1 (ja) リモコン機能設定プログラム、リモコンサーバ、リモコン機能設定方法、携帯端末
US20220276822A1 (en) Information processing apparatus and information processing method
JP6310624B1 (ja) コンピュータシステム、設定方法及びプログラム
CN113919457A (zh) 一种处理方法及相关设备
KR101488724B1 (ko) 전자태그를 이용하여 애플리케이션을 실행하는 방법 및 그를 위한 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16913483

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC ( EPO FORM 1205A DATED 27/05/2019 )

NENP Non-entry into the national phase

Ref country code: JP

122 Ep: pct application non-entry in european phase

Ref document number: 16913483

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载