+

WO2018108176A1 - Procédé, dispositif et terminal de commande d'appel vidéo de robot - Google Patents

Procédé, dispositif et terminal de commande d'appel vidéo de robot Download PDF

Info

Publication number
WO2018108176A1
WO2018108176A1 PCT/CN2017/116674 CN2017116674W WO2018108176A1 WO 2018108176 A1 WO2018108176 A1 WO 2018108176A1 CN 2017116674 W CN2017116674 W CN 2017116674W WO 2018108176 A1 WO2018108176 A1 WO 2018108176A1
Authority
WO
WIPO (PCT)
Prior art keywords
target object
target
robot
feature information
information
Prior art date
Application number
PCT/CN2017/116674
Other languages
English (en)
Chinese (zh)
Inventor
何坚强
Original Assignee
北京奇虎科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京奇虎科技有限公司 filed Critical 北京奇虎科技有限公司
Publication of WO2018108176A1 publication Critical patent/WO2018108176A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J11/00—Manipulators not otherwise provided for
    • B25J11/0005—Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions

Definitions

  • the present invention relates to the field of automatic control technologies, and in particular, to a robot video call control method, apparatus and terminal.
  • intelligent robots as one of the smart products, can replace or assist humans in some work, and are often used in all walks of life.
  • intelligent robots have gradually replaced thousands of households to deal with daily household chores. Although such robots have simple automatic control and movement functions, they cannot meet modern needs.
  • the present invention provides a robot video call control method and a corresponding device thereof.
  • a robot video call control method of the present invention includes the following steps: establishing a video call with a calling party, and transmitting a video stream obtained by the local camera unit to the calling party; Receiving the instruction of the calling by the calling party, parsing the target information included in the seeking instruction, determining the target feature information of the corresponding target object according to the target information; and starting the walking when the target object is not captured
  • the device performs local movement, performs image recognition on the video stream of the camera unit during the movement, determines an image containing the target feature information to capture the target object, and controls the walking device to make the local device after capturing the target object Maintain a preset distance range from the target object.
  • the present invention also provides a robot video call control apparatus comprising: at least one processor; and at least one memory communicably coupled to the at least one processor; the at least one memory including processing
  • the walking device is activated to perform local movement, and the video stream of the camera unit is image-recognized during the movement, and the image containing the target feature information is determined to capture the target object; when the target object is captured, the control is performed.
  • the walking device maintains a preset distance range between the unit and the target object.
  • the present invention also provides a video callable mobile robot comprising a processor for running a program to perform the steps of the robot video call control method.
  • the present invention also provides a computer program comprising computer readable code that causes the robotic video call control method to be executed when the video callable mobile robot runs the computer readable code.
  • the invention provides a computer readable medium storing the computer program of the fourth aspect.
  • the present invention has the following beneficial effects:
  • the method, device and terminal for controlling a video call of a robot provided by the present invention establish and initiate remote control through pre-stored images, relationship of characters, and contact information with a person to realize remote connection, and when remote control is transmitted remotely by a character.
  • the image of the target object stored in advance is used as the feature information for identification, and after the target object is captured, the extended feature information of the target object is collected, so as to facilitate A fast capture of the target object can be achieved by extending the feature information when subsequently capturing the target object.
  • the voice reminder is provided in the invention to ensure that the child receives the video call initiated by the parent in time, which helps the parent to monitor the child's activity anytime and anywhere.
  • the process of capturing the target object mainly adopted
  • the method involves voice and/or infrared positioning, so that the robot can locate the child's position more quickly while searching for children, greatly improving the time and accuracy of the person recognition, after capturing the child and with the child.
  • the distance measuring device is kept within a preset distance from the child, thereby ensuring the child's safe and clear reception of the child information and the maximum range of capturing the child's state.
  • the invention integrates the entertainment learning function of video communication, mobile and human-computer interaction, and any character stored in the robot database can initiate an instruction to start the robot, by receiving and parsing the instruction, completing and/or starting the instruction involved in the instruction.
  • Tasks and functions, the robot provided by the present invention can also provide children with human-computer interaction activities matching their age and/or IQ through observation and learning, so that the machine can achieve the maximum possible development of children's intelligence in the process of accompanying children. To reduce the child's loneliness, it is more practical in real life.
  • FIG. 1 is a flowchart of a method for controlling a video call of a robot according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a method for controlling a video call of a robot according to another embodiment of the present invention
  • FIG. 3 is a flowchart of a method for controlling a video call of a robot according to still another embodiment of the present invention.
  • FIG. 4 is a flowchart of a method for controlling a video call of a robot according to still another embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a video call control apparatus for a robot according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a sub-frame of a robot video call control device according to another embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a method for controlling a video call of a robot according to still another embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a method for controlling a video call of a robot according to still another embodiment of the present invention.
  • Figure 9 is a block diagram of a video callable mobile robot performing the method according to the present invention.
  • Figure 10 is a schematic diagram of a memory unit for holding or carrying program code implementing a method in accordance with the present invention.
  • terminal and terminal device used herein include both a wireless signal receiver device and a device having only a wireless signal receiver without a transmitting capability.
  • a device comprising receiving and transmitting hardware having a receiving and transmitting hardware capable of two-way communication over a two-way communication link.
  • Such devices may include cellular or other communication devices having a single line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which may combine voice, data Processing, fax, and/or data communication capabilities; PDA (Personal Digital Assistant), which can include radio frequency receivers, pagers, Internet/Intranet access, web browsers, notepads, calendars, and/or GPS (Global Positioning System (Global Positioning System) receiver; conventional laptop and/or palmtop computer or other device having a conventional laptop and/or palmtop computer or other device that includes and/or includes a radio frequency receiver.
  • PCS Personal Communications Service
  • PDA Personal Digital Assistant
  • terminal may be portable, transportable, installed in a vehicle (aviation, sea and/or land), or adapted and/or configured to operate locally, and/or Run in any other location on the Earth and/or space in a distributed form.
  • the "terminal” and “terminal device” used herein may also be a communication terminal, an internet terminal, a music/video playing terminal, and may be, for example, a PDA, a MID 10 (Mobile Internet Device), and/or have a music/video playing function.
  • Mobile phones can also be smart TVs, set-top boxes and other devices.
  • the robot involved in the present invention can understand the human language, talks with the operator in human language, and forms an external environment in which it can "survive” in its own “consciousness” through programming. It analyzes what is happening and adjusts its movements to meet the operator's requirements for robots. Robots can be programmed to make their intelligence reach the level of children. Robots can walk alone, “seeing” things and analyzing what they see, obeying instructions and answering questions in human language. More importantly, it has the ability to "understand”.
  • the robot video call control method enables a family member to view the situation of the home at any time and anywhere in the home of the guardian through the communication terminal, and the child at home is thinking about the parents and/or other at home.
  • members are members, they can also send timely communication to them through the robot end.
  • the robot can update the stored extended feature information in time by monitoring the change of the extended features of the extended part of the target object.
  • the robot can receive the message in time and transmit the video image of the situation at home, the robot of the present invention has Human-computer interaction function can also play, answer questions and help learn The role.
  • the robot video call control method disclosed in the following embodiments, as shown in FIG. 1, includes the following steps:
  • S100 Establish a video call with the calling party, and transmit the video stream obtained by the local camera unit to the calling party;
  • the information stored in the robot records the relationship between the robot and the family member, and establishes a connection mode with the direct communication terminal between the family members.
  • the calling party in step S100 is a family member, for example, each mobile terminal of the family member, such as a mobile phone. Computers, ipads, etc.
  • the application can be an app that controls the robot directly related to the robot, or a web link that controls the robot, in order to perform real-time monitoring of the role of the child, the family
  • the member establishes and activates the video communication transmission function of the robot in advance, directly establishes a video call with the communication terminal of the family member, or establishes a video call with the family member when receiving the transmission instruction of the family member, and real-time Transmitting a video image to a terminal of a family member, or directly receiving a call from a family member who controls the bot's App application or entering a web application that controls the machine, and issues a video call request, directly accepts the family member's Video call request and to The family member who is the video call, that is, the calling party transmits the video stream acquired by the local machine, wherein the past video stream transmitted by the robot is also displayed on the calling mobile terminal of the calling party where the controlling robot is installed, or the webpage opened by the calling mobile terminal. Displayed on the app,
  • S200 receiving an instruction of the calling party initiated by the calling party, parsing object information included in the seeking instruction, and determining target feature information of the corresponding target object according to the target information;
  • the robot accepts and sends back the video stream to the caller.
  • the caller can directly observe the situation at home through the video. If the caller cannot see the target object in the video, the caller can be on his own side.
  • the mobile terminal issues a seek instruction, and the search instruction includes information about the target object, and the information is stored in the robot or in the cloud connected to the robot. After the robot receives the seek instruction, the robot is in the local machine.
  • the target object information included in the searched instruction is parsed, and the target feature information of the target object corresponding to the target object is determined according to the target object information, and the robot searches and determines the target object based on the basis in the subsequent process.
  • the mother initiates a search for “looking for a daughter” to the robot at the mobile terminal, and the robot receives the “seeking daughter” search command and parses the information in the seek instruction in the local machine, that is, parses and extracts “ "Daughter” message, sending the message "daughter” to the store "daughter” a database of target information and corresponding feature information, and determining the feature information of the daughter by storing the target information of the daughter, wherein the storage relationship between the specific target feature information and the target information is detailed later, and the feature information is the daughter's Facial features, such as the contour and position of the entire face and facial features, will be used by the robot to find and determine the daughter based on this.
  • Another embodiment of the present step is that after the robot receives the instruction of seeking, the robot sends the searched instruction to the cloud, and the cloud parses the target information contained in the searched instruction, and sends the target information to the robot, and the robot according to the target.
  • the object information determines the target feature information of the target object corresponding to the target object, and the robot will find and determine the target object based on the basis in the subsequent process.
  • the above-mentioned mother initiates a search for “looking for a daughter” to the robot at the mobile terminal, and after receiving the instruction of “seeking the daughter”, the robot sends the instruction of “seeking the daughter” to the cloud, and the cloud analyzes the search.
  • the information in the instruction that is, parsing and extracting the information of "daughter”, and sending the information to the robot, the robot receives the parsed information, and sends the message "daughter" to the target information storing "daughter”
  • a database corresponding to the characteristic information, and determining the characteristic information of the daughter by storing the target information of the daughter the robot will find and determine the daughter based on the basis in the subsequent process.
  • Another embodiment of the present step is that after the robot receives the seek instruction, the robot sends the search command to the cloud, and the cloud parses the target information contained in the search command, and the cloud sends the target information to the storage target.
  • the cloud and the target feature information are in the cloud database, and the target feature information of the target object corresponding to the target object is determined according to the target information in the cloud, and the cloud sends the target feature information to the robot, and the robot will use the target in the subsequent process. Based on finding and determining the target object.
  • the above-mentioned mother initiates a search for “looking for a daughter” to the robot at the mobile terminal, and after receiving the instruction of “seeking the daughter”, the robot sends the instruction of “seeking the daughter” to the cloud, and the cloud analyzes the search.
  • the information in the instruction that is, parsing and extracting the information of "daughter”, determining the characteristic information of the daughter through the information, and transmitting the characteristic information to the robot, and the robot receives the characteristic information directly receiving the daughter, and based thereon Find and identify your daughter.
  • S300 when the target object is not captured, start the walking device to perform local movement, perform image recognition on the video stream of the imaging unit during the moving process, and determine an image including the target feature information to capture the target object;
  • step S200 After the feature information of the target object and the target object is determined in step S200, if it has been acquired In the video stream, image recognition is performed without image with target feature information, and the robot activates its own walking device to move the robot. During the moving process, the robot passes the acquired video stream of the camera unit and passes image recognition technology. Identifying the image in the video, determining whether the image contains the image of the target feature information, and capturing the target object, wherein the uncaptured object includes the following situations: 1. The robot does not recognize the image in the video stream.
  • the walking device is activated; The robot captures the target feature of the target object, and the distance from the target object remains within the preset range. If the target object suddenly moves away from the target feature that causes the robot to fail to recognize the target object, the walking device is activated, if the target object is far away and Before the robot locates the position of the target object, the robot finds that the target object is gradually approaching by ranging, and does not activate the walking device.
  • the robotic walking device receives the signal from the camera unit and converts it into an electrical signal of a controller that controls and electrically connects the traveling device, and the controller rotates the electrical signal to the driving device that activates the traveling device, and the driving device starts
  • the walking device realizes the movement of the robot, wherein the driving device may be a motor, the walking device may be a wheel, a crawler or a wheel and the like; the image recognition is to first store a picture in the robot and use it as a model, the robot processor The model is first preprocessed, and the contour with lines, the angle between the lines and the lines, the relative position of the lines and the lines, the colors covered in the outline, etc. are extracted, after capturing a video image.
  • the robot processor In order to determine whether the target object needs to be captured in the current image, the robot processor sequentially preprocesses each frame of the image in the image, and extracts the contour with lines, the angle between the lines and the lines, and the line and The relative position of the lines, the colors covered in the outline, etc. Comparative fitting, the fitting when it reaches the set value is considered to exist in the video image captured target.
  • the characteristic information of the daughter is determined in step S200, and the robot searches for the daughter at home based on the feature information, and the robot firstly at the position where he receives the instruction to seek.
  • the walking device is activated to cooperate The camera unit captures the target feature information of the daughter.
  • the walking device is activated, and the image is recognized by the video stream of the camera unit to capture the daughter.
  • the target features are the contour features of the daughter's face.
  • S400 After capturing the target object, controlling the walking device to maintain a preset distance range between the local machine and the target object.
  • the robot can preset the distance range between the robot and the target object and is M. After the robot captures the target object, the distance between the robot and the target object is first measured by a measuring device such as an ultrasonic sensor mounted thereon, if M ⁇ L, If the distance between the robot and the target object is far away and is not within the preset distance range, the robot moves to the preset distance range from the target object by the walking device, and the robot always keeps with the target object during the moving process of the target object. If the distance is set, if M ⁇ L, the distance between the robot and the target object is relatively close. At this time, the robot only needs to maintain the preset distance range from the target object.
  • a measuring device such as an ultrasonic sensor mounted thereon
  • step S300 if the mother searches for a daughter, in step S300, if the robot finds the daughter according to the characteristic information of the daughter, wherein the preset distance between the robot and the daughter is 3 m, the robot measures the distance between the robot and the daughter through its own measuring device. When 6m is greater than the preset distance of 3m, the robot moves to the preset distance range with the daughter through the walking device, and the robot always maintains a preset distance of 3m from the target object during the walking of the daughter.
  • S410 After capturing the target object, collecting extended feature information of the target object that belongs to the target feature information, and when the target feature information cannot be captured, positioning the target object according to the extended feature information The amplified part achieves the target object capture.
  • the extended feature information of the amplified part other than the target feature information on the target object is collected by the video, on the calling side.
  • Video call via robot and target object In the process, due to the movement of the target object and/or the movement of the robot, the video of the robot cannot capture the target feature information of the target object, that is, the robot cannot clearly recognize in the partial frame or the continuous frame picture in the current video stream.
  • the target feature corresponds to the outline with lines, the angle between the line and the line, the relative position of the line and the line, the color covered in the outline, etc.
  • the robot can expand through the amplification part other than the target feature information of the target object.
  • the feature information quickly finds and focuses on the target feature information of the target object. After the target object captures the target object, the target feature information of the target object cannot be recognized because the target object suddenly moves away.
  • the robot simultaneously identifies the target feature information and the extended feature information of the target object in the current video stream. If the augmented part of the target object is identified by the extended feature information, the target object is captured according to the augmented part, and the target feature of the target object is further determined based on the target object.
  • the camera captures other extended feature information of the daughter other than the facial feature information, such as the color and style of the daughter's clothes, pants, shoes, and the color of the hair.
  • the robot can not locate and capture the facial feature information of the daughter, and the robot can locate the target feature information and capture the target object through the extended feature information of the extended part.
  • the ratio decolorizes part of the frame or the continuous frame image, processes it as a black and white picture, extracts the outline feature of the body contour, and determines the target object as the extended feature information in the process of capturing the target object.
  • the walking device is started to continue searching for the target feature information around the augmented portion until the target feature information is acquired.
  • the robot When the robot cannot capture the target feature information of the target object, the robot first compares the feature information captured by the camera unit with the previously acquired extended feature information and locates the extended portion captured by the camera unit, and the robot starts the walking device to move around. The amplified portion continues to search for the target feature information until the camera unit captures and locates the target feature information. Continue to capture the target in the video call.
  • the robot can capture the facial features of the daughter. After the daughter stands up, the robot can only capture the body part of the daughter, the daughter. Wearing a pink dress, the style is a round neck, sleeveless tutu. At this time, the robot can determine the captured position by the extended feature information collected before, that is, the outline of the daughter's body, the color and style of the skirt.
  • the camera unit can capture the facial features of the daughter; if the daughter stands up and turns over the body, so that the back of the camera is facing the camera unit of the robot, the camera unit of the robot captures the color of the skirt of the extension of the daughter's torso and The extended feature information of the style and the back profile, then the robot needs to start the walking device Moving around the torso and searching for the target feature information by changing the angle of the camera unit, until the camera unit searches for the target feature information, the robot adjusts the distance with the daughter, and locates the target feature information before continuing to capture the target object, such as a robot.
  • the robot When the target feature information of the daughter is captured, and the distance from the daughter is kept within the preset range, when the outline of the video image of the robot camera unit is unclear, the robot temporarily does not activate the walking device, and after the outline of the video image is clear, the video is passed. After the image recognition in the stream, the recognized extended feature information is derived from the amplified portion of the daughter's back of the body, and the walking device is activated to capture the facial feature information of the daughter around the feature portion.
  • the extended feature information is collected from a moving scene image in which the image portion corresponding to the target feature information in the video stream moves together.
  • the target object belongs to the dynamic image in the video stream relative to other scenes.
  • the extended information is collected, the extended features on the target object in the video stream that move together with the target feature information are collected, such as a robot camera unit.
  • the other line contours in the partial frame or the continuous frame picture change with the line contour of the face and may be surrounded by the face contour in a closed
  • the part in the contour is used as an amplification site, and the contour of the amplified part, the relative position of each contour, and the color of the contour coating are collected after the amplification site is determined.
  • the daughter is relative to the home in the video stream.
  • the static item belongs to the dynamic object.
  • the daughter's torso, clothes, hair, etc. all move together following the daughter's movement.
  • the daughter's object is determined according to the facial feature information of the daughter in the video stream, some frames or consecutive frames in the video stream are collected.
  • the outline of the part of the frame picture that changes with the contour of the other side of the daughter's face, the relative position of each outline, and the color of the outline, and other line outlines can be surrounded with the outline of the face and in a closed outline
  • Whether the position of the part changes relative to the position of other items determines whether the daughter is in a state of motion, and if yes, it is exercise, otherwise the daughter is not in a state of motion.
  • the contour features of the face in the partial or continuous frame image of the video stream of the camera unit are changed relative to other items, and the contour position of the face is changed according to the change of the face feature position.
  • the change is changed with respect to the position of other items, the daughter is determined to be in a moving state, and the contours of other parts can be wrapped in a closed contour with the contour of the face, and the leg is determined according to the position of the contour of the other part relative to the face.
  • the target information and the target feature information are stored in a database in a mapping relationship, and the target feature information is determined according to the target information by querying the database.
  • the target information and the target feature information are all stored in the database in a one-to-one correspondence relationship.
  • the target feature information can be queried through the target information, and the database can be a local database. It can be a cloud database connected to the robot. If it is a local database, after obtaining the target information, the target feature information can be directly determined locally. If it is a cloud database, the robot sends the target information to the cloud, and the cloud determines the target feature information corresponding to the target information. , return the target feature information to the robot.
  • the mother when storing, the mother will store the common name of Xiaohong (including other family members to Xiaohong) and Xiaohong's facial feature information, and also include the collected extension.
  • the feature information as shown in Table 1, is the storage relationship between the target information and the target feature information in the database.
  • the step of controlling the walking device to maintain the preset distance range between the local device and the target object after capturing the target object acquiring the local distance sensor detection target and the target along with the running of the walking device The distance data between the objects, when the distance data exceeds the predetermined distance range, controls the walking device to start moving first, and otherwise controls the walking device to stop walking and pause the movement.
  • the distance sensor After capturing the target object, the distance sensor is always in the measurement state while maintaining the distance between the robot and the target object. The distance between the robot itself and the target object is measured. During the movement of the target object, the robot and the target When the distance between the objects exceeds the preset range, the robot automatically controls the walking device to start moving and moves. If the distance between the robot and the target object is measured by the distance sensor within a preset range, the robot automatically controls the walking device to stop the walking device. The robot pauses to move.
  • the distance sensor is always in the measurement state while the robot's walking device maintains its distance from the small red, and the distance between the robot itself and the target object is measured.
  • the robot automatically controls the walking device to start walking and moves. If the distance between the robot and the red red is measured by the distance sensor, the distance is within a preset range.
  • the robot automatically controls the walking device to stop the walking device to suspend the movement of the robot.
  • the robot automatically controls the walking device to start walking and move, making the robot more intelligent.
  • the target information is a name or an indicator of the target object.
  • the target object information carried in the command is parsed, and the target object information is the name or indicator of the target object, such as the name of the person, the computer, and the like.
  • the above-mentioned mother’s instruction as a calling party at the terminal is “looking for electricity”.
  • "brain” in which "find computer” character information
  • step S200 parses out the computer in "find computer” and uses it as an indicator of the target object, such as a computer as an indicator, that is, target information; for example, the daughter's name is Xiaohong, the instruction issued by the mother is "Looking for Xiaohong", in which "Look for Xiaohong” character information, step S200 parses out the little red in "Look for Xiaohong” and uses it as an indicator of the target object, ie
  • the target information Xiaohong is the name of the daughter's target object, that is, the target information; in addition, the indicator may also be the information stored by the calling party terminal and the target object, and the information of the target object is triggered at the calling party terminal, and the calling party terminal will The information of the target object generates an indicator corresponding to the target object, and send
  • the mother On behalf of Xiaohong’s image, the mother triggers the daughter’s name on the terminal or the image representing Xiaohong, and the caller’s terminal generates a search for Xiaohong. Character shown, and send the indicator to the robot.
  • the target feature information is facial feature information of the target object.
  • the target feature information of the target object is entered in advance, and the facial feature change can best express a person's mood or express the state at the time.
  • the facial feature is preferably used as the target feature information, which is convenient for the parent during the subsequent video call. Or the family members during the video call, or the video and/or photos taken, can first observe the emotions of the children and/or other family members at home through the facial expression.
  • the method further includes the following steps: after detecting that the extended feature information of the target object changes, re-collecting the extended feature information of the extended part.
  • the robot can save the extended feature information of the target object to the next time the extended feature information changes, and after the robot detects that the extended feature information of the stored target object changes, the new extended feature information is re-acquired, so as to expand the feature in the target object.
  • the target object needs to be captured, and the target feature information of the target object can be quickly located and the target object can be captured.
  • the extended feature information stored in the database is also the information of the pink dress.
  • the robot determines the object wearing the white dress by capturing the target feature information. After red, and found that the clothing extension feature information has changed, the extended feature information of the small red body and the white dress is re-acquired.
  • the distance sensor is an ultrasonic sensor, an infrared sensor, or a binocular ranging imaging device including the imaging unit.
  • the distance sensor involved in step S400 is an ultrasonic sensor, an infrared sensor, or a binocular ranging imaging device including the imaging unit, and the binocular ranging imaging device is convenient to use, and can initially determine the distance between the robot and the target object.
  • Ultrasonic has small error in long distance ranging and good effect.
  • the infrared sensor has small distance measurement error and good effect.
  • the invention combines with each other to optimize the distance error of the robot in the distance.
  • the extended feature information includes one or any of a plurality of features: a torso part feature, a clothing part feature, a facial contour feature, a hair outline feature information, or audio feature information.
  • the extended feature information includes one or any of a plurality of features: a torso part feature, a clothing part feature, a face outline feature, a fur outline feature information, or audio feature information.
  • the machine further includes an audio and/or infrared positioning unit, when the target object is captured, the local device turns on the audio and/or infrared positioning unit to acquire the target position to determine the starting direction of the walking device. .
  • an audio and/or infrared positioning unit that acquires the position of the target by turning on the audio and/or infrared positioning unit during capturing of the target object, and thereby determining the walking of the robotic walking device at the beginning direction.
  • the robot After determining that the target object is Xiaohong, Xiaohong laughs at the front of the robot at this time, and the robot acquires the audio of Xiaohong through the audio positioning unit, and locates the position of the red red as the robot. Directly forward, at this time, the robot directly activates the walking device to move the robot forward; if the robot radiates through the infrared light of the infrared positioning unit to sense the surrounding scenery and the infrared light reflected from the environment, the position of the small red is determined to be the right front of the robot. , the robot starts the walking device and moves to the right front to find the red.
  • the local machine measures the distance between the local device and the obstacle through the distance sensor, and controls the walking device to bypass and/or stay away from the obstacle.
  • the target object continues to be captured after being bypassed and/or away from the obstacle.
  • the robot In the process of searching for Xiaohong, the robot will inevitably encounter obstacles, such as stools and walls in the home. It is also possible to measure the distance between the robot and the obstacles in the figure through the distance sensor, and find the orientation of the red. In this case, the walking device is controlled to bypass and/or move away from the stool, the wall, and continue to capture the red.
  • obstacles such as stools and walls in the home. It is also possible to measure the distance between the robot and the obstacles in the figure through the distance sensor, and find the orientation of the red. In this case, the walking device is controlled to bypass and/or move away from the stool, the wall, and continue to capture the red.
  • the local machine further includes a voice reminding unit, when the local machine moves within a distance range from the target object, the voice reminding unit is activated, and a voice reminder is issued.
  • the robot In order to ensure that the target object can receive the message sent by the parent in time when the calling party initiates the video call, the robot prompts the voice reminding unit when the robot captures the target object and moves to a preset distance from the target object. And send a voice reminder.
  • the robot finds Xiaohong and moves to the distance range from Xiaohong, it will send a voice reminder to Xiaohong. For example, if the mother calls, the mother calls, and the mother calls, fast. Answer the call, answer the call quickly, and answer the call quickly.
  • the method further includes the following steps:
  • the robot continuously collects the video of the child's home through the camera unit.
  • the robot does not turn off the camera unit, but continuously collects videos of Xiaohong’s at home, learning, and the like.
  • the robot sends the video to a terminal connected thereto, and sends a text reminder and/or a voice reminder to the terminal.
  • the robot After a video capture is completed, the robot sends the collected video to the terminal and/or cloud of the family member connected to it, and sends a text reminder and/or a voice reminder to the terminal, and after collecting a video, the next video is continuously collected. .
  • the robot plays the video of Xiaohong at home, it is sent to the family member's terminal and/or the cloud, such as a mobile phone, a computer, an ipad, etc., and/or a cloud connected thereto, and the video is successfully transmitted.
  • Send text reminders and/or voice reminders to the terminal of the family member such as: there is a video with Xiaohong playing; if the robot is not connected to the cloud, it will only be sent to the terminal, and if it is sent to the cloud and the terminal, or the terminal is all
  • the status is off, only the cloud is sent, and when any terminal is turned on, an alert message is sent.
  • the robot when the robot acquires the target object, according to a change of the target object facial feature and/or audio feature and/or an interaction instruction issued by the target object, the local machine Initiating a voice interaction unit and/or initiating a video call to a mobile terminal connected to the local machine.
  • the robot changes the facial features of the target object through the camera unit during the process of collecting the target object. If the child is crying, the robot activates the human-computer interaction unit of the machine to make the child happy; according to the change of the audio characteristics of the target object, If the child is angry, the robot determines that the child belongs to the temper state through the audio characteristics, the robot activates the human-computer interaction unit of the machine to comfort the child; the robot sends an instruction according to the target object, such as the child submits flowers to the machine in English. Say, the robot answers according to the question raised by the child, the English of the flower is flower; and if the target child sends a call to the dad to the robot, the robot sends a video call request to the mobile terminal of the father.
  • the robot determines that Xiaohong is crying now by the change of the characteristics of the little red face and the change of the audio feature, and the robot starts the human-computer interaction unit to tell Xiaohong or tell the story. Jokes, etc., make Xiaohong happy; and Xiaohong gives the robot instructions to listen to the song, and the robot sings to Xiaohong. If Xiaohong says to the robot that I want to learn Tang poetry, the robot will ask questions based on Xiaohong’s usual time. Determine Xiaohong's intellectual development stage, and give Xiao Hongyan a Tang poem suitable for Xiaohong's intellectual stage learning and analyze it.
  • the camera unit further includes a photographing function to change according to a facial feature and/or an audio feature of the target object and/or an interactive instruction issued by the target object. Taking a picture of the target object.
  • the camera unit of the robot further includes a photographing function.
  • the robot collects the target object video
  • the facial features of the target object change. If the target object smiles really, the camera unit captures the state of the target object at this time. As in the target object has been silently talking to a person, the camera unit also captures the state of the target object at this time; and as the target object gives the robot a picture of me and the dog, the robot according to the target
  • the object's command activates the camera function of the camera unit, and takes a picture of the target object and the dog.
  • the method further includes the following steps:
  • S700 Receive an interaction instruction of the target object.
  • the target feature information of the family members at home can be stored in the local database of the robot and/or the cloud database connected to the robot, so members stored in the database can go to the machine.
  • the person sends an interactive instruction, and the robot first receives the interactive instruction issued by the current target object.
  • members of the Xiaohong family include Grandpa, Grandma, Dad, Mom, and Xiaohong himself, and the database stores the target feature information of all family members, that is, facial feature information.
  • the current family members include grandfather, grandma, and small. Red three people, if the current target identified by Xiaohong is Xiaohong, when multiple people send interactive commands to the robot at the same time, only the interactive command sent by Xiaohong is accepted.
  • the robot After the robot acquires the interactive instruction of the target object, it needs to parse the information contained in the interactive instruction, and parse the indicator corresponding to the local functional unit in the interactive instruction, so as to open the functional unit of the local machine.
  • the robot receives Xiaohong to send an interactive instruction
  • the interactive instruction is “Give me a story about the duckling”
  • the robot analyzes the “Little Duck Story” and “Speak” in the instruction, and “The Story of the Little Duck” Change to search for "Little Duck's Story” in the database or network, and extract it, and change the "speak” to start the indicator of the speech unit.
  • S900 Start a functional unit corresponding to the indicator.
  • the interaction instruction issued by the target object includes a functional indication that can achieve the purpose of the target object, and according to the indicator parsed in step S800, the functional unit that realizes the purpose of the target object is started, and the target object is issued. instruction.
  • the interactive instruction issued by Xiaohong "tell me the story of the duckling", after the parsing of step S800, searches for "the story of the duckling" through the database and/or the network, and extracts it, and starts the robot.
  • the voice function tells Xiaohong the story of "Little Duck”.
  • the database can be a local database or a cloud database. When searching, the database and the network can be searched at the same time, or only the local database can be searched when there is no network connection.
  • the interactive instruction issues a voice instruction for the target and/or a button corresponding to the function unit that the target clicks on the local machine.
  • the robot has a sensor for receiving voice, and a physical function button for man-machine interaction is set on the robot. If the robot is provided with a touch screen, the function mortgage can also be a virtual touch button.
  • the present invention also provides a robot video call control apparatus, including the following modes.
  • Piece
  • S10 a video module, configured to establish a video call with the calling party, and transmit the video stream obtained by the local camera unit to the calling party;
  • the information stored in the robot records the relationship between the robot and the family members, and establishes a connection mode with the direct communication terminal between the family members.
  • the calling party is a family member, for example, various mobile terminals of the family members such as a mobile phone, a computer, an ipad, etc.
  • the family members are set in advance.
  • the video module S10 directly establishes a video call with the communication terminal of the family member, or when receiving the transmission instruction of the family member, the video module S10 establishes a video call with the family member.
  • the video call request transmits the video stream acquired by the local party to the calling party that initiates the video call, wherein the past video stream transmitted by the robot is also displayed on the calling mobile terminal of the calling party where the controlling robot is installed, or is in the call
  • the mobile application opened by the mobile terminal displays the real-time video call.
  • S20 an analysis module, configured to receive an instruction of the calling party initiated by the calling party, parse target information included in the seeking instruction, and determine target feature information of the corresponding target object according to the target information;
  • the video module S10 establishes a video call with the calling party, and the robot accepts and sends back the video stream to the calling party, and the calling party can directly observe the situation at home through the video, if the calling party cannot see that he wants to see in the video.
  • the caller can issue a seek command on the mobile terminal on the side of the caller.
  • the search command contains information about the target object, and the information is stored in the robot or in the cloud connected to the robot.
  • the robot analysis module S20 parses the target object information included in the homing instruction in the local machine, and then determines the target feature information of the target object corresponding to the target object according to the target object information, and the robot in the subsequent process, Based on this, the target object will be found and determined.
  • the robot analysis module S20 receives the "seeking daughter” search command and parses the information in the seek command in the local machine, that is, parses and extracts the "daughter” information, and sends the "daughter” information to the storage.
  • "Daughter” is a database of target information and corresponding feature information, and determines the daughter's feature information by storing the daughter's target information as the daughter's facial features, that is, the outline and position of the entire face and facial features, and the robot in the subsequent process. In this, based on this, find and determine the daughter.
  • Another embodiment of the present step is that after the robot analysis module S20 receives the instruction of searching, the instruction of the search is sent to the cloud, and the cloud parses the target information contained in the searched instruction, and sends the target information to the robot.
  • the robot determines the target feature information of the target object corresponding to the target object according to the target object information, and the robot searches and determines the target object based on the basis in the subsequent process.
  • the above-mentioned mother initiates a search for “looking for a daughter” to the robot at the mobile terminal, and the robot analysis module S20 receives the instruction of “finding the daughter” and sends the instruction of “seeking the daughter” to the cloud.
  • the cloud parses the information in the search instruction, that is, parses and extracts the information of "daughter”, and sends the information to the robot.
  • the robot receives the parsed information and sends the message "daughter" to the storage "daughter".
  • a database of target information and corresponding feature information wherein the storage relationship between the specific target feature information and the target information is detailed later, and the daughter's feature information is determined by storing the daughter's target information, and the robot is in the subsequent process. In this, based on this, find and determine the daughter.
  • Another embodiment of the present step is that after the robot analysis module S20 receives the instruction of searching, the instruction of the search is sent to the cloud, and the cloud parses the target information contained in the searched instruction, and the cloud sends the target information to the cloud.
  • the cloud database storing the target information and the target feature information, and determining the target feature information of the target object corresponding to the target according to the target information in the cloud, the cloud transmitting the target feature information to the robot, and the robot in the subsequent process, Based on this, the target object will be found and determined.
  • the above-mentioned mother initiates a search for “looking for a daughter” to the robot at the mobile terminal, and the robot analysis module S20 receives the instruction of “finding the daughter” and sends the instruction of “seeking the daughter” to the cloud.
  • the cloud parses the information in the search instruction, that is, parses and extracts the information of “daughter”, determines the feature information of the daughter through the information, and sends the feature information to the robot, and the robot receives the feature information directly receiving the daughter, and This is the basis for seeking Find and confirm your daughter.
  • S30 a capture module, configured to: when the target object is not captured, start the walking device to perform local movement, perform image recognition on the video stream of the image capturing unit during the moving process, and determine an image that includes the target feature information, Capture the target object;
  • the robot After the analysis module S20 determines the feature information of the target object and the target object, if it is determined by the image recognition that there is no image with the target feature information in the acquired video stream, the robot starts its own walking device to move the robot, and moves In the process, the robot passes the acquired video stream of the camera unit, and through image recognition technology, identifies the image in the video, determines whether the image contains the image of the target feature information, and captures the target object, wherein the target object is not captured.
  • the object includes the following situations: 1. The robot does not recognize the target feature information and the extended feature information corresponding to the target object in the image in the video stream, and the robot passes the audio and infrared to the target object, and then passes the measurement and the target object.
  • the distance is greater than a preset distance range between the preset robot and the target object, and the walking device is activated; 2. the robot does not recognize the target feature information and the extended feature information corresponding to the target object in the image in the video stream, and the robot passes After the audio and infrared are positioned to the target object, The distance between the measured object and the target object is smaller than the preset distance range between the preset robot and the target object, and the robot starts the walking device; 3. The robot captures the target feature of the target object, and the distance from the target object remains within the preset range. During the process, when the outline of the video image of the robot camera unit is not clear, the robot does not start the walking device temporarily.
  • the walking device is activated; 4.
  • the robot captures the target feature of the target object, and the distance from the target object remains within the preset range, such as the target object suddenly moving away
  • the walking device is activated. If the target object is far away and the robot finds that the target object is gradually approaching by the distance measurement before the robot locates the target object, the traveling device is not activated.
  • the robotic walking device receives the signal from the camera unit and converts it into an electrical signal of a controller that controls and electrically connects the traveling device, and the controller rotates the electrical signal to the driving device that activates the traveling device, and the driving device starts
  • the walking device realizes the movement of the robot, wherein the driving device may be a motor, the walking device may be a wheel, a crawler or a wheel and the like; the image recognition is to first store a picture in the robot and use it as a model, the robot processor
  • the model is first preprocessed and extracted between the outlines with lines, lines and lines The relative position of the line, the line and the line, the color covered in the outline, etc.
  • the robot processor After capturing a video image, in order to determine whether the target object needs to be captured in the current image, the robot processor sequentially images each frame in the image. Pre-processing, and extracting the contour with lines, the angle between the lines and the lines, the relative position of the lines and the lines, the colors covered in the contour, etc., and the image models in the database are compared and fitted. When the fitness reaches the set value, the captured target object is considered to exist in the video image.
  • the analysis module S20 determines the characteristic information of the daughter, and the robot searches for the daughter at home based on the feature information, and the robot first passes the position at which the instruction of the search is received.
  • Each audio and/or infrared locates the position of the daughter. If the daughter is within a preset distance range and the robot cannot capture the target feature of the daughter in a partial frame or a continuous frame image of the current video stream, the walking device is activated to cooperate with the camera. The unit captures the target feature information of the daughter.
  • the walking device is activated, and the video stream of the camera unit is used for image recognition to capture the daughter's Target features such as the contour features of the daughter's face.
  • a maintenance module configured to control the walking device to maintain a preset distance range between the local device and the target object after the target object is captured.
  • the robot can preset the distance range between the robot and the target object. After the robot step to the target object, the distance between the robot and the target object is first measured by the measuring device installed thereon, if the distance between the robot and the target object is far and not preset Within the distance range, the robot moves to a preset distance range from the target object by the walking device, and during the moving of the target object, the robot always maintains a preset distance range with the target object through the maintenance module S40.
  • the robot finds the daughter according to the daughter's characteristic information through the capture module S30, and the robot measures the distance between the robot and the daughter through its own measuring device, if the robot is far away from the daughter and is not preset. Within the distance range, the robot moves to the preset distance range with the daughter by the walking device, and during the walking of the daughter, the robot always maintains a preset distance range with the target object through the maintenance module S40.
  • the capturing module S10 further includes an acquiring unit S31: after capturing the target object, acquiring extended feature information of the target object other than the target feature information thereof, and failing to capture the target feature Positioning the information according to the extended feature information
  • the target part is amplified by the target object.
  • the expansion unit amplifies the expansion of the target object except the target feature information through the acquisition unit S31.
  • Feature information in the process of the caller's video call through the robot and the target object, due to the movement of the target object itself and/or the movement of the robot, the camera unit of the robot cannot capture the target feature information of the target object, and the robot is in the current video.
  • the robot When a partial frame or a continuous frame picture in the stream cannot clearly identify the contour with the line corresponding to the target feature, the angle between the line and the line, the relative position of the line and the line, the color covered in the outline, etc., the robot
  • the target feature information of the target object can be quickly found and focused by the extended feature information of the augmented portion other than the target object target feature information. After the target object captures the target object, the target feature information of the target object cannot be recognized because the target object suddenly moves away.
  • the robot In order to further capture the target object, the robot simultaneously identifies the target feature information and the extended feature information of the target object in the current video stream. If the augmented part of the target object is identified by the extended feature information, the target object is captured according to the augmented part, and the target feature of the target object is further determined based on the target object.
  • the robot collects other extended feature information of the daughter other than the facial feature information, such as the color and style of the daughter's clothes, pants, and shoes, through the collecting unit S31.
  • the color and shape of the hair, the color, shape, style of the hat, the outline of the body, arms, legs, etc. during the process of the mother and daughter passing the robot video call, if the daughter is sitting while sitting, in the process, It is possible that the robot cannot locate and capture the facial feature information of the daughter, and the robot can locate the target feature information and capture the target object through the extended feature information of the extended part.
  • the ratio decolorizes part of the frame or the continuous frame image, processes it as a black and white picture, extracts the outline feature of the body contour, and determines the target object as the extended feature information in the process of capturing the target object.
  • the acquiring unit S31 further includes a positioning unit S311: after positioning the augmented portion of the target object according to the extended feature information, the starting walking device continues to search for the target feature information around the augmented portion until the target feature information is continued. After targeting the target feature information The target object is now captured.
  • the robot When the robot cannot capture the target feature information of the target object, the robot first compares the feature information captured by the capture module S30 with the extended feature information collected by the previous acquisition unit S31, and locates the current capture module S30 by the positioning unit S311. In the extended part, the robot initiates the walking device to continue to search for the target feature information around the augmented portion until the camera unit captures and locates the target feature information, and then continues to capture the target in the video call.
  • the robot capture module S30 can capture the facial feature information of the daughter when the daughter is sitting as described above, and the robot capture module S30 can only capture after the daughter stands up.
  • the daughter's body part, the daughter is wearing a pink dress, the style is a round neck, sleeveless tutu, at this time the robot can be determined by the extended feature information collected before the daughter's body contour, skirt color, style
  • the captured portion is determined, and the orientation of the rotation of the robot camera unit is determined by the capturing portion, that is, the collar, the shoulder form, and the characteristic information of the daughter's body captured by the robot through the camera unit, and it is determined that the body part of the daughter is captured at this time.
  • the chest, and by lifting the camera unit, can capture the facial features of the daughter; if the daughter stands up and turns over the body, so that the back of the camera is facing the camera unit of the robot, the camera unit of the robot captures the daughter's torso Extended skirt color and style, extended profile information of the back profile, then the machine Then, the walking device is required to move around the torso and search for the target feature information by changing the angle of the camera unit until the camera unit searches for the target feature information, the robot adjusts the distance with the daughter, and the target feature information is located by the positioning unit S311, and then the capture module The S30 continues to capture the target object.
  • the robot If the robot captures the target feature information of the daughter and the distance from the daughter remains within the preset range, the robot does not start the walking when the video image outline is unclear.
  • the device after the image of the video image is clear, after identifying the image in the video stream, the recognized extended feature information is derived from the amplified portion of the daughter's back of the body, and the walking device is activated to capture the facial feature information of the daughter around the feature portion.
  • the extended feature information is collected from a moving scene image in which the image portion corresponding to the target feature information in the video stream moves together.
  • the target object belongs to the dynamic image in the video stream relative to other scenes.
  • the extended information is collected, the collected video stream moves together with the target feature information.
  • the extended feature on the target object for example, after the robot camera unit captures the target object, wherein the facial feature information is used as the target feature information, and the other line contours in the partial frame or the continuous frame picture change with the line outline of the face occur.
  • the portion that changes and can be surrounded by the contour of the face in a closed contour serves as an augmentation site. After the amplification site is determined, the contour of the augmentation site, the relative position of each contour, and the color of the contour coating are collected.
  • the daughter in the video stream is a dynamic object relative to the static item in the home, and the daughter's torso, clothes, hair, etc. are all following the movement of the daughter, according to the daughter in the video stream.
  • the facial feature information determines the daughter's object, the outline of the part of the video stream in which the partial or continuous frame picture changes with the contour of the daughter's face changes, the relative position of each contour and the outline are covered.
  • the color, and other line contours can be surrounded with the contour of the face with a part of the closed contour, and according to the position of the other line contours and the contour of the face, it is determined that the other contour belongs to a certain part of the body, and according to the partial frame or the continuous frame Whether the contour of the face in the picture changes relative to the position of other items and the position of other parts follows whether the position of the face changes relative to the position of other items to determine whether the daughter is in a state of motion, and if yes, it is exercise, otherwise the daughter is not in motion. .
  • the contour features of the face in the partial or continuous frame image of the video stream of the camera unit are changed relative to other items, and the contour position of the face is changed according to the change of the face feature position.
  • the change is changed with respect to the position of other items, the daughter is determined to be in a moving state, and the contours of other parts can be wrapped in a closed contour with the contour of the face, and the leg is determined according to the position of the contour of the other part relative to the face.
  • the analysis module S20 further includes a query unit S21.
  • the target information and the target feature information are stored in a database in a mapping relationship, and are used to query the database to determine target feature information according to the target information.
  • the target information and the target feature information are all stored in the database in a one-to-one correspondence relationship.
  • the query unit S21 can query the target feature information through the target information, and the database can be local.
  • the database can also be a cloud database connected to the robot. If it is a local database, after obtaining the target information, the target feature information can be directly determined locally, and if it is a cloud database, the robot sends the target information. After the cloud determines the target feature information corresponding to the target information, the target feature information is returned to the robot.
  • Xiaohong is the target
  • the mother when storing, the mother will store the common name of Xiaohong (including the common name of other family members) and Xiaohong's facial feature information, and also include collecting. Extended feature information, as shown in Table 2.
  • Table 2 shows the storage relationship between the target information and the target feature information in the database.
  • the maintaining module S40 further includes a measuring unit S41: in the step of controlling the walking device to maintain a preset distance range between the local device and the target object after capturing the target object, accompanied by the walking device The operation is to obtain the distance data detected by the local distance sensor and the target object. When the distance data exceeds the predetermined distance range, the control walking device starts to move first to perform the movement, otherwise the control walking device stops walking and pauses. mobile.
  • the distance sensor of the measuring unit S41 is always in the measurement state while maintaining the distance between the robot and the target object, and the distance between the robot itself and the target object is measured.
  • the robot automatically controls the walking device to start moving and moves. If the distance between the robot and the target object is measured within a preset range by the distance sensor, the robot automatically controls the walking device. Stop the walking device to pause the robot.
  • the robot After the robot captures the small red, the distance between the measuring unit S41 and the target object is measured during the distance between the measuring unit S41 and the target object.
  • the robot automatically controls the walking device to start walking and move, if the distance between the robot and Xiaohong is measured by the distance sensor in a preset range. Inside, the robot automatically controls the walking device to stop the walking device so that the robot pauses and moves over the distance range.
  • the robot automatically controls the walking device to start walking and move, making the robot more intelligent.
  • the target information is a name or an indicator of the target object.
  • the analysis module S20 After the analysis module S20 receives the instruction of the incoming caller, the object information carried in the instruction is parsed, and the target object information is the name or indicator of the target object, such as the name of the person, the computer, and the like.
  • the analysis module S20 parses out the computer in the "find computer” and As the target object indicator, such as the computer is the indicator, that is, the target information; if the daughter's name is Xiaohong, the mother's instruction to find is "Look for Xiaohong", in which "Look for Xiaohong” character information, analysis
  • the module S20 parses out the red in the "find red” and uses it as an indicator of the target object, that is, the target information red is the name of the target object of the daughter, that is, the target information; in addition, the indicator may also be
  • the information stored by the calling party terminal and the target object triggers the information of the target object at the calling party terminal, and the calling party terminal generates an indicator corresponding to the target object by the information of the target object, and issues an indicator of the target object to the robot.
  • the mother as the caller stores information related to her daughter Xiaohong on her terminal, such as the name Xiaohong, the image representing Xiaohong, mother Triggered on the terminal daughter Alice's name or image on behalf of the red, the caller terminal generates red looking indicator, and the indicator is sent to the robot.
  • the target feature information is facial feature information of the target object.
  • the target feature information of the target object is entered in advance, and the facial feature change can best express a person's mood or express the state at the time.
  • the facial feature is preferably used as the target feature information, which is convenient for the parent during the subsequent video call. Or the family members during the video call, or the video and/or photos taken, can first observe the emotions of the children and/or other family members at home through the facial expression.
  • the collecting unit S31 further includes a monitoring unit S312: for monitoring the extended feature information of the extended part after detecting that the extended feature information of the target object is changed.
  • the robot can save the extended feature information of the target object to the next extended feature information.
  • the robot monitoring unit S312 detects that the extended feature information of the stored target object has changed, and then re-acquires the new extended feature information, so that after the target object is extended, the target information needs to be captured, and the target object can be quickly located. Go to the target object's target feature information and capture the target object.
  • the extended feature information stored in the database is also the information of the pink dress.
  • the robot determines the object wearing the white dress by capturing the target feature information.
  • the monitoring unit S312 finds that the extension feature information of the small red body and the white dress is re-acquired after the change of the clothing extension feature information is changed.
  • the distance sensor is an ultrasonic sensor, an infrared sensor, or a binocular ranging imaging device including the imaging unit.
  • the distance sensor in the maintenance module S40 is an ultrasonic sensor, an infrared sensor, or a binocular ranging imaging device including the imaging unit, and the binocular ranging imaging device is convenient to use, and can initially determine the distance between the robot and the target object, and the ultrasonic wave.
  • the error of long distance ranging is small and the effect is good.
  • the infrared sensor has small distance measurement error and good effect.
  • the invention combines with each other to optimize the distance error of the robot in the distance.
  • the extended feature information includes one or any of a plurality of features: a torso part feature, a clothing part feature, a facial contour feature, a hair outline feature information, or audio feature information.
  • the extended feature information collected by the collecting unit S31 includes one or any of a plurality of features: a trunk part feature, a clothing part feature, a face outline feature, a hair outline feature information, or audio feature information.
  • the capturing module S30 includes a positioning unit S311, and the local audio and/or infrared positioning unit acquires the target position when the target object is captured by the local audio and/or infrared positioning unit. To determine the starting direction of the walking device.
  • an audio and/or infrared positioning unit that acquires the position of the target by turning on the audio and/or infrared positioning unit during capturing of the target object, and thereby determining the walking of the robotic walking device at the beginning direction.
  • the robot after determining that the target object is Xiaohong, Xiaohong laughs at the front of the robot at this time, and the robot acquires the audio of Xiaohong through the audio positioning unit, and locates the small The red position is directly in front of the robot.
  • the robot directly activates the walking device to move the robot forward.
  • the robot uses the infrared light of the infrared positioning unit to radiate the infrared light reflected by the surrounding scene and the environment to determine the location of Xiaohong.
  • the position is the right front of the robot, and the robot activates the walking device and moves to the right front to find the red.
  • the measuring unit S41 is further configured to: when capturing the target object, when the obstacle is encountered, the local machine measures the distance between the local device and the obstacle through the distance sensor, and controls the walking device to bypass and / or away from the obstacle, continue to capture the target object after detouring and / or away from the obstacle.
  • the robot In the process of searching for Xiaohong, the robot will inevitably encounter obstacles, such as stools and walls in the home. It is also possible to measure the distance between the robot and the obstacles in the figure by the distance sensor of the measuring unit S41. With the orientation unchanged, the walking device is controlled to bypass and/or move away from the stool, the wall, and continue to capture the red.
  • the maintenance module S40 further includes a voice module S50, as shown in FIG. 6, for starting the voice reminding unit and issuing a voice reminder when the local machine moves within a distance range from the target object.
  • a voice module S50 as shown in FIG. 6, for starting the voice reminding unit and issuing a voice reminder when the local machine moves within a distance range from the target object.
  • the robot In order to ensure that the target object can receive the message sent by the parent in time when the calling party initiates the video call, the robot prompts the voice reminding unit when the robot captures the target object and moves to a preset distance from the target object. And send a voice reminder.
  • the robot finds Xiaohong and moves to the distance range from Xiaohong, it will send a voice reminder to Xiaohong. For example, if the mother calls, the mother calls, and the mother calls, fast. Answer the call, answer the call quickly, and answer the call quickly.
  • the video module S10 further includes:
  • S11 a shooting unit, after the calling party hangs up the video call, the robot camera unit continuously collects a video of the target object;
  • the robot continuously collects the child's home video by shooting the image unit S11.
  • the shooting unit S11 plays in the continuous collection of Xiaohong's home, learning and other videos.
  • S12 A transmission unit, the robot sends the video to a terminal connected thereto, and sends a text reminder and/or a voice reminder to the terminal.
  • the robot After the video capture of the shooting unit S11 is completed, the robot sends the collected video to the terminal and/or the cloud of the family member connected thereto through the transmission unit S12, and sends a text reminder and/or a voice reminder to the terminal through the transmission unit S12, and collects the video. After a video, continue to capture the next video.
  • the video shooting unit S11 plays the video of Xiaohong at home, it is sent to the family member's terminal and/or the cloud through the transmission unit S12, such as a mobile phone, a computer, an ipad, etc., and/or connected thereto.
  • the transmission unit S12 such as a mobile phone, a computer, an ipad, etc.
  • a text reminder and/or a voice reminder is sent to the terminal of the family member through the transmission unit S12, for example, a video with a small red play; if the robot is not connected to the cloud, it is only sent to the terminal, and then When it is sent to the cloud and the terminal, or when the terminal is all turned off, only the cloud is sent, and when any terminal is turned on, an alert message is sent.
  • the activation unit 60 is further configured to: when the robot acquires the target object, according to a change of the target object facial feature and/or audio feature and/or an interaction instruction issued by the target object, The local device initiates a voice interaction unit and/or initiates a video call to a mobile terminal connected to the local device.
  • the robot changes the facial features of the target object through the camera unit during the process of collecting the target object. If the child is crying, the robot starting unit 60 activates the human-computer interaction unit of the machine to make the child happy; according to the audio characteristics of the target object The change, such as the child is losing his temper, the robot determines that the child belongs to the temper state through the audio feature, the robot starting unit 60 activates the human-computer interaction unit of the machine to comfort the child; the robot sends an interactive instruction according to the target object, such as a child When the machine proposes a flower in English, the robot answers according to the question raised by the child, and the English of the flower is flower; and if the target child sends a call to the dad to the robot, the robot starting unit 50 activates the video module S10 to the mobile phone of the father. The terminal issues a video call request.
  • the robot determines that Xiaohong is crying now by the change of the characteristics of the little red face and the change of the audio feature, and the robot starting unit 60 activates the human-computer interaction unit to tell the story to Xiaohong. Or tell a joke, etc., to make a little red happy; and as Xiaohong gives the robot an instruction to listen to the song, the robot startup unit 60 activates the song function to sing to Xiaohong; Another example is Xiaohong and the robot saying that I want to learn Tang poetry. Then the robot determines the stage of intellectual development of Xiaohong according to the question of Xiao Hongping, and gives Xiao Hong the Tang poem that is suitable for the learning of Xiaohong intelligence stage.
  • the photographing unit S11 is further configured to, during the process of acquiring the target object video, the photographing unit further includes a photographing function to change according to a facial feature and/or an audio feature of the target object and/or Or an interactive instruction issued by the target object takes a picture of the target object.
  • the shooting unit S11 of the robot further includes a photographing function.
  • the robot collects the target object video, the facial features of the target object change. If the target object smiles really, the camera unit captures the target object at this time. State; as in the target object has been silently talking to a person, the camera unit also captures the state of the target object at this time; and as the target object gives the robot a picture of me and the dog, the robot is based on The instruction of the target object activates the camera function of the camera unit, and takes a picture of the target object and the dog.
  • the transmission unit S12 further includes:
  • S13 a receiving unit, configured to receive an interaction instruction of the target object
  • the target feature information of the family members at home can be stored in the database local to the robot and/or the cloud database connected to the robot, so members stored in the database can send interactive instructions to the robot, and the robot first receives the current target object.
  • members of the Xiaohong family include Grandpa, Grandma, Dad, Mom, and Xiaohong himself, and the database stores the target feature information of all family members, that is, facial feature information.
  • the current family members include grandfather, grandma, and small. Red three people, if the current target identified by Xiaohong is Xiaohong, when multiple people send interactive commands to the robot at the same time, only the interactive command sent by Xiaohong is accepted.
  • S14 an analyzing unit, configured to parse the interaction information included in the interaction instruction, and extract an indicator corresponding to the local function unit;
  • the robot After the robot acquires the interactive instruction of the target object, it needs to parse the information contained in the interactive instruction, and parse the indicator corresponding to the local functional unit in the interactive instruction, so as to open the functional unit of the local machine.
  • the robot analyzes the "Little Duck Story” and “Speak” in the instruction, transforms the “Little Duck Story” into a database or network to search for “Little Duck Story” and extract it. , "talk” to change the indicator that initiates the speech unit.
  • the interaction instruction issued by the target object includes a functional indication that can achieve the purpose of the target object, and according to the indicator parsed by the analysis unit S14, the functional unit that realizes the purpose of the target object is started, and the target object is issued. Instructions.
  • the interactive instruction issued by Xiaohong mentioned above "tell me the story of the duckling", after parsing by the analyzing unit S14, searching for "the story of the duckling” through the database and/or the network and extracting it, starting the robot
  • the voice function tells Xiaohong the story of "Little Duck”.
  • the database can be a local database or a cloud database. When searching, the database and the network can be searched at the same time, or only the local database can be searched when there is no network connection.
  • the interactive instruction issues a voice instruction for the target and/or a button corresponding to the function unit that the target clicks on the local machine.
  • the robot has a sensor for receiving voice, and a physical function button for man-machine interaction is set on the robot. If the robot is provided with a touch screen, the function mortgage can also be a virtual touch button.
  • the present invention also provides a terminal, including a processor, for executing a program to execute various steps of the robot video call control method, for example, the robot establishes various mobile terminals with a mother such as a mobile phone, a computer, an ipad.
  • a mother such as a mobile phone, a computer, an ipad.
  • the terminal downloads a video stream that is connected to the control and connected to the robot, and transmits the home situation acquired by the local camera unit to the mother's mobile terminal, since the mother wants to see the daughter Xiaohong in the mobile terminal now In the state of the home, and the video stream does not contain the image of Xiaohong at this time, the mother initiated a search for the "find daughter" to the robot on the mobile terminal, and the robot receives the instruction of "finding the daughter" and is in the local machine.
  • the information in the instruction of the search is parsed, that is, the information of "daughter” is parsed and extracted, and the characteristic information of the daughter is determined locally by the information, and the feature information is the facial feature of the daughter, that is, the outline of the entire face and the facial features.
  • the robot finds the daughter at home based on this characteristic information.
  • the robot first receives the finger in the search.
  • the camera unit of the robot rotates 360 degrees, and the image is recognized by the video stream of the camera unit to capture the target feature of the daughter. If the daughter is not captured, the robot starts its own walking device and moves, and the robot moves.
  • the video stream is acquired by the camera unit, and the image recognition technology is used to check whether there is a daughter's facial feature information in the current video image. If the robot finds the daughter according to the daughter's feature information, the robot passes its own measuring device. Measuring the distance between the robot and the daughter. If the robot is far away from the daughter and is not within the preset distance range, the robot moves to the preset distance from the daughter through the walking device, and during the walking of the daughter, the robot always Keep a preset distance range from your daughter Xiaohong.
  • processor of this embodiment may further implement other steps of the method in the foregoing embodiment.
  • processors in the foregoing method and no further details are provided herein.
  • Fig. 9 shows a video callable mobile robot (hereinafter referred to as a video call mobile robot collectively referred to as a device) that can implement robot video call control according to the present invention.
  • the device conventionally includes a processor 1010 and a computer program product or computer readable medium in the form of a memory 1020.
  • the memory 1020 may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), an EPROM, a hard disk, or a ROM.
  • the memory 1020 has a memory space 1030 for executing program code 1031 of any of the above method steps.
  • storage space 1030 for program code may include various program code 1031 for implementing various steps in the above methods, respectively.
  • the program code can be read from or written to one or more computer program products.
  • These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks.
  • Such a computer program product is typically a portable or fixed storage unit as described with reference to FIG.
  • the storage unit may have a storage section or a storage space or the like arranged similarly to the storage 1020 in FIG.
  • the program code can be compressed, for example, in an appropriate form.
  • the storage unit comprises program code 1031' for performing the steps of the method according to the invention, ie code that can be read by, for example, a processor such as 1010, which when executed by the device causes the device to perform the above Each step in the described method.
  • steps, measures, and solutions in the various operations, methods, and processes that have been discussed in the present invention may be alternated, changed, combined, or deleted. Further, other steps, measures, and schemes of the various operations, methods, and processes that have been discussed in the present invention may be alternated, modified, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and solutions in the prior art having various operations, methods, and processes disclosed in the present invention may also be alternated, changed, rearranged, decomposed, combined, or deleted.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Signal Processing (AREA)
  • Manipulator (AREA)

Abstract

Un procédé, un dispositif et un terminal de commande d'appel vidéo de robot, selon la présente invention, concernent principalement un robot intelligent capable de réaliser un appel vidéo par l'intermédiaire d'une télécommande. Le procédé comprend les étapes suivantes consistant à : établir un appel vidéo avec une partie appelante et transmettre des flux vidéo acquis de manière native à la partie appelante ; recevoir une instruction de ralliement lancée par la partie appelante, analyser les informations d'objet cible qui y sont contenues, et utiliser celles-ci en tant que base pour déterminer des informations de caractéristiques cibles correspondantes ; démarrer un dispositif de marche pour exécuter un mouvement du natif lorsqu'un objet cible n'est pas capturé, et effectuer une reconnaissance d'image sur les flux vidéo pendant le déplacement pour déterminer une image contenant les informations de caractéristiques cibles de façon à capturer l'objet cible ; et commander le dispositif de marche de manière à maintenir une plage de distance prédéfinie entre le natif et l'objet cible après la capture de l'objet cible. Le dispositif de marche de la présente invention coopère avec une unité de caméra pour capturer l'objet cible rapidement par l'intermédiaire de la technologie de reconnaissance d'image pendant le déplacement, effectue l'appel vidéo et réduit la solitude des enfants par l'intermédiaire d'une fonction d'interaction homme-ordinateur.
PCT/CN2017/116674 2016-12-15 2017-12-15 Procédé, dispositif et terminal de commande d'appel vidéo de robot WO2018108176A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611157928.6 2016-12-15
CN201611157928.6A CN106791565A (zh) 2016-12-15 2016-12-15 机器人视频通话控制方法、装置及终端

Publications (1)

Publication Number Publication Date
WO2018108176A1 true WO2018108176A1 (fr) 2018-06-21

Family

ID=58888280

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/116674 WO2018108176A1 (fr) 2016-12-15 2017-12-15 Procédé, dispositif et terminal de commande d'appel vidéo de robot

Country Status (2)

Country Link
CN (1) CN106791565A (fr)
WO (1) WO2018108176A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110221602A (zh) * 2019-05-06 2019-09-10 上海秒针网络科技有限公司 目标对象捕捉方法和装置、存储介质及电子装置
CN114079696A (zh) * 2020-08-21 2022-02-22 海能达通信股份有限公司 呼叫终端的方法、装置及电子设备

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106791565A (zh) * 2016-12-15 2017-05-31 北京奇虎科技有限公司 机器人视频通话控制方法、装置及终端
CN107659608A (zh) * 2017-07-24 2018-02-02 北京小豆儿机器人科技有限公司 一种基于养老机器人的亲情关怀系统
CN107516367A (zh) * 2017-08-10 2017-12-26 芜湖德海机器人科技有限公司 一种基于医院排队人员识别的座椅机器人控制方法
CN107825428B (zh) * 2017-12-08 2020-12-11 子歌教育机器人（深圳）有限公司 智能机器人的作业系统及智能机器人
CN108073112B (zh) * 2018-01-19 2024-02-20 冠捷电子科技(福建)有限公司 一种具有角色扮演的智能服务型机器人
CN110191300B (zh) * 2019-04-26 2020-02-14 特斯联（北京）科技有限公司 一种无人化停车场的可视通话设备及其系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040190754A1 (en) * 2003-03-31 2004-09-30 Honda Motor Co., Ltd. Image transmission system for a mobile robot
US20060147001A1 (en) * 2004-12-07 2006-07-06 Young-Guk Ha System and method for service-oriented automatic remote control, remote server, and remote control agent
CN103718125A (zh) * 2011-08-02 2014-04-09 微软公司 寻找被呼叫方
CN103926912A (zh) * 2014-05-07 2014-07-16 桂林赛普电子科技有限公司 一种基于家庭服务机器人的智能家庭监控系统
CN104800950A (zh) * 2015-04-22 2015-07-29 中国科学院自动化研究所 自闭症儿童辅助机器人及系统
CN105856260A (zh) * 2016-06-24 2016-08-17 深圳市鑫益嘉科技股份有限公司 一种带有随叫随到功能的机器人
CN106162037A (zh) * 2016-08-08 2016-11-23 北京奇虎科技有限公司 一种视频通话时进行互动的方法和装置
CN106791565A (zh) * 2016-12-15 2017-05-31 北京奇虎科技有限公司 机器人视频通话控制方法、装置及终端

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6925357B2 (en) * 2002-07-25 2005-08-02 Intouch Health, Inc. Medical tele-robotic system
CN102025964A (zh) * 2010-05-07 2011-04-20 中兴通讯股份有限公司 一种视频留言方法及终端
CN102176222B (zh) * 2011-03-18 2013-05-01 北京科技大学 多传感器信息采集分析系统及自闭症儿童监护辅助系统
CN104656653A (zh) * 2015-01-15 2015-05-27 长源动力（北京）科技有限公司 一种基于机器人的交互系统及方法
CN105182983A (zh) * 2015-10-22 2015-12-23 深圳创想未来机器人有限公司 基于移动机器人的人脸实时跟踪方法和跟踪系统
CN105301997B (zh) * 2015-10-22 2019-04-19 深圳创想未来机器人有限公司 基于移动机器人的智能提醒方法和系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040190754A1 (en) * 2003-03-31 2004-09-30 Honda Motor Co., Ltd. Image transmission system for a mobile robot
US20060147001A1 (en) * 2004-12-07 2006-07-06 Young-Guk Ha System and method for service-oriented automatic remote control, remote server, and remote control agent
CN103718125A (zh) * 2011-08-02 2014-04-09 微软公司 寻找被呼叫方
CN103926912A (zh) * 2014-05-07 2014-07-16 桂林赛普电子科技有限公司 一种基于家庭服务机器人的智能家庭监控系统
CN104800950A (zh) * 2015-04-22 2015-07-29 中国科学院自动化研究所 自闭症儿童辅助机器人及系统
CN105856260A (zh) * 2016-06-24 2016-08-17 深圳市鑫益嘉科技股份有限公司 一种带有随叫随到功能的机器人
CN106162037A (zh) * 2016-08-08 2016-11-23 北京奇虎科技有限公司 一种视频通话时进行互动的方法和装置
CN106791565A (zh) * 2016-12-15 2017-05-31 北京奇虎科技有限公司 机器人视频通话控制方法、装置及终端

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110221602A (zh) * 2019-05-06 2019-09-10 上海秒针网络科技有限公司 目标对象捕捉方法和装置、存储介质及电子装置
CN114079696A (zh) * 2020-08-21 2022-02-22 海能达通信股份有限公司 呼叫终端的方法、装置及电子设备
CN114079696B (zh) * 2020-08-21 2024-06-07 海能达通信股份有限公司 呼叫终端的方法、装置及电子设备

Also Published As

Publication number Publication date
CN106791565A (zh) 2017-05-31

Similar Documents

Publication Publication Date Title
WO2018108176A1 (fr) Procédé, dispositif et terminal de commande d'appel vidéo de robot
CN106873773B (zh) 机器人交互控制方法、服务器和机器人
US11080882B2 (en) Display control device, display control method, and program
US20230305530A1 (en) Information processing apparatus, information processing method and program
US20170185276A1 (en) Method for electronic device to control object and electronic device
US20130123658A1 (en) Child-Care Robot and a Method of Controlling the Robot
JP6759445B2 (ja) 情報処理装置、情報処理方法及びコンピュータプログラム
WO2019072104A1 (fr) Procédé et dispositif d'interaction
WO2020015682A1 (fr) Système et procédé de commande d'un véhicule aérien sans pilote
JP7375770B2 (ja) 情報処理装置、情報処理方法、およびプログラム
CN108881544B (zh) 一种拍照的方法及移动终端
US10339381B2 (en) Control apparatus, control system, and control method
CN110989689A (zh) 一种基于自动追踪的机器人系统
US11780097B2 (en) Information processing apparatus and method for processing information
CN112106004A (zh) 信息处理装置、信息处理方法和程序
US20220371178A1 (en) Information processing apparatus, information processing method, and program
US12263592B2 (en) Autonomous mobile body, information processing method, and information processing device
CN107452381B (zh) 一种多媒体语音识别装置及方法
WO2019123744A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
US12138794B2 (en) Information processing apparatus and information processing method
US11687049B2 (en) Information processing apparatus and non-transitory computer readable medium storing program
US11986959B2 (en) Information processing device, action decision method and program
TW200848229A (en) Method of locating objects using an autonomously moveable device
CN114488879B (zh) 一种机器人控制方法以及机器人
Menon et al. A Survey on Alzheimer's Patient's Assistive Technology

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17881594

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17881594

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载