JP7625315B1 - Information processing device, information processing method, and program - Google Patents

Abstract

An information processing device, an information processing method, and a program are provided that are capable of improving safety during ship navigation in a new way.
[Solution] An information processing device according to the present disclosure includes a control unit that executes a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a specified detection target detected by a detection unit of each first ship, a storage process for consolidating and storing the received detection target information, and a transmission process for transmitting at least one of the received detection target information and specified output data generated using the detection target information under predetermined generation conditions to a specified second ship.
[Selected Figure] Figure 1

Description

The present disclosure relates to an information processing device, an information processing method, and a program.

Traditionally, systems such as AIS (Automatic Identification System) have been known as a means of sharing a ship's position information with other ships to increase safety when sailing. However, many small ships, such as fishing boats, are not equipped with AIS, and there have been issues with information sharing.

In response to this, for example, Patent Document 1 proposes a system that uses a device with GPS functionality to transmit a ship's position information to other ships, so that even ships without AIS can share their own position information.

JP 2006-163765 A

However, it is practically difficult to equip all ships with AIS or GPS equipment.

Therefore, this disclosure has been made in consideration of the above problems, and its purpose is to provide an information processing device, an information processing method, and a program that can improve safety during ship navigation in a new way.

According to the present disclosure, a receiving process is performed to receive detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships;
A storage process of aggregating and storing the received detection target information;
An information processing device is provided that has a control unit that executes a transmission process to transmit at least one of the received detection target information and specified output data generated under predetermined generation conditions using the detection target information to a specified second ship.

This disclosure provides an information processing device, an information processing method, and a program that can improve safety during ship navigation in a new way.

FIG. 1 illustrates an example of a system according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating an example of an information processing device according to the embodiment. FIG. 4 is a flowchart illustrating a series of controls in the system according to the embodiment. FIG. 11 is a diagram showing an example of detection target information according to the embodiment. 2 shows an example of a ship detection system according to the embodiment. 2 shows an example of a vessel according to the embodiment. 4 is a flowchart illustrating a series of controls in the detection system according to the embodiment. FIG.

A preferred embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that in this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals to avoid redundant description.

Figure 1 is a conceptual diagram of a system using the information processing device of this embodiment. Figure 1 illustrates ships A1 and A2 currently sailing that can communicate with the information processing device 100, as well as a ship B, a floating object C, and a person D that cannot communicate with the information processing device 100.

The information processing device 100 shown in FIG. 1 functions as a management server (e.g., a cloud server). As shown in FIG. 2, the information processing device 100 includes a control unit 120, a memory unit 130, an input unit 140, an output unit 150, and a communication unit 160. In this example, the control unit 120, the memory unit 130, the input unit 140, the output unit 150, and the communication unit 160 are implemented in an information processing device (computer), but this is not limited thereto, and the system may be divided into multiple devices, or each unit (at least a portion) may be provided independently.

The control unit 120 includes a processor (computing device) such as a CPU, and is configured to be able to execute various information processing operations based on programs stored in the memory unit 130.

The memory unit 130 can store various information stored in advance, information generated by the control unit 120, user operation information input via the input unit 140, information received from the ship or an external device via the communication unit 160, etc. The memory unit 130 has a main memory device consisting of a volatile memory device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory device consisting of a non-volatile memory device such as a flash memory or a HDD (Hard Disc Drive).

The storage unit 130 can store two-dimensional or three-dimensional map data (cartography information) corresponding to real space, received information, various information generated based on the received information, various information related to the detection target, condition information for various processes executed by the control unit, etc. The various processes can store information related to various conditions such as the matching process, position prediction process, and target estimation process described below. The storage unit 130 can store information related to various parameters indicating the shapes (three-dimensional model data, etc.) and characteristics (characteristics related to power, speed, and turning amount) of various ships, navigation history information, navigation route information, tidal information, etc.

The input unit 140 accepts input information based on operations from the user. The input unit 140 is composed of mechanical buttons, switches, operating levers, a touch panel, a keyboard, a mouse, etc. The input unit 140 may also be equipped with a microphone that allows voice input.

The output unit 150 outputs various information in the form of images (video), audio, etc. The output unit 150 may include, for example, a display unit such as an LCD monitor or touch panel that displays images, an audio output unit such as a speaker that displays audio, a vibration generating unit (vibration device) that generates vibrations, etc.

The communication unit 160 is connected to a network such as the Internet or wireless communication, and receives and transmits data from the ship, external devices (management servers, control devices, terminal devices used by users, etc.), etc.

The control unit 120 can estimate the position, type, size, posture, etc. of an obstacle by, for example, performing image analysis on image data acquired by a camera (target estimation process). The control unit 120 may estimate the type of obstacle by selecting one of the candidate types of obstacles stored in advance in the storage unit 130. The information on the type of obstacle can include type information such as ships, specific ship classifications (fishing boats, water taxis, small multipurpose boats, recreational fishing boats, passenger boats, transportation boats, work boats, fireboats/patrol boats, pleasure yachts, pleasure motorboats, special work boats, tankers, cargo ships, etc.), marine structures, people (including swimming, surfing, yachts, paddle boards, etc.), rocks, shores, driftwood, seaweed, marine life, etc. The information on the type of obstacle is stored in association with size (area, height) information.

In the obstacle information estimation process, the control unit 120 may estimate the position, type, size, posture, etc. of the obstacle using machine learning technology using a learning model that has been trained in advance. When the control unit 120 estimates the position of the obstacle based on image data acquired by the camera, after estimating the three-dimensional position (coordinates) of the obstacle in the camera coordinate system, the control unit 120 can convert the camera coordinate system to a world coordinate system by a coordinate conversion process based on the camera position (coordinates) and shooting direction information, and calculate the position (coordinates) of the obstacle on the world coordinate system. The control unit 120 can estimate the bow direction of a ship as an obstacle, the direction of a person's face (front direction), and the direction (direction of travel) of an object used such as a surfboard by image analysis of the image data from the camera or analysis of three-dimensional data (point cloud data, model data, etc.).

The control unit 120 of the information processing device can execute a receiving process for receiving detection target information including position information of a predetermined detection target detected by the detection unit of each first ship from a plurality of first ships sailing, a storage process for aggregating and storing the received detection target information, and a transmission process for transmitting at least one of the received detection target information and the predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship. According to this embodiment, the information detected by the first ship can be aggregated and stored and transmitted to another second ship. As a result, for example, the position information of a ship that does not have an AIS or GPS can be shared with the second ship via the information processing device from the first ship, thereby improving safety during navigation. Note that the ship A1 in FIG. 1 may function as the first ship and the ship A2 may function as the second ship, or vice versa. Also, the ships A1 and A2 may function as the first ships, and the ships A1 and A2 may function as the second ships. Also, at least one of vessel A1 and vessel A2 may function as the first vessel, and another vessel (not shown) may function as the second vessel. In either case, if any of the first vessels can detect detection targets such as vessel B, floating object C, or person D, they can be shared with other second vessels that are unable to detect them, thereby improving safety during navigation.

For example, as shown in FIG. 3, the control unit 120 executes a reception process to acquire detection target information from a first vessel (S101). The reception process only needs to be able to receive detection target information from at least one first vessel. Furthermore, the information processing device maintains a continuous reception-enabled state unless the control unit 120 stops the reception function.

Next, the control unit 120 aggregates the detection target information and executes a storage process (S102). Aggregating and storing means consolidating and managing information acquired from multiple different ships in a centralized manner. The storage process is performed each time a reception process is performed. The control unit 120 may assign unique identification information (such as an ID including a number or letters) to each detection target and accumulate the information in the detection target information table. In addition, map information that reflects the position (coordinates) of the detection target in pre-stored map information may be stored.

Figure 4 shows an example of a detection target information table. In the example of Figure 4, the detection target information includes the type of detection target, position, attitude, movement speed, movement direction, size, detection time (time when the first ship detected it), reception time (time when the information processing device received it), and transmitting ship (the ship that detected the information and transmitted it to the information processing device), which are associated with the identification information and managed in an aggregated manner. The detection target information may include other information, for example, image information of each detection target captured by a camera of the first ship may be associated and stored. In addition, the detection target information received by the information processing device may include at least position information, and for example, the attitude, movement speed, etc. may be unknown. In the example of Figure 4, for example, for the "driftwood" with identification information "B0003" transmitted from the ship "A1", information regarding the attitude and movement cannot be detected and is unknown (indicated by "-" in the figure). In addition, the attitude and movement direction information is expressed as a numerical value up to 360, with any of the east-west-north-south directions being 0°, but is not limited to this. In addition, in the illustrated example, the size is expressed in three levels: "large," "medium," and "small," but this is not limited to this and may be two levels or less or four levels or more, the levels may be expressed numerically, or the width (maximum width, diameter, etc.), height, area, volume, etc. may be expressed numerically. This information can be estimated by known methods by analyzing image data acquired by the first vessel. Such analysis may be performed by the first vessel, or by an information processing device that receives the image data from the first vessel.

The control unit 120 may be configured to repeatedly update the stored information at predetermined intervals, regardless of whether reception processing is being performed. For example, if the detection target is moving, the control unit 120 may update the location information by predicting and storing the current location based on the elapsed time from a past time.

Then, the control unit 120 executes a process for determining the target ships (S103). The target ships may be all ships within the communication range, or only some of the ships. The "some ships" may be, for example, ships that have transmitted a transmission request, but are not limited to this, and may be only ships that satisfy other predetermined conditions. The other conditions may be, for example, ships that have transmitted information about their own ship (ship type, position, shipowner information, destination information, etc.), or only ships that have been registered with the system's prior approval for communication. In this case, the transmitted information may be assigned an identifier or encrypted so that only specific ships can obtain the information. Such conditions for determining the target ships are stored in the storage unit in advance.

Furthermore, the control unit 120 executes a process of transmitting output information to a specified second ship (S104). The control unit 120 can transmit the detection target information received so far to, for example, the ship (second ship) determined in the process of determining the transmission target ship (S103). The output information to be transmitted may be the detection target information received from all first ships transmitted in its original data format, or may be transmitted in a predetermined data format. In addition, various data generated based on the detection target information received from the first ship may be transmitted.

The transmission process may be repeated each time the reception process or storage process is performed, each time the storage unit is updated, at regular intervals, or in response to a reception request from the second vessel.

The second ships for which the transmission process is performed may be all ships with which communication is possible, or only some of the ships.

The types of objects to be detected may include other ships, floating objects, people, and fixed objects such as rocks and shorelines. Examples of other ships include fishing boats, water taxis, multipurpose boats, pleasure fishing boats, passenger boats, transportation boats, work boats, fireboats and patrol boats, pleasure yachts, pleasure motorboats, special work boats, tankers, cargo ships, etc. Examples of floating objects include driftwood, seaweed, and marine life. Examples of people include people swimming, surfing, yachting, paddleboarding, etc.

Detection target information is not limited to position information, but may also include posture (direction), presence or absence of movement, movement speed, movement route, type (other ships, floating objects, people, etc.), size information, etc., and various information is associated with identification information (such as a unique ID) assigned to each detection target, and then aggregated and stored. Position information and posture information may also be associated with time information and stored. Furthermore, various information such as position information may include not only the time when the information was acquired (such as the current time at the time of detection), but also predicted position information a predetermined time later. Size information may include at least one of the maximum diameter (m) in a planar view, area (m^2), volume (m^3), and height from the water surface (m).

The control unit 120 may transmit at least one of the detection target information and the output data in response to the transmission request information received from the second vessel. For example, the second vessel transmits transmission request information (signal) via wireless communication, satellite communication, the Internet, etc. to the information processing device to request information about the detection target. When the information processing device receives the transmission request information, it can transmit the detection target information in a similar manner to the second vessel that transmitted the transmission request information.

In this embodiment, the transmission request information includes the second ship's own ship position information, and the control unit 120 may transmit information about the detection target located within a predetermined distance range from the second ship's own ship position to the second ship. Specifically, the control unit 120 can extract detection targets located within a predetermined radius (100 m, 200 m, 500 m, 1 km, etc.) centered on the position information (coordinate information) of the second ship from the information in the storage unit to generate output data. In addition, the output data may include all information about the detection targets included in the range, or may be composed of only a portion of the information. For example, of the detection target information shown in FIG. 4, it may be composed of only position information, or may be composed of only type and position information, or may include other information such as image data. The transmission request information may include time information (for example, the current time when the second ship transmitted the transmission request signal). In that case, the control unit of the information processing device may generate output data based on the time information. For example, only information detected within a specified range from the time (within 1 minute, within 10 minutes, within 30 minutes, within 1 hour, etc.) may be extracted and transmitted to the second vessel. Alternatively, the position of each detection target shown in FIG. 4 may be predicted based on the moving speed and moving direction according to the elapsed time from the detection time of each detection target to the transmission request time of the second vessel, and output data including the predicted position may be generated and transmitted (position prediction processing). In this way, the control unit of the information processing device can generate information corresponding to the time information included in the transmission request information and transmit it to the second vessel.

The control unit 120 may generate output data that reflects the detection target information on a pre-stored map, and transmit the output data to the second vessel in a transmission process. In this case, the map data may be displayed on a display unit such as a monitor installed on the second vessel, making it easy to grasp the position of the detection target. Note that the map data is not limited to position information of the detection target, and various information such as type, size, movement direction, movement speed, etc. may be displayed in text or figures (icons, etc.). This makes it possible to confirm at a glance where and what type of detection target is present. Furthermore, map data similar to the output data may be generated by reflecting position information, etc. included in the detection target information received from the information processing device on map data pre-stored in the second vessel.

In this embodiment, the detection target information includes speed information (moving speed) and heading information (moving heading) of the detection target, and the control unit may generate output data including predicted position information of the detection target predicted based on the speed information and heading information. This makes it possible to transmit the movement of the detection target to the second ship, thereby improving safety and increasing the navigation efficiency of the second ship. That is, for example, if the movement direction of the detection target is a direction away from the second ship, there is no need to forcibly avoid it, and there is no need to make an unnecessary detour, so the movement efficiency of the second ship can be improved.

In this embodiment, the control unit may further execute a matching process to integrate overlapping detection target information by comparing the multiple pieces of detection target information received, and transmit output data based on the information after the matching process to the second vessel. With this configuration, for example, when the same detection target is detected by multiple first vessels, it is possible to avoid the risk of misidentifying that there are multiple detection targets. Information regarding the conditions of the matching process is stored in advance in the memory unit.

In this embodiment, the control unit may determine that two detection targets are the same detection target when the position information of the two detection targets is within a predetermined distance in the matching process. For example, the control unit may determine that "B0001" and "B0005" in FIG. 4 are the same detection target when the position information of these targets is within a predetermined distance range. In this case, "B0005" with the latest time information may be given priority and kept, and "B0001" may be deleted and updated.

In this embodiment, the control unit may determine that two detection targets are the same detection target when the position information and type information of the detection target satisfy a predetermined condition in the matching process. For example, the control unit may determine that "B0001" and "B0005" in FIG. 4 are the same type and the position information is within a predetermined distance range as the same detection target. In this case, "B0005" with the latest time information may be given priority and may be updated by deleting "B0001". By adding not only position information but also type information to the matching conditions, more accurate matching process is possible. Any information such as size information, moving speed, moving direction, etc. can be added to the conditions, and the combination can be changed as desired.

In this embodiment, the control unit may store the received position information of the detection target in association with time information during the storage process. The time information may include, for example, as shown in FIG. 4, either the detection time when the detection target was detected by the first vessel or the reception time when the information processing device received the detection target information, or both.

In this embodiment, the detection target may include floating objects, people, or other objects such as fixed objects. In this way, it is possible to obtain information about obstacles that cannot be obtained by AIS, etc.

In this embodiment, an information processing method is provided in which an information processing device executes a reception process for receiving detection target information from multiple sailing first ships, the detection target information including position information of a specific detection target detected by the detection unit of each first ship, a storage process for aggregating and storing the received detection target information, and a transmission process for transmitting at least one of the received detection target information and specific output data generated under predetermined generation conditions using the detection target information to a specific second ship.

In this embodiment, a program is provided that causes the control unit to execute a reception process for receiving detection target information from multiple sailing first ships, the detection target information including position information of a specific detection target detected by the detection unit of each first ship, a storage process for aggregating and storing the received detection target information, and a transmission process for transmitting at least one of the received detection target information and specific output data generated under predetermined generation conditions using the detection target information to a specific second ship.

In this embodiment, the detection target information may include information (including error information) on the accuracy of the detection unit. For example, the accuracy information is expressed as 0 to 100% (or 0 to 1), and the control unit of the information processing device can store the accuracy information and generate output information including the accuracy information. For example, when detection target information with different accuracy information is obtained for the position of the same detection target, the position information with the higher accuracy value may be adopted and stored, or a weighting corresponding to the accuracy value may be performed and a position between multiple detection target information may be calculated as the position information of the detection target. For example, based on first position (x1, y1) information with a sensor detection accuracy of 0.6 and second position (x2, y2) information with a sensor detection accuracy of 0.3, the control unit may estimate the position coordinates of the detection target as the position of the detection target, which is closer to the first position by the percentage of the difference in accuracy than the midpoint between the first position and the second position. Such estimation conditions are stored in the storage unit in advance.

In this embodiment, the detection target information may include information on the type (category) of the detection unit, such as a sensor or a camera. The control unit of the information processing device can estimate the accuracy from the information on the type of the detection unit. Then, output information can be generated based on the accuracy in the same manner as described above.

In this embodiment, the information transmitted from the first vessel may include wind speed and direction obtained by the first vessel (own vessel), estimated wave height, wavelength, tidal current, etc., and this information can also be shared by transmitting it to the second vessel. The second vessel can obtain various detection information that cannot be obtained by the own vessel, enabling safer and more efficient navigation planning.

In this embodiment, the detection target information is not limited to the first vessel, but may be information detected by sensors installed on land such as in a harbor or on a fixed buoy, or collected from the air by a mobile object such as a drone. In other words, the information can be received from any device equipped with a detection unit and a transmission unit, and shared (transmitted) to the second vessel, etc.

In this embodiment, the second vessel may be only a vessel that does not have a detection unit for other vessels, etc., or in which the detection unit is not functioning. This can increase safety, particularly for vessels that cannot detect the positions of other vessels, etc.

In this embodiment, the current position, destination position, intermediate positions, the scheduled time of arrival at each position, information on the navigation route, etc. may be received from the first vessel and transmitted to the second vessel.

Here, we will explain an example of a first ship (e.g., ships A1 and A2) that can communicate with the information processing device. The second ship may have the same functions as the first ship. In other words, it may be possible to display information on the object to be detected received from the information processing device, or to generate a travel route based on the information on the object to be detected.

The first vessel is equipped with a detection unit such as a sensor and a camera, and can detect the position of surrounding objects (detection targets). For example, the vessel is equipped with a camera, and by analyzing an image captured by the camera, the vessel can estimate the relative positions (three-dimensional coordinates in a camera coordinate system with the camera as the origin) of the vessel, floating objects, people, etc. contained in the image relative to the vessel (e.g., the camera) using known technology. The vessel also has a vessel position estimation function, and can estimate the position of the detection target in real space (world coordinate system) based on the vessel's position information and the position information of the target relative to the vessel. The detection target position estimation may be performed using other known methods. The vessel's position information may be estimated as the current position using coordinate information as positioning data acquired by a GNSS module, for example, or may be estimated by a self-position estimation process based on a comparison between map data (three-dimensional point cloud data) acquired from Lidar and map information stored in a memory unit.

The ship can store the information detected by the detection unit in association with information on the time of detection. It can also transmit the detection information and the detection time information to an information processing device.

The ship may receive and acquire various information including the ship's own position information (ship type, size, attitude, movement speed, movement direction, route information, etc.) of other ships, and may transmit the position information of the other ships to the information processing device. In this way, not only detection information but also received position information of other ships can be transmitted to the information processing device.

Figure 5 shows an example of an obstacle estimation system (hereinafter also simply referred to as the "system") provided on the first vessel according to this embodiment. The system of this embodiment can be installed on any type of vessel, and is particularly suitable for relatively small vessels. Specifically, the system of this embodiment can be adopted in fishing boats, water taxis, small multipurpose vessels, pleasure fishing boats, passenger boats, transportation boats, work boats, fireboats/patrol boats, pleasure yachts, pleasure motorboats, special work vessels, etc., and can also be adopted in large passenger ships, tankers, cargo ships, etc.

The system of this embodiment is an obstacle estimation system 1 for detecting obstacles (detection targets) when a ship is navigating. This system 1 includes, for example, a data acquisition unit 10, a control unit 20, a memory unit 30, an input unit 40, an output unit 50, a communication unit 60, and a ship operation control unit 70. Each component is connected by wire or wirelessly and can communicate with each other. In this example, the control unit 20, the memory unit 30, the input unit 40, the output unit 50, and the communication unit 60 are implemented in an information processing device (computer), but this is not limited to the above and each component may be independent.

The data acquisition unit 10 includes a plurality of data acquisition means, such as various sensors and cameras, and acquires various sensor data and image data. The data acquisition unit may include, for example, a camera, a radar device such as Lidar (Light detection and ranging), a marine radar, a millimeter wave radar, an ultrasonic ranging sensor, a GNSS module, a QZSS module, and other positioning devices, an AIS, an inertial measurement unit (IMU), an inertial sensor, a wind direction sensor, a wind speed sensor, a speed sensor (ground speed sensor, water speed sensor), a direction sensor, an acceleration sensor, a gyro sensor, a magnetic compass, a satellite compass, a temperature sensor, a humidity sensor, a pressure sensor, an altitude sensor, an infrared sensor, and the like. The data acquisition unit 10 can acquire information on the ship's current position (coordinates), attitude (direction such as bow direction), moving speed, moving direction, bow direction and turning amount, information on the surrounding environment, and information on obstacles such as other ships and people. The AIS is a system for transmitting and receiving various information between ships and between ships and land facilities, and can exchange ship information such as the position, attitude (course), moving speed, and destination of each ship by wireless communication. The direction sensor may be a magnetic direction sensor that uses geomagnetism to calculate the bow direction, a gyrocompass, a GPS compass, etc.

The control unit 20 includes a processor (arithmetic device) such as a CPU, and is configured to be able to execute various information processes based on the programs stored in the storage unit 30. The control unit 20 can execute ship information estimation processing to estimate ship position information and ship attitude information based on data (first data) from the data acquisition unit 10. The ship position information can be estimated as the current position using coordinate information as positioning data acquired by the GNSS module. The ship position information may be estimated by a self-position estimation processing based on a comparison between map data (three-dimensional point cloud data) acquired from Lidar and map information stored in the storage unit. The ship position information may be expressed by two-dimensional or three-dimensional coordinate information on a specific coordinate system, and may be expressed in latitude and longitude. The attitude of the ship is expressed by orientation (angle) information in a specific two-dimensional or three-dimensional coordinate system. The attitude information may be expressed in east-west-north-south orientation.

The control unit 20 can execute a ship information correction process to correct the ship's position information and the ship's attitude information based on the data (second data) from the data acquisition unit 10. The control unit 20 can correct the ship's position information and the ship's attitude information based on speed data acquired from, for example, Lidar or a camera.

The control unit 20 can execute a speed estimation process to estimate the ship's speed based on data from the data acquisition unit 10. For example, it can calculate the distance traveled in a given time from position information from a given time ago (0.01 seconds, 0.1 seconds, 1 second, etc.) and current position information, and estimate the speed from the information on the distance traveled in the given time.

The control unit 20 can execute an obstacle information estimation process that estimates the position and size of an obstacle based on the ship's position information and ship's attitude information corrected by the ship's information correction process, and the data (third data) from the data acquisition unit 10. The control unit 20 can also estimate the type of obstacle based on the ship's position information and ship's attitude information corrected by the ship's information correction process, and the data (third data) from the data acquisition unit 10.

The control unit 20 can estimate the type of obstacle by, for example, selecting one of the candidate types of obstacles stored in advance in the memory unit 30. Information on the type of obstacle can include type information such as ships, specific ship classifications (fishing boats, water taxis, multipurpose boats, recreational fishing boats, passenger boats, transportation boats, work boats, fireboats/patrol boats, pleasure yachts, pleasure motorboats, special work boats, etc.), marine structures, people (including swimming, surfing, yachts, paddle boards, etc.), rocks, shores, driftwood, seaweed, marine life, etc. Information on the type of obstacle is stored in association with size information (area, height).

The control unit 20 can estimate the position, posture, type, and size of the obstacle by, for example, image analysis of image data acquired by the camera. The control unit 20 may estimate the position, posture, type, and size of the obstacle using a machine learning technique using a learning model that has been trained in advance in the obstacle information estimation process. When the control unit 20 estimates the position of the obstacle based on the image data acquired by the camera, after estimating the three-dimensional position (coordinates) of the obstacle in the camera coordinate system, the camera coordinate system can be converted to a world coordinate system by a coordinate conversion process based on the camera position (coordinates) and shooting direction information, and the position (coordinates) of the obstacle on the world coordinate system can be calculated. The control unit 20 can estimate the bow direction of the ship as an obstacle, the direction of a person's face (front direction), and the direction (direction of travel) of an object used such as a surfboard by image analysis of the image data from the camera or analysis of three-dimensional data (point cloud data, model data, etc.). The control unit 20 can estimate the three-dimensional position and orientation of obstacles on or underwater based on sensor data from a radar device, Lidar, etc., or a combination of multiple sensors, rather than just image data from the camera described above.

The storage unit 30 may store candidates for types of obstacles and the corresponding size (information on area and height in plan view) and size range (upper limit, lower limit) in advance in association with each other. For example, the range of the area of a ship may be 3 m^2 or more and 9200 m^2 or less, and the range of the height of the ship (height above the water surface) may be 0.5 m or more and 100 m or less. The range of the area of a person may be 0.2 m^2 or more and 5 m^2 or less, and the range of the height of a person (height above the water surface) may be 0.1 m or more and 3 m or less. The storage unit 30 stores information on the appropriate avoidance distance associated with obstacle information such as the position, type, and size of the obstacle. The storage unit 30 may also store information on a calculation formula for calculating the appropriate avoidance distance. For example, the appropriate avoidance distance may be calculated according to the size (area in plan view) of the ship as an obstacle. Furthermore, the parameters used in the calculation may include information such as the type of obstacle, the speed of movement, and the direction of movement. The memory unit 30 may include information on a program that simulates movement along the ship's navigation route (planned route) and whether or not it will come into contact with an obstacle. The memory unit 30 stores information on each data acquisition means. Information on the data acquisition means may be, for example, the position of the data acquisition means on the ship (e.g., a position relative to a specific point such as the center or center of gravity of the ship), a relative attitude (e.g., a direction relative to the bow direction of the ship), etc., but is not limited to this.

The control unit 20 may estimate the moving speed, moving direction, and attitude of an obstacle based on the ship's position information and ship's attitude information corrected by the ship's information correction process, the ship's moving speed information, moving direction information, and data (third data) from the data acquisition unit 10. The control unit 20 can acquire moving speed information and moving direction information of obstacles such as other ships relative to the ship, based on data from Lidar. The control unit 20 can also acquire moving speed information, moving direction information, etc. of obstacles using data from a radar device.

In the obstacle information estimation process, the control unit 20 can analyze the image data acquired by the camera to estimate information such as the type, size, position, shape, and color of the obstacle. The control unit 20 may also estimate the information of the obstacle based on any information of the obstacle estimated from the image data and the obstacle-related information pre-stored in the storage unit. For example, the control unit 20 may estimate the size of the obstacle from the image data, and refer to the storage unit to determine (estimate) a candidate obstacle associated with a size range that matches the size as the type of the obstacle. The control unit 20 may also analyze the image data acquired by the camera to estimate the type of the obstacle, and refer to the storage unit to determine (estimate) a size candidate associated with the type as the size of the obstacle. The type, size, and position of the obstacle may be estimated based on any one or a combination of detection data acquired from a radar device and detection data from Lidar, without being limited to image data acquired by the camera. For example, 3D data of an obstacle can be generated from distance data to multiple points on the obstacle using Lidar, and the 3D shape, type, size, position, etc. of the obstacle can be estimated based on the 3D data. In this way, the control unit 20 can select from among the obstacle candidates by comparing the information of the obstacle candidates stored in the memory unit 30 with the data on the obstacle acquired from the data acquisition unit 10, and estimate the type, size, etc. of the detected obstacle. The control unit 20 can also estimate the position and size of the obstacle based only on the data from the data acquisition unit 10.

The control unit 20 can store various information such as generated information in the memory unit 30, update the information in the memory unit 30, output the information from the output unit 50, and transmit the information to external devices, other ships, etc. via the communication unit 60. The control unit 20 can also accept information acquired by the data acquisition unit 10, user operation information input via the input unit 40, and information received by the communication unit 60, and execute information processing based on the information.

The memory unit 30 can store various information stored in advance, information acquired from the data acquisition unit 10, information received from external devices or other ships via the communication unit 60, information generated by the control unit 20, information input via the input unit 40, etc., and is composed of a non-volatile memory, a hard disk, etc. The memory unit 30 can store information on various parameters indicating the shape of the ship (3D model data, etc.), characteristics (characteristics related to output, speed, and turning amount), information on obstacles, 2D or 3D map information, navigation history information, navigation plan information, tidal information, etc.

The input unit 40 accepts input information based on operations from the user. The input unit 40 is composed of mechanical buttons, switches, operating levers, touch panels, etc. The input unit 40 may also be equipped with a microphone that allows voice input.

The output unit 50 outputs various information in the form of images (video), audio, etc. The output unit 50 may include, for example, a display unit such as an LCD monitor or touch panel that displays images, an audio output unit such as a speaker that displays audio, a vibration generating unit (vibration device) that generates vibrations, etc.

The communication unit 60 is connected to a network such as the Internet or wireless communication, and can transmit data to external devices (such as a management server, a smartphone or tablet terminal used by a user) or other ships, and can receive data from external devices.

The vessel operation control unit 70 may include an auto-rudder that controls steering, an auto-throttle that controls the power unit (engine, motor, etc.) for rotating the propeller or thruster, etc. The vessel operation control unit 70 can control the operation of the vessel 70 (movements such as forward movement, backward movement, left and right movement, turning, etc.) based on instruction information from at least one of the control unit 20, the memory unit 30, the input unit 40, and the communication unit 60, and can navigate the vessel along a predetermined route.

Figure 6 shows an example of the present system installed on ship A. In the example of Figure 6, the system includes a lidar, a camera, a GNSS antenna, an AIS antenna, and a wind speed and direction meter as the data acquisition unit 10, a control unit as the control unit 20, a monitor as the output unit 50, and an auto-rudder and auto-throttle as the ship operation control unit 70. In the example of Figure 6, the lidar and camera are installed on the front (bow side) and rear (stern side) of the ship 100, respectively, allowing data around the ship 100, particularly the front and rear sides, to be detected with high accuracy. The configuration of the present system installed on the ship is not limited to the illustrated example, and other sensors, etc. may be included.

Figure 7 shows an example of a process flow for the symmetry detection method in the system of this embodiment.

The control unit 20 executes a ship information estimation process (S201) for estimating ship position information and ship attitude information based on first data acquired from at least one data acquisition means, a ship information correction process (S202) for correcting ship position information and ship attitude information based on second data acquired from at least one data acquisition means, an obstacle information estimation process (S203) for estimating the position and size of an obstacle based on the ship position information and ship attitude information corrected in the ship information correction process and third data acquired from at least one data acquisition means, and an output information generation process (S204) for generating output information based on the obstacle position and size information estimated in the obstacle information estimation process. The first data, second data, and third data may each include data acquired from a common data acquisition means. Furthermore, the first data, second data, and third data may each include data acquired from one data acquisition means or from multiple data acquisition means. Note that the ship information correction process is not required, and the obstacle information estimation process may be performed after the ship information estimation process.

In the ship information estimation process (S201), for example, the ship's three-dimensional position is estimated based on coordinate information from a GNSS antenna as a data acquisition means, and the ship's three-dimensional attitude information is estimated based on orientation data acquired from an inertial measurement unit. The ship information estimation process (S201) may be performed based on data acquired from only one data acquisition means, but from the perspective of improving estimation accuracy, it is preferable to perform the estimation process based on data acquired from multiple data acquisition means.

In the ship information correction process (S202), the ship position information and ship attitude information estimated in S101 are corrected based on, for example, speed information acquired from at least one of the Lidar and the camera. The control unit 20 estimates the current position and attitude (a predetermined time after the time) using, for example, the position (coordinates), movement direction information, and movement speed information at a time point a predetermined time ago (0.01 seconds, 0.1 seconds, 1 second, etc.). The estimated position and attitude information is then compared with the ship position information and ship attitude information estimated in the ship information estimation process (S201). If they do not match, the ship position information and ship attitude information can be corrected by adopting the numerical value between the two as accurate current position and current attitude information. Note that the ship information correction process may be performed in a different manner, and for example, the ship position information and ship attitude information may be corrected using the position information and attitude information calculated by the ship position and attitude estimation process based on image data from a camera, or a combination of these. Furthermore, from the perspective of increasing the accuracy of the correction process, it is preferable to perform the correction process based on data acquired from multiple data acquisition means.

Next, in the obstacle information estimation process (S203), for example, the position and attitude of the camera are estimated (the position and attitude of the camera in the ship's coordinate system) based on the ship's position information and ship's attitude information corrected in the ship's information correction process, and the position of the obstacle in the world coordinate system is estimated by coordinate conversion using the relative position data of the obstacle in the camera coordinate system. In addition, the camera image data is analyzed to estimate the type and size of the obstacle. The position, type and size of the obstacle may be estimated using the detection data acquired from the radar device alone or in combination.

In the output information generation process (S204), output information including various information such as the position, type, moving speed, moving direction, size, etc. of the obstacle can be generated and transmitted to the information processing device 100. The transmission process may be repeatedly executed at pre-stored intervals, or may be transmitted each time a new object is detected. The output information may include image data or audio data, and the position of the obstacle may be displayed on a map on the ship's monitor, the type and size of the obstacle may be displayed in text or images, or audio may be output from a speaker to alert the ship. The content of the output information is not particularly limited, and for example, navigation route information for avoiding obstacles and navigating the ship may be generated and transmitted to the information processing device 100, etc., displayed on a monitor, or applied to an autopilot system for automatic navigation. The output information may include the current position, the destination position, the intermediate positions, and the scheduled times of arrival at the destination position and the intermediate positions.

The output information may include information on the distance (appropriate avoidance distance) to be kept away from the obstacle. For example, if the size of the obstacle is small (e.g., 1 m^2 or less), the appropriate avoidance distance may be set small (5 m, 10 m, etc.), and the larger the size of the obstacle (e.g., 10 m^2 or more), the larger the appropriate avoidance distance may be set (20 m, 50 m, etc.). For example, if the obstacle is a person, the appropriate avoidance distance may be set large (20 m, 50 m, etc.), and if it is a floating object such as a buoy, driftwood, or seaweed, the appropriate avoidance distance may be set small (5 m, 10 m, etc.). The appropriate avoidance distance may be determined based on appropriate avoidance distance information pre-stored in the storage unit according to at least one of the type and size of the obstacle. In this case, information on the type and size of the obstacle and information on the appropriate avoidance distance are pre-stored in association with each other. In addition, if the obstacle is another ship, the appropriate avoidance distance may be calculated based on the moving speed and moving direction of the other ship.

In this embodiment, the control unit 20 may perform various processes based on information (including error information) related to the accuracy (precision) of the detection unit (sensor, camera, etc.) previously stored in the storage unit. For example, in the ship position estimation process, only information from a highly accurate sensor with an accuracy value equal to or greater than a predetermined threshold may be used preferentially (information from a low-accuracy sensor with an accuracy value below the threshold is not used), or detection information may be integrated and used based on weighting previously stored in the storage unit. For example, the accuracy information is expressed as 0 to 100% (may be expressed as a decimal between 0 and 1), and the control unit of the information processing device can store or update the accuracy information based on information input from the user. The various processes may include at least one of a process for estimating ship position information, ship attitude information, a process for correcting information, and a process for estimating obstacles. In this way, by using a detection unit with high accuracy preferentially for information processing, the accuracy of various processes such as ship position, ship attitude, and obstacle detection can be improved. For example, when the control unit 20 has multiple different sensors capable of estimating the ship's position information, the control unit 20 can estimate a position determined based on the weighting of the accuracy information between the positions detected by the multiple sensors. When detection target information with different accuracy information is obtained for the position of the same detection target among the multiple detected (estimated) ship's position information, the position information with the higher accuracy value may be adopted and stored, or a weighting corresponding to the accuracy value may be performed and a position between the multiple detection target information may be calculated as the position information of the detection target. For example, based on first position (x1, y1) information with a sensor detection accuracy of 0.6 and second position (x2, y2) information with a sensor detection accuracy of 0.3, the control unit may estimate the position coordinates of the detection target as the position of the detection target, which are closer to the first position than the midpoint between the first and second positions by the percentage of the difference in accuracy. Such estimation conditions are stored in the storage unit in advance.

Although the preferred embodiment of the present disclosure has been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field of the present disclosure can conceive of various modified or revised examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

The device described in this specification may be realized as a single device, or may be realized by multiple devices (e.g., cloud servers) some or all of which are connected via a network. For example, the control unit and the storage unit may be realized by different servers connected to each other via a network.

The series of processes performed by the device described in this specification may be realized using software, hardware, or a combination of software and hardware. A computer program for realizing each function of the control unit according to this embodiment may be created and implemented in a PC or the like. A computer-readable recording medium on which such a computer program is stored may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The above computer program may also be distributed, for example, via a network, without using a recording medium.

In addition, the processes described in this specification using flowchart diagrams do not necessarily have to be performed in the order shown. Some processing steps may be performed in parallel. Also, additional processing steps may be employed, and some processing steps may be omitted.

Furthermore, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology disclosed herein may achieve other effects that are apparent to a person skilled in the art from the description in this specification, in addition to or in place of the above effects.

Note that the following configurations also fall within the technical scope of the present disclosure.
(Item 1)
a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships;
A storage process of aggregating and storing the received detection target information;
a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship.
(Item 2)
2. The information processing device according to item 1, wherein the control unit transmits at least one of the detection target information and the output data in response to transmission request information received from the second vessel.
(Item 3)
the transmission request information includes ship position information of the second ship,
The control unit is
3. An information processing device according to item 2, further comprising: a second vessel configured to transmit information about the detection target located within a predetermined distance range from the second vessel's own position to the second vessel.
(Item 4)
2. The information processing device according to item 1, wherein the control unit generates output data that reflects the detection target information on a pre-stored map, and transmits the output data to the second ship in the transmission process.
(Item 5)
The detection target information includes speed information and traveling direction information of the detection target,
2. The information processing device according to item 1, wherein the control unit generates the output data including predicted position information of the detection target predicted based on the speed information and traveling direction information.
(Item 6)
The control unit further includes:
A matching process is performed to integrate overlapping pieces of the detection target information by comparing the received pieces of detection target information;
2. The information processing device according to item 1, further comprising: a first information processing device configured to transmit output data based on the information after the matching process to a second vessel;
(Item 7)
The control unit is
7. The information processing device according to item 6, wherein in the matching process, when the position information of two detection targets is within a predetermined distance, the detection targets are determined to be the same detection target.
(Item 8)
The control unit is
7. The information processing device according to item 6, wherein in the matching process, when position information of two detection targets and information on the type of the detection targets satisfy a predetermined condition, the detection targets are determined to be the same detection target.
(Item 9)
2. The information processing device according to item 1, wherein the control unit, in the storage process, stores the received position information of the detection target in association with time information.
(Item 10)
2. The information processing device according to item 1, wherein the detection target includes other ships.
(Item 11)
2. The information processing device according to item 1, wherein the detection target includes floating matter.
(Item 12)
2. The information processing device according to item 1, wherein the detection target includes a person.
(Item 13)
a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships;
A storage process of aggregating and storing the received detection target information;
An information processing method in which an information processing device executes a transmission process to transmit at least one of the received detection target information and specified output data generated using the detection target information under predetermined generation conditions to a specified second ship.
(Item 14)
a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships;
A storage process of aggregating and storing the received detection target information;
A program that causes a control unit to execute a transmission process to transmit at least one of the received detection target information and specified output data generated using the detection target information under predetermined generation conditions to a specified second ship.

REFERENCE SIGNS LIST 1 Travel path generation system 10 Data acquisition unit 20 Control unit 30 Storage unit 40 Input unit 50 Output unit 60 Communication unit 70 Ship operation control unit

Claims (12)

a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships;
A storage process of aggregating and storing the received detection target information;
a control unit that executes a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship,
The control unit transmits at least one of the detection target information and the output data in response to transmission request information received from the second vessel,
the transmission request information includes ship position information of the second ship,
The control unit is
An information processing device that transmits information about the detection target located within a predetermined distance range from the second ship's own position to the second ship . The information processing device according to claim 1, wherein the control unit generates output data that reflects the detection target information on a pre-stored map, and transmits the output data to the second vessel in the transmission process. The detection target information includes speed information and traveling direction information of the detection target,
The information processing device according to claim 1 , wherein the control unit generates the output data including predicted position information of the detection target predicted based on the speed information and the traveling direction information. The control unit further
A matching process is performed to integrate overlapping pieces of the detection target information by comparing the received pieces of detection target information;
The information processing device according to claim 1 , further comprising: a first information processing unit configured to transmit output data based on the information after the matching process to the second vessel. The control unit is
The information processing apparatus according to claim 4 , wherein, in the matching process, when the position information of two detection targets is within a predetermined distance, the detection targets are determined to be the same detection target. a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships;
A storage process of aggregating and storing the received detection target information;
a control unit that executes a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship,
The control unit further includes:
A matching process is performed to integrate overlapping pieces of the detection target information by comparing the received pieces of detection target information;
Transmitting output data based on the information after the matching process to the second vessel;
The control unit is
The information processing device determines that two detection targets are the same detection target when position information of the two detection targets and information on types of the detection targets satisfy a predetermined condition in the matching process. The information processing device according to claim 1, wherein the control unit stores the received position information of the detection target in association with time information during the storage process. The information processing device according to claim 1, wherein the detection target includes other ships. The information processing device according to claim 1, wherein the detection target includes floating objects. The information processing device according to claim 1, wherein the detection target includes a person. a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships;
A storage process of aggregating and storing the received detection target information;
a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship ;
the control unit of the information processing device transmits at least one of the detection target information and the output data in response to transmission request information received from the second vessel; and
the transmission request information includes ship position information of the second ship,
The control unit is
An information processing method for transmitting information about the detection target located within a predetermined distance range from the second ship's own position to the second ship . a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships;
A storage process of aggregating and storing the received detection target information;
a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second vessel ;
The control unit transmits at least one of the detection target information and the output data in response to transmission request information received from the second vessel,
the transmission request information includes ship position information of the second ship,
The control unit is
A program that transmits information about the detection target located within a predetermined distance range from the second ship's own position to the second ship .

Images