JP7725301B2 - Tubular structure investigation support device, tubular structure investigation support system, method for determining pipe type of tubular structure, and program - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies [Website address] Just Co., Ltd. product page https://www. just-ltd. co. jp/project/smakan/ [Publication date] July 26, 2021 [Publications, etc.] [Website address] Sewerage Exhibition '21 Osaka exhibitor introduction page https://www. gesuidouten. jp/exhibitors/info/? evcid=GT14J0456 [Publication date] July 26, 2021 [Publications, etc.] [Event name] Sewerage Exhibition '21 Osaka [Date held] August 17th - 20th, 2021

The present invention relates to a tubular structure inspection support device, a tubular structure inspection support system, a method for determining the pipe type of a tubular structure, and a program, and more particularly to determining the pipe type of a tubular structure such as a sewer pipe.

Inspections are conducted to check for damage and the condition of tubular structures such as water supply and sewerage pipes, cable pipes, and tunnels. During these inspections, a camera that travels inside the pipe is used to photograph the inner wall surface along the pipe, and image data of the photographed inner wall surface is used to create an unfolded image on a computer. Workers then visually inspect the unfolded image to determine damage and prepare an inspection report.

For example, Patent Document 1 describes an intra-pipe work device monitoring system consisting of an intra-pipe work device that can move within a pipeline and a ground device equipped with a monitor, in which the ground device displays an image showing the current position of the intra-pipe work device on an expanded image created based on image data of the inner wall of the pipeline.

Non-patent document 1 also describes a sewerage report creation system for creating sewer pipe inspection reports in a specified format.

Publication No. 2010-066070

Nozawa Electronics Co., Ltd., "Sewerage Report Creation System," [online], searched August 3, 2021, Internet, <URL:http://nozawa-densi.sakura.ne.jp/CCP004.html>

However, before workers can assess damage to sewer pipes and other structures, they need to know the pipe type (the material the pipe is made of). While the pipe type can be determined from local government ledger information, repairs and other work can sometimes result in partial or complete changes to the pipeline, which can lead to discrepancies with the ledger information.

The present invention was made in consideration of the above-mentioned problems, and aims to provide a tubular structure inspection support device that can determine the type of pipe, such as a sewer pipe, thereby reducing the workload for damage assessment work and report preparation work when inspecting tubular structures, and improving work efficiency.

The first invention for solving the above-mentioned problems is a tubular structure inspection support device comprising an image acquisition means for acquiring direct-view images, which are images taken with a wide-angle camera while moving inside a tubular structure in the pipeline direction, a pipe type determination means for determining the pipe type of the tubular structure based on the direct-view images acquired by the image acquisition means, and an output means for outputting the determination result by the pipe type determination means, wherein the pipe type determination means determines the pipe type by image recognition processing using artificial intelligence .

The tubular structure inspection support device of the first invention can determine the pipe type (pipe material) based on direct-view images taken of the interior of a tubular structure in the pipe line direction and output the determination results. This allows workers to accurately identify the pipe type, which is the basis for damage assessment criteria, improving work efficiency in damage assessment tasks and report preparation tasks during tubular structure inspections. Furthermore, because the determination is made based on direct-view images, which are smaller and contain more information than unfolded images, accurate determination results can be obtained. Furthermore, by using artificial intelligence (AI), there is no need for training data or preprocessing of the images to be determined (adjusting image size, brightness, etc.), resulting in efficient and accurate determination results.

In the first invention, the pipe type determination means determines the pipe type based on multiple frames of the direct-view image. The pipe type determination means also determines the pipe type by taking a majority vote of the determination results of the multiple frames. By determining multiple frames, erroneous determinations due to dirt, damage, etc. can be prevented, and determination accuracy can be improved.

It is also desirable that the pipe type determination means determine the direct-view image of the entire pipeline. Furthermore, it is desirable that the pipe type determination means detects locations where the pipe type has changed based on the determination results. This makes it possible to check the pipe type throughout the entire pipeline and detect changes in pipe type along the pipeline.

The second invention is a tubular structure inspection support system comprising an imaging device that photographs the inside of a tubular structure with a wide-angle camera while moving in the pipeline direction, and a tubular structure inspection support device connected to the imaging device via a network, wherein the tubular structure inspection support device comprises an image acquisition means that acquires a direct-view image that is an image captured by the imaging device, a pipe type determination means that determines the pipe type of the tubular structure based on the direct-view image acquired by the image acquisition means , and an output means that outputs the determination result by the pipe type determination means, wherein the pipe type determination means determines the pipe type by image recognition processing using artificial intelligence .

According to the tubular structure inspection support system of the second invention, a photographing device captures direct-view images of the interior of a tubular structure, a tubular structure inspection support device acquires the direct-view images, and the tubular structure inspection support device determines the pipe type of the tubular structure based on the acquired direct-view images and outputs the determination results. This allows workers to accurately know the pipe type, which is the basis for damage determination criteria, and improves work efficiency in damage determination tasks and report preparation tasks in tubular structure inspections. Furthermore, since the determination is made based on direct-view images, which are smaller and contain more information than unfolded images, accurate determination results can be obtained. Furthermore, by using artificial intelligence (AI), there is no need for training data or preprocessing of the images to be determined (adjusting image size, brightness, etc.), and this is efficient and allows for accurate determination results.

A third invention is a pipe type determination method executed by a computer, which includes the steps of acquiring a direct-view image, which is an image taken with a wide-angle camera while proceeding inside a tubular structure in the pipeline direction, determining the pipe type of the tubular structure based on the acquired direct-view image, and outputting the determination result , wherein the step of determining the pipe type of the tubular structure determines the pipe type by image recognition processing using artificial intelligence .

According to the third invention, a computer can be used to determine the pipe type (pipe material) based on direct-view images of the interior of a tubular structure taken in the direction of the pipe, and the determination results can be output. This allows workers to know the pipe type, which is the basis for damage determination criteria, and improves work efficiency in damage determination tasks and report preparation tasks during tubular structure inspections. Furthermore, because direct-view images, which are smaller and contain more information than unfolded images, are used as the target of determination, accurate determination results can be obtained. Furthermore, by using artificial intelligence (AI), there is no need for preprocessing of learning data or images to be determined (adjusting image size, brightness, etc.), and this is efficient and can produce accurate determination results.

A fourth invention is a program for causing a computer to function as an image acquisition means for acquiring a direct-view image, which is an image captured by a wide-angle camera while moving inside a tubular structure in a pipeline direction, a pipe type determination means for determining the pipe type of the tubular structure based on the direct-view image acquired by the image acquisition means, and an output means for outputting a determination result by the pipe type determination means,
The pipe type determination means is a program that determines the pipe type by image recognition processing using artificial intelligence .

The fourth invention allows a computer to function as the tubular structure inspection support device of the first invention.

This invention makes it possible to determine the type of pipe from images taken during the inspection of sewer pipes, etc., thereby providing a tubular structure inspection support device that reduces the workload involved in damage assessment and report preparation work during the inspection of tubular structures, and improves work efficiency.

FIG. 1 is a diagram showing an example of the overall configuration of a tubular structure inspection support system 1. A block diagram showing the functional configuration of the imaging system 2 and the tubular structure inspection support device 5. Flowchart showing the overall flow of the investigation process Flowchart showing the flow of display processing FIG. 10 is a diagram showing an example of a main screen 7. An example of a grid display Flowchart showing the flow of pipe type determination processing Example of pipe type display FIG. 10 is a diagram showing a display example of objects 971 and 972 indicating tubular structures (joints). Flowchart showing the flow of damage input processing A diagram showing an example of specifying points for damaged areas An example of specifying a box (range) for a damaged area Flowchart showing the preview process FIG. 1 shows an example of a preview screen 15.

The following describes in detail a preferred embodiment of the present invention with reference to the drawings.

Figure 1 is a diagram showing the overall configuration of a tubular structure inspection support system 1 according to the present invention. In the following explanation, as shown in Figure 1, a tubular structure inspection support system 1 will be described in which an imaging system 2, storage 3, and tubular structure inspection support device (hereinafter, inspection support device) 5 are communicatively connected via a network 4. Note that the system configuration example shown in Figure 1 is merely an example, and the present invention is not limited to this. For example, the storage 3 may be omitted, and the imaging system 2 and inspection support device 5 may be connected via the network 4. Alternatively, the imaging system 2 and inspection support device 5 may not be communicatively connected via the network 4, and image data and the like may be exchanged via a recording medium or the like.

The photography system 2 includes a photography device 22 that photographs the interior of the tubular structure 10 (such as a water supply/sewerage pipe, intake/exhaust pipe, cable pipe, or tunnel; hereinafter referred to as "pipe 10") to be inspected using a wide-angle camera 22A while traveling along the pipe, and an on-site PC 21, a computer terminal used at the inspection site. The photography device 22 is a traveling vehicle equipped with a wide-angle camera 22A and an encoder. The encoder is a cable or other device used to measure the distance from the pipe entrance to the wide-angle camera 22A. The on-site PC 21 includes an interface for acquiring images captured by the photography device 22 (hereinafter referred to as "direct-view images") and the distance measured by the encoder, a communications interface for connecting to the network 4, a control unit (CPU, ROM, RAM), a memory unit, an input unit, a display unit, etc. It is desirable for the on-site PC 21 to have the ability to generate unfolded images based on the direct-view images acquired from the photography device 22, but the generation of unfolded images may also be performed by a separate computer terminal.

An unfolded image is an image obtained by cutting an image of the inside of the pipe 10 in the pipeline direction and unfolding it into a flat surface. Methods for creating unfolded images are known, and are described, for example, in JP 2010-066070 A. The imaging system 2 can also be configured using the work equipment and ground equipment shown in the above-mentioned patent document, or similar known imaging systems. The on-site PC 21 records the distance information obtained by the encoder (the distance from the pipe entrance to the wide-angle camera 22A) as position information (position in the pipeline direction) in the captured image (direct-view image) and unfolded image.

The on-site PC 21 stores the captured direct-view images in storage 3 via network 4. When the on-site PC 21 generates an unfolded image, it associates the generated unfolded image with the direct-view image and stores it in storage 3. Alternatively, the on-site PC 21 may transmit the direct-view image and unfolded image to the investigation support device 5 via network 4. Alternatively, the on-site PC 21 may record the direct-view image and unfolded image on a recording medium.

Storage 3 is a storage server accessible via network 4, and has a memory area for saving direct-view images, unfolded images, construction information, etc. sent from the site PC 21. When storage 3 receives an image acquisition request from the investigation support device 5, it responds to the request by sending the corresponding direct-view images or unfolded images.

Next, the investigation support device 5 will be described. As shown in FIG. 2, the investigation support device 5 is configured from a computer in which a control unit 51, a memory unit 52, a communication unit 53, an input unit 54, a display unit 55, and a peripheral device I/F (interface) unit 56 are connected via a bus, and a PC, tablet, smartphone, etc. can be used. The configuration of the investigation support device 5 can be modified as appropriate. A tubular structure investigation support program is installed in the investigation support device 5, and the control unit 51 executes processing in accordance with the tubular structure investigation support program to realize the functions described below.

The control unit 51 is composed of a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc. The CPU loads programs stored in the memory unit 52, ROM, etc. into the work memory area on the RAM and executes them, driving and controlling each unit connected via the bus (memory unit 52, communication unit 53, input unit 54, display unit 55, peripheral device I/F unit 56). The ROM permanently stores programs such as the boot program and BIOS, as well as data. The RAM temporarily stores loaded programs and data and provides a work area used by the control unit 51 to perform various processes.

The memory unit 52 is a storage device such as flash memory or a hard disk, and stores acquired image data, input damage information, input pipe structure information, etc. The memory unit 52 also stores processing programs related to the functions described below (collectively referred to as "tubular structure inspection support programs" or "apps").

The communication unit 53 has a communication port for a wireless communication unit such as a WiFi antenna or Bluetooth, or a wired communication unit such as a LAN, and a communication control device, and is an interface that mediates communication with external devices.

The input unit 54 includes, for example, a touch panel, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad, and inputs input data to the control unit 51.

The display unit 55 is composed of a display such as an LCD panel and a logic circuit (such as a video adapter) that works in conjunction with the display to perform display processing, and displays display data input under the control of the control unit 51 on the display. The display unit 55 may also be a touch panel display in which an input device (input unit 54) such as a touch panel is integrally provided on the display screen.

The peripheral device I/F unit 56 is a port for connecting peripheral devices, and the control unit 51 sends and receives data to and from peripheral devices via the peripheral device I/F unit 56. The peripheral device I/F unit 56 is configured using a USB (Universal Serial Bus) or similar. The connection with the peripheral device can be wired or wireless.

Next, the functional configuration of the research support device 5 will be described with reference to FIG.
The inspection support device 5 has, as functional units, an image acquisition unit 511, a display processing unit 513, a tubular structure input unit 514, a damage input unit 515, a preview display unit 516, and a pipe type determination unit 518. The inspection support device 5 may also include an unfolded image generation unit 512 that generates an unfolded image. These functional units are realized by the CPU of the control unit 51 reading a processing program (tubular structure inspection support program) stored in the storage unit 52, calling it up into a work memory area on the RAM, and executing it.

The image acquisition unit 511 acquires direct-view images captured by the imaging system 2 and unfolded images generated via the communication unit 53 or the peripheral device I/F unit 56. Alternatively, the image acquisition unit 511 may acquire direct-view images and unfolded images stored on a storage medium by reading them, or the communication unit 53 may acquire direct-view images and unfolded images stored on another computer or on the storage 3 via a network 4 such as a LAN or the Internet, and import them into the control unit 51.

The unfolded image generation unit 512 generates an unfolded image based on the direct-view image acquired by the image acquisition unit 511 and stores it in the memory unit 52. Note that the unfolded image may be generated in real time while the direct-view image is being captured, or immediately after the direct-view image is captured, at the investigation site, and it may be confirmed at the investigation site whether the conversion was appropriate. In this case, the unfolded image is generated by the on-site PC 21, etc.

The display processing unit 513 executes display processing, and displays the direct-view image 72 acquired by the image acquisition unit 511, a partial range of the unfolded image 71, and the overall image 73, which is the unfolded image 71 covering the entire pipe 10, side by side on the display unit 55 (see Figure 5). In the display processing, the display processing unit 513 displays the position of the direct-view image 72 on the pipe 10 in conjunction with the display range of the unfolded image 71, and indicates the above position in the overall image 73 and the unfolded image 71. Details of the display processing and examples of display screens will be described later.

The pipe structure input unit 514 accepts input of information about structures such as joints and attachment pipes of the pipe 10 for images (mainly the unfolded image 71) acquired by the image acquisition unit 511. It then displays objects 971, 972 representing the structures (joints and attachment pipes) on the unfolded image 71 or the overall image 73 (see Figure 9). The pipe structure input unit 514 also adds identification information to each of the structures (joints, attachment pipes, etc.), links them to positional information on the pipe 10, and records them as pipe structure information.

When the user designates a position on the unfolded image 71, the damage input unit 515 displays the damage information input field 111 for inputting damage information (see Figures 11 and 12) and accepts the input of the damage information. The damage information entered in the damage information input field 111 is linked to the position designated by the user (position on the pipe 10) and recorded. The damage position can be designated by a point or a box (range). Details of the processing by the damage input unit 515 and examples of the display screen will be described later.

The preview display unit 516 displays a damage list 150, which is a list of recorded damage information, together with the overall image 73 (see Figure 14). Details of the preview display process and examples of the display screen will be described later.

The pipe type determination unit 518 determines the pipe type (material of the pipe 10) of the pipe 10 based on the direct-view image 72 acquired by the image acquisition unit 511. Because the direct-view image 72 contains more information than the unfolded image 71, it can perform a more accurate determination than determining the pipe type based on the unfolded image 71. The pipe type determination unit 518 determines the pipe type through image recognition processing using AI (artificial intelligence). The pipe type determination unit 518 also determines the pipe type using multiple frames of the direct-view image 72 as the determination target. The pipe type is then determined by taking a majority vote of the determination results for the multiple frames. The pipe type determination unit 518 may also perform pipe type determination for the entire pipeline. After performing pipe type determination for the entire pipeline and detecting a location where the pipe type has been changed, the location where the pipe type has been changed is reflected on the unfolded image 71, for example. The determination result (pipe type) by the pipe type determination unit 518 is stored in the memory unit 52 and is displayed, for example, on the main screen 7, damage information input field 111, preview screen 15, etc.

Next, we will explain the flow of a tubular structure inspection using the tubular structure inspection support system 1. First, we will explain the overall inspection flow with reference to the flowchart in Figure 3.

The worker carries the imaging system 2 to the inspection site and uses the imaging device 22 to capture video of the inside of the pipe 10 (step S101). The captured video (images) are sequentially imported into the on-site PC 21 as direct-view images 72. The on-site PC 21 executes an unfolded image generation process and generates an unfolded image 71 based on the imported direct-view images 72 (step S102).

If the unfolded image 71 needs to be generated again (step S103; No), return to step S101. If the unfolded image 71 was successfully generated (step S103; Yes), store the direct-view image 72 and unfolded image 71 (step S104).

Once the processing of steps S101 to S104 is complete, the work at the investigation site ends and the worker moves to the office (step S105). In the office, the worker performs work using the investigation support device 5.

When the app (tubular structure investigation support processing program) is launched (step S106), the control unit 51 of the investigation support device 5 executes the tubular structure investigation support processing. The control unit 51 acquires direct-view images 72 and unfolded images 71 from storage 3 or the like in accordance with the operator's operations (step S107), and performs display processing (step S108), pipe type determination processing (step S109), tubular structure input processing (step S110), damage input processing (step S111), preview display processing (step S112), and report creation processing (step S113), etc. Note that the order of steps S107 to S113 may be changed or some may be omitted depending on the work content and the operator. The processing of each step is explained below.

First, the display processing of step S108 will be described with reference to Figure 4. The control unit 51 of the research support device 5 displays the main screen 7, which is the main display screen, on the display unit 55, and displays the unfolded image 71, direct-view image 72, and full-view image 73 acquired in step S107 in each display area provided within the main screen 7 (step S301).

Figure 5 shows an example of the display of the main screen 7. In the example of Figure 5, the display area for the unfolded image 71 is arranged horizontally at the top of the screen, the display area for the direct view image 72 is arranged at the bottom left of the screen, and the display area for the full image 73 is arranged at the bottom right of the screen, but the layout and size of each display area are not limited to this.

The unfolded image 71 is displayed with the pipeline direction of the pipe 10 oriented horizontally. The vertical center of the unfolded image 71 corresponds to the bottom of the pipe 10, and the upper and lower ends of the unfolded image 71 correspond to the top of the pipe 10. The unfolded image 71 also has a scale 79 indicating the pipeline direction position on the pipe 10, an unfolded image handle 74, and buttons 77a and 77b for scrolling the display range of the unfolded image 71. The unfolded image handle 74 is an operation unit (first position designation means) that can be moved left and right by user operation, and the user can move the unfolded image handle 74 to designate any position (pipeline direction position) on the unfolded image 71. The pipeline direction position indicated by the scale 79 is the distance from the entrance of the pipe to the camera 22A.

The direct-view image 72 is a direct-view image 72 captured at a position specified by the expanded image handle 74 (or full image handle 76) among the direct-view images 72 captured as a series of video. That is, the frame captured at the position specified by the expanded image handle 74 (or full image handle 76) among the frames of the video is displayed. It is desirable for the direct-view image 72 to display positional information (the distance from the entrance of the pipe to the camera 22A) and joint numbers (identification information for the joints). A play button 75 (playback instruction input means) is provided near the direct-view image 72. When the play button 75 is operated to input an instruction to play the direct-view image 72, the control unit 51 plays the direct-view image 72 in the forward direction (from the start point of the pipe 10 (the start point of the image capture) to the end point (the end point of the image capture)). The control unit 51 also moves the entire display range of the expanded image 71 in conjunction with the direct-view image 72 being displayed. The control unit 51 moves the expanded image handle 74 in conjunction with the currently displayed direct-view image 72, and also moves the whole image handle 76 in conjunction with the direct-view image 72. That is, the position of the currently played direct-view image 72 is indicated on the whole image 73 and the expanded image 71. In addition to the play button 75, a reverse play button for reverse playback, buttons for fast-forwarding and fast-rewinding, and buttons for moving to the start point and end point may also be provided. If the play button 75 is operated again during video playback, the control unit 51 stops playback.

The entire image 73 is a reduced version of the unfolded image 71, displayed across the entire pipeline, with the pipeline direction of the pipe 10 oriented horizontally. As with the unfolded image 71, the vertical center of the entire image 73 corresponds to the bottom of the pipe 10, and the upper and lower ends of the entire image 73 correspond to the top of the pipe 10. The entire image 73 also has an entire image handle 76 and movement buttons 78a and 78b for moving the position of the entire image handle 76. The entire image handle 76 is an operation unit (second position designation means) that can be moved left and right by user operation, and the user can move the entire image handle 76 to designate any position (pipeline direction position) on the entire image 73. The display position of the entire image handle 76 is moved left and right in conjunction with the position in the pipeline displayed by the direct view image 72 and the position of the unfolded image handle 74.

The main screen 7 displays an unfolded image 71, a direct view image 72, an overall image 73, and other functions, as well as function buttons 81-85 for executing various functions. Function button 81 is operated when manually inputting the position of the joint (joint) of the pipe 10, function button 82 is operated when specifying the damaged location using a point (dot), function button 83 is operated when specifying the damaged location using a box (range), function button 84 is operated when manually inputting the position of the attached pipe, and function button 85 is operated when moving the display position of the unfolded image 71. Also provided are a display size (enlargement/reduction rate) change field 86 that is operated when changing (enlarging/reducing) the display range of the expanded image 71, a reset button 87 that is operated when returning the display size (enlargement/reduction rate) to its original state, a grid display/hide switching operation unit 88, a pipe information display field 89 that displays various information about the pipe 10 displayed on the main screen 7 (route number, manhole number, pipe length, total length, etc.), pipe change buttons 90 and 91 that are operated when changing the pipe 10 displayed on the main screen 7, and a preview button 92 that is operated when displaying a preview of damage information.

Returning to the description of FIG.
When the playback button 75 of the direct-view image 72 is operated on the main screen 7 (step S302; Yes), the control unit 51 plays back the direct-view image 72 and synchronizes the display range of the unfolded image 71 with the direct-view image 72 (step S303). The control unit 51 also moves and displays the positions of the unfolded image handle 74 and the whole image handle 76 so as to indicate positions corresponding to the currently displayed direct-view image 72 (step S304).

Also, if the expanded image handle 74 is operated (step S302; No → step S305; Yes), the control unit 51 displays the expanded image 71 while moving the display range so that the position of the expanded image handle 74 is always included in the display area, and also displays the direct-view image 72 corresponding to the position of the expanded image handle 74 (step S306). Furthermore, the position corresponding to the position of the expanded image handle 74 is displayed on the entire image 73 (moving the display position of the entire image handle 76; step S307).

If the whole image handle 76 is operated (step S305; No → step S308; Yes), the control unit 51 changes the display range of the expanded image 71 to include the position indicated by the whole image handle 76, and displays the direct-view image 72 corresponding to the position of the whole image handle 76 (step S309). The control unit 51 also displays the position corresponding to the position of the whole image handle 76 on the expanded image 71 (moves the display position of the expanded image handle 74; step S310).

If another operation is input on the main screen 7 (step S308; No → step S311; Yes), the control unit 51 executes processing corresponding to the operation (step S312). For example, when the grid display/hide switching operation unit 88 on the main screen 7 is operated, the control unit 51 switches between displaying and hiding the grid 95 on the unfolded image 71, as shown in FIG. 6. The display color of the grid 95 can be selected as white or black, and the grid width can also be changed to 10 mm, 50 mm, 100 mm, etc. Displaying the grid 95 on the unfolded image 71 makes it easier to recognize the location and size of damage and structures. Furthermore, when an operation such as a right-click of the mouse is input on the direct-view image 72, the control unit 51 enlarges and displays the direct-view image 72. Furthermore, when the function button 85 is selected and the unfolded image 71 is dragged with the mouse, the control unit 51 changes the display position of the unfolded image 71. Furthermore, when an arbitrary enlargement or reduction ratio is entered in the display size (enlargement/reduction ratio) change field 86, the control unit 51 displays the expanded image 71 at the specified enlargement or reduction ratio. When the reset button 87 is operated, the control unit 51 returns the expanded image 71 to its original enlargement or reduction ratio (or a predetermined size that is initially set) and displays it.

In steps S311 and S312, processing is performed according to the operation of the above-mentioned function buttons 81 to 85, full preview button 92, magnification change, etc. If no operation is input (step S311; No), the system returns to step S302 and waits for an operation.

Next, the pipe type determination process (step S109 in FIG. 3) will be described with reference to the flowchart in FIG. 7. When an operation to determine the pipe type is performed while the main screen 7 is displayed (step S401; Yes), the control unit 51 starts the pipe type determination process. The operation to perform the pipe type determination is, for example, the operation of the management button 61. First, the control unit 51 acquires a direct-view image 72 (step S402) and extracts multiple image frames to be determined from the acquired direct-view image 72 (step S403). The image frames to be determined are arbitrary; as an example, three frames are selected: those near the entrance, the center, and the end of the pipe 10. The control unit 51 determines the pipe type for each extracted image frame (step S404) and determines the pipe type by taking a majority vote based on the determination results for each image frame (step S405). In the determination process in step S404, the control unit 51 preferably performs image recognition processing using AI (artificial intelligence) to determine the pipe type. The AI uses training data consisting of numerous direct-view images of various pipes 10 made from a variety of materials, including concrete pipes (hume pipes), ceramic pipes, and PVC pipes.

Once the pipe type has been determined, the control unit 51 stores the determined pipe type in RAM or the memory unit 52 (step S406). Pipe type information is previously input by the operator based on ledger information and stored in the memory unit 52. However, if the pipe type determined in step S405 differs from the input (stored) information, the control unit 51 overwrites the pipe type information. The control unit 51 then outputs the pipe type determination result (step S407). As an example of output, for example, as shown in FIG. 8, when the management button 61 on the main screen 7 is operated, the pipe type 62 is displayed nearby, such as "Pipe type: Ceramic pipe." Furthermore, if pipe diameter 63 information has been input in advance, or if the pipe diameter 63 can be calculated from the direct-view image 72, the pipe diameter 63 is also displayed, such as "Pipe diameter: 250 mm."

As another output example, the pipe type may be displayed in the damage information input field 111 during the damage input process described below (Figures 11 and 12), or in the damage list 150 on the preview screen 15 during the preview display process (Figure 14). Furthermore, the pipe type may be displayed in the report during the report creation process described below.

In addition, by determining the pipe type throughout the entire pipeline, it is also possible to detect a change in pipe type along the pipeline (for example, changing from a ceramic pipe route to a PVC pipe due to a sewer pipe replacement). In this case, the control unit 51 may extract multiple frames for each specified section and determine the pipe type for each section. If there is a section along the pipeline where the pipe type has changed, an indication of the change in pipe type may be displayed at the corresponding position on the unfolded image 71.

Next, the pipe structure input process (step S110 in FIG. 3) will be described.
Function button 81 on the main screen 7 is a button for specifying a pipe joint. When function button 81 is selected and any position (pipeline direction position) on the unfolded image is specified by clicking or other operation, the control unit 51 of the inspection support device 5 displays objects 971, 972, ... indicating joints superimposed at the specified position. Furthermore, function button 84 is a button for specifying an attached pipe. When function button 84 is selected and any position (pipeline direction position) on the unfolded image is specified by clicking or other operation, the control unit 51 of the inspection support device 5 displays objects indicating attached pipes superimposed at the specified position.

For example, as shown in Figure 9, straight line objects 971, 972, etc. are displayed superimposed at the positions of the joints. Although not shown, for example, a circular object is displayed superimposed at the position of the attachment pipe. The control unit 51 also obtains the position information of these pipe structures (distance position on the pipe 10) from the unfolded image 71 and adds identification information to each pipe structure. It is desirable to add the identification information as a combination of an alphabet indicating the type of pipe structure and a number indicating the order from the start point of the pipe 10. For example, identification information such as "J1", "J2", etc. is added to the joints. The control unit 51 links the identification information and position information of each pipe structure and stores them in the memory unit 52 as pipe structure information.

As shown in Figure 9, objects 971 and 972 are displayed at the joints in the unfolded image 71, and identification information 971a "J1" and 972a "J2" are displayed above each object 971 and 972. In addition, pipe structure information 98 such as "pipe mouth" is displayed at the start and end points of the pipe 10.

Next, the damage input process (step S111 in FIG. 3) will be described with reference to the flowchart in FIG.
When the function button 82 on the main screen 7 is operated to switch to point input (step S501; point), and then an arbitrary position (point) on the unfolded image is clicked (step S502), the control unit 51 displays a point mark 110 at the clicked position (step S503) and acquires position information of the clicked position in the pipe 10 (step S504). The control unit 51 then displays the damage information input field 111 (step S505).

Figure 11 shows an example screen displaying a damage information input field 111 for point specification. As shown in Figure 11, a point mark 110 is displayed at the position specified by the user on the unfolded image 71, and a damage information display field 110a is displayed nearby. The damage information display field 110a displays damage information including the location information of the damage (pipe direction position), joint identification number, installation direction, location, and details. In addition, a damage information input field 111 is displayed at the bottom of the screen. The damage information input field 111 is provided with a content data tab 111a and a photo tab 111b, and Figure 11 shows the state when the content data tab 111a is selected. The content data tab 111a is provided with a distance input field 113, a location input field 114, a joint number display field 112, a remarks field 115, a damage number 116, a damage type input field 117, and a degree input field 118. The distance input field 113 displays the location information (pipe direction position) of the specified point based on the location information added to the unfolded image 71. The location input field 114 allows the user to select and input the location of the damage using a drop-down list. If pipe structure information has already been entered, the joint number display field 112 displays the corresponding joint number. The remarks field 115 allows the user to enter any characters, numbers, etc. In addition, information on the pipe type determined by the pipe type determination process is displayed in the remarks field 115 of the damage information input field 111, for example, near the Manage button. The damage type input field 117 and damage degree input field 118 allow the user to enter the type and degree of damage using drop-down lists. The damage number 116 indicates the identification information for each damage. Since there may be multiple damages in the same location, it is desirable to provide multiple damage type input fields 117 and damage degree input fields 118.

The photo tab 111b allows you to paste (input) a direct view image 72 or a close-up photo of the damage corresponding to the specified position. When the worker operates the cancel button 111c, the control unit 51 clears the damage information input field 111 and returns to the main screen 7. When the worker enters information into the damage information input field 111 (step S506) and operates the confirm button 111d, the control unit 51 stores the information and image entered into the damage information input field 111 as damage information in the memory unit 52 (step S507).

On the other hand, when the function button 83 on the main screen 7 is operated to switch to box input (step S501; box), and then an arbitrary range of the unfolded image is specified by mouse operation or the like (step S508), the control unit 51 displays a box mark 120 in the specified range (step S509) and obtains position information for the specified range in the pipe 10 (step S510). The control unit 51 then displays the damage information input field 111 (step S505).

Figure 12 shows an example screen displaying the damage information input field 111 by box specification. As shown in Figure 12, a box mark 120 is displayed within the range specified by the user on the unfolded image 71, and a damage information display field 120a is displayed nearby. The damage information display field 120a displays damage information including damage range information (range of pipe direction position), joint location, installation direction, part, and content. The damage information input field 111 is also displayed at the bottom of the screen. The damage information input field 111 has a content data tab 111a and a photo tab 111b, similar to Figure 11, and Figure 11 shows the state when the content data tab 111a is selected. The content data tab 111a has a distance input field 113, part input field 114, joint number display field 112, remarks field 115, damage number 116, damage type input field 117, and extent input field 118. The distance input field 113 reflects the location information of the box-specified range and displays it as "XXX m to △△ m" or the like. The rest is the same as the parts of the damage information input field 111 in Figure 11. The same applies to the photo tab 111b.

When the worker enters information into the damage information input field 111 (step S506) and operates the decision button 111d, the control unit 51 stores the information and images entered into the damage information input field 111 as damage information in the memory unit 52 (step S507).

Next, the preview display process (step S112 in FIG. 3) will be described with reference to the flowchart in FIG.
When the preview button 92 is operated on the main screen 7 (step S601; Yes), the control unit 51 of the research support device 5 displays the preview screen 15 on the display unit 55 (step S602).

The control unit 51 acquires the pipe type information, damage information, and pipe structure information stored in the memory unit 52, as well as images linked to the damage information (direct view image 72, unfolded image 71, etc.) (step S603), and displays a list of damage information reflecting the pipe structure information on the preview screen 15 (step S604).

Figure 14 is a diagram showing an example of the preview screen 15. As shown in Figure 14, the preview screen 15 displays a damage list 150, an overall image 73, and pipe information 89a. The overall image 73 displays dot marks 110 or box marks 120 at the locations of damage recorded as damage information. The damage list 150 displays a list of damage information 151, 152, etc. in order of damage number, and images 72, 71a, 71b, etc. linked to each damage information 151, 152, etc. The damage information 151, 152, etc. corresponds to the information entered in the damage information input field 111 in Figures 11 and 12, and displays location information, site information, joint number, notes, type of damage, pipe type, and degree (rank), etc. In addition, each piece of damage information 151, 152, etc. has a modify button 151a, 152a, etc., allowing the content to be modified.

When the edit button 151a, 152a, ... is operated (step S605; Yes), the control unit 51 accepts the edit of the damage information 151, 152, .... When the worker inputs the edit (step S606), the control unit 51 updates and stores the damage information reflecting the edit in the storage unit 52 (step S607). The control unit 51 stores the date and time of the edit, the content of the edit, information about the person who edited, etc. as edit history information in the storage unit 52 (step S608). If the edit button 151a, 152a, ... is not operated (step S605; No), the preview process ends.

Next, the control unit 51 executes a report creation process (step S113 in Figure 3). In the report creation process, the control unit 51 reads construction information, pipe information, damage information, images, etc. from the storage unit 25, and generates a report in which the construction information, pipe information, damage information, images, etc. are entered according to a predetermined format established by each local government. The control unit 51 displays the generated report on the display unit 55. In addition, the control unit 51 executes operations such as printing out the report, storing and transmitting the report data, according to instructions from the worker.

As explained above, in the tubular structure inspection support system 1, the tubular structure inspection support device 5 acquires the direct-view image 72 captured by the imaging system 2, determines the pipe type (material of the pipe 10), which is the basis of the damage assessment criteria, based on the direct-view image 72, and outputs the assessment result. Compared to the unfolded image 71, the direct-view image 72 is a smaller image with a larger amount of information, allowing for efficient and highly accurate pipe type assessment. This makes it possible to efficiently assess damage information and create reports.

The above describes preferred embodiments of the present invention with reference to the accompanying drawings, but the present invention is not limited to these examples. For example, the layout of each screen, the display size of each image, and the arrangement of function buttons, operation buttons, display fields, input fields, etc. are merely examples, and other layouts, sizes, and arrangements may be adopted. It is clear that a person skilled in the art could conceive of various other modifications or alterations within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally fall within the technical scope of the present invention.

1. Tubular structure investigation support system 2. Photography system 21. On-site PC
22: Photography device 3: Storage 4: Network 5: Tubular structure survey support device (computer)
51: Control unit 511: Image acquisition unit 512: Unfolded image generation unit 513: Display processing unit 514: Pipe structure input unit 515: Damage input unit 516: Preview display unit 61: Management button 62: Pipe type 63: Pipe diameter 7: Main screen 71: Unfolded image (partial range)
72: Direct view image 73: Whole image 74: Developed image handle (first position designation means)
75: Playback button 76: Whole image handle (second position designation means)
79: Scales indicating position information 81 to 85: Function buttons 86: Display size change field 88: Grid display/non-display switching operation section 92: Preview buttons 971, 972: Linear object (joint section)
110: Dot mark 120: Box mark 111: Damage information input field 15: Preview screen 10: Tubular structure (pipe)

Claims (8)

an image acquisition means for acquiring a direct-view image, which is an image taken by a wide-angle camera while traveling inside the tubular structure in the pipe line direction;
a pipe type determination means for determining the pipe type of the tubular structure based on the direct-view image acquired by the image acquisition means;
an output means for outputting the determination result by the pipe type determination means;
Equipped with
The pipe type determination means determines the pipe type by image recognition processing using artificial intelligence.
A tubular structure inspection support device characterized by: The tubular structure inspection support device described in claim 1, characterized in that the pipe type determination means determines multiple frames of the direct-view image. The tubular structure inspection support device described in claim 2, characterized in that the pipe type determination means determines the pipe type by taking a majority vote of the determination results of the multiple frames. A tubular structure inspection support device according to any one of claims 1 to 3, characterized in that the pipe type determination means determines the direct-view image of the entire pipeline. The tubular structure inspection support device according to claim 4, characterized in that it detects locations where the pipe type has been changed based on the results of the determination made by the pipe type determination means. an imaging device that photographs the inside of the tubular structure with a wide-angle camera while moving in the direction of the pipe;
a tubular structure inspection support device connected to the imaging device via a network,
The tubular structure inspection support device comprises:
an image acquisition means for acquiring a direct-view image, which is an image captured by the imaging device;
a pipe type determination means for determining the pipe type of the tubular structure based on the direct-view image acquired by the image acquisition means;
an output means for outputting the determination result by the pipe type determination means;
Equipped with
The pipe type determination means determines the pipe type by image recognition processing using artificial intelligence.
A tubular structure inspection support system characterized by: A pipe type determination method executed by a computer, comprising :
acquiring direct-view images that are images taken with a wide-angle camera while traveling inside the tubular structure in the pipe direction;
determining the pipe type of the tubular structure based on the acquired direct-view image;
outputting the determination result;
Including,
The step of determining the pipe type of the tubular structure determines the pipe type by image recognition processing using artificial intelligence.
A method for determining the type of pipe of a tubular structure. Computer,
an image acquisition means for acquiring a direct-view image, which is an image taken by a wide-angle camera while traveling inside the tubular structure in the pipe line direction;
a pipe type determination means for determining the pipe type of the tubular structure based on the direct-view image acquired by the image acquisition means;
an output means for outputting the determination result by the pipe type determination means;
A program for functioning as
The pipe type determination means determines the pipe type by image recognition processing using artificial intelligence.
A program characterized by: