+

WO2018154871A1 - Dispositif et système d'observation ainsi que procédé de commande de dispositif d'observation - Google Patents

Dispositif et système d'observation ainsi que procédé de commande de dispositif d'observation Download PDF

Info

Publication number
WO2018154871A1
WO2018154871A1 PCT/JP2017/040772 JP2017040772W WO2018154871A1 WO 2018154871 A1 WO2018154871 A1 WO 2018154871A1 JP 2017040772 W JP2017040772 W JP 2017040772W WO 2018154871 A1 WO2018154871 A1 WO 2018154871A1
Authority
WO
WIPO (PCT)
Prior art keywords
observation
optical system
imaging
image
imaging optical
Prior art date
Application number
PCT/JP2017/040772
Other languages
English (en)
Japanese (ja)
Inventor
宏樹 網野
聡司 原
健人 原
加藤　茂
松音　英明
児玉　裕
野中　修
Original Assignee
オリンパス株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オリンパス株式会社 filed Critical オリンパス株式会社
Publication of WO2018154871A1 publication Critical patent/WO2018154871A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements

Definitions

  • the present invention relates to an observation apparatus, an observation system, and an observation apparatus control method.
  • Japanese Patent Application Laid-Open No. 2005-295818 discloses a technique related to a cell culture device that takes an image of the surface of a culture vessel while changing the relative position between the imaging device having a magnifying optical system and the culture vessel.
  • Japanese Unexamined Patent Application Publication No. 2014-238558 discloses the position of the subject of interest in the optical axis direction based on the parallax between images acquired by two cameras.
  • a technique related to an imaging apparatus that acquires the image is disclosed.
  • an imaging device that images the surface of the culture vessel while changing the relative position between the imaging device and the culture vessel, and an imaging device that acquires the position in the optical axis direction of the subject of interest based on the parallax between the imaging And have different optical limitations.
  • An object of the present invention is to provide an observation apparatus, an observation system, and an observation apparatus control method capable of acquiring depth information of a subject of interest.
  • the observation apparatus is a first optical system that is an object-side non-telecentric optical system in which the angle formed by any one of the principal rays on the object side and the optical axis is 6 ° or more.
  • An imaging optical system that captures a sample using the first imaging optical system to obtain a first image, a moving mechanism that changes a relative position between the sample and the imaging unit, Three-dimensional information for acquiring three-dimensional information about the sample based on a plurality of the first images acquired at different positions of the imaging unit and information on each imaging position at the time of acquisition of the first image.
  • An acquisition unit is a first optical system that is an object-side non-telecentric optical system in which the angle formed by any one of the principal rays on the object side and the optical axis is 6 ° or more.
  • the observation system includes the observation device and a controller that acquires a user operation result, outputs the operation result to the observation device, and acquires the observation result of the observation device.
  • the observation apparatus control method is an enlargement optical system and a subject-side non-telecentric optical system in which an angle formed between any principal ray on the subject side and the optical axis is 6 ° or more.
  • the imaging unit including a certain first imaging optical system, imaging the sample by using the first imaging optical system to obtain the first image, and the relative relationship between the sample and the imaging unit Based on the change of position, the plurality of first images acquired at different positions of the imaging unit, and information on each imaging position at the time of acquisition of the first image, the sample And acquiring three-dimensional information.
  • an observation apparatus it is possible to provide an observation apparatus, an observation system, and an observation apparatus control method that can acquire depth information of a subject of interest.
  • FIG. 1 is a schematic diagram illustrating an example of an outline of the appearance of the observation system according to the first embodiment.
  • FIG. 2 is a block diagram illustrating an outline of a configuration example of the observation system according to the first embodiment.
  • FIG. 3 is a schematic diagram illustrating an example of a positional relationship between an imaging unit including the second imaging optical system according to the first embodiment and a sample.
  • FIG. 4 is a schematic diagram illustrating another example of the positional relationship between the imaging unit including the second imaging optical system according to the first embodiment and the sample.
  • FIG. 5 is a schematic diagram illustrating an example of a positional relationship between an imaging unit including the first imaging optical system according to the first embodiment and a sample.
  • FIG. 6 is a schematic diagram illustrating another example of the positional relationship between the imaging unit including the first imaging optical system according to the first embodiment and the sample.
  • FIG. 7 is a flowchart illustrating an example of the observation apparatus control process according to the first embodiment.
  • FIG. 8 is a flowchart of an example of the count scan process according to the first embodiment.
  • FIG. 9 is a schematic diagram illustrating an example of count scan processing information according to the first embodiment.
  • FIG. 10 is a schematic diagram illustrating an example of a movement pattern of the image acquisition unit in the count scan processing according to the first embodiment.
  • FIG. 11 is a flowchart illustrating an example of the 3D scan processing according to the first embodiment.
  • FIG. 12 is a schematic diagram illustrating an example of 3D scan processing information according to the first embodiment.
  • FIG. 13 is a schematic diagram illustrating an example of a movement pattern of the image acquisition unit in the 3D scan processing according to the first embodiment.
  • FIG. 14 is a flowchart illustrating an example of a controller control process according to the first embodiment.
  • FIG. 15 is a schematic diagram illustrating an example of a positional relationship between an imaging unit including the first imaging optical system according to the second embodiment and a sample.
  • FIG. 16 is a schematic diagram illustrating an example of a side observation image according to the second embodiment.
  • FIG. 17 is a flowchart illustrating an example of a side observation process according to the second embodiment.
  • the observation system according to the present embodiment is a system for photographing a cell, a cell group, a tissue or the like in culture and recording the number, form, etc. of the cell or cell group.
  • An example of an outline of the appearance of the observation system 1 is shown in FIG. 1 as a schematic diagram, and an outline of a configuration example of the observation system 1 is shown in FIG. 2 as a block diagram, and the configuration of the observation system 1 will be described with reference to these figures.
  • the observation system 1 includes an observation device 100 and a controller 200.
  • the observation apparatus 100 has a substantially flat plate shape.
  • a sample 300 to be observed is arranged on the upper surface of the observation apparatus 100, and the observation apparatus 100 and the sample 300 are installed in, for example, an incubator.
  • an X axis and a Y axis that are orthogonal to each other are defined in a plane parallel to a plane on which the sample 300 of the observation apparatus 100 is arranged, and a Z axis (observation axis) is orthogonal to the X axis and the Y axis. ) Is defined.
  • the observation apparatus 100 includes a housing 101, a transparent plate 102, an image acquisition unit 150, and a moving mechanism 160.
  • a transparent plate 102 is disposed on the upper surface of the housing 101.
  • the image acquisition unit 150 is provided inside the housing 101 and includes an imaging unit 151 and an illumination unit 155.
  • the imaging unit 151 includes an imaging optical system 152 and an imaging element 153.
  • the imaging unit 151 generates image data based on an image (subject image) formed on the imaging surface of the imaging element 153 via the imaging optical system 152.
  • the image acquisition unit 150 is moved by the moving mechanism 160 to change the relative position with the sample 300.
  • the image acquisition unit 150 illuminates the sample 300 through the transparent plate 102 while being moved, and photographs the sample 300 to acquire an image of the sample 300.
  • the controller 200 is installed outside the incubator, for example.
  • the observation apparatus 100 and the controller 200 communicate with each other.
  • the controller 200 controls the operation of the observation apparatus 100.
  • the shooting position in the Z-axis direction may be changed by the moving mechanism 160, or may be changed by changing the in-focus position of the imaging optical system 152.
  • the imaging optical system 152 is preferably a zoom optical system that can change the focal length.
  • the observation apparatus 100 performs a 3D scan process for acquiring depth information of the subject of interest, and a count scan process for acquiring the size, number, and the like of the subject of interest.
  • the imaging optical system 152 includes a first imaging optical system that is an optical system having non-telecentricity at least on the subject side.
  • a first image an image captured using the first imaging optical system is referred to as a first image.
  • the observation apparatus 100 acquires the first image while moving the image acquisition unit 150 including the first imaging optical system.
  • the observation apparatus 100 acquires, as depth information, the position in the optical axis direction (observation axis direction) of the first imaging optical system of the subject of interest included in the sample 300 based on parallax between imaging performed at different positions. To do.
  • the depth information includes the position in the optical axis direction of the first imaging optical system related to the subject of interest, a three-dimensional model (3D model), and the like.
  • the imaging optical system 152 further includes a second imaging optical system that is an optical system having telecentricity at least on the subject side.
  • a second imaging optical system that is an optical system having telecentricity at least on the subject side.
  • an image captured using the second imaging optical system is referred to as a second image.
  • the observation apparatus 100 acquires the second image while moving the image acquisition unit 150 including the second imaging optical system.
  • the observation apparatus 100 combines the second images captured at different positions, and acquires a wide range of high pixel images as if they were captured by capturing a wide range.
  • the observation apparatus 100 acquires the size, number, and the like of the subject of interest based on the second image or the high pixel image.
  • the imaging optical system 152 includes a diaphragm 152a and a plurality of lenses including at least an objective lens 152b and an imaging lens 152c.
  • the focal length on the image sensor 153 side of the imaging lens 152c is defined as a focal length Ft.
  • FIG. 3 An example of the positional relationship between the imaging unit 151 including the second imaging optical system according to the present embodiment and the sample 300 is shown in FIG. 3 as a schematic diagram.
  • the sample 300 includes a container 310 and a cell 324 that is a subject of interest.
  • the second imaging optical system is a double-sided telecentric optical system will be described as an example. As shown in FIG.
  • the second imaging optical system matches the positions of the exit side focal point of the stop 152a and the objective lens 152b with the incident side focal point of the imaging lens 152c, so that the subject side And an optical system having telecentricity on both sides of the image side.
  • a principal ray that has entered parallel to the optical axis of the optical system (a ray passing through the center of the stop 152a) is emitted in parallel to the optical axis of the optical system.
  • the point P1 on the cell 324 is located on the optical axis of the second imaging optical system.
  • the light rays (chief rays) that pass through the second imaging optical system so as to pass through the center of the stop 152a and enter the imaging element 153 are principal rays of the second imaging optical system.
  • the light ray R1 enters the objective lens 152b in parallel with the optical axis of the second imaging optical system, passes through the center of the diaphragm 152a and the imaging lens 152c, and the second imaging optical on the image sensor 153.
  • the light enters the point Q1 located on the optical axis of the system.
  • FIG. 4 shows a schematic diagram of the state after being applied.
  • the light rays (chief rays) that can enter the image sensor 153 through the second imaging optical system so as to pass through the center of the stop 152a are
  • the light ray R1 ′ is incident on the objective lens 152b in parallel with the optical axis of the second imaging optical system.
  • the mechanism configuration can be tilted, it is possible to apply the tilting movement to the optical axis.
  • a point on the image sensor 153 on which the light ray R1 ′ is incident is referred to as Q1 ′.
  • the amount of change in the image position (second image movement amount) on the imaging surface of the imaging element 153 is the distance between the point Q1 and the point Q1 ′. Further, in the state shown in FIG. 4, a point on the optical axis of the second imaging optical system and whose position in the optical axis direction is equal to the point P1 is defined as a point P11.
  • FIG. 5 An example of the positional relationship between the imaging unit 151 including the first imaging optical system according to the present embodiment and the sample 300 is shown in FIG. 5 as a schematic diagram. Similar to the case shown in FIGS. 3 and 4, the sample 300 includes a container 310 and a cell 324 that is a subject of interest.
  • the first imaging optical system is a subject-side non-telecentric optical system (image-side telecentric optical system) will be described as an example. As shown in FIG.
  • the first imaging optical system is configured to reduce the position of the objective lens 152b in the optical axis direction of the first imaging optical system in the double-sided telecentric optical system shown in FIG.
  • This is an optical system in which the telecentricity is broken only on the subject side by moving to the 152a side.
  • a light beam (principal light beam) that enters the optical system and is deflected by the objective lens 152b and then passes through the center of the stop 152a is parallel to the optical axis of the optical system. It is injected. Since the exit side focal points of the aperture stop 152a and the objective lens 152b are different from each other, the principal ray (in FIG.
  • the off-axis principal ray (shown by a broken line) has an optical axis and an inclination of the optical system between the subject of interest and the optical system.
  • the position of the point P1 on the cell 324 is on the optical axis of the first imaging optical system. Therefore, out of the light rays emitted from the point P1, the diaphragm 152a of the first imaging optical system.
  • a ray (principal ray) that can enter the image sensor 153 through the center of the image is a ray R2 that passes through the optical axis of the first image pickup optical system and enters the point Q2 on the image sensor 153.
  • FIG. 6 shows a schematic diagram of the state after being applied.
  • the principal ray out of the rays emitted from the point P1 has an inclination of the angle ⁇ with respect to the optical axis. It is a light ray R2 ′ that is incident on the objective lens 152b. Thereafter, the light ray R2 ′ is deflected by the objective lens 152b, passes through the center of the stop 152a, and enters the point Q2 ′ on the image sensor 153.
  • the amount of change in the image position (first image movement amount ⁇ X) on the imaging surface of the imaging element 153 is the distance between the point Q2 and the point Q2 ′.
  • a point on the optical axis of the first imaging optical system and whose position in the optical axis direction is equal to the point P1 is defined as a point P12.
  • each principal ray incident on the imaging element 153 is parallel to the optical axis of the second imaging optical system. 2 is incident on the imaging optical system 2. Therefore, when the relative position between the subject of interest and the optical axis position of the second imaging optical system changes, there is no change (parallax) in the direction of viewing the point P1 from the incident end of the second imaging optical system.
  • the optical axis position (imaging position) of the second imaging optical system moves from position X1 to position X1 ′. It is.
  • the second image movement amount is equal to a distance obtained by multiplying the interval between the imaging positions at which the second image is acquired (first X movement amount ⁇ X1) by the magnification of the second imaging optical system. Even if the subject is uneven, the appearance on the imaging surface does not change. This can also be expressed as the second image movement amount not including the image movement amount caused by the parallax that occurs when the parallax exists.
  • the pupil is infinite when viewed from the subject side, so that an image (second image) as if seen from a distance is obtained.
  • the second imaging optical system is parallel to the optical axis of the second imaging optical system.
  • the optical path of the light incident on the system does not change. Therefore, when combining a plurality of second images acquired while changing the imaging position to synthesize a wide range of high-pixel images, it is easy to process the joints between the images. Needless to say, based on a wide range of images, it is easy to compare the count of the number of cells and the size of the cells present in the imaging region.
  • the observation apparatus 100 performs imaging using the first imaging optical system when it is desired to acquire depth information.
  • the principal ray emitted from the optical axis has an inclination with respect to the optical axis. Therefore, for example, when the relative position of the subject of interest and the optical axis position changes so that the optical axis position (imaging position) of the first imaging optical system moves from the position X2 to the position X2 ′, the first Change (parallax) in the direction of viewing the point P1 from the incident end of the optical system.
  • the first image movement amount ⁇ X is equal to a distance obtained by multiplying the interval between the imaging positions at which the first image is acquired (second X movement amount ⁇ X2) by the magnification of the first imaging optical system.
  • the magnification on the imaging surface differs for a subject that is in a position before and after the focal plane on the subject side of the objective lens (a surface that includes P1 and is perpendicular to the optical axis).
  • the magnification on the imaging surface decreases as the distance from the objective lens 152b increases. That is, the ratio between the second X movement amount ⁇ X2 and the first image movement amount ⁇ X changes depending on the difference in the depth of the position of the point P1 to be set. This can also be expressed as the first image movement amount ⁇ X including an image movement amount caused by the parallax generated when the parallax exists.
  • the observation apparatus 100 can acquire depth information about a subject of interest such as the cell 324 using the principle of triangulation based on the first image movement amount ⁇ X, for example.
  • the depth information includes, for example, 3D information and 3D images including information such as thickness and unevenness. In the imaging using the first imaging optical system, such distance distribution and image information in each depth direction can be obtained, so that the object can be confirmed with various information.
  • the imaging unit 151 that acquires an image of the sample 300 using the observation optical system, the moving mechanism 160 that changes the relative position between the sample 300 and the imaging unit 151, and a plurality of images acquired at different positions of the imaging unit 151.
  • an observation apparatus provided with the three-dimensional information acquisition part which acquires this sample 300 as three-dimensional information based on the information regarding each imaging position at the time of acquisition of this image can be provided.
  • the observation optical system only needs to have an imaging function, and may be applied such that enlargement is electronically enlarged.
  • the triangle formed by the point S0, the point P1, and the point P12 indicating the center of the diaphragm 152a is the point K1 on the imaging lens 152c through which the point S0 and the light ray R2 ′ pass.
  • the distance between the point K1 and the point K0 is equal to the first image movement amount ⁇ X.
  • the first image movement amount ⁇ X is known because the corresponding point is detected and calculated as the movement amount of the corresponding point on the image plane in the image processing based on at least two first images.
  • the distance between the point S0 and the point K0 is the focal length Fo on the incident side of the imaging lens 152c and is known.
  • the distance between the point P1 and the point P12 is the distance between the optical axes (second X movement amount ⁇ X2) and is known.
  • the second imaging optical system shown in FIGS. 3 and 4 and the first imaging optical system shown in FIGS. 5 and 6 are shown as optical systems having telecentricity on the image side, respectively.
  • the observation apparatus 100 uses the imaging optical system 152 as the first imaging optical system in order to switch whether an image acquired by imaging includes depth information of the cell 324 or not. Or a second imaging optical system. Therefore, even if the first imaging optical system or the second imaging optical system is an optical system having non-telecentricity on the image side, the same effect can be obtained.
  • the first imaging optical system is an optical system having non-telecentricity on the subject side
  • the first imaging optical system is moved in the optical axis direction of the first imaging optical system in order to focus.
  • the corresponding point on the image sensor 153 moves.
  • the case where the first imaging optical system and the imaging element 153 are moved together in the direction away from the cell 324 (Z-direction) from the state shown in FIG. 6 will be described as an example.
  • an angle formed by a straight line passing through the points P1 and S0 and a straight line passing through the points P12 and S0 is small. Therefore, the point K1 moves to the optical axis side, and the point Q2 ′ also moves to the optical axis side. In this way, even when the first imaging optical system is moved in the optical axis direction of the first imaging optical system for focusing or the like in the middle of repeating the imaging while being moved, the corresponding point is moved. Obviously, it is possible to acquire depth information.
  • the first image movement amount ⁇ X is acquired for a plurality of corresponding points included in the first image, and the depth information on the surface is acquired for the cell 324.
  • a point P1 is positioned on the optical axis of the first imaging optical system as one of the first images used for obtaining depth information is captured.
  • the present invention is not limited to this.
  • points that are not located on the optical axis at the time of each imaging may be used as corresponding points.
  • the observation apparatus 100 acquires depth information using the first imaging optical system, the depth information includes depth information of the surface of the subject of interest, and the subject of interest It becomes easy to obtain a 3D model.
  • any principal ray on the subject side may be regarded as a subject-side telecentric optical system as long as the angle formed with the optical axis is 4 ° or less.
  • the subject-side non-telecentric optical system may be regarded as an optical system in which an angle formed by any principal ray on the subject side and the optical axis (angle ⁇ in FIG. 6) is 6 ° or more.
  • An image having a sufficient parallax between the plurality of first images can be obtained.
  • the angle (angle ⁇ in FIG. 6) formed by any principal ray on the subject side and the optical axis is 20 ° or more.
  • a sample 300 that is a measurement target of the observation system 1 is, for example, as follows.
  • the sample 300 includes, for example, a container 310, a culture medium 322, cells 324, and a reflection plate 360.
  • a medium 322 is placed in the container 310, and cells 324 are cultured in the medium 322.
  • the container 310 can be, for example, a petri dish, a culture flask, a multiwell plate, or the like.
  • the container 310 is a culture container for culturing a biological sample, for example.
  • the shape, size, etc. of the container 310 are not limited.
  • the medium 322 may be a liquid medium or a solid medium.
  • the measurement object is, for example, the cell 324, but this may be an adhesive cell or a floating cell.
  • the cell 324 may be a spheroid or a tissue.
  • the cell 324 may be derived from any organism, and may be a fungus or the like.
  • the sample 300 includes a biological sample that is a living organism or a sample derived from a living organism.
  • the reflection plate 360 is for illuminating the cells 324 by reflecting the illumination light incident on the sample 300 via the transparent plate 102, and is disposed on the upper surface of the container 310.
  • the transparent plate 102 disposed on the upper surface of the casing 101 of the observation apparatus 100 is made of, for example, glass.
  • the observation apparatus 100 is in a state in which the inside is sealed by a member including, for example, a housing 101 and a transparent plate 102.
  • the sample 300 is placed on the transparent plate 102.
  • FIG. 1 shows an example in which the entire upper surface of the housing 101 is formed of a transparent plate, but the observation apparatus 100 is provided with a transparent plate on a part of the upper surface of the housing 101.
  • the other part of the upper surface may be configured to be opaque.
  • transparency here shows that it is transparent with respect to the wavelength of illumination light.
  • the moving mechanism 160 includes a support portion 165, an X feed screw 161 for moving the support portion 165 in the X-axis direction, and an X actuator 162.
  • the moving mechanism 160 further includes a Y feed screw 163 and a Y actuator 164 for moving the support portion 165 in the Y-axis direction.
  • the moving mechanism 160 may include a Z feed screw and a Z actuator for moving the support portion 165 in the Z-axis direction.
  • the direction in which the support portion 165 moves away from the X actuator 162 is defined as the positive direction of the X direction (X + direction)
  • the direction of movement away from the Y actuator 164 is defined as the positive direction in the Y direction. (Y + direction)
  • the direction from the support 165 toward the sample 300 is the positive direction of the Z direction (Z + direction).
  • the illumination unit 155 included in the image acquisition unit 150 is provided on the support unit 165 included in the moving mechanism 160.
  • An imaging unit 151 is provided in the vicinity of the illumination unit 155.
  • the illumination unit 155 includes an illumination optical system 156 and a light source 157.
  • the illumination light emitted from the light source 157 is irradiated onto the sample 300 via the illumination optical system 156.
  • the light source 157 includes, for example, an LED.
  • the imaging unit 151 further includes a lens switching unit 154.
  • the lens switching unit 154 drives a lens included in the imaging optical system 152 in the optical axis direction so that the imaging optical system 152 becomes the first imaging optical system or the second imaging optical system.
  • the lens switching unit 154 according to the present embodiment performs, for example, a count scan process that counts the number of cells 324, an imaging process that acquires a wide range of high pixel images by combining a plurality of images acquired at a plurality of positions, and the like.
  • the imaging optical system 152 is used as the second imaging optical system.
  • the lens switching unit 154 uses the imaging optical system 152 when, for example, 3D scan processing for acquiring depth information of the cell 324, imaging processing for acquiring a three-dimensional image of the cell 324, or the like is performed. 1 imaging optical system.
  • the observation apparatus 100 causes the lens switching unit 154 to switch the imaging optical system 152 according to the type of observation, and causes the moving mechanism 160 to change the position of the image acquisition unit 150 in the X direction and the Y direction.
  • the sample 300 is repeatedly photographed while changing the optical axis of the imaging optical system 152 in the direction while being kept parallel to the observation axis, and a plurality of images are acquired.
  • the observation apparatus 100 further includes an observation side recording circuit 130.
  • the observation-side recording circuit 130 records, for example, programs and various parameters used in each unit included in the observation apparatus 100 and data obtained by the observation apparatus 100.
  • the observation-side recording circuit 130 temporarily records various data such as image data (pixel data), image data for recording, image data for display, and processing data during operation.
  • the observation-side recording circuit 130 records the focus position range in the optical axis direction of the imaging optical system 152 as, for example, a focus position range.
  • a focus position range for example, a value corresponding to the size of the sample 300 or the like is set in advance, or is set by a user input.
  • the observation apparatus 100 further includes an image processing circuit 120.
  • the image processing circuit 120 performs various image processing on the image data obtained by the imaging unit 151. Data after image processing by the image processing circuit 120 is recorded in, for example, the observation-side recording circuit 130 or transmitted to the controller 200. Further, the image processing circuit 120 may perform various analyzes based on the obtained image. For example, the image processing circuit 120 acquires depth information of a cell 324 or a cell group included in the sample 300 based on the obtained first image. For example, the image processing circuit 120 extracts an image of a cell 324 or a cell group included in the sample 300 based on the obtained second image, or calculates the number of cells or a cell group. The analysis result obtained in this way is also recorded in the observation-side recording circuit 130 or transmitted to the controller 200, for example.
  • the observation apparatus 100 further includes an observation side communication apparatus 140.
  • an observation side communication apparatus 140 wireless communication using, for example, Wi-Fi (registered trademark) or Bluetooth (registered trademark) is used.
  • the observation apparatus 100 and the controller 200 may be connected to each other by a wired communication to communicate with each other, or may be connected to an electrical communication line such as the Internet and communicate via an electrical communication line such as the Internet. It may be broken.
  • the observation apparatus 100 further includes an observation-side control circuit 110 and a clock unit 172.
  • the observation side control circuit 110 controls the operation of each unit included in the observation apparatus 100.
  • the observation-side control circuit 110 acquires various information related to the operation of the observation apparatus 100, performs various determinations related to the operation of the observation apparatus 100, and notifies and alerts the user based on the determination result.
  • the observation-side control circuit 110 includes a position control unit 111, an imaging control unit 112, an illumination control unit 113, a communication control unit 114, a recording control unit 115, a measurement control unit 116, and a distance conversion unit 117. It has a function.
  • the position control unit 111 controls the operation of the moving mechanism 160 and controls the position of the image acquisition unit 150.
  • the imaging control unit 112 controls the operation of the imaging unit 151 included in the image acquisition unit 150 and causes the imaging unit 151 to acquire an image of the sample 300.
  • the imaging control unit 112 includes a focus / exposure switching unit.
  • the imaging control unit 112 performs focus adjustment by moving a focusing lens included in the imaging optical system 152 in the optical axis direction, for example.
  • the focusing lens may be a lens having a variable focal length such as a liquid lens. A plurality of lenses with different focal points may be prepared for focusing. If the prepared lens is multi-lens, refocusing technology or the like can be used.
  • the focus / exposure switching unit adjusts the exposure by controlling the operation of the diaphragm 152a, for example, and adjusts the zoom by controlling the operation of the lens in the optical axis direction.
  • the illumination control unit 113 controls the operation of the illumination unit 155 included in the image acquisition unit 150.
  • the communication control unit 114 manages communication with the controller 200 via the observation side communication device 140.
  • the recording control unit 115 controls recording of data obtained by the observation apparatus 100.
  • the measurement control unit 116 controls the entire measurement such as the timing and number of times of measurement.
  • the distance conversion unit 117 acquires position information in the optical axis direction of the cell 324 that is the subject of interest, information on the unevenness of the cell 324, and the like as depth information based on the processing result of the image processing circuit 120, for example.
  • the clock unit 172 generates time information and outputs it to the observation side control circuit 110. The time information is used for determination related to the operation of the observation apparatus 100 when recording acquired data, for example.
  • observation-side control circuit 110 the image processing circuit 120, the observation-side recording circuit 130, and the observation-side communication device 140 described above are arranged inside the casing 101 as a circuit group 104, for example, as shown in FIG. Is provided.
  • the observation apparatus 100 includes functions as a three-dimensional information acquisition unit, a corresponding point acquisition unit, and a three-dimensional model generation unit.
  • the three-dimensional information acquisition unit is used as the three-dimensional information about the sample 300 based on the plurality of first images acquired at the positions of the different imaging units 151 and the information related to the respective imaging positions at the time of acquisition of the first images. get.
  • the three-dimensional information acquisition unit acquires information related to the unevenness of the sample 300, for example.
  • the three-dimensional information acquisition unit acquires, for example, information related to each imaging position at the time of acquiring the first image from the distance conversion unit 117.
  • the three-dimensional information includes depth information acquired by the distance conversion unit 117, for example.
  • the corresponding point acquisition unit acquires the corresponding point based on the correlation between the plurality of first images acquired while the imaging unit 151 is moved by the moving mechanism 160, for example, and acquires the first image movement amount ⁇ X. .
  • the three-dimensional model generation unit constructs a three-dimensional model of the subject of interest based on the depth information acquired by the distance conversion unit 117, for example.
  • the functions as the three-dimensional information acquisition unit, the corresponding point acquisition unit, and the three-dimensional model generation unit can be realized by the observation-side control circuit 110 and / or the image processing circuit 120, for example.
  • the reliability is high.
  • the structure can be easily handled and cleaned, and can prevent contamination and the like.
  • the controller 200 is, for example, a personal computer (PC), a tablet information terminal, or the like.
  • FIG. 1 illustrates a tablet information terminal.
  • the controller 200 is provided with an input / output device 270 including a display device 272 such as a liquid crystal display and an input device 274 such as a touch panel.
  • the input device 274 may include a switch, dial, keyboard, mouse, and the like in addition to the touch panel.
  • controller 200 is provided with a controller side communication device 240.
  • the controller side communication device 240 is a device for communicating with the observation side communication device 140.
  • the observation apparatus 100 and the controller 200 communicate with each other via the observation side communication apparatus 140 and the controller side communication apparatus 240.
  • the controller 200 includes a controller-side control circuit 210 and a controller-side recording circuit 230.
  • the controller side control circuit 210 controls the operation of each part of the controller 200.
  • the controller-side recording circuit 230 records, for example, programs used in the controller-side control circuit 210, various parameters, and data received from the observation apparatus 100.
  • the controller-side control circuit 210 has functions as a system control unit 211, a display control unit 212, a recording control unit 213, a communication control unit 214, and a network cooperation unit 215.
  • the controller-side control circuit 210 may further have functions as a corresponding point acquisition unit and a three-dimensional model generation unit.
  • the system control unit 211 performs various calculations related to control for measurement of the sample 300.
  • the display control unit 212 controls the operation of the display device 272.
  • the display control unit 212 causes the display device 272 to display necessary information and the like.
  • the recording control unit 213 controls information recording in the controller-side recording circuit 230.
  • the communication control unit 214 controls communication with the observation device 100 via the controller side communication device 240.
  • the network cooperation unit 215 controls the cooperation between the observation system 1 and a network server or the like outside the observation system 1 such as a cloud provided on a telecommunication line such as the Internet.
  • the network cooperation unit 215 provides, for example, an observation result such as an image acquired by the observation device 100 or an observation result acquired by the controller 200 on the observation side communication device 140 or the controller side communication device 240 on the network. Sent to the server.
  • the network cooperation unit 215 causes the image processing circuit or the like included in the network server to perform processing such as cell count and depth information calculation based on the observation result, and acquires the result of the processing.
  • the network cooperation unit 215 acquires information from an IoT device such as an incubator, an air conditioning facility, and a lighting facility connected to the Internet, and controls the IoT device.
  • the observation-side control circuit 110, the image processing circuit 120, and the controller-side control circuit 210 include an integrated circuit such as Central Processing Unit (CPU), Application Specific Integrated Circuit (ASIC), or Field Programmable Gate Array (FPGA). .
  • the observation side control circuit 110, the image processing circuit 120, and the controller side control circuit 210 may each be configured by one integrated circuit or the like, or may be configured by combining a plurality of integrated circuits. Further, the observation side control circuit 110 and the image processing circuit 120 may be configured by one integrated circuit or the like.
  • the position control unit 111, the imaging control unit 112, the illumination control unit 113, the communication control unit 114, the recording control unit 115, the measurement control unit 116, and the distance conversion unit 117 of the observation side control circuit 110 are each one integrated circuit. Etc., or a combination of a plurality of integrated circuits or the like. Further, two or more of the position control unit 111, the imaging control unit 112, the illumination control unit 113, the communication control unit 114, the recording control unit 115, the measurement control unit 116, and the distance conversion unit 117 are configured by one integrated circuit or the like. May be.
  • system control unit 211, the display control unit 212, the recording control unit 213, the communication control unit 214, and the network link unit 215 of the controller-side control circuit 210 may each be configured with one integrated circuit or the like. These integrated circuits or the like may be combined. Further, two or more of the system control unit 211, the display control unit 212, the recording control unit 213, the communication control unit 214, and the network link unit 215 may be configured by one integrated circuit or the like. The operation of these integrated circuits is performed according to a program recorded in a recording area in the observation-side recording circuit 130 or the controller-side recording circuit 230 or the integrated circuit, for example.
  • observation-side recording circuit 130 the controller-side recording circuit 230, or each of the elements included in the observation-side recording circuit 130 is a non-volatile memory such as a flash memory, for example, but is not limited to Static Random Access Memory (SRAM) Such a volatile memory may be further included.
  • the observation-side recording circuit 130 or each of the elements included therein and the controller-side recording circuit 230 or each of the elements included in the observation-side recording circuit 130 may be configured by one memory or the like, or a plurality of memories or the like may be combined. It may be configured. Further, a database or the like outside the observation system 1 may be used as a part of the memory.
  • step S101 the observation-side control circuit 110 stands by until a signal output from the controller 200 according to a user operation is received, for example.
  • step S102 the observation-side control circuit 110 determines whether, for example, a power-on signal for turning on the power of the observation apparatus 100 or a power-off signal for turning off the power of the observation apparatus 100 is received from the controller 200. The process proceeds to step S103 if it is determined that the power ON / OFF signal has been received, and proceeds to step S104 if it is not determined that it has been received.
  • step S103 when it is determined that the power-on signal is received in step S102, the observation-side control circuit 110 starts supplying power to each part of the observation apparatus 100, and receives the power-off signal in step S102. If determined, the supply of power to each unit of the observation apparatus 100 is terminated. Note that power is continuously supplied to the observation-side communication device 140 in any case in order to wait for communication. Thereafter, the process returns to step S101.
  • the observation device 100 is a communication device with low power consumption such as Bluetooth Low Energy (BLE) for transmission / reception of control signals and the like, and high-speed communication such as Wi-Fi for transmission / reception of data such as observation results including images. And a device.
  • BLE Bluetooth Low Energy
  • Wi-Fi wireless fidelity
  • the controller 200 when the power is turned on at step S103. That's fine.
  • the power of the observation apparatus 100 is turned on or off based on the power ON / OFF signal output from the controller 200, the present invention is not limited to this. For example, during culturing of cells, observation such as imaging may be performed by turning on or off the observation apparatus 100 at a preset time interval such as every minute.
  • step S104 the observation-side control circuit 110 determines whether or not control signals related to various settings are received from the controller 200, for example. The process proceeds to step S105 if it is determined that a control signal related to various settings has been received, and returns to step S101 if it is not determined that it has been received.
  • the observation-side control circuit 110 sets each part of the observation device 100 according to the control signals related to various settings received by the observation-side communication device 140 in step S104.
  • the information set here includes, for example, information related to an observation result such as an image acquired by the observation apparatus 100 or a transmission destination of the measurement result, imaging conditions, measurement conditions, and various parameters.
  • the transmission destination of the observation result or measurement result acquired by the observation apparatus 100 is, for example, the observation-side recording circuit 130 of the observation apparatus 100, the controller-side recording circuit 230 of the controller 200, a data server on the network, or the like.
  • an observation result or a measurement result is transmitted to a cloud or the like constructed on the network in this way, not only information sharing between different users is facilitated, but also an analysis of an acquired image and an image outside the observation system 1 Processing can be performed.
  • step S106 the observation-side control circuit 110 determines whether or not a control signal instructing execution of the count scan process is received from the controller 200, for example. The process proceeds to step S107 when it is determined that a control signal instructing execution of the count scan process has been received, and proceeds to step S108 when it is not determined that it has been received.
  • a measurement start time or the like is determined in advance, and measurement may be started at the determined measurement start time.
  • step S107 the observation-side control circuit 110 executes a count scan process and counts the number of cells 324. Details of the count scan process will be described later. Thereafter, the process proceeds to step S108.
  • step S108 the observation-side control circuit 110 determines whether or not a control signal instructing execution of 3D scan processing is received from the controller 200, for example. The process proceeds to step S109 if it is determined that a control signal instructing execution of the count process has been received, and proceeds to step S110 if it is not determined that it has been received.
  • step S109 the observation-side control circuit 110 executes a 3D scan process and acquires depth information such as three-dimensional information of the cell 324. Details of the 3D scanning process will be described later. Thereafter, the process proceeds to step S110.
  • step S110 the observation-side control circuit 110 determines whether or not to end the processing related to observation or measurement based on, for example, a control signal output by the controller 200 in response to a user operation. The process proceeds to step S111 if it is determined to end, and returns to step S104 if it is determined not to end.
  • the observation-side control circuit 110 determines whether or not a control signal that requests an observation result or a measurement result is received from the controller 200, for example.
  • the observation result or the measurement result includes, for example, various data obtained by the observation apparatus 100 such as a measurement measurement value, an acquired image, a photographing position, and an analysis result.
  • the shooting position includes an X coordinate, a Y coordinate, and a Z coordinate of the shooting position.
  • the X coordinate and the Y coordinate are values used in the control of the moving mechanism 160 and can be acquired from the position control unit 111, for example.
  • the Z coordinate is a value used for controlling the imaging optical system 152, and can be acquired from, for example, the imaging control unit 112, the distance conversion unit 117, and the like.
  • the process proceeds to step S112 if it is determined that a control signal requesting an observation result or a measurement result has been received, and returns to step S101 if it is not determined that it has been received.
  • step S112 the observation-side control circuit 110 transmits the results obtained by various observations and measurements such as an acquired image, the analysis results obtained by analyzing the results, and the like to the transmission destination set in step S105, for example. To do. Thereafter, the process returns to step S101.
  • step S107 of the observation apparatus control process is shown in FIG. 8 as a flowchart, and the operation of the observation system 1 during the count scan process will be described with reference to this.
  • step S201 the observation-side control circuit 110 causes the lens switching unit 154 to make the imaging optical system 152 the second imaging optical system.
  • the second imaging optical system is an optical system having telecentricity on the subject side as described above. Thereafter, the process proceeds to step S202.
  • step S202 the observation-side control circuit 110 executes pre-processing for starting a count scan based on, for example, the count-scan processing information recorded in the observation-side recording circuit 130.
  • the observation side control circuit 110 moves the image acquisition unit 150 to the moving mechanism 160 and returns it to the XY start position of the count scan.
  • the observation-side control circuit 110 controls the operations of the imaging optical system 152 and the imaging element 153 or the moving mechanism 160 so that the count scan can be started from the initial position in the Z direction. Thereafter, the observation side control circuit 110 starts a count scan.
  • count scan processing information according to the present embodiment is shown in FIG. 9, and information recorded as count scan processing information will be described with reference to this.
  • the information is set in advance, for example, or set in step S105 of the observation apparatus control process.
  • the count scan processing information includes information CSP relating to the count scan pattern, information CSJ relating to execution of the count scan processing, and information CSR obtained by the count scan processing.
  • the information CSP related to the count scan pattern includes, for example, a count scan start condition CSP1, a start position CSP2, an end condition CSP3, a first X movement pitch CSP5, a first Y movement pitch CSP6, and movement in the X direction to movement in the Y direction.
  • the first X ⁇ Y condition CSP10 which is a condition for switching to the X direction
  • the first Y ⁇ X condition CSP11 which is a condition for switching from the movement in the Y direction to the movement in the X direction.
  • the first X movement pitch CSP5 is a movement (imaging) interval in the X direction
  • the first Y movement pitch CSP6 is a movement (imaging) interval in the Y direction.
  • the image acquisition unit 150 captures images for each of these movement pitches and acquires a second image.
  • the information CSJ related to the execution of the count scan process is, for example, when it is determined that there is an observation defect based on a first NG determination condition CSJ1, which is a determination condition for determining an observation defect, for example, the first NG determination condition CSJ1.
  • a first retry determination condition CSJ2 which is a determination condition for determining whether or not to re-execute the count scan, is included.
  • the information CSR obtained by the count scan process is recorded in association with each image (second image) acquired by using the second imaging optical system in the count scan process, for example.
  • the first result CSR1 includes the first frame CSR11, the first time CSR12 when the first frame CSR11 is acquired, the first AF information CSR13, and the first imaging condition CSR14.
  • the shooting conditions include exposure conditions such as shutter speed and aperture, and other shooting conditions.
  • the photographing conditions here may be different for each photographing, may be different for each measurement, or may be common for all photographing.
  • the information may include information on a position where the second image is acquired, a result of counting the number of cells 324, and the like.
  • FIG. 10 an example of the movement pattern of the image acquisition unit 150 in the count scan processing according to the present embodiment is shown in FIG. 10 as a schematic diagram, and the movement of the image acquisition unit 150 in the count scan will be described with reference to this.
  • a case where the count scan process is executed while the image acquisition unit 150 is moved on the line CL1 shown in FIG. 10 will be described as an example.
  • the observation-side control circuit 110 moves the image acquisition unit 150 to the start position CP1, and acquires the second image.
  • the observation-side control circuit 110 moves the image acquisition unit 150 in the Y direction by the first Y movement pitch, and acquires the second image at the position after the movement. Thereafter, the observation-side control circuit 110 determines that the second image is in a state of being in a state corresponding to the first Y ⁇ X condition CSP11, for example, the image acquisition unit 150 exists at the position indicated by the point CP2.
  • the acquisition and the movement of the image acquisition unit 150 are repeated.
  • the observation side control circuit 110 switches the direction in which the image acquisition unit 150 is moved from the Y direction to the X direction. After the movement direction is switched to the Y direction, the observation-side control circuit 110 is in a state corresponding to the first X ⁇ Y condition CSP10, for example, the image acquisition unit 150 exists at the position indicated by the point CP3. Until it is determined, the process of moving the image acquisition unit 150 by the first X movement pitch and acquiring the second image is repeated. In this way, the observation-side control circuit 110 continues the count scan process until it is determined that the end condition CSP3 is satisfied, for example, when the image acquisition unit 150 reaches the point CP10.
  • step S203 the observation-side control circuit 110 determines the count scan state. In this determination, for example, when the image captured by the image processing circuit 120 is analyzed to detect an observation failure, or when the moving mechanism 160 detects an operation failure, it is determined that the count scan needs to be performed again.
  • the determination condition is recorded in the observation-side recording circuit 130 as count scan processing information, for example.
  • the specification which a user judges based on the live view (LV) display performed by the controller 200 by transmitting the image acquired at the time of count scanning to the controller 200 is also considered.
  • the process proceeds to step S204 when it is determined that re-execution of the count scan is necessary, and proceeds to step S205 when it is not determined.
  • step S204 the observation-side control circuit 110 alerts the user that an observation failure or the like has occurred in the count scan, or that the count scan needs to be re-executed, according to the determination result in step S203.
  • a control signal is generated and transmitted to the controller 200. Thereafter, the process returns to step S202.
  • count scan processing performed after returning to step S202 may be performed again with the image acquisition unit 150 returned to the start position or performed again from the current position, for example, according to the determination result in step S203. To do.
  • step S205 the observation-side control circuit 110 determines whether or not the count scan in a predetermined entire region has ended based on the end condition CSP3 recorded in the observation-side recording circuit 130 as count scan processing information, for example. To do. The process proceeds to step S211 when it is determined that the entire region has been completed, and proceeds to step S206 when it is not determined.
  • the observation-side control circuit 110 causes the imaging unit 151 to perform focus adjustment (Auto Focus: AF) on the cell 324 that is the subject of interest, and to perform an imaging operation to acquire a second image.
  • focus adjustment Auto Focus: AF
  • the imaging optical system is used during AF. 152, the size and position of the cell 324 do not change even if the position in the optical axis direction of the image sensor 153 or the like changes.
  • the observation-side control circuit 110 causes the observation-side communication device 140 to transmit the second image acquired as described above with reference to FIG. 9 to a preset transmission destination.
  • step S207 the observation-side control circuit 110 causes the image processing circuit 120 to analyze the acquired second image, count the number of cells 324 or cell groups, and also display the cell count result as the observation-side recording circuit 130. Or to the controller 200. Thereafter, the process proceeds to step S208.
  • the cell count may be performed by the controller 200.
  • the cell count is performed outside the observation system 1. Also good. Further, the cell count may be performed based on a wide range of high pixel images synthesized based on the acquired second image after the count scan process of the entire region is completed.
  • step S208 the observation-side control circuit 110 determines whether or not the current state satisfies the first X ⁇ Y condition CSP10 or the first Y ⁇ X condition CSP11 recorded as count scan processing information. judge. The process proceeds to step S209 when it is determined that the first X ⁇ Y condition CSP10 or the first Y ⁇ X condition CSP11 is satisfied, and the process proceeds to step S210 when it is not determined.
  • step S209 the observation-side control circuit 110 switches the direction in which the image acquisition unit 150 is moved according to the determination result in step S208. Thereafter, the process proceeds to step S210.
  • step S210 the observation-side control circuit 110 moves the image acquisition unit 150 by the first X movement pitch or the first Y movement pitch according to the movement direction at that time. Thereafter, the process returns to step S203.
  • step S211 the observation-side control circuit 110 causes the observation-side communication device 140 to transmit an end signal to the controller 200 when it is determined in step S205 that the count scan has been completed in the entire area.
  • the observation side control circuit 110 causes the image processing circuit 120 to synthesize a wide range of high pixel images based on the acquired second image. Thereafter, the process ends, and the process proceeds to step S108 of the observation apparatus control process.
  • step S109 of the observation apparatus control process is shown in FIG. 11 as a flowchart, and the operation of the observation system 1 during the 3D scan process will be described with reference to this flowchart.
  • step S301 the observation-side control circuit 110 causes the lens switching unit 154 to make the imaging optical system 152 the first imaging optical system that is a non-telecentric optical system on the subject side. Thereafter, the process proceeds to step S302.
  • the observation-side control circuit 110 executes pre-processing for starting 3D scanning based on, for example, 3D scan processing information recorded in the observation-side recording circuit 130.
  • the observation-side control circuit 110 moves the image acquisition unit 150 to the moving mechanism 160 and returns it to the XY designated position for 3D scanning.
  • the observation-side control circuit 110 controls the operations of the imaging optical system 152 and the imaging element 153, or the moving mechanism 160 so that 3D scanning can be started from a predetermined position in the Z direction. Thereafter, the observation-side control circuit 110 starts 3D scanning.
  • 3D scan processing information is shown in FIG. 12, and information recorded as 3D scan processing information will be described with reference to this.
  • the information is set in advance, for example, or set in step S105 of the observation apparatus control process.
  • the 3D scan processing according to the present embodiment is executed using the first imaging optical system that is a subject-side non-telecentric optical system.
  • the 3D scanning process according to the present embodiment is a process executed on an area (specific area) including a specific position designated by the user.
  • the 3D scan processing information includes information TSP related to the 3D scan pattern, information TSJ related to execution of the 3D scan processing, and information TSR obtained by the 3D scan processing.
  • the information TSP related to the 3D scan pattern is a designated position TSP1, a range setting TSP2, a second X movement pitch TSP5, a second Y movement pitch TSP6, and a second condition for switching from movement in the X direction to movement in the Y direction.
  • X ⁇ Y condition TSP10, and a second Y ⁇ X condition TSP11 that is a condition for switching from movement in the Y direction to movement in the X direction.
  • the second X movement pitch TSP5 is a movement (imaging) interval in the X direction
  • the second Y movement pitch TSP6 is a movement (imaging) interval in the Y direction.
  • the observation-side control circuit 110 determines the specific area based on the designated position TSP1 and the range setting TSP2, and also determines the start position and end position of the 3D scan.
  • the information TSJ related to the execution of the 3D scanning process is, for example, when it is determined that the observation is defective according to the second NG determination condition TSJ1, which is a determination condition for determining the observation defect, for example, the second NG determination condition TSJ1.
  • a second retry determination condition TSJ2 that is a determination condition for determining whether or not to re-execute the 3D scan is included.
  • the information TSR obtained by the 3D scanning process is recorded in association with each image (first image) acquired by using the first imaging optical system in the 3D scanning process.
  • the first result TSR1 includes the first frame TSR11, the first time TSR12 when the first frame TSR11 is acquired, the first depth information TSR13, and the first 3D shooting condition TSR14.
  • the depth information may include the position of the cell 324 in the optical axis direction of the first imaging optical system, information on a 3D model that can be constructed based on the position information, and the like.
  • FIG. 13 an example of the movement pattern of the image acquisition unit 150 in the 3D scan processing according to the present embodiment is shown in FIG. 13 as a schematic diagram, and the movement of the image acquisition unit 150 in the 3D scan will be described with reference to this.
  • a case where the image acquisition unit 150 performs the 3D scan process while being moved on the line TL1 illustrated in FIG. 13 will be described as an example.
  • the observation-side control circuit 110 moves the image acquisition unit 150 to the line TL1 in a region (specific region) that is generally centered on the designated position TSP1 indicated by the point TP0 according to the range setting TSP2.
  • the first image is acquired at each position by picking up an image while moving it up.
  • the movement of the image acquisition unit 150 during the 3D scan is substantially the same as the movement of the image acquisition unit 150 during the count scan described with reference to FIG.
  • the image acquisition unit 150 starts moving from the start position TP1, and moves in the Y direction until the second Y ⁇ X condition TSP11 is satisfied at the point TP2. Thereafter, the robot moves in the X direction at the point TP3 until the second X ⁇ Y condition TSP10 is satisfied. In this way, the observation-side control circuit 110 continues the 3D scan process until the image acquisition unit 150 reaches the point TP10.
  • step S303 the observation-side control circuit 110 determines the 3D scan state in the same manner as in step S203 of the count scan process.
  • the determination conditions used here are the second NG determination condition TSJ1 and the second retry determination condition TSJ2.
  • the process proceeds to step S304 if it is determined that the 3D scan needs to be re-executed, and proceeds to step S305 if it is not determined.
  • step S304 the observation-side control circuit 110 warns the user that an observation defect or the like has occurred in the 3D scan according to the determination result in step S303, and that it is necessary to re-execute the 3D scan.
  • a control signal is generated and transmitted to the controller 200. Thereafter, the process returns to step S302.
  • step S305 the observation-side control circuit 110 determines whether or not the 3D scan in the specific area has ended. The process proceeds to step S310 when it is determined that the 3D scan in the specific area has been completed, and proceeds to step S306 when it is not determined.
  • step S306 the observation-side control circuit 110 causes the imaging unit 151 to perform an imaging operation to acquire a first image.
  • the first imaging optical system which is an optical system having non-telecentricity on the subject side is used.
  • the observation-side control circuit 110 causes the observation-side communication device 140 to transmit the acquired first image to a preset transmission destination.
  • the observation-side control circuit 110 performs the same processes as steps S208 to S210 in the count scan process.
  • the observation-side control circuit 110 determines whether or not the second X ⁇ Y condition TSP10 or the second Y ⁇ X condition TSP11 is satisfied in step S307, and if it is determined that these conditions are satisfied.
  • the moving direction is switched.
  • the observation-side control circuit 110 moves the image acquisition unit 150 according to the movement direction and the value of the second X movement pitch TSP5 or the second Y movement pitch TSP6. Thereafter, the process returns to step S303.
  • step S310 the observation-side control circuit 110, as described above with reference to FIGS. 5 and 6, based on the result of causing the image processing circuit 120 to perform image processing on the first image, the depth associated with the cell 324. Get information.
  • the depth information includes information related to the unevenness of the cell 324 or the cell group in the optical axis direction of the first imaging optical system. Therefore, the depth information acquired here includes, for example, a 3D model indicating the three-dimensional shape of the cell 324 or the cell group generated by the three-dimensional model generation unit.
  • the depth information may be acquired by the controller 200.
  • the controller 200 when the controller 200 is the transmission destination of the first image and the controller 200 includes an image processing circuit, the depth information may be acquired by the controller 200.
  • the controller 200 when a server on a network such as a cloud is a transmission destination of the first image and the cloud has a function corresponding to an image processing circuit, the depth information is acquired outside the observation system 1. It may be broken.
  • step S311 the observation-side control circuit 110 causes the observation-side communication device 140 to transmit an end signal to the controller 200. Thereafter, the process ends, and the process proceeds to step S110 of the observation apparatus control process.
  • FIG. 14 An example of a controller control process performed by the controller 200 is shown in FIG. 14 as a flowchart, and the operation of the observation system 1 will be described with reference to this flowchart.
  • the process illustrated in the flowchart of FIG. 14 starts, for example, in a state where the observation apparatus 100 is waiting for communication.
  • step S401 the controller-side control circuit 210 generates display information for presenting various functions provided in the controller 200 to the user using text, icons, and the like, and causes the display device 272 to display the display information.
  • step S402 the controller-side control circuit 210 determines whether the activation of the inspection application is instructed based on, for example, a control signal output from the input device 274 in accordance with a user operation result.
  • the inspection application is application software having a program for communicating with the observation apparatus 100 to control the observation apparatus 100.
  • the controller control process proceeds to step S403 when it is determined that the activation of the inspection application is instructed, and returns to step S401 when it is not determined.
  • the controller 200 is, for example, a tablet PC or a smartphone, and in this step, a telephone application or a mail application can be selected in addition to the inspection application. In the following description, only the case where the inspection application is selected will be described as an example.
  • step S403 the controller side control circuit 210 accesses the designated camera.
  • the designated camera is an imaging device to be controlled by the inspection application selected in step S402, for example.
  • the description will be continued assuming that the designated camera is the observation apparatus 100.
  • step S ⁇ b> 404 the controller-side control circuit 210 performs an operation for turning on the observation apparatus 100 by the user or a power supply for the observation apparatus 100 based on, for example, a control signal output from the input apparatus 274 according to the operation result of the user. It is determined whether or not an operation to turn off (imaging ON / OFF operation) has been performed. The process proceeds to step S405 if it is determined that the imaging ON / OFF operation has been performed, and proceeds to step S406 if it is not determined.
  • imaging ON / OFF operation an operation to turn off
  • step S405 the controller-side control circuit 210 turns off the power ON signal for turning on the observation device 100 or the power to the observation device 100 based on the result of the user's imaging ON / OFF operation detected in step S404.
  • the controller-side communication device 240 is caused to transmit a power OFF signal to the observation device 100.
  • the process returns to step S403. Note that the process in this step corresponds to steps S102 to S103 in the observation apparatus control process.
  • step S406 the controller-side control circuit 210, for example, based on a control signal output from the input device 274 in accordance with a user operation result, the transmission destination of an observation result or measurement result such as an image acquired by the observation device 100 by the user. It is determined whether or not various settings including information, shooting conditions, measurement conditions, and various parameters are performed. The process proceeds to step S407 if it is determined that various settings have been made, and proceeds to step S408 if it is not determined.
  • step S407 the controller-side control circuit 210 causes the controller-side communication device 240 to transmit control signals related to various settings detected in step S406 to the observation device 100. Thereafter, the process proceeds to step S408. Note that the process in this step corresponds to steps S104 to S105 in the observation apparatus control process.
  • step S408 the controller-side control circuit 210 determines whether or not the user has instructed execution of the count scan process based on, for example, a control signal output from the input device 274 in accordance with the result of the user operation. The process proceeds to step S409 when it is determined that execution of the count scan process is instructed, and proceeds to step S410 when it is not determined.
  • step S409 the controller-side control circuit 210 causes the controller-side communication device 240 to transmit a control signal instructing execution of the count scan process to the observation device 100. Thereafter, the process proceeds to step S410. Note that the process in this step corresponds to steps S106 to S107 in the observation apparatus control process.
  • step S410 the controller-side control circuit 210 determines whether or not the user has instructed the execution of the 3D scanning process based on, for example, a control signal output from the input device 274 in accordance with the result of the user operation.
  • the controller-side control circuit 210 also determines whether or not the user has set the designated position TSP1, the range setting TSP2, and the like related to the specific area where the 3D scan is executed.
  • the process proceeds to step S411 when it is determined that the execution of the 3D scan process is instructed or the setting related to the specific area is performed, and when it is not determined, the process proceeds to step S412.
  • step S411 if the execution of the 3D scan process is instructed in step S410, the controller side control circuit 210 causes the controller side communication device 240 to transmit a control signal instructing the execution of the 3D scan process to the observation apparatus 100. .
  • the controller-side control circuit 210 causes the controller-side communication device 240 to transmit the control signal related to the setting to the observation device 100. Thereafter, the process proceeds to step S412. Note that the process of this step corresponds to steps S108 to S109 of the observation apparatus control process.
  • step S412 the controller-side control circuit 210 determines whether to receive a measurement result or the like from the outside of the controller 200, for example, according to the result of the user operation. The process proceeds to step S413 when it is determined that the measurement result is received, and proceeds to step S414 when it is not determined.
  • step S413 the controller-side control circuit 210 acquires the measurement results and the like acquired by the observation apparatus 100, and displays the measurement results and the like on the display device 272.
  • the measurement result or the like may be acquired from the observation apparatus 100, or may be acquired from the transmission destination of the measurement result of the observation apparatus 100 set in step S406. Thereafter, the process proceeds to step S414. Note that the process of this step corresponds to steps S111 to S112 of the observation apparatus control process.
  • step S414 the controller-side control circuit 210 determines whether or not to end the inspection application, for example, according to the operation result of the user. If it is determined that the process is to end, the inspection application is ended and the process returns to step S401. If it is determined that the process is not ended, the process returns to step S403.
  • the lens switching unit 154 is driven in the optical axis direction of the imaging optical system 152 of the lens included in the imaging optical system 152, and the first imaging optical system and the second imaging optical system are driven.
  • the observation apparatus 100 may include a first imaging optical system and a second imaging optical system separately.
  • the lens switching unit 154 moves the optical axis of the imaging optical system used for imaging out of the first imaging optical system and the second imaging optical system so as to be parallel to the observation axis.
  • the imaging optical system is switched.
  • the illumination unit 155 is described as being disposed on the support unit 165, it is only necessary that the light emitting unit of the illumination optical system 156 be disposed on the support unit 165.
  • the light source 157 may be used for observation. It may be placed anywhere on the device 100. Note that, for example, the intensity of illumination may be changed depending on the type of observation in order to reduce damage to the observation target such as the cell 324.
  • control of illumination light there can be a control method of intermittent illumination such that the sample 300 is illuminated only at the moment of photographing, and a control method of increasing or decreasing the number of lighting illuminations.
  • the case where the image acquisition unit 150 starts moving in the Y direction from the start position has been described as an example.
  • the present invention is not limited to this.
  • scanning starts from the start position in the X direction. May be.
  • the observation apparatus 100 causes the moving mechanism 160 to change the position of the image acquisition unit 150 in the X direction and the Y direction while maintaining the optical axis of the imaging optical system 152 in parallel with the observation axis.
  • the sample 300 is repeatedly photographed to acquire a plurality of images.
  • the observation apparatus 100 uses the second imaging optical system, which is a subject-side telecentric optical system, to acquire a second image suitable for synthesis of a wide range of high-pixel images.
  • the second image or the high pixel image is suitable for acquiring the shape and number of the subject of interest such as a cell.
  • the observation apparatus 100 according to the present embodiment uses a first imaging optical system that is a subject-side non-telecentric optical system, thereby causing parallax from a plurality of first images acquired at different imaging positions.
  • a first image movement amount ⁇ X including the image movement amount is acquired, and depth information of a subject of interest such as a cell is acquired based on the first image movement amount ⁇ X.
  • the lens switching unit 154 switches the imaging optical system 152 according to the type of observation.
  • the observation apparatus 100 includes the lens switching unit 154 that switches between the first imaging optical system and the second imaging optical system, and thus is suitable for, for example, acquisition of depth information and cell counting. Observations with different required optical characteristics, such as image acquisition, can be realized.
  • the observation system 1 can accurately project the position of the cell in the optical axis direction, the three-dimensional model of the cell, the shape of the cell, etc. only by specifying the desired observation method. Images, cell count results, etc. can be acquired.
  • the cell 324 is used when the first imaging optical system is used.
  • the case where the depth information of the subject of interest such as is acquired has been described as an example.
  • a light beam that can reach the imaging element 153 has an inclination between the cell 324 and the objective lens 152b with respect to the optical axis of the first imaging optical system. Therefore, the first image includes information related to the side surface of the subject that exists at a position other than the optical axis. Therefore, in the present embodiment, the observation system 1 that generates a side observation image as if the side surface of the cell 324 was imaged based on the first image acquired by using the first imaging optical system will be described. do.
  • the observation apparatus 100 further has functions as a stereoscopic image generation unit and a side information processing unit.
  • the stereoscopic image generation unit generates a stereoscopic image of the subject of interest based on, for example, a plurality of first images and depth information.
  • the side information processing unit acquires information related to various side observations such as image data and position information obtained by the side observation processing, and generates a side observation image based on the first image and the depth information.
  • the functions as the stereoscopic image generation unit and the side information processing unit can be realized by the observation-side control circuit 110 and / or the image processing circuit 120, for example.
  • the functions as the stereoscopic image generation unit and the side information processing unit may be realized by the controller-side control circuit 210, respectively.
  • FIG. 15 shows a schematic diagram of an example of the positional relationship between the imaging unit 151 including the first imaging optical system according to the present embodiment and the sample 300.
  • the state shown in FIG. 15 is a state where the point P1 on the cell 324 is moved by the third X movement amount ⁇ X3 in the X + direction from the state where it is located on the optical axis of the first imaging optical system. That is, when the image position of the point P1 is a corresponding point, the position of the corresponding point is in a state of moving from the point U1 to the point U1 ′.
  • the light ray R10 emitted from the point P1 on the cell 324 is incident on the point U1 ′ on the image sensor 153, and the light ray R30 emitted from the point P3 is incident on the point U3 on the image sensor 153.
  • the area A1 on the cell is imaged as the area A1 ′ on the image sensor 153.
  • the area between the position of the point P1 and the position of the point P3 corresponds to the in-focus range (within the depth of focus). That is, it is assumed that the area A1 is in focus.
  • the focused area A1 of the subject of interest is referred to as a specific range
  • the region A1 ′ obtained by capturing the specific range of the subject of interest in the first image is referred to as a specific range image. I will call it.
  • the observation apparatus 100 causes the image processing circuit 120 to cut out the specific range image from each first image. Further, the image processing circuit 120 synthesizes the specific range images based on depth information about the subject included in each specific range image, for example, and acquires a side observation image.
  • the side information processing unit included in the observation device 100 according to the present embodiment combines the plurality of specific range images acquired in this manner, thereby imaging the cell 324 from below, and the side information processing unit A side observation image such as a depth composite image obtained by imaging can be acquired.
  • FIG. 16 an example of a side observation image according to the present embodiment is shown in FIG. 16 as a schematic diagram.
  • the observation device 100 acquired from each first image, A plurality of specific range images including the first specific range image I10, the second specific range image I11, and the third specific range image I12 are synthesized.
  • the width W in the Z-axis direction (after image processing) in the side-view image of each specific range image to be synthesized is, for example, the optical axis between the points P1 and P3 when the region A1 ′ is the specific range image.
  • the width W is the difference between the depth Z3 and the depth Z1.
  • the width W in the side-view observation image of each specific range image can be different.
  • the side image is actively utilized to perform side observation, and it is more important and abundant than ever to inspect and observe the condition of the object with a simple configuration. It is possible to obtain image information (stereoscopic conditions such as shading, color, and structure).
  • the three-dimensional information acquisition unit acquires the three-dimensional information of the sample 300 based on the information related to each imaging position at the time of movement. At least one of the plurality of first images is the first imaging optical system.
  • the image obtained in (1) is an image other than on the optical axis of the optical system.
  • the width W in the Z-axis direction of each image may be determined based on, for example, the first image movement amount ⁇ X or the angle of view ⁇ , or may be determined based on the focal length and the width of the focusing range. May be.
  • the present invention is not limited to this.
  • the region on the first image cut out as the specific range image may be a region that is separated from the optical axis of the first imaging optical system by a predetermined threshold or more.
  • it is preferable that the range is less affected by image distortion that may be present at the periphery of the image sensor 153.
  • each acquired first Of these images a region that is focused on the cell 324 and is separated from the optical axis by a predetermined threshold or more may be cut out as a specific range image and image synthesis may be performed.
  • the image composition is performed using information on the imaging position acquired simultaneously with the image.
  • a score is obtained based on the presence or absence of overexposure or blackout, the degree of focus on the cell 324, etc., and is used for image synthesis according to the score.
  • the specific range image to be selected may be selected.
  • the range focused on the subject of interest in the first image is cut out as the specific range image.
  • the observation apparatus 100 determines a region to be imaged of the subject of interest, divides it into a plurality of regions, and repeats imaging focused on the divided regions to obtain the first image. Obtaining and using the range including the region of the first image as the specific range image.
  • the observation apparatus 100 uses an area corresponding to a specific area that is a predetermined threshold or more away from the optical axis on the image sensor 153 as an AF area, and performs first focusing so as to focus on the AF area. The Z position of the imaging optical system is adjusted.
  • the observation apparatus 100 may acquire the side observation image while fixing the region occupied by the specific range image in the first image. Further, for example, the observation apparatus 100 acquires the first image while moving the image acquisition unit 150 in each of the X direction, the Y direction, and the Z direction according to a preset movement pattern, as in 3D scanning. Then, the side observation image may be acquired by using the range focused from the acquired first image as the specific range image.
  • the present invention is not limited to this.
  • the acquisition of the side observation image may be performed while being moved, for example, in the X direction and the Y direction according to the set or selected movement pattern.
  • the side surface information processing unit included in the observation apparatus 100 according to the present embodiment cannot be observed using the subject-side telecentric optical system such as the sample 300 observed from the X + direction in the state illustrated in FIG. A side observation image that captures the side surface of the cell 324 can be acquired.
  • the side observation process is performed, for example, after step S205 and before step S210 in the observation apparatus control process described above with reference to FIG.
  • the side observation process receives a control signal instructing execution of the side observation process output by the controller 200 according to, for example, a user operation result in the same manner as the count scan process or the 3D scan process. It is started when it is determined.
  • various types of information such as scan patterns and determination conditions required in the side observation processing are recorded in the observation side recording circuit 130 as side observation processing information, for example.
  • the side observation processing information includes a result acquired in the side observation processing.
  • step S501 the observation-side control circuit 110 causes the lens switching unit 154 to set the imaging optical system 152 as the first imaging optical system in the same manner as in step S301 of the 3D scanning process. Thereafter, the process proceeds to step S502.
  • step S502 the observation-side control circuit 110 performs pre-processing in the same manner as in step S302 of 3D scanning processing. In addition, the observation side control circuit 110 starts scanning for acquiring a side observation image. Thereafter, the process proceeds to step S503.
  • steps S503 and S504 the observation-side control circuit 110 performs pre-processing, determination regarding NG determination conditions and retry determination conditions, warning processing as necessary, and the like in the same manner as in steps S303 and S304 of 3D scanning processing. I do.
  • the process proceeds to step S505 when it is determined in step S503 that the NG determination condition and the retry determination condition are not satisfied, and in step S504 when it is determined that the NG determination condition or the retry determination condition is satisfied in step S503. After giving a warning if necessary, the process returns to step S502.
  • step S505 the observation-side control circuit 110 determines whether or not the side observation image acquisition in the specific area is completed. If it is determined that the process has been completed, the process proceeds to step S508; otherwise, the process proceeds to step S506.
  • step S506 the observation-side control circuit 110 performs AF on a region (specific range) in which the distance from the optical axis of the first imaging optical system is equal to or greater than a predetermined threshold.
  • a region specifically range
  • the distance from the optical axis of the first imaging optical system is equal to or greater than a predetermined threshold.
  • step S507 the observation side control circuit 110 moves the image acquisition unit 150 to the moving mechanism 160 according to the scan pattern recorded as the side observation processing information. For example, the observation-side control circuit 110 moves in a direction corresponding to the inclination of the principal ray emitted from the specific range. Thereafter, the process returns to step S503.
  • the observation-side control circuit 110 causes the image processing circuit 120 to set a specific range from each first image acquired in the side-view observation scan in the specific area as described above with reference to FIG. The captured area is cut out as a specific range image.
  • the observation-side control circuit 110 causes the image processing circuit 120 to convert the plurality of specific range images into an appropriate width W based on the depth information, and to synthesize the converted images.
  • step S509 the observation side control circuit 110 transmits the side observation image to, for example, a preset transmission destination.
  • the transmission destination may be determined based on a control signal output by the controller 200 in accordance with a user operation result in this step. Thereafter, the process ends.
  • the observation system 1 according to the present embodiment has the following advantages in addition to the advantages obtained in the first embodiment.
  • the side information processing unit included in the observation apparatus 100 is an area on the cell 324 in an observation apparatus that performs observation or the like by switching between the first imaging optical system and the second imaging optical system according to the purpose.
  • the side surface of the cell 324 is imaged using the fact that there is an inclination between the principal ray radiated from a region separated from the optical axis of the first imaging optical system by a predetermined threshold or more and the optical axis. Acquired side observation image.
  • the first image including the side surface of the cell 324 that cannot be obtained by the observation device having telecentricity on the subject side, and the image based on the first image. Obtained side view images.
  • the observation apparatus 100 according to the present embodiment and the observation apparatus 100 according to the first embodiment can be combined.
  • the stereoscopic image generation unit performs a process of three-dimensionalizing the specific range image acquired as described in the present embodiment based on the depth information regarding each corresponding point included in the specific range image.
  • the observation device 100 acquires the stereoscopic image of the cell 324 by synthesizing the three-dimensional specific range image acquired in this way at the corresponding position of the stereoscopic model of the cell 324 described in the first embodiment. it can.
  • the objective lens 152b and the imaging lens 152c are illustrated as positive lenses for simplicity, but the present invention is not limited to this.
  • the objective lens may be a lens group having a negative refractive power in order to reduce the first image movement amount ⁇ X between the first images to be acquired and to increase the parallax. Needless to say, a plurality of lenses can be used according to the required performance.
  • observation apparatus 100 may perform the AF operation based on the depth information acquired from the parallax image when the first imaging optical system is used.
  • the observation apparatus 100 is used in an incubator, and emphasizes the use focused on cell observation.
  • the present invention can be generalized as an observation apparatus for enlarging and confirming details, which can realize acquisition of size and the like and acquisition of depth information related to an observation object.
  • the sample 300 can be put in and out of, for example, an incubator, a clean bench or the like while being placed on the upper surface of the observation apparatus 100.
  • the cell 324 will be affected by a temperature change and may receive a heat shock.
  • contamination may occur with taking in and out.
  • the present technology can appropriately acquire the shape and size of an observation object such as a cell or a cell group and depth information such as unevenness of the observation object such as a cell or a cell group.
  • the user is warned of the occurrence of observation failure according to the result of the determination using the NG determination condition and the retry determination condition for the image acquired by the observation apparatus 100.
  • the application of the present technology is not limited to this. The user can be warned even when an abnormality of the observation object, contamination, or the like is detected.
  • this technique evaluates the state of the culture medium based on the result of image analysis.
  • the observation-side control circuit 110, the image processing circuit 120, the observation-side recording circuit 130, and the observation-side communication device 140 are arranged inside the housing 101 as a circuit group 104.
  • the case where it is provided has been described as an example, it is not limited thereto.
  • one or more of these functions may be provided in the image acquisition unit 150.
  • the function as the observation-side communication device 140 may be provided in both the image acquisition unit 150 and the circuit group 104.
  • one or more functions of the observation side control circuit 110, the image processing circuit 120, and the observation side recording circuit 130 may be provided in the controller 200. That is, for example, some or all of the above-described various determinations, image processing, and the like may be performed by the controller 200.
  • some elements such as the input / output device 270 may be included in the observation device 100.
  • a configuration in which the observation apparatus 100 and the controller 200 are incorporated in one housing is also conceivable.
  • the observation system 1 in which the observation apparatus 100 and the controller 200 are integrated can be used when the user himself enters a use environment such as a temperature-controlled room.
  • the observation system 1 records and learns observation results such as image analysis, usage of the observation system 1 including the usage frequency of the user, incubator settings, and the like.
  • Artificial intelligence AI that presents parameters and the like may be included.
  • the AI may be built inside the observation system 1, for example, in a DSP or the like, or may be built on the Internet and outside the observation system 1.
  • the observation system 1 including such an AI can determine, for example, the state of the cell, the type, the state of the medium, the presence or absence of foreign matter, etc. with respect to the acquired image by referring to a database prepared on the server.
  • the present technology is also effective when applied to an imaging apparatus that is used at a position where the imaging optical system and the user are separated, such as a surveillance camera or an endoscope.
  • the user can acquire observation results such as images suitable for the purpose of use without having to replace the imaging device according to the purpose of imaging.
  • this invention is not limited to the said embodiment, In the implementation stage, it can change variously in the range which does not deviate from the summary. Further, the embodiments may be implemented in combination as appropriate, and in that case, the combined effect can be obtained. Furthermore, the present invention includes various inventions, and various inventions can be extracted by combinations selected from a plurality of disclosed constituent elements. For example, even if several constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the problem can be solved and an effect can be obtained, the configuration from which the constituent requirements are deleted can be extracted as an invention. *

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Optics & Photonics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Microscoopes, Condenser (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Apparatus Associated With Microorganisms And Enzymes (AREA)

Abstract

L'invention concerne un dispositif d'observation (100), comprenant : une unité d'imagerie (151) ; un mécanisme de déplacement (160) ; et une unité d'acquisition d'informations stéréoscopiques. L'unité d'imagerie (151) comprend un premier système optique d'imagerie, qui est un système optique grossissant et qui est un système optique non télécentrique côté sujet dans lequel un angle formé entre l'un des rayons principaux du côté sujet et l'axe optique est supérieur ou égal à 6°. L'unité d'imagerie (151) utilise le premier système optique d'imagerie pour mettre en image un échantillon (300) et pour acquérir une première image. Le mécanisme de déplacement (160) modifie des positions relatives de l'échantillon (300) et de l'unité d'imagerie (151). L'unité d'acquisition d'informations stéréoscopiques acquiert des informations stéréoscopiques de l'échantillon (300) sur la base d'une pluralité des premières images acquises à différentes positions de l'unité d'imagerie (151) et d'informations concernant des positions d'imagerie respectives utilisées pour acquérir les premières images.
PCT/JP2017/040772 2017-02-21 2017-11-13 Dispositif et système d'observation ainsi que procédé de commande de dispositif d'observation WO2018154871A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017029999A JP6330070B1 (ja) 2017-02-21 2017-02-21 観察装置、観察システム及び観察装置の制御方法
JP2017-029999 2017-02-21

Publications (1)

Publication Number Publication Date
WO2018154871A1 true WO2018154871A1 (fr) 2018-08-30

Family

ID=62186841

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/040772 WO2018154871A1 (fr) 2017-02-21 2017-11-13 Dispositif et système d'observation ainsi que procédé de commande de dispositif d'observation

Country Status (2)

Country Link
JP (1) JP6330070B1 (fr)
WO (1) WO2018154871A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210102887A1 (en) * 2018-06-08 2021-04-08 Olympus Corporation Observation device

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7410701B2 (ja) * 2019-12-09 2024-01-10 株式会社ミツトヨ アダプタ光学系および焦点距離可変光学系

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006011145A (ja) * 2004-06-28 2006-01-12 Olympus Corp 双眼顕微鏡装置
JP2013088490A (ja) * 2011-10-14 2013-05-13 Nikon Corp 顕微鏡、画像取得方法、プログラム、及び記録媒体
JP2016070720A (ja) * 2014-09-29 2016-05-09 株式会社ミツトヨ 画像測定装置及び画像測定装置のガイダンス表示方法
JP2016157197A (ja) * 2015-02-23 2016-09-01 株式会社リコー 自己位置推定装置、自己位置推定方法およびプログラム

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4740826B2 (ja) * 2006-02-23 2011-08-03 株式会社神戸製鋼所 形状測定装置、形状測定方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006011145A (ja) * 2004-06-28 2006-01-12 Olympus Corp 双眼顕微鏡装置
JP2013088490A (ja) * 2011-10-14 2013-05-13 Nikon Corp 顕微鏡、画像取得方法、プログラム、及び記録媒体
JP2016070720A (ja) * 2014-09-29 2016-05-09 株式会社ミツトヨ 画像測定装置及び画像測定装置のガイダンス表示方法
JP2016157197A (ja) * 2015-02-23 2016-09-01 株式会社リコー 自己位置推定装置、自己位置推定方法およびプログラム

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210102887A1 (en) * 2018-06-08 2021-04-08 Olympus Corporation Observation device
US11635364B2 (en) * 2018-06-08 2023-04-25 Evident Corporation Observation device

Also Published As

Publication number Publication date
JP6330070B1 (ja) 2018-05-23
JP2018136178A (ja) 2018-08-30

Similar Documents

Publication Publication Date Title
US8982457B2 (en) Microscope system and illumination intensity adjusting method
CN104111524B (zh) 数码显微镜以及优化数码显微镜中工作流程的方法
US10116848B2 (en) Illumination and imaging system for imaging raw samples with liquid in a sample container
CN106210520B (zh) 一种自动调焦电子目镜及系统
TW201126624A (en) System and method for inspecting a wafer (2)
CN110261385B (zh) 外观检查系统、图像处理装置、拍摄装置以及检查方法
US11119382B2 (en) Tunable acoustic gradient lens system with amplitude adjustment corresponding to z-height as indicated by calibration data
US9291750B2 (en) Calibration method and apparatus for optical imaging lens system with double optical paths
US11249225B2 (en) Tunable acoustic gradient lens system utilizing amplitude adjustments for acquiring images focused at different z-heights
JP6698451B2 (ja) 観察装置
JP6330070B1 (ja) 観察装置、観察システム及び観察装置の制御方法
US10827114B2 (en) Imaging system and setting device
JP2013034127A (ja) 撮像装置
CN109194851A (zh) 一种小型化超短焦视觉成像系统
JPH1172717A (ja) 顕微鏡デジタル写真撮影システム
JP6640610B2 (ja) 観察装置、測定システム及び観察方法
JP6815787B2 (ja) 観察装置、観察装置の制御方法及び観察装置の制御プログラム
CN113596441B (zh) 光轴调整装置、方法、系统及可读存储介质
JP2007327903A (ja) 外観検査装置
CN203759357U (zh) 光学取像系统及光学检测系统
CN114689281A (zh) 检测光学模组光瞳漂移的方法
JP2007103787A (ja) 固体撮像素子の検査装置
JP5362981B2 (ja) 撮像装置
WO2020039920A1 (fr) Système et procédé de traitement d'image, et programme
JP2019033436A (ja) 撮像装置、撮像システム及び撮像装置の制御方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17897409

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17897409

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载