US20080049993A1

US20080049993A1 - System and method for counting follicular units

Info

Publication number: US20080049993A1
Application number: US11/467,283
Authority: US
Inventors: Shehrzad A. Qureshi; Mohan Bodduluri
Original assignee: Restoration Robotics Inc
Current assignee: Venus Concept Inc
Priority date: 2006-08-25
Filing date: 2006-08-25
Publication date: 2008-02-28
Also published as: KR101030853B1; US8199983B2; BRPI0715630A2; CA2661660C; CN101505659A; US20120230561A1; WO2008024955A2; JP4988847B2; AU2007286606B2; US8290229B2; US20090052738A1; CN101505659B; AU2007286606A1; JP2010501288A; EP2053973A2; WO2008024955A3; EP2053973A4; WO2008024955A9; CA2661660A1; KR20090040336A

Abstract

A system and method for counting follicular units using an automated system comprises (i) acquiring a digital image of a body surface having skin and a plurality of follicular units; (ii) selecting a region of interest with the digital image; (iii) segmenting the selected image to produce a binary image; (iv) performing a morphological open operation on the binary image; and (v) performing noise filtering by removing objects having certain characteristics which do not correspond to hair.

Description

FIELD OF INVENTION

This invention relates generally to hair transplantation procedures and more particularly to a system and method for counting follicular units using digital imaging and processing techniques for use in hair transplantation procedures.

BACKGROUND

Hair transplantation procedures are well-known, and typically involve (in a patient having male pattern baldness) harvesting donor hair grafts from the side and back fringe areas (donor areas) of the patient's scalp, and implanting them in a bald area (recipient area). Historically, the harvested grafts were relatively large (3-5 mm), although more recently, the donor grafts may be single follicular units. In particular, “follicular units” (also referred to herein as FU or FUs) are naturally occurring aggregates of 1-3 (and much less commonly, 4-5) closely spaced hair follicles that are distributed randomly over the surface of the scalp.
The follicular units may be classified, or “typed,” based on the number of hairs in the unit and identified in shorthand as an “F1” for a single hair follicular unit, an “F2” for a two hair follicular unit and so on for follicular units with 3-5 hairs. In some cases of multiple hair follicular units, the hairs may appear to emanate from a single follicle or point in the skin. In other cases, the hairs may exit the skin surface at slightly spaced apart positions, but converge into a single follicular unit beneath the skin. Referring to FIG. 1, a print of a digital image of an exemplary section of a human scalp 11 having a variety of types of follicular units is shown. For example, the follicular unit 13 has two hairs and is therefore an F2, while follicular unit 15 is an F1 since it has only a single hair. Similarly, follicular unit 17 appears to be an F3 having three hairs.
There are several reasons it is important and desirable to count and classify follicular units in a region of interest on a body surface. For one, the number of follicular units can be used in the planning process for a transplantation procedure. For instance, if this number sets the limit on the number of follicular units in that area that can be harvested for transplantation. However, in many cases, the doctor may want to implant only a certain percentage of the follicular units available, thereby leaving some coverage in the area being harvested. In addition, in many hair restoration transplant procedures, certain classes of follicular units are preferred.
As for classification, there are several reasons it is important and desirable to identify and classify follicular units based on the number of hairs in the follicular unit. It may be desirable to utilize a variety of classes (also referred to as “types”) of follicular units to provide the desired attributes for the appearance of the transplanted hair. Such attributes can include the density of hair, the direction or orientation of hair, the particular mix of types of follicular units, and/or the appearance of randomness, among other possible attributes. An example of the use of various types of follicular units is as follows. It is preferable to transplant certain classes of follicular units into specific regions of the scalp. For example, single hair follicular units (F1s) are commonly implanted along the hairline that frames the face. Follicular units with more than one hair (F2s, F3s, etc.) are commonly implanted in the mid-scalp and crown. This arrangement of follicular unit distribution is thought to produce a more natural appearing aesthetic result.
Various procedures for hair transplantation have been previously disclosed, including both manual and mechanized to certain degrees of automation. In one well-known manual process, a linear portion of the scalp is removed from a donor area by dissection with a scalpel down into the fatty subcutaneous tissue. The strip is dissected (under a microscope) into the component follicular units, which are then implanted into a recipient area in respective puncture holes made by a needle. Forceps are typically used to grasp and place the follicular unit grafts into the needle puncture locations, although other instruments and methods are known for doing so.
In “Androgenetic Alopecia” (Springer 1996), M. Inaba & Y. Inaba disclose and describe a manual method for harvesting singular follicular units by positioning a hollow punch needle having a cutting edge and interior lumen with a diameter of 1 mm, which is about equal to the diameter of critical anatomical parts of a follicular unit. The needle punch is axially aligned with an axis of a follicular unit to be extracted and then advanced into the scalp to cut the scalp about the circumference of the selected follicular unit. Thereafter, the follicular units are easily removed, e.g., using forceps, for subsequent implantation into a recipient site with a specially devised insertion needle.
U.S. Pat. No. 6,585,746 discloses an automated hair transplantation system utilizing a robot, including a robotic arm and a hair follicle introducer associated with the robotic arm. A video system is used to produce a three-dimensional virtual image of the patient's scalp, which is used to plan the scalp locations that are to receive hair grafts implanted by the follicle introducer under the control of the robotic arm. The entire disclosure of U.S. Pat. No. 6,585,746 is incorporated herein by reference.
U.S. Pat. No. 6,585,746 discloses an automated hair transplantation system utilizing a robot, including a robotic arm and a hair follicle introducer associated with the robotic arm. A video system is used to produce a three-dimensional virtual image of the patient's scalp, which is used to plan the scalp locations that are to receive hair grafts implanted by the follicle introducer under the control of the robotic arm. The entire disclosure of U.S. Pat. No. 6,585,746 is incorporated herein by reference.
Automated systems and methods for transplanting are also disclosed in U.S. provisional patent application Ser. Nos. 60/722,521, filed Sep. 30, 2005, 60/753,602, filed Dec. 22, 2005, and 60/764,173, filed Jan. 31, 2006, and U.S. patent application Ser. Nos. 11/380,903, filed Apr. 28, 2006 and 11/380,907, filed Apr. 28, 2006. The foregoing applications are all hereby incorporated by reference into the present application in their entirety.
For example, U.S. patent application Ser. No. 11/380,907, referenced above, the disclosed system comprises a robotic arm having a harvesting and/or implantation tool mounted on the arm. One or more cameras are also mounted on the arm and are used to image the work space, such as a body surface. A processor is configured to receive and process images acquired by the cameras. A controller is operatively coupled to the processor and the robotic arm. The controller controls the movement of the robotic arm based, at least in part, on the processed images acquired by the cameras and the processor. The arm is controllably moveable to position the tool at a desired orientation and position relative to the body surface to perform transplantation of hairs.
In utilizing any of these systems and methods for hair transplantation, it is desirable to first plan the transplantation to select the follicular units to be harvested and transplanted and to determine the precise location where the hairs are to be implanted. Accordingly, in planning a hair transplantation procedure, specific follicular units from a specific location on a body surface may be selected for harvesting and transplantation into a different part of the body surface. The follicular units to be transplanted may be selected based on certain criteria, for example, the type of follicular unit (i.e. F1, F2, etc.), the orientation of the hair in the follicular unit, the density of the hair, etc. However, the process of counting, and characterizing each follicular unit can be tedious and time consuming. Therefore, there is a need for a system and method for counting and/or classifying follicular units using an automated system. A system and method for classifying follicular units is described in U.S. patent application Serial No. (not yet assigned), filed on or about Aug. 25, 2006, entitled SYSTEM AND METHOD FOR CLASSIFYING FOLLICULAR UNITS, the contents of which is incorporated by reference herein in its entirety.

SUMMARY

In accordance with a general aspect of the inventions disclosed herein, a system and method for counting follicular units using an automated system is provided. The system and method of the present invention may be utilized with systems and methods for transplantation of hair follicular units on a body surface, such as a human scalp. The system and method of the present invention is especially useful when implemented on, or integrated with, an automated system for hair transplantation.
In one aspect of the present invention, the method of counting follicular units comprises first acquiring a digital image of a body surface having skin and follicular units. A region of interest within the digital image which is known to contain one or more follicular units (FU) is selected. This region of interest within the digital image is called the selected image. The selected image is digitally filtered using a band-pass filter to remove components in the image which correspond to the skin. The filtered image is then segmented to produce a binary image of the region of interest.
A morphological open operation is performed on the binary image to further process the image in preparation for the subsequent noise filtering step. A morphological open operation is a standard image processing technique known by those of ordinary skill in the art. Noise filtering is then performed on the image resulting from the morphological open operation. The noise filtering removes objects which do not meet criteria corresponding to a follicular unit. For example, the area, location or orientation of an object in the image may be one whose area, location or orientation does not correspond to an actual follicular unit (it could be cut hair that happens to be remaining on the scalp, for example). Whether the characteristics of an image of an object corresponds to hair may be determined by statistical comparison to the global nature of the same characteristics for images of objects in the selected image which are known to be hair, or alternatively, the characteristics can be compared to predetermined criteria based on patient sampling or other data (e.g., if the patient parts the hair in a certain way we know that the hairs should mostly be pointing in a given direction).
Each of the objects remaining in the image after the noise filtering is counted as a follicular unit. Thus, the method may be used to count follicular units.
In another aspect of the method of counting follicular units, the step of digitally filtering the selected image using a band-pass filter may comprise a first filtering step using a low-pass filter having a first kernel and a second filtering step using a low-pass filter having a second kernel. In another feature of the present invention, the low-pass kernels may be Gaussian kernels. Those of ordinary skill in the art are familiar with, and understand how to implement, such low-pass filters and Gaussian filters.
In another embodiment of the method of the present invention, the step of acquiring the digital image of the body surface includes a method for tracking the FU of interest and aligning the camera(s) to obtain the digital image. First and second cameras which provide stereo images. The stereo images may be used to track an FU of interest within the digital images of the first and second cameras to adjust for movement of the body surface and/or movement of the cameras. In addition, the first and second cameras are aligned with the general orientation of the hair of the FU. The stereo images may also be used to compute coordinate positions of the hairs. Then, images having a computed coordinate position which is inconsistent with a hair on said body surface can also be filtered out.
The system for counting follicular units using an automated system may comprise any of the transplantation systems described in the background above. For instance, the system described in U.S. patent application Ser. No. 11/380,907 may be programmed and configured to perform the methods of counting follicular units according to the present invention. The cameras on the system can provide stereo digital images and the robotic arm can properly position and orient the cameras. The selection of a region of interest may be performed by an operator at the user interface of the system (such as a computer having a monitor and input devices) or it could be automated through programming of the computer and/or controller.
Accordingly, a system and method for counting follicular units is provided. Other and further embodiments, objects and advantages of the invention will become apparent from the following detailed description when read in view of the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which:

FIG. 1 is a print of a digital image of an exemplary section of a human scalp having a plurality of follicular units.

FIG. 2 is a print of the digital image of FIG. 1 after it has been filtered using a band-pass filter.

FIG. 3 is a print of the digital image of FIG. 2 after the image has been segmented.

FIG. 4 is print of the digital image of FIG. 3 after a morphological open operation has been performed on the segmented image.

FIG. 5 is a print of the digital image of FIG. 4 after noise filtering has been performed on the image.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

Referring first to FIG. 1, the system and method for counting follicular units according to the present invention generally begins with acquiring a digital image 10 of a body surface 11 using one or more digital cameras. The body surface 11 has skin 12 and a plurality of follicular units 14 each having one or more hairs 13 (only a few of the follicular units 14 and hairs 13 are labeled in the figures). The photo of FIG. 1 is an image of a section of human scalp 11, but it is understood that the body surface 11 could be any area of any body having hair. The digital image 10 shows a variety of types of follicular units 14 (FU) on the scalp 11.
The digital image 10 may be acquired using one or more digital cameras of an automated hair transplantation system, such as the cameras described in the hair transplantation system of U.S. patent application Ser. No. 11/380,907, which is incorporated by reference herein in its entirety. The image from just one of the cameras can be used to produce the digital image 10. Alternatively, the process for obtaining the digital image 10 may be acquired by a more involved process which aligns the camera(s) to improve the image used to classify a follicular unit of interest. In this process, a first camera and a second camera are used. The cameras are arranged and configured to obtain stereo images of a body surface at which they cameras are directed. The cameras are first positioned to be directed at the body surface in an area known to have hair. A first digital image is acquired from the first camera and a follicular unit (FU) of interest is selected from within the first digital image. A second digital image of about the same region of the body surface as the first camera (except from a slightly different angle as provided by stereo cameras) is acquired from the second camera and the same FU of interest is selected from within the second digital image. The FU of interest can be selected in the digital images by an operator of the system or automatically by the system using a selection algorithm. The transplantation system is now able to track the FU of interest within the first and second digital images from the first and second cameras. The tracking procedure can be used to adjust for movement of the body surface and movement of the cameras when they are aligned to acquire the digital image(s) used for classifying the FU. Next, the first and second cameras are moved and oriented to be aligned with the general orientation of the hair of the FU. As the cameras are moved, additional digital images may be acquired and processed by the system in order to track the FU of interest. By aligning the cameras with the hair of the FU, a better digital image for classifying the FU can be acquired. With the cameras in the desired alignment, the cameras acquire the digital images to be used in the next steps of the method of classifying a follicular unit.
After the digital image 10 is acquired, a region of interest 19 which could be the entire digital image 10 or a sub-area is selected. In the example described herein, the selected region of interest 19 is co-extensive with the digital image 10. However, the selected region of interest 19 can be any subset area of the digital image 10. The region of interest 19 may be selected by an operator or the selection may be automated by the system. This region of interest within the digital image is called the selected image.
Then, the selected image 19 is digitally filtered using a band-pass filter to remove components in the image which correspond to the skin 12. FIG. 2 shows a print of the digital image after the original selected image has been filtered using a band-pass filter. The band-pass filter may comprise any suitable filter as known by those of ordinary skill in the art. For example, the selected image may be filtered using a band-pass filter. The band-pass filtering can be accomplished by low-pass filtering the selected image twice and then subtracting the two resulting filtered images. The band-pass filter may comprise a first filtering step using a low-pass filter having a first kernel and a second filtering step using a low-pass filter having a second kernel. The first kernel is preferably different from the second kernel. In one embodiment of the present invention, the kernels of the low-pass filter(s) may be Gaussian kernels. The first Gaussian kernel may have substantially the following characteristics: support 21 pixels, sigma of 1.0. The second Gaussian kernel may have substantially the following characteristics: support 21 pixels, sigma of 0.075.
Next, the resulting image after the band-pass filtering is segmented using well-known digital image processing techniques to produce a binary image of the selected image. FIG. 3 is a print of the binary image after the image has been segmented.
Then, a morphological open operation is performed on the binary image to remove artifacts from the image. FIG. 4 shows the resulting image after the morphological open operation. As stated above, a morphological open operation is a known, standard image processing technique. As can be seen in FIG. 4, the image may still contain many objects which do not correspond to a the hair 13 of follicular unit 14. There are many objects which appear to be too long, too large, randomly oriented and/or in a location which probably does not contain hair.
Accordingly, noise filtering is then performed on the image resulting from the morphological open operation. The noise filtering removes objects which do not meet criteria corresponding to a follicular unit 14. For example, the area, location or orientation of the image of the follicular unit may be whose area, location or orientation do not correspond to hair. Referring back to FIG. 4, the object 22 appears to be much longer and have a much larger area than the other objects in the image 19. Thus, it can be assumed that this object is probably not a hair 13 and therefore should be filtered out of the image. Turning now to the print of the image after the noise filtering step of FIG. 5, it can be seen that the object 22 has been filtered out of the image. The noise filtering step can filter based on a wide range of characteristics of the objects in the image, including without limitation, length, area, orientation and/or location. Whether the characteristics of an image of an object corresponds to hair may be determined by statistical comparison to the global nature of the same characteristics for images of objects in the selected image which are known to be hair, or alternatively, the characteristics can be compared to predetermined criteria based on patient sampling or other data. For instance, the noise filtering filter can be based on characteristics of a sampling of the other hairs on the body surface of the particular patient, or the characteristics of a sampling of hairs on a sample of patients, or on known predetermined data based on studies or research.
Any or all of the systems and methods for classifying a follicular unit as described herein may be used in conjunction with the system and method of harvesting and transplanting hair as described in U.S. patent application Ser. No. 11/380,903 and U.S. patent application Ser. No. 11/380,907.
The foregoing illustrated and described embodiments of the invention are susceptible to various modifications and alternative forms, and it should be understood that the invention generally, as well as the specific embodiments described herein, are not limited to the particular forms or methods disclosed, but to the contrary cover all modifications, equivalents and alternatives falling within the scope of the appended claims. By way of non-limiting example, it will be appreciated by those skilled in the art that the invention is not limited to the use of a robotic system including a robotic arm, and that other automated and semi-automated systems may be utilized. Moreover, the system and method of counting follicular units of the present invention can be a separate system used along with a separate automated transplantation system or even with a manual transplantation procedure.

Claims

1. A method of identifying follicular units on a body surface having skin and hair, comprising:

acquiring a digital image of a body surface;

selecting a region of interest within said digital image called a selected image;

digital filtering said selected image, via a band-pass filter, to remove components corresponding to the skin;

segmenting said selected image to produce a binary image;

performing morphological open operation on said binary image; and

performing noise filtering by removing objects having certain characteristics which do not correspond to such characteristics of hair.

2. The method of claim 1, wherein said step of filtering said selected image via a band-pass filter comprises low-pass filtering the selected image twice and then subtracting the two resulting filtered images.

3. The method of claim 1, wherein said step of filtering said selected image via a band-pass filter comprises a first filter step in which said selected image is filtered using a low-pass filter having a first kernel and a second filter step in which said selected image is filtered using a low-pass filter having a second kernel.

4. The method of claim 3, wherein said first kernel is a Gaussian kernel having substantially the following characteristics, support 21 pixels, sigma of 1.0.

5. The method of claim 3, wherein said second kernel is a Gaussian kernel having substantially the following characteristics, support 21 pixels, sigma of 0.75.

6. The method of claim 3 wherein said first kernel is a Gaussian kernel having substantially the following characteristics, support 21 pixels, sigma of 1.0 and said second kernel is a Gaussian kernel having substantially the following characteristics, support 21 pixels, sigma of 0.75.

7. The method of claim 1, further comprising the following steps:

acquiring a second digital image in stereo correspondence to said first digital image;

computing the coordinate position of a hair using said first and second digital images; and

filtering out images having a computed coordinate position which is inconsistent with a hair on said body surface.

8. The method of claim 1, further comprising counting the discrete images of hairs using said processed image.

9. The method of claim 1, wherein said step of noise filtering comprises filtering out any object having an area that differs by more than two standard deviations from the mean object size within the selected image.

10. The method of claim 1, wherein said step of noise filtering comprises filtering out any image with an area that is larger than two standard deviations larger than the mean object size.

11. The method of claim 1, wherein said characteristics include one or more of area, location and orientation.

12. The method of claim 1, wherein said step of noise filtering comprises filtering out objects whose characteristics do not correspond to such characteristics for the other objects in the region of interest.

13. The method of claim 1, wherein said step of noise filtering comprises filtering out objects whose characteristics do not correspond to such characteristics for hair based on a sampling of hairs on the body surface.

14. The method of claim 1, wherein said step of noise filtering comprises filtering out objects whose characteristics do not correspond to such characteristics expected for hairs based on predetermined data.

15. The method of claim 3 wherein said first kernel is different from said second kernel.