WO1986002741A1

WO1986002741A1 - Three dimensional images

Info

Publication number: WO1986002741A1
Application number: PCT/GB1985/000484
Authority: WO
Inventors: Christopher John Gill; Elizabeth Jane Gill
Original assignee: National Research Development Corporation
Priority date: 1984-10-23
Filing date: 1985-10-21
Publication date: 1986-05-09
Also published as: JPS62501463A; GB8426775D0; EP0229067A1; BR8507262A; AU4963385A

Abstract

Various arrangements for recording and displaying three dimensional images. In one arrangement, five video cameras (11 to 15) are spaced apart laterally with regard to a camera viewing direction. Four of the cameras are positioned at the four corners of a square and the fifth camera is positioned at the middle of the square. The cameras (11 to 15) are focused onto a single point and the resulting five video signals are mixed in a predetermined sequence in a mixing circuit (36) to obtain a single video signal. When the single video signal is displayed on a monitor (40), a three dimensional effect is obtained. In another arrangement, a board is provided with five apertures, four of which are positioned at the four corners of a square and the fifth is located at the middle of the square. Pictures are then taken with a conventional cine camera which is located at each aperture in turn in a predetermined sequence, a single frame being exposed at each aperture. When the resulting cine film is shown, a three dimensional effect is obtained. In a further arrangement, still transparencies are taken from five viewpoints, four of which are located at the four corners of the square and the fifth is located at the middle of the square. The five transparencies are then projected onto a screen using five projection and a shutter arrangement which causes the individual pictures to be projected in a predetermined sequence. A three dimensional effect is obtained.

Description

"THREE DIMENSIONAL IMAGES"

Technical Field

This invention relates to creation and display of three dimensional images.

Three dimensional images are at present produced by photographing a scene from two viewpoints which are normally separated from each other by the interpupillary distance of 65mm. Where only a still picture is required, a single pair of pictures is taken and when moving pictures are required a series of pairs of pictures are taken. The resulting pair or pairs of pictures are then viewed by an observer using either one of several types of special viewing apparatus or a special viewing screen so that one picture of the or each pair is viewed by one eye and the other picture is viewed by the other eye. These methods of producing three dimensional images suffer from the need to provide either a special viewing apparatus or a special screen.

in US Patent 1,714,849, US Patent 3,457,364, Canadian Patent 1,107,100, and German Patent Application 2,546,387, arrangements are proposed in which images taken from a pair of spaced apart viewpoints are projected alternately onto a screen. US Patent 1,714,849 also discusses projecting images taken from three viewpoints, two of the viewpoints corresponding to left and right eye positions and he third viewpoint being located midway between the first and second viewpoint. With these arrangements, it is alleged that a three dimensional effect may obtained by viewing the screen directly and without using any special view apparatus. However, these arrangements suffer from the dis¬ advantage that the three dimensional effect is not stable.

Disclosure of invention

It is an object of this invention to provide both a method and an apparatus for producing three dimensional images in which the above problems are overcome.

According to one aspect of this invention, there is provided an apparatus for creating data representative of a set of images using information representative of pictures taken from at least three viewpoints spaced laterally apart from each other with regard to a common viewing direction, each image being composed from information from at least one viewpoint and the composition of each image or group of images with respect to the viewpoints differing from the composition of an adjacent image or group of images, characterized in that at least three of said viewpoints are spaced around a central position.

If the data is used to project the images onto a screen a three dimensional effect is obtained which is sub¬ stantially more stable than the three dimensional effect obtained in the prior art arrangement discussed above.

Preferably, the data is representative of a sequence of images.

Although the present invention may be performed with only three viewpoints, it has been found that an improvement is obtained where there are four viewpoints and that a further improvement is obtained where there are five viewpoints. It is envisaged that satisfactory results will be obtained with more than five viewpoints although the complexity of the apparatus will increase as the number of viewpoints increases. Accordingly, it is preferred that the information is representative of pictures taken from five viewpoints.

It has been found that, where one viewpoint occupies the central position and the remaining viewpoints are spaced around this central position, when the images are displayed a further improvement in stability is obtain. Where no viewpoint occupies the central position, some apparent movement is perceived between parts of the images. Although such relative movement may be useful for certain special effects, it is _* preferred for most purposes that one viewpoint occupies the central position.

In one arrangement, the data creating means comprises a set of television cameras spaced apart laterally with reference- o a common viewing direction, camera control means for operating said cameras, and a mixing circuit for mixing the individual video signals from said cameras to provide a single video signal representative of said set of images .

The camera control means may provide an individual control signal for each camera. Alternatively, the camera control means may provide a common control signal for all of said cameras.

In the single video signal produced by said mixing circuit, each field may be formed from the video signal from a respective one of said cameras. Alternatively, some or all of the fields of said single video signal may be formed by mixing the video signals from two or more of said cameras . In another arrangement, the data creating means comprises a single camera and an optical arrangement for receiving light at a set of viewpoints and projecting the light from each viewpoint in turn onto said camera.

The optical arrangement may include rotatable reflecting elements .

In a further arrangement, the data creating means comprises a computer program to produce said data from control information.

According to another aspect of this invention there is provided a method for creating data for producing three dimensional images, said method comprising creating data representative of a set of images using information representative of pictures taken from at least three viewpoints spaced laterally apart from each other with regard to a common viewing direction, each image being composed from information form at least one viewpoint and the composition of each image or group of images with respect to the viewpoints differing from the composition of an adjacent image or group of images, characterized in that at least three of said viewpoints are spaced around a central position

Preferably, the data is repesentative of a sequence of images.

Preferably, the information is representative of pictures taken from five viewpoints.

The data may be created by arranging a camera so that it may be positioned at each one of a set of viewpoints,moving the camera from viewpoint to viewpoint in accordance with a predetermined sequence, and taking at least one picture at each viewpoint.

Where the camera is arranged to take moving pictures the sequence is repeated and, where the camera is arranged to take still pictures, the sequence need be performed once only.

According to a further aspect of this invention there is provided a recording medium on which is stored data created by the method set out above.

According to a still further aspect of this invention, there is provided a method of displaying three dimensional images comprising projecting a set images of a scene onto a screen, the images using information representative of pictures taken from at least there viewpoints spaced laterally apart from each other with regard to a common viewing direction, each image being composed from information from at least one viewpoint, and the composition of each image or group of images with regard to the viewpoints differing from the composition of an adjacent image or group of images, characterized in that at least three of said viewpoints are spaced around a central position.

The images may be projected with the aid of a cine projector, or a set of still projectors, or a television set.

According to a still further aspect of this invention there is provided an apparatus for projecting three dimensional images onto a screen, characterized in that said apparatus comprises three or more still projectors arranged to project images onto a common screen, and a shutter arrangement associated with the projectors for permitting each projector to project an image onto the screen with a predetermined timing. Brief Description of Drawings

This invention will now be described in more detail, by way of example, with reference to the drawings in which:- Figure 1 is a front view of a camera mount used in an apparatus embodying this invention;

Figure 2 is a side view of the camera mount of Figure 1;

Figure 3 is a block diagram of the electronic circuitry used with the camera mount of Figure 1; Figures 4a to 4f are a set of sequence charts for describing operation of the apparatus shown in Figures

1 to 3;

Figure 5 is an aperture board used in a method

_ embodying this invention; Figure 6 is a diagrammatic view of a projection apparatus embodying this invention; and Figure 7 is a view of an aperture disc used with the projection apparatus shown in Figure 6.

Modes for carrying out invention

As will be described in more detail below, in the apparatus shown in Figures 1 to 3, five video cameras are positioned at five separate viewpoints which are spaced apart laterally with regard to a common viewing direction. The five video cameras are then focussed onto a single point of a scene to be photographed and the resulting five video signals are mixed in a predetermined sequence to obtain a single video signal. The single video signal may be supplied to a television monitor directly or, alternatively, it may be stored using a video cassette recorder and then replayed at a later stage. When the signal is displayed using the television monitor, it has been found that the resulting images provide a three dimensional effect similar to that obtained with conventional stereoscopic techniques. However, there is no need to use a special screen or viewing apparatus.

Referring now to Figures 1 and 2, five television cameras 11 to 15 are shown positioned in a camera mount 16. The camera mount 16 has four channel shaped members 17 to 20 arranged in the shape of a cross with the open parts of these members facing outwardly. The camera 15 is mounted in a tube 21 positioned in the square-shaped opening defined by the four inner walls of the channel- shaped members 17 to 20. Each of the cameras 11 to 14is mounted in a respective one of square-shaped tubes 22 to 25 which are positioned respectively inside the members 17 to 20. The square tube 22 is provided with a mounting bracket 27 at its front end and five pairs of apertures 28 are formed in the front end of the channel shaped member 17. The front, end of the square tube 22 is pivotally mounted on the front end of channel shaped member 17 by a pivot rod 28a which is passed through a selected one of -the pairs of apertures 28 and apertures in the bracket 27. Thus, the displacement of camera 11 from camera 15 is set when selecting one of the pairs of apertures 28. The rear end of tube 22 is connected to a screw thread 29 which passes through a guide 30 mounted on the end of member 17. A wing nut 29a is secured to screw thread 29 and the inclination of camera 11 may be set by adjusting nut 30. Tubes 23 to 25 are mounted in a similar manner. Thus, the cameras 11 to 15 are mounted at five viewpoints, four of which will normally be at the corners of a square and the fifth is at the middle of the square. The separation between the viewpoints may be adjusted, as described before, and the inclination of cameras 11 to 14 may be adjusted with regard to the viewing direction of camera 15.

As shown in Figure 3, the cameras 11 to 15 are each connected to a respective one of camera control units 31 to 35. The camera control units 31 to 35 receive synchronising signals from a mixing circuit 36 and each of these units 31 to 35 supplies the video signal from its associated camera 11 to 15 to circuit 36. In the mixing circuit 36, the five video signals are mixed to provide data in the form of a single video signal which is supplied on a line 37 to the input terminal of a video cassette recorder 38. This single video signal is supplied either directly or during replay through a line 39 to a television monitor 40. If it is not desired to store the video signal, the video cassette recorder 38 may be removed and the video signal supplied directly fr •om circuit 36 to monitor 40. •__

The video signals are mixed in circuit 36 according to a predetermined sequence and this sequence may be set by a programmer 41. Six possible sequences are shown in Figures 4a to 4f and, in these figures, each rectangle represents a single video field. Each video frame is of course formed from two video fields.

In the sequence shown in Figure 4a, the video signal commences with a first field of a first frame from camera 11, a second field of the first frame from camera 12, a first field of a second frame from camera 13, a second field of a second frame from camera 14, followed by a first field of a third frame from camera 15. The sequence then continues with a second field of a third frame from camera 11, a first field from a fourth frame from camera 12, a second field from a fourth frame from camera 13, a first field from a fifth frame from camera 14, and a second field from a fifth frame from camera 15. The sequence is then repeated in a similar manner.

In the sequence shown in Figure 4b, the single video signal commences with a single field from each of cameras 11 to 15 in turn followed by a single field from camera 13. The sequence is then repeated.

The sequence shown in Figure 4c commences with a single field from each of cameras 11, 13, 12, 13, 15, 13, 14, and 13 and the sequence then repeats.

The sequence shown in Figure 4d commences with a single field from camera 11 followed by four fields from camera 13, a field from camera 12, four fields from camera 13, a field from camera 15, four fields from camera 13, a field from camera 14, and four fields from camera 13. The sequence is then repeated.

It has been found that each of the sequences shown in Figures 4b, 4c and 4d provides an improvement in stability over the previously described sequence. It will also be observed that each of the sequences shown in Figures 4b, 4c and 4d includes a higher content of the video signal from the central camera 13 than the previous sequence and this is the reason for the improvement.

In each of the sequences described in Figures 4a to 4d, each video field of the resulting single video signal is formed from a video field of one of the cameras 11 to 15. As will now be described, in the sequences shown in Figures 4e and 4f, some of the video fields of the single video signal are formed from a combination of video fields from two cameras.

In the sequence shown in Figure 4e, the first field of the single video signal is formed from the upper half of a field from camera 11 and the lower half of a field from camera 13, the second field comprises a whole field from camera 13, and the third field comprises a lower half of a field from camera 11 and an upper half of a field from camera 13. This is followed by two whole fields from camera 13, the upper half of a field from camera 12 and the lower half of a field from camera 13, a field from camera 13, the lower half of a field from camera 12 and the upper half of a field from camera 13, and then two fields from camera 13. As shown, the sequence then continues in a generally similar manner.

In the sequence shown in Figure 4f, the first field of the single video signal is formed from the upper third of a field from camera 11 together with the lower two thirds of a field from camera 13, and this is follo _wed by a field which comprises the middle third of a field from camera.12 together with the upper third and lower third of a field from camera 13, and this is followed by a single field from camera 13. The sequence then continues in a_,generally similar manner as that shown.

Although not illustrated, the video signals from cameras 11 to 15 may be mixed so that in each field of the single video signal adjacent pixels or picture points are provided from the video signals of different cameras. For example, in the first field of the single video signal, the first five pixels may be provided respectively from cameras 11 to 15 and in the second field the first five pixels may be provided respectively from cameras 14, 15, 13, 11, 12.

In the various sequences described above, each field of the single video signal is composed from the video signals of one or more of the individual cameras 11 to

15. Moreover, at the end of each field or each group of^' fields, the composition is changed.

In the apparatus shown in Figure 3, there is a risk that the camera control units 31 to 35 will not be exactly matched thereby causing distortions in the display images. In order to eliminate this the units 31 to 35 may be replaced by a single camera control unit to drive all cameras 11 to 15.

In the apparatus described with reference to Figures 1 to 3, the upper and lower edges of the displayed images from the outer cameras will normally converge. If the convergence is sufficient to cause a problem, this may be eliminated by driving the cameras 11, 12, 14 and 15 in such a way that the convergence is eliminated.

Referring now to Figure 5, there is shown an aperture board 50 which is used for recording three dimensional images in accordance with this invention. The aperture board 50 has formed therein five apertures A, B, C, D and Ξ. The apertures A, B, D and E are positioned at the four corners of a square and the aperture C is positioned at the centre of the square. In one method, a cine camera is positioned at each aperture in turn in a predetermined sequence and one or more pictures are taken at each aperture. For each picture, the camera is aimed and focussed at a central point of the object to be photographed, in order to view the pictures, the film is projected using a conventional projector and the screen is viewed by an observer using either one or both eyes simultaneously and a three dimensional effect is obtained without the need for using any special screen or viewing apparatus. Alternatively, the pictures may be taken using a video camera.

The aperture board 50 may also be used with a still camera using transparency film and, in this case, a picture is taken at each aperture. The resulting five transparencies are displayed using the projector arrangement shown in Figure 6. The projection apparatus shown in Figure 6 comprises a central projector 60, two upper side projectors 61 and 62, and two lower side projectors, not shown, positioned respectively beneath projector 61 and 62.

Projector 60 has a transparency holder 63 and an objective lens 64. As shown for projectors 61 and 62, each side projector is provided with a transparency holder 65, a mirror 66 for reflecting the beam from the projector through 90° and an objective lens 67. An aperture disc 68 is positioned immediately in front of objective lenses 64, 67 and mounted for rotation about an axis 69. Although only half of the disc 68 is shown in Figure 6, this disc is shown in greater detail by way of a front view in Figure 7. Arcuate apertures are formed in disc 68 which permit projection of images from the individual projectors onto a screen 70. Specifically, the disc has four apertures 71 which permit projection of images from the projector 60, apertures 72 and 73 permit projection of images from projectors 61 and 62, and apertures 74 and 75 which permit projection of images from the two lower projectors .

The apertures are arranged so that only one projector is permitted to project its image at any one time. Each of the apertures 72 to 75 subtends an equal angle with respect to the centre of the disc 68 so that images from the four side projectors are projected for equal periods of time. The angle subtended by the apertures 71 is three times greater than the angle subtend by the apertures 72 to 75 with the result that the image from projector 60 will be displayed for periods which are three times greater than the periods for the images from the side projectors.

In operation, the disc 68 is rotated and the resulting images displayed on screen 70 provide a three dimensional effect without the need for special viewing apparatus, it has been that particularly good results occur when the disc 68 is rotated at a speed such that each aperture 71 corresponds to a projection time of

1/30 second and each aperture 72 to 74 corresponds to a projection time of 1/90 second.

in a modification of the apparatus shown in Figure 6 the disc 68 can be replaced with an individual shutter for each projector, the shutters being operated by a single control device which could be a computer.

Another apparatus for recording three dimensional pictures comprises an optical arrangement which receives light at a set of viewpoints and projects the light from each -viewpoint in a predetermined sequence onto a single camera.

In an experiment with this apparatus, it has been found that very satisfactory results are obtained when the images from the individual viewpoints are projected at a frequency of 70 images per second. in the arrangement described with reference to Figs. 1 to 4, the standard field frequency of 50 fields/second has been used. However, in view of the very satisfactory results obtained when projecting 70 images per second, it is considered that an improvement could be obtainedby increasing the field frequency to 70 fields second.

Also the apparatus shown in Figure 6 could be used for recording pictures by replacing the five projectors with five video cameras and combining the five video signals into a single video signal. In the various arrangements described above, the individual images are recorded sequentially. However, the present invention is not limited to recording images in this way. For example, a cine camera could be constructed which records pictures from five viewpoints at successive instants of time, at each instant pictures from all five viewpoints being recorded simultaneously. The five pictures for each instant could be recorded alongside each other so as to form a row of pictures on the cine film with the rows for successive instants being spaced along the length of the film. It would then be necessary to construct a suitable projection for projecting the individual pictures in a desired pattern.

In the various examples described above for recording pictures, no dimensions have been given for the spacing between the viewpoints. The optimum spacing will depend upon various factors including the distance between the viewpoints and the subject. In general, the optimum spacing will increase with subject distance and the optimum spacing may be found by experiment. Wich the aperture board shown in Figure 5, a particularly satisfactory result for a subject distance of 3.7 metres has been obtained where the centres of the four outer apertures are located at the corners of a square which has a side of 44mm. It is believed that this spacing would also produce a similarly satisfactory result with the apparatus shown in Figures 1 to 4 but a modification would be required to achieve this spacing.

Although the present invention has been described with reference to forming images using light, the present invention may also be applied to the formation of images using different types of radiation and may, for example, be used to form images using X-rays or acoustic beams. The present invention may also be applied to the creation of images using a computer which is programmed to form images representative of pictures taken from different viewpoints. As may be readily appreciated, a computer can be programmed to create pictures without the necessity of using a camera.

Although in the various embodiments of the invention described above, an entire scene is photographed so as to have a three dimensional effect, the present invention may be applied to the case where only the foreground or rearground of the scene £as the three dimensional effect. In this case, the part of the scene which is to have the three dimensional effect is created using any of the methods described above whereas the remainder of the screen is created using conventional techniques.

Claims

1. An apparatus for creating data for producing three dimensional images, said apparatus comprising means (11 to 16, 31 to 36) for creating data representative of a set of images using information representative of pictures taken from at least three viewpoints spaced laterally apart from each other with regard to a common viewing direction, each image being composed from information from at least one viewpoint and the composition of each image or group of images with respect to the viewpoints differing from the composition of an adjacent image or group of images, characterized in that at least three of said viewpoints are spaced around a central position.

2. An apparatus as claimed in claim 1 characterized in that the data is representative of a sequence of images .

3. An apparatus as claimed in claim 1, characterized in that there are four viewpoints.

4. An apparatus as claimed in claim 1, characterized in that there are five viewpoints.

5. An apparatus as claimed in claim 1, characterized in that there are at least four viewpoints and one of said viewpoints is located at said central position.

5. An apparatus as claimed in claim 1, characterized in that the data creating means comprises a set of television cameras (11 to 15) spaced apart laterally with reference to a common viewing direction, camera control means (31 to 35) for operating said cameras (11 to 15), and a mixing circuit (36) for mixing the individual video signals from said camera (11 to 15) to provide a single video signal representative of said set of images.

7. An apparatus as claimed in claim 6, characterized in that the camera control means (31 to 35) provides an individual control signal for each camera.

8. An apparatus as claimed in claim 6, characterized in that the camera control means provides a common control signal for all of said cameras.

9. An apparatus as claimed in claim 6, characterized in that, in the single video signal produced by said mixing circuit (36), each field is formed from the video signal from a respective one of said cameras (11 to 15).

10. An apparatus as claimed in claim 6, characterized in that, in the single video signal produced by said mixing circuit (36), at least some of the fields are formed by mixing the video signals form two or more of said cameras (11 to 15).

11. An apparatus as claimed in claim 1, characterized in that the data creating means comprises a single camera and an optical arrangement for receiving light at a set of viewpoints and projecting the light from each viewpoint in turn onto said camera.

12. A apparatus as claimed in claim 11, characterized in that the apparatus includes rotatable reflecting elements .

13. An apparatus as claimed in claim 1, characterized in that the data creating mean comprises a computer program to produce data from control information.

14. A method for creating data for producing three dimensional images, said method comprising creating data representative of a set of images using information representative of pictures taken from at least three viewpoints spaced laterally apart from each other with regard to a common viewing direction, each image being composed from information form at least one viewpoint and the composition of each image or group of images with respect to the viewpoints differing from the composition of an adjacent images or group of images, characterized in that at least three of said viewpoints are spaced around a central position

15. A method as claimed in claim 14 characterized in that the data is representative of a sequence of images

16. A method as claimed in claim 14, characterized in that the information is representative of pictures taken from five viewpoints.

17. A method as claimed in claim 14, characterized in that there are at least four viewpoints and one of said viewpoints is located at the central position.

18. A method as claimed in claim 14, characterized in that the data is created by arranging a camera so that it may be positioned at each one of a set of viewpoints, moving the -camera from viewpoint to viewpoint in accordance with a predetermined sequence, and taking at least one picture at each viewpoint.

19. A recording medium on which is stored data created by the method according to claim 14. 20. A method of displaying three dimensional images comprising projecting a set images of a scene onto a screen, the images using information representative of pictures taken from at least there viewpoints spaced laterally apart from each other with regard to a common viewing direction, each image being composed from information from at least one viewpoint, and the composition of each image or group of images with regard to the viewpoints differing from the composition of an adjacent image or group of images, characterized in that at least three of said viewpoints are spaced around a central position.

21. An apparatus for projecting three dimensional images onto a screen characterised in that said apparatus comprises at least three still projectors (60, 61, 62) arranged to project imges onto a common screen (70), and a shutter arrangement (68) associated with the projectors (60, 61, 62) for permitting each projector to project an image onto the screen with a predetermined timing.