+

WO2008093167A2 - Procédés et systèmes pour une interaction avec un système de visualisation en 3d au moyen d'une interface 2d ('dextrolap') - Google Patents

Procédés et systèmes pour une interaction avec un système de visualisation en 3d au moyen d'une interface 2d ('dextrolap') Download PDF

Info

Publication number
WO2008093167A2
WO2008093167A2 PCT/IB2007/004506 IB2007004506W WO2008093167A2 WO 2008093167 A2 WO2008093167 A2 WO 2008093167A2 IB 2007004506 W IB2007004506 W IB 2007004506W WO 2008093167 A2 WO2008093167 A2 WO 2008093167A2
Authority
WO
WIPO (PCT)
Prior art keywords
virtual
tool
cursor
virtual world
mouse
Prior art date
Application number
PCT/IB2007/004506
Other languages
English (en)
Other versions
WO2008093167A3 (fr
Inventor
Ng Hern
Luis Serra
Original Assignee
Bracco Imaging S.P.A.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bracco Imaging S.P.A. filed Critical Bracco Imaging S.P.A.
Publication of WO2008093167A2 publication Critical patent/WO2008093167A2/fr
Publication of WO2008093167A3 publication Critical patent/WO2008093167A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/25—User interfaces for surgical systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays

Definitions

  • the present invention relates to interactive visualization of three-dimensional data sets, and more particularly to enabling the functionality of 3D interactive visualization systems on a 2D data processing system, such as a standard PC or laptop.
  • 3D visualization systems allow a user to view and interact with one or more 3D datasets.
  • An example of such a 3D visualization system is the DextroscopeTM running associated RadioDexterTM software, both provided by Volume Interactions Pte Ltd of Singapore. It is noted that 3D visualization systems often render 3D data sets stereoscopically. Thus, they render two images for each frame, one for each eye. This provides a user with stereoscopic depth cues and thus enhances the viewing experience.
  • a 3D dataset generally contains virtual objects, such as, for example, three dimensional volumetric objects.
  • a user of a 3D visualization system is primarily concerned with examining and manipulating volumetric objects using virtual tools.
  • virtual tools can include, for example, a virtual drill to remove a portion of an object, picking tools to select an object from a set of objects, manipulation tools to rotate, translate or zoom a 3D object, cropping tools to specify portions of an object, and measurement tools such as a ruler tool to measure distances, either absolute linear Cartesian distances or distances along a surface or section of one of the virtual objects.
  • a 3D visualization system presents a 3D environment
  • 3D interfaces such as, for example, two 6D controllers, where, for example, one can be held in a user's left hand for translating and rotating virtual objects in 3D space, and the other can be held, for example, in a user's right hand to operate upon the virtual objects using various virtual tools.
  • controllers for example, a user can move in three dimensions throughout the 3D environment.
  • a virtual control panel can be placed in the virtual world.
  • a user can select, via the virtual control panel, various objects and various tools to perform various manipulation, visualization and editing operations.
  • a virtual control panel can be used to select various display modes for an object, such as, for example, full volume or tri-planar display.
  • a virtual control panel can also be used, for example, to select a target object for segmentation, or to select two volumetric objects to be co-registered.
  • a virtual control panel can be invoked by touching a surface with, for example, a right hand controller as described above.
  • the "natural" interface to a 3D visualization system is the set of 3D control devices described above, sometimes it is desired to implement, to the extent possible, a 3D visualization system on a desktop or laptop PC, or other conventional data processing system having only 2D interface devices, such as a mouse and keyboard.
  • control of virtual tools, and the positioning, rotation and manipulation of virtual objects in 3D space needs to be mapped to the available mouse and keyboard. This can be difficult, inasmuch as while in 3D a user can move in three dimensions throughout the model space, when using a 2D interface, such as a mouse, for example, only motion in two dimensions can be performed. A mouse only moves in a plane, and provides no convenient manner to specify a z-value.
  • a mouse has no z control, while a 3D cursor or icon being controlled by the movement of a 2D mouse needs to have a z value associated with it to properly function in a 3D model space, the 2D mouse simply has no means to provide any such z value. If the z value of the cursor is ignored, the cursor or icon will nearly always seem to appear at a different depth than the object it is pointing to.
  • two view ports are generally used; one view port to display what the left eye sees and the other view port to display what the right eye sees. If the z value of a cursor or icon is simply ignored, the cursor or icon will always be displayed at some fixed depth set by the system.
  • a cursor or icon could be always displayed at the convergence plane (i.e., a plane in front of the viewpoint and perpendicular to it where there is no disparity between the left and right eyes). This introduces three problems.
  • the mouse cursor will appear as floating in front of the object, its motion constrained to a plane, and thus never reaching the objects which a user intends to manipulate.
  • This option requires that a user manipulate the virtual object from a distance in front of it, making interactions awkward. This situation is illustrated in Fig. 1A.
  • a 3D visualization system can be ported to a laptop or desktop PC, or other standard 2D computing environment, which uses a mouse and keyboard as user interfaces.
  • a cursor or icon can be drawn at a contextually appropriate depth, thus preserving 3D interactivity and visualization while only having available 2D control.
  • a spatially correct depth can be automatically found for, and assigned to, the various cursors and icons associated with various 3D tools, control panels and other manipulations.
  • this can preserve the three- dimensional experience of interacting with a 3D data set even though the 2D interface used to select objects and manipulate them cannot directly provide a third dimensional co-ordinate.
  • a variety of 3D virtual tools and functionalities can be implemented and controlled via a standard 2D computer interface.
  • Figs. 1A-B depicts problems in displaying a cursor at an arbitrary fixed plane within a 3D data set
  • Figs. 2A-B depicts automatically setting the depth of a cursor using ray casting according to an exemplary embodiment of the present invention
  • Figs. 3A-B depicts drawing a cursor stereoscopically according to an exemplary embodiment of the present invention
  • Fig. 4 depicts an exemplary virtual control panel and virtual buttons used to manipulate 3D objects according to an exemplary embodiment of the present invention
  • Figs. 5A-B depict an exemplary mapping of translation in 3D to a two-button mouse according to an exemplary embodiment of the present invention
  • Figs. 6A-B depict an exemplary mapping of rotation in 3D to a two-button mouse according to an exemplary embodiment of the present invention
  • Figs. 7A-B depict an exemplary volume tool operating upon a fully rendered virtual object according to an exemplary embodiment of the present invention
  • Figs. 7C-D depict an exemplary volume tool operating on another exemplary virtual object according to an exemplary embodiment of the present invention
  • Figs. 8A-B depict an exemplary volume tool operating upon a tri-planar display of the exemplary virtual object of Figs. 7A-B according to an exemplary embodiment of the present invention
  • Figs. 8C-D depict an exemplary volume tool operating upon a tri-planar display of the exemplary virtual object of Figs. 7C-D according to an exemplary embodiment of the present invention
  • FIGS. 9A-B depict use of an exemplary drill tool according to an exemplary embodiment of the present invention.
  • Figs. 9C-G depict use of an exemplary drill tool on the exemplary virtual object of Figs. 7C-D according to an exemplary embodiment of the present invention
  • Fig. 10 depicts use of an exemplary ruler tool according to an exemplary embodiment of the present invention
  • Figs. 11A-C depict interactions with the exemplary virtual object of Figs. 7C-D that has had two points placed upon it according to an exemplary embodiment of the present invention.
  • Figs. 12A-B depict another interaction with the exemplary virtual object of Figs. 7C-D, where two different points have been placed upon it according to an exemplary embodiment of the present invention.
  • a 3D visualization system can be implemented on a standard laptop or desktop PC, or other standard 2D computing environment, which uses a mouse and keyboard as interfaces.
  • a mapping allows 3D visualization system functionality to be made available on a conventional PC.
  • Such a mapping can, for example, include depth appropriate positioning of cursors and icons as well as assigning various 3D interactive control functions to a mouse and keyboard.
  • a cursor controlled by a mouse, trackball, or other 2D device can be automatically drawn so as to have a contextually appropriate depth in a virtual 3D world, such as would be presented, for example, by a fully functional 3D visualization system.
  • Figs. 2A and 2B depict an exemplary method for setting cursor depth according to exemplary embodiments of the present invention.
  • a mouse's position can be acquired. Given the 2D realm in which a mouse moves, a mouse's position can only specify two co-ordinates, taken to be (x,y). Thus, a mouse's movement on a mouse pad can be mapped to movement anywhere in a plane. It is noted that mapping the mouse's motion to (x,y) is analogous to the mouse's physical movement. While this convention is not strictly necessary, it could be confusing if the 2D mouse motion was mapped to, say (x,z) or (y,z)).
  • the mouse's position in the mouse co-ordinate system
  • the mouse's position can be transformed into a position in the virtual world's eye co-ordinate system.
  • movement in a 3D model space requires the specification of three co-ordinates to locate each point.
  • a ray can be cast from the viewpoint (middle position between the eyes, shown at the bottom of Fig.
  • the first hit point (P' eye ) of such a ray with any virtual object can then, for example, be set as the new 3D cursor position. This is illustrated, for example, in Fig. 2A.
  • the previous z value of the cursor can be used. For example, as shown in Fig. 2B, where in frame N+1 - because no object was found upon casting a ray through P eye - the previous value (P' ey e) that was found in frame N can be used.
  • the cursor can be drawn as a texture image in the virtual world at the position P' ey e. It can, for example, be drawn such that its shape is not occluded by other object.
  • the middle position between the eyes can be taken as the viewpoint and can be calculated from these two positions.
  • the cursor can, for example, be an image or bitmap.
  • such a cursor image can be put in a 3D world by creating a 2D polygon (usually four sided, for example) and then use the image as a texture to map onto the polygon.
  • the polygon can then, for example, be positioned in the 3D world.
  • the polygon can be drawn in a rendering pass, with no depth test, so that it appears unoccluded by any other polygons.
  • a cursor In a system using a stereoscopic display, the cursor can be perceived with depth-cues and thus "falls” or “snaps" on the surface of virtual objects, as is shown in Fig. 3. For objects with a convex surface, such a cursor position does not introduce any occlusion problem and thus makes stereoscopic convergence easy. Furthermore, in exemplary embodiments of the present invention a cursor can be, for example, displayed transparently to create a see-through effect, so that even if the surface is concave and partially occludes the cursor, stereoscopic convergence can be preserved.
  • the size of the cursor can change as a function of the position at which it is displayed. This can give a sense of depth even when the display is not stereoscopic. However, in such a cursor size changing approach, if a cursor is positioned too near the eyes, it can appear as very large. Therefore, in exemplary embodiments of the present invention, some restriction on the maximum size of the cursor can be imposed, as provided below.
  • the following pseudocode can be used to implement the computation of the depth ("z" position) and control the size of an exemplary cursor.
  • the pesudocode assumes an exemplary stereoscopic system; for monoscopic systems the convergence plane can be, for example, any plane in front of the viewer at a convenient depth.
  • SIZEcursor MAXsize; scale cursor by a factor of SIZEcursor;
  • a graphics engine prior to drawing to the monitor a graphics engine generally has to transform all points from world coordinates to eye coordinates.
  • the eye coordinate system presents the virtual world from the point of view of a user's eye. These points in the eye co-ordinate system can then be projected onto the screen to produce a scene using a perspective projection.
  • a user can thus position a cursor or other icon anywhere within a 3D data set using only a mouse.
  • various virtual tools can be selected or engaged, and controlled using such a mouse in conjunction with a keyboard, thus allowing a 3D visualization system to be controlled from a 2D interface.
  • Virtual tools in 3D visualization systems can be classified as being one of three types: manipulation tools, placement tools and pointing tools. This is true, for example, in the DextroscopeTM.
  • Manipulation tools are those that interact directly on virtual objects. In the DextroscopeTM system, for example, these tools are mapped to the position and control of a controller that can be held in a user's right hand. To illustrate such mappings according to exemplary embodiments of the present invention, in what follows, the following exemplary manipulation tools shall be discussed: Volume tool, Drill and Restorer tool, Ruler tool and Picking tool.
  • Placement tools are those that can be used to position and/or rotate one or more virtual objects in 3D space.
  • a Placement tool can be mapped to the position, orientation and status of the control switch of a 6D controller held in a user's right hand.
  • a Pointing tool can interact with a virtual control panel which is inserted into a virtual world.
  • a virtual control panel allows a user to select form a palette of operations, color look up tables and various visualization and processing operations and functionalities.
  • DextroscopeTM for example, whenever a user reaches into a virtual control panel with a pointing tool (a 3D input), the virtual control panel appears and a displayed virtual image of the pointing tool is replaced by an image of whatever tool was selected from the control panel.
  • the selection and control of virtual tools can be mapped to a 2D interface device such as, for example, a mouse.
  • a 2D interface device such as, for example, a mouse.
  • a user In a 3D visualization system a user generally points, using a Pointing tool, to a 3D position of a virtual control panel to indicate that it should be activated.
  • a pointing tool can be a virtual image of a stylus type tool commonly held in a user's right hand, and used to point to various virtual buttons on a virtual control panel.
  • a control panel button Once a control panel button is chosen for a virtual tool of some kind, the control and position and orientation of the chosen tool can be mapped to the same right hand stylus, and thus the virtual image can be changed to reflect the functionality of the chosen tool.
  • a stylus can be used to select a drill tool from a virtual control panel.
  • a virtual image of a generic stylus is displayed in the 3D virtual world whose position, orientation track that of the physical tool being held in the user's right hand.
  • the image of the stylus can change to appear as a virtual drill, whose position and orientation still track the physical tool held in the user's right hand, but whose drilling functionality can also be controlled by physical buttons or other input/interface devices on the physical tool or stylus.
  • the virtual image of the user's right hand tool can, for example, return to the generic pointer tool.
  • the virtual control panel can be activated, for example, by a physical tool being brought into a certain defined 3D volume. This requires tracking of the physical tool or input device held in a user's hand.
  • a tracking device to signal to a 3D visualization system that a user wishes to see (and interact with) a virtual control panel
  • a button on the right side of the screen can be provided to invoke a virtual control panel whenever it is clicked, such as is shown, for example, in Fig.
  • control panel By the "control panel” button provided at the bottom of the four buttons shown on the right of the figure.
  • a user can depress a defined key on the keyboard, such as, for example, the space bar, to invoke a control panel.
  • a virtual control panel has a 3D position, whenever a 2D controlled 3D cursor reaches into that position (by being projecting onto it, as described above) then, for example, a control panel can be activated.
  • interaction with a virtual control panel via a mouse controlled cursor can be easily accomplished, inasmuch as a cursor can, using the techniques described above, be automatically positioned on the face of the selected button, as shown in Fig. 4, for example, where a cursor (functioning as a "pointer tool”) points to the ruler button of the control panel (said ruler button being generically labeled as "The Control Panel Tool” in Fig. 4).
  • a mouse only provides two degrees of interactive freedom.
  • a 3D virtual Placement tool can, for example, be decoupled into a Positioning tool and a Rotation tool to more easily map to a 2D interface device such as a mouse.
  • a Positioning tool by sliding a mouse horizontally or vertically and holding down a defined button, for example its left mouse button, a Positioning tool can move a virtual object horizontally or vertically.
  • a defined button for example its left mouse button
  • two outer regions on each of the left and right sides of the screen can be defined, such that if the Positioning tool slides vertically in either of these regions, the virtual object can be caused to move nearer towards, or further away, from the user.
  • a user can press a defined mouse button, for example the right one, to achieve the same effect.
  • a horizontal/vertical move is chosen, for example by pressing the left mouse button
  • an "xy” icon appears (cross made of two perpendicular double sided arrows labeled "xy”).
  • a nearer/farther (in a depth sense) translation is chosen, for example, by pressing a left mouse button or by moving the Positioning tool to one of the active regions as described above
  • a "z" icon can, for example, appear (double sided arrow labeled "z").
  • Figs. 5A and 5B the large cross in the center of the frame is not the mouse controlled cursor. It indicates the point on the object that can be used, for example, as a focus of zoom (FOZ).
  • FOZ focus of zoom
  • the object will move such that the FOZ will be at the center of the "zoom box.” This is an important feature, as it controls which part of the object to zoom into/out of view. This is described more fully in U.S. Patent Application No. 10/725,773, under common assignment herewith.
  • a FOZ icon can appear, as in Figs. 5, and a user can then use the Positioning tool to move the object relative to the icon to select a center of zoom on the object.
  • a Rotation tool by sliding a 2D device such as a mouse horizontally or vertically, and holding down, for example, one of its buttons, for example the left button, a Rotation tool can rotate a virtual object either horizontally (about the x axis) or vertically (about the y axis).
  • two outer regions can be defined, for example, on the left and the right sides of the screen such that if the Rotation tool slides vertically, i.e., the mouse is rolled vertically down the screen, in either of these regions, a roll rotation (i.e., rotation about the z axis) can be performed on the virtual object.
  • a roll rotation i.e., rotation about the z axis
  • a user can press the right mouse button to achieve the same effect.
  • FIG. ⁇ A and B Exemplary Rotation tool functionality is shown in Figs. ⁇ A and B, where the left image shows the result of a Rotation tool with the left mouse button pressed and thus implementing a rotation about either the x or y axes (depending on which direction the mouse is moved), and the right image shows the result of a Rotation tool with the right mouse button pressed thus implementing a rotation about the z axis.
  • buttons on, for example, the right side of the display screen can, for example, be provided.
  • This exemplary embodiment is depicted in Fig. 4.
  • a user can, for example, click on the appropriate button on the right side of the screen to activate the appropriate tool.
  • the three buttons can, for example, always be visible on the screen regardless of which mode the tool is actually in, if any.
  • a user can keep the keyboard's ⁇ ctrl> key, for example, pressed down to switch to the Positioning tool, or keep the ⁇ shift> key, for example, pressed down to switch to the Rotation tool.
  • the system can, for example, revert to the Manipulation tool.
  • a user can, for example, choose a manipulation tool via the control panel. He does not need to click on the Rotation Tool or Position Tool button, but can, for example, press the ⁇ ctrl> or ⁇ shift> key at any time to switch to rotation or position tool mode.
  • the Manipulation tool button need only be used if the user has clicked on the Position tool button or Rotation tool button and later wanted to use the Manipulation tool he had been using previously.
  • a Volume tool can be used to interact with a 3D object.
  • a Volume tool can be used, for example, to crop a volume and to move its tri-planar planes, when depicted in tri-planar view.
  • Any 3D object can be enclosed in a bounding box defined by its X, Y and Z maximum coordinates. It is common to want to see only a part of a given 3D object, for which purpose a "cropping" tool is often used. Such a cropping tool allows a user to change, for example, the visible boundaries of the bounding box.
  • a Volume tool can allow a user to crop (resize the bounding box) the object or roam around it (to move the bounding box in 3D space without resizing it).
  • An exemplary Volume tool performing cropping operations is shown in Figs. 7A and 7B, and an exemplary Volume tool performing roaming operations on a different volumetric object is shown in Figs. 7C and 7D.
  • the 3D object is a volumetric object, that is, it is made of voxels
  • two types of volumetric displays are possible, and hence two types of interactive manipulations are generally required.
  • One type of volumetric display is known as fully rendered, where all the voxels of the object (within the visible bounding box) are displayed.
  • the second type of volumetric display is a tri-planar rendering, in which three intersecting orthogonal planes of the object are displayed.
  • Figs. 8A and B depict exemplary tri-planar views of a volumetric object
  • Figs. 8C and D depict exemplary tri-planar views of another
  • volumetric object (the same one depicted in Figs. 7C and 7D).
  • a 3D object is rendered in a tri-planar manner, such a Volume tool can be used, for
  • a Volume tool generally casts a ray from its tip and if the ray touches the side of
  • VOL volumetric object
  • VOL is fully rendered (as shown, for example, in Figs. 7):
  • VOL is rendered in a tri-planar manner (as shown, for example, in Figs. 8): a. IF the cursor touches any of the planes, and WHILE the mouse button is pressed, move the plane according to the mouse movement; b. Project a ray from viewpoint through the cursor position; c. Find the face of the crop box of VOL which intersects the ray; d. IF the face is found and WHILE the mouse button is pressed, move the face of the crop box according to the cursor movement.
  • Figs. 7A and 7B depict an exemplary Volume tool working on a fully rendered exemplary volumetric object, here a heart
  • Figs. 7C and 7D depict an exemplary Volume tool working on another fully rendered exemplary volumetric object, here a human head.
  • Figs. 7A and 7C depict a mouse controlled cursor picking and moving the front face of a crop box (note in Fig. 7A the front face is selected and thus its borders shown in red, and in Fig. 7C the front face is selected and thus its borders shown in grey)
  • Figs. 7B and 7D show the mouse cursor outside the crop box and therefore roaming the crop box through the volumetric object.
  • Figs. 8 illustrate an exemplary Volume tool working on exemplary volumetric objects rendered in tri-planar mode.
  • the cursor picks and moves a horizontal (xz) plane
  • the mouse controlled cursor has not picked any plane (there was none that intersected with a projection from the viewpoint thorugh the cursor position) and thus picks and moves the front face of the crop box.
  • no plane was picked because the displayed cursor position does not snap onto any of the three depicted tri-planar planes.
  • the tri-planar volumetric object could be rotated (causing an intersection of a ray from viewpoint through the cursor position with the plane) until a desired plane is selected.
  • a Drill tool can be used to make virtual spherical holes on a selected volumetric object, and it can undo those holes when placed in restorer mode.
  • the implementation of a Drill tool using a mouse is basically the same as that on a 3D visualization system except that it is useful to restrict the depth of the mouse cursor so that it will not suddenly fall beyond any holes created in the object (i.e., into the virtual world in a direction away from the viewpoint). This unwanted behavior can happen, for example, when drilling a skull object that contains a large cavity. Without such a restriction, a cursor or icon could fall through the hole, then drop through the entire cavity and snap onto the opposite side of the skull (on its interior). It would be more intuitive to keep the cursor or icon at or near the surface of the object that has just been drilled, even if it is "floating" above the hole that was just made by the Drill.
  • the following pseudocode can be used, for example, to map a Drill tool to a mouse, and restrict its depth range to within [- THRESHOLD, THRESHOLD] of the z position that it had at the point it started drilling:
  • set Z mouse cursor's depth
  • THRESHOLD Choosing THRESHOLD to be sufficiently small will keep the Drill tool icon near the position it had when it had a surface to drill. As THRESHOLD becomes smaller, the cursor or icon is effectively held at the z position it had when drilling began, so as to "hover" over (or under, depending on the surface) the hole that has been created.
  • Figs. 9A (left image) and 9B (right image) illustrate the use of an exemplary 3D Drill tool mapped to a mouse.
  • Fig. 9A shows the effect of an exemplary mouse button being pressed and drilling out a hole in a volumetric object.
  • Fig. 9B shows the effect of a mouse being moved to the right while its button being continually pressed, which drills out a trail of holes, akin to the effect of a router.
  • Figs. 9C-9G depict a series of interactions with a different volumetric object ⁇ i.e., the head of Figs. 7C-D).
  • the Drill tool is held over the skull, but the mouse button is not pressed.
  • the Drill tool icon the circular area, is visible, but the volumetric object is not affected.
  • drilling operations are performed. In the depicted exemplary embodiment the Drill stays near the z position it had originally, when drilling began, even though the center of the Drill tool icon is hovering over a hole, and in Fig. 9G the entire Drill tool icon is hovering over a hole.
  • a Drill tool icon can be said to be located where its center point is (here the center of the circle) and thus need to be held “hovering" as in Figs. 9E and 9F when there is no longer any surface at the z position of its center, or alternatively, it can remain at a z position of any portion of a surface within the entire circle of its icon, and only need to be "hovered", such as is illustrated by the pseudocode above, or, for example, as in Fig. 2B, when the entire circle, for example, is above a hole.
  • a Ruler tool can measure distance in 3D space by placing a starting point and an ending point of the distance to be measured. Variants of the Ruler tool can measure distances between two points along a defined surface. This functionality is sometimes known as "curved measurement," as described in U.S. Patent Application No. 11/288,567, under common assignment herewith. In either variation, a Ruler tool or its equivalent needs to facilitate the placement of two points on a surface. In exemplary embodiments of the present invention, putting points on surfaces can be made trivial using a mouse (or other 2D device) controlled cursor, inasmuch as in exemplary embodiments of the present invention such a cursor can be automatically "snapped" onto the nearest surface behind it, as described above and as illustrated in Figs. 2. Exemplary Ruler tool functionality is depicted in Fig. 10, where two points have been set. Once the second point is set (in Fig. 10 the leftmost point) the distance between them can be displayed, for example.
  • Figs. 11A-C depict a series of interactions with a volumetric object that has had two points placed on it. Once the points are placed they remain fixed as the object is rotated and/or translated.
  • Figs. 12A-B depict a series of interactions with the volumetric object that has had two different points placed on it.
  • Fig. 12B shows the object of Fig. 12A after rotation and a visualization change, so that a large part of the skull is cropped away, but the two selected points, and the surface measurement line, remain.
  • a Picking tool can, for example, be used to pick or select any virtual object from among a group of virtual objects, and then position and orient such object for interactive examination.
  • determining the picked, or selected, object using a mouse can be made trivial inasmuch as the system inherently knows which virtual object the mouse's cursor has been snapped onto, as described above. If the two objects do not overlap completely, a user can always find a point where the object that is desired to be selected is not covered, and then pick it.
  • translations can be mapped to a 2D interface, such as a mouse, as follows.
  • a Picking tool By sliding a mouse horizontally or vertically and keeping, for example, its left button down, a Picking tool can be directed to move a picked object horizontally or vertically.
  • a user i.e., movement along the depth or "z" direction
  • he can, for example, slide the mouse in a defined direction (either horizontally or vertically) while pressing, for example, the right mouse button.
  • a user can, for example, slide a mouse horizontally or vertically while pressing down the ⁇ alt> key on a keyboard and a left mouse button.
  • a user can, for example, slide the mouse while pressing down the ⁇ alt> key and right mouse button.
  • Implementations of functionalities of other 3D visualization and manipulation tools using a 2D interface, such as a mouse, can be effected in similar fashion as the functions and tools that have been described above.
  • Such other tools can include, for example, tools to:

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Digital Computer Display Output (AREA)
  • Processing Or Creating Images (AREA)

Abstract

Des modes de réalisation représentatifs de la présente invention ont trait à un système de visualisation en 3D pouvant être transféré sur un ordinateur portable ou un ordinateur de bureau, ou sur un autre environnement informatique 2D classique utilisant une souris et un clavier comme interfaces utilisateur. Grâce aux procédés des modes de réalisation représentatifs de la présente invention, on peut amener un curseur ou une icône à une profondeur appropriée selon le contexte, ce qui permet de conserver une interactivité et une visualisation en 3D tout en disposant uniquement d'une commande en 2D. Des modes de réalisation représentatifs de la présente invention permettent de déterminer automatiquement une profondeur spatialement correcte pour les différents curseurs et icônes associés à différents outils, panneaux de commande et autres manipulations en 3D, et de leur affecter cette profondeur. Dans des modes de réalisation représentatifs de la présente invention, cela permet de conserver l'expérience tridimensionnelle d'une interaction avec un jeu de données en 3D bien que l'interface 2D servant à sélectionner les objets et à les manipuler ne puisse pas fournir directement de troisième coordonnée dimensionnelle. Dans certains modes de réalisation représentatifs de la présente invention, en se basant sur la position affectée au curseur ou à l'icône en 3D, sur la fonctionnalité de tel ou tel outil, et sur le fait d'avoir appuyé ou non sur de quelconques boutons d'un dispositif d'interface 2D, et si oui dans quel ordre, on peut mettre en œuvre un certain nombre d'outils et de fonctionnalités virtuels en 3D et les commander au moyen d'une interface d'ordinateur classique en 2D.
PCT/IB2007/004506 2006-09-19 2007-09-19 Procédés et systèmes pour une interaction avec un système de visualisation en 3d au moyen d'une interface 2d ('dextrolap') WO2008093167A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US84565406P 2006-09-19 2006-09-19
US60/845,654 2006-09-19

Publications (2)

Publication Number Publication Date
WO2008093167A2 true WO2008093167A2 (fr) 2008-08-07
WO2008093167A3 WO2008093167A3 (fr) 2008-12-18

Family

ID=39674554

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2007/004506 WO2008093167A2 (fr) 2006-09-19 2007-09-19 Procédés et systèmes pour une interaction avec un système de visualisation en 3d au moyen d'une interface 2d ('dextrolap')

Country Status (1)

Country Link
WO (1) WO2008093167A2 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2426586A1 (fr) * 2010-09-07 2012-03-07 Sony Corporation Dispositif de traitement d'informations, programme et procédé de traitement d'informations
CN110942519A (zh) * 2019-10-29 2020-03-31 华南师范大学 一种计算机组装虚拟实验系统及其实现方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020105529A1 (en) * 2000-02-11 2002-08-08 Jason Bowser Generation and display of multi-image video streams
US7605826B2 (en) * 2001-03-27 2009-10-20 Siemens Corporate Research, Inc. Augmented reality guided instrument positioning with depth determining graphics
CA2507213C (fr) * 2002-11-27 2013-01-22 Vision Iii Imaging, Inc. Balayage de parallaxe par manipulation de position d'objet

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2426586A1 (fr) * 2010-09-07 2012-03-07 Sony Corporation Dispositif de traitement d'informations, programme et procédé de traitement d'informations
CN102402281A (zh) * 2010-09-07 2012-04-04 索尼公司 信息处理装置、程序及信息处理方法
US8836642B2 (en) 2010-09-07 2014-09-16 Sony Corporation Information processing device, program, and information processing method
CN110942519A (zh) * 2019-10-29 2020-03-31 华南师范大学 一种计算机组装虚拟实验系统及其实现方法
CN110942519B (zh) * 2019-10-29 2023-04-28 华南师范大学 一种计算机组装虚拟实验系统及其实现方法

Also Published As

Publication number Publication date
WO2008093167A3 (fr) 2008-12-18

Similar Documents

Publication Publication Date Title
US20080094398A1 (en) Methods and systems for interacting with a 3D visualization system using a 2D interface ("DextroLap")
Grossman et al. Multi-finger gestural interaction with 3d volumetric displays
Vanacken et al. Exploring the effects of environment density and target visibility on object selection in 3D virtual environments
US20040246269A1 (en) System and method for managing a plurality of locations of interest in 3D data displays ("Zoom Context")
EP3136212B1 (fr) Interface utilisateur graphique de type joystick pour ajuster un plan en coupe transversale dans un volume 3d
US5689628A (en) Coupling a display object to a viewpoint in a navigable workspace
EP2681649B1 (fr) Système et procédé de navigation dans un environnement tridimensionnel à l'aide d'une interface multi-entrées
Schmalstieg et al. Using transparent props for interaction with the virtual table
US7782319B2 (en) Three-dimensional orientation indicator and controller
Zeleznik et al. Unicam—2D gestural camera controls for 3D environments
US7889227B2 (en) Intuitive user interface for endoscopic view visualization
Coffey et al. Slice WIM: a multi-surface, multi-touch interface for overview+ detail exploration of volume datasets in virtual reality
US20070279436A1 (en) Method and system for selective visualization and interaction with 3D image data, in a tunnel viewer
EP2669781A1 (fr) Interface utilisateur de navigation dans un environnement tridimensionnel
WO2024012208A1 (fr) Procédé et appareil de navigation de vue
US7477232B2 (en) Methods and systems for interaction with three-dimensional computer models
Sun et al. Selecting and Sliding Hidden Objects in 3D Desktop Environments.
Ohnishi et al. Virtual interaction surface: Decoupling of interaction and view dimensions for flexible indirect 3D interaction
WO2008093167A2 (fr) Procédés et systèmes pour une interaction avec un système de visualisation en 3d au moyen d'une interface 2d ('dextrolap')
Djajadiningrat et al. Cubby: a multiscreen movement parallax display for direct manual manipulation
Glueck et al. Multiscale 3D reference visualization
Guan et al. Volume-based tumor neurosurgery planning in the Virtual Workbench
Serra et al. An interface for precise and comfortable 3D work with volumetric medical datasets
Serra et al. Interaction techniques for a virtual workspace
JP5247398B2 (ja) 表示調整装置、表示調整方法及びコンピュータプログラム

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07872477

Country of ref document: EP

Kind code of ref document: A2

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载