4156 results about "Viewpoints" patented technology

A multiple camera tracking system for interfacing with an application program running on a computer is provided. The tracking system includes two or more video cameras arranged to provide different viewpoints of a region of interest, and are operable to produce a series of video images. A processor is operable to receive the series of video images and detect objects appearing in the region of interest. The processor executes a process to generate a background data set from the video images, generate an image data set for each received video image, compare each image data set to the background data set to produce a difference map for each image data set, detect a relative position of an object of interest within each difference map, and produce an absolute position of the object of interest from the relative positions of the object of interest and map the absolute position to a position indicator associated with the application program.

Depth maps are generated from a monoscopic source images and asymmetrically smoothed to a near-saturation level. Each depth map contains depth values focused on edges of local regions in the source image. Each edge is defined by a predetermined image parameter having an estimated value exceeding a predefined threshold. The depth values are based on the corresponding estimated values of the image parameter. The depth map is used to process the source image by a depth image based rendering algorithm to create at least one deviated image, which forms with the source image a set of monoscopic images. At least one stereoscopic image pair is selected from such a set for use in generating different viewpoints for multiview and stereoscopic purposes, including still and moving images.

Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In several embodiments, detecting parallax involves using competing subsets of images to estimate the depth of a pixel location in an image from a reference viewpoint. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Systems and methods are described for generating restricted depth of field depth maps. In one embodiment, an image processing pipeline application configures a processor to: determine a desired focal plane distance and a range of distances corresponding to a restricted depth of field for an image rendered from a reference viewpoint; generate a restricted depth of field depth map from the reference viewpoint using the set of images captured from different viewpoints, where depth estimation precision is higher for pixels with depth estimates within the range of distances corresponding to the restricted depth of field and lower for pixels with depth estimates outside of the range of distances corresponding to the restricted depth of field; and render a restricted depth of field image from the reference viewpoint using the set of images captured from different viewpoints and the restricted depth of field depth map.

Single-camera image processing methods are disclosed for 3D navigation within ordinary moving video. Along with color and brightness, XYZ coordinates can be defined for every pixel. The resulting geometric models can be used to obtain measurements from digital images, as an alternative to on-site surveying and equipment such as laser range-finders. Motion parallax is used to separate foreground objects from the background. This provides a convenient method for placing video elements within different backgrounds, for product placement, and for merging video elements with computer-aided design (CAD) models and point clouds from other sources. If home users can save video fly-throughs or specific 3D elements from video, this method provides an opportunity for proactive, branded media sharing. When this image processing is used with a videoconferencing camera, the user's movements can automatically control the viewpoint, creating 3D hologram effects on ordinary televisions and computer screens.

An image-based tele-presence system forward warps video images selected from a plurality fixed imagers using local depth maps and merges the warped images to form high quality images that appear as seen from a virtual position. At least two images, from the images produced by the imagers, are selected for creating a virtual image. Depth maps are generated corresponding to each of the selected images. Selected images are warped to the virtual viewpoint using warp parameters calculated using corresponding depth maps. Finally the warped images are merged to create the high quality virtual image as seen from the selected viewpoint. The system employs a video blanket of imagers, which helps both optimize the number of imagers and attain higher resolution. In an exemplary video blanket, cameras are deployed in a geometric pattern on a surface.

In certain embodiments, a graphical representation of a two-dimensional radial menu is displayed in a graphical user interface. The graphical representation of the two-dimensional radial menu is transformed into a graphical representation of a three-dimensional radial menu in the graphical user interface. In certain embodiments, the displaying comprises utilizing data representative of a three-dimensional radial menu model to render the graphical representation of the two-dimensional radial menu, based on a first viewpoint, in the graphical user interface, and the transforming comprises utilizing the data representative of the three-dimensional radial menu model to render the graphical representation of the three-dimensional radial menu, based on a second viewpoint, in the graphical user interface. In certain embodiments, the transforming comprises repositioning a viewpoint associated with the graphical representation of the two-dimensional radial menu to produce the graphical representation of the three-dimensional radial menu.

A computer graphics display system such as a video game system provides virtual camera 3-D viewpoint panning control based on a pointer. When the pointer is displayed within a virtual camera panning control region, the system automatically pans the virtual camera toward the pointer. When the pointer is displayed within a different region, panning is deactivated and the user can freely move the cursor (e.g., to control the direction a weapon is pointed) without panning the virtual camera viewpoint. Flexible viewpoint control and other animated features are provided based on a pointing device such as a handheld video game optical pointing control. Useful applications include but are not limited to first person shooter type video games.

A patient fall prediction system receives video image frames from a surveillance camera positioned in a patient's room and analyses the video image frames for movement that may be a precursor to a patient fall. In set up phase, the viewpoint of the camera is directed at a risk area associated with patient falls, beds, chairs, wheelchairs, etc. A risk area is defined graphically in the viewport. The patient fall prediction system generates a plurality of concurrent motion detection zones that are situated proximate to the graphic markings of the risk areas. These motion detection zones are monitored for changes between video image frames that indicate a movement. The pattern of detections is recorded and compared to a fall movement detection signature. One fall movement detection signature is a sequential detection order from the motion detection zone closest to the risk area in the frames associated with patient falls, to the motion detection zone farthest away from the risk area. The patient fall prediction system continually monitors the motion detection zones for changes between image frames and compiles detections lists that are compared to known movement detection signatures, such as a fall movement detection signature. Once a match is identified, the patient fall prediction system issues a fall warning to a healthcare provider.

Methods and apparatuses to generate stereoscopic views for image guided surgical navigation. One embodiment includes transforming a first image of a scene into a second image of the scene according to a mapping between two views of the scene. Another embodiment includes generating a stereoscopic display of the scene using a first image and a second image of a scene during a surgical procedure, where a position and orientation of an imaging device is at least partially changed to capture the first and second images from different viewpoints. A further embodiment includes: determining a real time location of a probe relative to a patient during a surgical procedure; determining a pair of virtual viewpoints according to the real time location of the probe; and generating a virtual stereoscopic image showing the probe relative to the patient, according to the determined pair of virtual viewpoints.

A system and process for generating, and then rendering and displaying, an interactive viewpoint video in which a user can watch a dynamic scene while manipulating (freezing, slowing down, or reversing) time and changing the viewpoint at will. In general, the interactive viewpoint video is generated using a small number of cameras to capture multiple video streams. A multi-view 3D reconstruction and matting technique is employed to create a layered representation of the video frames that enables both efficient compression and interactive playback of the captured dynamic scene, while at the same time allowing for real-time rendering.

There are provided a method and an apparatus for generating a multi-viewpoint depth map, and a method for generating a disparity of a multi-viewpoint image. A method for generating a multi-viewpoint depth map according to the present invention includes the steps of: (a) acquiring a multi-viewpoint image constituted by a plurality of images by using a plurality of cameras (b) acquiring an image and depth information by using a depth camera; (c) estimating coordinates of the same point in a space in the plurality of images by using the acquired depth information; (d) determining disparities in the plurality of images with respect to in the same point by searching a predetermined region around the estimated coordinates; and (e) generating a multi-viewpoint depth map by using the determined disparities. According to the above-mentioned present invention, it is possible to generate a multi-viewpoint depth map within a shorter time and generate a multi-viewpoint depth map having higher quality than a multi-viewpoint depth map generated by using known stereo matching.

A 3-dimensional video game apparatus comprises display 40 for displaying an image; image-processing means 16 for creating a 3-dimensional image from the viewpoint of a simulated camera and displaying that image on a display screen; and game controller 14 which conducts the game in response to reactions of the player, responsive to images displayed on display 40. The game apparatus further comprises camera unit 42 which can be arranged in such an orientation that its field of view is the play area in front of the display screen, and which periodically captures images of the play area in the state thus arranged; a position recognizing part which detects the player's head from the captured image, and detects the two-dimensional movement of the detected head position; and viewpoint change processor which moves the viewpoint of said simulated camera in conjunction with the change direction and change amount of the position of the detected marked region.

A surgical navigation system has a computer with a memory and display connected to a surgical instrument or pointer and position tracking system, so that the location and orientation of the pointer are tracked in real time and conveyed to the computer. The computer memory is loaded with data from an MRI, CT, or other volumetric scan of a patient, and this data is utilized to dynamically display 3-dimensional perspective images in real time of the patient's anatomy from the viewpoint of the pointer. The images are segmented and displayed in color to highlight selected anatomical features and to allow the viewer to see beyond obscuring surfaces and structures. The displayed image tracks the movement of the instrument during surgical procedures. The instrument may include an imaging device such as an endoscope or ultrasound transducer, and the system displays also the image for this device from the same viewpoint, and enables the two images to be fused so that a combined image is displayed. The system is adapted for easy and convenient operating room use during surgical procedures.

Systems and methods for stereo imaging with camera arrays in accordance with embodiments of the invention are disclosed. In one embodiment, a method of generating depth information for an object using two or more array cameras that each include a plurality of imagers includes obtaining a first set of image data captured from a first set of viewpoints, identifying an object in the first set of image data, determining a first depth measurement, determining whether the first depth measurement is above a threshold, and when the depth is above the threshold: obtaining a second set of image data of the same scene from a second set of viewpoints located known distances from one viewpoint in the first set of viewpoints, identifying the object in the second set of image data, and determining a second depth measurement using the first set of image data and the second set of image data.

This invention relates to an apparatus and a method for video capture of a three-dimensional region of interest in a scene using an array of video cameras. The video cameras of the array are positioned for viewing the three-dimensional region of interest in the scene from their respective viewpoints. A triggering mechanism is provided for staggering the capture of a set of frames by the video cameras of the array. The apparatus has a processing unit for combining and operating on the set of frames captured by the array of cameras to generate a new visual output, such as high-speed video or spatio-temporal structure and motion models, that has a synthetic viewpoint of the three-dimensional region of interest. The processing involves spatio-temporal interpolation for determining the synthetic viewpoint space-time trajectory. In some embodiments, the apparatus computes a multibaseline spatio-temporal optical flow.

Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

A system and method for generating and playing back three-dimensional (3D) movies are disclosed. The system is capable of partially rendering frames without relying upon exact viewpoint information. The partially rendered frames may be rendered to the extent possible without performing viewpoint-dependent processes, and then compressed and stored to a carrier medium. To reduce the amount of data to be stored, the viewer's possible viewpoints may be restricted (e.g., by defining a viewpoint-limiting volume or region). The resulting partially-rendered geometry data may be compressed using geometry compression. During playback, the compressed frames are read as a stream, and decompressed. Any final viewpoint-dependent rendering operations may then be performed (e.g., some lighting calculations and atmospheric effects, some fogging, specular highlighting, and reflections). A sensor such as a head-tracker may provide real-time viewpoint information that may be used by the playback system. After rendering, the frames are rasterized and then displayed in stereo.

A method and apparatus for creating motion blur, depth of field, and screen door effects when rendering three-dimensional graphics data are disclosed. A graphics system configured with a graphics processor, a super-sampled sample buffer, and a sample-to-pixel calculation unit is disclosed. The graphics processor may be configured to use a sample mask to select different subsets of sample coordinates to be rendered for a particular frame. Each subset may be rendered applying a different set of attributes, and the resulting samples may then be stored together in the sample buffer. The sample-to-pixel calculation unit may be configured to filter the samples into output pixels that are provided to a display device. The attributes that may be changed from subset to subset include the viewpoint, the time at which objects in the data are rendered, which objects or geometric primitives in the data are rendered, the position of objects in the data, the color of objects in the data, the transparency of objects in the data, and the shape of objects in the data.

A system and method for navigation within a surgical field are presented. In exemplary embodiments according to the present invention a micro-camera can be provided in a hand-held navigation probe tracked by a tracking system. This enables navigation within an operative scene by viewing real-time images from the viewpoint of the micro-camera within the navigation probe, which are overlaid with computer generated 3D graphics depicting structures of interest generated from pre-operative scans. Various transparency settings of the camera image and the superimposed 3D graphics can enhance the depth perception, and distances between a tip of the probe and any of the superimposed 3D structures along a virtual ray extending from the probe tip can be dynamically displayed in the combined image. In exemplary embodiments of the invention a virtual interface can be displayed adjacent to the combined image on a system display, thus facilitating interaction with various navigation related functions. In exemplary embodiments according to the present invention virtual reality systems can be used to plan surgical approaches with multi-modal CT and MRI data. This allows for generating 3D structures as well as marking ideal surgical paths. The system and method presented thus enable transfer of a surgical planning scenario to a real-time view of an actual surgical field, thus enhancing navigation.

A system and method for performing object and class recognition that allows for wide changes of viewpoint and distance of objects is disclosed. The invention provides for choosing pose-invariant interest points of a three-dimensional (3D) image, and for computing pose-invariant feature descriptors of the image. The system and method also allows for the construction of three-dimensional (3D) object and class models from the pose-invariant interest points and feature descriptors of previously obtained scenes. Interest points and feature descriptors of a newly acquired scene may be compared to the object and / or class models to identify the presence of an object or member of the class in the new scene.

This invention is a system, method, and article of manufacture for imaging tubular structures of the human body, such as the digestive tract of a living person, with a medical imaging device such as a computed tomography (CT) scanner and a computer work station. The system comprises receiving a first image data set representative of a portion of the colon in a prone position and a second image data set representative of a portion of the colon in a supine position, at a series of viewpoints. At each of the viewpoints, an image is generated of the colon in the prone and supine positions. The prone and supine images of the colon are simultaneously displayed on a screen display in a dual view mode.

The position and orientation of the viewpoint of an observer (100) are acquired. The position and orientation of a stylus (120) are acquired. A list image is laid out near the position of the stylus (120). An image of a virtual space after laying out the list image, which is seen in accordance with the position and orientation of the viewpoint, generated. The generated image is output to the display screen of an HMD (110).

A background image generator 51 renders a first 3D model representing a landscape to generate a background image, an image generator 52 generates a second 3D model to be located between a camera viewpoint and the first 3D model, an image adhering unit 53 selects a specified range of the background image and adheres the background image within the selected range as a texture to the second 3D model, and a combined image generator 54 renders the second 3D model having the background image within the specified range adhered thereto and combines the rendered model with the background image to generate a combined image.

A system for selecting oblique images from a collection of geo-referenced oblique images and viewing them within the context of a virtual, three- or four-dimensional (3D space and time) geographic scene, providing the ability to analyze and interact with the oblique image being viewed. The system automatically selects and displays the best fit oblique image from an image warehouse based on the user's current 3D / 4D viewpoint, and continuously maintains the geo-registration of the oblique image as the user adjusts the viewpoint.

A system for capturing super wide-angle panoramic images. In particular, a two-reflector system is disclosed which is substantially self-correcting in which optical aberrations are substantially eliminated, such as field curvature, astigmatism and the like. Moreover, the super wide-angle panoramic imaging apparatus of the invention captures a super-wide field of view from a substantially single reference viewpoint. The invention provides a substantially compact viewpoint, while also having a substantially flat and stigmatic image plane, in the context of a super wide-angle panoramic system. Devices and methods for capturing panoramic images of super wide-angle scenes are provided. In a particular embodiment of the invention, two reflectors are provided (e.g., one a hyperboloidal mirror, the other a concave ellipsoidal or spherical mirror), a relay system (e.g., optics such as a mirror, a lens, a pinhole and the like) and an image sensor (e.g., an electronic photo-sensor, a film and the like).