33312results about "Steroscopic systems" patented technology

A head mounted device provides an immersive virtual or augmented reality experience for viewing data and enabling collaboration among multiple users. Rendering images in a virtual or augmented reality system may include capturing an image and spatial data with a body mounted camera and sensor array, receiving an input indicating a first anchor surface, calculating parameters with respect to the body mounted camera and displaying a virtual object such that the virtual object appears anchored to the selected first anchor surface. Further operations may include receiving a second input indicating a second anchor surface within the captured image that is different from the first anchor surface, calculating parameters with respect to the second anchor surface and displaying the virtual object such that the virtual object appears anchored to the selected second anchor surface and moved from the first anchor surface.

A powerful, scaleable, and reconfigurable image processing system and method of processing data therein is described. This general purpose, reconfigurable engine with toroidal topology, distributed memory, and wide bandwidth I / O are capable of solving real applications at real-time speeds. The reconfigurable image processing system can be optimized to efficiently perform specialized computations, such as real-time video and audio processing. This reconfigurable image processing system provides high performance via high computational density, high memory bandwidth, and high I / O bandwidth. Generally, the reconfigurable image processing system and its control structure include a homogeneous array of 16 field programmable gate arrays (FPGA) and 16 static random access memories (SRAM) arranged in a partial torus configuration. The reconfigurable image processing system also includes a PCI bus interface chip, a clock control chip, and a datapath chip. It can be implemented in a single board. It receives data from its external environment, computes correspondence, and uses the results of the correspondence computations for various post-processing industrial applications. The reconfigurable image processing system determines correspondence by using non-parametric local transforms followed by correlation. These non-parametric local transforms include the census and rank transforms. Other embodiments involve a combination of correspondence, rectification, a left-right consistency check, and the application of an interest operator.

A method and apparatus for mapping depth of an object (22) in a preferred arrangement uses a projected light pattern to provide a selected texture to the object (22) along the optical axis (24) of observation. An imaging system senses (32, 34) first and second images of the object (22) with the projected light pattern and compares the defocused of the projected pattern in the images to determine relative depth of elemental portions of the object (22).

A method is presented for correcting errors in a 3D scanner. Measurement errors in the 3D scanner are determined by scanning each of a plurality of calibration objects in each of a plurality of sectors in the 3D scanner's field of view. The calibration objects have a known height, a known width, and a known length. The measurements taken by the 3D scanner are compared to the known dimensions to derive a measurement error for each dimension in each sector. An estimated measurement error is calculated based on scans of each of the plurality of calibration objects. When scanning target objects in a given sector, the estimated measurement error for that sector is used to correct measurements obtained by the 3D scanner.

An image conversion system for converting monoscopic images for viewing in three dimensions including: an input means adapted to receive the monoscopic images; a preliminary analysis means to determine if there is any continuity between a first image and a second image of the monoscopic image sequence; a secondary analysis means for receiving monoscopic images which have a continuity, and analyzing the images to determine the speed and direction of motion, and the depth, size and position of objects; a first processing means for processing the monoscopic images based on data received from the preliminary analysis means or the secondary analysis means; a second processing means capable of further processing images received from the first processing means; a transmission means capable of transferring the processed images to a stereoscopic display system.

A camera array captures plural component images which are combined into a single scene from which “panning” and “zooming” within the scene are performed. In one embodiment, each camera of the array is a fixed digital camera. The images from each camera are warped and blended such that the combined image is seamless with respect to each of the component images. Warping of the digital images is performed via pre-calculated non-dynamic equations that are calculated based on a registration of the camera array. The process of registering each camera in the arrays is performed either manually, by selecting corresponding points or sets of points in two or more images, or automatically, by presenting a source object (laser light source, for example) into a scene being captured by the camera array and registering positions of the source object as it appears in each of the images. The warping equations are calculated based on the registration data and each scene captured by the camera array is warped and combined using the same equations determined therefrom. A scene captured by the camera array is zoomed, or selectively steered to an area of interest. This zooming- or steering, being done in the digital domain is performed nearly instantaneously when compared to cameras with mechanical zoom and steering functions.

An interactive video display system. A display screen is for displaying a visual image for presentation to a user. A camera is for detecting an object in an interactive area located in front of the display screen, the camera operable to capture three-dimensional information about the object. A computer system is for directing the display screen to change the visual image in response to the object.

A display device is mounted on and / or inside the eye. The eye mounted display contains multiple sub-displays, each of which projects light to different retinal positions within a portion of the retina corresponding to the sub-display. The projected light propagates through the pupil but does not fill the entire pupil. In this way, multiple sub-displays can project their light onto the relevant portion of the retina. Moving from the pupil to the cornea, the projection of the pupil onto the cornea will be referred to as the corneal aperture. The projected light propagates through less than the full corneal aperture. The sub-displays use spatial multiplexing at the corneal surface. Various electronic devices interface to the eye mounted display.

Depth maps are generated from a monoscopic source images and asymmetrically smoothed to a near-saturation level. Each depth map contains depth values focused on edges of local regions in the source image. Each edge is defined by a predetermined image parameter having an estimated value exceeding a predefined threshold. The depth values are based on the corresponding estimated values of the image parameter. The depth map is used to process the source image by a depth image based rendering algorithm to create at least one deviated image, which forms with the source image a set of monoscopic images. At least one stereoscopic image pair is selected from such a set for use in generating different viewpoints for multiview and stereoscopic purposes, including still and moving images.

An augmented reality device for inserting virtual imagery into a user's view of their physical environment, the device comprising: a display device through which the user can view the physical environment; an optical sensing device for sensing at least one surface in the physical environment; and, a controller for projecting the virtual imagery via the display device; wherein during use, the controller uses wave front modulation to match the curvature of the wave fronts of light reflected from the display device to the user's eyes with the curvature of the wave fronts of light that would be transmitted through the device display if the virtual imagery were situated at a predetermined position relative to the surface, such that the user sees the virtual imagery at the predetermined position regardless of changes in position of the user's eyes with respect to the see-through display.

A sentient system combines detection, tracking, and immersive visualization of a cluttered and crowded environment, such as an office building, terminal, or other enclosed site using a network of stereo cameras. A guard monitors the site using a live 3D model, which is updated from different directions using the multiple video streams. As a person moves within the view of a camera, the system detects its motion and tracks the person's path, it hands off the track to the next camera when the person goes out of that camera's view. Multiple people can be tracked simultaneously both within and across cameras, with each track shown on a map display. The track system includes a track map browser that displays the tracks of all moving objects as well as a history of recent tracks and a video flashlight viewer that displays live immersive video of any person that is being tracked.

A system projects a user-viewable, computer-generated or -fed image, wherein a head-mounted projector is used to project an image onto a retro-reflective surface, so only the viewer can see the image. The projector is connected to a computer that contains software to create virtual 2-D and or 3-D images for viewing by the user! Further, on projector each is mounted on either side of the user's head, and, by choosing for example a retro angle of less than about 10 degrees, each eye can only see the image of one of the projectors at a give distance up to 3 meters, in this example, from the retro-reflective screen. The retro angle used may be reduced with larger viewing distance desired. These projectors use lasers to avoid the need for focusing, and in some cases there projectors use instead of lasers highly collimated LEO light sources to avoid the need for focusing.

A camera phone includes a phone stage for generating voice signals, a first image sensor for generating a first sensor output, a first fixed focal length wide angle lens for forming a first image of the scene on the first image sensor, a second image sensor for generating a second sensor output, and a second fixed focal length telephoto lens pointing in the same direction as the first lens and forming a second image of the same scene on the second image sensor. A control element selects either the first sensor output from the first image sensor or the second sensor output from the second image sensor. A processing section produces the output image signals from the selected sensor output, and a cellular stage processes the image and voice signals for transmission over a cellular network.

A vehicle control system and method includes structure and steps for capturing an image of a front seat of the vehicle and outputting image data corresponding thereto. A processor is provided which receives the image data output form the imaging device, compares the received image data with stored image data, and outputs a vehicle equipment control signal based on the comparison. Preferably, the vehicle equipment control signal controls one or more of airbag activation, mirror reflectance, vehicle lights activation, and vehicle intruder alarms. Preferably, the imaging device comprises a single chip camera disposed adjacent the vehicle rearview mirror.

There are many, many inventions described herein. In one aspect, what is disclosed is a digital camera including a plurality of arrays of photo detectors, including a first array of photo detectors to sample an intensity of light of a first wavelength and a second array of photo detectors to sample an intensity of light of a second wavelength. The digital camera further may also include a first lens disposed in an optical path of the first array of photo detectors, wherein the first lens includes a predetermined optical response to the light of the first wavelength, and a second lens disposed in with an optical path of the second array of photo detectors wherein the second lens includes a predetermined optical response to the light of the second wavelength. In addition, the digital camera may include signal processing circuitry, coupled to the first and second arrays of photo detectors, to generate a composite image using (i) data which is representative of the intensity of light sampled by the first array of photo detectors, and (ii) data which is representative of the intensity of light sampled by the second array of photo detectors; wherein the first array of photo detectors, the second array of photo detectors, and the signal processing circuitry are integrated on or in the same semiconductor substrate.

Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In several embodiments, detecting parallax involves using competing subsets of images to estimate the depth of a pixel location in an image from a reference viewpoint. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

There are many, many inventions described herein. In one aspect, what is disclosed is a digital camera including a plurality of arrays of photo detectors, including a first array of photo detectors to sample an intensity of light of a first wavelength and a second array of photo detectors to sample an intensity of light of a second wavelength. The digital camera further may also include a first lens disposed in an optical path of the first array of photo detectors, wherein the first lens includes a predetermined optical response to the light of the first wavelength, and a second lens disposed in with an optical path of the second array of photo detectors wherein the second lens includes a predetermined optical response to the light of the second wavelength. In addition, the digital camera may include signal processing circuitry, coupled to the first and second arrays of photo detectors, to generate a composite image using (i) data which is representative of the intensity of light sampled by the first array of photo detectors, and (ii) data which is representative of the intensity of light sampled by the second array of photo detectors; wherein the first array of photo detectors, the second array of photo detectors, and the signal processing circuitry are integrated on or in the same semiconductor substrate.

A stereo camera supporting apparatus of the present invention comprises a joining member constituted in such a manner as to support a stereo camera on a vehicle, and a control device which controls posture or position of the stereo camera supported on the vehicle by the joining member. The control device controls the posture or position of the stereo camera with respect to video obtained by the stereo camera in such a manner that a contour portion present in the highest level position in a contour of a noted subject in the video is positioned in a frame upper end of the video or its vicinity irrespective of a change of the posture or position of the vehicle.

Apparatus for generating computer models of individuals is provided comprising a booth (1) that is connected to a server (2) via the Internet (3). Image data of an individual is captured using the booth (1) and a computer model corresponding to the individual is then generated by comparing the captured image data relative to a stored generic model. Data representative of a generated model is then transmitted to the server (2) where it is stored. Stored data can then be retrieved via the Internet using a personal computer (4) having application software stored therein. The application software on the personal computer (4) can then utilise the data to create graphic representations of an individual in any one of a number of poses.

The various embodiments include systems and methods for rendering images in a virtual or augmented reality system that may include capturing scene images of a scene in a vicinity of a first and a second projector, capturing spatial data with a sensor array in the vicinity of the first and second projectors, analyzing captured scene images to recognize body parts, and projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts. Additional rendering operations may include tracking movements of the recognized body parts, applying a detection algorithm to the tracked movements to detect a predetermined gesture, applying a command corresponding to the detected predetermined gesture, and updating the projected images in response to the applied command.

Systems and methods are described for generating restricted depth of field depth maps. In one embodiment, an image processing pipeline application configures a processor to: determine a desired focal plane distance and a range of distances corresponding to a restricted depth of field for an image rendered from a reference viewpoint; generate a restricted depth of field depth map from the reference viewpoint using the set of images captured from different viewpoints, where depth estimation precision is higher for pixels with depth estimates within the range of distances corresponding to the restricted depth of field and lower for pixels with depth estimates outside of the range of distances corresponding to the restricted depth of field; and render a restricted depth of field image from the reference viewpoint using the set of images captured from different viewpoints and the restricted depth of field depth map.

A blind spot monitoring system for a vehicle includes two pairs of stereo cameras, two displays and a controller. The stereo cameras monitor vehicle blind spots and generate a corresponding pair of digital signals. The display shows a rearward vehicle view and may replace, or work in tandem with, a side view mirror. The controller is located in the vehicle and receives two pairs of digital signals. The controller includes control logic operative to analyze a stereopsis effect between each pair of stereo cameras and the optical flow over time to control the displays. The displays will show an expanded rearward view when a hazard is detected in the vehicle blind spot and show a normal rearward view when no hazard is detected in the vehicle blind spot.

Methods and apparatus to create and display stereoscopic and panoramic images are disclosed. Apparatus is provided to control the position of a lens in relation to a reference lens. Methods and apparatus are provided to generate multiple images that are combined into a stereoscopic or a panoramic image. An image may be a static image. It may also be a video image. A controller provides correct camera settings for different conditions. An image processor creates a stereoscopic or a panoramic image from the correct settings provided by the controller. A panoramic video wall system is also disclosed.

An object of the present invention is to provide a stereoscopic video recording method, a stereoscopic video recording medium, a stereoscopic video reproducing method, a stereoscopic video recording apparatus, and a stereoscopic video reproducing apparatus that are capable of more optimally performing recording and reproduction of stereoscopic video. According to one solving means of the present invention, a stereoscopic video recording method records, in a recording medium, stereoscopic video including left-eye images (2) and right-eye images (1) utilizing parallax information, by using a stereoscopic video recording apparatus (6). From video content that contains stereoscopic images, the method grasps the amount of variation in parallactic angle having a given or larger value, a variation time that the variation in parallactic angle takes, and the number of times that the variation in parallactic angle occurs. The method then calculates an evaluation value that corresponds to the degree of eye fatigue on the basis of the amount of variation, the variation time, and the number of times of the variation, encodes the video content in such a manner that the evaluation value is within a given range, and records the encoded video content in the recording medium.

Single-camera image processing methods are disclosed for 3D navigation within ordinary moving video. Along with color and brightness, XYZ coordinates can be defined for every pixel. The resulting geometric models can be used to obtain measurements from digital images, as an alternative to on-site surveying and equipment such as laser range-finders. Motion parallax is used to separate foreground objects from the background. This provides a convenient method for placing video elements within different backgrounds, for product placement, and for merging video elements with computer-aided design (CAD) models and point clouds from other sources. If home users can save video fly-throughs or specific 3D elements from video, this method provides an opportunity for proactive, branded media sharing. When this image processing is used with a videoconferencing camera, the user's movements can automatically control the viewpoint, creating 3D hologram effects on ordinary televisions and computer screens.

An electronic camera for producing an output image of a scene from a captured image signal includes a first imaging stage comprising a first image sensor for generating a first sensor output and a first lens for forming a first image of the scene on the first image sensor, and a second imaging stage comprising a second image sensor for generating a second sensor output and a second lens for forming a second image of the scene on the second image sensor, where the lenses have different focal lengths. A processing stage uses the sensor output from one of the imaging stages as the captured image signal and uses the images from both imaging stages to generate a range map identifying distances to the different portions of the scene.

The present invention discloses methods of digitally converting 2D motion pictures or any other 2D image sequences to stereoscopic 3D image data for 3D exhibition. In one embodiment, various types of image data cues can be collected from 2D source images by various methods and then used for producing two distinct stereoscopic 3D views. Embodiments of the disclosed methods can be implemented within a highly efficient system comprising both software and computing hardware. The architectural model of some embodiments of the system is equally applicable to a wide range of conversion, re-mastering and visual enhancement applications for motion pictures and other image sequences, including converting a 2D motion picture or a 2D image sequence to 3D, re-mastering a motion picture or a video sequence to a different frame rate, enhancing the quality of a motion picture or other image sequences, or other conversions that facilitate further improvement in visual image quality within a projector to produce the enhanced images.

Embodiments of the invention provide systems and methods for three-dimensional imaging with wide field of view and precision timing. In accordance with one aspect, a three-dimensional imaging system includes an illumination subsystem configured to emit a light pulse with a divergence sufficient to irradiate a scene having a wide field of view. A sensor subsystem is configured to receive over a wide field of view portions of the light pulse reflected or scattered by the scene and including: a modulator configured to modulate as a function of time an intensity of the received light pulse portion to form modulated received light pulse portions; and means for generating a first image corresponding to the received light pulse portions and a second image corresponding to the modulated received light pulse portions. A processor subsystem is configured to obtain a three-dimensional image based on the first and second images.

Systems and methods in accordance with embodiments of the invention are disclosed that use super-resolution (SR) processes to use information from a plurality of low resolution (LR) images captured by an array camera to produce a synthesized higher resolution image. One embodiment includes obtaining input images using the plurality of imagers, using a microprocessor to determine an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images, and using a microprocessor to determine a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image. In addition, each forward imaging transformation corresponds to the manner in which each imager in the imaging array generate the input images, and the high resolution image synthesized by the microprocessor has a resolution that is greater than any of the input images.