50 results about "Nonrigid motion" patented technology

A method of analysing a sequence of images of a deformable object in non-rigid motion involves modeling the boundary using a non-rigid contour. A representation of movement of the contour through the sequence of images is calculated using a tracking space shape. The calculated movement representation is decomposed using an interpretational space shape that is different than the tracking space shape.

A semantic object tracking method tracks general semantic objects with multiple non-rigid motion, disconnected components and multiple colors throughout a vector image sequence. The method accurately tracks these general semantic objects by spatially segmenting image regions from a current frame and then classifying these regions as to which semantic object they originated from in the previous frame. To classify each region, the method perform a region based motion estimation between each spatially segmented region and the previous frame to computed the position of a predicted region in the previous frame. The method then classifies each region in the current frame as being part of a semantic object based on which semantic object in the previous frame contains the most overlapping points of the predicted region. Using this method, each region in the current image is tracked to one semantic object from the previous frame, with no gaps or overlaps. The method propagates few or no errors because it projects regions into a frame where the semantic object boundaries are previously computed rather than trying to project and adjust a boundary in a frame where the object's boundary is unknown.

The invention provides a sparse and dense characteristic matching combined image registration method. According to the method, sparse characteristic matching is combined with dense characteristic matching to obtain a new mathematic model which includes two variables, nonrigid geometric transformation and disperse displacement flow field, wherein the nonrigid geometric transformation is applicable to sparse matching flows, and adjusted through introducing local linear constraint to be well posed; and the disperse displacement flow field is applicable to dense matching flows, a model similar to SIFT flows is used and meanwhile a belief propagation algorithm is adopted for optimized solution, and accurate pixel comparison matching can be obtained for a remote sensing image including nonrigid movements.

Treatment targets such as tumors or lesions, located within an anatomical region that undergoes motion (which may be periodic with cycle P), are dynamically tracked. A 4D mathematical model is established for the non-rigid motion and deformation of the anatomical region, from a set of CT or other 3D images. The 4D mathematical model relates the 3D locations of part(s) of the anatomical region with the targets being tracked, as a function of the position in time within P. Using fiducial-less non-rigid image registration between pre-operative DRRs and intra-operative x-ray images, the absolute position of the target and / or other part(s) of the anatomical region is determined. The cycle P is determined using motion sensors such as surface markers. The radiation beams are delivered using: 1) the results of non-rigid image registration; 2) the 4D model; and 3) the position in time within P.

Methods and apparatus for robust rigid and non-rigid motion tracking. An image, a next image, and a mask corresponding to an area in the image may be obtained. The area includes a plurality of points; each point indicates a location of the point in a position space and a color of the point in a color space. An iterative closest point algorithm may be applied that iteratively computes correspondences from a transformation and computes a new transformation from the correspondences. The algorithm tracks motion of the area in the image between the image and the next image. The algorithm matches points indicated by the mask to points in the next image in both position space and color space. An indication of an area in the next image that corresponds to the area in the image as tracked by the algorithm is generated.

The invention discloses a three-dimensional reconstruction method and system for combined optimization of the human body posture and the appearance model. The method comprises the following steps of performing depth map photographing on a human body to obtain a single depth image; converting the single depth image into a three-dimensional point cloud, and obtaining the three-dimensional point cloud, rebuilding model vertexes, and parameterizing matching point pairs among the human body model vertexes; according to the matching point pairs, building an energy function, and jointly solving non-rigid motion position transformation parameters and parameterization human body posture model parameters of each vertex on the rebuilt model; solving the energy function, and performing alignment on the rebuilt model and the three-dimensional point cloud according to the obtained result; updating and complementing the aligned model through the depth map to perform real-time human body dynamic three-dimensional rebuilding. The method can effectively improve the rebuilding real-time performance, robustness and accuracy and is high in expandability and simple and easy to implement.

The invention discloses a three-dimensional reconstruction method and device uniting rigid motion and non-rigid deformation. The method comprises the steps that photographing based on a depth camera is performed on a target object to obtain a single depth image; three-dimensional framework extraction is performed on a depth point cloud through a three-dimensional framework extraction algorithm; amatching point pair between a three-dimensional point cloud and each reconstruction model vertex is acquired; an energy function is established according to the matching point pairs and three-dimensional framework information, non-rigid motion position transformation parameters of each vertex on a reconstruction model are solved, and object framework parameters are optimized; GPU optimal solving is performed on the energy function to obtain non-rigid deformation of each surface vertex, and the reconstruction three-dimensional model in the previous frame is deformed according to the solving result, so that the deformed model is aligned to the three-dimensional point cloud in the current frame; and the updated model in the current frame is obtained to enter iteration of the next frame. Through the method, the instantaneity, robustness and accuracy of reconstruction can be effectively improved, extensibility is high, and the method is simple and easy to implement.

Dynamic tracking of moving targets. Treatment targets, such as tumors or lesions, located in an anatomical region that undergoes motion (which may be cyclic motion with cycles P) are dynamically tracked. Build a 4D mathematical model of non-rigid motion and deformation of an anatomical region from a set of CT or other 3D images. The 4D mathematical model relates the 3D position of the portion of the anatomical region to the target being tracked as a function of position in time in P. Determine the absolute position of targets and / or other parts of an anatomical region using fiducial-less, non-rigid image alignment between pre-operative DRR and intra-operative x-ray images. Cycle P is determined using a motion sensor such as a surface marker. Release the radiation beam using: 1) the result of non-rigid image alignment; 2) the 4D model; and 3) the position in time in P.

The invention discloses a skeleton tracking-based dynamic real-time three-dimensional reconstruction method and a skeleton tracking-based dynamic real-time three-dimensional reconstruction system. Themethod comprises the following steps: performing depth image shooting on a human body to acquire a single depth image; converting the single depth image into three-dimensional point cloud and acquiring a matched point pair between the three-dimensional point cloud and a reconstruction model vertex; establishing an energy function according to the matched points and jointly solving a non-rigid motion position conversion parameter and a human body skeleton motion parameter on each vertex on the reconstruction model; and solving the energy function, aligning the reconstruction model with the three-dimensional point cloud according to the solving result, and updating and complementing the aligned model by using the depth image to realize three-dimensional human body reconstruction. By the method, the human body can be shot by a depth image, so that the depth image is acquired to serve as system input information; furthermore, the function of performing real-time three-dimensional reconstruction is completed based on the depth image, and the method and the system are accurate and robust in solution and are simple and easy to realize.

The invention discloses a monocular RGB-D camera real-time face reconstruction method and device, wherein, that method comprise: detecting the position of the face feature point on the input face RGBimage through an advanced face feature point detection algorithm; Obtaining three-dimensional coordinates of each feature point of the current frame according to the position of the feature points ofthe face; Acquiring the current three-dimensional coordinates of each facial feature point on the key frame; The global rigid motion from the key frame to each frame is obtained according to the three-dimensional coordinates and the current three-dimensional coordinates, so as to obtain the rigid motion result; The result of rigid motion is used as the initialization of ICP to fine-tune the rigidmotion of human face. The rigid motion results are applied to the key frame model to update the TSDF representation of the model. This method effectively removes the depth of non-face region, eliminates the influence of rigid motion, and improves the accuracy of rigid motion estimation by using facial feature points.

The invention applies to the technical field of video processing and provides a video frame rate improvement method and device. The method comprises the steps of carrying out motion estimation on twoadjacent frames, thereby obtaining a motion vector of each pixel; segmenting a previous frame of image; clustering the motion vectors of all pixels, thereby obtaining a plurality of clustering centers; dividing the segmented areas into a plurality of areas according to the clustering centers, wherein the divided areas of which preset parameters are smaller than a first threshold are background areas, the divided areas of which the preset parameters are greater than or equal to the first threshold are motion areas, if the same object comprises the motion areas and the background areas and the preset parameters of the motion areas are smaller than a second threshold, the same object is a background object, otherwise, the same object is a motion object; finishing segmenting a foreground and abackground; carrying out image compensation; and inserting frames. According to a motion estimation algorithm based on sparse optical flow in the invention, on the basis of image segmentation, motionvector clustering is carried out on the pixels in the areas, the motion vectors of the foreground and the background are estimated, the motion compensation is realized, and the motion estimation accuracy of a nonrigid motion objet is improved.

The invention discloses a three-dimensional geometry and eigencomponent reconstruction method and device based on light and shadow optimization. The method comprises the following steps of: obtainingthree-dimensional color point cloud time series shooting by an RGBD camera; obtaining matching point pairs between a three-dimensional depth point cloud and vertexes of a reconstructed model, and obtaining a point pair set; establishing a joint energy function based on eigendecomposition according to the matching point pairs and a current view angle color image, and solving non-rigid motion position transformation parameters of each vertex on the reconstruction model; solving the energy function to obtain a deformation transformation matrix of vertexes of a surface model and eigencomponents ofeach item on the image; deforming the geometry of the previous frame three-dimensional model according to the solution result, complementing and updating the geometry and eigencomponents of the current frame model, and realizing three-dimensional geometry and eigencomponent reconstruction. The method can improve the track deformation robustness of a dynamic object in a sparse and single viewpointcondition, can obtain accurate solution, is low in requirement for devices, and has broad application prospect.

The invention discloses a 3D synthesis fitting method and system. The method comprises the following steps: 1) obtaining two photos of a garment; 2) identifying a contour line and key nodes of the photos of the garment; 3) calculating the same fitting points of the two pictures for fitting, and generating a garment virtual three-dimensional model of the basic model; 4) carrying out depth image shooting on the human body to obtain a single depth image; 5) converting the single depth image into a three-dimensional point cloud, and obtaining matching point pairs between the three-dimensional point cloud and the vertex of the reconstruction model and between the three-dimensional point cloud and the vertex of the parameterized human body model; 6) establishing an energy function according to the matching point pair, and solving a non-rigid motion position transformation parameter and a parameterized human body posture model parameter of each vertex on the reconstruction model together, and7) solving the energy function, so that the 3D composite image generated based on the body data and the clothes data can be used for experiencing fitting conveniently and quickly.

The invention discloses a dynamic human body three-dimensional reconstruction method, device and equipment and a medium, and relates to the technical fields of computer vision, computer graphics, three-dimensional reconstruction, virtual reality, augmented reality and the like. According to the specific implementation scheme, a human body three-dimensional model is reconstructed according to an RGB image and a depth image, collected based on a single view angle, of a target human body; posture estimation is performed on the target human body according to the RGB image; according to the estimated corresponding relation between the two-dimensional posture and the human body three-dimensional model and the estimated corresponding relation between the three-dimensional posture and the human body three-dimensional model, bone movement and non-rigid movement of the human body three-dimensional model are estimated, and therefore the robustness of dynamic human body three-dimensional reconstruction based on a single view angle is improved. The voxel fusion is performed on the human body three-dimensional model after movement according to the matching result of the semantic information of the target node before and after movement, so that the fusion of the wrong human body surface is avoided.

The embodiments relate to a method for producing a digital volume model of a body volume by a sensor device, which sensor device includes a plurality of radiation sensors, of which each produces a pixel value in a projection. In order to produce the volume model, a plurality of projections from different projection angles (a) are produced and the volume model is computed from sensor positions of the radiation sensors and pixel values of the radiation sensors. For at least one projection angle (a), the sensor positions are corrected by a respective correction vector for rigid motion compensation. The problem addressed is that of also compensating the non-rigid motion of the body volume (i.e., the deformation) in the computation of the volume model. This problem is solved in that, in order to correct the sensor positions, the projection surface provided by the totality of the radiation sensors is divided into a plurality of sub-surfaces and a separate correction vector is determined for each of the sub-surfaces independently of each other.

In a magnetic resonance (MR) navigator-based method and apparatus, MR data are acquired from a large joint of a patient, which is not modelable as a whole based on a single rigid body model. The field of view which the MR data are acquired is divided in a processor into multiple sub-sections, with each sub-section being modelable based on a rigid body model. MR navigator signals are acquired from each of the sub-sections, and these navigator signals are used in a motion tracking algorithm that is based on a rigid body model in order to generate a modeling result that tracks the movement of the overall joint within the field of view. The modeling result can be used for prospective or retrospective motion correction of the MR data.

The present invention discloses a method for generating an object space swept volume, relates to the fields of virtual simulation, computer aided manufacturing (CAM) and product lifecycle management (PLM), and in particular to a virtual model motion swept volume generation method. According to the method, a three-dimensional voxel method is adopted to obtain a swept volume, of an object, formed by motion transformation in a three-dimensional space, or called as an envelope volume, an envelope space, a swept space and a swept track. Compared with the current existing swept volume calculation method, the method provided by the present invention is applicable to rigid and nonrigid motion transformation of any complex three-dimensional shape, and can perform real-time three-dimensional display and surface reconstruction on a swept volume at each sampling moment during a motion transformation process. Besides, the method disclosed by the present invention further proposes accelerating a swept volume calculation process by adopting a massive parallel computing method, and the method is superior to the methods known in the art in the aspects of calculation efficiency, calculation accuracy and robustness of an algorithm.

The invention discloses a real-time reconstruction method and device for a hand and object interaction process, and the method comprises the following steps: carrying out the data segmentation of to-be-processed depth image data, obtaining a hand and object segmentation result, and obtaining gesture posture prediction data, and the hand and object segmentation result comprising object data and hand data; dividing the prediction data and the hand and object into constraint terms to optimize the unified energy function, so as to obtain a hand motion sequence and an object non-rigid motion sequence; and fusing the object data into the object model to obtain an object static model, and obtaining a real-time reconstruction result of the hand and object interaction process according to the handmotion sequence, the object non-rigid motion sequence and the object static model. The method can effectively solve the real-time reconstruction problem of hand and object interactive motion, and canobtain hand motion, object motion and object geometric models in real time.

The invention discloses a real-time human body dynamic three-dimensional reconstruction method and system driven by a clothes physical model, and the method comprises the steps: carrying out the photographing of a human body through a depth camera, and obtaining a color-depth image sequence; performing parameterized human body model matching, model surface non-rigid motion tracking and model updating based on depth fusion by using the input depth sequence; performing multi-view three-dimensional human body semantic segmentation based on a deep learning method by using the color image sequenceto obtain independent geometric models of different clothes, and forming a multi-layer human body model in combination with the parameterized human body model; according to the depth sequence input, performing human body motion tracking by using the double-layer human body surface to obtain corresponding human body posture information; conducting physical simulation on clothes movement according to human body movement, constructing depth fitting external force by combining input depth information, and restraining a clothes physical simulation result to be matched with depth input. The method is accurate and efficient in solution, and can realize human body dynamic three-dimensional reconstruction with clothes fine dynamic details.

The invention provides an object tracking method based on state fusion of multiple cell blocks, belonging to the technical field of visual object tracking. The object tracking method based on the state fusion of the multiple cell blocks can be used for effectively solving the non-rigid movement changes such as object rotation, object distortion, object scaling as well as a tracking problem under sheltering. The object tracking method based on the state fusion of the multiple cell blocks comprises the following steps of: selecting and determining a to-be-tracked target object from an initial image; automatically extracting by virtue of a moving target detecting method or manually appointing by virtue of a man-machine interaction method; setting a target cell block on a central point position which is randomly generated in a target object area; extracting a video image which is acquired by a camera and stored in a storage area under a real-time treatment condition, decomposing the video image which is used as a to-be-tracked video file into an image sequence consisting of a plurality of frames, extracting the frame image one by one as an input image; if the input image is null, ending the whole process; configuring the state of each cell block and determining the best configuration according to the corresponding target cell block. Target location is used for estimating the state of the existing target.

The invention provides a dense non-rigid motion structure algorithm based on a Grassmann manifold. The main content of the algorithm comprises Grassmann manifold construction, non-rigid motion structure formulation and three-dimensional reconstruction experiment and analysis. According to the process of the algorithm, first, Grassmann formulation representation is performed on a track space; second, Grassmann formulation representation is performed on a shape space; third, spatial-temporal formulation is performed, and bilinear optimization is performed; and last, a three-dimensional reconstruction experiment is performed, and control variable analysis and runtime analysis are performed. According to the algorithm, when a non-rigid motion structure is processed based on the Grassmann manifold, no prior template is needed, non-linear deformation with noise can be processed, and a final result based on a basic dataset can be provided; and the algorithm has better data extensibility and higher reconstruction precision, and comprehensive performance is optimal.

The invention discloses a magnetic resonance imaging rectification method, device and equipment based on 3D shape and appearance measurement, and the method comprises the steps: collecting the shape and appearance information of a head part and a body part; obtaining the rigid and non-rigid movement information of a human body according to the shape and appearance information of the head part andthe body part; synchronously collecting the magnetic resonance data; and carrying out the movement rectification of human body imaging according to the rigid and non-rigid movement information of thehuman body through combining with the magnetic resonance imaging state. The method can achieve the precise measurement of the rigid and non-rigid movement of the human body according to the shape andappearance information of the head part and the body part, so as to achieve the movement rectification of human body imaging. The method is simpler in operation, can effectively speed up the imaging,improves the imaging quality, and improves the measurement precision.

The invention discloses a real-time dynamic human body three-dimensional reconstruction method based on an inward looking-around multi-RGBD camera, and the method comprises the following steps: S1, building a multi-RGBD camera group, calibrating and collecting data, and obtaining a high-precision model; S2, extracting an embedded node and parameterizing the embedded node; S3, binding embedded nodes to the 3D skeleton, solving motion parameters and optimizing the number of the parameters, and propagating the parameters to the embedded nodes; s4, solving the non-rigid motion; and S5, carrying out real-time mapping. The method is low in implementation cost, simple and convenient to build, high in automation degree and free of manual intervention under the condition of having no special requirements for wearing, action posture, amplitude and speed of a user in a daily scene, and a real-time dynamic high-definition human body three-dimensional model is obtained through a series of algorithms based on a GPU. The method can be widely applied to the fields of remote holographic communication, remote teaching, entertainment and the like.

The invention discloses a steady non-rigid motion reconstruction method, and relates to the field of three-dimensional reconstruction in computer vision, in particular to a steady non-rigid motion reconstruction method. By use of the method, the defects in the prior art that the value of an orthogonal discrete cosine base number K needs to be continuously debugged to obtain a reconstruction resultwhich approaches to a truth value to reduce in a true non-rigid reconstruction application so as to be difficult in use can be solved, statistics can be carried out to obtain a reconstruction resultthrough the calculation of a plurality of rectification bases on a premise that the orthogonal discrete cosine base number K does not need to be preset or debugged, and the applicability and the stability of a non-rigid reconstruction algorithm based on a trajectory base are enhanced.