Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Flower 3D reconstruction method based on ORB and U-net

A 3D reconstruction and flower technology, applied in the field of 3D reconstruction, can solve problems such as inability to reconstruct, hollow reconstruction model, easy loss of camera pose, etc., to achieve the effect of saving time and improving accuracy

Active Publication Date: 2019-01-15
QINGDAO RES INST OF BEIHANG UNIV
View PDF3 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003]The current 3D reconstruction method, for the scanning of heavily occluded objects such as flowers, due to the TOF or structured light technology adopted by the kinect camera, the occluded part has no depth data and unable to rebuild
In practice, it is also not feasible to ensure that the scan can cover all points on the flower surface (flower objects are often placed in the corner and the bottom of the flower cannot be scanned), thus resulting in the use of existing 3D reconstruction methods. a large number of voids
At the same time, because the kinectfusion method only performs pose estimation by registration between the current frame and the previous frame, there is no global pose optimization, loop detection and loop closure, which makes the pose estimation for the depth map very inaccurate during 3D reconstruction. At the same time The robustness of the fast movement of the camera itself is very low, and only slow movement modeling can be performed, which is easy to lose the camera pose

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Flower 3D reconstruction method based on ORB and U-net
  • Flower 3D reconstruction method based on ORB and U-net

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] The present invention utilizes the U-net convolutional neural network to train the TSDF voxel model data obtained by scanning the depth camera, completes the completion of the 3D model through the completion of the TSDF voxel data, and innovatively uses U- The combination of net network and 3D reconstruction is used to solve the problem of 3D reconstruction of flowers with serious occlusion. Due to the small amount of data in the 3D data sets of flowers, U-net can show that when targeting smaller training data sets Great advantage. The data set used in the training comes from a large number of flower models modeled in Maya, that is, groundtruth. By simulating the scanning trajectory of the camera, part of the TSDF data of the flower model can be obtained as the input of the network. At the same time, in the 3D reconstruction part of the depth camera, global pose optimization, loop detection and loop closure are added on the basis of kinectfusion, and a relocation soluti...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a method based on ORB and U-NET, including ORB-based 3D reconstruction of depth camera and u-Net. In the part of 3D reconstruction, the global posture optimization, loop detection and relocation are added to kinectfusion, which greatly improves the accuracy of the camera posture estimation and can get the high-quality flower 3D reconstruction model. Three-dimensional reconstruction is divided into three parts, because the three-dimensional data set of the flower objects is small, so the three-dimensional reconstruction is based on U-NET network structure, can obtain thebetter effect under the condition of having the smaller data set. Deep learning method can fill a large number of holes in the traditional dense slam method. The end-to-end method is implemented in the network to fill the holes, which can save a lot of time to fill the holes.

Description

technical field [0001] The invention relates to a three-dimensional reconstruction technology, in particular to a flower three-dimensional reconstruction method. Background technique [0002] In the current process of 3D reconstruction technology, the 3D reconstruction of small scene objects is mainly based on the dense slam technology of kinectfusion. By using kinect and other depth cameras to collect depth images around scene objects, and then process the depth images to obtain 3D reconstruction of small scene objects Model. The technical process of Kinectfusion is as follows: First, use the kinect camera to collect depth images around small scene objects, use the kinect camera internal reference to convert the original 2D depth image into a 3D point cloud in the camera coordinate system and calculate the normal vector; use ICP to iterate the minimization point The energy function to the plane is used to calculate the camera pose of the current frame; the 3D point cloud i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06T17/00G06T7/73
CPCG06T7/74G06T17/00G06T2207/20084G06T2207/20081G06T2207/30244
Inventor 齐越刘麟祺孙涛
Owner QINGDAO RES INST OF BEIHANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products