Smoothing by Example: Mesh Denoising by Averaging with Similarity-based Weights
In this research, we propose a new and powerful mesh/soup denoising
technique. Our approach is inspired by recent non-local image
denoising schemes and naturally extends bilateral mesh smoothing
methods. The main idea behind the approach is very simple.
A new position of vertex P of a noisy mesh is obtained as
a weighted mean of mesh vertices Q with nonlinear weights
reflecting a similarity between local neighborhoods of P and Q.
We demonstrated that our technique outperforms recent
state-of-the-art smoothing methods.
We also suggest a new approach for comparing different
mesh/soup denoising methods.
This research is collaborated with Prof. Dr. A. G. Belyaev and Prof. Dr. H.-P. Seidel. The paper was published in Proc. IEEE International Conference on Shape Modeling and Applications (SMI), pp. 38-44, June 14-16, 2006 at Matsushima, Japan.
C++ Code and Java3D Viewer
You can download the C++ code and Java3D Viewer which is TAR.GNUZIP file. After applied gunzip and tar -xf, please read README.txt and the following simple manual. Questions and Bug report are welcome to my e-mail: shin[_at_]riken.jp.
Copyright
All rights are reserved by Shin Yoshizawa. This C++ and Java source files are allowed
for only a primary user for research and educational purposes. Don't use
secondary use: copy, distribution, diversion, business purpose, and etc.. In no event shall the author be liable to any party for direct, indirect,
special, incidental, or consequential damage arising out of the use of this
program and source files.
(*): The functions of the class SvdSolve and GaussianRN are written in
"Numerical Recipes in C++", The Art of Scientific Computing, William H. Press
, Saul A. Teukolsky, William T. Vetterling, Brian P. Flannery. The copyrights of these functions of SvdSolve and Eigens class are remained to them.
Commercial and Business Use
If you would like to use this C++ and Java sources in your commercial software then please contact with me and RIKEN.
Simple Manual
How to use
- Program Organization: The program is constructed by two parts: C++ and Java3D Viewer. The C++ program do actual computations to smooth triangle soup: normal computations, RBF fitting, spatial-tonal normalized convolution for normal direction. The Java3D program can visualize the smoothed triangle soup, and can produce an input file for the C++ code easily. Of course you can use only the C++ code but you have to write the input and visualization parts by yourself. NonLocalCODE.tar.gz includes two directories: SmoothingCODE (C++ sources) and SmoothingViewer (Java sources).
- Compile:
- C++ code, by using attached Makefile, you may run "make" in SmoothingViewer directory. Then you will get the executable file "Smoothing". Please move Smoothing to the NonLocalCODE directory (% mv Smoothing ../SmoothingViewer/) if you would like to use the Java3D Viewer. Otherwise you can deal with the Smoothing file. You should use the later versions of g++ 3.3.5
- Java code, in SmoothingViewer directory you may run "javac -O MeshSmoothing.java". MeshSmoothing.java includes main() function. Then you will get the executable classes. You should use the later versions of JDK 1.4.
- Execute:
- C++ code, You can run Smoothing itself as (% ./Smoothing input.txt output.txt). The formats of input.txt and output.txt are described in File Formats section of this simple manual.
- Java3D Viewer,
You need JDK and Java3D API environment. You may run java (% java MeshSmoothing). It is better to specify max and initial heap sizes as (% java -mx600m -ms600m MeshSmoothing) to avoid OutofMemoryError. Please read the Quick Practice and Viewer Options sections to play the Java3D Viewer. Note that the executable file Smoothing should be located in same directory of MeshSmoothing class files.
- File Formats:
- Input for Java3D Viewer: The input triangle mesh have to be constructed by the PLY2 format.
- Input of C++ code (input.txt): Similar format of the PLY2 format but it is not an exact PLY2 format. 1st line is 4 integers: fscanf(in,"%d %d %d %d",&Stype,&SNum,&pickID,&opti); SNum is a number of iterations for Non-Local mean mesh filtering. opti represents
methods of standard deviation sigma1 for height kernel of Non-Local mean mesh filtering
- 1: standard deviation mesured by a picked vertex with 2sigma1 radius neighbor vertices where the picked vertex ID is given by pickID.
- 2: average standard deviation mesured by all vertices 2sigma1 radius neighbor vertices.
- Otherwise: Manual
Stype represents type of method.
- 0: Measuring similarity for a vertex whose ID is pickID.
- 1: Adding Gaussian noise where standard deviation is sigma1.
- Otherwise: Non-Local Mean mesh filtering.
2nd line is 3 doubles:fscanf(in,"%lf %lf %lf",&eta1,&eta2,&eta3); Here |e| is the average edge length. eta1|e| represents a standard deviation for height. 2eta2|e| gives a integration domain size. 2eta3|e| gives a similarity kernel size. See the paper. 3rd line is a number of vertices of an input mesh. 4th line is a number of triangles of an input mesh. After these four lines, there are vertex coordinates and face information as PLY2 format but there is no 3 for face line (only three vertex IDs).
- Outputs of C++ code:
- output.txt: If Stype:0 then
- 1st line is a number of vertices of an input mesh (Integer), say V. 2nd-EOF lines, there V-doubles which are similarities between a vertex (pickID) and all vertices where line number minus 1 corresponds a vertex ID of the similarities.
If Stype is not 0 then
- 1st line is a number of vertices of an input mesh (Integer), say V. 2nd lines, there V-three doubles: fprintf(in,"%lf %lf %lf\n",point[i]->x,point[i]->y,point[i]->z); where (point[i]->x,point[i]->y,point[i]->z) is a smoothed vertex whose ID is a line number minus 1.
- For Windows User: Please use MeshSmoothingWindows.java instead of MeshSmoothing.java because you may have Smoothing.exe instead of Smoothing. If you have Smoothing then it is OK to use MeshSmoothing.java on the Windows OS. Cygwin may help to compile my code on Windows.
Quick Practice
Note that the C++ executable file (Smoothing) and the Java class files (XXXX.class) should be in the same directory.
- Road to ready:
% gunzip NonLocalCODE.tar.gz
% tar -xf NonLocalCODE.tar
% cd ./NonLocalCODE/SmoothingCODE
% make
% mv Smoothing ../SmoothingViewer
% cd ../SmoothingViewer
% javac -O MeshSmoothing.java
% java -mx600m -ms600m MeshSmoothing
- Let's play the Java3D Viewer:
Click the top menu: file->load->load (poly2)
Select the file noisyfandisk.ply2.
In Smoothing Options,
Change "Similarity Standard Deviation (sigma1)" to "Average Height" and 1.0.
Set "Iteration:" to 4.
Change "Coloring: Flat Sheading with" to "Mean Curvature".
Press "Smoothing" button.
Viewer Options
- Menubar:
- file->exit program: to exit program.
- file->load:
- load (poly2): load an input PLY2 formated triangle mesh.
- load view: load a view point.
- load (poly2) for point[]: load a points from PLY2 to set a temporary.
- file->save:
- save (poly2): save the current triangle mesh by PLY2 format.
- save view: save a view point.
- save image (.jpg): capture a screen image in the canvas region via JPEG.
- Original Mesh->exist: On: visualize the original mesh. Off: unbind the original mesh.
- Original Mesh->Face: mesh via the flat shading.
- Original Mesh->WireFrame: visualize the mesh via wire frame model.
- Original Mesh->Points: visualize the mesh vertices.
- Original Mesh->2 Light: extra lighting on/off.
- Original Mesh->Back Face: visualize the back face of the mesh on/off.
- Original Mesh->Transparency: half-transparent rendering of the mesh on/off.
- Camera: select camera positions. Nate that it also changes orientation for mouse motions.
- Smoothing Options Panel:
- Smoothing: Apply NL-mean smoothing, similarity for picked vertex with an entire mesh, and Gaussian noise with sigma 1 standard deviation. It depends on the first Choice box options.
- set Positions: set current visualized vertices to init positions.
- Init Positions: initialize visualized vertices to "init positions".
- When we push "Smoothing" button, Choice box for
- Non Local Mean: applying our non local mean mesh filtering.
- Similarity: calculating similarities for all vertices associated with the picked vertex where similarity domain size is given in sigma1.
- Gaussian Noise sigma1: adding additive Gaussian noise with zero mean and sigma1 standard deviation.
- Iteration: an iteration number for non local mean mesh filtering. It should be (Integer) and greater than zero.
- Sigma = Eta|e| where |e| is an average edge length. The values which are equal to eta1, eta2 and eta3, then in the C++ program these values are automatically multiplied with |e| to sigma1, sigma2 and sigma3. We recommend {Eta1,Eta2} = {0.75,1.0} to {1.0,2.0}. If you choose large numbers for Eta1 and Eta2 then the program could be very slow.
- Sigma1: Similarity Standard Deviation. It should be (Double) and greater than 0.0.
- pick: On: visualise a picking sphere. You can pick a vertex on the mesh.
- Choice box for how to select sigma1:
- Manual: Manually choose Eta1.
- Picked Height: Sigma1 is given by a standard deviation of the picked vertex with 2Eta1|e| radius.
- Average Height: Sigma1 is given by an average standard deviation of all vertices with 2Eta1|e| radius.
- Where if the above Choice box is selected to "Manual", Eta1 which should be (Double) and greater than 0.0. If "Picked Height" or "Average Height" then Eta1 (this TextFiled) represents a similarity kernel size and then Sigma1 is automatically calculated by either a standard deviation of picked vertex with 2Eta1|e| size patch or an average standard deviation of all vertices with 2Eta1|e| size patch, respectively for "Picked Height" and "Average Height".
- Sigma2: Integral Domain Size. The value which you input is equal to Eta2 which should be (Double) and greater than 0.0.
- Sigma3: Similarity Kernel Size. The value which you input is equal to Eta3 which should be (Double) and greater than 0.0.
- Coloring: Flat Shading with
- Constant: a default color.
- Similarity: similarities between the picked vertex with all vertices.
- Mean Curvature: discrete mean curvature profiles.
- Method Noise: differences between the original (init position) and smoothed vertices. If they describe some features then it means that the method also eliminates the features. You should apply at least one smoothing.
- Picking sphere radius size.
- Threshold for the Z-score visualization.
- Mesh Processing Tool Panel: This panel is not directly related to the non local mean mesh filtering.
- Subdivide: apply subdivision to the mesh via Loop, Butterfly, or Linear. The subdivision results overwrite the original mesh. Loop N: Loop subdivision without projection to the limit position. Loop: Loop with limit position projection. TextField represents a number of subdivision iterations. Note that please do not apply "Subdivide" the large mesh or many times iterations because of my memory expensive implementation (you will have OutofMemoryError after a long time waiting).
- GH-Decimation: apply Garlnd-Heckbert mesh simplification. "P:" is a reduction ratio (percentage). The decimation result overwrites the original mesh.
Denoising Examples
Log
May 25, 2011: Fix a bug (initialization, no effect for UNIX/Linux) in IDList.h for the latest Windows users, thanks to Yuee Liu.
April 12, 2011: Fix small memory leak in IDSet.cxx, thanks to Anuwat Dechvijankit.
April 12, 2011: Fix a bug for standard deviation calculation.
April 18, 2008: Move to RIKEN.
April 18, 2008: Solve g++ (GCC) version 4 and com.sun.image.codec problems.
March 24, 2006: Upload.
Sep 5, 2006: fix a bug for allocating points (thanks for Ding Sheng).