generic-library/opencv
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Removed Sphinx documentation files

Browse Source
This commit is contained in:
Maksim Shabunin
2014-12-24 18:37:57 +03:00
parent 61991a3330
commit d01bedbc61
338 changed files with 0 additions and 73040 deletions
Download Patch File Download Diff File Expand all files Collapse all files

70
doc/py_tutorials/py_photo/py_inpainting/py_inpainting.rst
View File

@@ -1,70 +0,0 @@
.. _inpainting:
Image Inpainting
**********************
Goal
======
In this chapter,
* We will learn how to remove small noises, strokes etc in old photographs by a method called inpainting
* We will see inpainting functionalities in OpenCV.
Basics
===========
Most of you will have some old degraded photos at your home with some black spots, some strokes etc on it. Have you ever thought of restoring it back? We can't simply erase them in a paint tool because it is will simply replace black structures with white structures which is of no use. In these cases, a technique called image inpainting is used. The basic idea is simple: Replace those bad marks with its neighbouring pixels so that it looks like the neigbourhood. Consider the image shown below (taken from `Wikipedia <http://en.wikipedia.org/wiki/Inpainting>`_):
.. image:: images/inpaint_basics.jpg
:alt: Inpainting example
:align: center
Several algorithms were designed for this purpose and OpenCV provides two of them. Both can be accessed by the same function, **cv2.inpaint()**
First algorithm is based on the paper **"An Image Inpainting Technique Based on the Fast Marching Method"** by Alexandru Telea in 2004. It is based on Fast Marching Method. Consider a region in the image to be inpainted. Algorithm starts from the boundary of this region and goes inside the region gradually filling everything in the boundary first. It takes a small neighbourhood around the pixel on the neigbourhood to be inpainted. This pixel is replaced by normalized weighted sum of all the known pixels in the neigbourhood. Selection of the weights is an important matter. More weightage is given to those pixels lying near to the point, near to the normal of the boundary and those lying on the boundary contours. Once a pixel is inpainted, it moves to next nearest pixel using Fast Marching Method. FMM ensures those pixels near the known pixels are inpainted first, so that it just works like a manual heuristic operation. This algorithm is enabled by using the flag, ``cv2.INPAINT_TELEA``.
Second algorithm is based on the paper **"Navier-Stokes, Fluid Dynamics, and Image and Video Inpainting"** by Bertalmio, Marcelo, Andrea L. Bertozzi, and Guillermo Sapiro in 2001. This algorithm is based on fluid dynamics and utilizes partial differential equations. Basic principle is heurisitic. It first travels along the edges from known regions to unknown regions (because edges are meant to be continuous). It continues isophotes (lines joining points with same intensity, just like contours joins points with same elevation) while matching gradient vectors at the boundary of the inpainting region. For this, some methods from fluid dynamics are used. Once they are obtained, color is filled to reduce minimum variance in that area. This algorithm is enabled by using the flag, ``cv2.INPAINT_NS``.
Code
===========
We need to create a mask of same size as that of input image, where non-zero pixels corresponds to the area which is to be inpainted. Everything else is simple. My image is degraded with some black strokes (I added manually). I created a corresponding strokes with Paint tool.
::
import numpy as np
import cv2
img = cv2.imread('messi_2.jpg')
mask = cv2.imread('mask2.png',0)
dst = cv2.inpaint(img,mask,3,cv2.INPAINT_TELEA)
cv2.imshow('dst',dst)
cv2.waitKey(0)
cv2.destroyAllWindows()
See the result below. First image shows degraded input. Second image is the mask. Third image is the result of first algorithm and last image is the result of second algorithm.
.. image:: images/inpaint_result.jpg
:alt: Inpainting result
:align: center
Additional Resources
=========================
#. Bertalmio, Marcelo, Andrea L. Bertozzi, and Guillermo Sapiro. "Navier-stokes, fluid dynamics, and image and video inpainting." In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, vol. 1, pp. I-355. IEEE, 2001.
#. Telea, Alexandru. "An image inpainting technique based on the fast marching method." Journal of graphics tools 9.1 (2004): 23-34.
Exercises
================
#. OpenCV comes with an interactive sample on inpainting, ``samples/python2/inpaint.py``, try it.
#. A few months ago, I watched a video on `Content-Aware Fill <http://www.youtube.com/watch?v=ZtoUiplKa2A>`_, an advanced inpainting technique used in Adobe Photoshop. On further search, I was able to find that same technique is already there in GIMP with different name, "Resynthesizer" (You need to install separate plugin). I am sure you will enjoy the technique.

139
doc/py_tutorials/py_photo/py_non_local_means/py_non_local_means.rst
View File

@@ -1,139 +0,0 @@
.. _non_local_means:
Image Denoising
************************
Goal
=========
In this chapter,
* You will learn about Non-local Means Denoising algorithm to remove noise in the image.
* You will see different functions like **cv2.fastNlMeansDenoising()**, **cv2.fastNlMeansDenoisingColored()** etc.
Theory
=========
In earlier chapters, we have seen many image smoothing techniques like Gaussian Blurring, Median Blurring etc and they were good to some extent in removing small quantities of noise. In those techniques, we took a small neighbourhood around a pixel and did some operations like gaussian weighted average, median of the values etc to replace the central element. In short, noise removal at a pixel was local to its neighbourhood.
There is a property of noise. Noise is generally considered to be a random variable with zero mean. Consider a noisy pixel, :math:`p = p_0 + n` where :math:`p_0` is the true value of pixel and :math:`n` is the noise in that pixel. You can take large number of same pixels (say :math:`N`) from different images and computes their average. Ideally, you should get :math:`p = p_0` since mean of noise is zero.
You can verify it yourself by a simple setup. Hold a static camera to a certain location for a couple of seconds. This will give you plenty of frames, or a lot of images of the same scene. Then write a piece of code to find the average of all the frames in the video (This should be too simple for you now ). Compare the final result and first frame. You can see reduction in noise. Unfortunately this simple method is not robust to camera and scene motions. Also often there is only one noisy image available.
So idea is simple, we need a set of similar images to average out the noise. Consider a small window (say 5x5 window) in the image. Chance is large that the same patch may be somewhere else in the image. Sometimes in a small neigbourhood around it. What about using these similar patches together and find their average? For that particular window, that is fine. See an example image below:
.. image:: images/nlm_patch.jpg
:alt: Similar patches
:align: center
The blue patches in the image looks the similar. Green patches looks similar. So we take a pixel, take small window around it, search for similar windows in the image, average all the windows and replace the pixel with the result we got. This method is Non-Local Means Denoising. It takes more time compared to blurring techniques we saw earlier, but its result is very good. More details and online demo can be found at first link in additional resources.
For color images, image is converted to CIELAB colorspace and then it separately denoise L and AB components.
Image Denoising in OpenCV
===================================
OpenCV provides four variations of this technique.
#. **cv2.fastNlMeansDenoising()** - works with a single grayscale images
#. **cv2.fastNlMeansDenoisingColored()** - works with a color image.
#. **cv2.fastNlMeansDenoisingMulti()** - works with image sequence captured in short period of time (grayscale images)
#. **cv2.fastNlMeansDenoisingColoredMulti()** - same as above, but for color images.
Common arguments are:
* h : parameter deciding filter strength. Higher h value removes noise better, but removes details of image also. (10 is ok)
* hForColorComponents : same as h, but for color images only. (normally same as h)
* templateWindowSize : should be odd. (recommended 7)
* searchWindowSize : should be odd. (recommended 21)
Please visit first link in additional resources for more details on these parameters.
We will demonstrate 2 and 3 here. Rest is left for you.
1. cv2.fastNlMeansDenoisingColored()
------------------------------------------
As mentioned above it is used to remove noise from color images. (Noise is expected to be gaussian). See the example below:
::
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('die.png')
dst = cv2.fastNlMeansDenoisingColored(img,None,10,10,7,21)
plt.subplot(121),plt.imshow(img)
plt.subplot(122),plt.imshow(dst)
plt.show()
Below is a zoomed version of result. My input image has a gaussian noise of :math:`\sigma = 25`. See the result:
.. image:: images/nlm_result1.jpg
:alt: Result of denoising
:align: center
2. cv2.fastNlMeansDenoisingMulti()
------------------------------------------
Now we will apply the same method to a video. The first argument is the list of noisy frames. Second argument `imgToDenoiseIndex` specifies which frame we need to denoise, for that we pass the index of frame in our input list. Third is the `temporalWindowSize` which specifies the number of nearby frames to be used for denoising. It should be odd. In that case, a total of `temporalWindowSize` frames are used where central frame is the frame to be denoised. For example, you passed a list of 5 frames as input. Let `imgToDenoiseIndex = 2` and `temporalWindowSize = 3`. Then frame-1, frame-2 and frame-3 are used to denoise frame-2. Let's see an example.
::
import numpy as np
import cv2
from matplotlib import pyplot as plt
cap = cv2.VideoCapture('vtest.avi')
# create a list of first 5 frames
img = [cap.read()[1] for i in xrange(5)]
# convert all to grayscale
gray = [cv2.cvtColor(i, cv2.COLOR_BGR2GRAY) for i in img]
# convert all to float64
gray = [np.float64(i) for i in gray]
# create a noise of variance 25
noise = np.random.randn(*gray[1].shape)*10
# Add this noise to images
noisy = [i+noise for i in gray]
# Convert back to uint8
noisy = [np.uint8(np.clip(i,0,255)) for i in noisy]
# Denoise 3rd frame considering all the 5 frames
dst = cv2.fastNlMeansDenoisingMulti(noisy, 2, 5, None, 4, 7, 35)
plt.subplot(131),plt.imshow(gray[2],'gray')
plt.subplot(132),plt.imshow(noisy[2],'gray')
plt.subplot(133),plt.imshow(dst,'gray')
plt.show()
Below image shows a zoomed version of the result we got:
.. image:: images/nlm_multi.jpg
:alt: Denoising a frame
:align: center
It takes considerable amount of time for computation. In the result, first image is the original frame, second is the noisy one, third is the denoised image.
Additional Resources
========================
#. http://www.ipol.im/pub/art/2011/bcm_nlm/ (It has the details, online demo etc. Highly recommended to visit. Our test image is generated from this link)
#. `Online course at coursera <https://www.coursera.org/course/images>`_ (First image taken from here)
Exercises
============

49
doc/py_tutorials/py_photo/py_table_of_contents_photo/py_table_of_contents_photo.rst
View File

@@ -1,49 +0,0 @@
.. _PY_Table-Of-Content-Photo:
Computational Photography
--------------------------------
Here you will learn different OpenCV functionalities related to Computational Photography like image denoising etc.
* :ref:`non_local_means`
.. tabularcolumns:: m{100pt} m{300pt}
.. cssclass:: toctableopencv
=========== ======================================================
|photo_1| See a good technique to remove noises in images called Non-Local Means Denoising
=========== ======================================================
.. |photo_1| image:: images/nlm_icon.jpg
:height: 90pt
:width: 90pt
* :ref:`inpainting`
.. tabularcolumns:: m{100pt} m{300pt}
.. cssclass:: toctableopencv
=========== ======================================================
|photo_2| Do you have a old degraded photo with many black spots and strokes on it? Take it. Let's try to restore them with a technique called image inpainting.
=========== ======================================================
.. |photo_2| image:: images/inpainticon.jpg
:height: 90pt
:width: 90pt
.. raw:: latex
\pagebreak
.. We use a custom table of content format and as the table of content only informs Sphinx about the hierarchy of the files, no need to show it.
.. toctree::
:hidden:
../py_non_local_means/py_non_local_means
../py_inpainting/py_inpainting