updated demos and tutorial regarding the Retina class transfer to bioinspired module.

doc/CMakeLists.txt

@@ -17,7 +17,7 @@ if(BUILD_DOCS AND HAVE_SPHINX)
   set(OPTIONAL_DOC_LIST "")
 
 
-  set(OPENCV2_BASE_MODULES core imgproc highgui video calib3d features2d objdetect ml flann gpu photo stitching nonfree contrib legacy)
+  set(OPENCV2_BASE_MODULES core imgproc highgui video calib3d features2d objdetect ml flann gpu photo stitching nonfree contrib legacy bioinspired)
 
   # build lists of modules to be documented
   set(OPENCV2_MODULES "")
@@ -122,4 +122,4 @@ if(BUILD_DOCS AND HAVE_SPHINX)
     install(FILES "${f}" DESTINATION "${OPENCV_DOC_INSTALL_PATH}" OPTIONAL)
   endforeach()
 
-endif()
+endif()

doc/tutorials/tutorials.rst

@@ -156,17 +156,17 @@ As always, we would be happy to hear your comments and receive your contribution
                  :width:  80pt
                  :alt: gpu icon
 
-*  :ref:`Table-Of-Content-Contrib`
+*  :ref:`Table-Of-Content-Bioinspired`
 
    .. tabularcolumns:: m{100pt} m{300pt}
    .. cssclass:: toctableopencv
 
-   =========== =======================================================
+   ============= =======================================================
-   |Contrib|       Discover additional contribution to OpenCV.
+   |Bioinspired|       Algorithms inspired from biological models.
 
-   =========== =======================================================
+   ============= =======================================================
 
-   .. |Contrib| image:: images/retina.jpg
+   .. |Bioinspired| image:: images/retina.jpg
                  :height: 80pt
                  :width:  80pt
                  :alt: gpu icon
@@ -219,6 +219,6 @@ As always, we would be happy to hear your comments and receive your contribution
    objdetect/table_of_content_objdetect/table_of_content_objdetect
    ml/table_of_content_ml/table_of_content_ml
    gpu/table_of_content_gpu/table_of_content_gpu
-   contrib/table_of_content_contrib/table_of_content_contrib
+   bioinspired/table_of_content_bioinspired/table_of_content_bioinspired
    ios/table_of_content_ios/table_of_content_ios
    general/table_of_content_general/table_of_content_general

include/opencv2/opencv.hpp

@@ -52,6 +52,7 @@
 #include "opencv2/calib3d.hpp"
 #include "opencv2/highgui.hpp"
 #include "opencv2/contrib.hpp"
+#include "opencv2/bioinspired.hpp"
 #include "opencv2/ml.hpp"
 
 #endif

modules/bioinspired/include/opencv2/bioinspired.hpp

@@ -47,5 +47,5 @@
 #include "opencv2/bioinspired/retina.hpp"
 #include "opencv2/bioinspired/retinafasttonemapping.hpp"
 
-using namespace cv::hvstools;
+using namespace cv::hvstools; // used to avoid complex namespace inclusions cv::hvstools::Retina => cv::Retina preferred
 #endif

modules/contrib/doc/retina/index.rst

@@ -61,9 +61,12 @@ Here is an overview of the abstract Retina interface, allocate one instance with
     // parameters setup instance
     struct RetinaParameters; // this class is detailled later
 
-    // main method for input frame processing
+    // main method for input frame processing (all use method, can also perform High Dynamic Range tone mapping)
     void run (InputArray inputImage);
 
+    // specific method aiming at correcting luminance only (faster High Dynamic Range tone mapping)
+    void applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage)
+
     // output buffers retreival methods
     // -> foveal color vision details channel with luminance and noise correction
     void getParvo (OutputArray retinaOutput_parvo);
@@ -138,6 +141,10 @@ This retina filter code includes the research contributions of phd/research coll
 
 * take a look at *imagelogpolprojection.hpp* to discover retina spatial log sampling which originates from Barthelemy Durette phd with Jeanny Herault. A Retina / V1 cortex projection is also proposed and originates from Jeanny's discussions. More informations in the above cited Jeanny Heraults's book.
 
+* Meylan&al work on HDR tone mapping that is implemented as a specific method within the model :
+
+.. [Meylan2007] L. Meylan , D. Alleysson, S. Susstrunk, "A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images", Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816
+
 Demos and experiments !
 =======================
 
@@ -161,11 +168,13 @@ Take a look at the provided C++ examples provided with OpenCV :
 
    Then, take a HDR image using bracketing with your camera and generate an OpenEXR image and then process it using the demo.
 
-   Typical use, supposing that you have the OpenEXR image *memorial.exr* (present in the samples/cpp/ folder)
+   Typical use, supposing that you have the OpenEXR image such as *memorial.exr* (present in the samples/cpp/ folder)
 
-   **OpenCVReleaseFolder/bin/OpenEXRimages_HighDynamicRange_Retina_toneMapping memorial.exr**
+   **OpenCVReleaseFolder/bin/OpenEXRimages_HighDynamicRange_Retina_toneMapping memorial.exr [optionnal: 'fast']**
 
       Note that some sliders are made available to allow you to play with luminance compression.
+      
+      If not using the 'fast' option, then, tone mapping is performed using the full retina model [Benoit2010]_. It includes spectral whitening that allows luminance energy to be reduced. When using the 'fast' option, then, a simpler method is used, it is an adaptation of the algorithm presented in [Meylan2007]_. This method gives also good results and is faster to process but it sometimes requires some more parameters adjustement.
 
 
 Methods description
@@ -275,7 +284,7 @@ Retina::printSetup
 
     Outputs a string showing the used parameters setup
 
-    :return: a string which contains formatted parameters information
+    :return: a string which contains formated parameters information
 
 Retina::run
 +++++++++++
@@ -286,6 +295,17 @@ Retina::run
 
     :param inputImage: the input Mat image to be processed, can be gray level or BGR coded in any format (from 8bit to 16bits)
 
+Retina::applyFastToneMapping
+++++++++++++++++++++++++++++
+
+.. ocv:function:: void Retina::applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage)
+
+    Method which processes an image in the aim to correct its luminance : correct backlight problems, enhance details in shadows. This method is designed to perform High Dynamic Range image tone mapping (compress >8bit/pixel images to 8bit/pixel). This is a simplified version of the Retina Parvocellular model (simplified version of the run/getParvo methods call) since it does not include the spatio-temporal filter modelling the Outer Plexiform Layer of the retina that performs spectral whitening and many other stuff. However, it works great for tone mapping and in a faster way.
+Check the demos and experiments section to see examples and the way to perform tone mapping using the original retina model and the method.
+
+    :param inputImage: the input image to process (should be coded in float format : CV_32F, CV_32FC1, CV832F_C3, CV832F_C4, the 4th channel won't be considered).
+    :param outputToneMappedImage: the output 8bit/channel tone mapped image (CV_8U or CV_8UC3 format).
+
 Retina::setColorSaturation
 ++++++++++++++++++++++++++
 

modules/contrib/include/opencv2/contrib.hpp

@@ -633,7 +633,6 @@ CV_EXPORTS_W void applyColorMap(InputArray src, OutputArray dst, int colormap);
 CV_EXPORTS bool initModule_contrib();
 }
 
-#include "opencv2/contrib/retina.hpp"
 #include "opencv2/contrib/openfabmap.hpp"
 
 #endif

samples/cpp/CMakeLists.txt

@@ -5,7 +5,7 @@
 
 SET(OPENCV_CPP_SAMPLES_REQUIRED_DEPS opencv_core opencv_flann opencv_imgproc
     opencv_highgui opencv_ml opencv_video opencv_objdetect opencv_photo opencv_nonfree opencv_softcascade
-    opencv_features2d opencv_calib3d opencv_legacy opencv_contrib opencv_stitching opencv_videostab)
+    opencv_features2d opencv_calib3d opencv_legacy opencv_contrib opencv_stitching opencv_videostab opencv_bioinspired)
 
 ocv_check_dependencies(${OPENCV_CPP_SAMPLES_REQUIRED_DEPS})
 

samples/cpp/OpenEXRimages_HighDynamicRange_Retina_toneMapping.cpp

@@ -10,8 +10,9 @@
 #include <iostream>
 #include <cstring>
 
-#include "opencv2/contrib.hpp"
+#include "opencv2/bioinspired.hpp" // retina based algorithms
-#include "opencv2/highgui.hpp"
+#include "opencv2/imgproc.hpp" // cvCvtcolor function
+#include "opencv2/highgui.hpp" // display
 
 static void help(std::string errorMessage)
 {
@@ -127,7 +128,7 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
      normalize(imageInputRescaled, imageInputRescaled, 0.0, 255.0, cv::NORM_MINMAX);
  }
 
- cv::Ptr<cv::Retina> retina;
+ cv::Ptr<Retina> retina;
  int retinaHcellsGain;
  int localAdaptation_photoreceptors, localAdaptation_Gcells;
  static void callBack_updateRetinaParams(int, void*)
@@ -175,6 +176,12 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
      }
 
      bool useLogSampling = !strcmp(argv[argc-1], "log"); // check if user wants retina log sampling processing
+     int chosenMethod=0;
+     if (!strcmp(argv[argc-1], "fast"))
+     {
+         chosenMethod=1;
+         std::cout<<"Using fast method (no spectral whithning), adaptation of Meylan&al 2008 method"<<std::endl;
+     }
 
      std::string inputImageName=argv[1];
 
@@ -210,17 +217,22 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
           * -> if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision)
           */
          if (useLogSampling)
-                {
+         {
-                     retina = cv::createRetina(inputImage.size(),true, cv::RETINA_COLOR_BAYER, true, 2.0, 10.0);
+             retina = createRetina(inputImage.size(),true, RETINA_COLOR_BAYER, true, 2.0, 10.0);
                  }
          else// -> else allocate "classical" retina :
-             retina = cv::createRetina(inputImage.size());
+             retina = createRetina(inputImage.size());
 
-        // save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup"
+         // create a fast retina tone mapper (Meyla&al algorithm)
-        retina->write("RetinaDefaultParameters.xml");
+         std::cout<<"Allocating fast tone mapper..."<<std::endl;
+         //cv::Ptr<cv::RetinaFastToneMapping> fastToneMapper=createRetinaFastToneMapping(inputImage.size());
+         std::cout<<"Fast tone mapper allocated"<<std::endl;
+         
+         // save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup"
+         retina->write("RetinaDefaultParameters.xml");
 
-                 // desactivate Magnocellular pathway processing (motion information extraction) since it is not usefull here
+         // desactivate Magnocellular pathway processing (motion information extraction) since it is not usefull here
-                 retina->activateMovingContoursProcessing(false);
+         retina->activateMovingContoursProcessing(false);
 
          // declare retina output buffers
          cv::Mat retinaOutput_parvo;
@@ -230,20 +242,19 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
          histogramClippingValue=0; // default value... updated with interface slider
          //inputRescaleMat = inputImage;
          //outputRescaleMat = imageInputRescaled;
-         cv::namedWindow("Retina input image (with cut edges histogram for basic pixels error avoidance)",1);
+         cv::namedWindow("Processing configuration",1);
-         cv::createTrackbar("histogram edges clipping limit", "Retina input image (with cut edges histogram for basic pixels error avoidance)",&histogramClippingValue,50,callBack_rescaleGrayLevelMat);
+         cv::createTrackbar("histogram edges clipping limit", "Processing configuration",&histogramClippingValue,50,callBack_rescaleGrayLevelMat);
 
-         cv::namedWindow("Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", 1);
          colorSaturationFactor=3;
-         cv::createTrackbar("Color saturation", "Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", &colorSaturationFactor,5,callback_saturateColors);
+         cv::createTrackbar("Color saturation", "Processing configuration", &colorSaturationFactor,5,callback_saturateColors);
 
          retinaHcellsGain=40;
-         cv::createTrackbar("Hcells gain", "Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping",&retinaHcellsGain,100,callBack_updateRetinaParams);
+         cv::createTrackbar("Hcells gain", "Processing configuration",&retinaHcellsGain,100,callBack_updateRetinaParams);
 
          localAdaptation_photoreceptors=197;
          localAdaptation_Gcells=190;
-         cv::createTrackbar("Ph sensitivity", "Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", &localAdaptation_photoreceptors,199,callBack_updateRetinaParams);
+         cv::createTrackbar("Ph sensitivity", "Processing configuration", &localAdaptation_photoreceptors,199,callBack_updateRetinaParams);
-         cv::createTrackbar("Gcells sensitivity", "Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", &localAdaptation_Gcells,199,callBack_updateRetinaParams);
+         cv::createTrackbar("Gcells sensitivity", "Processing configuration", &localAdaptation_Gcells,199,callBack_updateRetinaParams);
 
 
          /////////////////////////////////////////////
@@ -257,11 +268,28 @@ static void drawPlot(const cv::Mat curve, const std::string figureTitle, const i
          while(continueProcessing)
          {
              // run retina filter
-             retina->run(imageInputRescaled);
+             if (!chosenMethod)
-             // Retrieve and display retina output
+             {
-             retina->getParvo(retinaOutput_parvo);
+                 retina->run(imageInputRescaled);
-             cv::imshow("Retina input image (with cut edges histogram for basic pixels error avoidance)", imageInputRescaled/255.0);
+                 // Retrieve and display retina output
-             cv::imshow("Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", retinaOutput_parvo);
+                 retina->getParvo(retinaOutput_parvo);
+                 cv::imshow("Retina input image (with cut edges histogram for basic pixels error avoidance)", imageInputRescaled/255.0);
+                 cv::imshow("Retina Parvocellular pathway output : 16bit=>8bit image retina tonemapping", retinaOutput_parvo);
+                 cv::imwrite("HDRinput.jpg",imageInputRescaled/255.0);
+                 cv::imwrite("RetinaToneMapping.jpg",retinaOutput_parvo);
+             }
+             else
+             {
+                 // apply the simplified hdr tone mapping method
+                 cv::Mat fastToneMappingOutput;
+                 retina->applyFastToneMapping(imageInputRescaled, fastToneMappingOutput);
+                 cv::imshow("Retina fast tone mapping output : 16bit=>8bit image retina tonemapping", fastToneMappingOutput);
+             }
+             /*cv::Mat fastToneMappingOutput_specificObject;
+             fastToneMapper->setup(3.f, 1.5f, 1.f);
+             fastToneMapper->applyFastToneMapping(imageInputRescaled, fastToneMappingOutput_specificObject);
+             cv::imshow("### Retina fast tone mapping output : 16bit=>8bit image retina tonemapping", fastToneMappingOutput_specificObject);
+*/
              cv::waitKey(10);
          }
      }catch(cv::Exception e)

samples/cpp/OpenEXRimages_HighDynamicRange_Retina_toneMapping_video.cpp

@@ -14,8 +14,9 @@
 #include <stdio.h>
 #include <cstring>
 
-#include "opencv2/contrib.hpp"
+#include "opencv2/bioinspired.hpp" // retina based algorithms
-#include "opencv2/highgui.hpp"
+#include "opencv2/imgproc.hpp" // cvCvtcolor function
+#include "opencv2/highgui.hpp" // display
 
 static void help(std::string errorMessage)
 {
@@ -160,7 +161,7 @@ static void rescaleGrayLevelMat(const cv::Mat &inputMat, cv::Mat &outputMat, con
 
  }
 
- cv::Ptr<cv::Retina> retina;
+ cv::Ptr<Retina> retina;
  int retinaHcellsGain;
  int localAdaptation_photoreceptors, localAdaptation_Gcells;
  static void callBack_updateRetinaParams(int, void*)
@@ -280,10 +281,10 @@ static void loadNewFrame(const std::string filenamePrototype, const int currentF
           */
          if (useLogSampling)
                 {
-                     retina = cv::createRetina(inputImage.size(),true, cv::RETINA_COLOR_BAYER, true, 2.0, 10.0);
+                     retina = createRetina(inputImage.size(),true, RETINA_COLOR_BAYER, true, 2.0, 10.0);
                  }
          else// -> else allocate "classical" retina :
-             retina = cv::createRetina(inputImage.size());
+             retina = createRetina(inputImage.size());
 
         // save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup"
         retina->write("RetinaDefaultParameters.xml");

samples/cpp/retinaDemo.cpp

@@ -9,7 +9,7 @@
 #include <iostream>
 #include <cstring>
 
-#include "opencv2/contrib.hpp"
+#include "opencv2/bioinspired.hpp"
 #include "opencv2/highgui.hpp"
 
 static void help(std::string errorMessage)
@@ -106,15 +106,15 @@ int main(int argc, char* argv[]) {
     try
     {
         // create a retina instance with default parameters setup, uncomment the initialisation you wanna test
-        cv::Ptr<cv::Retina> myRetina;
+        cv::Ptr<Retina> myRetina;
 
         // if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision)
         if (useLogSampling)
                 {
-                        myRetina = cv::createRetina(inputFrame.size(), true, cv::RETINA_COLOR_BAYER, true, 2.0, 10.0);
+                        myRetina = createRetina(inputFrame.size(), true, RETINA_COLOR_BAYER, true, 2.0, 10.0);
                 }
         else// -> else allocate "classical" retina :
-            myRetina = cv::createRetina(inputFrame.size());
+            myRetina = createRetina(inputFrame.size());
 
         // save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup"
         myRetina->write("RetinaDefaultParameters.xml");
@@ -143,7 +143,8 @@ int main(int argc, char* argv[]) {
             cv::imshow("retina input", inputFrame);
             cv::imshow("Retina Parvo", retinaOutput_parvo);
             cv::imshow("Retina Magno", retinaOutput_magno);
-            cv::waitKey(10);
+
+            cv::waitKey(5);
         }
     }catch(cv::Exception e)
     {