ABSTRACTDynamic weather conditions such as rain, result in effecting the performance of vision algorithms. These vision algorithms are employed for video surveillance and analysis tasks. Many methods have been proposed to reduce the effects of raindrop that are regarded as noise. However, removing of rain-drop is quite challenging for single images since each of them affects only on a very small region of an image.
Expulsion of rain from still pictures is a complex and a testing undertaking. The rain drops influences just on a little area of a picture, and subsequently, prompting a disarray to figure out which area ought to be considered and which ought to not. Using the image impainting techniques the clear image can obtained with partial enhancement. By applying Neural Networks better results are obtained.
ACKNOWLEDGEMENTI wish to record my deep sense of gratitude and profound thanks to my research supervisor Ms. Manjusha R, Asst Professor, Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Coimbatore, for his keen interest, inspiring guidance, constant encouragement with my work during all stages, to bring this thesis into fruition.I am extremely indebted to Dr. Latha Parameswaran, Chairperson, Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Coimbatore, for their valuable suggestions and support during the course of my research work.I also thank the faculty and non-teaching staff members of the Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Coimbatore, for their valuable support throughout the course of my research work. HIMABINDU YTABLE OF CONTENTSCHAPTER NO.TITLE PAGE NO. TOC “Heading 2,4,Heading 3,5,Heading 4,6,Heading 5,7,Heading 6,8,cfr_Abstractheading,1,cfr_ChapterHead,3,cfr_ReferHead,9,cfr_Tablesandfigures,2” ABSTRACT PAGEREF _Toc418848707 h iiiLIST OF FIGURESviIntroduction 71.
1introduction81.1.1IMAGE RESTORATION81.
1.2APPLICATIONS OF IMAGE RESTORATION81.2LITERATURE9EXPERIMENTAL TECHNIQUES 162.1SCOPE AND OBJECTIVE162.2kEY CHALLENGES162.
3IMPLEMENTATION162.4OUTPUT SCREENSHOTS172.5Future work18REFERENCES19List of figuresFigure noTitlepage no1 Image restoration 82 derainNet Architecture 123 jorder architecture 15Introduction introduction to digital image processingDigital image processing deals with manipulation of digital images through a digital computer.
Digital Image processing is a strategy to play out a few tasks on a picture, with the end goal to get an improved picture or to remove some valuable data from it. The mathematical function f(x , y) where x and y are the two co-ordinates horizontally and vertically define an Image. An image is given as input, using efficient algorithms and operations the image can be processed and enhanced.
Digital image processing is one of the popular research topics among the researcher. The digital image processing techniques are used in several areas such as medical visualisation, image enhancement, image restoration, satellite views, surveillance purpose and many more. The two major tasks that Digital image processing focuses are the improvement of pictorial information for human interpretation and processing of image data for storage, transmission and representation for autonomous machine perception.There are different essential advances associated with the Image Processing that is representation of pictures, preprocessing of pictures, enhancement, restoration, reconstruction of pictures and compression. Image RestorationThe concerns of the image restoration are the removal or decrease of corruptions which are incorporated during the acquisition of pictures e.g.
, Noise, pixel value errors, out of focus blur or camera motion blur using prior information of the degradation phenomenon. The process or the operation carried out to obtain a clean image from a corrupted or noisy image is said to be Image Restoration. The main objective of Image Restoration is to reduce the noise and recover the resolution loss.Applications of Image RestorationIn medical imaging such as computerised tomography(CT) and magnetic resonance imaging (MRI).For satellite imagery, the resolution of captured images can be improved using image restoration techniques.One of the major area is surveillance. Now a days, most of the places are kept under surveillance thus if the cameras are outdoor cameras then they are affected by bad weather conditions. The places like shopping malls, traffic signals and many more employ cameras outside which in turn are effected by rain and sometimes snow.
This leads to degradation of the images or videos taken during that period. Figure 1:Image RestorationLITERATURE surveyThe research on raindrop detection and removal from images as well as videos started in 90s. In case of a degraded video, it can be restored using past frames. But for a single image it is challenging task to restore or reconstruct to proper state.In 1 they have proposed a method that characterizes and enhances the fundamental image constituents like salient edges. Their approach relates edge-preserving smoothing.
The objective was to maintain and enhance the most prominent set of edges.The objective i.e.
, removal of raindrops has been achieved by smoothing operation. The salient regions were identified by smoothing operation. By confining the number of non-zero gradients the highest-contrast edges were identified in global manner. The input was a discrete signal and the count of amplitude changes was taken.c(f) = #{ p | | fp – fp+1| != 0 }(1)p and p+1 are the indices of neighboring samples. |fp – fp+1 | is the gradient with respect to p. #{} denotes the counting operator that gives the count of p that satisfies the condition i.e.
, non-zero components. To abstract structural information i.e., to flatten the details and sharpen the main edges the method was combined with a general constraint, that is, there exists a structural similarity between input signal g and the result f.
(2)Algorithm 1:Input: image I, smoothing weight ?, smoothing rate ?, parameters ? and ?maxOutput: smoothed image SInitialization: S ? I, ?max ? ?, ? = 2.0, ? = 2E – 2, ?max = 1E5Reading input image IEvaluating FFT on Input image IIntroducing variables h and v? = 2* ?While ? < ?maxSolve (h, v) Solve S Evaluating IFFT? = ? * ?EndOutput image SEnhance SAlgorithm explains the steps of the smoothing activity. The parameter ? was consequently refreshed in every iteration beginning from an initial value, it is increased by ? each time. Using Contrast Enhancement the pixel brightness is maintained and controlled. Histogram stretching is used to shift the pixel values to fill the brightness resulting in high contrast.
This yields the better quality of images.The method proposed in 2 was a hybrid approach. The input color image was converted to YCBCR color space to obtain accurate gray component. The algorithm starts with a guess about centroid of cluster and iterated until a local optimum has found. (3)Using the above equation the distance between clusters is measured.
Using a histogram the peaks and valleys are found which inturn are used for locating the clusters in the image. From this clusters they obtained the segmented regions and by combining the boundary extraction they got the image features. In 2 they have used elbow algorithm to find the exact number of clusters. The peak value of histogram was calculated based on the value of number of clusters.Algorithm 2 : K-Means Cluster1. Set K value2. Assign K items randomly from the list to initial cluster K3.
doa. Calculate the difference between the item and the center.b.
Assigning the item to the cluster whose centroid is the most similar to the item intensity.c. Recalculating the centroid s because of the item removal or item adding4. End while convergence achieved.After the potential raindrops areas are distinguished by the saliency strategy, a methods for check is applied to affirm if the area contains a raindrop or not. Using Hough circle transform the shape information is obtained.Algorithm 3: circle Extraction1: Give the input image.2: Smoothen the input image.
3: Get the edge of the processed image.4: Based on the peak value find the threshold.5: Based on the object boundary filling the region.6: Get the segmented circle as result.Using Algorithm 2 and algorithm 3 the raindrop boundary can be extracted. After classifying the regions using k-means, canny edge detection was used to obtain edges and circle boundary information. The raindrops were detected by subtraction of background region.
Using median filter the drops were removed.In 3, “DerainNet” is proposed for removal of raindrops from images, where the deep CNN is the base. Using DerainNet the nonlinear mapping function between clean and rainy detail layers is learnt directly and automatically. To improve the visual effect both the rain removal and the image enhancement are carried out. Image processing domain knowledge was used to improve the de-rain quality.
Each input image was decomposed into a low-frequency and a high-frequency layer. They were termed as base and detail layers respectively. The detail layer is the input to the CNN. An image enhancement step to sharpen the results of both layers was performed to improve the visual quality. After this decomposition they train the CNN on the detail layer image. After training the network, the de-rained image was obtained by directly adding the output detail layer to the base layer.
.Figure 2 : DerainNet ArchitectureTaken from : Taken from : Fu, Xueyang, et al. “Clearing the skies: A deep network architecture for single-image rain removal.” IEEE Transactions on Image Processing 26.6 (2017)In 4, the solution for the problem of rain streak removal was addressed using layer decomposition. The image O was decomposed into two layers i.e.
, the rain free background layer B and the rain streak layer R.O = B + R(4)The method used here assigns patch-based priors for the decomposed layers. Multiple orientations and scales of rain streaks were handled using these priors. These priors are based on Gaussian mixture models.(5)(6)(7)Algorithm 4:Input: input image O; GMMs for two layers: GB and GR;Initialization: B ? O; R ? 0; ? ? ??;repeatupdate H using Eq. 5;solve {B, R} by Eq. 6;solve {gBi ,gRi} by Eq.
7;? = 2 * ?;until convergence or maximum iteration number;Output: The estimation of two layers B and R;In 5, a method which does not require any pixel-based statistical information was presented. This method is suitable for both single images as well as videos as it does not require any alignment before detection.(8)Using this, the intensity produced by rain on a pixel is calculated and compute the reference image i.e., which is not effected by raindrops. The obtained reference image gave rough contours but it does not have those edges which are effected by rain streaks.
Thus using a guided filter, an edge preserving filter I’ is obtained. Using these reference images and guided image the effected regions are restored.Here in 6, a procedure without explicit motion analysis is introduced. The method analyses the temporal changes and selects rain corrupted pixels. Further, the selected pixels are filtered for camera and object motion using spatio-temporal consistency constraints. Intensity subtraction was used for recovering the corrupted pixels.
In 7, using a binary map in addition to an existing model, rain streak regions were located and a new model consisting of two components, one for representing rain streak and another for representing various shapes and directions of overlapping rain streaks was created. A multitask deep learning architecture was used which learns the binary mapping, appearance of rain streaks and clean background. A recurrent rain detection and removal network was introduced in order to handle the accumulation of rain streaks. In this model they introduced a generalized rain model.
O = B + SR(9)Where B is background scene without rain streaks and S is the rain streak layer. R is a new region dependent variable that gives the location of individual visible rain drops. R elements take binary values. Thus the rain streak and non rain streak regions were operated differently preserving the background details. Convolutional multi-task network JORDER(joint rain detection and removal) was constructed. This detects the rain effected regions and a contextualized dilated network was used for extracting the rain distinguishing features.Figure 3: JORDER ArchitectureTaken from : Dodkey, Nitesh.
“Rain Streaks Detection and Removal in Image based on Entropy Maximization and Background Estimation.” Threshold 164.11 (2017).
In 8, an orientation filter was used for finding the rain drops. Using the entropy maximization remaining rain drops were found and background removal was carried out. Here the decomposition was based on the orientation of rain drops. The background estimation is performed by using a low pass filter.
The background in images was obtained by high frequency suppression and orientation filter was again applied for removing rain drops. In 9, using a dark channel prior the raw atmospheric transmission map was estimated and then, based on atmospheric scattering the image was restored back. The dark channel prior gives a kind of statistics of outdoor haze-free images. It was a key observation- the pixels effected have very low intensity in atleast in one color channel. Using the prior the thickness of haze was estimated and the image was restored. PROPOSED WORKA detailed experimental procedure adopted in this investigation is presented in this chapter.
In addition, the theoretical formulations involved in the computation of various acoustic parameters are also discussed.SCOPE AND OBJECTIVE The removal of rain streaks has recently received much attention in the research work in the field of image processing. The rain removal is just like the image enhancement and may come in the category of image noise removal or image restoration.kEY CHALLENGESIdentifying the raindropsDifferentiating the raindrops from objectsRestoring the image without loosing information.iMPLEMENTATIONInitially performed subtraction of the background information from image i.
e., separating the rain streak layer and object layer. By using high-pass and low- pass filter along with certain threshold setting this objective was partially achieved.Now tried to fill those raindrop effected regions using image impainting technique. The mean of surround pixels is used for filling that region.Applying the L0 gradient smoothening technique resulted in partial rain free image.2.
4Output Screenshots:Figure 4:Rainy layerFigure 5: After rain removal2.5FUTURE WORKFor the obtained result applying a deep learning architecture for obtained an enhanced image with maximum removal of rain streaks and with a better accuracy.REFERENCES BIBLIOGRAPHY l 1033 Manu, B.
N. “Rain removal from still images using L0 gradient minimization technique.” Information Technology and Electrical Engineering (ICITEE), 2015 7th International Conference on. IEEE, 2015.
Kanthan, M. Ramesh, and S. Naganandini Sujatha. “Rain drop detection and removal using K-Means clustering.” Computational Intelligence and Computing Research (ICCIC), 2015 IEEE International Conference on. IEEE, 2015.Fu, Xueyang, et al.
“Clearing the skies: A deep network architecture for single-image rain removal.” IEEE Transactions on Image Processing 26.6 (2017).Li, Yu, et al.
“Rain streak removal using layer priors.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.Xu, Jing, et al.
“Removing rain and snow in a single image using guided filter.” Computer Science and Automation Engineering (CSAE), 2012 IEEE International Conference on. Vol. 2. IEEE, 2012.
Subhani, M. F., and J. P.
Oakley. “Low latency mitigation of rain induced noise in images.” (2008): 13-13.Yang, Wenhan, et al. “Deep joint rain detection and removal from a single image.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.Dodkey, Nitesh.
“Rain Streaks Detection and Removal in Image based on Entropy Maximization and Background Estimation.” Threshold 164.11 (2017).He, Kaiming, Jian Sun, and Xiaoou Tang.
“Single image haze removal using dark channel prior.” IEEE transactions on pattern analysis and machine intelligence 33.12 (2011): 2341-2353.Zhu, Lei, et al. “Joint bilayer optimization for single-image rain streak removal.” Proceedings of the IEEE international conference on computer vision.
2017.Liu, Risheng, et al. “Deep Layer Prior Optimization for Single Image Rain Streaks Removal.
” 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018.Luo, Yu, Yong Xu, and Hui Ji.
“Removing rain from a single image via discriminative sparse coding.” Proceedings of the IEEE International Conference on Computer Vision. 2015.Bai, Zongwen, et al. “Algorithm designed for image inpainting based on decomposition and fractal.” Electronic and Mechanical Engineering and Information Technology (EMEIT), 2011 International Conference on. Vol. 3.
IEEE, 2011.Chen, Libo, and Jin Wu. “Image inpainting algorithm based on self-adaptive structural group sparse representation.
” 2018 13th IEEE Conference on Industrial Electronics and Applications (ICIEA). IEEE, 2018.
