ABSTRACTDynamic weather conditions such as rain, result in effecting the performance of vision algorithms. These vision algorithms are employed for video surveillance and analysis tasks.
Many methods have been proposed to reduce the effects of raindrop that are regarded as noise. However, removing of rain-drop is quite challenging for single images since each of them affects only on a very small region of an image. Expulsion of rain from still pictures is a complex and a test undertaking. The rain drops influence just on a little area of a picture, and subsequently, prompting a disarray to figure out which area ought to be considered and which ought to not. Using the image impainting techniques the clear image can be obtained. By applying Neural Networks better results are obtained.ACKNOWLEDGEMENTI wish to record my deep sense of gratitude and profound thanks to my research supervisor Ms Manjusha R, Asst Professor, Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Coimbatore, for his keen interest, inspiring guidance, constant encouragement with my work during all stages, to bring this thesis into fruition.I am extremely indebted to Ms Manjusha R, Asst Professor, Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Coimbatore, for their valuable suggestions and support during the course of my research work.
I also thank the faculty and non-teaching staff members of the Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Coimbatore, for their valuable support throughout the course of my research work. HIMABINDU YTABLE OF CONTENTSCHAPTER NO.TITLE PAGE NO.
TOC “Heading 2,4,Heading 3,5,Heading 4,6,Heading 5,7,Heading 6,8,cfr_Abstractheading,1,cfr_ChapterHead,3,cfr_ReferHead,9,cfr_Tablesandfigures,2” ABSTRACT PAGEREF _Toc418848707 h iiiLIST OF FIGURESviIntroduction 71.1introduction81.1.1Image Restoration81.
1.2Applications of Image Restoration81.2LITERATURE9EXPERIMENTAL TECHNIQUES 162.1SCOPE AND OBJECTIVE162.2kEY CHALLENGES162.3IMPLEMENTATION162.4OUTPUT SCREENSHOTS172.5Future work18REFERENCES19List of figuresFigure noTitlepage no1 Image restoration 82 derainNet Architecture 123 jorder architecture 15Introduction introduction to digital image processingDigital image processing deals with manipulation of digital images through a digital computer.
Digital Image processing is a strategy to play out a few tasks on a picture, with the end goal to get an improved picture or to remove some valuable data from it. The mathematical function f(x , y) where x and y are the two co-ordinates horizontally and vertically define an Image. An image is given as input, using efficient algorithms and operations the image can be processed and enhanced.Digital image processing is one of the popular research topics among the researcher. The digital image processing techniques are used in several areas such as medical visualisation, image enhancement, image restoration, satellite views, surveillance purpose and many more. The two major tasks that Digital image processing focuses on, are the improvement of pictorial information for human interpretation and processing of image data for storage, transmission and representation for autonomous machine perception.There are different essential advances associated with the Image Processing that is representation of pictures, preprocessing of pictures, enhancement, restoration, reconstruction of pictures and compression. Image RestorationThe concerns of the image restoration are the removal or decrease of corruptions which are incorporated during the acquisition of pictures e.
g., Noise, pixel value errors, out of focus blur or camera motion blur using prior information of the degradation phenomenon. The process or the operation carried out to obtain a clean image from a corrupted or noisy image is said to be Image Restoration. The main objective of Image Restoration is to reduce the noise and recover the resolution loss.Applications of Image RestorationIn medical imaging such as computerised tomography(CT) and magnetic resonance imaging (MRI).
For satellite imagery, the resolution of captured images can be improved using image restoration techniques.One of the major area is surveillance. Now a days, most of the places are kept under surveillance thus if the cameras are outdoor cameras then they are affected by bad weather conditions. The places like shopping malls, traffic signals and many more employ cameras outside which in turn are effected by rain and sometimes snow. This leads to degradation of the images or videos taken during that period. Figure 1:Image RestorationLITERATURE surveyThe research on raindrop detection and removal from images as well as videos started in 90s. In case of a degraded video, it can be restored using past frames.
But for a single image it is challenging task to restore or reconstruct to proper state.In this paper they have proposed a method that characterizes and enhances the fundamental image constituents like salient edges. This approach relates edge-preserving smoothing. The objective is to maintain and enhance the most prominent set of edges.
The objective that is removal of raindrops is achieved by smoothing operation. The salient regions are identified by smoothing operation. By confining the number of non-zero gradients the highest-contrast edges are identified in global manner. The input is a discrete signal and the count of amplitude changes is taken.
c(f) = #{ p | | fp – fp+1| != 0 }(1)p and p+1 are the indices of neighboring samples. |fp – fp+1 | is the gradient with respect to p. #{} denotes the counting operator that gives the count of p that satisfies the condition i.
e., non-zero components. To abstract structural information i.e., to flatten the details and sharpen the main edges the method is combined with a general constraint, that is, there exists a structural similarity between input signal g and the result f.(2)Thus says there exist k non-zero gradients.Algorithm 1:Input: image I, smoothing weight ?, smoothing rate ?, parameters ? and ?maxOutput: smoothed image SInitialization: S ? I, ?max ? ?, ? = 2.
0, ? = 2E – 2, ?max = 1E5Reading input image IEvaluating FFT on Input image IIntroducing variables h and v? = 2* ?While ? < ?maxSolve (h, v) Solve S Evaluating IFFT? = ? * ?EndOutput image SEnhance SAlgorithm explains the steps of the smoothing activity. The parameter ? is consequently refreshed in every iteration beginning from an initial value, it is increased by ? each time. Using Contrast Enhancement the pixel brightness is maintained and controlled. Histogram stretching is used to shift the pixel values to fill the brightness resulting in high contrast. This yields the better quality of images.1The method proposed in this paper is a hybrid approach.
The input color image is converted to YCBCR color space to obtain accurate gray component. The algorithm starts with a guess about centroid of cluster and iterated until a local optimum has found. (3)Using the above equation the distance between clusters can be measured. Using a histogram the peaks and valleys are found which in turn are used for locating the clusters in the image. From this clusters we obtain the segmented region and by combining the boundary extraction we get the image features. In this paper they have used elbow algorithm to find the exact number of clusters. The peak value of histogram is calculated based on the value of number of clusters.Algorithm 2 : K-Means Cluster1.
Set K value2. Assign K items randomly from the list to initial cluster K3. doa. Calculate the difference between the item and the center.b. Assigning the item to the cluster whose centroid is the most similar to the item intensity.
c. Recalculating the centroid s because of the item removal or item adding4. End while convergence achieved.After the potential raindrops areas are distinguished by the saliency strategy, a methods for check is applied to affirm if the area contains a raindrop or not.
Using Hough circle transform the shape information is obtained.Algorithm 3: circle Extraction1: Give the input image.2: Smoothen the input image.
3: Get the edge of the processed image.4: Based on the peak value find the threshold.5: Based on the object boundary filling the region.6: Get the segmented circle as result.
Using Algorithm 2 and algorithm 3 the raindrop boundary can be extracted. After classifying the regions using k-means, canny edge detection is used to obtain edges and circle boundary information. The raindrops are detected by subtraction of background region.
Using median filter the drops are removed.2In this paper, “DerainNet” is proposed for removal of raindrops from images, where the deep CNN is the base. Using DerainNet the nonlinear mapping function between clean and rainy detail layers is learnt directly and automatically.
To improve the visual effect both the rain removal and the image enhancement are carried out. Image processing domain knowledge is used to improve the de-rain quality. Each input image is decomposed into a low-frequency and a high-frequency layer. They are termed as base and detail layers respectively. The detail layer is the input to the CNN. An image enhancement step to sharpen the results of both layers is performed to improve the visual quality. After this decomposition we train the CNN on the detail layer image.
After training the network, the de-rained image can be obtained by directly adding the output detail layer to the base layer.3.Figure 2 : DerainNet ArchitectureTaken from : Fu, Xueyang, et al. “Clearing the skies: A deep network architecture for single-image rain removal.” IEEE Transactions on Image Processing 26.6 (2017)In this paper, the solution for the problem of rain streak removal is addressed using layer decomposition.
The image O is decomposed into two layers i.e., the rain free background layer B and the rain streak layer R.O = B + R(4)The method used here assigns patch-based priors for the decomposed layers. Multiple orientations and scales of rain streaks can be handled using these priors. These priors are based on Gaussian mixture models.4(5)(6)(7)Algorithm 4:Input: input image O; GMMs for two layers: GB and GR;Initialization: B ? O; R ? 0; ? ? ??;repeatupdate H using Eq.
5;solve {B, R} by Eq. 6;solve {gBi ,gRi} by Eq. 7;? = 2 * ?;until convergence or maximum iteration number;Output: The estimation of two layers B and R;In this paper, a method which does not require any pixel-based statistical information is presented. This method is suitable for both single images as well as videos as it does not require any alignment before detection.
(8)Using this, the intensity produced by rain on a pixel is calculated and compute the reference image i.e., which is not effected by raindrops.
The obtained reference image gives rough contours but it does not have those edges which are effected by rain streaks. Thus using a guided filter, an edge preserving filter I’ is obtained. Using these reference image and guided image the effected regions are restored.5Here in this paper, a procedure without explicit motion analysis is introduced. The method analyses the temporal changes and selects rain corrupted pixels. Further, the selected pixels are filtered for camera and object motion using spatio-temporal consistency constraints. Intensity subtraction is used for recovering the corrupted pixels.
6In this paper, using a binary map in addition to an existing model, rain streak regions are located and a new model consisting of two components, one for representing rain streak and another for representing various shapes and directions of overlapping rain streaks is created. A multitask deep learning architecture is used which learns the binary mapping,appearance of rain streaks and clean background. A recurrent rain detection and removal network is introduced in order to handle the accumulation of rain streaks. In this model they introduced a generalized rain model.O = B + SR(9)Where B is background scene without rain streaks and S is the rain streak layer. R is a new region dependent variable that gives the location of individual visible rain drops.
R elements take binary values. Thus the rain streak and non rain streak regions are operated differently preserving the background details. Convolutional multi-task network JORDER(joint rain detection and removal) is constructed. This detects the rain effected regions and a contextualized dilated network is used for extracting the rain distinguishing features.
Figure 3: JORDER ArchitectureTaken from : Dodkey, Nitesh. “Rain Streaks Detection and Removal in Image based on Entropy Maximization and Background Estimation.” Threshold 164.11 (2017).In this paper, an orientation filter is used for finding the rain drops. Using the entropy maximization remaining rain drops are found and background removal is carried out.
Here the decomposition is based on the orientation of rain drops. The background estimation is performed by using a low pass filter.The background in images is obtained by high frequency suppression and orientation filter is again applied for removing rain drops. 8In this paper, using a dark channel prior the raw atmospheric transmission map is estimated and then, based on atmospheric scattering the image is restored back. The dark channel prior is gives a kind of statistics of outdoor haze-free images. It is a key observation- the pixels effected have very low intensity in atleast one color channel.
Using the prior the thickness of haze is estimated and the image is restored. 9PROPOSED WORKA detailed experimental procedure adopted in this investigation is presented in this chapter. In addition, the theoretical formulations involved in the computation of various acoustic parameters are also discussed.
SCOPE AND OBJECTIVE The removal of rain streaks has recently received much attention in the research work in the field of image processing. The rain removal is just like the image enhancement and may come in the category of image noise removal or image restoration.kEY CHALLENGESIdentifying the raindropsDifferentiating the raindrops from objectsRestoring the image without loosing information.iMPLEMENTATIONInitially performed subtraction of the background information from image i.
e., separating the rain streak layer and object layer. By using high-pass and low- pass filter along with certain threshold setting this objective was partially achieved.Now tried to fill those raindrop effected regions using image impainting technique. The mean of surround pixels is used for filling that region.Applying the L0 gradient smoothening technique resulted in partial rain free image.2.4Output Screenshots:Figure 4:Rainy layerFigure 5: After rain removal2.
5FUTURE WORKFor the obtained result applying a deep learning architecture for obtained an enhanced image with maximum removal of rain streaks and with a better accuracy.REFERENCES BIBLIOGRAPHY l 1033 Manu, B. N. “Rain removal from still images using L0 gradient minimization technique.” Information Technology and Electrical Engineering (ICITEE), 2015 7th International Conference on. IEEE, 2015.Kanthan, M. Ramesh, and S.
Naganandini Sujatha. “Rain drop detection and removal using K-Means clustering.” Computational Intelligence and Computing Research (ICCIC), 2015 IEEE International Conference on. IEEE, 2015.Fu, Xueyang, et al. “Clearing the skies: A deep network architecture for single-image rain removal.
” IEEE Transactions on Image Processing 26.6 (2017).Li, Yu, et al. “Rain streak removal using layer priors.
” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.Xu, Jing, et al. “Removing rain and snow in a single image using guided filter.” Computer Science and Automation Engineering (CSAE), 2012 IEEE International Conference on. Vol. 2.
IEEE, 2012.Subhani, M. F., and J.
P. Oakley. “Low latency mitigation of rain induced noise in images.” (2008): 13-13.Yang, Wenhan, et al. “Deep joint rain detection and removal from a single image.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
2017.Dodkey, Nitesh. “Rain Streaks Detection and Removal in Image based on Entropy Maximization and Background Estimation.
” Threshold 164.11 (2017).He, Kaiming, Jian Sun, and Xiaoou Tang. “Single image haze removal using dark channel prior.
” IEEE transactions on pattern analysis and machine intelligence 33.12 (2011): 2341-2353.Zhu, Lei, et al. “Joint bilayer optimization for single-image rain streak removal.” Proceedings of the IEEE international conference on computer vision. 2017.Liu, Risheng, et al.
“Deep Layer Prior Optimization for Single Image Rain Streaks Removal.” 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018.Luo, Yu, Yong Xu, and Hui Ji. “Removing rain from a single image via discriminative sparse coding.” Proceedings of the IEEE International Conference on Computer Vision. 2015.Bai, Zongwen, et al.
“Algorithm designed for image inpainting based on decomposition and fractal.” Electronic and Mechanical Engineering and Information Technology (EMEIT), 2011 International Conference on. Vol. 3. IEEE, 2011.
Chen, Libo, and Jin Wu. “Image inpainting algorithm based on self-adaptive structural group sparse representation.” 2018 13th IEEE Conference on Industrial Electronics and Applications (ICIEA). IEEE, 2018.
