Last updated on 2024-06-18 | Edit this page
Overview
Questions
- How can we apply a low-pass blurring filter to an image?
Objectives
- Explain why applying a low-pass blurring filter to an image isbeneficial.
- Apply a Gaussian blur filter to an image using scikit-image.
In this episode, we will learn how to use scikit-image functions toblur images.
When processing an image, we are often interested in identifyingobjects represented within it so that we can perform some furtheranalysis of these objects, e.g., by counting them, measuring theirsizes, etc. An important concept associated with the identification ofobjects in an image is that of edges: the lines that representa transition from one group of similar pixels in the image to anotherdifferent group. One example of an edge is the pixels that represent theboundaries of an object in an image, where the background of the imageends and the object begins.
When we blur an image, we make the colour transition from one side ofan edge in the image to another smooth rather than sudden. The effect isto average out rapid changes in pixel intensity. Blurring is a verycommon operation we need to perform before other tasks such as thresholding. There are severaldifferent blurring functions in the ski.filters module, sowe will focus on just one here, the Gaussian blur.
Filters
In the day-to-day, macroscopic world, we have physical filters whichseparate out objects by size. A filter with small holes allows onlysmall objects through, leaving larger objects behind. This is a goodanalogy for image filters. A high-pass filter will retain the smallerdetails in an image, filtering out the larger ones. A low-pass filterretains the larger features, analogous to what’s left behind by aphysical filter mesh. High- and low-pass, here, referto high and low spatial frequencies in the image. Detailsassociated with high spatial frequencies are small, a lot of thesefeatures would fit across an image. Features associated with low spatialfrequencies are large - maybe a couple of big features per image.
Blurring
To blur is to make something less clear or distinct. This could beinterpreted quite broadly in the context of image analysis - anythingthat reduces or distorts the detail of an image might apply. Applying alow-pass filter, which removes detail occurring at high spatialfrequencies, is perceived as a blurring effect. A Gaussian blur is afilter that makes use of a Gaussian kernel.
Kernels
A kernel can be used to implement a filter on an image. A kernel, inthis context, is a small matrix which is combined with the image using amathematical technique: convolution. Different sizes, shapesand contents of kernel produce different effects. The kernel can bethought of as a little image in itself, and will favour features ofsimilar size and shape in the main image. On convolution with an image,a big, blobby kernel will retain big, blobby, low spatial frequencyfeatures.
Gaussian blur
Consider this image of a cat, in particular the area of the imageoutlined by the white square.
Now, zoom in on the area of the cat’s eye, as shown in the left-handimage below. When we apply a filter, we consider each pixel in theimage, one at a time. In this example, the pixel we are currentlyworking on is highlighted in red, as shown in the right-hand image.
When we apply a filter, we consider rectangular groups of pixelssurrounding each pixel in the image, in turn. The kernel isanother group of pixels (a separate matrix / small image), of the samedimensions as the rectangular group of pixels in the image, that movesalong with the pixel being worked on by the filter. The width and heightof the kernel must be an odd number, so that the pixel being worked onis always in its centre. In the example shown above, the kernel issquare, with a dimension of seven pixels.
To apply the kernel to the current pixel, an average of the colourvalues of the pixels surrounding it is calculated, weighted by thevalues in the kernel. In a Gaussian blur, the pixels nearest the centreof the kernel are given more weight than those far away from the centre.The rate at which this weight diminishes is determined by a Gaussianfunction, hence the name Gaussian blur.
A Gaussian function maps random variables into a normal distributionor “Bell Curve”. 
The shape of the function is described by a mean value μ, and avariance value σ². The mean determines the central point of the bellcurve on the X axis, and the variance describes the spread of thecurve.
In fact, when using Gaussian functions in Gaussian blurring, we use a2D Gaussian function to account for X and Y dimensions, but the samerules apply. The mean μ is always 0, and represents the middle of the 2Dkernel. Increasing values of σ² in either dimension increases the amountof blurring in that dimension.
The averaging is done on a channel-by-channel basis, and the averagechannel values become the new value for the pixel in the filtered image.Larger kernels have more values factored into the average, and thisimplies that a larger kernel will blur the image more than a smallerkernel.
To get an idea of how this works, consider this plot of thetwo-dimensional Gaussian function:
Imagine that plot laid over the kernel for the Gaussian blur filter.The height of the plot corresponds to the weight given to the underlyingpixel in the kernel. I.e., the pixels close to the centre become moreimportant to the filtered pixel colour than the pixels close to theouter limits of the kernel. The shape of the Gaussian function iscontrolled via its standard deviation, or sigma. A large sigma valueresults in a flatter shape, while a smaller sigma value results in amore pronounced peak. The mathematics involved in the Gaussian blurfilter are not quite that simple, but this explanation gives you thebasic idea.
To illustrate the blurring process, consider the blue channel colourvalues from the seven-by-seven region of the cat image above:
The filter is going to determine the new blue channel value for thecentre pixel – the one that currently has the value 86. The filtercalculates a weighted average of all the blue channel values in thekernel giving higher weight to the pixels near the centre of thekernel.
This weighted average, the sum of the multiplications, becomes thenew value for the centre pixel (3, 3). The same process would be used todetermine the green and red channel values, and then the kernel would bemoved over to apply the filter to the next pixel in the image.
Image edges
Something different needs to happen for pixels near the outer limitsof the image, since the kernel for the filter may be partially off theimage. For example, what happens when the filter is applied to theupper-left pixel of the image? Here are the blue channel pixel valuesfor the upper-left pixel of the cat image, again assuming aseven-by-seven kernel:
OUTPUT
x x x x x x x x x x x x x x x x x x x x x x x x 4 5 9 2 x x x 5 3 6 7 x x x 6 5 7 8 x x x 5 4 5 3The upper-left pixel is the one with value 4. Since the pixel is atthe upper-left corner, there are no pixels underneath much of thekernel; here, this is represented by x’s. So, what does the filter do inthat situation?
The default mode is to fill in the nearest pixel value fromthe image. For each of the missing x’s the image value closest to the xis used. If we fill in a few of the missing pixels, you will see howthis works:
OUTPUT
x x x 4 x x x x x x 4 x x x x x x 4 x x x 4 4 4 4 5 9 2 x x x 5 3 6 7 x x x 6 5 7 8 x x x 5 4 5 3Another strategy to fill those missing values is to reflectthe pixels that are in the image to fill in for the pixels that aremissing from the kernel.
OUTPUT
x x x 5 x x x x x x 6 x x x x x x 5 x x x 2 9 5 4 5 9 2 x x x 5 3 6 7 x x x 6 5 7 8 x x x 5 4 5 3A similar process would be used to fill in all of the other missingpixels from the kernel. Other border modes are available; youcan learn more about them in the scikit-imagedocumentation.
This animation shows how the blur kernel moves along in the originalimage in order to calculate the colour channel values for the blurredimage.
scikit-image has built-in functions to perform blurring for us, so wedo not have to perform all of these mathematical operations ourselves.Let’s work through an example of blurring an image with the scikit-imageGaussian blur function.
First, import the packages needed for this episode:
PYTHON
import imageio.v3 as iioimport ipymplimport matplotlib.pyplot as pltimport skimage as ski%matplotlib widgetThen, we load the image, and display it:
PYTHON
image = iio.imread(uri="data/gaussian-original.png")# display the imagefig, ax = plt.subplots()ax.imshow(image)
Next, we apply the gaussian blur:
PYTHON
sigma = 3.0# apply Gaussian blur, creating a new imageblurred = ski.filters.gaussian( image, sigma=(sigma, sigma), truncate=3.5, channel_axis=-1)The first two arguments to ski.filters.gaussian() arethe image to blur, image, and a tuple defining the sigma touse in ry- and cx-direction, (sigma, sigma). The thirdparameter truncate is meant to pass the radius of thekernel in number of sigmas. A Gaussian function is defined from-infinity to +infinity, but our kernel (which must have a finite,smaller size) can only approximate the real function. Therefore, we mustchoose a certain distance from the centre of the function where we stopthis approximation, and set the final size of our kernel. In the aboveexample, we set truncate to 3.5, which means the kernelsize will be 2 * sigma * 3.5. For example, for a sigma of1.0 the resulting kernel size would be 7, while for a sigmaof 2.0 the kernel size would be 14. The default value fortruncate in scikit-image is 4.0.
The last argument we passed to ski.filters.gaussian() isused to specify the dimension which contains the (colour) channels.Here, it is the last dimension; recall that, in Python, the-1 index refers to the last position. In this case, thelast dimension is the third dimension (index 2), since ourimage has three dimensions:
PYTHON
print(image.ndim)OUTPUT
3Finally, we display the blurred image:
PYTHON
# display blurred imagefig, ax = plt.subplots()ax.imshow(blurred)
Visualising Blurring
Somebody said once “an image is worth a thousand words”. What isactually happening to the image pixels when we apply blurring may bedifficult to grasp. Let’s now visualise the effects of blurring from adifferent perspective.
Let’s use the petri-dish image from previous episodes:
Graysacle version of the Petri dish image
What we want to see here is the pixel intensities from a lateralperspective: we want to see the profile of intensities. For instance,let’s look for the intensities of the pixels along the horizontal lineat Y=150:
PYTHON
# read colonies color image and convert to grayscaleimage = iio.imread('data/colonies-01.tif')image_gray = ski.color.rgb2gray(image)# define the pixels for which we want to view the intensity (profile)xmin, xmax = (0, image_gray.shape[1])Y = ymin = ymax = 150# view the image indicating the profile pixels positionfig, ax = plt.subplots()ax.imshow(image_gray, cmap='gray')ax.plot([xmin, xmax], [ymin, ymax], color='red')
Grayscale Petri dish image marking selectedpixels for profiling
The intensity of those pixels we can see with a simple line plot:
PYTHON
# select the vector of pixels along "Y"image_gray_pixels_slice = image_gray[Y, :]# guarantee the intensity values are in the [0:255] range (unsigned integers)image_gray_pixels_slice = ski.img_as_ubyte(image_gray_pixels_slice)fig, ax = plt.subplots()ax.plot(image_gray_pixels_slice, color='red')ax.set_ylim(255, 0)ax.set_ylabel('L')ax.set_xlabel('X')
Intensities profile line plot of pixels alongY=150 in original image
And now, how does the same set of pixels look in the correspondingblurred image:
PYTHON
# first, create a blurred version of (grayscale) imageimage_blur = ski.filters.gaussian(image_gray, sigma=3)# like before, plot the pixels profile along "Y"image_blur_pixels_slice = image_blur[Y, :]image_blur_pixels_slice = ski.img_as_ubyte(image_blur_pixels_slice)fig, ax = plt.subplots()ax.plot(image_blur_pixels_slice, 'red')ax.set_ylim(255, 0)ax.set_ylabel('L')ax.set_xlabel('X')
Intensities profile of pixels along Y=150 inblurred image
And that is why blurring is also called smoothing.This is how low-pass filters affect neighbouring pixels.
Now that we have seen the effects of blurring an image from twodifferent perspectives, front and lateral, let’s take yet another lookusing a 3D visualisation.
3D Plots with matplotlib
The code to generate these 3D plots is outside the scope of thislesson but can be viewed by following the links in the captions.
A 3D plot of pixel intensities across the wholePetri dish image before blurring. Explorehow this plot was created with matplotlib. Image credit: Carlos H Brandt.
A 3D plot of pixel intensities after Gaussianblurring of the Petri dish image. Note the ‘smoothing’ effect on thepixel intensities of the colonies in the image, and the ‘flattening’ ofthe background noise at relatively low pixel intensities throughout theimage. Explorehow this plot was created with matplotlib. Image credit: Carlos H Brandt.
Experimenting with sigma values (10 min)
The size and shape of the kernel used to blur an image can have asignificant effect on the result of the blurring and any downstreamanalysis carried out on the blurred image. The next two exercises askyou to experiment with the sigma values of the kernel, which is a goodway to develop your understanding of how the choice of kernel caninfluence the result of blurring.
First, try running the code above with a range of smaller and largersigma values. Generally speaking, what effect does the sigma value haveon the blurred image?
Generally speaking, the larger the sigma value, the more blurry theresult. A larger sigma will tend to get rid of more noise in the image,which will help for other operations we will cover soon, such asthresholding. However, a larger sigma also tends to eliminate some ofthe detail from the image. So, we must strike a balance with the sigmavalue used for blur filters.
Experimenting with kernel shape (10 min - optional, not included in timing)
Now, what is the effect of applying an asymmetric kernel to blurringan image? Try running the code above with different sigmas in the ry andcx direction. For example, a sigma of 1.0 in the ry direction, and 6.0in the cx direction.
PYTHON
# apply Gaussian blur, with a sigma of 1.0 in the ry direction, and 6.0 in the cx directionblurred = ski.filters.gaussian( image, sigma=(1.0, 6.0), truncate=3.5, channel_axis=-1)# display blurred imagefig, ax = plt.subplots()ax.imshow(blurred)
These unequal sigma values produce a kernel that is rectangularinstead of square. The result is an image that is much more blurred inthe X direction than in the Y direction. For most use cases, a uniformblurring effect is desirable and this kind of asymmetric blurring shouldbe avoided. However, it can be helpful in specific circ*mstances, e.g.,when noise is present in your image in a particular pattern ororientation, such as vertical lines, or when you want to removeuniform noise without blurring edges present in the image in aparticular orientation.
Other methods of blurring
The Gaussian blur is a way to apply a low-pass filter inscikit-image. It is often used to remove Gaussian (i.e., random) noisein an image. For other kinds of noise, e.g., “salt and pepper”, a medianfilter is typically used. See theskimage.filters documentation for a list of availablefilters.
Key Points
- Applying a low-pass blurring filter smooths edges and removes noisefrom an image.
- Blurring is often used as a first step before we performthresholding or edge detection.
- The Gaussian blur can be applied to an image with the
ski.filters.gaussian()function. - Larger sigma values may remove more noise, but they will also removedetail from an image.
