How I used Machine Learning in my Photo Editing Application(Android)

As a hobby I was interested working on a photo editing application,

Basic feature of this application was blurring background.

The first stage of the application involved manual blurring of specific regions of the image by the user through touch input. Manually blurring the image placed a significant burden on the user, especially since their primary objective was likely to blur only the background.

Two key functionalities could alleviate the burden of manual blurring

• Auto detecting background in an image
• Blur only the background

Detecting Background of Image using Machine Learning:

Identifying background in an image can be accomplished through an image segmentation model. Numerous open-source image segmentation models are accessible online. The one I employed was the TensorFlow Lite version of the Deeplab 3 image segmentation model. I seamlessly integrated this model into my application using the TensorFlow Lite library. Essentially, when you input an image into this specific model, it discerns the number of segments present in the image.

As you can see from the above gif, it will clear, this model takes an image as input and returns a segmented map as an output

Model Input:

Model Output:

Blur only the background:

With the segmented image providing clear distinction between the background and objects present, our objective now is to implement this functionality on real-time images to blur the background.

Step 1: Execute the Image Segmentation Model. I won’t delve into the technical intricacies of running a machine learning model on an Android application, as this GitHub sample provides a solid

Step 2: We then use PorterDuff.Mode feature from the Pain object to blend the original image with the segmented image in a canvas, inturn we remove the background from the original image.

fun blendOriginalBitmapWithSegmentedBitmap(original: Bitmap, segmentedBitmap: Bitmap?, canvas: Canvas) {
            if (original == null
                || segmentedBitmap == null
            ) {
                return
            }

            val maskPaint = BitmapSelector.getPaintObject(PorterDuff.Mode.DST_IN)
            val w = original.width
            val h = original.height
            if (w <= 0 || h <= 0) {
                return
            }
            canvas.drawBitmap(original, 0f, 0f, null)
            canvas.drawBitmap(segmentedBitmap, 0f, 0f, maskPaint)
        }

Step 3: In this last phase, we blend the outcome of our earlier image processing with the blurred version of the original image, effectively creating a blurred background effect.(assumption we should have a blurred version of the original image, which can achieved via android default library)

        // you should pass the canvas object of the previous method.
        fun blurBakcgroundOfImage(blurBitmap: Bitmap, canvas: Canvas) {
            if (blurBitmap == null) {
                return
            }
            val paint = BitmapSelector.getPaintObject(PorterDuff.Mode.DST_OVER)
            val w = blurBitmap.width
            val h = blurBitmap.height
            if (w <= 0 || h <= 0) {
                return
            }
            canvas.drawBitmap(blurBitmap, 0f, 0f, paint)
        }

The Canvas will have the resulting image, which you can later convert it to a bitmap and then write it in the file,

So whole method might look like the following.

           val resultBitmap = Bitmap.createBitmap(
                originalBitmap!!.width,
                originalBitmap!!.height,
                Bitmap.Config.ARGB_8888
            )
            val canvas = Canvas(resultBitmap)
   
            if (segmentedBitmap != null && bluredBitmap != null) {
                blendOriginalBitmapWithSegmentedBitmap(
                    originalBitmap,
                    segmentedBitmap,
                    canvas
                )
                blurBakcgroundOfImage(
                    bluredBitmap,
                    canvas
                )
            }
//result bitmap will have the bckground blurred image 

This approach not only assisted me in blurring the background of an image but also enabled me to achieve some remarkable features.

I’ll share a few examples below. Moreover, with the same technique, you can even replace the background of any image. Pretty incredible, isn’t it?

If you enjoyed this article, please consider following me on LinkedIn for more similar content. Your comments below would greatly encourage me to continue writing more articles. Thank you for your support!