Face Detection App with MLKit Android [PROJECT]
6 min readMar 9, 2023
Face detection is a powerful technique that allows developers to identify human faces in images or live camera streams. With MLKit and Kotlin, building a face detection app for Android has become easier than ever before.
In this article, we will discuss how to create a face detection app with MLKit and Kotlin on Android.
What is MLKit?
MLKit is a mobile SDK that provides developers with pre-built machine learning models and tools to easily integrate them into their applications. It is a powerful tool that can help developers create intelligent and responsive apps without having to become machine learning experts themselves.
What is the Face Detection?
With ML Kit’s face detection API, you can detect faces in an image, identify key facial features, and get the contours of detected faces. Note that the API detects faces, it does not recognize people .
With face detection, you can get the information you need to perform tasks like embellishing selfies and portraits, or generating avatars from a user’s photo. Because ML Kit can perform face detection in real time, you can use it in applications like video chat or games that respond to the player’s expressions.
Key capabilities
- Recognize and locate facial features Get the coordinates of the eyes, ears, cheeks, nose, and mouth of every face detected.
- Get the contours of facial features Get the contours of detected faces and their eyes, eyebrows, lips, and nose.
- Recognize facial expressions Determine whether a person is smiling or has their eyes closed.
- Track faces across video frames Get an identifier for each unique detected face. The identifier is consistent across invocations, so you can perform image manipulation on a particular person in a video stream.
- Process video frames in real time Face detection is performed on the device, and is fast enough to be used in real-time applications, such as video manipulation.
Face detection concepts
Face detection locates human faces in visual media such as digital images or video. When a face is detected it has an associated position, size, and orientation; and it can be searched for landmarks such as the eyes and nose.
Here are some of the terms that we use regarding the face detection feature of ML Kit:
- Face tracking extends face detection to video sequences. Any face that appears in a video for any length of time can be tracked from frame to frame. This means a face detected in consecutive video frames can be identified as being the same person. Note that this isn’t a form of face recognition; face tracking only makes inferences based on the position and motion of the faces in a video sequence.
- A landmark is a point of interest within a face. The left eye, right eye, and base of the nose are all examples of landmarks. ML Kit provides the ability to find landmarks on a detected face.
- A contour is a set of points that follow the shape of a facial feature. ML Kit provides the ability to find the contours of a face.
- Classification determines whether a certain facial characteristic is present. For example, a face can be classified by whether its eyes are open or closed, or if the face is smiling or not.
Build Face Detection App
build.gradle
Add the dependencies for the ML Kit Android libraries to your module’s app-level gradle file, which is usually app/build.gradle and we will use viewbinding, we should not forget to add the necessary additions in it.
android {
namespace 'com.ibrahimcanerdogan.facedetectionapp'
compileSdk 33
...
buildFeatures {
viewBinding true
}
}// Face Detection
implementation 'com.google.mlkit:face-detection:16.1.5'
// CameraX
def camerax_version = "1.3.0-alpha04"
implementation "androidx.camera:camera-core:${camerax_version}"
implementation "androidx.camera:camera-camera2:${camerax_version}"
implementation "androidx.camera:camera-lifecycle:${camerax_version}"
implementation "androidx.camera:camera-video:${camerax_version}"
implementation "androidx.camera:camera-view:${camerax_version}"
implementation "androidx.camera:camera-mlkit-vision:${camerax_version}"
implementation "androidx.camera:camera-extensions:${camerax_version}"
// CameraSource
implementation 'com.google.android.gms:play-services-vision-common:19.1.3'AndroidManifest.xml
The following permissions must be added for camera access.
<uses-permission android:name="android.permission.CAMERA" />Change “android:theme” for NoActionBar.
<application
...
android:theme="@style/Theme.AppCompat.NoActionBar"
...
</application>Create Graphic Classes
RectangleOverlay.kt
Draw rectangle with GraphicOverlay.
class RectangleOverlay(
private val overlay: GraphicOverlay<*>,
private val face : Face,
private val rect : Rect
) : GraphicOverlay.Graphic(overlay) {
private val boxPaint : Paint = Paint()
init {
boxPaint.color = Color.GREEN
boxPaint.style = Paint.Style.STROKE
boxPaint.strokeWidth = 3.0f
}
override fun draw(canvas: Canvas) {
// See github repository for CameraUtils.
// Calculate Rectangle for detected face.
val rect = CameraUtils.calculateRect(
overlay,
rect.height().toFloat(),
rect.width().toFloat(),
face.boundingBox
)
canvas.drawRect(rect, boxPaint)
}
}Camera Analyzer
BaseCameraAnalyzer.kt
abstract class BaseCameraAnalyzer<T : List<Face>> : ImageAnalysis.Analyzer {
abstract val graphicOverlay : GraphicOverlay<*>
// ovverride analyze from ImageAnalysis.Analyzer
@SuppressLint("UnsafeOptInUsageError")
override fun analyze(imageProxy: ImageProxy) {
val mediaImage = imageProxy.image
mediaImage?.let { image ->
// detect face in image
detectInImage(InputImage.fromMediaImage(image, imageProxy.imageInfo.rotationDegrees))
.addOnSuccessListener { results ->
// process face
onSuccess(results, graphicOverlay, image.cropRect)
imageProxy.close()
}
.addOnFailureListener {
onFailure(it)
imageProxy.close()
}
}
}
// use InputImage
protected abstract fun detectInImage(image : InputImage) : Task<T>
abstract fun stop()
// function that will be affected if error occurs
protected abstract fun onSuccess(
results : List<Face>,
graphicOverlay: GraphicOverlay<*>,
rect : Rect
)
// function that will be affected if error occurs
protected abstract fun onFailure(e : Exception)
}CamerAnalyzer.kt
class CameraAnalyzer(
private val overlay: GraphicOverlay<*>
) : BaseCameraAnalyzer<List<Face>>(){
override val graphicOverlay: GraphicOverlay<*>
get() = overlay
// build FaceDetectorOptions
private val cameraOptions = FaceDetectorOptions.Builder()
.setPerformanceMode(FaceDetectorOptions.PERFORMANCE_MODE_ACCURATE)
.setLandmarkMode(FaceDetectorOptions.LANDMARK_MODE_ALL)
.setClassificationMode(FaceDetectorOptions.CLASSIFICATION_MODE_ALL)
.setMinFaceSize(0.15f)
.enableTracking()
.build()
// set options in client
private val detector = FaceDetection.getClient(cameraOptions)
override fun detectInImage(image: InputImage): Task<List<Face>> {
return detector.process(image)
}
override fun stop() {
try {
detector.close()
} catch (e : Exception) {
Log.e(TAG , "stop : $e")
}
}
// add rectangle graphic and clear
override fun onSuccess(results: List<Face>, graphicOverlay: GraphicOverlay<*>, rect: Rect) {
graphicOverlay.clear()
results.forEach {
val faceGraphic = RectangleOverlay(graphicOverlay, it, rect)
graphicOverlay.add(faceGraphic)
}
graphicOverlay.postInvalidate()
}
override fun onFailure(e: Exception) {
Log.e(TAG, "onFailure : $e")
}
companion object {
private const val TAG = "CameraAnalyzer"
}
}Camera Manager
There are functions to manage camera operations on the UI.
CameraManager.kt
class CameraManager(
private val context : Context,
private val previewView : PreviewView,
private val graphicOverlay: GraphicOverlay<*>,
private val lifecycleOwner : LifecycleOwner
) {
private lateinit var cameraProvider: ProcessCameraProvider
private lateinit var preview : Preview
private lateinit var imageAnalysis: ImageAnalysis
private lateinit var camera : Camera
private var cameraExecutor : ExecutorService = Executors.newSingleThreadExecutor()
// Start Camera
fun cameraStart() {
val cameraProcessProvider = ProcessCameraProvider.getInstance(context)
cameraProcessProvider.addListener(
{
cameraProvider = cameraProcessProvider.get()
preview = Preview.Builder().build()
// ImageAnalysis
imageAnalysis = ImageAnalysis.Builder()
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.build()
.also {
it.setAnalyzer(cameraExecutor, CameraAnalyzer(graphicOverlay))
}
// Camera Selector for options.
val cameraSelector = CameraSelector.Builder()
.requireLensFacing(cameraOption)
.build()
setCameraConfig(cameraProvider, cameraSelector)
},
ContextCompat.getMainExecutor(context)
)
}
// Set Camera Config
private fun setCameraConfig(cameraProvider: ProcessCameraProvider, cameraSelector: CameraSelector) {
try {
cameraProvider.unbindAll()
camera = cameraProvider.bindToLifecycle(
lifecycleOwner,
cameraSelector,
preview,
imageAnalysis
)
preview.setSurfaceProvider(previewView.surfaceProvider)
} catch (e : Exception) {
Log.e(TAG, "setCameraConfig : $e")
}
}
// Return Camera
fun changeCamera() {
cameraStop()
cameraOption = if (cameraOption == CameraSelector.LENS_FACING_BACK) CameraSelector.LENS_FACING_FRONT
else CameraSelector.LENS_FACING_BACK
CameraUtils.toggleSelector()
cameraStart()
}
// Stop Camera
fun cameraStop () {
cameraProvider.unbindAll()
}
companion object {
private const val TAG : String = "CameraManager"
var cameraOption : Int = CameraSelector.LENS_FACING_FRONT
}
}MainActivity.kt
// Access to CameraManager
private lateinit var cameraManager: CameraManager
// Binding
private val binding by lazy { ActivityMainBinding.inflate(layoutInflater) }
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(binding.root)
// set CameraManager
cameraManager = CameraManager(
this,
binding.viewCameraPreview,
binding.viewGraphicOverlay,
this
)
askCameraPermission()
buttonClicks()
}// Button Clicks
private fun buttonClicks() {
// call changeCamera function for turn camera.
binding.buttonTurnCamera.setOnClickListener {
cameraManager.changeCamera()
}
// call cameraStop function for stop.
binding.buttonStopCamera.setOnClickListener {
cameraManager.cameraStop()
buttonVisibility(false)
}
// call cameraStartfunction for start.
binding.buttonStartCamera.setOnClickListener {
cameraManager.cameraStart()
buttonVisibility(true)
}
}
private fun buttonVisibility(forStart : Boolean) {
if (forStart) {
binding.buttonStopCamera.visibility = View.VISIBLE
binding.buttonStartCamera.visibility = View.INVISIBLE
} else {
binding.buttonStopCamera.visibility = View.INVISIBLE
binding.buttonStartCamera.visibility = View.VISIBLE
}
}onRequestPermissionsResult;
private fun askCameraPermission() {
if (arrayOf(android.Manifest.permission.CAMERA).all {
ContextCompat.checkSelfPermission(this, it) == PackageManager.PERMISSION_GRANTED
}) {
cameraManager.cameraStart()
} else {
ActivityCompat.requestPermissions(this, arrayOf(android.Manifest.permission.CAMERA), 0)
}
}
override fun onRequestPermissionsResult(
requestCode: Int,
permissions: Array<out String>,
grantResults: IntArray
) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults)
if (requestCode == 0 && ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED) {
cameraManager.cameraStart()
} else {
// if camera permission is not accepted
Toast.makeText(this, "Camera Permission Denied!", Toast.LENGTH_SHORT).show()
}
}
