Non-verbal communication-based emotion detection in online interviews

Abstract

The outcome of interviews is strongly influenced by the behavior of the interviewed candidate. In some situations, the answers given by the candidate do not reflect the true feelings or the actual potential of the candidate for the interviewed job position. Usually in interviews, it is not possible to use extra devices to detect the confidence level of the given answers. The situation has become very critical in online interviews, which are happening in abundance at present. In online interviews, the candidate faces the interview panel virtually, limiting the opportunity of the panel to observe facial expressions, body language and other soft skills of the candidate. Analyzing the required soft skills for a particular job is a good parameter in the long term for a candidate to hold on to the job. Hence, selecting the most suitable candidate who is good ‘on paper’ as well as good ‘in person’ is very challenging. This research proposes a method of identifying emotions of candidates based on a captured video sequence of the candidate during the interview to identify the evoked emotional states and the characteristic expressions of the human face. The primary objective of an emotion recognition system is to interpret the input signals from different modalities and use them to convey information about the interpreted emotion. The system consists of four main modules: image pre-processing, feature extraction, identification of feature occurrences and intensities, and classifying emotions. Based on an emotion model, the system will be able to predict the confidence level of the candidate. Thus, it is expected that fine-grained speaker-specific continuous emotion recognition system developed in this research will help the interview panels to identify the most suitable candidate in online interviews. Emotions will alter the message significantly: often, it is not what has been said that is most relevant, nor how it has been said. Faces tend to be the most obvious means of contact between feelings. However, as opposed to the voice and other modes of speech, they are often easily manipulated in reaction to various social circumstances.