text analytics

Found Out About You: Should your Organization be Passively Observing Employees?

Note: Thanks to Dr. Gordon Schmidt for the band suggestion to base this post on. In case you’re wondering the lyrics are from the Gin Blossoms song ‘Found About You’

Rumors follow everywhere you go
Like when you left and I was last to know

A trend in the HR technology space is that of passive listening or monitoring of employees. While these tools have existed for almost 10 years they have primarily been used to monitor customer comments and sentiment. This technique has gained some traction in conversations with human resources as talent becomes scarcer and retention becomes a larger issue. The technique involves evaluating employee communication (i.e. email) for sentiment to offer a real-time ‘pulse’ for employee attitudes.  

                On LinkedIn, an article about a new tool that scans employee emails and evaluates sentiment has been making the rounds. The company is called Receptiviti, a Toronto-based startup co-founded by Dr. James. W. Pennebaker, a giant in the field of natural language processing (NLP). The science behind Receptiviti is more than likely sound since it is likely based on much of Pennebaker’s seminal language work. The creators also claim that it has some ability to assess mental health. However, the idea of this tool and similar tools gives me pause.

                I shared this article with some I-O psychology colleagues. In this case Dr. Vivian Woo and Dr. Daniel Meltzer. We ended up having an interesting discussion and I’ll be referencing some comments they made in this post.

Is there a line that I could write
That's sad enough to make you cry?
All the lines you wrote to me were lies

                Organizations that would attempt to use this tool are operating under some core assumptions and like all assumptions its important to interrogate those assumptions to identify what those assumptions mean. This piece isn’t criticism of Receptiviti in particular. I have not used the tool, nor have I read anything about it other than the Quartz article. However, Receptiviti and tools like it are turning into a more viable options in the HR tech marketplace and we should think about what these tools offer and what they mean for organizational life.

Assumption 1: Employee surveys don’t contain honest feedback.

                One reason to use a passive sentiment analysis tool is because an organization may feel it doesn’t receive honest feedback from its employees. Survey researchers and organizational practitioners are aware of the potential faking problem and have suggested some solutions to it. Survey research has been the tried and true method of assessing employee satisfaction for over 70 years. Most organizational survey practitioners use reverse coded items and other techniques to assess for consistency. Natural language processing has yet to show that it measures employee sentiment more effectively than traditional surveys.

Assumption 2: Employees will consent to this use of email communication.

                The Receptiviti tool includes employee consent and reports data by gender and by department not at the individual level. Some employees are allowed to opt out which mirrors the nonresponse bias we see in traditional survey research. The consent issue is one of great interest. When I spoke with Dr. Meltzer he mentioned that like many software agreements, people rarely read informed consent forms. In fact, most people won’t read the consent form at all especially if the implication is that it’s a condition of using email which is an absolute necessity in the modern workplace. Most consent forms are challenging to read and are often ignored completely by employees. The same would most likely be true for employees. Most people cannot remember what they consent to nor do they completely understand what it is that they’ve agreed to. In addition, its your employer asking you to participate and there’s additional pressure.

                An automated email sentiment analysis tool differs from other forms of surveillance. If cameras exist in the workplace, employees don’t believe that they are used to assess them in any way. Reviewing emails for feelings has a distinctly darker tinge.

Assumption 3: The data will be consistent and reliable

                Once employees realize that their emails or other forms of communication are being analyzed by department and gender to measure sentiment there’s no guarantee what the reaction would be. Dr. Woo mentioned that there’s a possibility of gaming the system by including only positive language in email and finding alternative routes by which to communicate. Or using the system to create a sense of negativity or positivity where that wasn’t the case. Organizations will also need to develop their own dictionaries including unique acronyms and terminology so that the NLP will better understand employee communication. This would require some effort to determine how language is used in the organization and what specific terminology means. Regional issues may also arise.

In addition to the gaming the system, Dr. Woo mentioned the potential for abuse by those in power with the introduction of any new tool or system. Imagine being brought into your boss’ office for sending too many funny emails. A manager could use the tool to conduct a sentiment analysis to find out what you really think about them based on emails.

 I found out about you
I found out about you

Assumption 4: Companies will take action on real-time sentiment analysis and create healthier workplace cultures

            An issue that Reciptivi hopes to address is toxic workplace culture involving harassment and stress. This is a laudable goal and it seems that many of these tools have arisen to help organizations unover hidden activities happening within the workplace as we have seen in the #MeToo movement. However, this assumes that organizations will take action on what they find. In a piece I wrote (along with my colleagues, Dr. Zoe Zhu, Dr. Holly Jacobs, and Dr. Ranjit Nair) for Industrial Organizational Psychology: Perspectives on Science and Practice, we identified that most harassers were not punished by their organizations despite organizational policy. In a recent SHRM #NextChat, the topic was toxic culture and a consistent theme was a lack of consequences for abusers and follow through on company policy.  Ultimately, the issues around workplace culture don’t seem to be around whether an organization used a survey or not. The real issue is following through on what an organization has learned from a hotline, a survey, or even a sentiment analysis tool.

            Before investing in a sentiment analysis tool like this, make sure that your organization plans on using what you discover. If your organization doesn’t action on employee survey results will they really action on sentiment analysis? From a scientific standpoint, this idea is very cool. It’s great that Pennebaker and co. are innovating in this way. But be mindful of investing in something like this if you’re not actually going to do something with it. If your employees trust your organization enough to participate in this program it may prove useful. However, if your organization has challenges and problems be aware of unintended consequences.

The Unexpected, Unplanned Magical Mystery Tour

magical-mystery-tour.jpeg

We have been quiet here on the Talent Metrics blog, but not for lack of trying. With SIOP submissions, some recent manuscript submissions and client projects we have been busy. We just haven't had time to post on the blog. The blog is a key part of what we're about at Talent Metrics because part of our company mission is to advocate for evidence-based data driven management and people practices. A large piece of that mission is being a part of local organizations. Each of the principal consultants at Talent Metrics is involved in some local organization related to human resources, talent management and I-O Psychology. Our advocacy comes in multiple forms whether its poster presentations at SIOP or the Association for Psychological Science  or writing research articles. This has expanded to participating in podcasts and speaking engagements. 

The very cool Department 12 podcast logo! 

The very cool Department 12 podcast logo! 

 

Recently Mike and I were on the Department 12 Podcast with our pal Dr. Ben Butina. Give the podcast a listen. Ben does a wonderful job interviewing some great names in I-O and talent management, you can learn a lot. Check out Mike and I discussing Text Analytics here

moxxie logo.jpeg

Over the next two weeks Mike and I will be participating in a series of speaking engagements. On November 9th I'll be speaking with the Moxxie Mentoring Network Moxxie is a terrific "women's business community of high-achievers that provides resources, connections and experiential knowledge to peers" The organization provides young women the opportunity to connect, network, and learn from other women in business. I'll be discussing talent management strategy and employer branding. 

 

atd nyc horizontal logo.gif

On November 16th Mike and I will be presenting at the Association for Talent Development NYC I have been a member of ATD for quite a while and I currently serve on the ATD Long Island board (my local chapter!) Mike and I will facilitate a Conversation Hour about People Analytics. You can register for the event here or reach out to us by email if you have specific questions. 

Finally, on November 18th I'l be participating on a Diversity and Inclusion panel at NYU as part of a NY Organizational Psychology Student Association event. Check out the flier below for more information.

 

diversity and inclusion panel.jpg

We'd like to see you at any of these events. If you can't attend one of the above events and would like us to present feel free to reach out to us. We're always happy to visit!