aNewDomain — Do your emails hit just the right spot, or do they tend to thud and sink when they land in other people’s mailboxes? IBM is now offering up the services of its Watson supercomputer — free of charge to anyone — for analyzing the “tonal” quality of emails and other written communications and giving suggestions for improving what you write. In this review, we put Watson to the test in rating a trio of emails.
Yeah, this is the same Watson (but with tons of enhancements since then) that beat world champion Garry Kasperov at chess as well as beating human players at Jeopardy way back in the 1990s. Yet like most humans, computers have their quirks, and Watson’s overall performance hasn’t been flawless.
The Demo
In an online demo, IBM lets you drop in your own text — an email, presentation, snail mail letter, lengthy report, or whatever — for “tone analysis” by Watson. Watson examines each word in the document to give color-coded ratings across three different types of tones: emotional, social and writing style. A Word Count feature displays the numbers of words associated with emotional tones like anger and cheerfulness, social tones like agreeableness and conscientiousness, and writing tones such as confident and tentative. Beyond that, Watson delivers suggestions for synonyms for often-used words in the text. All this to help you build a better doc.
The Test
In an informal test, I created three mock emails (displayed in full at the end of this article) for Watson to examine. Each is a message from a customer seeking resolution of a glitch made by a company which delivered the wrong size shoes from an order placed online. I intentionally wrote the letters so they would be progressively more angry and less agreeable. From my human perspective, the customer is reasonable in email number one, and presumably likely to get a helpful response. In email number two, the tone gets nasty, and email number three is even meaner.
The Results
For cloud-based linguistic analysis applications like this, Watson displays two sets of results. The Word Count view shows the percentage of “words per tone” (emotional, social and writing style) as well as the total number of words for each “tone trait” (such as anger and confident).
The Percentile view presents numbers drawn from comparisons between a raw output score and a preset scorecard, according to info on the site. For the online demo, IBM uses a scorecard called “Business Email.”
Well, the Percentile scores for these three emails concurred with some of my expectations.
Watson gave email number one an extremely low rating on anger (zero) and a high rating for agreeableness (85 percent). To my surprise, though, this email didn’t score high on measures such as openness (only 6 percent) and conscientiousness (27 percent).
Also as I anticipated, Watson gave emails number two and three extremely high ratings on anger (100%). On the face of things, though, it seems kind of perplexing that the computer also bestowed high marks on agreeableness to the two nasty notes (also 85 percent for each).
The Documentation
To keep myself from knowing in advance what Tone Analyzer might find, I wrote the three emails before viewing IBM’s online documentation. As it turns out, this documentation is extensive, and it helps to explain Watson’s findings by giving descriptions of what IBM means by terms such as cheerfulness, conscientiousness and confidence.
According to the documentation, Watson’s analysis of social tones are based on the Big Five personality model developed by Costa and Norman. Tone Analyzer’s emotional and writing tones, on the other hand, are derived from the Linguistic Inquiry and Word Count (LIWC) psycholinguistics dictionary (Tausczik and Pennebaker, 2010, and Pennebaker et al., 2007).
Tone Analyzer suggests improvements to docs through the use of WordNet, a lexical database for the English language made available by Princeton University.
IBM also notes that the online demo represents an experimental release of Watson Tone Analyzer service, and that the service might change in the future. IBM does not recommend the experimental release for production use.
Conclusion
The Watson demo doesn’t predict how other people will treat your emails. That’s up to you to try to figure out! It does serve up tips that might boost the quality of many of your docs.
If you think your email might come across as too much of a rant, it’s generally best to sit on things for a day or two, anyway, until you’ve chilled out.
At that point, you might want to get some analysis and recommendations from Watson.
Have fun playing with Watson online!
Here Are The Emails
Here’s the full text of the three mock emails that I created for Watson to analyze. (The incident with the shoes didn’t really happen. These are just examples of how various customers might react to an issue like that.)
Have fun hanging out with Watson online!
Email Number One
Dear Customer Support Team,
I hope that you can help me with a problem that’s cropped up concerning my purchase of a pair of shoes from XZY’s web site.
I attempted to resolve this matter with XZY over the telephone today, but I wasn’t able to reach anyone who could provide me with some answers.
Here’s a quick summary of the problem. I bought the shoes (item #125689ABC) from the site seven days ago. Although they were shipped to me immediately, and I received them within three days, the company sent me the item in the wrong size. I would now like to exchange
them as quickly as possible for the same shoe in the correct size. However, the item is no longer listed on the site. Consequently, I want to find out whether the shoE is available in my size at any of your brick-and-mortar retail store locations, and if so, whether I can bring the shipment I received to that retail outlet for an exchange.
Thank you very much for your time and consideration.
Email Number Two
Hi. I’m very annoyed at the moment over a hassle I’m having with XYZ. Since nobody at your company can manage to give me an answer over the phone, I really hope that your online support crew can do a better job!!!
For some unknown reason, XYZ has sent me a pair of shoes in size 6 instead of 7. I like the shoes, but when I checked this morning, you’d already removed the item from your web site. I just got off the phone after three fruitless and frustrating hours of trying to find somebody who’ll give me answers to a couple of simple questions. Number one, do you (OR DO YOU NOT?) have this shoe in stock in the right size at any of your retail sites? Number two, if I schlep over to a store, can I trade in the useless pair that you people delivered to me by mistake?
Please get back to me as soon as possible, or I’ll contact every consumer complaint place I can think of to tell them about your customer service runaround games.
Email Number Three
XYZ, YOU ARE A BUNCH OF JERKS!!!
First, some moron in your warehouse sends me the wrong shoes. Then some other idiot at XYZ makes the item disappear from your stupid web site. After that, I waste three hours of my precious time hanging on your phone line today, getting routed to six different so-called customer support reps and dangling on hold forever each time around. Plus, half of your staff barely speaks Engish. Are they working from grass huts somewhere? And did I mention that your phone connections are awful, too? The phone conversations got all garbled and then the phone call just…dropped.
Is your company too cheap to hire people who know what they’re doing????
I’M NEVER BUYING ANYTHING FROM XYZ AGAIN!!!!!
For aNewDomain, I’m Jacqueline Emigh.
