Every executive I work with has been told the same thing. We summarise employee feedback with the model so leaders see the signal. The vendor pitch is uniform. We preserve the signal.
Nobody asks which parts of the signal get preserved and which get dropped. I have been asking it for a year. The room goes quiet every time.
The CPO in Zurich
Last month I sat with a chief people officer at a European asset manager in Zurich. She showed me the quarterly employee-listening summary her executive team reads. The vendor dashboard had a bright green check next to a setting labelled preserve critical and minority concerns. She was proud of the program. The response rate was high. The engagement score was up.
I asked a different question. From last quarter's raw data, which concerns did only one team raise. Of those, which made it into the summary the CEO read.
She did not know. The vendor could not tell her either. There was a sentiment distribution. There was no retention report.
That is the whole problem.
What the audit measured
Last week Thilo Tamme at TU Munich, Anton Hantel at MIT and Bijan Khosrawi-Rad at Leuphana published a voice-retention audit on a real deployment. A global professional-service company. 2,586 free-text responses across 538 project codes and 260 distinct teams. They generated 45 leader summaries with Claude-Opus-4-8 under three system prompts, including one explicitly told to retain critical and minority concerns.
The headline is uncomfortable.
Single-comment themes retained at 0.14 [95% CI 0.05, 0.31] versus multi-comment themes retained at 0.74. Roughly 86 percent of singleton concerns were dropped, and the preserve prompt failed to recover them.
A logistic model nails the mechanism. Log source-unit frequency drives retention at coefficient +1.61, p<<0.001. The critical-theme indicator adds no independent lift, +0.22, p=0.55. The filter is prevalence. Sentiment carries no weight on its own.
For a cross-lingual workforce the gap widens. German-only theme retention collapsed to 0.20 against 0.61 overall. Short comments survived at 0.27 against 0.65 for elaborated ones. Brevity in the input compounds the frequency filter.
The one-time concern is often the one that matters. The junior analyst flagging a client-relationship risk nobody else sees. The one team raising a safety issue. The person who says the thing no one else will say out loud. Those are the voices your summariser drops first. If an LLM sits between your workforce and your leadership inbox, the audit to run is per-theme retention under your production prompt. Ask the vendor for the singleton number before you cite the summary at the next board meeting as what your people are saying.