Why negation detection?

The recognition of negations is an important task in medical text mining: during the treatment of patients many examinations are carried out and both the presence and absence of symptoms and conspicuous findings are documented. The following are examples of negated findings.

• Patient shows no meningism, normal light reaction, no indication of scleral icterus, oral mucosa and pharyngeal ring are irritation-free and inconspicuous, thyroid gland not enlarged, nausea did not occur.

Due to the complexity of human language there is the particular challenge of e.g. double negations, pseudo-negations etc. In the following examples, findings are not negated, although there are clear indicator words (“inconspicuous”, “not”, “excluded”) in the same sentence.

• Apart from a slight skin rash, the physical examination not inconspicuous.
• A tumor cannot be excluded with certainty.
• Spleen and liver not palpable due to ascites.

Machine learning forms the basis for improved negation detection

In rule-based approaches, these complex patterns are recognized via a wealth of rules. Here Averbis uses the text mining rule engine UIMA RUTA, which is developed by Averbis and made available to the community open-source. Many negation patterns can already be reliably recognized using rules.

Internal analyses have shown that the reliable detection of complex negation patterns leads to an enormous complexity in the rules and the maintainability suffers greatly from this. For this reason, we have added a machine learning approach to this procedure in recent weeks. For this purpose, we have compiled and annotated a large internal training data set consisting of English and German training data. Additionally, we used data from the public i2b2-2010 dataset.

For each diagnosis, the ML model gets information about the words before and after each diagnosis, whether the diagnosis is in an enumeration and whether a negation indicator such as “none” or “not” is present. The rest of the decision making process is fully automated. We have improved individual errors of the ML model by applying specific post-processing rules.

Negation status of diagnoses can be determined with high accuracy

Fortunately, we were able to increase the performance of the negation detection to 96% F1 score (English) and 95% (German) by combining the rule-based and ML method. An in-depth analysis of the errors showed that the ML approach was also able to identify faulty annotations in the gold standard.