caa a22 4500 463251377 CHVBK 20180405153341.0 cr unu---uuuuu 170326e20071001xx s 000 0 eng 10.1007/s10791-007-9027-7 doi (NATIONALLICENCE)springer-10.1007/s10791-007-9027-7 An empirical study of tokenization strategies for biomedical information retrieval [Elektronische Daten] [Jing Jiang, ChengXiang Zhai] Due to the great variation of biological names in biomedical text, appropriate tokenization is an important preprocessing step for biomedical information retrieval. Despite its importance, there has been little study on the evaluation of various tokenization strategies for biomedical text. In this work, we conducted a careful, systematic evaluation of a set of tokenization heuristics on all the available TREC biomedical text collections for ad hoc document retrieval, using two representative retrieval methods and a pseudo-relevance feedback method. We also studied the effect of stemming and stop word removal on the retrieval performance. As expected, our experiment results show that tokenization can significantly affect the retrieval accuracy; appropriate tokenization can improve the performance by up to 96%, measured by mean average precision (MAP). In particular, it is shown that different query types require different tokenization heuristics, stemming is effective only for certain queries, and stop word removal in general does not improve the retrieval performance on biomedical text. Springer Science+Business Media, LLC, 2007 Biomedical information retrieval nationallicence Tokenization nationallicence Stemming nationallicence Stop word nationallicence Jiang Jing Department of Computer Science, University of Illinois at Urbana-Champaign, 201 N Goodwin Ave, 61801, Urbana, IL, USA aut Zhai ChengXiang Department of Computer Science, University of Illinois at Urbana-Champaign, 201 N Goodwin Ave, 61801, Urbana, IL, USA aut Information Retrieval Springer Netherlands 10/4-5(2007-10-01), 341-363 1386-4564 10:4-5<341 2007 10 10791 https://doi.org/10.1007/s10791-007-9027-7 text/html Onlinezugriff via DOI 1 research-article jats NATIONALLICENCE 856 40 https://doi.org/10.1007/s10791-007-9027-7 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Jiang Jing Department of Computer Science, University of Illinois at Urbana-Champaign, 201 N Goodwin Ave, 61801, Urbana, IL, USA aut NATIONALLICENCE 700 1- Zhai ChengXiang Department of Computer Science, University of Illinois at Urbana-Champaign, 201 N Goodwin Ave, 61801, Urbana, IL, USA aut NATIONALLICENCE 773 0- Information Retrieval Springer Netherlands 10/4-5(2007-10-01), 341-363 1386-4564 10:4-5<341 2007 10 10791 Metadata rights reserved Springer special CC-BY-NC licence nationallicence BK010053 XK010053 XK010000 NATIONALLICENCE NATIONALLICENCE NL-springer