Improving context-based medical image retrieval by incorporating semantic-based retrieval

The rapid increasing amount of medical images has led to demands for more effective and efficient medical image retrieval technology. In recent years, context-based retrieval has begun to attract more research interest due to the limitation of current content-based technology. Traditional text retrieval methods determine relevance of documents based on term matching, and suffer from the severe ambiguity problem existed in biomedical domain. This paper proposes to combine semantic-based retrieval with traditional text-based retrieval for context-based medical image retrieval. In our approach, each query or document has a text-based representation and a concept-based representation. For semantic-based retrieval, semantic similarity measure is used to comparing query concepts and document concepts, and asymmetric similarity measures are also proposed by modifying the existing symmetric measures. Then the inter-concept similarities are aggregated to compute the relevance score of a document. Finally, this semantic-based retrieval is combined with text-based retrieval. Our approach is evaluated on ImageCLEFmed 2010 dataset, which contains more than 77,000 images and their captions from online medical journals. The experimental results indicate incorporating semantic-based retrieval can improve the performance of context-based medical image retrieval, and our asymmetric semantic similarity measures can achieve better MAP than symmetric ones.