An entropy-based query expansion approach for learning researchers' dynamic information needs

Literature survey is one of the most important steps in the process of academic research, allowing researchers to explore and understand topics. However, novice researchers without sufficient prior knowledge lack the skills to determine proper keywords for searching topics of choice. To tackle this problem, we propose an entropy-based query expansion with a reweighting (E-QE) approach to revise queries during the iterative retrieval process. We designed a series of experiments that consider the researcher's changing information needs during task execution. Three topic change situations are considered in this work: minor, moderate and dramatic topic changes. The simulation-based pseudo-relevance feedback technique is applied during the search process to evaluate the effectiveness of the proposed approach without the intervention of human effort. We measured the effectiveness of the TFIDF and E-QE approaches for different types of topic change situations. The results show that the proposed E-QE approach achieves better search results than the TFIDF, helping researchers to revise queries. The results also confirm that the E-QE approach is effective when considering the relevant and irrelevant pages during the relevance feedback process at different levels of topic change.