User evaluation of textual results clustering for web search

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Purpose: Clustering web search results into dynamic clusters and hierarchies provides a promising way to alleviate the overabundance of information typically found in ranked list search engines. This study seeks to investigate the usefulness of clustering textual results in web search by analysing the search performance and users' satisfaction levels with and without the aid of clusters and hierarchies. Design/methodology/approach: This study utilises two evaluation metrics. One is a usability test of clustering interfaces measured by users' search performances; the other is a comprehension test measured by users' satisfaction levels. Various methods were used to support the two tests, including experiments, observations, questionnaires, interviews, and search log analysis. Findings: The results showed that there was no significant difference between the ranked list and clustering interfaces, although participants searched slightly faster, retrieved a larger number of relevant pages, and were more satisfied when using the ranked list interface without clustering. Even so, the clustering interface offers opportunities for diversified searching. Moreover, the repetitive ratio of relevant results found by each participant was low. Other advantages of the clustering interface are that it highlights important concepts and offers richer contexts for exploring, learning and discovering related concepts; however, it may induce a certain amount of anxiety about missing or losing important information. Originality/value: The evaluation of a clustering interface is rather difficult, particularly in the context of the web search environment, which is used by a large heterogeneous user population for a wide variety of tasks. The study employed multiple data collection methods and in particular designed a combination of usability and comprehension tests to offer preliminary results on users' evaluation of real-world clustering search interfaces. The results may extend the understanding of search characteristics with a cluster-based web search engine, and could be used as a vehicle for further discussion of user evaluation research into this area.

Original languageEnglish
Pages (from-to)855-874
Number of pages20
JournalOnline Information Review
Volume34
Issue number6
DOIs
Publication statusPublished - 2010 Dec 24

Fingerprint

Search engines
evaluation
search engine
comprehension
evaluation research
data collection method
Experiments
performance
anxiety
questionnaire
experiment
methodology
interview
learning
Values

Keywords

  • Search engines
  • Worldwide web

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this

User evaluation of textual results clustering for web search. / Pu, Hsiao-Tieh.

In: Online Information Review, Vol. 34, No. 6, 24.12.2010, p. 855-874.

Research output: Contribution to journalArticle

@article{25271c84d6434a909827436548020946,
title = "User evaluation of textual results clustering for web search",
abstract = "Purpose: Clustering web search results into dynamic clusters and hierarchies provides a promising way to alleviate the overabundance of information typically found in ranked list search engines. This study seeks to investigate the usefulness of clustering textual results in web search by analysing the search performance and users' satisfaction levels with and without the aid of clusters and hierarchies. Design/methodology/approach: This study utilises two evaluation metrics. One is a usability test of clustering interfaces measured by users' search performances; the other is a comprehension test measured by users' satisfaction levels. Various methods were used to support the two tests, including experiments, observations, questionnaires, interviews, and search log analysis. Findings: The results showed that there was no significant difference between the ranked list and clustering interfaces, although participants searched slightly faster, retrieved a larger number of relevant pages, and were more satisfied when using the ranked list interface without clustering. Even so, the clustering interface offers opportunities for diversified searching. Moreover, the repetitive ratio of relevant results found by each participant was low. Other advantages of the clustering interface are that it highlights important concepts and offers richer contexts for exploring, learning and discovering related concepts; however, it may induce a certain amount of anxiety about missing or losing important information. Originality/value: The evaluation of a clustering interface is rather difficult, particularly in the context of the web search environment, which is used by a large heterogeneous user population for a wide variety of tasks. The study employed multiple data collection methods and in particular designed a combination of usability and comprehension tests to offer preliminary results on users' evaluation of real-world clustering search interfaces. The results may extend the understanding of search characteristics with a cluster-based web search engine, and could be used as a vehicle for further discussion of user evaluation research into this area.",
keywords = "Search engines, Worldwide web",
author = "Hsiao-Tieh Pu",
year = "2010",
month = "12",
day = "24",
doi = "10.1108/14684521011099379",
language = "English",
volume = "34",
pages = "855--874",
journal = "Online Information Review",
issn = "1468-4527",
publisher = "Emerald Group Publishing Ltd.",
number = "6",

}

TY - JOUR

T1 - User evaluation of textual results clustering for web search

AU - Pu, Hsiao-Tieh

PY - 2010/12/24

Y1 - 2010/12/24

N2 - Purpose: Clustering web search results into dynamic clusters and hierarchies provides a promising way to alleviate the overabundance of information typically found in ranked list search engines. This study seeks to investigate the usefulness of clustering textual results in web search by analysing the search performance and users' satisfaction levels with and without the aid of clusters and hierarchies. Design/methodology/approach: This study utilises two evaluation metrics. One is a usability test of clustering interfaces measured by users' search performances; the other is a comprehension test measured by users' satisfaction levels. Various methods were used to support the two tests, including experiments, observations, questionnaires, interviews, and search log analysis. Findings: The results showed that there was no significant difference between the ranked list and clustering interfaces, although participants searched slightly faster, retrieved a larger number of relevant pages, and were more satisfied when using the ranked list interface without clustering. Even so, the clustering interface offers opportunities for diversified searching. Moreover, the repetitive ratio of relevant results found by each participant was low. Other advantages of the clustering interface are that it highlights important concepts and offers richer contexts for exploring, learning and discovering related concepts; however, it may induce a certain amount of anxiety about missing or losing important information. Originality/value: The evaluation of a clustering interface is rather difficult, particularly in the context of the web search environment, which is used by a large heterogeneous user population for a wide variety of tasks. The study employed multiple data collection methods and in particular designed a combination of usability and comprehension tests to offer preliminary results on users' evaluation of real-world clustering search interfaces. The results may extend the understanding of search characteristics with a cluster-based web search engine, and could be used as a vehicle for further discussion of user evaluation research into this area.

AB - Purpose: Clustering web search results into dynamic clusters and hierarchies provides a promising way to alleviate the overabundance of information typically found in ranked list search engines. This study seeks to investigate the usefulness of clustering textual results in web search by analysing the search performance and users' satisfaction levels with and without the aid of clusters and hierarchies. Design/methodology/approach: This study utilises two evaluation metrics. One is a usability test of clustering interfaces measured by users' search performances; the other is a comprehension test measured by users' satisfaction levels. Various methods were used to support the two tests, including experiments, observations, questionnaires, interviews, and search log analysis. Findings: The results showed that there was no significant difference between the ranked list and clustering interfaces, although participants searched slightly faster, retrieved a larger number of relevant pages, and were more satisfied when using the ranked list interface without clustering. Even so, the clustering interface offers opportunities for diversified searching. Moreover, the repetitive ratio of relevant results found by each participant was low. Other advantages of the clustering interface are that it highlights important concepts and offers richer contexts for exploring, learning and discovering related concepts; however, it may induce a certain amount of anxiety about missing or losing important information. Originality/value: The evaluation of a clustering interface is rather difficult, particularly in the context of the web search environment, which is used by a large heterogeneous user population for a wide variety of tasks. The study employed multiple data collection methods and in particular designed a combination of usability and comprehension tests to offer preliminary results on users' evaluation of real-world clustering search interfaces. The results may extend the understanding of search characteristics with a cluster-based web search engine, and could be used as a vehicle for further discussion of user evaluation research into this area.

KW - Search engines

KW - Worldwide web

UR - http://www.scopus.com/inward/record.url?scp=78650322303&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78650322303&partnerID=8YFLogxK

U2 - 10.1108/14684521011099379

DO - 10.1108/14684521011099379

M3 - Article

AN - SCOPUS:78650322303

VL - 34

SP - 855

EP - 874

JO - Online Information Review

JF - Online Information Review

SN - 1468-4527

IS - 6

ER -