Concept shift detection for frequent itemsets from sliding windows over data streams

Jia Ling Koh, Ching Yi Lin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Citations (Scopus)

Abstract

In a mobile business collaboration environment, frequent itemsets analysis will discover the noticeable associated events and data to provide important information of user behaviors. Many algorithms have been proposed for mining frequent itemsets over data streams. However, in many practical situations where the data arrival rate is very high, continuous mining the data sets within a sliding window is unfeasible. For such cases, we propose an approach whereby the data stream is monitored continuously to detect any occurrence of a concept shift. In this context, a "concept-shift" means a significant number of frequent itemsets in the up-to-date sliding window are different from the previously discovered frequent itemsets. Our goal is to detect the notable changes offrequent itemsets according to an estimated changing rate of frequent itemsets without having to perform mining of the frequent itemsets at every time point. Consequently, for saving the computing costs, it is triggered to discover the complete set of new frequent itemsets only when any significant change is observed. The experimental results show that the proposed method detects concept shifts of frequent itemsets both effectively and efficiently.

Original languageEnglish
Title of host publicationDatabase Systems for Advanced Applications - DASFAA 2009 International Workshops
Subtitle of host publicationBenchmarX, MCIS, WDPP, PPDA, MBC, PhD
Pages334-348
Number of pages15
DOIs
Publication statusPublished - 2009 Sep 28
EventInternational Workshops on Database Systems for Advanced Applications, DASFAA 2009: BenchmarX, MCIS, WDPP, PPDA, MBC, PhD - Brisbane, QLD, Australia
Duration: 2009 Apr 202009 Apr 23

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5667 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherInternational Workshops on Database Systems for Advanced Applications, DASFAA 2009: BenchmarX, MCIS, WDPP, PPDA, MBC, PhD
CountryAustralia
CityBrisbane, QLD
Period09/4/2009/4/23

Fingerprint

Frequent Itemsets
Sliding Window
Data Streams
Mining
Costs
Industry
Mean Shift
Concepts
User Behavior
Computing
Experimental Results

Keywords

  • Change Detection
  • Data Streams
  • Frequent Itemsets

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Koh, J. L., & Lin, C. Y. (2009). Concept shift detection for frequent itemsets from sliding windows over data streams. In Database Systems for Advanced Applications - DASFAA 2009 International Workshops: BenchmarX, MCIS, WDPP, PPDA, MBC, PhD (pp. 334-348). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5667 LNCS). https://doi.org/10.1007/978-3-642-04205-8_28

Concept shift detection for frequent itemsets from sliding windows over data streams. / Koh, Jia Ling; Lin, Ching Yi.

Database Systems for Advanced Applications - DASFAA 2009 International Workshops: BenchmarX, MCIS, WDPP, PPDA, MBC, PhD. 2009. p. 334-348 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5667 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Koh, JL & Lin, CY 2009, Concept shift detection for frequent itemsets from sliding windows over data streams. in Database Systems for Advanced Applications - DASFAA 2009 International Workshops: BenchmarX, MCIS, WDPP, PPDA, MBC, PhD. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5667 LNCS, pp. 334-348, International Workshops on Database Systems for Advanced Applications, DASFAA 2009: BenchmarX, MCIS, WDPP, PPDA, MBC, PhD, Brisbane, QLD, Australia, 09/4/20. https://doi.org/10.1007/978-3-642-04205-8_28
Koh JL, Lin CY. Concept shift detection for frequent itemsets from sliding windows over data streams. In Database Systems for Advanced Applications - DASFAA 2009 International Workshops: BenchmarX, MCIS, WDPP, PPDA, MBC, PhD. 2009. p. 334-348. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-04205-8_28
Koh, Jia Ling ; Lin, Ching Yi. / Concept shift detection for frequent itemsets from sliding windows over data streams. Database Systems for Advanced Applications - DASFAA 2009 International Workshops: BenchmarX, MCIS, WDPP, PPDA, MBC, PhD. 2009. pp. 334-348 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{7b0705977de94232891acbad9296163e,
title = "Concept shift detection for frequent itemsets from sliding windows over data streams",
abstract = "In a mobile business collaboration environment, frequent itemsets analysis will discover the noticeable associated events and data to provide important information of user behaviors. Many algorithms have been proposed for mining frequent itemsets over data streams. However, in many practical situations where the data arrival rate is very high, continuous mining the data sets within a sliding window is unfeasible. For such cases, we propose an approach whereby the data stream is monitored continuously to detect any occurrence of a concept shift. In this context, a {"}concept-shift{"} means a significant number of frequent itemsets in the up-to-date sliding window are different from the previously discovered frequent itemsets. Our goal is to detect the notable changes offrequent itemsets according to an estimated changing rate of frequent itemsets without having to perform mining of the frequent itemsets at every time point. Consequently, for saving the computing costs, it is triggered to discover the complete set of new frequent itemsets only when any significant change is observed. The experimental results show that the proposed method detects concept shifts of frequent itemsets both effectively and efficiently.",
keywords = "Change Detection, Data Streams, Frequent Itemsets",
author = "Koh, {Jia Ling} and Lin, {Ching Yi}",
year = "2009",
month = "9",
day = "28",
doi = "10.1007/978-3-642-04205-8_28",
language = "English",
isbn = "364204204X",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "334--348",
booktitle = "Database Systems for Advanced Applications - DASFAA 2009 International Workshops",

}

TY - GEN

T1 - Concept shift detection for frequent itemsets from sliding windows over data streams

AU - Koh, Jia Ling

AU - Lin, Ching Yi

PY - 2009/9/28

Y1 - 2009/9/28

N2 - In a mobile business collaboration environment, frequent itemsets analysis will discover the noticeable associated events and data to provide important information of user behaviors. Many algorithms have been proposed for mining frequent itemsets over data streams. However, in many practical situations where the data arrival rate is very high, continuous mining the data sets within a sliding window is unfeasible. For such cases, we propose an approach whereby the data stream is monitored continuously to detect any occurrence of a concept shift. In this context, a "concept-shift" means a significant number of frequent itemsets in the up-to-date sliding window are different from the previously discovered frequent itemsets. Our goal is to detect the notable changes offrequent itemsets according to an estimated changing rate of frequent itemsets without having to perform mining of the frequent itemsets at every time point. Consequently, for saving the computing costs, it is triggered to discover the complete set of new frequent itemsets only when any significant change is observed. The experimental results show that the proposed method detects concept shifts of frequent itemsets both effectively and efficiently.

AB - In a mobile business collaboration environment, frequent itemsets analysis will discover the noticeable associated events and data to provide important information of user behaviors. Many algorithms have been proposed for mining frequent itemsets over data streams. However, in many practical situations where the data arrival rate is very high, continuous mining the data sets within a sliding window is unfeasible. For such cases, we propose an approach whereby the data stream is monitored continuously to detect any occurrence of a concept shift. In this context, a "concept-shift" means a significant number of frequent itemsets in the up-to-date sliding window are different from the previously discovered frequent itemsets. Our goal is to detect the notable changes offrequent itemsets according to an estimated changing rate of frequent itemsets without having to perform mining of the frequent itemsets at every time point. Consequently, for saving the computing costs, it is triggered to discover the complete set of new frequent itemsets only when any significant change is observed. The experimental results show that the proposed method detects concept shifts of frequent itemsets both effectively and efficiently.

KW - Change Detection

KW - Data Streams

KW - Frequent Itemsets

UR - http://www.scopus.com/inward/record.url?scp=70349317040&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70349317040&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-04205-8_28

DO - 10.1007/978-3-642-04205-8_28

M3 - Conference contribution

AN - SCOPUS:70349317040

SN - 364204204X

SN - 9783642042041

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 334

EP - 348

BT - Database Systems for Advanced Applications - DASFAA 2009 International Workshops

ER -