An efficient algorithm for extracting appliance-time association using smart meter data.

Demand Response (DR) programs play a significant role for developing energy management solutions. Gaining home residents trust and respecting their appliances usage preferences are essential factors for promoting these programs. Extracting resident's usage behaviour is a challenging task with the infinite massive amount of data being generated from smart meters. The main contribution of this paper is to extract temporal association patterns of energy consumption at appliance level. The proposed approach extends the Utility-oriented Temporal Association Rules Mining (UTARM) algorithm to discover appliances usage preference at a time. The results achieved from the proposed work succeeded to discover appliance-time association considering appliances usage priority as a utility factor with respect to the 24-hours of the day as a temporal partitioning factor.

Introduction

In responding to the rapid demand of energy, large deployments of smart meters are driven by governments and industries. In the United States, smart meters deployment has reached 76 million devices in 2016 and expected to reach 90 million devices by the end of 2020 according to the Institute for Electric Innovation (Cooper, 2017). In Germany, smart meters deployment has reached 47.9 million devices. The deployment in Italy has reached 36.7 million devices, 35 million devices have been deployed in France and 27.77 million devices in Spain (Zhou and Brown, 2017).

Smart meters are considered as a key component for initializing smart grid environment. It is sometimes called as “cornerstone of smart grid” (Palacios-Garcia et al., 2015). Smart meters measure the consumed power frequently based on a specified time interval and send these data to utilities. Infinite stream of data is being transmitted from smart meters. These data are time series data where the consumption patterns and usage preferences are changing continuously over time. For example, the heater is heavily used in winter while the air conditioner is used in summer. Time series data reveal that some findings may expire with time and new ones should be discovered. This introduces the paradigm of appliance-time association where an appliance usage is always associated with a certain time based on user preference. This time can be a certain hour, day, week, month or season.

Analysing these data is very promising to extract precious information and develop energy management solutions. Such analysis has a great benefit for both utilities and home residents. Utilities can develop several applications such as monitoring the consumed power, detecting faults and blackouts, making power redistribution decisions and developing Demand Response (DR) programs for clustered customers with similar consumption behaviour (Wang et al., 2018). Home residents failed to save energy in their daily activity due to the lack of knowledge of their appliances usage consumption. The only measure that residents have is their electricity bills which do not help them to understand what behaviour they are doing to make it high or low. Home residents can have a better control on their energy usage when providing them with information about it in real time. Raising their awareness will guide them to a better usage behaviour and saving energy.

Energy consumed in residential premises is related to users' activities and their behaviour of appliances usage. Residents daily life follow some routines habit which makes it possible to study and understand their preferences. For example, microwave and coffee machine are used during breakfast activity, oven and kitchen lights are used during dinner activity. Understanding residents’ behaviour and respecting their preferences will promote DR programs and guarantee the efficient use of energy. DR programs have not reached the target results for reducing energy consumption in the residential sector. One of the main limitations is the lack of knowledge of residents usage preferences (Torstensson and Wallin, 2015). Thus, residents are not motivated to respond to these programs. It is important to tailor DR programs based on these preferences. This paper addresses this challenge by extracting temporal usage patterns of home appliances by considering two essential factors: the time factor and the utility factor. The time factor is the hour at which appliances are preferred to be used. The utility factor is the priority of using an appliance during a certain hour. The basic idea for using these two factors is to enhance the quality of the discovered temporal patterns. For example, the microwave has higher priority than the laptop at 7 am during the breakfast activity.

The proposed methodology employs the Utility-oriented Temporal Association Rules Mining (UTARM) algorithm to enhance the quality of the mined temporal associations considering the utility factor (Maragatham and Lakshmi, 2015). The algorithm is applied to the UK Domestic Appliance-Level Electricity (UK-DALE) dataset (Kelly and Knottenbelt, 2015). The dataset is fine-grained collecting usage consumption on appliance level for 5 dwellings during 4.3 years starting from 2012 to 2017 with resolution of 6 seconds.

The rest of this paper is organized as follows: related work is presented in section 2. Section 3 introduces the proposed methodology. Section 4 evaluates the conducted results. Discussion is presented in section 5 and finally conclusion and future work are derived in section 6.

Related work

With the continuous deployment of smart meters over the globe and the availability of its fine-grained rich data at appliance-level, it has been the focus to analyse these data and develop solutions for energy management. A lot of research techniques have been developed to achieve various purposes such as extracting load profiles, developing scheduling algorithms for appliances, extracting usage patterns and predicting consumption patterns (Osama et al., 2017). Most of the previous work focus on associations between appliances and pay a little attention on extracting temporal patterns to discover appliance-time association.

Chen et al. (2012) developed a Time-slot Probability Usage Pattern (TPUP) algorithm that calculates the probability that an appliance will be operating at each time slot per day. The time slot used was 3 hours generating 8 slots per day. The authors generated a dataset by logging consumption data every 4 seconds using wireless power meter for six appliances; microwave, dish-washer, washer-dryer, light, oven and air-condition. The dataset holds consumption data logged for 3 years for one dwelling.

Singh and Yassine (2017) developed an algorithm extending k-means through dynamic programming approach to cluster appliances incrementally through intervals of time. The time interval used was 30 minutes so that each day consumption data was partitioned into 48 clusters and each cluster's size indicates the appliance-time association strength. The developed algorithm was applied to the UK-DALE dataset (Kelly and Knottenbelt, 2015).

Gajowniczek and Zkabkowski (2015) proposed an approach for clustering appliances usage with respect to time using agglomerative hierarchical clustering. The authors used the hour as a partitioning factor and extracted a probability distribution for each appliance per hour. The authors proposed their approach as a proof of concept for only one week and used dendogram for representing the achieved results. The dataset used hold consumption data of five appliances: washing_machine, dish_washer, tumble_dryer, kettle and microwave oven for one dwelling during 44 days. Table 1 presents a summary of the approaches used for extracting appliance-time association.

Table 1

Extracting appliance-time association approaches.

Approach	Partitioning Factor	Dataset	Duration
TPUP (Chen et al., 2012)	3 hours	Data logged from one dwelling	3 years
K-means (Singh and Yassine, 2017)	30 minutes	UK-DALE (Kelly and Knottenbelt, 2015)	3 years
Hierarchical Clustering (Gajowniczek and Zkabkowski, 2015)	1 hour	Data logged from one dwelling	44 days

As shown in Table 1, discovering appliance-time association requires determining a partitioning factor to generate a number of intervals. The reviewed approaches succeeded to find out appliances’ association level to each interval. However, the fact that residents behaviour may change with time was ignored and as a consequence discovered associations will be changed. Moreover, temporal patterns were extracted by calculating the frequency of appliances usage ignoring its weight or importance for residents.

In the proposed approach, exhibition period for each discovered association is discovered since that smart meter's data is a time series data. The main purpose for extracting exhibition periods is to identify the validity of the discovered patterns. Thereby, a behavioural history can be obtained that maintains the old discovered associations and discovers the new ones as well.

Methodology

Appliance-time association is the extraction of appliances that are preferred for home residents to be used at a certain time. This time can be an hour, a day, a week, a month or a season. Thereby, the chosen time factor is used to partition the temporal database. In this context, we have used the hour as our time factor and since the aim is to discover appliances association level to a certain hour. We have represented an appliance by its activity state as being OFF or ON, i.e., S = {0,1} and the value of power consumed is ignored.

The proposed methodology has utilized the UTARM algorithm (Maragatham and Lakshmi, 2015) for extracting appliance-time association. The basic idea of using the UTARM algorithm is to extract significant temporal associations that are highly impacted by resident's usage taking into consideration the importance of using an appliance at a time in addition to identifying the lifespan of the discovered rule. The UTARM algorithm merges two data mining tasks together which are temporal association mining and utility-oriented mining.

The temporal association mining is the process of extracting temporal association rules from a time series data. Temporal association rules are an extension of frequent items association rules with aspect of time dimension. The basic idea of the time dimension is that each association rule can be valid for a period of time sometimes called as an exhibition period or a lifespan (Ale and Rossi, 2000). Thus, discovering appliances that are frequently appearing together at specific time and lasts for an exhibition period.

The utility-oriented mining is the process of extracting frequent itemsets subjected to a weight or importance factor. Each item in the itemset has a different weight. This weight factor is expressed as a utility factor which is a function of internal and external utility measures. The Internal Utility (IU) is a quantitative value that measures the quantity of an item in an itemset. The External Utility (EU) is a value that reflects the significance of an item in an itemset (Zida et al., 2015).

The data is mined in batches of 24 hours at the end of each day. First, the raw data is preprocessed by generating a usage matrix and then the utility data is updated. Finally, the usage preference is extracted by discovering appliance-time association. The proposed methodology is illustrated in Fig. 1.

Fig. 1

The proposed methodology.

The proposed approach is achieved through two phases:

Data Preparation.

Extracting Appliance-Time Association.

Data preparation

The UK-DALE dataset (Kelly and Knottenbelt, 2015) is a real-world dataset that holds consumption logs for 5 dwellings with different durations. The readings were logged at appliance level with time granularity of 6 seconds. Each consumption log carries the power consumed in watt and its timestamp. Table 2 shows a summary of the dataset specification.

Table 2

UK-Dale dataset specification.

Dwelling	Number of Appliances	Duration	Start Date	End Date
Dwelling 1	52	4.3 years	09-11-2012	26-04-2017
Dwelling 2	18	8 months	17-02-2013	10-10-2013
Dwelling 3	4	2 months	27-02-2013	08-04-2013
Dwelling 4	5	7 months	09-03-2013	01-10-2013
Dwelling 5	24	5 months	29-06-2014	13-11-2014

The pre-processing phase of the data generated from smart meters is essential in the regard of its huge amount. In this work, the UK-DALE dataset has been pre-processed in chunks of 24-hours at the end of each day. Only one record in the database has been stored for each day holding the date and a generated usage matrix of size 24 *N where N is the number of appliances. Since our focus is on the appliances’ association with the hour, each cell in the usage matrix holds a flag of 0 or 1 indicating the activity of each appliance during the corresponding hour as an OFF or ON state respectively. The main objective from this phase is to extract only the needed data to achieve a better performance as well as to reduce the storage needed for the database.

Extracting appliance-time association

The basic idea of this work is to extract appliance-time association rules considering utility factor calculated per each hour in addition to the exhibition period for the discovered association rule. The exhibition period defines the validity of the discovered rule since residents behaviour and preferences change with time.

The Utility (U) factor is a value that reflects the weight or the importance for having an appliance in the active state at an hour. Each appliance has an internal utility and external utility. The internal utility is a value for measuring the quantity of an item in an event. Thus, it is calculated as the number of days that the appliance was recorded as active at a certain hour. The external utility is a value for measuring the importance or the priority of an item. It is calculated as the probability that an appliance is active at a certain hour all over the recorded days. Measuring the utility of an appliance is the multiplication of the internal utility and the external utility at each hour in the day.

Each partition in the transaction database has a utility value named as the Transaction Weighted Utility (TWU). Since we have used the hour as our partitioning factor then each hour has a weight value calculated as the maximum IU multiplied by the maximum EU generated by any appliance at the corresponding hour.

Infinite stream of data is being generated from smart meters. Thus, data should be processed in an incremental approach. The proposed algorithm mines the data at the end of each day in chunks of 24 hours.

First, the algorithm starts by updating the utility measures for each appliance at each hour. If the appliance has been recorded as active in an hour, the IU value will be incremented by one. The EU value will be updated as the number of the recorded days has been incremented in addition to updating the utility value of the hour.

The next step of the algorithm is to generate candidate 2-itemsets of home appliances. A candidate 2-itemset is represented by the combination of two appliances, i.e., appliance 1 (a1) and appliance 2 (a2) represents a candidate. Each candidate is processed at each hour in the day. If the candidate has been recorded as active at the hour, the exhibition period of the candidate will be updated with the date of the processed day and the frequency of the candidate will be incremented. The Frequency (FU) is the number of days having a1 and a2 are active at the hour.

The utility value for each candidate will be updated either it has been recorded as active or not. The utility of candidate 2-itemsets is the summation of the utility values for each item in the candidate. Each candidate has a Frequent Temporal Utility (FTU) that measures the support value of the candidate considering the temporal factor. It is a function of candidate 2-itemset utility value and TWU.

Algorithm 1 outlines the steps used for the proposed approach by extending the UTARM algorithm.

The UTARM algorithm identifies the validity of the association rule through its exhibition period as some rules may expire with the behavioural change of residents.

Results

In this section, a comprehensive analysis was conducted to explain our results. To the best of our knowledge, our proposed methodology is the first to consider appliances utility with respect to temporal mining. The architecture of the proposed methodology has succeeded to mine smart meters data progressively without mining the whole database whenever new data is generated.

This paper conducts the results for each house using heatmap where appliances are plotted on the y-axis and the 24-hours are plotted on the x-axis as represented in Figs. 2, 3, 4, 5, and 6. The higher the support value computed is the higher the association between appliance usage and the hour. Moreover, we have presented appliances with support values higher than 0.5 along with their exhibition periods for each house.

Fig. 2

House 1 appliance-time association.

Fig. 3

House 2 appliance-time association.

Fig. 4

House 3 appliance-time association.

Fig. 5

House 4 appliance-time association.

Fig. 6

House 5 appliance-time association.

The results of house 1 are represented in Fig. 2.

It is noted that the solar_thermal_pump is highly used at the afternoon and this is because the sunlight exists mostly at this time but it is not active at the evening hours. Also, some appliances are associated together at the same hours for example the usage of amp_livingroom, subwoofer_livingroom and livingroom_lamp_tv are having similar support values. Moreover, samsung_charger and bedroom_chargers are used together at the evening.

Table 3 represents a sample of house 1 appliances support values with respect to hour and their exhibition periods.

Table 3

Sample of house 1 appliance-time association.

Appliance	FTU	Hour	From	To
boiler	0.84–0.89	0–23	1-1-2013	26-4-2017
solar_thermal_pump	0.58	12	1-1-2013	26-4-2017
solar_thermal_pump	0.70	13	1-1-2013	26-4-2017
solar_thermal_pump	0.74	14	1-1-2013	26-4-2017
solar_thermal_pump	0.65	15	1-1-2013	26-4-2017
tv	0.57	22	1-1-2013	25-4-2017
tv	0.51	23	1-1-2013	25-4-2017
kitchen_lights	0.53	10	1-1-2013	26-4-2017
kitchen_lights	0.57	19	1-1-2013	25-4-2017
kitchen_lights	0.56	20	1-1-2013	25-4-2017
htpc	0.60	21	1-1-2013	25-4-2017
htpc	0.77	22	1-1-2013	25-4-2017
htpc	0.77	23	1-1-2013	25-4-2017
fridge	0.98–1.00	0–23	1-1-2013	26-4-2017
amp_livingroom	0.59	22	1-1-2013	25-4-2017
amp_livingroom	0.53	23	1-1-2013	25-4-2017
adsl_router	0.97–0.98	0–23	1-1-2013	24-3-2017
lighting_circuit	0.54	10	13-3-2013	26-4-2017
lighting_circuit	0.54	19	12-3-2013	26-4-2017
lighting_circuit	0.56	20	12-3-2013	25-4-2017
subwoofer_livingroom	0.54	22	12-3-2013	25-4-2017
gas_oven	0.91	0–23	14-3-2013	26-4-2017
data_logger_pc	0.94	0–23	14-3-2013	26-4-2017

It is noted that solar_thermal_pump has high support values at the afternoon hours from 12:00 to 15:00.

The results of house 2 are represented in Fig. 3.

It is obvious that speakers, server, router and modem are the most used appliances during the 24-hours revealing that this house could be an office. This observation proves that household characteristics such as house type, number of residents and their ages can be extracted from appliances usage.

Table 4 represents a sample of house 2 appliances support values with respect to hour and their exhibition periods.

Table 4

Sample of house 2 appliance-time association.

Appliance	FTU	Hour	From	To
laptop	0.56	22	17-2-2013	9-10-2013
laptop	0.60	23	17-2-2013	9-10-2013
monitor	0.51	22	17-2-2013	9-10-2013
monitor	0.55	23	17-2-2013	9-10-2013
speakers	0.97–1.00	0–23	17-2-2013	10-10-2013
server	0.97–1.00	0–23	17-2-2013	10-10-2013
router	0.97–1.00	0–23	18-2-2013	10-10-2013

It is noted that the laptop and monitor are mostly used at hours 22 and 23 while the speakers, server and router are used during the 24-hours.

The results of house 3 are represented in Fig. 4.

It is noted that the laptop was highly used at hours 1, 2 and 3 while the kettle usage was not associated to a specific hour. Although the kettle was logged as active at hour 22 in the database, it was not extracted in the association since its usage was not frequent.

Table 5 represents a sample of house 3 appliances support values with respect to hour and their exhibition periods.

Table 5

Sample of house 3 appliance-time association.

Appliance	FTU	Hour	From	To
laptop	1	1	12-3-2013	26-3-2013
laptop	1	2	12-3-2013	26-3-2013
laptop	1	3	12-3-2013	27-3-2013
laptop	0.75	4	12-3-2013	12-3-2013
laptop	0.64	5	12-3-2013	28-3-2013
laptop	0.64	6	12-3-2013	12-3-2013
laptop	0.64	7	12-3-2013	12-3-2013
laptop	0.64	8	12-3-2013	4-4-2013
laptop	1	9	12-3-2013	3-4-2013
laptop	0.64	10	12-3-2013	27-3-2013
projector	1	0	26-3-2013	2-4-2013
projector	0.84	1	13-3-2013	26-3-2013
projector	1	23	25-3-2013	3-4-2013

It is noted that the projector usage was affected by a change in residents’ preference. From 13-3 until 26-3 it was preferred to be used at hour 1 then starting from 26-3 until 2–4 the projector usage was preferred at hour 0 and this proves the importance of extracting the association exhibition period as usage preference change with time.

The results of house 4 are represented in Fig. 5.

It is noted that the freezer and the gas_boiler are always active during the 24-hours but with different support values and this is common for appliances with thermostat component that switches their activity status based on sensing its state.

Table 6 represents a sample of house 4 appliances support values with respect to hour and their exhibition periods.

Table 6

Sample of house 4 appliance-time association.

Appliance	FTU	Hour	From	To
tv_dvd_digibox_lamp	0.96–1.00	0–23	10-3-2013	1-10-2013
gas_boiler	0.96–1.00	0–23	10-3-2013	1-10-2013
freezer	0.96–1.00	0–23	10-3-2013	1-10-2013

It is observed that freezer and gas_boiler support values has high range from 0.96 to 1.00 during the 24-hours.

The results of house 5 are represented in Fig. 6.

It is noted that appliances with thermostat component like the fridge_freezer is active during the 24-hours. This is the same observation noted before for the freezer and the gas_boiler in house 4 and for the fridge in house 1.

Table 7 represents a sample of house 5 appliances support values with respect to hour and their exhibition periods.

Table 7

Sample of house 5 appliance-time association.

Appliance	FTU	Hour	From	To
i7_desktop	0.80	12	30-6-2014	13-11-2014
i7_desktop	0.81	13	30-6-2014	13-11-2014
i7_desktop	0.76	14	30-6-2014	13-11-2014
i7_desktop	0.79	15	30-6-2014	13-11-2014
i7_desktop	0.80	16	30-6-2014	13-11-2014
i7_desktop	0.80	17	30-6-2014	13-11-2014
i7_desktop	0.77	18	29-6-2014	13-11-2014
i7_desktop	0.78	19	29-6-2014	13-11-2014
i7_desktop	0.72	20	29-6-2014	12-11-2014
i7_desktop	0.69	21	29-6-2014	12-11-2014
i7_desktop	0.66	22	29-6-2014	12-11-2014
i7_desktop	0.56	23	29-6-2014	12-11-2014
core2_server	0.97–1.00	0–23	29-6-2014	13-11-2014
fridge_freezer	0.97–1.00	0–23	29-6-2014	13-11-2014
microwave	0.78–0.81	0–23	8-7-2014	12-11-2014
washer_dryer	0.72–0.76	0–23	30-6-2014	13-11-2014

It is observed that i7_desktop is preferred to be used starting from 12:00 until 23:00 with peak support hours from 12:00 to 20:00.

Discussion

The proposed work succeeded to extract temporal patterns for each home appliance. The results prove that the priority of appliances usage differ with respect to hour. Moreover, residents behaviour change over the time so some findings may expire. In this regard, association findings should have an exhibition period that reveals its validity. The achieved results conducted the following observations:

Appliances with thermostat component such as fridge and freezer are always active during the 24 hours and are not controlled by home residents.

Appliances that are solar powered will be active during the morning hours.

Appliances with similar usage patterns reveal inter-appliance associations indicating the preference of being used together (Osama et al., 2019).

Household characteristics can be discovered from appliances usage as observed in house 2. Server, modem, router and speakers are always active in the 24 hours. In addition, laptop and monitor are logged active starting from 9 am till midnight. These consumption patterns may reveal that this house may be an office.

Behavioural change can be detected by examining exhibition periods of the discovered associations as observed in house 3.

Extracting appliance-time associations can achieve promising results if integrated with DR programs through Home Energy Management Systems (HEMS). DR programs develop dynamic pricing strategies and expect home residents to lower their consumption usage at critical time pricing during peak hours. HEMS receive data from the utilities about the expected demand. Then, HEMS respond to utilities by lowering the total consumption to be below a certain limit. This is achieved by scheduling home appliances to operate during the off-peak hours instead. However, that may affect home residents comfort level and demotivate them to respond to DR programs as a consequence. This issue is addressed by the proposed work by extracting their preferences per each hour of the day. Based on the extracted associations, HEMS can schedule home appliances by considering the weight of its usage to home residents at the action time. As an example considering the conducted results of house 3, laptop and projector appliances are logged as frequent active appliances at hour 1 however the laptop has higher utility value than the projector. If utilities send to HEMS to lower the demand. Then, HEMS should respond to utilities by giving the priority to the laptop to be on then the projector based on the provided limit.

Conclusion

Understanding and respecting home residents’ preferences are crucial factors for gaining their trust and promoting DR programs. The tremendous stream of data being generated from smart meters makes the mining task very challenging. The work proposed in this paper succeeded to extract appliances association with respect to time using the UTARM algorithm. The association is measured based on two factors: temporal factor which is the hour and utility factor which is the probability of using an appliance at the hour.

This paper presents home residents preferences for using their appliances at a time and indicates that home resident's usage behaviour follow some daily routine patterns. Hence, DR programs can be tailored based on these preferences for gaining their trust and motivating them for using energy efficiently. The results achieved can be integrated with HEMS for scheduling appliances based on the extracted preferences.

In the future work, our proposed approach can be extended to extract appliance-time association taking into account the value of power consumed.

Declarations

Author contribution statement

Sarah Osama: Conceived and designed the experiments; Performed the experiments; Wrote the paper.

Marco Alfonse: Analyzed and interpreted the data.

Abdel-Badeeh M. Salem: Contributed reagents, materials, analysis tools or data.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

Data associated with this study has been deposited at http://jack-kelly.com/data/.

Algorithm 1 Extracting Appliance-Time Association
Require: minimum support minsup
Ensure: Utility-Oriented Temporal Association Rules
1:	for eachhour (h) in 24-hoursdo
2:	for eachappliance (a) in dwelling appliancesdo
3:	ifa.isActivehthen
4:	a.IUh= a.IUh+ 1
5:	end if
6:	a.EUh= active days count/total days count
7:	a.Uh= a.IUh* a.EUh
8:	end for
9:	TWUh= max (IUh) * max (EUh)
10:	end for
11:	candidate2 ← generate candidate 2-itemsets
12:	for eachitem (a1, a2) in candidate2do
13:	for eachhour (h) in 24-hoursdo
14:	ifitem.isActivehthen
15:	ifitem.startAthis nullthen
16:	item.startAth= date of the day
17:	end if
18:	item.endAth= date of the day
19:	item.FUh= item.FUh+ 1
20:	end if
21:	item.U^h=(IUa1h∗EUa1h)+(IUa2h∗EUa2h)
22:	item.FTU^h=(FU^h∗Ua1,a2h)/(2∗TWU^h)
23:	ifitem.FTUh> minsupthen
24:	a1.FTU^h=(FUa1h∗Ua1h)/(TWU^h)
25:	a2.FTU^h=(FUa2h∗Ua2h)/(TWU^h)
26:	end if
27:	end for
28:	end for