A New Wavelet-Based Privatization Mechanism for Probability Distributions.

In this paper, we propose a new privatization mechanism based on a naive theory of a perturbation on a probability using wavelets, such as a noise perturbs the signal of a digital image sensor. Wavelets are employed to extract information from a wide range of types of data, including audio signals and images often related to sensors, as unstructured data. Specifically, the cumulative wavelet integral function is defined to build the perturbation on a probability with the help of this function. We show that an arbitrary distribution function additively perturbed is still a distribution function, which can be seen as a privatized distribution, with the privatization mechanism being a wavelet function. Thus, we offer a mathematical method for choosing a suitable probability distribution for data by starting from some guessed initial distribution. Examples of the proposed method are discussed. Computational experiments were carried out using a database-sensor and two related algorithms. Several knowledge areas can benefit from the new approach proposed in this investigation. The areas of artificial intelligence, machine learning, and deep learning constantly need techniques for data fitting, whose areas are closely related to sensors. Therefore, we believe that the proposed privatization mechanism is an important contribution to increasing the spectrum of existing techniques.

1. Introduction

Probability models capable of capturing the fundamental information contained in modern data, as those used for artificial intelligence [1] and big data [2], as well as models presenting unique features, have promoted derivations of novel continuous probability distributions [3,4].

Numerous and diverse approaches have been proposed over time to generate new probability or statistical distributions [5]. One of the most common approaches allows us to enhance the functionality of a base continuous cumulative distribution function (CDF). This can be achieved utilizing various transformations based on exponential, logarithmic, power, or other functions [6].

On this topic, we may refer to the so-called “families of probability distributions”, as described in [7,8]. The new probability distributions may be employed efficiently in diverse settings, as described in [9,10]. We may also refer to the work stated in [11] pointing out the importance of continuous probability distributions in the definition of various measures.

In view of the impacts of the current research on probability distributions [12], diverse applications related to the areas of artificial intelligence [1], machine learning [13], and deep learning [14] constantly require new techniques for data fitting, whose areas are closely related to sensors. Additionally, to aid in the progress of computer sciences, new approaches are welcome to expand the options of a reference probability distribution [15].

An application of probability models can be introduced by perturbing a CDF additively, similarly to how a noise perturbs the signal of a digital image sensor [16]. Surprisingly, such a strategy does not appear to have received much attention in the literature. More precisely, given a continuous CDF, one can add this function to another (the perturbation function) in such a way that the resulting function is also a continuous CDF.

To propose a manageable perturbation [17], one can employ a special, well-known function called wavelet [18,19]. Basically, such a function has a wave-like oscillation with an amplitude that starts at zero and increases or decreases before returning to zero, one or more times. Wavelets may be utilized to extract information from a wide range of data, including audio signals and images often related to sensors [20], as unstructured data. To thoroughly analyze data, wavelet sets might be used. For more information on wavelets, we refer the reader to [21,22,23]. More specifically, in [24], transients and their wavelet coefficients are modeled as mixed Laplace probability density functions (PDFs). In [25], image segmentation based on a wavelet feature descriptor and dimensionality reduction was applied to remote sensing. Thus, one could involve a wavelet function to define a valid perturbation, and then a privatized probability distribution can be obtained through theoretical and practical tools.

The main objectives of this article are to propose and derive a naive theory of an additive perturbation on a continuous probability distribution based on a wavelet approach, and to illustrate it with a sensor-related application. The use of wavelets in this probability distribution setting is original, and our findings offer up a new modeling horizon, which are examined in depth. Therefore, we offer a mathematical method for choosing a suitable probability distribution to model data by starting from some guessed-at initial probability distribution. Examples for the proposed method are also presented. For the computational experiments, we utilize a database-sensor and two related algorithms.

The rest of the article is organized as follows. Section 2 introduces the new wavelet approach. In Section 3, we discuss the choice of a perturbation for an arbitrary probability distribution. Section 4 proposes a correction for statistical moments due to the perturbation. Then, in Section 5, the generalization of the perturbation approach at further levels is presented. In Section 6, we provide an empirical application of our approach. Finally, Section 7 gives the concluding remarks.

2. Background and Wavelet Approach

Suppose we have a random variable X with a continuous CDF . Let us consider an additive (functional) perturbation, denoted as -perturbation, so that with the CDF stated in (1) being a privatized CDF.

Note that, in the expression defined in (1), the CDF of the variable X has been perturbed and a new function is obtained. However, the choice of the perturbation cannot be arbitrary because it could break the requirements to deal only with a probability distribution. The following conditions must be met by the perturbation:

;

is derivable and satisfies , where denotes the PDF related to the CDF .

The conditions (C1) and (C2) above stated guarantee that is also a CDF. This new distribution could be seen as a privatized version of the reference distribution.

To describe our new wavelet approach, some definitions need to be given. Let us begin with the mathematical definition of a wavelet.

(Wavelet function). A wavelet is a Lebesgue measurable function

On the one hand, from the expression established in (2), observe that the absolute value of is integrable over the entire real line and its result is equal to zero (0). On the other hand, in the formula stated in (3), note that the square of is also integrable over and its result is equal to one (1). Keep in mind that, in this study, we deal with compactly supported wavelets [26], that is, the closure of the set upon which the wavelet stands non-vanishing is a compact set. Specifically, if is a wavelet function, then is a compact set, and we say is a wavelet of compact support. Henceforth, we assume that support , which plays a crucial role in our proposal [21,27]. The next definition presents the notion of wavelet cumulative function in this setting.

(Wavelet cumulative function). A wavelet cumulative function is defined by

Since only compactly supported wavelets are considered, the wavelet cumulative function given in (4) can be simplified to

Thus, from the expression stated in (5), the following properties can be verified:

Note that the properties formulated in (6)–(8) are helpful. To begin with, let us deal with the uniform distribution, denoted as , whose CDF is given by , for , where , for , and , for . A mapping is proposed to bring the support [0, 1] of the uniform distribution to the support of the wavelet, that is, . Then, we propose to choose a particular perturbation according to

For the particular choice stated in (9), the new distribution defined in (1) has the same support as the original distribution, with no perturbation added. Furthermore, imposing the condition , it follows that

From the expression established in (10), we can guarantee that , for all . Therefore, the condition is assured, for all . Hence, we must determine whether is always a non-descending function or not. Thus, we examine the behavior of the corresponding PDF formulated as implying where denotes the PDF related to the CDF .

From the formulas given in (11) and (12), it follows that and , for all x, thereby proving that this is indeed a valid PDF to be considered. Then, this new PDF and its associated CDF might be visualized as a privatized version of the reference distribution, with the privatization mechanism being named wavelet perturbation. This is that we call “privatization analysis”.

As an example, let us first consider a compactly supported wavelet defined within proposed in [28] and mathematically defined as

Figure 1 shows the original distribution, that is, [0, 1], and the new distribution generated by the perturbation identified in (13).

Figure 1

Plots of: (a) a wavelet perturbation to be applied to the distribution; and (b) wavelet perturbation (— blue), uniform (- - red), and perturbed uniform (- · - orange) CDFs.

Another family of compactly supported wavelets with parameters that can be adjusted is the beta wavelet family [29]. One of the advantages of adopting beta wavelet perturbations consists of the easy replacement of shape () and scale () parameters to make the perturbation flexible. In other words, this wavelet family allows for a simple parametrization that drives the asymmetry of the resulting probability distribution. The plots of two beta wavelet perturbations are shown in Figure 2 as examples.

Figure 2

Plots of the beta wavelet perturbations: (a) ; and (b) .

Figure 3 displays perturbed uniform distributions that are generated as a result of applying the perturbations of Figure 2. This approach can be employed to introduce asymmetries in a chosen probability distribution, controlled by the beta wavelet parameter. Among the compactly supported wavelets, certainly the most used are the Daubechies (DB4) wavelets [27]. Expressions close to approximately the DB4 wavelets of any order have been proposed in [30]. Using Matlab commands, these continuous approximations were employed to plot the DB4 perturbation adapted to the distribution, denoted by , in Figure 4.

Figure 3

Plots of: (a) beta wavelet perturbations to be applied to the distribution; and (b) perturbed uniform (⋯ blue), perturbed uniform (- · - blue), and uniform (— red) CDFs.

Figure 4

Plots of: (a) a DB4 wavelet perturbation to be applied to the distribution; and (b) DB4 wavelet perturbation (— blue) and uniform (- · - red) CDFs.

3. Choosing a Perturbation for an Arbitrary Probability Distribution

Now, we offer a valid perturbation for an arbitrary CDF . For a given compactly supported wavelet with its cumulative function (see Definition 2), consider a new chosen CDF according to with

From (11) and (14), note that , and with stated in (15) given by

Then, is a valid perturbation because the condition (C1) is satisfied. In addition, we have due to , so that the condition (C2) is also satisfied, since by (16), having . Thus, any wavelet of compact support can be used to induce a different perturbation in the vicinity of the probability distribution initially assigned. From the expressions stated in (14)–(17), note that, after applying the perturbation, the resulting function is also a CDF.

In summary, given a random variable X with CDF , a perturbation can be added, which guarantees that the modified function is still a CDF around the original CDF. This new CDF, and its associated distribution, as mentioned, are privatized versions of the reference distribution using a wavelet-based privatization mechanism.

4. Moments Correction Due to the Perturbation

Based on the random variable X, the hypothesized distribution (initial or prior distribution around which the wavelet perturbation is introduced) has its k-th moment defined by providing its existence in the mathematical sense. By introducing the perturbation defined in (9), the new (adjusted/privatized) k-th moment is stated as

Consider the equation given by Then, by using the expressions given in (18) and (19), it follows that

The second term on the right side of (20) accounts for a moment correction due to the introduced wavelet perturbation.

Let us consider now the particular case of a perturbation in a (normalized) uniform distribution, that is, . To evaluate the moments of the new CDF , under the wavelet perturbation with a compact support , we have

Note that the moment of the wavelet used to build the additive perturbation also adds to the moment of the starting distribution, because

If the support set is the unit interval, that is , then the formulas stated in (21) and (22) may be utilized. In the general case, if has a support , we can build a modified (supported-normalized) wavelet defined as

Hence, we have that

Under the assumption that the integral term given in (23) vanishes, the moments of the new and hypothesized distributions coincide.

5. Generalizing the Perturbation Approach at Further Levels

In the case that a beta perturbation occurs over a distribution, it depends on its parameters and of the perturbation wavelet. Thus, it is worth rewriting, via the equations stated in  (1)–(9), that

The interpretation presented in (24) of wavelet theory (approximation + detail) can be generalized into the lines of a wavelet tree with several levels. First, we present level-1 parameters by means of

In Figure 3, we can see examples of this case. Second, we introduce level-2 LH parameters considering

An example can be provided using the parameters , and , . These parameters are similar to those employed in Figure 3. However, note that different wavelets may be selected to fit different segments of the initial distribution support. For instance, in a level-2 perturbation, the sub-level-L can use a beta wavelet, whereas the sub-level-H may employ a Mexican-hat wavelet, denoted by , as in Figure 5. The parameterization , and is used in Figure 6, with the corresponding perturbation denoted by .

Figure 5

Plots of: (a) a Mexican-hat wavelet perturbation to be applied to the distribution; and (b) Mexican-hat wavelet perturbation (— blue) and uniform (- · - red) CDFs.

Figure 6

Plots of: (a) a level-2 beta wavelet perturbation to be applied to the distribution; and (b) level-2 beta wavelet perturbation (— blue) and uniform (- · - red) CDFs.

Next, we present level-4 LL LH HL HH parameters, namely, stated as

An example of this level-4 approach is illustrated utilizing the values given by

An interpretation for this approach is considering a distinct perturbation in each quartile of the distribution such as:

First quartile driven by .

Second quartile driven by .

Third quartile driven by .

Fourth quartile driven by .

In short, the privatization mechanism allows us to perturb a probability distribution employing levels (applying a partition on the compact support), which may be very attractive when fitting data. We can use the expression stated in (25) when implementing one level, in (26) when implementing two levels, and in (27) when implementing four levels.

6. Empirical Application

Next, we apply our privatization approach to a real-world problem. An e-commerce company sells products on the Internet and wants to analyze the possibility of adding more servers or changing its most important server. By collecting daily data, we find many days in which the best server has almost all its hardware resources consumed of the time. Looking at the empirical PDF and CDF, we see that a triangular distribution, with support on the set [0, 1] and mode equal to 0.7, might represent the data well. However, when performing goodness-of-fit tests, the results tell us that a triangular distribution is not the best option. However, a “quasi-triangular” distribution could be an appropriate probability model for the random variable X that measures the daily proportion of times with full resource consumption of the best server. Among the known techniques to fit data, the privatization mechanism that we propose in this work is an excellent option to slightly perturb the triangular distribution and describe the data well. For the computational experiments, we utilize a database-sensor and two related algorithms.

Let X be a continuous variable, which is triangularly distributed, with support on the interval [0, 1], and whose mode is m, for . The PDF and CDF of X are, respectively, given by and

Now, we use the wavelet function defined in (13). Figure 7 shows the graphical plot of the CDF corresponding to X (original triangular distribution) and also the graphical plot of the privatized version that corresponds to the random variable (perturbed triangular distribution). We consider the value in the calculations carried out. Note that, in the perturbed triangular distribution, the CDF values are greater than when compared to the original triangular distribution, for values of X less than , while for values of X greater than , the opposite occurs. This behavior is due to the wavelet function employed in such an empirical application. In practice, this method is flexible allowing us to choose the most convenient wavelet to fit the data.

Figure 7

Plots of: (a) PDF and CDF of the triangular distribution; and (b) wavelet perturbation (— blue), triangular (- - red), and perturbed triangular (- · - orange) CDFs.

For the computational experiments that were carried out, a database-sensor was used. Algorithm 1 shows the steps to perturb a probability distribution with compact support. If a perturbation by levels is required, we propose Algorithm 2 as a generalization of Section 5, where the number k of levels is left to the consideration of the data analyst.

Consider a random variable X with compact support .

Select a wavelet with compact support to perturb the distribution of the previous step, with the computations being performed by a first process denoted by A that sends the generated data to a database.

State a sensor in the database that detects the entry of new data, so that, using a trigger, the sensor responds sending a copy of the stored data to a second process denoted by B.

Establish that process B receives the perturbed data and is responsible for building the CDF of the resulting distribution.

Confirm that process B generates the corresponding plots showing, between a and b, the original distribution, wavelet used, and perturbed distribution.

Select a probability distribution with compact support .

Apply a partition of k subintervals over the interval (not necessarily equispaced).

Use Algorithm 1 on the interval , for each i from 1 to k.

Perform computations to unify the results on the interval of the previous step.

Generate unified plots on the interval for the original distribution, wavelet used, and perturbed distribution.

7. Concluding Remarks

This paper has presented a new method for building an additive wavelet-based perturbation, as a privacy mechanism, to modify a given continuous probability distribution. Then, the initial guess could be perturbed as some sort of “prospecting within the ensemble of possible probability distributions around the starting distribution”.

The method we have proposed in this investigation is flexible with respect to the perturbation function that may be employed to fit the data, since different wavelets are available. A procedure was also offered to employ four different perturbations, one in each quartile of the original distribution, which can be quite attractive when fitting data. Examples of the proposed method were discussed. Computational experiments were carried out using a database-sensor and two related algorithms. Several knowledge areas can benefit from using the new method proposed in this study.

Stochastic programming, simulation studies, and multivariate analysis [31,32,33,34], among other areas of knowledge, may also benefit from the utilization of the new approach proposed in this investigation. The Internet of things, robotics, monitoring stations, telemetry, and the use of sensors are also important fields for data reading and fitting. Concrete applications via this new approach may now emerge, with an efficient configuration for the involved functions. Another benefit of this technique is its ease of implementation in any programming language. Software developers must be the first to get involved to make this technique available to data analysts. The areas of artificial intelligence, machine learning, and deep learning [35] constantly require new techniques for data fitting, whose areas are closely related to sensors. Accordingly, we think that the proposed privatization mechanism is an important contribution to increasing the spectrum of existing techniques. An avenue of future work to be considered is to provide a method that allows us to determine the most appropriate wavelet during data fitting.