Examples of convolution (continuous case)

The method of convolution is a great technique for finding the probability density function (pdf) of the sum of two independent random variables. We state the convolution formula in the continuous case as well as discussing the thought process. Some examples are provided to demonstrate the technique and are followed by an exercise.

The Convolution Formula (Continuous Case)
Let X and Y be independent continuous random variables with pdfs f_X(x) and f_Y(y), respectively. Let Z=X+Y. Then the following is the pdf of Z.

\displaystyle \begin{aligned}f_Z(z)&=\int_{-\infty}^\infty f_Y(z-x) f_X(x) \ dx \\&=\int_{-\infty}^\infty f_X(z-y) f_Y(y) \ dy  \end{aligned}

Let’s look at the thought process behind the formula. Since X and Y are independent, the joint pdf of X and Y is f_{X,Y}(x,y)=f_X(x) \ f_Y(y). The pdf of Z is simply the sum of the “joint density” at the points of the line z=x+y. In Figure 1 below, every point at the line z=x+y is of the form (x,z-x). The joint density at each such point is f_X(x) \ f_Y(z-x). Summing the values of these joint density produces the probability density function of Z.

Setting the limits of the integral depends on knowing the range of possible values of x or y for a given line z=x+y. If X and Y can only take on positive values, then for a given line z=x+y, both x or y can range from 0 to z (see Example 1 below).

Example 1
Let X and Y be independent exponentially distributed variables with common density f(x)=\alpha e^{-\alpha x} where \alpha>0. Then the following is the pdf of Z=X+Y.

\displaystyle \begin{aligned}f_Z(z)&=\int_{0}^z f_Y(z-x) f_X(x) \ dx \\&=\int_{0}^z \alpha e^{-\alpha (z-x)} \alpha e^{-\alpha x} \ dx \\&=\int_{0}^z \alpha^2 e^{-\alpha z} \ dx =\alpha^2 \ z \ e^{-\alpha z}  \end{aligned}

The above pdf indicates that the independent sum of two identically distributed exponential variables has a Gamma distribution with parameters 2 and \alpha.

Example 2
Let X and Y be independent uniformly distributed variables, U(0,10) and U(0,20), respectively. The pdf of Z=X+Y is:

\displaystyle f_Z(z)=\left\{\begin{matrix} \displaystyle \frac{z}{200}&\ \ \ \ \ \ 0 \le z <10 \\{\text{ }}& \\{\displaystyle \frac{1}{20}}&\ \ \ \ \ \ 10 \le z <20 \\{\text{ }}& \\{\displaystyle \frac{30-z}{200}}&\ \ \ \ \ \ 20 \le z <30  \end{matrix}\right.

The convolution formula is applied three times. For the first case, the line x+y=z ranges in 0 \le z < 10. For each such line, we have 0<x<z. Figure 2 below is a representative diagram.

\displaystyle \begin{aligned}f_Z(z)&=\int_{0}^{z} f_Y(z-x) f_X(x) \ dx \\&=\int_{0}^{z} \frac{1}{20} \frac{1}{10} \ dx =\frac{z}{200}  \end{aligned}

For the second case, the line x+y=z ranges in 10 \le z < 20. For each such line, x ranges from 0 to 10. Figure 3 below is a representative diagram.

\displaystyle \begin{aligned}f_Z(z)&=\int_{0}^{10} f_Y(z-x) f_X(x) \ dx \\&=\int_{0}^{10} \frac{1}{10} \frac{1}{20} \ dx =\frac{1}{20}  \end{aligned}

For the third case, the line x+y=z ranges in 20 \le z < 30. Figure 4 below is a representative diagram.

\displaystyle \begin{aligned}f_Z(z)&=\int_{z-20}^{10} f_Y(z-x) f_X(x) \ dx \\&=\int_{z-20}^{10} \frac{1}{10} \frac{1}{20} \ dx =\frac{30-z}{200}  \end{aligned}

The following is the graph of the pdf of Z=X+Y.

Exercise
Suppose that X is an exponentially distributed variable with pdf f(x)=\alpha e^{-\alpha x} and Y has the uniform distribution U(0,1). Find the pdf of the independent sum Z=X+Y.

\displaystyle f_Z(z)=\left\{\begin{matrix} \displaystyle 1-e^{-\alpha z}&\ \ \ \ \ \ 0 \le z <1 \\{\text{ }}& \\{\text{ }}& \\{\displaystyle e^{-\alpha z}(e^\alpha-1)}&\ \ \ \ \ \ 1 \le z <\infty  \end{matrix}\right.

Advertisements

12 responses

  1. Student at Birkbeck College(MSc Applied Statistics) | Reply

    I have a standard Normal N (0,1), distribution, so is the method of finding the convolution integral the same method as the above examples?

    1. To find the pdf of an independent sum of normal variables, you can certainly apply the convolution method. But a better approach is to use moment generating functions (mgfs). The mgf of an independent sum is obtained by taking the product of the individual mgfs. If each individual mgf is from a normal distribution, their product has the same form as a normal mgf.

  2. Thank you for the examples, especially the second one. I’m facing an exam and had difficulties for a while to determine the boundaries of the convolution integral and this set my mind straight 🙂

  3. Hi Dan.

    Thank you for the nice explanation. By the way, I guess the answer for the exercise is not correct. If 1<z, shouldn't it be exp( -a(z-1 )?

    1. Hi, 21alive

      The answer given is correct. I just worked it out and I got \displaystyle e^{- \alpha (z-1)} - e^{- \alpha z}. I then simplified and I got \displaystyle e^{- \alpha z} (e^{\alpha} - 1).

      Hope this helps.

      Dan

  4. Thank you very much for the great example. Would it be possible to help me understand why they did not apply convolution to number 108 in the May sample problems (http://www.beanactuary.org/exams/preliminary/exams/syllabi/ExamPSamplequestions.pdf)?

    1. Hi, Renee

      Thanks for your question.

      SOA Exam P sample question #108 is essentially about finding the density function of the sum of two independent exponential distributions, one with mean 1 and the other with mean 1/2. You can actually solve this problem using convolution. However, the sample solution provided by SOA did not use convolution. But that solution is long and is no more efficient than doing convolution. So convolution is at least as good as the sample solution. However, the approach in the sample solution is worth knowing.

      Let me explain the SOA sample solution. We have S where the density is f(s)=e^{-s} and T where the density is g(t)=0.5 e^{-0.5 t}. Since S and T are independent, the following is the joint distribution of S and T.

        \displaystyle f(s,t)=\biggl(e^{-s}\biggr) \ \biggl(0.5 e^{-0.5 t}\biggr)

      Let X=S+T. We first find the cdf of X. Then take the derivative to find the pdf of X.

      To find the cdf of X, compute P(X \le x), which is accomplished by the following double integral.

        \displaystyle \begin{aligned} P(X \le x)&=\int_0^x \int_0^{x-s} e^{-s} \ 0.5 e^{-0.5 t} \ dt \ ds \\&\text{ } \\&=1-2 \ e^{-0.5 x}+e^{-x}  \end{aligned}

      After you take derivative of the above, you would get e^{-0.5 x}-e^{-x}, which is the answer.

      The above double integral is to sum the joint density f(s,t) over the following traingular region:

        \displaystyle 0<s<x, \ \ \ \ 0<t<x-s

      You can draw this region to get a sense of what you are summing. The sum of all the density values in this triangular region is the probability that X=S+T \le x.

      To do the double integral efficiently and skillfully requires some practice. If you plan on finding the density function of an independent sum by first finding the cdf of the sum and then differentiate, make sure you can take care of the required calculation in a reasonable amount of time during the exam. In fact, it should be a good practice doing both approaches.

      1. I have two questions regarding this problem,

        1) Why do you automatically assume it is the sum of one exponential with mean 1/2 and the other with mean 1? The problem states they have a common density.

        2) I am having a lot of difficulty setting up the ranges for my integration when using the cdf technique and was wondering if you could provide some insight on the best way to set these ranges.

        Thanks

  5. Thank you so much!!! That was really clear and helpful. I was trying to work through it using a convolution but I kept getting the wrong answer. I got confused because it was 2*T1 and I was not sure if it could be applied. This is what I tried http://gyazo.com/9bc3dc2a215707f70556e536ee1337ea

    1. Hi, Renee

      Any constant multiple of an exponential distribution is also exponential. So I would regard 2T_1 as the exponential distribution with mean 2 (since T_1 has mean 1). No matter what approach you use, the two densities you work with are 1 \ e^{-s} (the exponential with mean 1) and \frac{1}{2} \ e^{-\frac{1}{2} t} (the one with mean 1/2). You can then do convolution using these two density functions or do the cdf approach.

      1. Renee Campolongo

        Thank you that makes sense that T1 would have a mean of 2. Just to confirm you could not apply a convolution method for Z= 2*X+Y if it were not an exponential or geometric distribution?

      2. Convolution is a general method for finding the density of the sum of two independent random variables. So it is not just limited to exponential or geometric. Exponential distribution takes a prominent place in Exam P since the calculation involving exponential distribution is very tractable.

        Even though this may be obvious, another comment is that you should not try to apply convolution to every independent sum problem. For example the independent sum of X and Y where they share the same density function \alpha \ e^{-\alpha x} is the following.

          \displaystyle \alpha^2 \ x \ e^{-\alpha x}

        If an exam P problem hinges on this fact, you should know it. Spending exam time to derive this fact means you less time to work enough problems to pass.

        Hope this helps.

        Dan Ma

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: