Solution Manual for Adaptive Filter Theory, 5th Edition

Name: Solution Manual for Adaptive Filter Theory, 5th Edition
Brand: Academic Documents Platform
Price: 15 USD
Availability: InStock
Author: Seller Steve

Solution Manual for Adaptive Filter Theory, 5th Edition provides you with expert textbook solutions that ensure you understand every concept thoroughly.

Seller Steve

Contributor

4.0

4 months ago

Preview (16 of 423)

Solution Manual
for
Adaptive Filter Theory 5e
Created by
Simon Haykin
and
Kelvin Hall
McMaster University
HAMILTON, ONTARIO
CANADA

Table of Contents
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P ii
Corrections for Question-Descriptions in the Textbook . . . . . . . . . . . . . . P iv
Notes on Computer Simulations and Provided Programs . . . . . . . . . . . . P vi
Chapter 1: Stochastic Processes and Models . . . . . . . . . . . . . . . . . . . . . . . . P 1
Chapter 2: Wiener Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 21
Chapter 3: Linear Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 48
Chapter 4: Method of Steepest Descent . . . . . . . . . . . . . . . . . . . . . . . . . . P 102
Chapter 5: Method of Stochastic Gradient Descent . . . . . . . . . . . . . . . . P 120
Chapter 6: The Least-Mean-Square(LMS) Algorithm . . . . . . . . . . . . . . P 128
Chapter 7: Normalized Least-Mean-Square(LMS) Algorithm
and Its Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 167
Chapter 8: Block-Adaptive Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 178
Chapter 9: Method of Least-Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 190
Chapter 10: The Recursive Least-Squares (RLS) Algorithm . . . . . . . . P 214
Chapter 11: Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 229
Chapter 12: Finite-Precision Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 243
Chapter 13: Adaptation in Nonstationary Environments . . . . . . . . . . . . P 251
Chapter 14: Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 304
Chapter 15: Square-Root Adaptive Filtering Algorithms . . . . . . . . . . . P 324
Chapter 16: Order-Recursive Adaptive Filtering Algorithm . . . . . . . . P 341
Chapter 17: Blind Deconvolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 380
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P 394
iii

Notes on the Computer Simulations and Provided Programs
The computer experiments completed for this solutions manual were completed al-
most exclusively using Matlab®, for ease of readability. To improve the accessibility of
the solutions, to the users of this manual, specialized signal processing toolkits were not
used in programs included. Graphical solutions are provided along with the .m files in
case the user of the textbook is interested in completing the exercises in a different pro-
gramming language, in which case a graphical solution is available for comparison.
The solutions of the computer problems in Chapter 13 were completed by Ashique
Rupam Mahmood, Computer Science, University of Alberta, the creator of the Autostep
algorithm. The solutions being completed prior to the rest of the manual are only avail-
able in the programming language python, which is similar to Matlab® and therefore quite
readable.
vi

Chapter 1
Problem 1.1
Let
ru(k) = E[u(n)u∗(n − k)] (1)
ry(k) = E[y(n)y∗(n − k)] (2)
we are given that
y(n) = u(n + a) − u(n − a) (3)
Hence, substituting Equation (3) into Equation (2), and then using Equation (1), we get
ry(k) =E[(u(n + a) − u(n − a))(u∗(n + a − k) − u∗(n − a − k))]
=2ru(k) − ru(2a + k) − ru(−2a + k)
Problem 1.2
We know that the correlation matrix R is Hermitian; that is to say that
RH = R
Given that the inverse matrix R−1 exists, we may write
R−1RH = I
where I is the identity matrix. Taking the Hermitian transpose of both sides:
RR−H = I
1

PROBLEM 1.3. CHAPTER 1.
Hence,
R−H = R−1
That is, the inverse matrix R−1 is Hermitian.
Problem 1.3
For the case of a two-by-two matrix, it may be stated as
Ru = Rs + Rν
=
[r11 r12
r21 r22
]
+
[σ2 0
0 σ2
]
=
[r11 + σ2 r12
r21 r22 + σ2
]
For Ru to be nonsingular, we require
det(Ru) 6 = 0
(r11 + σ2)(r22 + σ2) − r12r22 6 = 0
With r12 = r21 for real data, this condition reduces to
(r11 + σ2)(r22 + σ2) − r2
12 6 = 0
Since this is a quadratic in σ2, we may impose the following conditions on σ2 for nonsin-
gularity of Ru:
σ2 6 = 1
2(r11 + r22)
(√
1 − 4∆r
(r11 + r22)2 − 1
)
where ∆r = r11r22 − r2
12
Problem 1.4
We are given
R =
[1 1
1 1
]
2

Loading page 6...

PROBLEM 1.5. CHAPTER 1.
This matrix is positive definite because it satisfies the condition:
aT R a = [a1 a2
] [1 1
1 1
] [a1
a2
]
=a2
1 + 2a1a2 + a2
2
=(a1 + a2)2 > 0 for all nonzero values of a1 and a2
But the matrix R is singular because:
det(R) = (1)2 − (1)2 = 0
Hence, it is possible for a matrix to be both positive definite and singular at the same time.
Problem 1.5
a)
RM +1 =
[r(0) rH
r RM
]
(1)
Let
R−1
M +1 =
[a bH
b CM
]
(2)
wherea, b and C are to be determined. Multiply Equation (1) by Equation (2):
IM +1 =
[r(0) rH
r RM
] [a bH
b C
]
Where IM +1 is the identity matrix. Therefore,
r(0)a + rH b = 1 (3)
ra + RM b = 0 (4)
rbH + RM C = IM (5)
r(0)bH + rH C = 0 (6)
Equation (4) can be rearranged to solve for b as:
b = −R−1
M ra (7)
3

Loading page 7...

PROBLEM 1.5. CHAPTER 1.
Hence, from equations (3) and (7):
a = 1
r(0) − rH R−1
M r (8)
Correspondingly,
b = − R−1
M rrH R−1
M
r(0) − rH R−1
M r (9)
From Equation (5):
C = R−1
M − R−1
M rbH
C = R−1
M + R−1
M rrH R−1
M
r(0) − rH R−1
M r (10)
As a check, the results of Equations (9) and (10) should satisfy Equation (6)
r(0)bH + rH C = − r(0)rH R−1
M
r(0) − rH R−1
M r + rH R−1
M + rH R−1
M rrH R−1
M
r(0) − rH R−1
M R
= 0
We have thus shown that
R−1
M +1 =
[0 0
0 R−1
M
]
+ a
[ 1 −rH R−1
M
R−1
M r R−1
M rrH R−1
M
]
=
[0 0
0 R−1
M
]
+ a
[ 1
−R−1
M r
] [
1 −rHR−1
M
]
where the scalar a is defined by Equation (8)
b)
RM +1 =
[RM rB∗
rBT r(0)
]
(11)
Let
R−1
M +1 =
[ D e
eH f
]
(12)
4

Loading page 8...

PROBLEM 1.5. CHAPTER 1.
where D, e and f are to be determined. Multiplying Equation (11) by Equation (12) you
get:
IM +1 =
[RM rB∗
rBT r(0)
] [ D e
eH f
]
Therefore:
RM D + rB∗eH = I (13)
RM e + rB∗f = 0 (14)
rBT e + r(0)f = 1 (15)
rBT D + r(0)eH = 0 (16)
From Equation (14):
e = −R−1
M rB∗ (17)
Hence, from Equation (15) and Equation (17):
f = 1
r(0) − rBT R−1
M rB∗ (18)
Correspondingly,
e = − R−1
M rB∗
r(0) − rBT R−1
M rB∗ (19)
From Equation (13):
D = R−1
M − R−1
M rB∗eH
= R−1
M + R−1
M rB∗rBT R−1
M
r(0) − rBT R−1
M rB∗ (20)
As a check, the results of Equation (19) and Equation (20) must satisfy Equation (16):
rBT D + r(0)eH = 0
rBT R−1
M + rBT R−1
M rB∗rBT R−1
M
r(0) − rBT R−1
M rB∗ − r(0)rBT R−1
M
r(0) − rBT R−1
M rB∗ = 0
We have thus shown that
R−1
M +1 =
[R−1
M 0
0 0
]
+ f
[R−1
M rB∗rBT R−1
M R−1
M rB∗
−rBT R−1
M 1
]
=
[R−1
M 0
0 0
]
+ f
[−R−1
M rB∗
1
] [−rBT R−1
M 1]
where the scalar f is defined by Equation (18)
5

Loading page 9...

PROBLEM 1.6. CHAPTER 1.
Problem 1.6
a)
We express the difference equation describing the first-order AR process u(n) as
u(n) = ν(n) + w1u(n − 1)
where w1 = −a1 . Solving the equation by repeated substitution, we get
u(n) =ν(n) + w1ν(n − 1) + w1u(n − 2)
=ν(n) + w1ν(n − 1) + w2
1 ν(n − 2) + . . . + wn−1
1 ν(1) (1)
Here we used the initial condition
u(0) = 0
Taking the expected value of both sides of Equation (1) and using
E[ν(n)] = μ
we get the geometric series
E[u(n)] = μ + w1μ + w2
1 μ + . . . + wn−1
1 μ
=
{ μ(1−wn
1
1−w1 ), w1 6 = 1
μn, w1 = 1
}
This result shows that if μ 6 = 0, then E[u(n)] is a function of time n. Accordingly, the AR
process u(n) is not stationary. If, however, the AR parameter satisfies the condition:
|a1| < 1 or |w1| < 1
then
E[u(n)] → μ
1 − w1
as n → ∞
Under this condition, we say that the AR process is asymptotically stationary to order one.
b)
When the white noise process ν(n) has zero mean, the AR process u(n) will likewise have
zero mean. Then
var[ν(n)] = σ2
ν
6

Loading page 10...

PROBLEM 1.6. CHAPTER 1.
var[u(n)] = E[u2(n)] (2)
Substituting Equation (1) into Equation (2), and recognizing that for the white noise pro-
cess
E[ν(n)ν(k)] =
{ σ2
ν n = k
0, n 6 = k (3)
we get the geometric series
var[u(n)] =σ2
ν (1 + w2
1 + w4
1 + . . . + w2n−2
1 )
=



σ2
ν (1 − w2n
1
1 − w2
1
), w1 6 = 1
σ2
ν n, w1 = 1
When |a1| < 1 or |w1| < 1, then
var[u(n)] ≈ σ2
ν
1 − w2
1
= σ2
ν
1 − a2
1
for large n
c)
The autocorrelation function of the AR process u(n) equals E[u(n)u(n − k)]. Substituting
Equation (1) into this formula, and using Equation (3), we get
E[u(n)u(n − k)] = σ2
ν (wk
1 + wk+2
1 + . . . + wk+2n−2
1 )
=
{
σ2
ν wk
1 (1−w2n
1
1−w2
1
), w1 6 = 1
σ2
ν n, w1 = 1
For |a1| < 1 or |w1| < 1, we may therefore express this autocorrelation function as
r(k) =E[u(n)u(n − k)]
≈ σ2
ν wk
1
1 − w2
1
for large n
Case 1: 0 < a1 < 1
In this case, w1 = −a1 is negative, and r(k) varies with k as follows:
7

Loading page 11...

PROBLEM 1.7. CHAPTER 1.9
for large n
Case 1: 0 < a1 < 1
In this case, w1 = -a1 is negative, and r(k) varies with k as follows:
Case 2: -1 < a1 < 0
In this case, w1 = -a1 is positive and r(k) varies with k as follows:
1.7 (a) The second-order AR process u(n) is described by the difference equation:
Hence
and the AR parameters equal
Accordingly, we write the Yule-Walker equations as
σv w1
1 w1
2
–
---------------≈
-4
-3
-2
-1
0
+1
+2
+3
+4 k
r(k)
-4 -2 -1 0 +2 +3 +4 k
r(k)
-3 +1
u n( ) u n 1–( ) 0.5u n 2–( )– v n( )+=
w1 1=
w2 0.5–=
a1 1–=
a2 0.5=
Case 2: −1 < a1 < 0
In this case, w1 = −a1 is positive, and r(k) varies with k as follows:9
for large n
Case 1: 0 < a1 < 1
In this case, w1 = -a1 is negative, and r(k) varies with k as follows:
Case 2: -1 < a1 < 0
In this case, w1 = -a1 is positive and r(k) varies with k as follows:
1.7 (a) The second-order AR process u(n) is described by the difference equation:
Hence
and the AR parameters equal
Accordingly, we write the Yule-Walker equations as
σv
2w1
k
1 w1
2
–
---------------≈
-4
-3
-2
-1
0
+1
+2
+3
+4 k
r(k)
-4 -2 -1 0 +2 +3 +4 k
r(k)
-3 +1
u n( ) u n 1–( ) 0.5u n 2–( )– v n( )+=
w1 1=
w2 0.5–=
a1 1–=
a2 0.5=
Problem 1.7
a)
The second-order AR process u(n) is described by the difference equation:
u(n) = u(n − 1) − 0.5u(n − 2) + ν(n)
which, rewritten, states
w1 = 1
w2 = −0.5
as the AR parameters are equal to:
a1 = −1
a2 = 0.5
Accordingly, the Yule-Walker equation may be written as:
[r(0) r(1)
r(1) r(0)
] [ 1
−0.5
]
=
[r(1)
r(2)
]
b)
Writing the Yule-Walker equations in expanded form:
r(0) − 0.5r(1) = r(1)
8

Loading page 12...

PROBLEM 1.8. CHAPTER 1.
r(1) − 0.5r(0) = r(2)
Solving the first relation for r(1):
r(1) = 2
3r(0) (1)
Solving the second relation for r(2):
r(2) = 1
6r(0) (2)
c)
Since the noise ν(n) has zero mean, the associated AR process u(n) will also have zero
mean. Hence,
var[u(n)] = E[(u2)]
= r(0)
It is known that
σ2
ν =
2∑
k=0
akr(k)
=r(0) + a1r(1) + a2r(2) (3)
Substituting Equation (1) and Equation (2) into Equation (3), and solving for r(0), we
get:
r(0) = σ2
ν
1 + 2
3 a1 + 1
6 a2
= 1.2
Problem 1.8
By Definition,
P0 = Average power of the AR process u(n)
=E[|u(n)|2]
=r(0) (1)
where r(0) is the autocorrelation function of u(n) with zero lag. We note that
{a1, a2, . . . , aM }
{r(1)
r(0), r(2)
r(0), . . . , r(M )
r(0)
}
9

Loading page 13...

PROBLEM 1.9. CHAPTER 1.
Equivalently, except for the scaling factor r(0),
{a1, a2, . . . , aM } {r(1), r(2), . . . , r(M )} (2)
Combining Equation (1) and Equation (2):
{P0, a1, a2, . . . , aM } {r(0), r(1), r(2), . . . , r(M )} (3)
Problem 1.9
a)
The transfer function of the MA model of Fig. 1.3 is
H(z) = 1 + b∗
1z−1 + b∗
2z−2 + . . . + b∗
K z−K
b)
The transfer function of the ARMA model of Fig. 1.4 is
H(z) = b0 + b∗
1z−1 + b∗
2z−2 + . . . + b∗
K z−K
1 + a∗
1z−1 + a∗
2z−2 + . . . + a∗
M z−M
c)
The ARMA model reduces to an AR model when
b0 = b1 = . . . = bK = 0
The ARMA model reduces to MA model when
a1 = a2 = . . . = aM = 0
10

Loading page 14...

PROBLEM 1.10. CHAPTER 1.
Problem 1.10
∗ Taking the z-transform of both sides of the correct equation:
X(z) = (1 + 0.75z−1 + 0.25z−2)V (z)
Hence, the transfer function of the MA model is:
X(z)
V (z) =1 + 0.75z−1 + 0.75z−1
= 1
(1 + 0.75z−1 + 0.75z−1)−1 (1)
Using long division we may perform the following expansion of the denominator in Equa-
tion (1):
(1 + 0.75z−1 + 0.75z−1)−1
= 1 − 3
4z−1 + 5
16z−2 − 3
64z−3 − 11
256z−4 − 45
1024z−5
− 91
4096z−6 + 93
16283z−7 − 85
65536z−8 − 627
262144z−9 + 1541
1048576z−10 + . . .
≈ 1 − 0.75z−1 + 0.3125z−2 − 0.0469z−3 − 0.043z−4 − 0.0439z−5
− 0.0222z−6 + 0.0057z−7 − 0.0013z−8 − 0.0024z−9 + 0.0015z−10
(2)
a)
M = 2
Retaining terms in Equation (2) up to z−2, we may approximate the MA model with an
AR model of order two as follows:
X(z)
V (z) ≈ 1
1 − 0.75z−1 + 0.3125z−2
∗Correction: the question was meant to ask the reader to consider an MA process x(n) of order two
described by the difference equation
x(n) = ν(n) + 0.75ν(n − 1) + 0.25ν(n − 2)
not the equation
x(n) = ν(n) + 0.75ν(n − 1) + 0.75ν(n − 2)
11

Loading page 15...

PROBLEM 1.11. CHAPTER 1.
b)
M = 5
Retaining terms in Equation (2) up to z−5, we may approximate the MA model with an
AR model of order two as follows:
X(z)
V (z) ≈ 1
1 − 0.75z−1 + 0.3125z−2 − 0.0469z−3 − 0.043z−4 + 0.0439z−5
c)
M = 10
Retaining terms in Equation (2) up to z−10, we may approximate the MA model with an
AR model of order two as follows:
X(z)
V (z) ≈ 1
D(z)
where D(z) is given by the polynomial on the right-hand side of Equation (2).
Problem 1.11
a)
The filter output is
x(n) = wH u(n)
where u(n) is the tap-input vector. The average power of the filter output is therefore
E[|x(n)|2] = E[wH u(n)uH (n)w]
= wH E[u(n)uH (n)]w
= wH Rw
b)
If u(n) is extracted from a zero-mean white noise with variance σ2, then
R = σ2I
where I is the identity matrix. Hence,
E[|x(n)|2] = σ2wH w
12

Loading page 16...

13 more pages available. Scroll down to load them.

Preview Mode

100%

Study Now!

XY-Copilot AI

Unlimited Access

Secure Payment

Instant Access

24/7 Support

Document Chat

Document Details

Subject

Electrical Engineering & Electronics

Solution Manual for Adaptive Filter Theory, 5th Edition

Study Now!

Document Details

Related Documents

Lab 9 Energy Skate Park

Common Electric Meters Used in the Industry

Experiment #17 DC Circuits Colin McCormick

Student Exploration Circuit Builder Build and Test

IS 3413 M03 Lab Signal Coding

Computed and digital radiography notes

How to Repair a Broken HDMI Port-A Step-by-Step Gu

Flip-Flop Timing Parameters and Circuit Behavior Analysis

Bridge Rectifier and its Analysis: A Study of Output Voltage and Ripple Effects

FM Receiver Design and Functionality: Analysis of Key Stages and Components Week 6 Assignment

Company

Explore

Study Tools