| Author |
Message
|
| bit4bit |
 Posted: Sun Apr 20, 2008 12:36 pm Post subject: Maxima/Minima/Saddles. |
 |
|
Forum Bachelors Degree
Joined: 14 Jul 2007
Posts: 492
|
I've been working on some problems for critical points on graphs 3D functions (Maxima/minima/saddle points), using the matrix of second partials:
[ ∂2f/∂x2 ∂2f/∂x∂y ]
[ ∂2f/∂y∂x ∂2f/∂y2 ]
I've been using the fact that the gradient vector is equal to zero to find critical points, and then using the determinant of this matrix to see whether they are maxima, minima or saddles at those points.
Using the notation D(x,y) to be the determinant of the matrix, I've been using these facts to solve the problems:
D(x,y) = (∂2f/∂x2)(∂2f/∂y2) - ( ∂2f/∂x∂y)2
1.) D(x,y)>0 --> Minimum or Maximum at (x,y)
--Then using: ∂2f/∂x2>0 -->Min, and ∂2f/∂x2<0 -->Max
2.) D(x,y)<0 --> Saddle at (x,y)
3.) D(x,y)=0 --> Could be minimum, maximum or saddle ("or something much more bizzare", the book says) at (x,y), not enough information to know.
I've been answering the problems using 1.) and 2.) with no problems at all, but for the case of 3.), I'm not sure how to do them. I thought about using the second test (as in 1.), but this only takes account of whether it is a maximum or a minimum and not whether it is a saddle point or "something much more bizzare"....unless ∂2f/∂x2=0 gives a saddle point?
Also, I'm still not clear on exactly how points 1,2 and 3 are derived from the definition of taking the determinant of the matrix. I can see how the ∂2f/∂x2 terms are positive for a minimum, and negative for a maximum, as it's basically the same thing as for a normal second derivative, but the ∂2f/∂x∂y terms, with relation to the saddle points and the determinat are confusing me a bit. Because of that, I can't grasp why 3.) is defined as being any of the possibilities.
Thanks.
_________________
Chance favours the prepared mind. |
|
| Back to top |
|
 |
| bit4bit |
| Posted: Mon Apr 21, 2008 7:03 pm Post subject: |
|
|
Forum Bachelors Degree
Joined: 14 Jul 2007
Posts: 492
|
Can anyone help? Here is an example of what i mean: "Find the local maxima, minima, and saddles of the following function: f(x,y)=x3-xy+y2 "
so ∇f=<0,0>=<∂f/∂x,∂f/∂y>= <3x2-y,2y+x>
3x2-y=0 (1)
2y+x=0 (2)
(1)*2 --> 6x2+2y=0 (3)
(3)-(2) --> 0=6x2-x=x(6x-1)
so x = 0 and 1/6
From (1) y=-3x2
so for x=0, y=0, and for x=1/6, y=-1/12
So there's critical points at (0,0) and (1/6,-1/12).
The matrix of second partials is:
|6x -1|
| 1 2 |
So the determinant is 12x+1. At (0,0), 12*0+1 = 1 >0, so according to my book, there must be a max/min here. The book says that to decide whether it is in fact a max/min point, to look at ∂2f/∂x2 or ∂2f/∂y2 and if they are positive, it's a minimum, but if they are negative, it's a maximum. ∂2f/∂x2 =0 at (0,0), but ∂2f/∂y2 >0, so can I say it's a minimum based on ∂2f/∂y2?
For (1/6,-1/12) the determinant is 3>0, so there is a max/min, and ∂2f/∂x2 and ∂2f/∂y2 >0, so it must be another minimum?
I'm still not understanding the whole mechanism of how the outcome of the determinant will decide what type of point it is.
If
a=∂2f/∂x2
b=∂2f/∂x∂y
c=∂2f/∂y∂x
d=∂2f/∂y2
Then we have the determinant:
|a b|
|c d| = ad-bc = A
From that there are a few possibilities of A being less than, more than, or equal to zero, and I don't understand how the ∂2f/∂x∂y terms come into it. You can visualise the ∂2f/∂x2 terms as a plane intersecting the graph, and then as a normal second derivative, but what about the other type? I can't see its significance in regards to the determinant.
_________________
Chance favours the prepared mind. |
|
| Back to top |
|
|
| serpicojr |
| Posted: Mon Apr 21, 2008 9:34 pm Post subject: |
|
|
Forum Ph.D.
Joined: 17 Jul 2007
Posts: 871
Location: JRZ
|
My multivariable calc is rusty, so I don't know how much I can help. However, I did come up with a way (well, presumably, the way) of showing that if the determinant of the Hessian (matrix of second partials) is positive, then we have a minimum/maximum, and if it's negative, then we have a saddle point. Unfortunately, the proof is not so intuitive or even accessible unless you have a decent amount of linear algebra under your belt. The basic idea is this, though: for any vector u = [a,b] in the (x,y)-plane, we can form the directional derivative along u (I think we were talking about this a while ago, and you may already know what this is... if not, stop me). If we take the second derivative of a function f(x,y) along u, we get:
fuu = a2 fxx + 2ab fxy + b2 fyy
(Let me know if you don't understand the notation.) This is a quadratic form in a and b, and its coefficients come from the second partial derivatives of f. We can always attach a symmetric matrix to a quadratic form, which is the Hessian in our case. If the determinant of this matrix is positive, then the matrix is called definite: it always takes on positive values whenever a≠0 or b≠0 or it always takes on negative values whenever a≠0 or b≠0. If the determinant of this matrix is negative, then the matrix is called indefinite: it will necessarily take on negative and positive values.
Now fuu tells us the direction of the concavity of f in the (u,z)-plane. f will have a minimum or maximum if the concavity is the same direction for all u, whereas f will have a saddle point if the concavity changes from up to down as u varies. The former corresponds to definite quadratic forms, i.e. positive determinant Hessians, while the latter corresponds to indefinite quadratic forms, i.e. negative determinant Hessians. |
|
| Back to top |
|
|
| bit4bit |
| Posted: Tue Apr 22, 2008 4:20 pm Post subject: |
|
|
Forum Bachelors Degree
Joined: 14 Jul 2007
Posts: 492
|
Thanks for the reply. Yeh, I've done the directional derivative before, and know that for a unit vector u,
∇uf=∇.u=<∂f/∂x,∂f/∂y>.<a,b>=a(∂f/∂x)+b(∂f/∂y)
I'm assuming your notation means fuu = ∂2f/∂u2, and so on?
So form the expression you've got it looks like, you've squared the gradient vector so that:
∇uf=afxx+bfyy
(∇uf)2=(afx+bfy)2
=a2fx2+2afxbfy+b2fy2
=a2fxx+2abfxy+b2fyy
I can understand how to get to that definition, but not why it works. For example, basically you have (∇uf)2, so is it the case that to find the nth derivative of a function along a vector, we just rasie the directional derivative to the nth power?...so if we wanted the third derivative, we would just use (∇uf)3? Why is that?
Also, you lost me at attaching a matrix to a quadratic form. I have touched on linear algebra before (not in depth), and seem to remember that systems of equations can be written out as a determinant. For example, the co-efficients from two simultaneous equations,
ax+b=0
cx+d=0
can be written out as a determinant,
|a b|
|c d|
...though I can't remember the significance of the determinant in relation to solving for x...I'm sure the determinant doesn't equal x, but it somehow helps you to find it more easily? Has that got anything to do with what you're showing?
Thanks alot
_________________
Chance favours the prepared mind. |
|
| Back to top |
|
|
| william |
| Posted: Tue Apr 22, 2008 4:38 pm Post subject: Re: Maxima/Minima/Saddles. |
|
|
Forum Ph.D.
Joined: 23 Jun 2006
Posts: 905
Location: USA
|
It's been a long time since I did problems like these, so I'd have to re-learn to be able to help with your specific question. But I wanted to comment on this;
| bit4bit wrote: |
| D(x,y) = (∂2f/∂x2)(∂2f/∂y2) - ( ∂2f/∂x∂y)2 |
The second term is true only if the order of differentiation doesn't matter. That is, if ∂2f/∂x∂y = ∂2f/∂y∂x. I can't quite recall, but I think they are equal if the function is smooth and continuous....
Cheers
_________________
"... the polhode rolls without slipping on the herpolhode lying in the invariable plane."
~Footnote in Goldstein's Mechanics, 3rd ed. p. 202
About my avatar: This is a smoothed particle hydrodynamics (SPH) simulation of the merger of two galaxies. The code was written by Volker Springel of the Max Planck Institute for Astrophysics at Garching Germany. This simulation uses 20,000 disk particles (stars) and 40,000 halo particles (dark matter) per galaxy. The three views are, from left to right, the x-y plane, x-z plane, and y-z plane. |
|
| Back to top |
|
|
| bit4bit |
| Posted: Tue Apr 22, 2008 5:06 pm Post subject: |
|
|
Forum Bachelors Degree
Joined: 14 Jul 2007
Posts: 492
|
Yeh the determinant is actually
D(x,y) = (∂,2f/∂x2)(∂2f/∂y2) - (∂2f/∂x∂y)(∂2f/∂y∂x),
The book writes it as (∂2f/∂x∂y)2, which makes sense to me algebraically since:
(∂2f/∂x∂y)(∂2f/∂y∂x) =[∂2f∂2f] / [∂x∂y∂x∂y] = [∂2f]2 / [∂x∂y]2 = (∂2f/∂x∂y)2
I think so long as the function is differentiable, then it's valid to do this, and thats assumed by the book really. For the case where the order of differentiation matters, i think thats probably beyond my scope at the moment, or only happens in special cases.
_________________
Chance favours the prepared mind. |
|
| Back to top |
|
|
| serpicojr |
| Posted: Tue Apr 22, 2008 5:30 pm Post subject: |
|
|
Forum Ph.D.
Joined: 17 Jul 2007
Posts: 871
Location: JRZ
|
So, in general, a (binary) quadratic form in the variables x and y is a function f(x,y) = Ax2+Bxy+Cy2, A, B, and C numbers. We attach to it the matrix:
F = [ A B/2]
[B/2 C ]
We then have that, for the row vector v = [x,y], f(x,y) = vFvt, where vt is the transpose of v, i.e. the column vector with x in the first slot and y in the second slot. Depending on the author, the quantity -det(F) or -4det(F) is the discriminant of the form. (Note that this, in the latter case, is B2-4AC, the discriminant in the quadratic formula.) The important thing for our discussion is that the behavior of the form is essentially determined by the sign of the discriminant (i.e., the opposite sign of the determinant). Negative discriminant/positive determinant implies a definite form, positive discriminant/negative determinant implies an indefinite form, and zero determinant/discriminant implies a "degenerate form", basically one which can be factored as the square of a linear form, i.e. f(x,y) = ±(Ex+Fy)2. |
|
| Back to top |
|
|
| Chemboy |
| Posted: Tue Apr 22, 2008 8:43 pm Post subject: Re: Maxima/Minima/Saddles. |
|
|
Forum Ph.D.
Joined: 01 Jul 2006
Posts: 959
Location: NY
|
| william wrote: |
| The second term is true only if the order of differentiation doesn't matter. That is, if ∂2f/∂x∂y = ∂2f/∂y∂x. I can't quite recall, but I think they are equal if the function is smooth and continuous... |
Yes, the order of differentiation won't matter if the function is "nice enough" (not my words). Though I guess maybe that's been figured out by this point...
_________________
"There is a kind of lazy pleasure in useless and out-of-the-way erudition." -Jorge Luis Borges |
|
| Back to top |
|
|
| bit4bit |
| Posted: Fri Apr 25, 2008 9:37 am Post subject: |
|
|
Forum Bachelors Degree
Joined: 14 Jul 2007
Posts: 492
|
Hi, been offline for a few days, had to give a presentation today, and was rushing to get it done. Went OK though.
| serpicojr wrote: |
So, in general, a (binary) quadratic form in the variables x and y is a function f(x,y) = Ax2+Bxy+Cy2, A, B, and C numbers. We attach to it the matrix:
F = [ A B/2]
[B/2 C ] |
So in this case A=fxx, B=fxy, and C=fyy?
| Quote: |
| We then have that, for the row vector v = [x,y], f(x,y) = vFvt, where vt is the transpose of v, i.e. the column vector with x in the first slot and y in the second slot. |
I don't get what the purpose of v is, and why you have to multiply F by both v, and vt. Am I right in saying that:
vF = [Ax Bx/2]
[By/2 Cy]
and,
vFvt= [Ax2 Bxy/2]
[Bxy/2 Cy2]
Is that right?
| Quote: |
| Depending on the author, the quantity -det(F) or -4det(F) is the discriminant of the form. (Note that this, in the latter case, is B2-4AC, the discriminant in the quadratic formula.) The important thing for our discussion is that the behavior of the form is essentially determined by the sign of the discriminant (i.e., the opposite sign of the determinant). Negative discriminant/positive determinant implies a definite form, positive discriminant/negative determinant implies an indefinite form, and zero determinant/discriminant implies a "degenerate form", basically one which can be factored as the square of a linear form, i.e. f(x,y) = ±(Ex+Fy)2. |
Whats Ex, and Fy?
Thanks
_________________
Chance favours the prepared mind. |
|
| Back to top |
|
|
| serpicojr |
| Posted: Fri Apr 25, 2008 10:08 am Post subject: |
|
|
Forum Ph.D.
Joined: 17 Jul 2007
Posts: 871
Location: JRZ
|
A 2x2 matrix times a 2x1 matrix (2 rows, 1 column, i.e. a column vector) gives a 2x1 matrix. A 1x2 matrix (1 row, 2 columns, i.e. a row vector) times a 2x1 matrix gives a 1x1 matrix, a scalar. In our case:
[x y][ A B/2][x] =
[B/2 C ][y]
[x y][Ax+By/2] =
[Bx/2+Cy]
[Ax^2+Bxy/2+Bxy/2+Cy^2] = Ax^2+Bxy+Cy^2
In our case, fxy = B/2, as we can see by looking at the coefficient on ab in our form.
E and F are just whatever coefficients that we get when we factor. Like:
4x2-12xy+9y2 = (2x-3y)2
Here, A = 4, B = -12, C = 9, E = 2, and F = -3. Note that we also have that the discriminant D = (-12)2-4*4*9 = 144-144 = 0. |
|
| Back to top |
|
|
| bit4bit |
| Posted: Sat Apr 26, 2008 12:23 pm Post subject: |
|
|
Forum Bachelors Degree
Joined: 14 Jul 2007
Posts: 492
|
I've tried to follow your proof, and I can't get it. I don't think I've got enough linear algebra under my belt, as you say is needed. For now, I'm just going to be happy with using the definitions, rather than understanding their proofs. For when D(x,y)=0, and there is not enough information about the point to know what it is, I read that you need to do a higher derivative test on the function.... maybe making a matrix of third partials, and checking that?
For now though, I'm just gonna carry on with my book, and use these definitions when needed. Thanks for trying to explain anyway. 
_________________
Chance favours the prepared mind. |
|
| Back to top |
|
|
| serpicojr |
| Posted: Sat Apr 26, 2008 1:04 pm Post subject: |
|
|
Forum Ph.D.
Joined: 17 Jul 2007
Posts: 871
Location: JRZ
|
Yeah, you don't learn about quadratic forms in a first course in linear algebra typically. I think we only got to eigenvalues by the end of the semester. If you want a flipping fantastic linear algebra book which goes into proofs but is completely and utterly pleasant to read, look for Hoffman and Kunze's Linear Algebra. I can't recommend it enough. And I think they go over all sorts of forms.
(Oh, and a lot of my knowledge about quadratic forms actually comes from number theory. Binary quadratic forms with integer coefficients were studied all the way back to Gauss, and their study is very important in understanding quadratic irrationals, i.e. irrational numbers that satisfy quadratic equations with integer coefficients.) |
|
| Back to top |
|
|
| bit4bit |
| Posted: Sun Apr 27, 2008 12:32 pm Post subject: |
|
|
Forum Bachelors Degree
Joined: 14 Jul 2007
Posts: 492
|
Thanks for the recommendation...I'll see if I can get my hands on a copy.....before I got my copy of a multivariable calc book, I was using this page, which covers eigenvectors, and other linear algebra stuff. My book is a bit more simplified than that, and is really a dumbed down course, but I'd forgotten about that page, and might start reading it alongside my book.
As for number theory, I've never really looked into it in much depth, as it has no immediate practical applications, so it doesn't interest me so much. I've heard of Gauss though who was apperntly an outstanding mathematician...his name seems to pop up all the time in loads of areas of maths.
_________________
Chance favours the prepared mind. |
|
| Back to top |
|
|
|
|
|
