Notices
Results 1 to 13 of 13

Thread: Maxima/Minima/Saddles.

  1. #1 Maxima/Minima/Saddles. 
    Forum Masters Degree bit4bit's Avatar
    Join Date
    Jul 2007
    Posts
    621
    I've been working on some problems for critical points on graphs 3D functions (Maxima/minima/saddle points), using the matrix of second partials:

    [ ∂<sup>2</sup>f/∂x<sup>2</sup> ∂<sup>2</sup>f/∂x∂y ]
    [ ∂<sup>2</sup>f/∂y∂x ∂<sup>2</sup>f/∂y<sup>2</sup> ]

    I've been using the fact that the gradient vector is equal to zero to find critical points, and then using the determinant of this matrix to see whether they are maxima, minima or saddles at those points.

    Using the notation D(x,y) to be the determinant of the matrix, I've been using these facts to solve the problems:

    D(x,y) = (∂<sup>2</sup>f/∂x<sup>2</sup>)(∂<sup>2</sup>f/∂y<sup>2</sup>) - ( ∂<sup>2</sup>f/∂x∂y)<sup>2</sup>

    1.) D(x,y)>0 --> Minimum or Maximum at (x,y)
    --Then using: ∂<sup>2</sup>f/∂x<sup>2</sup>>0 -->Min, and ∂<sup>2</sup>f/∂x<sup>2</sup><0 -->Max
    2.) D(x,y)<0 --> Saddle at (x,y)
    3.) D(x,y)=0 --> Could be minimum, maximum or saddle ("or something much more bizzare", the book says) at (x,y), not enough information to know.

    I've been answering the problems using 1.) and 2.) with no problems at all, but for the case of 3.), I'm not sure how to do them. I thought about using the second test (as in 1.), but this only takes account of whether it is a maximum or a minimum and not whether it is a saddle point or "something much more bizzare"....unless ∂<sup>2</sup>f/∂x<sup>2</sup>=0 gives a saddle point?

    Also, I'm still not clear on exactly how points 1,2 and 3 are derived from the definition of taking the determinant of the matrix. I can see how the ∂<sup>2</sup>f/∂x<sup>2</sup> terms are positive for a minimum, and negative for a maximum, as it's basically the same thing as for a normal second derivative, but the ∂<sup>2</sup>f/∂x∂y terms, with relation to the saddle points and the determinat are confusing me a bit. Because of that, I can't grasp why 3.) is defined as being any of the possibilities.

    Thanks.


    Chance favours the prepared mind.
    Reply With Quote  
     

  2.  

    Related Discussions:

     

  3. #2  
    Forum Masters Degree bit4bit's Avatar
    Join Date
    Jul 2007
    Posts
    621
    Can anyone help? Here is an example of what i mean: "Find the local maxima, minima, and saddles of the following function: f(x,y)=x<sup>3</sup>-xy+y<sup>2</sup> "

    so ∇f=<0,0>=<∂f/∂x,∂f/∂y>= <3x<sup>2</sup>-y,2y+x>

    3x<sup>2</sup>-y=0 (1)
    2y+x=0 (2)

    (1)*2 --> 6x<sup>2</sup>+2y=0 (3)

    (3)-(2) --> 0=6x<sup>2</sup>-x=x(6x-1)

    so x = 0 and 1/6

    From (1) y=-3x<sup>2</sup>

    so for x=0, y=0, and for x=1/6, y=-1/12

    So there's critical points at (0,0) and (1/6,-1/12).

    The matrix of second partials is:

    |6x -1|
    | 1 2 |

    So the determinant is 12x+1. At (0,0), 12*0+1 = 1 >0, so according to my book, there must be a max/min here. The book says that to decide whether it is in fact a max/min point, to look at ∂<sup>2</sup>f/∂x<sup>2</sup> or ∂<sup>2</sup>f/∂y<sup>2</sup> and if they are positive, it's a minimum, but if they are negative, it's a maximum. ∂<sup>2</sup>f/∂x<sup>2</sup> =0 at (0,0), but ∂<sup>2</sup>f/∂y<sup>2</sup> >0, so can I say it's a minimum based on ∂<sup>2</sup>f/∂y<sup>2</sup>?

    For (1/6,-1/12) the determinant is 3>0, so there is a max/min, and ∂<sup>2</sup>f/∂x<sup>2</sup> and ∂<sup>2</sup>f/∂y<sup>2</sup> >0, so it must be another minimum?

    I'm still not understanding the whole mechanism of how the outcome of the determinant will decide what type of point it is.

    If

    a=∂<sup>2</sup>f/∂x<sup>2</sup>
    b=∂<sup>2</sup>f/∂x∂y
    c=∂<sup>2</sup>f/∂y∂x
    d=∂<sup>2</sup>f/∂y<sup>2</sup>

    Then we have the determinant:

    |a b|
    |c d| = ad-bc = A

    From that there are a few possibilities of A being less than, more than, or equal to zero, and I don't understand how the ∂<sup>2</sup>f/∂x∂y terms come into it. You can visualise the ∂<sup>2</sup>f/∂x<sup>2</sup> terms as a plane intersecting the graph, and then as a normal second derivative, but what about the other type? I can't see its significance in regards to the determinant.


    Chance favours the prepared mind.
    Reply With Quote  
     

  4. #3  
    Forum Professor serpicojr's Avatar
    Join Date
    Jul 2007
    Location
    JRZ
    Posts
    1,069
    My multivariable calc is rusty, so I don't know how much I can help. However, I did come up with a way (well, presumably, the way) of showing that if the determinant of the Hessian (matrix of second partials) is positive, then we have a minimum/maximum, and if it's negative, then we have a saddle point. Unfortunately, the proof is not so intuitive or even accessible unless you have a decent amount of linear algebra under your belt. The basic idea is this, though: for any vector u = [a,b] in the (x,y)-plane, we can form the directional derivative along u (I think we were talking about this a while ago, and you may already know what this is... if not, stop me). If we take the second derivative of a function f(x,y) along u, we get:

    f<sub>uu</sub> = a<sup>2</sup> f<sub>xx</sub> + 2ab f<sub>xy</sub> + b<sup>2</sup> f<sub>yy</sub>

    (Let me know if you don't understand the notation.) This is a quadratic form in a and b, and its coefficients come from the second partial derivatives of f. We can always attach a symmetric matrix to a quadratic form, which is the Hessian in our case. If the determinant of this matrix is positive, then the matrix is called definite: it always takes on positive values whenever a≠0 or b≠0 or it always takes on negative values whenever a≠0 or b≠0. If the determinant of this matrix is negative, then the matrix is called indefinite: it will necessarily take on negative and positive values.

    Now f<sub>uu</sub> tells us the direction of the concavity of f in the (u,z)-plane. f will have a minimum or maximum if the concavity is the same direction for all u, whereas f will have a saddle point if the concavity changes from up to down as u varies. The former corresponds to definite quadratic forms, i.e. positive determinant Hessians, while the latter corresponds to indefinite quadratic forms, i.e. negative determinant Hessians.
    Reply With Quote  
     

  5. #4  
    Forum Masters Degree bit4bit's Avatar
    Join Date
    Jul 2007
    Posts
    621
    Thanks for the reply. Yeh, I've done the directional derivative before, and know that for a unit vector u,

    ∇<sub>u</sub>f=∇.u=<∂f/∂x,∂f/∂y>.<a,b>=a(∂f/∂x)+b(∂f/∂y)

    I'm assuming your notation means f<sub>uu</sub> = ∂<sup>2</sup>f/∂u<sup>2</sup>, and so on?

    So form the expression you've got it looks like, you've squared the gradient vector so that:

    ∇<sub>u</sub>f=af<sub>xx</sub>+bf<sub>yy</sub>

    (∇<sub>u</sub>f)<sup>2</sup>=(af<sub>x</sub>+bf<sub>y</sub>)<sup>2</sup>
    =a<sup>2</sup>f<sub>x</sub><sup>2</sup>+2af<sub>x</sub>bf<sub>y</sub>+b<sup>2</sup>f<sub>y</sub><sup>2</sup>
    =a<sup>2</sup>f<sub>xx</sub>+2abf<sub>xy</sub>+b<sup>2</sup>f<sub>yy</sub>

    I can understand how to get to that definition, but not why it works. For example, basically you have (∇<sub>u</sub>f)<sup>2</sup>, so is it the case that to find the nth derivative of a function along a vector, we just rasie the directional derivative to the nth power?...so if we wanted the third derivative, we would just use (∇<sub>u</sub>f)<sup>3</sup>? Why is that?

    Also, you lost me at attaching a matrix to a quadratic form. I have touched on linear algebra before (not in depth), and seem to remember that systems of equations can be written out as a determinant. For example, the co-efficients from two simultaneous equations,

    ax+b=0
    cx+d=0

    can be written out as a determinant,

    |a b|
    |c d|

    ...though I can't remember the significance of the determinant in relation to solving for x...I'm sure the determinant doesn't equal x, but it somehow helps you to find it more easily? Has that got anything to do with what you're showing?

    Thanks alot
    Chance favours the prepared mind.
    Reply With Quote  
     

  6. #5 Re: Maxima/Minima/Saddles. 
    Forum Ph.D. william's Avatar
    Join Date
    Jun 2006
    Location
    Wherever I go, there I am
    Posts
    935
    It's been a long time since I did problems like these, so I'd have to re-learn to be able to help with your specific question. But I wanted to comment on this;

    Quote Originally Posted by bit4bit
    D(x,y) = (∂<sup>2</sup>f/∂x<sup>2</sup>)(∂<sup>2</sup>f/∂y<sup>2</sup>) - ( ∂<sup>2</sup>f/∂x∂y)<sup>2</sup>

    The second term is true only if the order of differentiation doesn't matter. That is, if ∂<sup>2</sup>f/∂x∂y = ∂<sup>2</sup>f/∂y∂x. I can't quite recall, but I think they are equal if the function is smooth and continuous....



    Cheers
    "... the polhode rolls without slipping on the herpolhode lying in the invariable plane."
    ~Footnote in Goldstein's Mechanics, 3rd ed. p. 202
    Reply With Quote  
     

  7. #6  
    Forum Masters Degree bit4bit's Avatar
    Join Date
    Jul 2007
    Posts
    621
    Yeh the determinant is actually

    D(x,y) = (∂,<sup>2</sup>f/∂x<sup>2</sup>)(∂<sup>2</sup>f/∂y<sup>2</sup>) - (∂<sup>2</sup>f/∂x∂y)(∂<sup>2</sup>f/∂y∂x),

    The book writes it as (∂<sup>2</sup>f/∂x∂y)<sup>2</sup>, which makes sense to me algebraically since:

    (∂<sup>2</sup>f/∂x∂y)(∂<sup>2</sup>f/∂y∂x) =[∂<sup>2</sup>f∂<sup>2</sup>f] / [∂x∂y∂x∂y] = [∂<sup>2</sup>f]<sup>2</sup> / [∂x∂y]<sup>2</sup> = (∂<sup>2</sup>f/∂x∂y)<sup>2</sup>

    I think so long as the function is differentiable, then it's valid to do this, and thats assumed by the book really. For the case where the order of differentiation matters, i think thats probably beyond my scope at the moment, or only happens in special cases.
    Chance favours the prepared mind.
    Reply With Quote  
     

  8. #7  
    Forum Professor serpicojr's Avatar
    Join Date
    Jul 2007
    Location
    JRZ
    Posts
    1,069
    So, in general, a (binary) quadratic form in the variables x and y is a function f(x,y) = Ax<sup>2</sup>+Bxy+Cy<sup>2</sup>, A, B, and C numbers. We attach to it the matrix:

    <pre>F = [ A B/2]
    [B/2 C ]</pre>

    We then have that, for the row vector v = [x,y], f(x,y) = vFv<sup>t</sup>, where v<sup>t</sup> is the transpose of v, i.e. the column vector with x in the first slot and y in the second slot. Depending on the author, the quantity -det(F) or -4det(F) is the discriminant of the form. (Note that this, in the latter case, is B<sup>2</sup>-4AC, the discriminant in the quadratic formula.) The important thing for our discussion is that the behavior of the form is essentially determined by the sign of the discriminant (i.e., the opposite sign of the determinant). Negative discriminant/positive determinant implies a definite form, positive discriminant/negative determinant implies an indefinite form, and zero determinant/discriminant implies a "degenerate form", basically one which can be factored as the square of a linear form, i.e. f(x,y) = ±(Ex+Fy)<sup>2</sup>.
    Reply With Quote  
     

  9. #8 Re: Maxima/Minima/Saddles. 
    Moderator Moderator AlexP's Avatar
    Join Date
    Jul 2006
    Location
    NY
    Posts
    1,838
    Quote Originally Posted by william
    The second term is true only if the order of differentiation doesn't matter. That is, if ∂<sup>2</sup>f/∂x∂y = ∂<sup>2</sup>f/∂y∂x. I can't quite recall, but I think they are equal if the function is smooth and continuous...
    Yes, the order of differentiation won't matter if the function is "nice enough" (not my words). Though I guess maybe that's been figured out by this point...
    "There is a kind of lazy pleasure in useless and out-of-the-way erudition." -Jorge Luis Borges
    Reply With Quote  
     

  10. #9  
    Forum Masters Degree bit4bit's Avatar
    Join Date
    Jul 2007
    Posts
    621
    Hi, been offline for a few days, had to give a presentation today, and was rushing to get it done. Went OK though.

    Quote Originally Posted by serpicojr
    So, in general, a (binary) quadratic form in the variables x and y is a function f(x,y) = Ax<sup>2</sup>+Bxy+Cy<sup>2</sup>, A, B, and C numbers. We attach to it the matrix:

    <pre>F = [ A B/2]
    [B/2 C ]</pre>
    So in this case A=f<sub>xx</sub>, B=f<sub>xy</sub>, and C=f<sub>yy</sub>?

    We then have that, for the row vector v = [x,y], f(x,y) = vFv<sup>t</sup>, where v<sup>t</sup> is the transpose of v, i.e. the column vector with x in the first slot and y in the second slot.
    I don't get what the purpose of v is, and why you have to multiply F by both v, and v<sup>t</sup>. Am I right in saying that:

    <pre>vF = [Ax Bx/2]
    [By/2 Cy]</pre>

    and,

    <pre>vFv<sup>t</sup>= [Ax<sup>2</sup> Bxy/2]
    [Bxy/2 Cy<sup>2</sup>]</pre>

    Is that right?

    Depending on the author, the quantity -det(F) or -4det(F) is the discriminant of the form. (Note that this, in the latter case, is B<sup>2</sup>-4AC, the discriminant in the quadratic formula.) The important thing for our discussion is that the behavior of the form is essentially determined by the sign of the discriminant (i.e., the opposite sign of the determinant). Negative discriminant/positive determinant implies a definite form, positive discriminant/negative determinant implies an indefinite form, and zero determinant/discriminant implies a "degenerate form", basically one which can be factored as the square of a linear form, i.e. f(x,y) = ±(Ex+Fy)<sup>2</sup>.
    Whats Ex, and Fy?

    Thanks
    Chance favours the prepared mind.
    Reply With Quote  
     

  11. #10  
    Forum Professor serpicojr's Avatar
    Join Date
    Jul 2007
    Location
    JRZ
    Posts
    1,069
    A 2x2 matrix times a 2x1 matrix (2 rows, 1 column, i.e. a column vector) gives a 2x1 matrix. A 1x2 matrix (1 row, 2 columns, i.e. a row vector) times a 2x1 matrix gives a 1x1 matrix, a scalar. In our case:

    <pre>[x y][ A B/2][x] =
    [B/2 C ][y]

    [x y][Ax+By/2] =
    [Bx/2+Cy]

    [Ax^2+Bxy/2+Bxy/2+Cy^2] = Ax^2+Bxy+Cy^2</pre>

    In our case, f<sub>xy</sub> = B/2, as we can see by looking at the coefficient on ab in our form.

    E and F are just whatever coefficients that we get when we factor. Like:

    4x<sup>2</sup>-12xy+9y<sup>2</sup> = (2x-3y)<sup>2</sup>

    Here, A = 4, B = -12, C = 9, E = 2, and F = -3. Note that we also have that the discriminant D = (-12)<sup>2</sup>-4*4*9 = 144-144 = 0.
    Reply With Quote  
     

  12. #11  
    Forum Masters Degree bit4bit's Avatar
    Join Date
    Jul 2007
    Posts
    621
    I've tried to follow your proof, and I can't get it. I don't think I've got enough linear algebra under my belt, as you say is needed. For now, I'm just going to be happy with using the definitions, rather than understanding their proofs. For when D(x,y)=0, and there is not enough information about the point to know what it is, I read that you need to do a higher derivative test on the function.... maybe making a matrix of third partials, and checking that?

    For now though, I'm just gonna carry on with my book, and use these definitions when needed. Thanks for trying to explain anyway.
    Chance favours the prepared mind.
    Reply With Quote  
     

  13. #12  
    Forum Professor serpicojr's Avatar
    Join Date
    Jul 2007
    Location
    JRZ
    Posts
    1,069
    Yeah, you don't learn about quadratic forms in a first course in linear algebra typically. I think we only got to eigenvalues by the end of the semester. If you want a flipping fantastic linear algebra book which goes into proofs but is completely and utterly pleasant to read, look for Hoffman and Kunze's Linear Algebra. I can't recommend it enough. And I think they go over all sorts of forms.

    (Oh, and a lot of my knowledge about quadratic forms actually comes from number theory. Binary quadratic forms with integer coefficients were studied all the way back to Gauss, and their study is very important in understanding quadratic irrationals, i.e. irrational numbers that satisfy quadratic equations with integer coefficients.)
    Reply With Quote  
     

  14. #13  
    Forum Masters Degree bit4bit's Avatar
    Join Date
    Jul 2007
    Posts
    621
    Thanks for the recommendation...I'll see if I can get my hands on a copy.....before I got my copy of a multivariable calc book, I was using this page, which covers eigenvectors, and other linear algebra stuff. My book is a bit more simplified than that, and is really a dumbed down course, but I'd forgotten about that page, and might start reading it alongside my book.

    As for number theory, I've never really looked into it in much depth, as it has no immediate practical applications, so it doesn't interest me so much. I've heard of Gauss though who was apperntly an outstanding mathematician...his name seems to pop up all the time in loads of areas of maths.
    Chance favours the prepared mind.
    Reply With Quote  
     

Bookmarks
Bookmarks
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •