Recently my father and I got into a debate about artificial intelligence. He didn't believe we'd ever be able to fully simulate human intelligence, and even if we ever did, it wouldn't actually be intelligent. Obviously, I thought the exact opposite.
An interesting thought concerning this is from the classic book, "Xenocide" by Orson Scott Card (if you haven't read it, I recommend it). At some point in the book, Miro and Ender discuss free will, and if it even exists at all. Ender pointed out that if something with "free will" is created by someone/something else, the creator is almost like a puppet master. The master programs his puppets to behave in set ways. Computer programs are created by people, so they don't have any free will. He also pointed out that people are created by their environment, and therefore don't really have free will - it's only a script they play out.
This then begs the question - if we're ever able to simulate human intelligence, what's the difference between a "simulation" and the real thing? How do we know that the simulation isn't genuine, that the program isn't acting out a preset script given to it by the programmer?