The theory of relativity states that as the velocity of an object nears the speed of light, time slows down.
If person A travels for a few hours at half the speed of light and returns to Earth, that person A has aged much slower than person B who stayed on Earth. This difference in aging is because time passed much slower for person A than person B, because person A was traveling closer to the speed of light. While just a few hours passed for person A, person B experienced many years.
Let's say we have a wormhole which mouth is portable. It is then carried on a spacecraft traveling at 50 percent of light speed. The other end of the wormhole is on Earth.
When the spacecraft returned to Earth a few hours (according to the spacecrafts clock)later a few years would have passed on Earth.
Okay, so far so good.
But do we now have a time machine? No, we do not. Time has only passed slower for the other end of the wormhole.
The effect of going through the wormhole would be the same as if you stepped through a space shuttle's porthole after it returned from a mission. Time in the shuttle would have only passed slower, it does not become a time machine into the past. You can't go back when there is nothing to go back to. Except maybe if you consider the shuttle is the past, because it has aged slower.
Ps. I'd be happy if anyone could show me if I'm wrong. Many of the world renown scientist seem to me as a bunch of idiots when it comes to time travel and I sure would be happy if it was just me who is the idiot.