Let's see: first there's relativity, that imposes and absolute speed limit, the speed of light, and worse, says that as something with non-zero rest mass accelerates toward the speed of light its mass increases with no upper bound, meaning more and more energy is needed to accelerate by the same amount; second there's the fact that most of the universe is very far away; and finally there's the fact that either of the dodges to these problems (wormholes and the solution to Einstein's equation called "the Alcubier warp drive" because it looks mathematically like what a warp drive would look like) require amounts of negative mass on the order of the positive mass of Jupiter if they're to be usable by something as big as us and we've only observed negative mass at the quantum-mechanical scale and always with a larger positive mass in close proximity that would make it useless for either making a wormhole or a warp drive.
Of course, relativity has a positive side. As something accelerates toward the speed of light, time as experience by it, slows down in comparison to time in a non-accelerated frame of reference, so the long journey that takes centuries from the point of view of the earthbound might take a few years from the point of view of the traveler (if only that pesky increase of mass effect didn't make going that fast take too much energy).
