Verified answer

Some do. Doctor Who has his (now her) TARDIS or Time And Relative Dimension In Space ...

But you could circle around a black hole, that works for time dilation as well.

Haven't you ever used Microsoft Word?  You can choose what to anchor a frame to.  To text, to the page, to other frames.  So if you are traveling to another time on the same point RELATIVE TO THE EARTH you connect your dimensional anchor to that point.  Whereas if you want to go to a point relative to a galaxy you set your dimensional anchor to that galaxy prior to initiating time shift sequencing.  It is all outlined in "Time travel for dummies".

Doctor Who often gets lost.

The writers are creating a fictional story. Writing reality would not sell and would be counter productive. Some people, actually do write one book of reality then, go broke.

They account for it in Primer but generally films don't even get real science correct.

First, there is no absolute position in space. Space doesn't have a coordinate system painted on it. 

Second, time travel to the past is impossible. It would create paradoxes.

Third, writers of films are stupid. The film's rarely get any science right. And they rarely even have any logic or reason in their plots. 

I get tired of time travel excuses in the movies and TV storylines. It's treated as an easy thing to do and incredibly compact. When in reality the power of a star is needed.

The concept of time travel doesn't work on many levels. This is just one of them. 

maybe they forgot to mention that