What property are you getting the attachment URL from? I’m guessing its download_url? That link is only valid for two minutes; see the docs. I think you need to use permanent_url.
See Question about attachment url fields for more discussion. @Frederic_Malenfant may have more input here as well.