We are observing random occurrence of following error on our builds
An error occurred in a before(:suite) hook.
Failure/Error: bucket.object(path).get(response_target: destination)
Aws::S3::Errors::RequestTimeTooSkewed:
The difference between the request time and the current time is too large
stack version for agents: 5.21.0
Curious to know if anyone else is having the same problem
That is a curious error. Our elastic stack should have NTP configured by default, are you modifying anything about the system time on your instances? Or are you running the command in a docker image that might be interfering with time?
I couldn’t find a configuration option for the AWS CLI to ignore or modify its errors related to time. So I think the best bet is to investigate where the time difference is coming from. I found this article which might help: amazon web services - AWS S3 upload fails: RequestTimeTooSkewed - Stack Overflow. And this article might be helpful as well How to fix S3 (RequestTimeTooSkewed) error | Ottomatik Help Center
And one last thought, that looks like ruby from that snippet, is it possible that your tests are changing anything with the system time?
Yes our ruby test are running in docker container
we configured to get times this error happens
NTP time is 2023-10-05 13:44:03 +1100 (1696473843)
Local time (Time.zone) is 2023-10-05 00:00:00 +1100 (1696424400)
Local time (Process.clock_gettime) is 2023-10-05 13:44:03 +1100 (1696473843.316994)
Hey @farhan!
Would it be possible to email us at support@buildkite.com with a link to at least one of the builds you’re seeing this error in, but preferably more than 1 and including the first build you saw it occur in?
Thanks!
sending now