I am keeping resources for my skills in an S3 bucket accessible via Cloudfront as <prefix>.cloudfront.net. This includes icons for the skills, in skill graphics, and short mp3 sounds played by the skill using SSML.
Everything's been fine literally for years but since early September I noticed that some skill errors and it turns out that some of the sounds randomly do not play (icons and graphics kept kept in the same bucket & cloudtront distribution always seem to load fine though). Normally they play without a noticable gap, but in those cases after the 10 second CloudFront timeout the skill gets back an Error from the Amazon Alexa service which I see in my internal logs:
INVALID_RESPONSE: Connect to <prefix>.cloudfront.net:443 [d<prefix>.cloudfront.net/<ip1>, <prefix>.cloudfront.net/<ip2>, <prefix>.cloudfront.net/188.8.131.52, dunq4g40piwiv.cloudfront.net/<ip3>] failed: connect timed out for requestId amzn1.echo-api.request.<id>
Note the 3 IP addresses (which I intentionally obscured), it seems that Cloudfront tries 3 times accessing different edge locations (?) and then finally times out. I looked into Cloudfront but retries cannot be increased beyond 3 and this had anyway never happened before.
I also noticed that it does not happen all the time, but in my case on 3rd and 4th of September and then again 5th and 6th of October 2021. This is really odd, and my skills usage does not significantly change on those days.
It is breaking the skill's user experience because after such error the Alexa device usually just stops executing the skill session.
You can try to cause this by e.g. "Alexa, open My Fluffy Friend". There is a small clip played at the very start and today on average every second time it doesn't and Alexa as a result terminates the skill. Interestingly the pattern is play / not play / play / not play ... i.e. every second time is fine. Hmm.
Anybody else experiencing this?
Did anything change in Cloudfront since late August/September which might be causign this?
Is there any way this can be further investigated, or any settings further modified to work around it?