question

We got our first feedback yesterday after 5 days, but I've heard for some people it has taken as long as 3 weeks.

About a week is what it has been going on. Depends on the skill. The simplest fortune cookie skill I've done (Demotivate Me) took two days. The most complicated skill I've done (Starlanes) took a month and a half. However, that's only the first step. Something has changed on the certification team. They are now bouncing things for highly pedantic and arbitrary reasons. Unless your skill is moronically simple, they will probably find some reason to bounce it. Even if you "fix" what they talk about, there's no guarantee that it will go back to the same person for re-submission. So the highly subjective standards will be tested by someone else's subjectivity. [E.g. I had one skill come back with [i]four[/i] new complaints about an area that I hadn't changed upon re-certification.] You can fill in a comments section during submission, ostensibly, to pass notes to the certification team. However I have no evidence that they read this. The only evidence I have is to the contrary, indicating that they do not read it. [E.g. I mention that one intent is used for internal automated testing only, and is not intended to be exposed to the user. The certification team complained that there were no intents for that utterance and that users couldn't call it.] You can reply to the mail they send. But there is no evidence that anyone reads it. Certainly questions I've asked about some of their subjective assessments both in e-mail and in the comments section have gone completely ignored. [E.g. they say that my Blackjack game was deemed "unsuitable for some people" and required me to put in a warning. I pointed out that I had pointed out there was no betting, and that it was, thus, just a card game. No different than "go fish". And that the other blackjack game, approved just a week before I submitted mine, carried no such disclaimer. The resubmission came back with exact same thing on it.] So, given that many areas of the certification "requirements" are subjective, and there is no way to engage in dialog with the people doing certification, I consider the certification process broken. If you have a completely trivial or demo skill, good luck to you. If you have something more serious, or that you are trying to build a business with... I hope you have a large appetite for frustration.

Hi, I'd say that you can probably expect a reply within the next few days.The amount of time it takes to review a skill can depend on a number of factors, including volume of submitted skills and complexity. Of course, we are always trying to improve the certification process, including the time it takes for skills to be certified, and are happy for any suggestions users have regarding how we are doing. Rest assured that we do read all feedback submitted to us. However, we aren't always perfect, so things can sometimes be missed. Thanks, Nick

Nick, >Rest assured that we do read all feedback submitted to us Can you be precise about what feedback you read? Feedback submitted with a skill does not appear to be read. I've had skills returned with comments that make it clear the reviewer has never read the feedback submitted. I suggest you read the feedback submitted with the skill. Feedback submitted in the forum appears to be read, but not engaged. There is no dialog in the forum between the community, and whoever is responsible for feature definition, design and implementation of the Alexa Skill Kit. The only input from Amazon in the forum is answering basic questions or "thank you, we will pass it on to the relevant team". Since there are never any comments from "the relevant team" there is no indication that they pay any attention. I suggest that you get the "relevant team" to engage with the community in the forum. >are happy for any suggestions users have regarding how we are doing. Your wording is very careful. You assure us all comments are 'read', but not that they are listened to. You assure us that you are 'happy' for suggestions, but not that you are going to take them under advisement. Engaging in genuine communication and less weasel words ( https://en.wikipedia.org/wiki/Weasel_word) would be to everyone's benefit.

NickG@Amazon, if you're serious about being interested in feedback on the certification process, I'd really appreciate a running dialogue with someone from the Alexa team about Section 4.1 of your guidelines, as described in this thread: https://forums.developer.amazon.com/forums/thread.jspa?threadID=10158&tstart=0

Thanks for the feedback, I'll give that thread a read through and do some research and see what I find. As said, we've only been doing Alexa certification a few months so I'm sure there's still plenty of things to improve on. -Nick

After we submitted our skill, the Amazon team responded with: "The url you provided for account linking has no place for users to enter account information. This issue blocked our team from fully testing your skill. Please see our documentation on account linking for more information." I responded with a screenshot of the page, and the URL, and asked them to send me a screenshot if they saw anything different. They sent this: "The link you've just sent does appear to be different from what was seen by our certification team when trying to link with your skill. Because your skill is no longer in certification, we cannot capture a screenshot of what we've seen. However, feel free to resubmit your skill if you believe an error has been made and we would be happy to review your account linking page again." I sent a screenshot of the dashboard page for our skill, which plainly showed the same URL, which we have not changed. I can not imagine what URL they looked at. There is only one URL for login. We changed it 2 weeks before we submitted the app. I can not imagine what they saw.

There's a lot of things about the certification process that fall into the "cannot imagine" category. (Like being told how wonderful and innovative my reprompt feature was, but could I please shorten the time down a bit.) It is well documented the ferocious rate of turnover there is in Amazon. At any one time I rather suspect that half the certification team has been on the job less than a month. So there's a 50% chance you are going to be certified by someone with less experience of Alexa than you have. This means they are going to make mistakes (like using the wrong URL, or mistaking an intrinsic Echo feature for a skill feature). Some of them gloriously stupid. That's irritating. But I understand Amazon's constraints. What moves that from irritating to intolerable is the inability to engage in a persistent dialog. Right now we throw our skills over the wall. They are reviewed by a random person. They throw them back. The only recourse is to start the conversation again, potentially with a different person. (You were luck to get any reply out of them at all. I haven't in quite a while.) It seems like it would be operationally trivial to ensure that the same person get assigned to a resubmitted skill. Or that internal notes were kept on a skill, so subsequent certifiers could have access to dialog from previous certification attempts. Or that instead of just chucking the skill back, a grace period for dialog was allowed. Or if their certifiers actually read the notes you submitted with your skill. The lack of any of this makes it hard to professionally engage with skill development.

jjaquinta -- I strongly agree. Amazon needs to fix the many problems with the certification process.

Hi Hankjin, To answer your original question, we provide feedback or publish your skill within 7 business days in most cases, depending on submission volume and testing complexities.