question

I've seen a lot of different people's certification failures, but never seen that before. The only thing that sounds similar to is that a couple people had skills rejected based on the fact that they appeared to be reading books, which the cert team said should be redirected over to the audiobook functionality. It's also very strange that they call out the Echo specifically, since you are submitting a skill for Alexa, not for a specific device. They've gone to great lengths to treat them as distinct-but-related platforms, and this doesn't really match the messaging we've been getting. If I had to guess, I'd imagine there was supposed to be more to that message. Make sure to update this thread once you get clarification.

That's a new one on me. If you tell us a bit more about your skill I can hazard a better guess as to why it was rejected. There are plenty of things in the terms and conditions that would cause it to be rejected. (e.g. pornography) But usually they would cite that specifically.

I agree with the others, this sounds a little odd. The oddest part being that they don't really indicate exactly why the skill is being rejected, their wording here is just slightly obscure. It appears they are saying: "Your skill is inherently tied to the user using an Amazon Echo, and we feel that if the user was using another Alexa Enabled device, they would have a bad user experience." So frankly, a lot really depends on what your skill is, and how it is virtually forcing the user to be using it on an Echo. I can understand Amazon wanting a "skill" to be compatible with all Alexa Enabled devices, especially since they are seeing Alexa as the "Platform" and Echo simply as an implementation. Currently, there is nothing within the Alexa Skills catalog which allows you to indicate which devices it is or isn't compatible with, so a user of any Alexa device could find your skill, and install it, only to find it failed for some reason or another because they didn't install it on the "right" device. If there are clear reasons that an app could/should be able to run on the Echo, but not the Dot, Tap, or whatever, then perhaps Amazon needs to consider a compatibility matrix to allow proper targeting, or you need to consider how to modify your app to "do the right thing" when it is installed on something besides the Echo. -Robert

The app is similar two other apps in the skill store, namely "The Bible" and "Bible" and includes additional references to religious texts. At the time I started developing the app I did not know the other apps existed. In four submission rounds, I have never received feed back it would be better as a audio book.

God knows that Amazon has no policy against duplicate skills. Because, you know, five bit-coin rate quoting skills isn't nearly enough. Do you think that your skill, how the Bible is quoted, what commentary is presented, and/or the other religious texts referenced might be considered offensive to people? It's a fine line when expressing religious or political views and companies tend to be on the conservative side. I don't recall a specific line in the terms and conditions about that, but it's also something they would be vague about anyway. Any company of sufficient side has to be both religiously tolerant, and also not piss off customers. For example, there is a clear prohibition on "sexually explicit" material in Amazon's terms and condition. Now, most of their language refers to visually explicit, which has no meaning on Alexa. But we kind of decided that a "sex positions" skill would get rejected. (Come on! It's a perfect use case for hands free instructions!) Our fallback plan for that idea was to produce a "Kama Sutra" skill. This would be a standard fortune cookie skill reciting parts of Vatsyayana's work. It is, after all, a religious work, and there are tons of passages in there with good advice about parenting and being a dutiful child/spouse. But since the whole thing would really be a sort of bait-and-switch "ha ha, you thought you were getting smut" type of skill, we figured it would probably offend just about anyone. Those looking for smut would be offended because they were getting actual religious advise. Those respectful of Vatsyayana would be offended because of his religious writings being used as a joke. So, although the skill probably met a narrow interpretation of Amazon's terms and conditions, we decided against the project. And this kind of goes back to the problem of inconsistency in certification. At times they interpret the guidelines narrowly (see my thread on the dreaded 4.1 rule) and other times broadly (assuming this is one of those).

Now I get it... I was being far to literal in reading their rejection reason of "not all skills will be made available on Amazon Echo". Instead, being more of a "we don't like your skill" is even odder. Especially since it has to be either subjective, or prescriptive. If subjective, then that means different people would evaluate your skill differently, which isn't sustainable in an app market. If prescriptive, then that means they have some rather firm guidelines they are following I can imagine that there might be some such guidelines which may be difficult to share (example: Amazon has a planned change in how Alexa/Echo works, and they don't want to approve a skill that would be invalidated once the change happens, but they also don't want to tip their hand on what this change will be yet... although I doubt this is the situation in this case). However, I think a bland comment about "we don't have to approve all skills" is far too vague, especially since your skill is "similar" to other skills which did make it through. They should try to give you enough of a rationale so you can consider updating your skill to be more appropriate. ...but without knowing why they chose not to approve the skill, there is really no way in knowing how valid their reason is, or how valid their "not sharing" their reason is. -Robert

> The app is similar two other apps in the skill store, > namely "The Bible" and "Bible" and includes > additional references to religious texts. That's likely your problem - they've started treating Bible skills as "better fit for audiobook". See this thread here: https://forums.developer.amazon.com/forums/thread.jspa?messageID=26944楀 > At the > time I started developing the app I did not know the > other apps existed. It doesn't matter that there are other duplicate skills. They're perfectly happy to let multiple skills compete with each other. > In four submission rounds, I > have never received feed back it would be better as a > audio book. It's weird that they didn't bring this up during your first three rounds. You should call in to the Weekly Dev Office Hours today and ask about this.

tldr; Yes, "not all skills will be made available on Amazon Echo" = app rejected because we don't like it. Update: 1. Never got a response from multiple "Amazon Mobile App Distribution Support Team" request. 2. Never got a response from any request put in the "Testing Instructions" section of the skill submission. 3. Never got a official response here in the forrums. 4. Did get a response from David Isbitski (Chief Alexa Evangelist). Dave was very helpful and I am grateful he replied. He stated. "I forwarded this thread direct to cert team let me know if they send you any updates. I think this may have something to do with invocation phrase and not the content of the skill. I was actually the one who worked with Lifechurch/Youversion on their Bible App so content should not be an issue. =)" 5. Never heard back from the cert team. I changed the "invocation phrase" multiple times and still get the same response below. All indications seem to imply it is the content of the skill, not a technical issue, that is preventing certification. If you want the skill it available at. https://github.com/laynr/Gospel-Library --- Thank you for the recent submission of your skill Gospel Library Your skill submission has failed the certification process on account of the following issues that need to be resolved for successful certification. Request you to please address these issues and resubmit the skill at your earliest convenience. We consider a variety of factors as we evaluate each skill, including the customer experience offered on the device and compatibility with the device. While our goal is to offer a broad and compelling assortment of skills to Amazon customers, not all skills will be made available on Amazon Echo. Please do not reply to this e-mail. To share specific feedback or receive additional clarity on your skill's certification results, please use our contact form here. Please note that you will be directed to a login page before submitting your feedback. Providing your skill's name and application id will assist us in helping you as quickly as possible. For other general questions, we encourage you to visit our forums. These are actively monitored by Amazon employees, and contain many posts by our community that may help to quickly resolve your issue(s). You can also signup to attend our Alexa Skills Kit developer office hours. Additional links: Submission Checklist Regards, The Alexa Skills Team

How long have you been at it? It took 5 months to get Acoustic Tuner to be certified. All it does is play a tone that you can adjust the pitch of your instrument to. In the end it took ranting in the forum to get it over the line. It's pretty annoying. TsaTsaTzu has a fairly comprehensive set of logging tools that we instrument all of our skills with. Because of that, I have some pretty precise measurements of how much effort Amazon puts into certification. The stats below use three certification rounds of Acoustic Tuner as the baseline. These were done between 3/29 and 4/14, so they are recent. There were a lot of start + end sessions done in quick succession. These, I assume, were some sort of automated "is it there" type checking. I discount those from the analysis. Between 3 and 5 unique IDs accessed the skill during the certification processes. Sometimes it's clear that two IDs using the skill in quick succession are, in fact, one person testing enablement. But for most sessions, the bulk of the work is done by a single ID over about half of the total time. Between 50 and 100 minutes were spent cumulatively by Amazon engineers per certification attempt. This includes gaps between session when those gaps were less than ten minutes. Engineers were assumed to be doing paperwork, taking notes, etc, rather than multi-plexing. Each re-certification took less time. Based on glassdoor.com, someone in the Software Engineer I position in Seattle makes $118,750 in total compensation per year. Based on the quality of work, I figured it was being done by interns or new-hires, but I was told by an Amazonian that this was not the case. So I picked this job classification as entry-level-ish. The usual metric to determine cost-to-company is to double the salary cost, which, with rounding, brings it to about $2 a minute. [b]Conclusions[/b] There are a fair number of assumptions above. I'm happy to make my raw data available if someone wants to test their own assumptions. My conclusions here are another level of inference above that. They may be wildly off, but they are probably within the ballpark. Analysis is primarily done by one person, and, it gets shorter each time. If you just look at the time spent by the primary contributor, it gets more dramatically shorter. So there is moderate evidence that the same person gets re-assigned each time. This is a positive for Amazon in that the person is already somewhat familiar with the skill and can do the analysis in less time (and therefore cost less). This is a negative for developers, as any bias the person has developed will be repeated in subsequent certification attempts. In this particular run certain comments from previous certifications were cut and paste into subsequent certifications, with no reference made to the testing notes for feedback submission. Amazon spends from $100 to $200 per certification. With 640 skills in the tab, and assuming it takes 2-3 submissions to get a skill certified, that comes to about a $275,000 investment in certification. Or, in other terms, one full time engineer. Given the size of the teams involved, that isn't a lot. Amazon can probably deal with any spike in skill submissions. So if their aim is quantity, rather than quality (as evidenced by the multiple "write a skill in an hour" articles, tutorials and hack-a-thons), they are well positioned to respond to that. One response to the variability seen in previous certification attempts late last year was to just resubmit the unchanged skill multiple times. With the assumption that eventually the randomness would work in your favor. With this data that recourse seems less likely to succeed. Repeated submissions won't "drain the coffers" of Amazon because it just cost them that much. The switch to having the same person look at it repeatedly seems to have moved the problem from randomness to confirmation bias. In the community call the other day, we were told not to just make changes to get through certification, and then reverse them afterwards. That Amazon will check-point skills and revoke their certification if they are changed later. Although this analysis does conclude the cost of certification isn't that high, if they were to regularly re-certify all the skills in the tabs that would still prove exorbitant. Most likely, if this isn't just theoretical rhetoric, they either respond to user complaints, or else only check-point specifically popular skills. Overall, laynr, this mostly points of flaws in the system, rather than a way to address your specific problem. Amazon's culture is that of a service provider, rather than a business partner. So all of their systems gravitate to a "throw it over the wall" approach. The only thing that I've seen work, sad to say, is harassment. When I've been in this position (as with Acoustic Tuner) I ended up continually resubmitting the skill, posting feedback and bumping the thread in the forum daily. David's a nice guy, so I only wrote to him weekly, rather than daily. All of these things are really just "shooting the messenger". But since Amazon does such a good job at hiding everyone except the messengers, that really is the only recourse. If you pursue this course, keep this thead updated. They do read the forum. And they may eventually get tired of reading this one thread and give you a cogent answer to your questions. You do deserve at least that. Best of luck.

After reading @jjaquinta's reply I tried submitting it one more time under a different Skill ID thinking it may be assigned a different reviewer. It received the same rejection. I interpret the multilateral skill content rejections as being an internal policy decision, but not listed in the public "Policy: Content Guidelines" ( https://developer.amazon.com/public/support/faq#Policy:%20Content%20Guidelines) Created and added to "Rejected Skills List" https://forums.developer.amazon.com/forums/thread.jspa?threadID=12732&tstart=0