question

Well, thanks for spending the time and sharing your results. I did a quick reply to the major points on your blog. But I'll get into a few more details here. >Amazon understands that the certification and community engagement processes still need work I am not certain that what they think needs work is what we think needs work. >Failed process benefits nobody. See. This is the thing. This is why it matters what they see as the role for the Alexa platform. If the process fails for the sorts of things they aren't interested in, then it's not a failure from their perspective. So, you might call me cynical for saying it but, yes, it is a platitude if there is not clarification about what their goals are vis-a-vis what our goals are. >if the people on the call are a representative cross-section It was not. I've been on their side of calls like this before. When you are dealing with a "high maintenance customer" (as we used to call them, and, ironically, I find myself one of!) you get the top people involved. These people were selected intentionally, they had a meeting before hand to discuss how the meeting would go, and they were either all in the same room, or had a back-channel going on during the meeting to coordinate responses. These are the tricks of the trade. It's what I would do. When I had my 4-on-1 meeting with them it was the same spread of people and I was pretty certain the same thing was happening. >the people in a position to handle certification were sort of "at the bottom rung of the ladder" There is clear evidence from multiple replies to certification that [i]frequently[/i] the people doing certification are not subject matter experts. Only one person on your call was involved in doing the actual certification. The rest were part of the "crisis team" to deal with an irritated customer. >Once we were able to explain to them... See, that, in a nutshell, is the [b]actual[/b] problem. Once they were willing to engage you and treat you as something other than an "I learned to code in an hour" moron, you were able to work something out. But if you look at all the promised actions from they call, they are all about changing the [i]guidelines[/i], not the [i]process[/i]. It's not the [i]guidelines[/i] that are broken. At least not severely. And, even the bits that are broken would matter less if the [i]process[/i] was fixed. If Amazon was willing to engage the people developing for their platform more, all the irritation could be better sorted out. It doesn't need to be a hour long call with four of their people. There are plenty things they could do with a lower time investment: * Have the certifiers actually read the testing notes * Set it up so that e-mails can be exchanged during certification. I.e. that it's just not a "throw it over the wall" exchange * Have actual subject matter experts occasionally read and comment in the forums. (Currently the people who respond in the Alexa forum respond in ALL forums for Amazon Mobile App.) * Explain some of Amazon's road map so that there is a better understanding of what their goals for the platform are. Etc, etc. It doesn't look like any outcome of your meeting involved changing the process. Just changing the guidelines. I do not think that will improve things. It will only lead to different subjective points to argue over.

> * Have the certifiers actually read the testing > notes We sort of accidentally caught them on this one during the call. At one point they were describing a potential issue with one of our workarounds, and how it may not work for various reasons. The thing is, we had done considerable testing of it and provided links to our test case results to prove it, and when I asked what they thought about those docs, it became clear nobody had looked at them. To his credit, though, the "Business Guy" on the call totally recognized this and promised to fix that process going forward. > Etc, etc. It doesn't look like any outcome of your > meeting involved changing the process. Just changing > the guidelines. I do not think that will improve > things. It will only lead to different subjective > points to argue over. I'd say this call probably didn't even change any guidelines. In theory there could be a precedent set that others could follow, in terms of specific gray areas, but I doubt that we'll see actually see the rules get outwardly updates as a result. That said, I'm considerably less cynical on whether or not the process will improve. They spent a not-inconsiderable amount of time on this exact topic, to the point that it sort of cut into the time we needed to discuss specific issues. While they obviously didn't provide any details about WHAT would change (aside from reading testing notes, mentioned above), they did put out there that it would be cost prohibitive for them to have to do conference calls with a cross-functional team just to solve simple problems, so it's obviously in their best interest to come up with a solution that mitigates that.

Missed this the first time through > Only > one person on your call was involved in doing the > actual certification. The rest were part of the > "crisis team" to deal with an irritated customer. I'm actually pretty sure that this is not entirely true. The engineer on the call spoke as if she had been consulted on certain aspects of one of our problems. Now, it's possible that she was just briefed before the meeting, as you suggest, but when I randomly compared our situation to that of another skill facing our issue, she was immediately able to explain why the certification for that team had resolved the way it did. That, at the very least, tells me that she was involved in that certification process. Now, this obviously doesn't tell us that non-certifiers are always involved in the certification process. Maybe they start getting outside people involved only when developers start raising concerns. Who knows? But at least in this one case, there is anecdotal evidence that there are discussions with the engineering teams going on behind the scenes.

There is still a lack of transparency about what Amazon wants from the Alexa eco-system, and what they intend to do to achieve their goals. For a bit of perspective, it is worth thinking about Matt Cutts and his role at Google. I recall back in the era of 2008-2010 I worked at a startup that was pushing the limits of what we could do with certain Google APIs. At that time, Matt Cutts was very public as an evangelist. I could send him an email and get a response. If a question about Google APIs came up on Hacker News, Matt Cutts would enter the thread and answer those questions directly. For several years, it was possible to read his blog and learn a huge amount about the Google APIs: https://www.mattcutts.com/blog/ To my mind, that was the ideal way for a tech company to promote a technology to 3rd party developers. I would love to see something like that from Amazon. I would love to see that kind of transparency. I would love to see them appoint an evangelist and say "This person is riding point on Alexa, please direct all of your questions to them." Think about what that would be like, and then compare it to the situation that we actually face today. I find it sobering, as a reality check.

They have that person and his name is David Isbitski, and he is going to be at the Alexa Roadshow in your city on February 25th, as I mentioned on another thread. You should definitely sign up for it. Maybe Tweeting at him is an approach we should be taking, though.

After I met David at the Dev Day in Boston I e-mailed and tweeted him for a while. But he didn't write back much or re-tweet things I tweeted about (like my book launch, which I would have thought was a big deal to the developer community at the time) so I gave up on him as a point of communication. I got the impression he didn't have leeway to stray too far from the message from corporate. More like what I would call "technical sales" rather than "developer advocate". But titles shift meaning every year.