question

hayden avatar image
hayden asked

Testing a Smart Home Skill

I am have written some tests that follow the examples provided in the documentation and it responds correctly to them, however when I use the Smart Home Debugger or the Smart Home Test in the Developer Console the tests fail.

What are the message response types I need to implement for it to pass or at least partially pass?

alexa smart homesmart home
10 |5000

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

KirkC@Amazon avatar image
KirkC@Amazon answered

Broadly speaking, for a given discovered smart home device, you must support ReportState directives, and any other directives associated with the interfaces supported by your device.


For example, if my device supports the Alexa.PowerController interface, I must support that interface's "TurnOn" and "TurnOff" directives:
https://developer.amazon.com/en-US/docs/alexa/device-apis/alexa-powercontroller.html#directives


If my device also supports the Alexa.BrightnessController interface, then I must also support this interface's SetBrightness and AdjustBrightness directives:
https://developer.amazon.com/en-US/docs/alexa/device-apis/alexa-brightnesscontroller.html#directives


To bring it back to your specific scenario, the "Smart Home Test" section of the "Test" tab will allow you to download both the test cases run for your device as well as the test results of those tests. Here's a screen-shot showing where those documents can be downloaded:

test-tab-sh-results.png


Using my own tests as an example take a look at the following test case from my "TestCases" json:

{
    "name": "DevRe_1.0",
    "initialSetups": [
        {
            "capabilityState": {
                "namespace": "Alexa.PowerController",
                "name": "powerState",
                "value": "OFF"
            },
            "directive": {
                "header": {
                    "namespace": "Alexa.PowerController",
                    "name": "TurnOff"
                },
                "payload": {}
            }
        }
    ],
    "directive": {
        "header": {
            "namespace": "Alexa.PowerController",
            "name": "TurnOn"
        },
        "payload": {}
    },
    "expectedCapabilityStates": [
        {
            "namespace": "Alexa.PowerController",
            "name": "powerState",
            "value": "ON"
        }
    ],
    "capabilityTolerances": [
        {
            "namespace": "Alexa.PowerController",
            "name": "powerState",
            "percentThreshold": 0
        }
    ],
    "capabilityTestPlan": "PowerController"
}



In this "DevRe_1.0" test case, you can see that my skill will initially be sent an Alexa.PowerController TurnOff directive so that the capabilityState.value of my device is "OFF" before the test begins.

Then for the test, I will be sent an Alexa.PowerController TurnOn directive. The expected result of this, as indicated by the "expectedCapabilityStates" object, is that my skill will report the device's powerState as "ON"

If you have a look at the "EvaluationResults" JSON, there also exists a "DevRe_1.0" object which shows the results of the test performed above:


{
    "name": "DevRe_1.0",
    "status": "PASSED",
    "expectedCapabilityStates": [
        {
            "namespace": "Alexa.PowerController",
            "name": "powerState",
            "value": "ON"
        }
    ],
    "directive": {
        "header": {
            "namespace": "Alexa.PowerController",
            "name": "TurnOn"
        },
        "payload": {}
    },
    "actualCapabilityStates": [
        {
            "namespace": "Alexa.PowerController",
            "name": "powerState",
            "value": "ON"
        }
    ]
}


As you can see the "status" of the "DevRe_1.0" test is "PASSED" and the "expectedCapabilityStates" matches the "actualCapabilityStates"


10 |5000

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.