Skip to content

Instantly share code, notes, and snippets.

@valgaze

valgaze/actions-on-google.md

Last active May 11, 2020 17:16
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save valgaze/dcd07f6d93f654de6d14d76a341d9450 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save valgaze/dcd07f6d93f654de6d14d76a341d9450 to your computer and use it in GitHub Desktop.
Download ZIP
DialogFlow Payloads
Raw
actions-on-google.md

Actions On Google

simpleResponse, basicCard, carouselBrowse, mediaResponse, tableCard

Rich Response Items

Rich visual elements that deliver resources & visually-interesting content to user

Visual selection responses

AKA Carousel or List, don't use both in a single response

https://developers.google.com/assistant/conversational/selection-responses

Samples

Samples
"Kitchen sink" intent handler
Kitchen Sink JSON Rich Response
Skeleton JSON Rich Response

Resources

Skeleton response

{
    "webhookPayload": {
        "google": {
            "systemIntent": {
                "intent": "actions.intent.OPTION",
                "data": {
                    "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
                    "listSelect": {
                        "title": "List Title",
                        "items": []
                    },
                    "carouselSelect": {
                        "items": []
                    }
                },
                "richResponse": {
                    "items": [],
                    "suggestions": [],
                    "linkOutSuggestion": {}
                }
            }
        }
    }
}

simpleResponse

https://developers.google.com/assistant/conversational/simple-responses#SimpleResponseSamples

Note: This must be included as a fallback if any rich responses are used as a fallback

Note: You can optionally pass "speech" for voice to text (in that case, text is available on textToSpeech rather than displayText)

Usage

conv.ask(new SimpleResponse({
  speech: 'this is speech too! [textToSpeech]'
}))

conv.add('this is speech too! [textToSpeech]')
conv.ask('this is speech too! [textToSpeech]')

Response (webhookPayload.google.richResponse.items)

All 3 above will return this output

{
  "simpleResponse": {
    "textToSpeech": "this is speech too! [textToSpeech]"
  }
}

Usage

conv.ask(
    new SimpleResponse({
      text: `Abcdefg Hijklmop`
    })
);

Response (webhookPayload.google.richResponse.items)

{
	"simpleResponse": {
		"displayText": "Abcdefg Hijklmop"
	}
}

Usage

conv.ask(
    new SimpleResponse({
      speech: 'This is speech [textToSpeech]',
      text: 'This is display text [displayText]'
    })
 )

**Response (webhookPayload.google.richResponse.items)**

```json
{
	"simpleResponse": {
		"textToSpeech": "This is speech [textToSpeech]",
		"displayText": "This is display text [displayText]"
	}
}

⬆️_TOP

BasicCard

https://developers.google.com/assistant/conversational/rich-responses#BasicCardSamples

Usage

  app.intent('my intent name', (conv, parameters) => {
  
    const flavor = 'mint'
    const size = 'large'
    const imageURL = `https://i.imgur.com/W9Eeuu1.jpg`

    conv.ask(
      new BasicCard({
        text: `Here is your 🍦!!! This type of ${flavor} ice cream is great  \nEverybody after this line break loves it.`,
        subtitle: "Here's your subtitle",
        title: `Here's your ${size} ${flavor}`,
        buttons: new Button({
          title: `Learn more about ${flavor}`,
          url: `https://duckduckgo.com/?q=${flavor}+ice+cream`
        }),
        image: new Image({
          url: imageURL,
          alt: `${flavor} mmm...`
        })
      })
    );

    
  })

Response (webhookPayload.google.richResponse.items)

{
	"basicCard": {
		"title": "Here's your large mint",
		"subtitle": "Here's your subitle",
		"formattedText": "Here is your 🍦!!! This type of mint ice cream is great",
		"image": {
			"url": "https://i.imgur.com/W9Eeuu1.jpg",
			"accessibilityText": "mint mmm..."
		},
		"buttons": [{
			"title": "Learn more about mint",
			"openUrlAction": {
				"url": "https://duckduckgo.com/?q=mint+ice+cream"
			}
		}]
	}
}

⬆️_TOP

Image

Note: Images on their own get wrapped in a card

Note: Image URLs must be publicly accessible (or you can base64, ex data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHwAAACECAYAAABS30)

Usage

  app.intent('my intent name', (conv, parameters) => {
  
    conv.ask(new Image({
      url:
        "http://storage.googleapis.com/automotive-media/album_art.jpg",
         alt: "Image becomes a card"
    }))
	  
  })

Response (webhookPayload.google.richResponse.items)

{
	"basicCard": {
		"image": {
			"url": "http://storage.googleapis.com/automotive-media/album_art.jpg",
			"accessibilityText": "Image becomes a card"
		}
	}
}

⬆️_TOP

Suggestions/"Chips"

https://developers.google.com/assistant/conversational/rich-responses#suggestion_chips

Note: Images on their own get wrapped in a card

Usage

  app.intent('my intent name', (conv, parameters) => {
  
    // Suggestions
    conv.ask(new Suggestions(['Suggestion 1', 'Suggestion 2']));

	  
  })

Response (webhookPayload.google.richResponse.suggestions)

{
    "suggestions": [{
        "title": "Suggestion 1"
    }, {
        "title": "Suggestion 2"
    }]
}

⬆️_TOP

Suggestion B: "Linkout Suggestion"

Note: only one linkout suggestion per response. URL must be valid

Usage

  app.intent('my intent name', (conv, parameters) => {
  
    // LinkOut Suggestion
    conv.ask(new LinkOutSuggestion({
      name: 'Suggestion bongo',
      url: 'https://assistant.google.com/',
    }));
	  
  })

Response (webhookPayload.google.richResponse.suggestions)

{
    "suggestions": [],
     "linkOutSuggestion": {
   	"destinationName": "Jams",
   	 "url": "https://www.youtube.com/watch?v=UzYibo3igGU"
   }
}

⬆️_TOP

tableCard

https://developers.google.com/assistant/conversational/rich-responses#TableCardSamples

Usage

  app.intent('my intent name', (conv, parameters) => {
  
    // Table
    conv.ask(new Table({
      dividers: true,
      columns: ['header 1', 'header 2', 'header 3'],
      rows: [
        ['row 1 item 1', 'row 1 item 2', 'row 1 item 3'],
        ['row 2 item 1', 'row 2 item 2', 'row 2 item 3'],
      ],
    }));
	  
  })

Response (webhookPayload.google.richResponse.suggestions)

{
	"tableCard": {
		"rows": [{
			"cells": [{
				"text": "row 1 item 1"
			}, {
				"text": "row 1 item 2"
			}, {
				"text": "row 1 item 3"
			}],
			"dividerAfter": true
		}, {
			"cells": [{
				"text": "row 2 item 1"
			}, {
				"text": "row 2 item 2"
			}, {
				"text": "row 2 item 3"
			}],
			"dividerAfter": true
		}],
		"columnProperties": [{
			"header": "header 1"
		}, {
			"header": "header 2"
		}, {
			"header": "header 3"
		}]
	}
}

⬆️_TOP

mediaResponse

https://developers.google.com/assistant/conversational/rich-responses#MediaResponseSamples

Note: Resource URL must be publicly accessible

Usage

  app.intent('my intent name', (conv, parameters) => {
    // Media Object
    conv.ask(
      new MediaObject({
        name: "Jazz in Paris",
        url:
          "http://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
        description: "A funky Jazz tune",
        icon: new Image({
          url:
            "http://storage.googleapis.com/automotive-media/album_art.jpg",
          alt: "Media icon"
        })
      })
    )
	  
  })

Response (webhookPayload.google.richResponse.items)

{
    "mediaResponse": {
        "mediaType": "AUDIO",
        "mediaObjects": [{
            "contentUrl": "http://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
            "description": "A funky Jazz tune",
            "icon": {
                "url": "http://storage.googleapis.com/automotive-media/album_art.jpg",
                "accessibilityText": "Media icon"
            },
            "name": "Jazz in Paris"
        }]
    }
}

⬆️_TOP

carouselBrowse

https://developers.google.com/assistant/conversational/rich-responses#BrowsingCarouselSamples

Note: Resource URL must be publicly accessible

Usage

  app.intent('my intent name', (conv, parameters) => {
    // Browse Carousel
    
    conv.ask(new BrowseCarousel({
      items: [
        new BrowseCarouselItem({
          title: 'Title of item 1',
          url: 'https://example.com',
          description: 'Description of item 1',
          image: new Image({
            url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
            alt: 'Image alternate text',
          }),
          footer: 'Item 1 footer',
        }),
        new BrowseCarouselItem({
          title: 'Title of item 2',
          url: 'https://example.com',
          description: 'Description of item 2',
          image: new Image({
            url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
            alt: 'Image alternate text',
          }),
          footer: 'Item 2 footer',
        }),
      ],
    }));

	  
  })

Response (webhookPayload.google.richResponse.items)

        {
                        "carouselBrowse": {
                            "items": [
                                {
                                    "title": "Title of item 1",
                                    "openUrlAction": {
                                        "url": "https://example.com"
                                    },
                                    "description": "Description of item 1",
                                    "footer": "Item 1 footer",
                                    "image": {
                                        "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                                        "accessibilityText": "Image alternate text"
                                    }
                                },
                                {
                                    "title": "Title of item 2",
                                    "openUrlAction": {
                                        "url": "https://example.com"
                                    },
                                    "description": "Description of item 2",
                                    "footer": "Item 2 footer",
                                    "image": {
                                        "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                                        "accessibilityText": "Image alternate text"
                                    }
                                }
                            ]
                        }
                    }

⬆️_TOP

carouselSelect

Note: Taps should emit event actions_intent_OPTION (hopefully an intent "listening" for that to handle)

Usage

  app.intent('my intent name', (conv, parameters) => {
	  
	  
  // Constants for list and carousel selection
  const SELECTION_KEY_GOOGLE_ASSISTANT = "googleAssistant";
  const SELECTION_KEY_GOOGLE_PAY = "googlePay";
  const SELECTION_KEY_GOOGLE_PIXEL = "googlePixel";
  const SELECTION_KEY_GOOGLE_HOME = "googleHome";

  // Constant for image URLs
  const IMG_URL_AOG =
    "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png";
  const IMG_URL_GOOGLE_PAY =
    "https://storage.googleapis.com/actionsresources/logo_pay_64dp.png";
  const IMG_URL_GOOGLE_PIXEL =
    "https://storage.googleapis.com/madebygoog/v1/Pixel/Pixel_ColorPicker/Pixel_Device_Angled_Black-720w.png";
  const IMG_URL_GOOGLE_HOME =
    "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw";

    // Carousel select
    conv.ask(
      new Carousel({
        items: {
          // Add the first item to the carousel
          [SELECTION_KEY_GOOGLE_ASSISTANT]: {
            synonyms: ["Assistant", "Google Assistant"],
            title: "Item #1",
            description: "Description of Item #1",
            image: new Image({
              url: IMG_URL_AOG,
              alt: "Google Assistant logo"
            })
          },
          // Add the second item to the carousel
          [SELECTION_KEY_GOOGLE_PAY]: {
            synonyms: ["Transactions", "Google Payments"],
            title: "Item #2",
            description: "Description of Item #2",
            image: new Image({
              url: IMG_URL_GOOGLE_PAY,
              alt: "Google Pay logo"
            })
          },
          // Add third item to the carousel
          [SELECTION_KEY_GOOGLE_PIXEL]: {
            synonyms: ["Pixel", "Google Pixel phone"],
            title: "Item #3",
            description: "Description of Item #3",
            image: new Image({
              url: IMG_URL_GOOGLE_PIXEL,
              alt: "Google Pixel phone"
            })
          },
          // Add last item of the carousel
          [SELECTION_KEY_GOOGLE_HOME]: {
            title: "Item #4",
            synonyms: ["Google Home"],
            description: "Description of Item #4",
            image: new Image({
              url: IMG_URL_GOOGLE_HOME,
              alt: "Google Home"
            })
          }
        }
      })
    );
  })

Response (webhookPayload.google.systemIntent.data.carouselSelect)

{
    "webhookPayload": {
        "google": {
            "richResponse": {
                "items": [],
                "suggestions": []
            },
            "systemIntent": {
                "intent": "actions.intent.OPTION",
                "data": {
                    "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
                    "carouselSelect": {
                        "items": [{
                                "optionInfo": {
                                    "key": "googleAssistant",
                                    "synonyms": [
                                        "Assistant",
                                        "Google Assistant"
                                    ]
                                },
                                "description": "Description of Item #1",
                                "image": {
                                    "url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
                                    "accessibilityText": "Google Assistant logo"
                                },
                                "title": "Item #1"
                            },
                            {
                                "optionInfo": {
                                    "key": "googlePay",
                                    "synonyms": [
                                        "Transactions",
                                        "Google Payments"
                                    ]
                                },
                                "description": "Description of Item #2",
                                "image": {
                                    "url": "https://storage.googleapis.com/actionsresources/logo_pay_64dp.png",
                                    "accessibilityText": "Google Pay logo"
                                },
                                "title": "Item #2"
                            },
                            {
                                "optionInfo": {
                                    "key": "googlePixel",
                                    "synonyms": [
                                        "Pixel",
                                        "Google Pixel phone"
                                    ]
                                },
                                "description": "Description of Item #3",
                                "image": {
                                    "url": "https://storage.googleapis.com/madebygoog/v1/Pixel/Pixel_ColorPicker/Pixel_Device_Angled_Black-720w.png",
                                    "accessibilityText": "Google Pixel phone"
                                },
                                "title": "Item #3"
                            },
                            {
                                "optionInfo": {
                                    "key": "googleHome",
                                    "synonyms": [
                                        "Google Home"
                                    ]
                                },
                                "description": "Description of Item #4",
                                "image": {
                                    "url": "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw",
                                    "accessibilityText": "Google Home"
                                },
                                "title": "Item #4"
                            }
                        ]
                    }
                }
            }
        }
    }
}

⬆️_TOP

listSelect

Note: Taps should emit event actions_intent_OPTION (hopefully an intent "listening" for that to handle)

Note: title & option keys are required fields

Usage

  app.intent('my intent name', (conv, parameters) => {
    // Constants for list and carousel selection
    const SELECTION_KEY_GOOGLE_ASSISTANT = "googleAssistant";
    const SELECTION_KEY_GOOGLE_PAY = "googlePay";
    const SELECTION_KEY_GOOGLE_PIXEL = "googlePixel";
    const SELECTION_KEY_GOOGLE_HOME = "googleHome";

    // Constant for image URLs
    const IMG_URL_AOG =
      "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png";
    const IMG_URL_GOOGLE_PAY =
      "https://storage.googleapis.com/actionsresources/logo_pay_64dp.png";
    const IMG_URL_GOOGLE_PIXEL =
      "https://storage.googleapis.com/madebygoog/v1/Pixel/Pixel_ColorPicker/Pixel_Device_Angled_Black-720w.png";
    const IMG_URL_GOOGLE_HOME =
      "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw";

    conv.ask(new List({
      title: 'List Title',
      items: {
        // Add the first item to the list
        [SELECTION_KEY_GOOGLE_ASSISTANT]: {
          synonyms: [
            'Assistant',
            'Google Assistant',
          ],
          title: 'Item #1',
          description: 'Description of Item #1',
          image: new Image({
            url: 'https://www.gstatic.com/images/branding/product/2x/assistant_48dp.png',
            alt: 'Google Assistant logo',
          }),
        },
        // Add the second item to the list
        [SELECTION_KEY_GOOGLE_PAY]: {
          synonyms: [
            'Transactions',
            'Google Payments',
            'Google Pay',
          ],
          title: 'Item #2',
          description: 'Description of Item #2',
          image: new Image({
            url: 'https://www.gstatic.com/images/branding/product/2x/pay_48dp.png',
            alt: 'Google Pay logo',
          }),
        },
        // Add the third item to the list
        [SELECTION_KEY_GOOGLE_PIXEL]: {
          synonyms: [
            'Pixel',
            'Google Pixel',
            'Pixel phone'
          ],
          title: 'Item #3',
          description: 'Description of Item #3',
          image: new Image({
            url: 'https://storage.googleapis.com/madebygoog/v1/Pixel/Pixel_ColorPicker/Pixel_Device_Angled_Black-720w.png',
            alt: 'Google Pixel phone',
          }),
        },
        // Add the last item to the list
        [SELECTION_KEY_GOOGLE_HOME]: {
          title: 'Item #4',
          synonyms: [
            'Home',
            'Google Home',
          ],
          description: 'Description of Item #4',
          image: new Image({
            url: 'https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw',
            alt: 'Google Home',
          }),
        },
      },
    }));  
	  
	  
  })

Response (webhookPayload.google.systemIntent.data.listSelect)

{
    "webhookPayload": {
        "google": {
            "richResponse": {
                "items": [],
                "suggestions": []
            },
            "systemIntent": {
                "intent": "actions.intent.OPTION",
                "data": {
                    "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
                    "listSelect": {
                        "title": "List Title",
                        "items": [{
                                "optionInfo": {
                                    "key": "googleAssistant",
                                    "synonyms": [
                                        "Assistant",
                                        "Google Assistant"
                                    ]
                                },
                                "description": "Description of Item #1",
                                "image": {
                                    "url": "https://www.gstatic.com/images/branding/product/2x/assistant_48dp.png",
                                    "accessibilityText": "Google Assistant logo"
                                },
                                "title": "Item #1"
                            },
                            {
                                "optionInfo": {
                                    "key": "googlePay",
                                    "synonyms": [
                                        "Transactions",
                                        "Google Payments",
                                        "Google Pay"
                                    ]
                                },
                                "description": "Description of Item #2",
                                "image": {
                                    "url": "https://www.gstatic.com/images/branding/product/2x/pay_48dp.png",
                                    "accessibilityText": "Google Pay logo"
                                },
                                "title": "Item #2"
                            },
                            {
                                "optionInfo": {
                                    "key": "googlePixel",
                                    "synonyms": [
                                        "Pixel",
                                        "Google Pixel",
                                        "Pixel phone"
                                    ]
                                },
                                "description": "Description of Item #3",
                                "image": {
                                    "url": "https://storage.googleapis.com/madebygoog/v1/Pixel/Pixel_ColorPicker/Pixel_Device_Angled_Black-720w.png",
                                    "accessibilityText": "Google Pixel phone"
                                },
                                "title": "Item #3"
                            },
                            {
                                "optionInfo": {
                                    "key": "googleHome",
                                    "synonyms": [
                                        "Home",
                                        "Google Home"
                                    ]
                                },
                                "description": "Description of Item #4",
                                "image": {
                                    "url": "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw",
                                    "accessibilityText": "Google Home"
                                },
                                "title": "Item #4"
                            }
                        ]
                    }
                }
            }
        }
    }
}

⬆️_TOP

Raw
kitchensink.js
const {
Image,
Carousel,
BrowseCarousel,
BrowseCarouselItem,
BasicCard,
List,
Suggestion,
MediaObject,
Suggestions,
SimpleResponse,
Table,
Button,
LinkOutSuggestion,
} = require("actions-on-google");
module.exports = function(app, intentName = "intent") {
app.intent(intentName, (conv, parameters) => {
const choice = Math.random() > 0.5 ? 'carousel' : 'list'
// Constants for list and carousel selection
const SELECTION_KEY_GOOGLE_ASSISTANT = "googleAssistant";
const SELECTION_KEY_GOOGLE_PAY = "googlePay";
const SELECTION_KEY_GOOGLE_PIXEL = "googlePixel";
const SELECTION_KEY_GOOGLE_HOME = "googleHome";
// Constant for image URLs
const IMG_URL_AOG =
"https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png";
const IMG_URL_GOOGLE_PAY =
"https://storage.googleapis.com/actionsresources/logo_pay_64dp.png";
const IMG_URL_GOOGLE_PIXEL =
"https://storage.googleapis.com/madebygoog/v1/Pixel/Pixel_ColorPicker/Pixel_Device_Angled_Black-720w.png";
const IMG_URL_GOOGLE_HOME =
"https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw";
// Text
conv.ask(
new SimpleResponse({
text: `This is simple text`, // returned as simpleResponse.displayText
// speech: `this is vocal' // returned as simpleResponse.textToSpeech`
})
);
// equivalent to simpleResponse
conv.ask(`This is simple text too...`)
// Suggestions
conv.ask(new Suggestions(['Suggestion 1', 'Suggestion 2']));
// Link out suggestion
// NOTE: Only 1 is permitted
conv.ask(new LinkOutSuggestion({
name: 'Suggestion bongo',
url: 'https://assistant.google.com/',
}));
// Image
// Note: Images without anything else become a card
conv.ask(new Image({
url: "http://storage.googleapis.com/automotive-media/album_art.jpg",
alt: "Image becomes a card"
}))
// Media Object
conv.ask(
new MediaObject({
name: "Jazz in Paris",
url: "http://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
description: "A funky Jazz tune",
icon: new Image({
url: "http://storage.googleapis.com/automotive-media/album_art.jpg",
alt: "Media icon"
})
})
)
// BasicCard
const flavor = 'mint'
const size = 'large'
const imageURL = `https://i.imgur.com/W9Eeuu1.jpg`
conv.ask(
new BasicCard({
text: `Here is your 🍦!!! This type of ${flavor} ice cream is great \nEverybody after this line break loves it.`,
subtitle: "Build Actions",
title: `Here's your ${size} ${flavor}`,
buttons: new Button({
title: `Learn more about ${flavor}`,
url: `https://duckduckgo.com/?q=${flavor}+ice+cream`
}),
image: new Image({
url: imageURL,
alt: `${flavor} mmm...`
})
})
);
// Table
conv.ask(new Table({
dividers: true,
columns: ['header 1', 'header 2', 'header 3'],
rows: [
['row 1 item 1', 'row 1 item 2', 'row 1 item 3'],
['row 2 item 1', 'row 2 item 2', 'row 2 item 3'],
],
}));
// Browse Carousel
conv.ask(new BrowseCarousel({
items: [
new BrowseCarouselItem({
title: 'Title of item 1',
url: 'https://example.com',
description: 'Description of item 1',
image: new Image({
url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
alt: 'Image alternate text',
}),
footer: 'Item 1 footer',
}),
new BrowseCarouselItem({
title: 'Title of item 2',
url: 'https://example.com',
description: 'Description of item 2',
image: new Image({
url: 'https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png',
alt: 'Image alternate text',
}),
footer: 'Item 2 footer',
}),
],
}));
if (choice === 'carousel') {
// Carousel
conv.ask(
new Carousel({
items: {
// Add the first item to the carousel
[SELECTION_KEY_GOOGLE_ASSISTANT]: {
synonyms: ["Assistant", "Google Assistant"],
title: "Item #1",
description: "Description of Item #1",
image: new Image({
url: IMG_URL_AOG,
alt: "Google Assistant logo"
})
},
// Add the second item to the carousel
[SELECTION_KEY_GOOGLE_PAY]: {
synonyms: ["Transactions", "Google Payments"],
title: "Item #2",
description: "Description of Item #2",
image: new Image({
url: IMG_URL_GOOGLE_PAY,
alt: "Google Pay logo"
})
},
// Add third item to the carousel
[SELECTION_KEY_GOOGLE_PIXEL]: {
synonyms: ["Pixel", "Google Pixel phone"],
title: "Item #3",
description: "Description of Item #3",
image: new Image({
url: IMG_URL_GOOGLE_PIXEL,
alt: "Google Pixel phone"
})
},
// Add last item of the carousel
[SELECTION_KEY_GOOGLE_HOME]: {
title: "Item #4",
synonyms: ["Google Home"],
description: "Description of Item #4",
image: new Image({
url: IMG_URL_GOOGLE_HOME,
alt: "Google Home"
})
}
}
})
);
} else if (choice === 'list') {
// List
conv.ask(new List({
title: 'List Title',
items: {
// Add the first item to the list
[SELECTION_KEY_GOOGLE_ASSISTANT]: {
synonyms: [
'Assistant',
'Google Assistant',
],
title: 'Item #1',
description: 'Description of Item #1',
image: new Image({
url: 'https://www.gstatic.com/images/branding/product/2x/assistant_48dp.png',
alt: 'Google Assistant logo',
}),
},
// Add the second item to the list
[SELECTION_KEY_GOOGLE_PAY]: {
synonyms: [
'Transactions',
'Google Payments',
'Google Pay',
],
title: 'Item #2',
description: 'Description of Item #2',
image: new Image({
url: 'https://www.gstatic.com/images/branding/product/2x/pay_48dp.png',
alt: 'Google Pay logo',
}),
},
// Add the third item to the list
[SELECTION_KEY_GOOGLE_PIXEL]: {
synonyms: [
'Pixel',
'Google Pixel',
'Pixel phone'
],
title: 'Item #3',
description: 'Description of Item #3',
image: new Image({
url: 'https://storage.googleapis.com/madebygoog/v1/Pixel/Pixel_ColorPicker/Pixel_Device_Angled_Black-720w.png',
alt: 'Google Pixel phone',
}),
},
// Add the last item to the list
[SELECTION_KEY_GOOGLE_HOME]: {
title: 'Item #4',
synonyms: [
'Home',
'Google Home',
],
description: 'Description of Item #4',
image: new Image({
url: 'https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw',
alt: 'Google Home',
}),
},
},
}));
}
});
}
Raw
kitchensink_response.json
{
"responseId": "11111111-11111111-11111111-11111111",
"queryResult": {
"fulfillmentMessages": [],
"outputContexts": [],
"queryText": "kitchen sink",
"speechRecognitionConfidence": 1,
"action": "",
"parameters": {},
"allRequiredParamsPresent": true,
"fulfillmentText": "",
"webhookPayload": null,
"intent": {
"outputContexts": [],
"parameters": [],
"name": "projects/bongo-aaabbbccc/agent/intents/11111111-11111111-11111111-11111111",
"displayName": "Demo"
},
"intentDetectionConfidence": 1,
"languageCode": "en",
},
"session": "12345678989765643",
"webhookPayload": {
"google": {
"expectUserResponse": true,
"systemIntent": {
"intent": "actions.intent.OPTION",
"data": {
"@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
"carouselSelect": {
"items": [
{
"optionInfo": {
"key": "googleAssistant",
"synonyms": [
"Assistant",
"Google Assistant"
]
},
"description": "Description of Item #1",
"image": {
"url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
"accessibilityText": "Google Assistant logo"
},
"title": "Item #1"
},
{
"optionInfo": {
"key": "googlePay",
"synonyms": [
"Transactions",
"Google Payments"
]
},
"description": "Description of Item #2",
"image": {
"url": "https://storage.googleapis.com/actionsresources/logo_pay_64dp.png",
"accessibilityText": "Google Pay logo"
},
"title": "Item #2"
},
{
"optionInfo": {
"key": "googlePixel",
"synonyms": [
"Pixel",
"Google Pixel phone"
]
},
"description": "Description of Item #3",
"image": {
"url": "https://storage.googleapis.com/madebygoog/v1/Pixel/Pixel_ColorPicker/Pixel_Device_Angled_Black-720w.png",
"accessibilityText": "Google Pixel phone"
},
"title": "Item #3"
},
{
"optionInfo": {
"key": "googleHome",
"synonyms": [
"Google Home"
]
},
"description": "Description of Item #4",
"image": {
"url": "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw",
"accessibilityText": "Google Home"
},
"title": "Item #4"
}
]
}
}
},
"richResponse": {
"items": [
{
"simpleResponse": {
"displayText": "This is simple text"
}
},
{
"simpleResponse": {
"textToSpeech": "This is simple text too..."
}
},
{
"basicCard": {
"image": {
"url": "http://storage.googleapis.com/automotive-media/album_art.jpg",
"accessibilityText": "Image becomes a card"
}
}
},
{
"mediaResponse": {
"mediaType": "AUDIO",
"mediaObjects": [
{
"contentUrl": "http://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
"description": "A funky Jazz tune",
"icon": {
"url": "http://storage.googleapis.com/automotive-media/album_art.jpg",
"accessibilityText": "Media icon"
},
"name": "Jazz in Paris"
}
]
}
},
{
"basicCard": {
"title": "Here's your large mint",
"subtitle": "Build Actions",
"formattedText": "Here is your 🍦!!! This type of mint ice cream is great \nEverybody after this line break loves it.",
"image": {
"url": "https://i.imgur.com/W9Eeuu1.jpg",
"accessibilityText": "mint mmm..."
},
"buttons": [
{
"title": "Learn more about mint",
"openUrlAction": {
"url": "https://duckduckgo.com/?q=mint+ice+cream"
}
}
]
}
},
{
"tableCard": {
"rows": [
{
"cells": [
{
"text": "row 1 item 1"
},
{
"text": "row 1 item 2"
},
{
"text": "row 1 item 3"
}
],
"dividerAfter": true
},
{
"cells": [
{
"text": "row 2 item 1"
},
{
"text": "row 2 item 2"
},
{
"text": "row 2 item 3"
}
],
"dividerAfter": true
}
],
"columnProperties": [
{
"header": "header 1"
},
{
"header": "header 2"
},
{
"header": "header 3"
}
]
}
},
{
"carouselBrowse": {
"items": [
{
"title": "Title of item 1",
"openUrlAction": {
"url": "https://example.com"
},
"description": "Description of item 1",
"footer": "Item 1 footer",
"image": {
"url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
"accessibilityText": "Image alternate text"
}
},
{
"title": "Title of item 2",
"openUrlAction": {
"url": "https://example.com"
},
"description": "Description of item 2",
"footer": "Item 2 footer",
"image": {
"url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
"accessibilityText": "Image alternate text"
}
}
]
}
}
],
"suggestions": [
{
"title": "Suggestion 1"
},
{
"title": "Suggestion 2"
}
],
"linkOutSuggestion": {
"destinationName": "Suggestion bongo",
"url": "https://assistant.google.com/"
}
}
}
},
"fulfillmentText": ""
}
Raw
skeleton_response.json
{
"webhookPayload": {
"google": {
"systemIntent": {
"intent": "actions.intent.OPTION",
"data": {
"@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
"listSelect": {
"title": "List Title",
"items": []
},
"carouselSelect": {
"items": []
}
},
"richResponse": {
"items": [],
"suggestions": [],
"linkOutSuggestion": {}
}
}
}
}
}
Raw
v2.ts
/**
* Copyright 2018 Google Inc. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// DO NOT MANUALLY EDIT: this file contains types generated from protobuf messages
/* tslint:disable:no-any max-line-length auto generated from protobufs */
/*
From here: https://github.com/actions-on-google/actions-on-google-nodejs/blob/master/src/service/actionssdk/api/v2.ts
https://developers.google.com/assistant/conversational/rich-responses
*/
// webhookPayload
export interface GoogleCloudDialogflowV2webhookPayload {
"systemIntent"?: {
"intent": string,
"data"?: {
"@type": string,
"listSelect": GoogleCloudDialogflowV2IntentMessageListSelect,
"carouselSelect": GoogleCloudDialogflowV2IntentMessageCarouselSelect
},
"richResponse": {
"items": (GoogleCloudDialogflowV2IntentMessage)[],
"suggestions": GoogleCloudDialogflowV2IntentMessageSuggestions[],
"linkOutSuggestion": GoogleCloudDialogflowV2IntentMessageLinkOutSuggestion
}
}
}
// https://github.com/actions-on-google/actions-on-google-nodejs/blob/master/src/common.ts#L84
/** @hidden */
export interface ApiClientObjectMap<TValue> {
[key: string]: TValue
}
export type GoogleCloudDialogflowV2IntentDefaultResponsePlatforms =
'PLATFORM_UNSPECIFIED'|'FACEBOOK'|'SLACK'|'TELEGRAM'|'KIK'|'SKYPE'|'LINE'|
'VIBER'|'ACTIONS_ON_GOOGLE'
export type GoogleCloudDialogflowV2IntentMessagePlatform =
'PLATFORM_UNSPECIFIED'|'FACEBOOK'|'SLACK'|'TELEGRAM'|'KIK'|'SKYPE'|'LINE'|
'VIBER'|'ACTIONS_ON_GOOGLE'
export type GoogleCloudDialogflowV2IntentTrainingPhraseType =
'TYPE_UNSPECIFIED'|'EXAMPLE'|'TEMPLATE'
export type GoogleCloudDialogflowV2IntentWebhookState =
'WEBHOOK_STATE_UNSPECIFIED'|'WEBHOOK_STATE_ENABLED'|
'WEBHOOK_STATE_ENABLED_FOR_SLOT_FILLING'
export interface GoogleCloudDialogflowV2Context {
name?: string
lifespanCount?: number
parameters?: ApiClientObjectMap<any>
}
export interface GoogleCloudDialogflowV2EventInput {
name?: string
parameters?: ApiClientObjectMap<any>
languageCode?: string
}
export interface GoogleCloudDialogflowV2Intent {
name?: string
displayName?: string
webhookState?: GoogleCloudDialogflowV2IntentWebhookState
priority?: number
isFallback?: boolean
mlDisabled?: boolean
inputContextNames?: string[]
events?: string[]
trainingPhrases?: GoogleCloudDialogflowV2IntentTrainingPhrase[]
action?: string
outputContexts?: GoogleCloudDialogflowV2Context[]
resetContexts?: boolean
parameters?: GoogleCloudDialogflowV2IntentParameter[]
messages?: GoogleCloudDialogflowV2IntentMessage[]
defaultResponsePlatforms?:
GoogleCloudDialogflowV2IntentDefaultResponsePlatforms[]
rootFollowupIntentName?: string
parentFollowupIntentName?: string
followupIntentInfo?: GoogleCloudDialogflowV2IntentFollowupIntentInfo[]
}
export interface GoogleCloudDialogflowV2IntentFollowupIntentInfo {
followupIntentName?: string
parentFollowupIntentName?: string
}
export interface GoogleCloudDialogflowV2IntentMessage {
text?: GoogleCloudDialogflowV2IntentMessageText
image?: GoogleCloudDialogflowV2IntentMessageImage
quickReplies?: GoogleCloudDialogflowV2IntentMessageQuickReplies
card?: GoogleCloudDialogflowV2IntentMessageCard
payload?: ApiClientObjectMap<any>
simpleResponses?: GoogleCloudDialogflowV2IntentMessageSimpleResponses
basicCard?: GoogleCloudDialogflowV2IntentMessageBasicCard
suggestions?: GoogleCloudDialogflowV2IntentMessageSuggestions
linkOutSuggestion?: GoogleCloudDialogflowV2IntentMessageLinkOutSuggestion
listSelect?: GoogleCloudDialogflowV2IntentMessageListSelect
carouselSelect?: GoogleCloudDialogflowV2IntentMessageCarouselSelect
platform?: GoogleCloudDialogflowV2IntentMessagePlatform
}
export interface GoogleCloudDialogflowV2IntentMessageBasicCard {
title?: string
subtitle?: string
formattedText?: string
image?: GoogleCloudDialogflowV2IntentMessageImage
buttons?: GoogleCloudDialogflowV2IntentMessageBasicCardButton[]
}
export interface GoogleCloudDialogflowV2IntentMessageBasicCard {
title?: string
openUriAction?:
GoogleCloudDialogflowV2IntentMessageBasicCardButtonOpenUriAction
}
export interface GoogleCloudDialogflowV2IntentMessageBasicCardButtonOpenUriAction {
uri?: string
}
export interface GoogleCloudDialogflowV2IntentMessageCard {
title?: string
subtitle?: string
imageUri?: string
buttons?: GoogleCloudDialogflowV2IntentMessageCardButton[]
}
export interface GoogleCloudDialogflowV2IntentMessageCardButton {
text?: string
postback?: string
}
export interface GoogleCloudDialogflowV2IntentMessageCarouselSelect {
items?: GoogleCloudDialogflowV2IntentMessageCarouselSelectItem[]
}
export interface GoogleCloudDialogflowV2IntentMessageCarouselSelectItem {
info?: GoogleCloudDialogflowV2IntentMessageSelectItemInfo
title?: string
description?: string
image?: GoogleCloudDialogflowV2IntentMessageImage
}
export interface GoogleCloudDialogflowV2IntentMessageImage {
imageUri?: string
accessibilityText?: string
}
export interface GoogleCloudDialogflowV2IntentMessageLinkOutSuggestion {
destinationName?: string
uri?: string
}
export interface GoogleCloudDialogflowV2IntentMessageListSelect {
title?: string
items?: GoogleCloudDialogflowV2IntentMessageListSelectItem[]
}
export interface GoogleCloudDialogflowV2IntentMessageListSelectItem {
info?: GoogleCloudDialogflowV2IntentMessageSelectItemInfo
title?: string
description?: string
image?: GoogleCloudDialogflowV2IntentMessageImage
}
export interface GoogleCloudDialogflowV2IntentMessageQuickReplies {
title?: string
quickReplies?: string[]
}
export interface GoogleCloudDialogflowV2IntentMessageSelectItemInfo {
key?: string
synonyms?: string[]
}
export interface GoogleCloudDialogflowV2IntentMessageSimpleResponse {
textToSpeech?: string
ssml?: string
displayText?: string
}
export interface GoogleCloudDialogflowV2IntentMessageSimpleResponses {
simpleResponses?: GoogleCloudDialogflowV2IntentMessageSimpleResponse[]
}
export interface GoogleCloudDialogflowV2IntentMessageSuggestion {
title?: string
}
export interface GoogleCloudDialogflowV2IntentMessageSuggestions {
suggestions?: GoogleCloudDialogflowV2IntentMessageSuggestion[]
}
export interface GoogleCloudDialogflowV2IntentMessageText {
text?: string[]
}
export interface GoogleCloudDialogflowV2IntentParameter {
name?: string
displayName?: string
value?: string
defaultValue?: string
entityTypeDisplayName?: string
mandatory?: boolean
prompts?: string[]
isList?: boolean
}
export interface GoogleCloudDialogflowV2IntentTrainingPhrase {
name?: string
type?: GoogleCloudDialogflowV2IntentTrainingPhraseType
parts?: GoogleCloudDialogflowV2IntentTrainingPhrasePart[]
timesAddedCount?: number
}
export interface GoogleCloudDialogflowV2IntentTrainingPhrasePart {
text?: string
entityType?: string
alias?: string
userDefined?: boolean
}
export interface GoogleCloudDialogflowV2OriginalDetectIntentRequest {
source?: string
payload?: ApiClientObjectMap<any>
}
export interface GoogleCloudDialogflowV2QueryResult {
queryText?: string
languageCode?: string
speechRecognitionConfidence?: number
action?: string
parameters?: ApiClientObjectMap<any>
allRequiredParamsPresent?: boolean
fulfillmentText?: string
fulfillmentMessages?: GoogleCloudDialogflowV2IntentMessage[]
webhookSource?: string
webhookPayload?: ApiClientObjectMap<any>
outputContexts?: GoogleCloudDialogflowV2Context[]
intent?: GoogleCloudDialogflowV2Intent
intentDetectionConfidence?: number
diagnosticInfo?: ApiClientObjectMap<any>
}
export interface GoogleCloudDialogflowV2WebhookRequest {
session?: string
responseId?: string
queryResult?: GoogleCloudDialogflowV2QueryResult
originalDetectIntentRequest?:
GoogleCloudDialogflowV2OriginalDetectIntentRequest
}
export interface GoogleCloudDialogflowV2WebhookResponse {
fulfillmentText?: string
fulfillmentMessages?: GoogleCloudDialogflowV2IntentMessage[]
source?: string
payload?: ApiClientObjectMap<any>
outputContexts?: GoogleCloudDialogflowV2Context[]
followupEventInput?: GoogleCloudDialogflowV2EventInput
}
@valgaze
Copy link
Author

valgaze commented Apr 11, 2020 •
edited
Loading

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/dialogflow-fulfillment/index.d.ts

https://gist.github.com/iamarcel/f39a08820f6f9181b92eee042c53c908

    {
            fulfillmentMessages: [{
            "platform": "ACTIONS_ON_GOOGLE",
            "simpleResponses": {
              "simpleResponses": [
                {
                  "textToSpeech": "abcdefg!",
                  "ssml": "",
                  "displayText": ""
                }
              ]
            },
            "message": "simpleResponses"
          },
           {
              "platform": "PLATFORM_UNSPECIFIED",
              "text": {
                  "text": [
                      "abcdefg v2"
                  ]
              },
              "message": "text"
            }]
          }
{
  "fulfillmentText": "replicated message",
  "fulfillmentMessages": [
    {
      "platform": "FACEBOOK",
      "text": {
        "text": [
          "replicated message"
        ]
      },
      "message": "text"
    },
    {
      "platform": "ACTIONS_ON_GOOGLE",
      "simpleResponses": {
        "simpleResponses": [
          {
            "textToSpeech": "replicated message",
            "ssml": "",
            "displayText": ""
          }
        ]
      },
      "message": "simpleResponses"
    },
    {
      "platform": "FACEBOOK",
      "payload": {
        "fields": {
          "flag": {
            "stringValue": "followup",
            "kind": "stringValue"
          }
        }
      },
      "message": "payload"
    },
    {
      "platform": "ACTIONS_ON_GOOGLE",
      "payload": {
        "fields": {
          "flag": {
            "stringValue": "followup",
            "kind": "stringValue"
          }
        }
      },
      "message": "payload"
    },
    {
      "platform": "FACEBOOK",
      "text": {
        "text": [
          "This is a response for facebook messenger"
        ]
      },
      "message": "text"
    },
    {
      "platform": "ACTIONS_ON_GOOGLE",
      "simpleResponses": {
        "simpleResponses": [
          {
            "textToSpeech": ">>SIMPLE RESPONSE CONFIGURED FROM DIALOGFLOW<<",
            "ssml": "",
            "displayText": ">>SIMPLE RESPONSE CONFIGURED FROM DIALOGFLOW<<"
          }
        ]
      },
      "message": "simpleResponses"
    },
    {
      "platform": "ACTIONS_ON_GOOGLE",
      "listSelect": {
        "items": [
          {
            "info": {
              "synonyms": [],
              "key": "chocolateoptionkey"
            },
            "title": "choclate title",
            "description": "chocolate desc",
            "image": {
              "imageUri": "https://i.imgur.com/uMtF8ah.jpg",
              "accessibilityText": "chocolate"
            }
          },
          {
            "info": {
              "synonyms": [],
              "key": "mintoptionkey"
            },
            "title": "mint title",
            "description": "mint desc",
            "image": {
              "imageUri": "https://i.imgur.com/W9Eeuu1.jpg",
              "accessibilityText": "mint"
            }
          },
          {
            "info": {
              "synonyms": [],
              "key": "strawberryoptionkey"
            },
            "title": "strawberry title",
            "description": "strawberry desc",
            "image": {
              "imageUri": "https://i.imgur.com/GSZEI39.jpg",
              "accessibilityText": "mint accessibility"
            }
          },
          {
            "info": {
              "synonyms": [],
              "key": "vanillaoptionkey"
            },
            "title": "vanilla title",
            "description": "vanilla desc",
            "image": {
              "imageUri": "https://i.imgur.com/zqKeYU5.jpg",
              "accessibilityText": "vanilla accessibility"
            }
          }
        ],
        "title": "List 123",
        "subtitle": ""
      },
      "message": "listSelect"
    },
    {
      "platform": "ACTIONS_ON_GOOGLE",
      "suggestions": {
        "suggestions": [
          {
            "title": "aaa"
          },
          {
            "title": "bbbb"
          },
          {
            "title": "ccc"
          }
        ]
      },
      "message": "suggestions"
    },
    {
      "platform": "PLATFORM_UNSPECIFIED",
      "text": {
        "text": [
          "replicated message"
        ]
      },
      "message": "text"
    },
    {
      "platform": "PLATFORM_UNSPECIFIED",
      "payload": {
        "fields": {
          "flag": {
            "stringValue": "followup",
            "kind": "stringValue"
          }
        }
      },
      "message": "payload"
    }
  ]
}

@valgaze
Copy link
Author

valgaze commented Apr 17, 2020 •
edited
Loading

Core response:

{
	"queryResult": {
		"fulfillmentMessages": [],
		"outputContexts": [],
		"queryText": "hiii",
		"action": "",
		"parameters": {},
		"fulfillmentText": "",
		"webhookPayload": {},
		"intent": {
			"name": "projects/xxxx/agent/intents/123456789-aaaa-bbbb-123456789987654321",
			"displayName": "my great intent name"
		},
		"languageCode": "en"
	}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment