❤️Hi! Azure.❤️从零开始打造一个语音机器人，跟你的电脑聊聊天！❤️

╰半夏微凉° 2022-09-16 11:05 208阅读 0赞

这篇文章带大家来做一个简单的聊天机器人，碰巧遇到微软Azure招募开发者，可以免费试用人工智能服务，果断申请。

> 先决条件：
> 
> 1.  熟悉 Azure 服务和 Azure 门户（也可以用其它的AI开放平台）
> 2.  拥有 Python 编程经验
> 3.  熟悉API调用

## 序言 ##

首先需要了解一下做对话机器人需要涉及到的技术，可以看一下这个流程图。

![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_18_color_FFFFFF_t_70_g_se_x_16_pic_center]  
理清思路之后，我们发现涉及到的技术有三种：语音识别、对话式问答、语音合成。

> 简单解释一下这三个技术：
> 
> 1.  语音识别：准确地将口述语音听录为文本，[功能演示][Link 1]
> 2.  对话式问答：提取文本信息，并将其转化为一目了然的对话式答案
> 3.  语音合成：将文本转换为逼真的语音，[功能演示][Link 2]

目前来说最流行的语音算法主要是依赖于深度学习的神经网络算法，深度学习的应用真正让语音识别/合成达到了商用级别。

不过如果要自己做一个语音识别系统，从零开始训练语言模型势必会非常耗时，虽然有很多预训练的模型，但是对于没有深度学习基础的同学来说还是比较困难，所以我们借助Azure的[认知服务][Link 3]开源的语音服务API来是想这些功能。

首先需要使用你的 Microsoft 帐户登录到[ Azure 门户][Azure]。  
![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_20_color_FFFFFF_t_70_g_se_x_16]  
然后创建一个认知服务资源。  
![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_20_color_FFFFFF_t_70_g_se_x_16 1]  
创建并且等待部署完成后，在左侧窗格中的“资源管理”下，选择“密钥和终结点” 。  
![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_20_color_FFFFFF_t_70_g_se_x_16 2]  
每个订阅有两个密钥；可在应用程序中使用任意一个密钥。

## 语音识别+语音合成 ##

#### 安装和导入语音 SDK ####

需要先安装语音 SDK，然后才能执行任何操作。

pip install azure-cognitiveservices-speech

安装语音 SDK 后，将其导入到 Python 项目中。

import azure.cognitiveservices.speech as speechsdk

#### 创建语音识别配置 ####

若要使用语音 SDK 调用语音服务，需要创建 SpeechConfig。 此类包含有关订阅的信息，例如密钥和关联的位置/区域、终结点、主机或授权令牌。

import azure.cognitiveservices.speech as speechsdk
    
    # 需要填入自己的密钥和位置
    speech_key, service_region = "YourSubscriptionKey", "YourServiceRegion"
    speech_config = speechsdk.SpeechConfig(subscription=speech_key, region=service_region,
                                           speech_recognition_language="zh-cn")
    
    # Creates a recognizer with the given settings
    speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config)

#### 从麦克风识别 ####

若要使用设备麦克风识别语音，只需创建 SpeechRecognizer（无需传递 AudioConfig），并传递 speech\_config。

import azure.cognitiveservices.speech as speechsdk
    
    def from_mic():
        speech_config = speechsdk.SpeechConfig(subscription="<paste-your-speech-key-here>", region="<paste-your-speech-location/region-here>")
        speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config)
        
        print("Speak into your microphone.")
        result = speech_recognizer.recognize_once_async().get()
        print(result.text)
    
    from_mic()

#### 错误处理 ####

前面的示例只从 result.text 获取已识别的文本，但要处理错误和其他响应，需要编写一些代码来处理结果。 以下代码评估 result.reason 属性并：

*  输出识别结果：speechsdk.ResultReason.RecognizedSpeech
 *  如果没有识别匹配项，通知用户：speechsdk.ResultReason.NoMatch
 *  如果遇到错误，则输出错误消息：speechsdk.ResultReason.Canceled

if result.reason == speechsdk.ResultReason.RecognizedSpeech:
        print("Recognized: {}".format(result.text))
    elif result.reason == speechsdk.ResultReason.NoMatch:
        print("No speech could be recognized: {}".format(result.no_match_details))
    elif result.reason == speechsdk.ResultReason.Canceled:
        cancellation_details = result.cancellation_details
        print("Speech Recognition canceled: {}".format(cancellation_details.reason))
        if cancellation_details.reason == speechsdk.CancellationReason.Error:
            print("Error details: {}".format(cancellation_details.error_details))

#### 语音合成 ####

同样，我们可以搭建一个语音合成的功能。

def speak(text_words):
    	speech_config.speech_synthesis_language = "zh-cn"
    	audio_config = AudioOutputConfig(use_default_speaker=True)
    	speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config, audio_config=audio_config)
    
    	result = speech_synthesizer.speak_text_async(text_words).get()
    
    	# Checks result.
    	if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
    		print("Speech synthesized to speaker for text [{}]".format(text_words))
    	elif result.reason == speechsdk.ResultReason.Canceled:
    		cancellation_details = result.cancellation_details
    		print("Speech synthesis canceled: {}".format(cancellation_details.reason))
    		if cancellation_details.reason == speechsdk.CancellationReason.Error:
    			if cancellation_details.error_details:
    				print("Error details: {}".format(cancellation_details.error_details))
    		print("Did you update the subscription info?")

## 图灵机器人服务构建聊天功能 ##

对话式机器人现在已经成为相对成熟的AI解决方案，机器人以聊天的方式与用户交互，比如edge自带的小冰虚拟女友：  
![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_15_color_FFFFFF_t_70_g_se_x_16]  
为了实现这种对话式的聊天功能，我们需要借助[图灵机器人开放平台][Link 4]，通过图灵机器人开放平台，软硬件产品开发者可快速为自己的产品接入一款具备个性化身份属性特征、满足不同场景多轮对话及上下文对话的人工智能机器人，实现产品的对话式交互。

#### 创建机器人 ####

进入图灵机器人管理控制台后。点击“创建机器人”，按要求填写机器人名称及应用信息，即可完成创建，当前每个账号只能创建5个机器人。

![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_14_color_FFFFFF_t_70_g_se_x_16]

然后在机器人设置中可以定义机器人的一些属性。  
![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_20_color_FFFFFF_t_70_g_se_x_16 3]

#### API接入 ####

目前图灵机器人API接口可调用聊天对话、语料库、技能三大模块的语料：

*  聊天对话是指平台免费提供的近10亿条公有对话语料，满足用户对话娱乐需求；
 *  语料库是指用户在平台上传的私有语料，仅供个人查看使用，帮助用户最便捷的搭建专业领域次的语料。
 *  技能服务是指平台打包的26种实用服务技能。涵盖生活、出行、购物等多个领域，一站式满足用户需求。

**编码方式**  
UTF-8（调用图灵API的各个环节的编码方式均为UTF-8）

**接口地址**  
[http://openapi.turingapi.com/openapi/api/v2][http_openapi.turingapi.com_openapi_api_v2]

**请求方式**  
HTTP POST

**请求参数**  
请求参数格式为 json

**请求示例：**

{ 
    	"reqType":0,
        "perception": { 
            "inputText": { 
                "text": "附近的酒店"
            },
            "inputImage": { 
                "url": "imageUrl"
            },
            "selfInfo": { 
                "location": { 
                    "city": "北京",
                    "province": "北京",
                    "street": "信息路"
                }
            }
        },
        "userInfo": { 
            "apiKey": "",
            "userId": ""
        }
    }

**参数说明**

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否必须</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>reqType</td> 
   <td>int</td> 
   <td>N</td> 
   <td>-</td> 
   <td>输入类型:0-文本(默认)、1-图片、2-音频</td> 
  </tr> 
  <tr> 
   <td>perception</td> 
   <td>-</td> 
   <td>Y</td> 
   <td>-</td> 
   <td>输入信息</td> 
  </tr> 
  <tr> 
   <td>userInfo</td> 
   <td>-</td> 
   <td>Y</td> 
   <td>-</td> 
   <td>用户参数</td> 
  </tr> 
 </tbody> 
</table>

**perception**

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否必须</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>inputText</td> 
   <td>-</td> 
   <td>N</td> 
   <td>-</td> 
   <td>文本信息</td> 
  </tr> 
  <tr> 
   <td>inputImage</td> 
   <td>-</td> 
   <td>N</td> 
   <td>-</td> 
   <td>图片信息</td> 
  </tr> 
  <tr> 
   <td>inputMedia</td> 
   <td>-</td> 
   <td>N</td> 
   <td>-</td> 
   <td>音频信息</td> 
  </tr> 
  <tr> 
   <td>selfInfo</td> 
   <td>-</td> 
   <td>N</td> 
   <td>-</td> 
   <td>客户端属性</td> 
  </tr> 
 </tbody> 
</table>

注意：输入参数必须包含inputText或inputImage或inputMedia！  
***inputText***

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否必须</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>text</td> 
   <td>String</td> 
   <td>Y</td> 
   <td>1-128字符</td> 
   <td>直接输入文本</td> 
  </tr> 
 </tbody> 
</table>

***inputImage***

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否必须</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>url</td> 
   <td>String</td> 
   <td>Y</td> 
   <td></td> 
   <td>图片地址</td> 
  </tr> 
 </tbody> 
</table>

***inputMedia***

***selfInfo***

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否必须</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>location</td> 
   <td>-</td> 
   <td>N</td> 
   <td>-</td> 
   <td>地理位置信息</td> 
  </tr> 
 </tbody> 
</table>

***location***

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否必须</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>city</td> 
   <td>String</td> 
   <td>Y</td> 
   <td>-</td> 
   <td>所在城市</td> 
  </tr> 
  <tr> 
   <td>province</td> 
   <td>String</td> 
   <td>N</td> 
   <td>-</td> 
   <td>省份</td> 
  </tr> 
  <tr> 
   <td>street</td> 
   <td>String</td> 
   <td>N</td> 
   <td>-</td> 
   <td>街道</td> 
  </tr> 
 </tbody> 
</table>

**userInfo**

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否必须</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>apiKey</td> 
   <td>String</td> 
   <td>Y</td> 
   <td>32位</td> 
   <td>机器人标识</td> 
  </tr> 
  <tr> 
   <td>userId</td> 
   <td>String</td> 
   <td>Y</td> 
   <td>长度小于等于32位</td> 
   <td>用户唯一标识</td> 
  </tr> 
  <tr> 
   <td>groupId</td> 
   <td>String</td> 
   <td>N</td> 
   <td>长度小于等于64位</td> 
   <td>群聊唯一标识</td> 
  </tr> 
  <tr> 
   <td>userIdName</td> 
   <td>String</td> 
   <td>N</td> 
   <td>长度小于等于64位</td> 
   <td>群内用户昵称</td> 
  </tr> 
 </tbody> 
</table>

****

**输出示例：**

{ 
        "intent": { 
            "code": 10005,
            "intentName": "",
            "actionName": "",
            "parameters": { 
                "nearby_place": "酒店"
            }
        },
        "results": [
            { 
             	"groupType": 1,
                "resultType": "url",
                "values": { 
                    "url": "http://m.elong.com/hotel/0101/nlist/#indate=2016-12-10&outdate=2016-12-11&keywords=%E4%BF%A1%E6%81%AF%E8%B7%AF"
                }
            },
            { 
             	"groupType": 1,
                "resultType": "text",
                "values": { 
                    "text": "亲，已帮你找到相关酒店信息"
                }
            }
        ]
    }

**参数说明**

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否必须</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>intent</td> 
   <td>-</td> 
   <td>Y</td> 
   <td>-</td> 
   <td>请求意图</td> 
  </tr> 
  <tr> 
   <td>results</td> 
   <td>-</td> 
   <td>N</td> 
   <td>-</td> 
   <td>输出结果集</td> 
  </tr> 
 </tbody> 
</table>

**intent**

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否包含</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>code</td> 
   <td>int</td> 
   <td>Y</td> 
   <td>-</td> 
   <td>输出功能code</td> 
  </tr> 
  <tr> 
   <td>intentName</td> 
   <td>String</td> 
   <td>N</td> 
   <td>-</td> 
   <td>意图名称</td> 
  </tr> 
  <tr> 
   <td>actionName</td> 
   <td>String</td> 
   <td>N</td> 
   <td>-</td> 
   <td>意图动作名称</td> 
  </tr> 
  <tr> 
   <td>parameters</td> 
   <td>Map</td> 
   <td>N</td> 
   <td>-</td> 
   <td>功能相关参数</td> 
  </tr> 
 </tbody> 
</table>

**results**

<table> 
 <thead> 
  <tr> 
   <th>参数</th> 
   <th>类型</th> 
   <th>是否包含</th> 
   <th>取值范围</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>resultType</td> 
   <td>String</td> 
   <td>Y</td> 
   <td>文本(text);连接(url);音频(voice);视频(video);图片(image);图文(news)</td> 
   <td>输出类型</td> 
  </tr> 
  <tr> 
   <td>values</td> 
   <td>-</td> 
   <td>Y</td> 
   <td>-</td> 
   <td>输出值</td> 
  </tr> 
  <tr> 
   <td>groupType</td> 
   <td>int</td> 
   <td>Y</td> 
   <td>-</td> 
   <td>‘组’编号:0为独立输出，大于0时可能包含同组相关内容 (如：音频与文本为一组时说明内容一致)</td> 
  </tr> 
 </tbody> 
</table>

#### Code ####

def chat(text_words=""):
    	api_key = "1dde879fa**********1ee88269f3852"
    	api_url = "http://openapi.tuling123.com/openapi/api/v2"
    	headers = { "Content-Type": "application/json;charset=UTF-8"}
    
    	req = { 
    		"reqType": 0,
    		"perception": { 
    			"inputText": { 
    				"text": text_words
    			},
    			"selfInfo": { 
    				"location": { 
    					"city": "天津",
    					"province": "天津",
    					"street": "天津科技大学"
    				}
    			}
    		},
    		"userInfo": { 
    			"apiKey": api_key,
    			"userId": "Alex"
    		}
    	}
    
    	req["perception"]["inputText"]["text"] = text_words
    	response = requests.request("post", api_url, json=req, headers=headers)
    	response_dict = json.loads(response.text)
    
    	result = response_dict["results"][0]["values"]["text"]
    	print("AI Robot said: " + result)
    	return result

## 汇总 ##

最后，把语音识别、语音合成的代码放在一起，再加上一个简单的流程控制，所有的代码加起来不超过一百行，一个能跟你聊天的机器人就完成了。

import json
    
    import requests
    import azure.cognitiveservices.speech as speechsdk
    
    from azure.cognitiveservices.speech.audio import AudioOutputConfig
    
    speech_key, service_region = "6d5afa148**********5d8801db8e408", "chinanorth2"
    speech_config = speechsdk.SpeechConfig(subscription=speech_key, region=service_region,
                                           speech_recognition_language="zh-cn")
    
    
    def record():
    	# Creates a recognizer with the given settings
    	speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config)
    
    	print("Say something...")
    	result = speech_recognizer.recognize_once()
    
    	# Checks result.
    	if result.reason == speechsdk.ResultReason.RecognizedSpeech:
    		print("Recognized: {}".format(result.text))
    	elif result.reason == speechsdk.ResultReason.NoMatch:
    		print("No speech could be recognized: {}".format(result.no_match_details))
    	elif result.reason == speechsdk.ResultReason.Canceled:
    		cancellation_details = result.cancellation_details
    		print("Speech Recognition canceled: {}".format(cancellation_details.reason))
    		if cancellation_details.reason == speechsdk.CancellationReason.Error:
    			print("Error details: {}".format(cancellation_details.error_details))
    
    	return result.text
    
    
    def chat(text_words=""):
    	api_key = "1dde879f**********f1ee88269f3852"
    	api_url = "http://openapi.tuling123.com/openapi/api/v2"
    	headers = { "Content-Type": "application/json;charset=UTF-8"}
    
    	req = { 
    		"reqType": 0,
    		"perception": { 
    			"inputText": { 
    				"text": text_words
    			},
    			"selfInfo": { 
    				"location": { 
    					"city": "天津",
    					"province": "天津",
    					"street": "天津科技大学"
    				}
    			}
    		},
    		"userInfo": { 
    			"apiKey": api_key,
    			"userId": "Alex"
    		}
    	}
    
    	req["perception"]["inputText"]["text"] = text_words
    	response = requests.request("post", api_url, json=req, headers=headers)
    	response_dict = json.loads(response.text)
    
    	result = response_dict["results"][0]["values"]["text"]
    	print("AI Robot said: " + result)
    	return result
    
    
    def speak(text_words):
    	speech_config.speech_synthesis_language = "zh-cn"
    	audio_config = AudioOutputConfig(use_default_speaker=True)
    	speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config, audio_config=audio_config)
    
    	result = speech_synthesizer.speak_text_async(text_words).get()
    
    	# Checks result.
    	if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
    		print("Speech synthesized to speaker for text [{}]".format(text_words))
    	elif result.reason == speechsdk.ResultReason.Canceled:
    		cancellation_details = result.cancellation_details
    		print("Speech synthesis canceled: {}".format(cancellation_details.reason))
    		if cancellation_details.reason == speechsdk.CancellationReason.Error:
    			if cancellation_details.error_details:
    				print("Error details: {}".format(cancellation_details.error_details))
    		print("Did you update the subscription info?")
    
    
    if __name__ == '__main__':
    	while True:
    		text = record()
    		if "退出" in text:
    			break
    		res = chat(text)
    		speak(res)

代码地址：[https://github.com/Matrix-King-Studio/IntelligentVoiceRobot][https_github.com_Matrix-King-Studio_IntelligentVoiceRobot]  
演示视频：[https://www.bilibili.com/video/BV1BU4y1F7s7/][https_www.bilibili.com_video_BV1BU4y1F7s7]

[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_18_color_FFFFFF_t_70_g_se_x_16_pic_center]: /images/20220828/725578c3bdfa4af280c0c68df86e3ca6.png
[Link 1]: https://azure.microsoft.com/zh-cn/services/cognitive-services/speech-to-text/#features
[Link 2]: https://azure.microsoft.com/zh-cn/services/cognitive-services/text-to-speech/#features
[Link 3]: https://azure.microsoft.com/zh-cn/services/cognitive-services/#overview
[Azure]: https://portal.azure.com/
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_20_color_FFFFFF_t_70_g_se_x_16]: /images/20220828/badd8c3620c3400e962ea31f017a83b7.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_20_color_FFFFFF_t_70_g_se_x_16 1]: /images/20220828/bce1ca2bd0734a9aa581c21aefc509e7.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_20_color_FFFFFF_t_70_g_se_x_16 2]: /images/20220828/3acc849d30584f4c94eb5b887fb575b1.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_15_color_FFFFFF_t_70_g_se_x_16]: /images/20220828/4201a0ddc4d341639f79ee8778afaa98.png
[Link 4]: http://www.turingapi.com/
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_14_color_FFFFFF_t_70_g_se_x_16]: /images/20220828/945d4a0f446b436c857b02bd06e0a09d.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAQWxleCBIdWI_size_20_color_FFFFFF_t_70_g_se_x_16 3]: /images/20220828/772fc815b4b84a6182f08716f5384a39.png
[http_openapi.turingapi.com_openapi_api_v2]: http://openapi.turingapi.com/openapi/api/v2
[https_github.com_Matrix-King-Studio_IntelligentVoiceRobot]: https://github.com/Matrix-King-Studio/IntelligentVoiceRobot
[https_www.bilibili.com_video_BV1BU4y1F7s7]: https://www.bilibili.com/video/BV1BU4y1F7s7/