人工智能培训

搜索

人工智能论文:状态规则的递归神经网络(State-Regularized Recurrent Neural Networks)

[复制链接]
sulee 发表于 2019-1-28 11:46:06 | 显示全部楼层 |阅读模式
sulee 2019-1-28 11:46:06 226 0 显示全部楼层
人工智能论文:状态规则的递归神经网络(State-Regularized Recurrent Neural Networks)递归神经网络是一种广泛使用的神经结构类型。然而,它们有两个缺点。首先,很难理解他们学到的东西。其次,尽管原则上具有这种能力,但它们往往对需要长期记忆的序列工作不佳。我们的目标是利用一类在小区应用之间使用随机状态转换机制的循环网络来解决这两个缺点。这种机制,我们称之为状态正则化,使得RNN在有限的可学习状态之间转换。我们在(1)常规语言上评估状态正则化的RNN以用于自动提取; (2)非平常语言,如平衡括号,回文,以及需要外在记忆的复制任务; (3)实词序列学习任务强化分析,视觉对象识别和语言建模。我们证明了状态正则化(a)简化了有限状态自动机的提取,模拟了RNN的状态转移动力学; (b)迫使RNN更像自动机与外部存储器,而不像有限状态机; (c)使RNN具有更好的可解释性和可解释性。
Recurrent neural networks are a widely used class of neural architectures.They have, however, two shortcomings.First, it is difficult to understand whatexactly they learn.Second, they tend to work poorly on sequences requiringlong-term memorization, despite having this capacity in principle.We aim toaddress both shortcomings with a class of recurrent networks that use astochastic state transition mechanism between cell applications.Thismechanism, which we term state-regularization, makes RNNs transition between afinite set of learnable states.We evaluate state-regularized RNNs on (1)regular languages for the purpose of automata extraction;(2) nonregularlanguages such as balanced parentheses, palindromes, and the copy task whereexternal memory is required;and (3) real-word sequence learning tasks forsentiment analysis, visual object recognition, and language modeling.We showthat state-regularization (a) simplifies the extraction of finite stateautomata modeling an RNN's state transition dynamics;(b) forces RNNs tooperate more like automata with external memory and less like finite statemachines;(c) makes RNNs have better interpretability and explainability.人工智能论文:状态规则的递归神经网络(State-Regularized Recurrent Neural Networks) tvZrUFFvxdDf9399.jpg
URL地址:https://arxiv.org/abs/1901.08817     ----pdf下载地址:https://arxiv.org/pdf/1901.08817    ----人工智能论文:状态规则的递归神经网络(State-Regularized Recurrent Neural Networks)
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则 返回列表 发新帖

sulee当前离线
新手上路

查看:226 | 回复:0

快速回复 返回顶部 返回列表