人工智能培训

搜索

人工智能论文:完全连通神经网络中的TUCKER张量层(Tucker Tensor Layer in Fully Connected Neural Network

[复制链接]
bigrc 发表于 2019-3-15 12:48:17 | 显示全部楼层 |阅读模式
bigrc 2019-3-15 12:48:17 827 0 显示全部楼层
人工智能论文:完全连通神经网络中的TUCKER张量层(Tucker Tensor Layer in Fully Connected Neural Networks)我们引入Tucker Tensor Layer(TTL),它是前馈神经网络(NN)完全连接层的密集矩阵的替代方案,以回答压缩NN并改善其解释性的长期任务。这是通过将这些权重矩阵视为更高阶权重张量的展开来实现的。这使得我们能够引入用于利用权重张量的多路性质的框架,以便通过张量分解的压缩特性来有效地减少参数的数量。 Tucker分解(TKD)用于将权重张量分解为核心张量和因子矩阵。通过将矩阵导数的概念扩展到张量,在这个框架内得到反向传播。通过这种方式,通过针对每个因子矩阵计算梯度的过程,利用TKD的物理可解释性来获得对训练的见解。所提出的框架在合成数据和Fashion-MNIST数据集上得到验证,强调了各种数据特征在训练中的相对重要性,从而减轻了NN固有的“黑箱”问题。在MNIST和Fashion-MNIST上的实验说明了TTL的压缩特性,实现了66.63倍的压缩,同时保持了与未压缩的NN相当的性能。
We introduce the Tucker Tensor Layer (TTL), an alternative to the denseweight-matrices of the fully connected layers of feed-forward neural networks(NNs), to answer the long standing quest to compress NNs and improve theirinterpretability.This is achieved by treating these weight-matrices as theunfolding of a higher order weight-tensor.This enables us to introduce aframework for exploiting the multi-way nature of the weight-tensor in order toefficiently reduce the number of parameters, by virtue of the compressionproperties of tensor decompositions.The Tucker Decomposition (TKD) is employedto decompose the weight-tensor into a core tensor and factor matrices.Were-derive back-propagation within this framework, by extending the notion ofmatrix derivatives to tensors.In this way, the physical interpretability ofthe TKD is exploited to gain insights into training, through the process ofcomputing gradients with respect to each factor matrix.The proposed frameworkis validated on synthetic data and on the Fashion-MNIST dataset, emphasizingthe relative importance of various data features in training, hence mitigatingthe "black-box" issue inherent to NNs.Experiments on both MNIST andFashion-MNIST illustrate the compression properties of the TTL, achieving a66.63 fold compression whilst maintaining comparable performance to theuncompressed NN.人工智能论文:完全连通神经网络中的TUCKER张量层(Tucker Tensor Layer in Fully Connected Neural Networks) isC15sCc9hH44n98.jpg
URL地址:https://arxiv.org/abs/1903.06133     ----pdf下载地址:https://arxiv.org/pdf/1903.06133    ----人工智能论文:完全连通神经网络中的TUCKER张量层(Tucker Tensor Layer in Fully Connected Neural Networks)
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则 返回列表 发新帖

bigrc当前离线
新手上路

查看:827 | 回复:0

快速回复 返回顶部 返回列表