人工智能培训

搜索

人工智能教程:非线性系数 - 预测深度神经网络的过拟合(The Nonlinearity Coefficient - Predicting Overfittin

[复制链接]
dabiao2008 发表于 2018-6-4 09:06:43 | 显示全部楼层 |阅读模式
dabiao2008 2018-6-4 09:06:43 650 0 显示全部楼层
人工智能教程:非线性系数 - 预测深度神经网络的过拟合(The Nonlinearity Coefficient - Predicting Overfitting in Deep Neural  Networks)长期以来,设计表现出高性能的神经架构被认为是需要专家手动调整的黑暗艺术。其中一个着名的建筑设计准则是避免了渐变的渐变,但即使这个准则仍然相对模糊和具有实质性。我们引入非线性系数(NLC),这是一种基于梯度大小的神经网络计算函数复杂性的度量。通过广泛的实证研究,我们证明NLC是测试误差的强大预测因子,获得正确尺寸的NLC对于获得最佳性能至关重要.NLC展现出一系列有趣且重要的特性。它与计算单个网络梯度所获得的信息量密切相关。它与用线性运算代替网络中的非线性运算时产生的误差相关。它不会受到乘法缩放,加性偏移和层宽的混杂因素的影响。它从一层到另一层都是稳定的。因此,我们认为NLC是深度网络中过度拟合的第一个稳健预测器。
For a long time, designing neural architectures that exhibit high performancewas considered a dark art that required expert hand-tuning.One of the fewwell-known guidelines for architecture design is the avoidance of explodinggradients, though even this guideline has remained relatively vague andcircumstantial.We introduce the nonlinearity coefficient (NLC), a measurementof the complexity of the function computed by a neural network that is based onthe magnitude of the gradient.Via an extensive empirical study, we show thatthe NLC is a powerful predictor of test error and that attaining a right-sizedNLC is essential for optimal performance.The NLC exhibits a range of intriguing and important properties.It isclosely tied to the amount of information gained from computing a singlenetwork gradient.It is tied to the error incurred when replacing thenonlinearity operations in the network with linear operations.It is notsusceptible to the confounders of multiplicative scaling, additive bias andlayer width.It is stable from layer to layer.Hence, we argue that the NLC isthe first robust predictor of overfitting in deep networks.人工智能教程:非线性系数 - 预测深度神经网络的过拟合(The Nonlinearity Coefficient - Predicting Overfitting in Deep Neural  Networks) l3J550b7hZ5Hh3SI.jpg
URL地址:https://arxiv.org/abs/1806.00179     ----pdf下载地址:http://arxiv.org/pdf/1806.00179    ----人工智能教程:非线性系数 - 预测深度神经网络的过拟合(The Nonlinearity Coefficient - Predicting Overfitting in Deep Neural  Networks)
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则 返回列表 发新帖

dabiao2008当前离线
新手上路

查看:650 | 回复:0

快速回复 返回顶部 返回列表