人工智能培训

搜索

人工智能论文:泛化的最优传输观(An Optimal Transport View on Generalization)

[复制链接]
376156679 发表于 5 天前 | 显示全部楼层 |阅读模式
376156679 5 天前 194 0 显示全部楼层
人工智能论文:泛化的最优传输观(An Optimal Transport View on Generalization)我们基于它们的\ emph {算法传输成本}推导出学习算法的泛化误差的上界:输出假设和输出假设之间的预期的Wassersteindistance以输入示例为条件。边界提供了一种新方法来研究从最优传输视图中学习算法的一般化和对损失函数的无限制约束,例如亚高斯或有界。我们进一步根据总变差距离,相对熵(或KL-发散)和VC维度提供算法传输成本的几个上界,从而进一步用统计学习理论桥接最优传输理论和信息理论。此外,我们还研究了损失函数的不同条件,在这些条件下,学习算法的泛化误差可以在与输出假设和/或输入数据相关的分布之间的不同概率度量上限。最后,在我们建立的框架下,我们分析了深入研究并得出结论:随着层数的增加,深度神经网络(DNN)中的泛化误差呈指数减小为零。深度学习中泛化误差的分析主要是利用DNN中的层次结构和$ f $ -diver的收缩性质,这可能对分析其他具有层次结构的学习模型具有独立的意义。
We derive upper bounds on the generalization error of learning algorithmsbased on their \emph{algorithmic transport cost}: the expected Wassersteindistance between the output hypothesis and the output hypothesis conditioned onan input example.The bounds provide a novel approach to study thegeneralization of learning algorithms from an optimal transport view and imposeless constraints on the loss function, such as sub-gaussian or bounded.Wefurther provide several upper bounds on the algorithmic transport cost in termsof total variation distance, relative entropy (or KL-divergence), and VCdimension, thus further bridging optimal transport theory and informationtheory with statistical learning theory.Moreover, we also study differentconditions for loss functions under which the generalization error of alearning algorithm can be upper bounded by different probability metricsbetween distributions relating to the output hypothesis and/or the input data.Finally, under our established framework, we analyze the generalization indeeplearning and conclude that the generalization error in deep neural networks(DNNs) decreases exponentially to zero as the number of layers increases.Ouranalyses of generalization error in deep learning mainly exploit thehierarchical structure in DNNs and the contraction property of $f$-divergence,which may be of independent interest in analyzing other learning models withhierarchical structure.人工智能论文:泛化的最优传输观(An Optimal Transport View on Generalization) H33575X2823sc5ZD.jpg
URL地址:https://arxiv.org/abs/1811.03270     ----pdf下载地址:https://arxiv.org/pdf/1811.03270    ----人工智能论文:泛化的最优传输观(An Optimal Transport View on Generalization)
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则 返回列表 发新帖

376156679当前离线
新手上路

查看:194 | 回复:0

快速回复 返回顶部 返回列表