人工智能培训

搜索

深度学习论文:结合贝叶斯优化和LIPSCHITZ优化(Combining Bayesian Optimization and Lipschitz Optimiz

[复制链接]
zwb521 发表于 2018-10-11 09:07:43 | 显示全部楼层 |阅读模式
zwb521 2018-10-11 09:07:43 157 0 显示全部楼层
深度学习论文:结合贝叶斯优化和LIPSCHITZ优化(Combining Bayesian Optimization and Lipschitz Optimization)贝叶斯优化和Lipschitz优化已经开发出用于优化黑盒功能的替代技术。它们各自利用关于函数的不同形式的先验。在这项工作中,我们探索了这些技术的策略,以便更好地进行全局优化。特别是,我们提出了在传统BO算法中使用Lipschitz连续性假设的方法,我们称之为Lipschitz贝叶斯优化(LBO)。这种方法不会增加渐近运行时间,并且在某些情况下会大大提高性能(而在最坏的情况下,性能类似)。实际上,在一个特定的环境中,我们证明使用Lipschitz信息产生与后悔相同或更好的界限,而不是单独使用贝叶斯优化。此外,我们提出了一个简单的启发式方法来估计Lipschitz常数,并证明Lipschitz常数的增长估计在某种意义上是“无害的”。我们对具有4个采集函数的15个数据集进行的实验表明,在最坏的情况下,LBO的表现类似于底层BO方法,而在某些情况下,它的表现要好得多。特别是汤普森采样通常看到了极大的改进(因为Lipschitz信息已经得到了很好的修正) - 探索“现象”及其LBO变体通常优于其他采集功能。
Bayesian optimization and Lipschitz optimization have developed alternativetechniques for optimizing black-box functions.They each exploit a differentform of prior about the function.In this work, we explore strategies tocombine these techniques for better global optimization.In particular, wepropose ways to use the Lipschitz continuity assumption within traditional BOalgorithms, which we call Lipschitz Bayesian optimization (LBO).This approachdoes not increase the asymptotic runtime and in some cases drastically improvesthe performance (while in the worst case the performance is similar).Indeed,in a particular setting, we prove that using the Lipschitz information yieldsthe same or a better bound on the regret compared to using Bayesianoptimization on its own.Moreover, we propose a simple heuristics to estimatethe Lipschitz constant, and prove that a growing estimate of the Lipschitzconstant is in some sense "harmless".Our experiments on 15 datasets with 4acquisition functions show that in the worst case LBO performs similar to theunderlying BO method while in some cases it performs substantially better.Thompson sampling in particular typically saw drastic improvements (as theLipschitz information corrected for it's well-known "over-exploration"phenomenon) and its LBO variant often outperformed other acquisition functions.深度学习论文:结合贝叶斯优化和LIPSCHITZ优化(Combining Bayesian Optimization and Lipschitz Optimization) o9k9uNS5SsJ00dgh.jpg
URL地址:https://arxiv.org/abs/1810.04336     ----pdf下载地址:https://arxiv.org/pdf/1810.04336    ----深度学习论文:结合贝叶斯优化和LIPSCHITZ优化(Combining Bayesian Optimization and Lipschitz Optimization)
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则 返回列表 发新帖

zwb521当前离线
新手上路

查看:157 | 回复:0

快速回复 返回顶部 返回列表