人工智能培训

搜索

机器学习论文:您的模型是否知道数字6不是猫?(Does Your Model Know the Digit 6 Is Not a Cat? A Less Bia

[复制链接]
aarryy7 发表于 2018-9-14 08:57:04 | 显示全部楼层 |阅读模式
aarryy7 2018-9-14 08:57:04 152 0 显示全部楼层
机器学习论文:您的模型是否知道数字6不是猫?(Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation  of "Outlier" Detectors)在现实世界中,学习系统可以接收看起来像在训练期间看到的任何东西的输入,这可能导致不可预测的行为。因此,我们需要知道任何给定的输入是否属于训练数据的人口分布,以防止在部署的系统中出现不可预测的行为。最近对这个问题的兴趣激增促使深度学习文学中的复杂技术的发展。然而,由于缺乏标准化的问题制定或详尽的评估,我们在实践中是否可以依赖这些方法并不明显。使这个问题与典型的监督学习环境不同的是,我们无法在实践中模拟分布式样本的多样性。训练中使用的异常值的分布可能与应用程序中遇到的异常值的分布不同。因此,学习内部函数与仅使用twodatasets的异常值的经典方法可以产生乐观的结果。我们引入了OD测试,这是一种三数据评估方案,作为评估该问题进展的实用且更可靠的策略。 OD测试基准提供了一种简单的比较方法,用于解决分布式样本检测问题的方法。我们对图像分类任务相关领域的广泛方法进行了详尽的评估。此外,我们表明,对于高维图像的现实应用,现有方法具有较低的准确性。我们的分析揭示了每种方法的优点和缺点。
In the real world, a learning system could receive an input that looksnothing like anything it has seen during training, and this can lead tounpredictable behaviour.We thus need to know whether any given input belongsto the population distribution of the training data to prevent unpredictablebehaviour in deployed systems.A recent surge of interest on this problem hasled to the development of sophisticated techniques in the deep learningliterature.However, due to the absence of a standardized problem formulationor an exhaustive evaluation, it is not evident if we can rely on these methodsin practice.What makes this problem different from a typical supervisedlearning setting is that we cannot model the diversity of out-of-distributionsamples in practice.The distribution of outliers used in training may not bethe same as the distribution of outliers encountered in the application.Therefore, classical approaches that learn inliers vs. outliers with only twodatasets can yield optimistic results.We introduce OD-test, a three-datasetevaluation scheme as a practical and more reliable strategy to assess progresson this problem.The OD-test benchmark provides a straightforward means ofcomparison for methods that address the out-of-distribution sample detectionproblem.We present an exhaustive evaluation of a broad set of methods fromrelated areas on image classification tasks.Furthermore, we show that forrealistic applications of high-dimensional images, the existing methods havelow accuracy.Our analysis reveals areas of strength and weakness of eachmethod.机器学习论文:您的模型是否知道数字6不是猫?(Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation  of "Outlier" Detectors) b9NR4ZZl008dcRNl.jpg
URL地址:https://arxiv.org/abs/1809.04729     ----pdf下载地址:https://arxiv.org/pdf/1809.04729    ----机器学习论文:您的模型是否知道数字6不是猫?(Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation  of "Outlier" Detectors)
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则 返回列表 发新帖

aarryy7当前离线
新手上路

查看:152 | 回复:0

快速回复 返回顶部 返回列表