人工智能培训

搜索

机器学习论文:深度自回归模型的分块并行解码(Blockwise Parallel Decoding for Deep Autoregressive Models

[复制链接]
comceo 发表于 5 天前 | 显示全部楼层 |阅读模式
comceo 5 天前 207 0 显示全部楼层
机器学习论文:深度自回归模型的分块并行解码(Blockwise Parallel Decoding for Deep Autoregressive Models)深度自回归序列到序列模型近年来在各种各样的任务中表现出令人印象深刻的性能。虽然常规架构类(例如循环,卷积和自注意网络)在每层所需的计算量与训练时的关键路径长度之间进行不同的权衡,但生成仍然是一个固有的顺序过程。为了克服这个限制,我们提出了一种新颖的块状并行解码方案,其中我们并行地对多个时间步骤进行预测,然后回到由评分模型验证的最长前缀。当应用于可以并行处理输出序列的架构时,这允许在生成速度方面进行实质性的理论改进。我们通过一系列实验验证我们的方法,使用最先进的自我关注模型进行机器翻译和图像超分辨率,在基线贪婪解码器上实现高达2倍的迭代减少,质量无损失,或高达7倍性能略有下降。就挂钟时间而言,我们最快的型号的实时加速比标准贪婪解码高出4倍。
Deep autoregressive sequence-to-sequence models have demonstrated impressiveperformance across a wide variety of tasks in recent years.While commonarchitecture classes such as recurrent, convolutional, and self-attentionnetworks make different trade-offs between the amount of computation needed perlayer and the length of the critical path at training time, generation stillremains an inherently sequential process.To overcome this limitation, wepropose a novel blockwise parallel decoding scheme in which we make predictionsfor multiple time steps in parallel then back off to the longest prefixvalidated by a scoring model.This allows for substantial theoreticalimprovements in generation speed when applied to architectures that can processoutput sequences in parallel.We verify our approach empirically through aseries of experiments using state-of-the-art self-attention models for machinetranslation and image super-resolution, achieving iteration reductions of up to2x over a baseline greedy decoder with no loss in quality, or up to 7xinexchange for a slight decrease in performance.In terms of wall-clock time, ourfastest models exhibit real-time speedups of up to 4x over standard greedydecoding.机器学习论文:深度自回归模型的分块并行解码(Blockwise Parallel Decoding for Deep Autoregressive Models) hIeCCCAEKA88Ex1A.jpg
URL地址:https://arxiv.org/abs/1811.03115     ----pdf下载地址:https://arxiv.org/pdf/1811.03115    ----机器学习论文:深度自回归模型的分块并行解码(Blockwise Parallel Decoding for Deep Autoregressive Models)
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则 返回列表 发新帖

comceo当前离线
新手上路

查看:207 | 回复:0

快速回复 返回顶部 返回列表