A Convolutional Attention Network for Extreme Summarization of Source Code

Miltiadis Allamanis, Hao Peng, Charles Sutton

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Attention mechanisms in neural networks have proved useful for problems in which the input and output do not have fixed dimension. Often there exist features that are locally translation invariant and would be valuable for directing the model’s attention, but previous attentional architectures are not constructed to learn such features specifically. We introduce an attentional neural network that employs convolution on the input tokens to detect local time-invariant and
long-range topical attention features in a context-dependent way. We apply this architecture to the problem of extreme summarization of source code snippets into short, descriptive function name-like summaries. Using those features, the model sequentially generates a summary by marginalizing over two attention mechanisms: one that predicts the next summary token based on the attention weights of the input tokens and another that is able to copy a code token as-is directly into the summary. We demonstrate our convolutional attention neural network’s performance on 10 popular Java projects showing that it achieves better performance compared to previous attentional mechanisms.
Original languageEnglish
Title of host publicationProceedings of The 33rd International Conference on Machine Learning, PMLR
Place of PublicationNew York, United States
Number of pages10
Publication statusPublished - 24 Jun 2016
Event33rd International Conference on Machine Learning: ICML 2016 - New York, United States
Duration: 19 Jun 201624 Jun 2016

Publication series

NameProceedings of Machine Learning Research
ISSN (Electronic)2640-3498


Conference33rd International Conference on Machine Learning
Abbreviated titleICML 2016
Country/TerritoryUnited States
CityNew York
Internet address


Dive into the research topics of 'A Convolutional Attention Network for Extreme Summarization of Source Code'. Together they form a unique fingerprint.

Cite this