April 18 - 23, 2015
About Crossings »
April 18 - 23, 2015
About Crossings »
The CHI review process for 2015 will continue to make use of the subcommittee structure introduced for CHI 2009. The key points in undertaking reviews are:
To improve the reviewing process, authors submit their papers to sub-committees that best reflect their most salient contribution, where each sub-committee is responsible for various topic areas typically seen at CHI (see: Selecting a subcommittee for details). Each sub-committee will comprise of a sub-chair and associate chairs who are knowledgeable in these topics. The idea is that, as specialists in the theme areas, they should be able to find good referees (such as you) for each submission, and that as specialists they should be able to better handle the meta-review process.
However, as a reviewer, you should not judge the paper by how well it fits the subcommittee theme(s). Many papers will not cleanly fit into a particularly sub-committee for a variety of reasons, and we do not want to penalize authors for this. Remember, the subcommittee organization is there only to try to improve reviewer matches and to better handle the volume of submissions. If you have a paper that does not fit the subcommittee theme, evaluate it as best you can with respect to its own quality. Any topic is valid, as long as it fits within the interests of a reasonable fraction of the overall CHI audience - the primary criterion remains the contribution to HCI.
The primary criterion for evaluation of all Papers and Notes is the research's contribution to HCI. In all cases, a CHI Paper or Note must break new ground and make an original research contribution. However, it is important to recognize that there are many ways for which a paper can make a contribution to HCI, and you should review the paper appropriately. Please see Contributions to HCI for a list of some of the types of contributions a paper can make to HCI, and the associated criteria that you can use to assess this type of contribution.
Papers and Notes are both reviewed within the same rigorous review process and at the highest level are judged by very similar criteria (i.e., does this paper or note provide a strong contribution to the field of HCI?). However, it is important as a reviewer to realize that the type of content that is appropriate for each is somewhat different. In particular, Notes present brief and focused research contributions that are noteworthy, but may not be as comprehensive or provide the same depth of results as a full paper. For details about these differences see Papers versus Notes: What is the difference?
The numeric ratings on the review form are on a 5 point scale (with half steps) starting at 1.0 (the lowest ratings) to 5.0 (the highest). Be as accurate with your ratings as possible. As in previous years, it is your written appraisal that is crucial. Make sure your review is balanced, and that its details reflect the numeric rating. The appraisal of a paper should always indicate why the paper deserves that rating. Your review may be discounted if, for example, you numerically rate a paper highly but only indicate its flaws. As well, please be polite to authors. Even if you rate a paper poorly, you can critique it in a positive voice. As part of polite reviewing practice, you should always state what is good about a paper first, followed by your criticisms. If possible, you should offer suggestions for improvement along with your criticism.
Content appearing at CHI should be new and ground-breaking. Therefore, material that has been previously published in widely disseminated publications should not be republished unless the work has been "significantly" revised. Guidelines for determining "significance" of a revision are stated in the ACM Policy on Pre-Publication Evaluation and the ACM Policy on Prior Publication and Simultaneous Submissions. Roughly, a significant revision would contain more than 25% new content material (i.e., material that offers new insights, new results, etc.) and significantly amplify or clarify the original material. These are subjective measures left to the interpretation and judgment of the reviewers and committee members - authors are advised to revise well beyond the Policy guidelines.
An exception is for work that has previously been presented or published in a language other than English. Such work may be translated and published in English at CHI. The original author should typically also be the author (or co-author) of work translated into English and it should be made clear that this is a translation. For more information see: http://www.sigchi.org/conferences.
The debate about what makes a good CHI paper has been ongoing for several years. If you are interested, the papers below touch upon this debate and contain references to additional papers that concern it.
