SC20 Is Everywhere We Are

Virtual Event FAQ
Artboard 1
Register
Home Submit Paper Submissions Papers FAQ

Papers FAQ

Find answers to common questions that may arise while you prepare your submission. If your question is not addressed here, or by carefully reviewing the Paper Submissions page, please contact the Papers Committee.

 

Contribution Areas

Paper Eligibility

Review, Response, Revision

Double-Blind Review Policy

Artifact Description (AD) and Artifact Evaluations (AE) Appendices

Plagiarism

 

Contribution Areas

 

Is it mandatory for authors to select a primary contribution area?

Yes. Authors must indicate their primary area of contribution from the area choices on the submissions form. We understand that contributions may straddle more than one area. In such cases, we encourage authors to indicate a secondary area of contribution.

How are reviewers assigned to submitted papers?

Papers are assigned to the most suitable program committee for the area of contribution. The primary and secondary areas selected by paper authors are used as guidelines, but do not exclude more suitable program committees.

What constitutes a State of the Practice (SOP) paper?

An SOP paper can describe a first-of-its-kind technology or methodology, or can capture a unique perspective or experience on issues, challenges, and solutions for dealing with aspects of unprecedented scale and complexity, particularly experience and knowledge that can be generalized to a wide range of systems and usages. Concrete case studies within a conceptual framework (i.e., experiential topics) would likely serve as the basis for SOP papers, and generalizing the experience to wider applicability should be explored.

Will State of the Practice (SOP) papers be rejected because they are being held to the same rigorous standards as other papers?

Although SOP papers will be reviewed with the same academic rigor as the papers in other areas, the acceptance criteria will be tailored to value new and generalizable insights as gained from experience with particular HPC machines, operations, applications, or benchmarks; overall analysis of the status quo of a particular metric of the entire field; or historical review of the progress of the field.

Such papers are common in other academic disciplines, including branches of computer science. For example, software engineering highly values the “experience papers” of particular frameworks and methodologies; human–computer interaction produces numerous analyses of human behavior given particular interfaces; social sciences collect data on social phenomena and produce meaningful insights based on statistical analysis.

If I used machine learning to improve X, should I select the Machine Learning and HPC (ML&HPC) area, or the area for X?

Authors should send papers whose focus is on introducing new machine learning techniques that are not specific to another Papers area to the ML&HPC area. Authors should send papers whose work is specific to another area (Applications, Architecture and Networks, System Software, etc.) to that area. ML&HPC may be listed as the secondary contribution area if appropriate.

 

Paper Eligibility

 

Can I put my paper on arXiv before submitting it to SC?

Yes. Papers that have not previously been published in a peer-reviewed conference or workshop are eligible for submission to SC.

If my paper was already published in a workshop, can I also submit it to SC?

No. Unless a paper has been substantially enhanced (i.e. 30% new material) since prior publication, it is not eligible for submission to SC.

 

Review, Response, Revision

 

Is there an author response/rebuttal stage?

Yes. Once authors receive initial feedback from reviewers, they are given the opportunity to improve their paper accordingly before the next round of reviews. For example, if a reviewer indicates that a paper requires some critical measurement or explanation in order to be accepted to the conference, the authors can supply that missing information. The revised paper (with revised text indicated in a blue font) is then submitted with an accompanying response document highlighting the changes made.

This is different from the rebuttal stage for many other conferences in that the focus is less on promises to fix problems for the final document and more on actually fixing the problems.

If I don’t have time to gather the additional data requested by my reviewers before the revise-and-respond deadline, can I do so by the final paper deadline?

Yes. The reviewers may want to see a major revision in this case to ensure that no information critical to the paper’s argument is missing.

After reading the reviews of my paper, can I reorganize it and upload a new version during the revise-and-respond period?

Yes. We encourage you to submit this kind of change as a revision, provided that you clearly document your changes. Given the committee members’ limited time in which to read and act on your revision, you must specify what changes you have made and why.

If I obtain new results for my paper after submission, can I upload them during the revise-and-respond period?

Yes. Again, you must clearly document any changes and their rationale in your author response.

Doesn’t this incentivize authors to initially submit a very rough draft and only make revisions after reviewer feedback?

Perhaps. But as the old adage goes, there’s no second chance to make a first impression. Your reviewers will have much more time to consider your first draft, and their initial scores will be based on it. SC introduced the revise-and-respond process with the belief that an actual revision is far more persuasive than promises made in a rebuttal. That does not mean that revisions will change a reviewer’s mind if their first impression of a paper was completely negative. Reviewers will have limited time to scrutinize your revisions, so you should strive to make every stage of the submission process count.

If I have concerns that a reviewer seemed to lack basic knowledge of the area on which my paper focuses, how should I address them in my author response and revision?

We have all received reviews that made us angry, especially on first reading. The revision period is short and doesn’t allow for the cooling-off period that authors have before they write a response to a journal review.

We provide you the opportunity to submit a revision and response to address any issues, but advise you to be careful with the wording of your response. For example, we recommend you don’t write: “If reviewer X had just taken the time to read my paper carefully, they would have realized that our algorithm was rotation invariant.” Instead, write: “Unfortunately, Section 4 must not have been as clear as we had hoped because Reviewer X did not understand that our algorithm was rotation invariant and they were therefore skeptical about the general applicability of our approach. The revised version of the second paragraph in Section 4 should clear up this confusion.”

The SC revise-and-respond process offers more opportunity than other conferences’ review processes do to convince your reviewers of the merits of your work; use such opportunities wisely.

 

Double-Blind Review Policy

 

What is double-blind reviewing?

Double-blind reviewing means that authors do not know the identities of their paper’s reviewers, nor do reviewers know the identities of the paper’s authors until after acceptance decisions have been made. Reviewers will not learn the identities of the authors of rejected papers.

Why does SC employ double-blind reviewing?

Studies indicate that double-blind reviewing goes a long way in reducing unconscious bias (e.g., based on institution, seniority, nationality, or gender) on the part of reviewers. For complete information, see SC’s Double-Blind Review Policy.

How should I prepare my paper for double-blind review?

SC does not demand that only an intelligence agency could possibly determine who the authors are; we simply require that there be some level of doubt as to the authors’ identities. Here are some suggestions:

  • Don’t state the authors’ names, institutions, or other identifying information anywhere in the paper. In particular, remove the acknowledgement section at submission time.
  • Write in the third person about your prior work, software, etc.
  • Don’t avoid citing your own work. But cite it in the third person, as in the example below.
  • Don’t obscure the hardware platforms you used/built, even if relatively few people have access to that hardware.
  • Don’t obscure the software you used/developed, even if relatively few people have access to that software.

For example, consider the following alternatives:

Bad

  • We enhanced our INTERCAL compiler [1] to accept keywords in Pig Latin.
  • [1] First name last name. “A high-performance parallel INTERCAL compiler”, International Conference on HPC-Related Stuff, 2019.

Good

  • We enhanced Bloggs and Doe’s INTERCAL compiler [1] to accept keywords in Pig Latin.
  • Joe Bloggs and Jane Doe. “A high-performance parallel INTERCAL compiler”, International Conference on HPC-Related Stuff, 2019.

Bad

  • We ran all of our experiments on a large HPC system at a high-security supercomputing center.

Good

  • We ran all of our experiments on the WOPR supercomputer at the Cheyenne Mountain Complex.

Should I still prepare my paper for double-blind review if I don’t mind reviewers knowing who I am?

Yes. The point is to avoid bias, not only against certain groups but also in favor of certain groups. We don’t want reviewers automatically favoring a paper written by a Turing Award winner at MIT over a paper written by a first-year student at a lesser-known college.

If I made my paper available under the same name with the same authors as an institutional technical report, arXiv preprint, or other non-peer-reviewed release, won’t reviewers be able to easily figure out who I am?

They could, but reviewers are asked to avoid trying to learn authors’ identities.

 

Artifact Description (AD) and Artifact Evaluation (AE) Appendices

 

For complete information, see SC’s Transparency & Reproducibility Initiative.

 

What is the AD appendix?

The AD appendix, generated by the SC submission site from content gathered via a form, allows authors to share with readers a description of the data, software, and hardware artifacts relied on to produce the paper’s results and provide links to persistently archived data and software products.

Does writing an AD appendix imply I can skip or shorten my paper’s experimental setup or other such sections?

No. An SC paper must stand on its own and will be reviewed as such. The AD appendix provides additional information to appendix reviewers and readers, including information that would break the double-blind review process of the paper as well as information explaining specifically how to reproduce and build on the results.

What is the AE appendix?

The AE appendix, generated by the SC submission site from a free-text form field, lets authors explain the trustworthiness of their data by detailing their approach to verification and validation, statistics gathered, uncertainty-quantification techniques applied, conditions controlled/not controlled for, and other details that convey the sense of confidence on the reported results. This is especially useful for results obtained in computational environments that are not widely accessible and/or environments that are not easily reproducible.

Are the AD and AE appendices mandatory or optional?

The AD appendix is mandatory for all papers in the Technical Program, and optional for posters and workshop papers. The AE appendix is optional but strongly recommended.

What if my paper does not have any associated artifacts?

Although we expect this to be a rare occurrence for SC papers, the first question in the required Artifact Description form enables authors to indicate that they do not have any software, hardware, or data artifacts associated with the submitted paper.

Will paper reviewers also review the AD and AE appendices?

No. Papers and Appendices will be reviewed separately, the latter by AD/AE reviewers (who are a completely separate team from paper reviewers). Paper reviewers will only see the Appendix reviews and factor them into the overall decision.

Doesn’t an AD conflict with double-blind reviewing?

The Appendix evaluation is double open. AD/AE reviewers may communicate with authors to help improve the AD/AE appendices. Since paper reviewers will not see the appendices, there is no conflict.

 

Plagiarism

 

Is is possible for authors to plagiarize their own work? 

Yes. This is called self-plagiarism and it is not acceptable. Please see the IEEE guidelines for identifying plagiarism for complete information. 

AD/AE Appendix FAQ & Author Kit

 

Resources for preparing and submitting AD/AE appendices with your paper.

View FAQ

Back To Top Button