NeurIPS Rejects Fuel ICLR Submissions: Authors Debate Feedback Implementation

The Post-Rejection Pivot: Navigating Feedback for ICLR

The machine learning research community is gearing up for another submission cycle, with many authors facing the familiar sting of a NeurIPS rejection. As the dust settles from one major conference, attention quickly shifts to the next, primarily ICLR. This transition period, however, brings a critical question to the forefront for researchers: How much of the feedback received from NeurIPS reviewers actually influences the revised paper submitted to ICLR? The sentiment on platforms like Reddit's r/MachineLearning suggests a spectrum of approaches, from addressing every critique to selectively incorporating only what is deemed essential.

For many, a NeurIPS rejection is not an endpoint but a redirection. The immediate impulse is to salvage the work, refine it based on the feedback, and aim for a different venue. This process, however, is far from uniform. Some researchers feel compelled to address every single point raised by reviewers, viewing each comment as a potential avenue for improvement. Others adopt a more pragmatic stance, prioritizing changes that align with their vision for the paper or that address what they perceive as genuine flaws. This selective approach often stems from a belief that not all reviewer comments are equally valuable or even accurate, particularly when dealing with subjective assessments like novelty or significance.

The core of this debate lies in the interpretation and application of reviewer feedback. While reviewers provide a valuable external perspective, their comments can sometimes be contradictory, misinformed, or simply not aligned with the authors' intended contribution. The challenge for researchers is to discern which feedback points are constructive and which might lead the paper astray from its original intent or impact. This decision-making process is often influenced by the perceived stakes of the new submission and the authors' confidence in their original work.

The Novelty and Significance Conundrum

A recurring theme in discussions about rejections, particularly for papers submitted to top-tier conferences like NeurIPS, revolves around comments questioning the paper's novelty or significance. These critiques can be particularly disheartening and difficult to address. Reviewers might state:

• “The contribution is incremental.”
• “Not sufficiently different from prior work.”
• “The empirical gains don’t justify the proposed method.”
• “The problem itself isn’t significant enough.”
• “Theoretical contribution is limited.”

When faced with such feedback, authors must grapple with how to demonstrate a substantial leap forward. Simply tweaking an existing method or presenting minor performance improvements often falls short of what is considered groundbreaking in the field. For papers that might represent a subtle but important shift in perspective or a novel application of existing techniques to a new domain, articulating this value clearly to reviewers can be a significant hurdle. The subjective nature of 'significance' means that what one reviewer finds incremental, another might see as a crucial step.

Consider a paper that introduces a novel regularization technique. A reviewer might deem the empirical gains insufficient to warrant a new method if the performance uplift is only a few percentage points. However, the real value might lie in the theoretical properties of the regularization or its potential to generalize to a wider range of tasks, even if not fully explored in the current submission. The challenge for the authors is to decide whether to pivot the paper's narrative to emphasize these aspects, conduct additional experiments to bolster empirical claims, or accept that the reviewers' perception of significance may differ fundamentally from their own.

The decision to address novelty or significance feedback often requires a strategic re-evaluation of the paper's core message. If reviewers consistently point to a lack of differentiation from prior work, authors might need to conduct a more thorough literature review, explicitly contrast their method with existing approaches, or even reframe their problem statement to highlight a unique angle. This can be particularly demanding when the original research was conceived with a specific problem or contribution in mind. It forces authors to consider if their initial hypothesis about what constitutes a significant contribution was misaligned with the community's current expectations.

Selective Implementation vs. Comprehensive Overhaul

The decision of how much feedback to implement often boils down to a trade-off between preserving the original contribution and appeasing reviewer concerns. A researcher who meticulously addresses every single comment might produce a paper that is significantly different from the original submission, potentially diluting its core message or introducing new, unaddressed weaknesses. Conversely, a researcher who ignores substantial criticism risks facing similar objections from the next set of reviewers.

The pragmatic approach, favored by many, involves a careful assessment of each reviewer comment. This means distinguishing between feedback that offers actionable insights for improvement and feedback that might be based on a misunderstanding or a reviewer's personal preference. For instance, a suggestion to add a new experimental setting to test robustness is often valuable. However, a demand to completely rewrite a theoretical section based on a reviewer's alternative, unproven hypothesis might be considered less critical, especially if it deviates significantly from the paper's established theoretical framework.

This selective process requires a deep understanding of the research, the field, and the conference's expectations. Authors must ask themselves: Does this feedback genuinely strengthen the paper's claims, clarity, or reproducibility? Or is it a request that would fundamentally alter the paper's contribution in a way that doesn't align with its core goals? The goal is not merely to satisfy reviewers but to produce a stronger, more impactful piece of research. Sometimes, this means engaging in a dialogue with reviewers (if the conference allows for rebuttal) to clarify misunderstandings or to explain why certain suggestions might not be applicable.

What nobody has addressed yet is the psychological toll of this iterative process. Authors invest months, sometimes years, into their research. A rejection, followed by the task of dissecting potentially hundreds of pages of feedback, can be exhausting. The decision to resubmit to ICLR, or another venue, is often fueled by a desire for validation and recognition, but the path there requires careful navigation of conflicting advice and a steadfast commitment to the research's integrity.

The ICLR Threshold: What Constitutes 'Meaningful Changes'?

The ultimate goal in resubmitting to ICLR is to demonstrate that the paper has been significantly improved. But what constitutes 'meaningful changes'? Is it the number of pages added, the extent of experimental results, or the depth of theoretical revisions? For papers that were rejected due to concerns about novelty or significance, the 'meaningful changes' must directly address these issues. This might involve:

• Introducing a substantially new experimental setup that validates the method on a wider range of tasks.
• Providing a more rigorous theoretical analysis that strengthens the claims of novelty or performance guarantees.
• Clearly articulating the paper's unique contributions and differentiating it more explicitly from existing literature.
• Conducting ablation studies that demonstrate the essential components of the proposed method and their impact.

The success of a resubmission often hinges on the reviewers' perception of these changes. If the core issues raised during the NeurIPS review are not visibly and convincingly addressed, the paper is likely to face similar criticism. For authors, this means not just making changes, but making the right changes and clearly communicating those improvements in the revised manuscript and cover letter.

The journey from a NeurIPS rejection to an ICLR submission is a testament to the resilience and iterative nature of scientific research. It highlights the complex interplay between authorial intent, reviewer feedback, and the evolving standards of top-tier conferences. The decision on how much feedback to implement is a strategic one, balancing the desire for acceptance with the imperative to produce sound, impactful research.