Dual-Anonymous Guidelines
ALMA is strongly committed to ensuring that the proposal review process is as fair and impartial as possible. Analysis of the proposal rankings in previous cycles has identified systematics that may signify the presence of biases in the review process (see Systematics in the ALMA Proposal Review Rankings). To reduce any biases as much as possible, ALMA will use a dual-anonymous proposal review process starting in Cycle 8 2021. In a dual-anonymous review, the proposal team does not know the identity of the reviewers, and the reviewers do not know the identity of the proposal team. The ultimate goal is that proposal reviews will focus on the science of the proposal, rather than the scientist or scientific team that wrote the proposal. While proposers will still enter their names and affiliations in the ALMA Observing Tool (OT), this information will not appear on the proposal cover sheet, nor in the tools used by the reviewers. It is the responsibility of the proposers to ensure anonymity is preserved when writing the scientific and technical justification sections of their proposals.
Guidelines are provided below on how to write proposals in an anonymous fashion. Most of the changes are in the writing style, including references. Proposers should remember to make the relevant changes, especially when resubmitting a proposal from a previous cycle.
General Guidelines pertaining to all Programs
- Do not identify the PI or any of the co-PIs or co-Is in the proposal. This includes, but is not limited to, the text, abstract, figures, footnotes, and tables, as well as the technical justification.
- Proposers should use third person or neutral wording when referencing their own work. For example, instead of:
In Smith et al. (2018), we demonstrated…
proposers can include references in the following formats:
As demonstrated in Smith et al. (2018),....
or
As demonstrated in [1],...
where in the latter case, [1] would correspond to the full citation in a reference list.
- Do not refer to data from ALMA or other observatories in a self-identifying fashion. If data from a project have been published, then a reference to that publication may be used. Unpublished data can be referenced by the project code. For example, instead of
Figure 1 shows the image from our Cycle 7 ALMA program (2019.1.02045.S, PI Smith).
proposers can write
Figure 1 shows the image from the Cycle 7 ALMA program 2019.1.02045.S.
- Software and datasets that are available in a public repository (e.g., GitHub) or in a public paper can be referenced per normal practices. If the software or data are not public, it can be referenced as “obtained via private communication” or similar language, but a name should not be specified since it could strongly imply who may be an investigator on the proposal. For example, instead of:
We use our group’s line identification package STAR… or
We use the line identification package STAR by co-I Sandra Smith…
proposers can write
We use the line identification package STAR (obtained via private communication)...
- Do not include references and links to papers in preparation or submitted that are stored on personal web pages. References to submitted papers on public archives (e.g., arXiv) are acceptable.
- Do not include personal acknowledgements or the source of any grant funding that may identify the proposers.
- While proposers may note if they are resubmitting an ongoing Cycle 7 proposal, they should not indicate the proposal code and investigators of the previously accepted proposal. For example, instead of
This is a resubmission of our ongoing Cycle 7 program 2019.1.02045.S (PI: Smith). Half of our targets have been observed and we are resubmitting the proposal to obtain the remaining half.
proposers can write
This is a resubmission of our ongoing Cycle 7 program. Half of our targets have been observed and we are resubmitting the proposal to observe the remaining half.
Resubmissions that present unpublished data in a figure may reference the project code using the example presented in the third bullet point, above.
Example text
Here is an example text that would need to be modified according to the guidelines, with the text to be changed in bold:
We propose to perform a multi-band, beam-matched spectral scan of the central molecular zone of the nearby starburst galaxy NGC 253 in order to obtain the first template of extragalactic molecular complexity and calibrate extragalactic molecular diagnostics. To sample a wide range of molecular excitation states, we will scan the full ALMA bands 3, 4, 6, and 7. From our previous ALMA observations (Mangum+2015), we estimate that in band 6 and 7 we will obtain confusion limited spectra in most of the central region. Our pioneering studies of multi-band spectral scans (e.g., Costagliola+2015) show that the combined effect of more optically thin tracers and proper treatment of molecular excitation can lead to a tenfold increase in the sensitivity of molecular diagnostics to the physical properties of the ISM.
Here is the same text revised according to the guidelines:
We propose to perform a multi-band, beam-matched spectral scan of the central molecular zone of the nearby starburst galaxy NGC 253 in order to obtain the first template of extragalactic molecular complexity and calibrate extragalactic molecular diagnostics. To sample a wide range of molecular excitation states, we will scan the full ALMA bands 3, 4, 6, and 7. Based on previous ALMA observations (Mangum+2015), we estimate that in band 6 and 7 we will obtain confusion limited spectra in most of the central region. Previous studies with multi-band spectral scans (e.g., Costagliola+2015) show that the combined effect of more optically thin tracers and proper treatment of molecular excitation can lead to a tenfold increase in the sensitivity of molecular diagnostics to the physical properties of the ISM.
Guidelines pertaining only to Large Programs
Proposals for Large Programs will now consist of two parts.
1. The first part contains the scientific justification that must be prepared following the dual-anonymous guidelines above. The scientific justification should motivate the proposed science and observations, include an assessment of the scheduling feasibility, describe the data products that will be delivered, and present the publication plan. The scientific justification is allowed to be 6 pages maximum and is submitted through the ALMA Observing Tool (OT) as with any other proposal.
2. The second part consists of a one-page PDF document that contains the management plan and a description of the available computing resources. Names and affiliations of at least the key members of the collaboration should be included in this one-page document. This statement must be submitted through the ALMA OT at the time of proposal submission.
Large Programs will be assessed initially based on the scientific justification only. After the scientific evaluation has been completed by the review panels and the ALMA Proposal Review Committee (APRC), the APRC will review the management plans and computing resources. At this point, the APRC may recommend to the ALMA Director that a proposal be rejected only if they feel the proposal team is not qualified to carry out the program or does not have the necessary computing resources. However, the scientific rankings of the proposals will not change. The ALMA Director will make the final decision on which Large Programs are accepted.
Compliance
PIs are required to anonymize their proposals. PIs who do not anonymize their proposals in accordance with the guidelines may have their proposal rejected. Extreme violations will be subject to disqualification before peer review. Proposals with minor violations (e.g., unintentional first person reference that should have been in third person) will be allowed to continue through the review process; reviewers will be instructed to flag such proposals to the JAO. PIs will be provided with feedback regarding any detected violations, particularly to avoid similar violations in future cycles. In some cases, a proposal may be very specialized and the identity of the proposal team may seem obvious to the reviewers even after the text is anonymized. As long as the guidelines to anonymize the proposals are followed, the proposal will not be considered in violation.
Guidelines for Reviewers
Proposals will be reviewed through either the distributed peer review process or a panel process. Here we refer to “reviewers” as those participating in the distributed review, as well as science assessors participating in the panel review. The role of reviewers is to evaluate the scientific merit of the proposals. In order to keep the process focused on the science, here we recommend guidelines to follow during the proposal review:
- Reviewers should rank (for distributed review) or score (for panel review) the proposals based on scientific merit.
- Reviewers should not try to guess the identities of the PI or the proposing team.
- In the case of panel review, panel chairs will be responsible for refocusing the discussion on the science merit of the proposal if the panel begins to discuss the identity of the proposal team.
- Even if a reviewer feels that a proposal has not followed the guidelines for anonymity, they should continue to review the proposal based on scientific merit. The JAO will make the final decision on the proposal. In these cases, reviewers may notify the JAO by the following methods:
- For reviewers participating in the distributed peer review, please use the box “Comment to JAO” via the Reviewer Tool.
- Participants of the panel review should contact the Proposal Handling Team directly.
Frequently Asked Questions
Answers to frequently asked questions are available on the dual-anonymous peer review FAQ page.
Return to the main ALMA Proposal Review page