The ALMA proposal review process is organized by the Proposal Handling Team (PHT) at the Joint ALMA Observatory (JAO). ALMA proposals are selected by competitive peer review through either the distributed peer review process or the ALMA Proposal Review Committee (APRC).

Proposals that request less than 50 h on the 12-m Array or less than 150 h on the 7-m Array in standalone mode are reviewed using the distributed peer review system, in which the proposal team designates one member of the proposal team to participate in the review process. The outcomes of the distributed peer review process are:

• A scientifically ranked list of proposals
• Individual comments for each proposal written by the reviewers that are sent to the Principal Investigators (PIs).

Large Programs, i.e., proposals that request more than 50 h on the 12-m Array or more than 150 h on the 7-m Array in standalone mode, are reviewed by the APRC, a panel composed of experts selected from the international astronomical community. To gain further expert assessment, external Science Assessors will provide reviews on Large Programs, that will be considered by the APRC. The outcomes of the APRC review process are:

• A list of recommended Large Programs
• A consensus report for each Large Program that summarizes the strengths and weaknesses of the proposal.

All proposals are reviewed in a dual anonymous fashion in which the proposers do not know the identity of the reviewers and the reviewers do not know the identity of the proposers. All proposals need to be prepared in accordance with the dual-anonymous guidelines.

 

Distributed peer review plenary session

The PHT will be hosting a series of live Webinars, during which the PHT will present a video that describes distributed peer review and will also be available to answer your questions. The video will be posted here before the Webinars. Reviewers and mentors are strongly encouraged to view the video or attend one of the sessions. The Webinars will be held between 9-14 May at different times to accommodate reviewers and mentors in each of the ALMA regions. Reviewers and mentors can attend any one of the three scheduled Webinars that is most convenient to them. The schedule for the three sessions is below:

Session 1: Thursday May 9, 17:00 UTC

Session 2: Friday May 10, 13:00 UTC

Session 3: Tuesday May 14, 2:00 UTC 

Prior to the Webinars, the PHT will send an email to all reviewers and mentors with more detailed information.

Plenary Session Slides

 

Available documents for the proposal review process

 

Document	Description
Dual-anonymous Guidelines	Provides guidelines for PIs on how to write their proposals in a dual-anonymous fashion
Distributed Peer Review	Detailed description of the distributed peer review process
Guidelines for Reviewers	Guidelines for reviewing proposals and writing comments to the PIs
How to use the Reviewer Tool	Reviewer Tool manual for the distributed peer review process
Frequently asked questions	Contains answers to numerous questions about the dual-anonymous format and distributed peer review

 

New in Cycle 11

Large Programs proposal format and Management Plan

The format of proposals for Large Programs has been modified. The Management Plan, which was previously submitted as a separate document through the OT, must now be included as part of the Scientific Justification. The Management Plan should include a description of (1) Organization of the team, (2) Data reduction plan and available computing resources, (3) Data products, and (4) Publication plan. The management plan must be properly anonymized following instructions on the Science Portal.  The page limit for the Scientific Justification has been increased to 7 pages, and at least one page should be devoted to presenting the management plan.

Proposals teams will also upload a one-page PDF statement detailing the "Team Expertise” that is submitted through the OT. The statement should contain information about the most relevant members of the team and their role in the project. This statement does not need to follow the anonymous guidelines, and should include investigator names and affiliations.

For further details on Large Programs, please refer to Section 5 of the Proposer’s Guide. 

 

Distributed peer review

• As in Cycle 10, if a reviewer needs to contact the ALMA Proposal Handling Team (PHT) during the proposal review process, they should do so by opening an  ALMA helpdesk ticket (with the "Proposal Review Support" department).

 

Proposal review timeline

Table 1 summarizes the timeline of the Cycle 11 proposal review process. A brief description of each step is given below. 

Table 1: Timeline of the Cycle 11 Proposal Review Process

Date	Milestone
30 April 2024	Deadline for reviewers to provide their scientific expertise and their list of conflicts of interest before proposals are assigned.
08 May 2024	Proposals are released to all reviewers. Stage 1 begins for both distributed peer review and the APRC.
15 May 2024	Deadline to provide conflicts of interest
05 June 2024	Stage 1 deadline for all reviewers.
06 June 2024	Distributed peer review Stage 2 begins.
20 June 2024	Distributed peer review Stage 2 deadline.
24-28 June 2024	APRC meeting.
August 2024	Results sent to PIs.

 

Distributed peer review

Here is a quick guide to the basic steps in distributed peer review, from proposal submission to the end of the review process. A complete description of the process can be found here.

1. Submit a proposal through the ALMA Observing Tool. When submitting the proposal, the PI must designate one member of the proposal team to be a reviewer.  
2. Reviewers should indicate their scientific expertise through their user preferences on the ALMA Science Portal by 30 April 2024, 15:00 UTC. All efforts will be made by the PHT to assign proposals to reviewers that match their field/s of expertise. Click here for more information on how the keywords are used by the PHT to assign proposals.
3. Reviewers have the option to provide a list of investigators for their conflicts of interest through their user preferences on the ALMA Science Portal. Reviewers need to provide this list by 30 April 2024, 15:00 UTC, at which point the PHT will begin the proposal assignment process. The PHT will not assign proposals to a reviewer in which a PI, co-PI, or co-I is in this list. Click here for more information on declaring conflicts of interest.
4. The PHT will assign 10 proposals (a “Proposal Set”) to the reviewer on behalf of each proposal in which they are the designated reviewer.
5. In Stage 1, the reviewers will rank the proposals within a Proposal Set from 1 (strongest) to 10 (weakest) and write a review for each proposal that indicates the strengths and weaknesses. Click here for guidance on writing helpful reviews. Stage 1 is mandatory. Reviewers must submit their reviews by the Stage 1 deadline (05 June 2024, 15:00 UTC) or the corresponding proposal for which they were identified as the designated reviewer will be rejected.
6. In Stage 2, reviewers can read the anonymized comments from the other reviewers and revise their own ranks and reviews as needed. Participation in Stage 2 is strongly encouraged, and the deadline is 20 June 2024, 15:00 UTC. If a reviewer does not submit the Stage 2 ranks and reviews, then the Stage 1 ranks and reviews will be considered final.

 

ALMA Proposal Review Committee

Large Programs will be sent for review to the APRC. The general steps for the assessment of these proposals are outlined below.

Stage 1

The APRC members review their assigned proposals, provide a numerical score for the proposals, and write comments summarizing the strengths and weaknesses of the proposal. The PHT assigns each Large Program a Primary Assessor, who is responsible for leading the proposal discussing in Stage 2 and writing the consensus report that will be sent to the PI.

External Science Assessors review a small number (1-3) of Large Programs in their area of expertise and provide a written report with their reviews.

Stage 2

The APRC  read the reports of the external Science Assessors in preparation for the meeting. The APRC discusses the proposals and recommends which Large Programs should be scheduled. The Primary Assessors write consensus reports to reflect the final scientific assessment.

 

Queue building

The main outcome of the proposal review process is a scientifically ranked list of proposals. The inputs used to generate the ranked list are 1) the proposal rankings of Proposal Sets from individual reviewers in distributed peer review, and 2) the ranked list of Large Programs recommended by the APRC.

In distributed peer review, each reviewer ranks their assigned proposals from 1 (strongest) to 10 (weakest). The rankings from individual reviewers are combined to form an overall ranked list. For a given proposal, the highest and lowest ranks are dropped to remove possible outliers. The remaining ranks are averaged.  The sorted list of average ranks determines the overall scientific ranked list of proposals.

The ranked list of proposals only establishes the priority order to build the observing queue.  The Large Programs recommended by the APRC have first priority in building the queue, but they are limited to how much time they can fill in a configuration/LST as described in the Proposer’s Guide. Creating the observing queue also takes into consideration the available time in a configuration and LST, the amount of time allocated to each executive, and the amount of time available based on historical weather conditions for a given observing band and observing frequency. It is during the queue-building stage that the priority grades (A, B, C) are assigned to the accepted proposals, and at this stage a proposal may be declined depending on the amount of telescope time available. Because numerous factors are considered in building the queue, it is possible for a high ranked proposal to be declined while a lower ranked proposal is accepted.

After the ranked list is generated and the observing queue is created, the ranked list and the priority grades are sent to the ALMA Director for approval. After the Director’s approval, the results are sent to representatives from East Asia, Europe, North America, and Chile for final approval. Principal Investigators are then notified of the results.