What is the Delphi Method and How Do You Use It in Surveys?

The Delphi method is a structured research technique that gathers and refines expert opinions through multiple rounds of surveys to reach consensus. Originating from the RAND Corporation in the 1950s, it’s now widely used in healthcare, policy, education, and market forecasting. The process involves selecting a panel of experts, conducting iterative survey rounds, and providing anonymous feedback until opinions converge. Its strengths include anonymity, flexibility, and reliability in complex decision-making, though it can be time-consuming and costly. By combining expert insights and data-driven analysis, the Delphi method provides an effective framework for achieving well-informed, collective judgments on uncertain or emerging issues.

Need an expert opinion? The problem with expert opinions is that if you aren’t satisfied with them, you’ll want a second opinion. And then maybe a third or fourth. The Delphi research method provides a solution to that by essentially combining experts’ opinions and whittling them down into one final answer. The method is named after Delphi, referencing the ancient Greek oracle, which inspired this modern forecasting approach. Of course, it is more complex than that. This blog will dive into the Delphi methodology and how it works.

Create your Delphi survey or questionnaire now!

What is the Delphi Method?

The Delphi methodology gathers several expert opinions and insights on a particular topic or issue. It is a structured group communication process that enables experts to address complex issues through iterative feedback and consensus-building collaboratively. It aims to reach a consensus among this group, usually by conducting a series of surveys. At the end of the series of Delphi surveys, the hope is that some consensus will be reached. The Delphi method is particularly useful when dealing with complex or uncertain issues where there might not be a single “correct” answer, but by tapping into the collective wisdom of experts, there may be a “convergence of opinion.”

History of the Delphi Method

The Delphi method was developed in the 1950s as a way to gather expert opinions in order to address complex or uncertain problems. The name ‘Delphi’ comes from the ancient Oracle of Delphi, a woman who resided in a Greek temple near Athens, and who was known to provide advice. Here are some key events in its development.

• 1950s: The RAND Corporation, a nonprofit think tank in the United States, created the Delphi research method as a systematic way to collect and combine expert opinions for military and policy-related issues. The Delphi method was specifically designed as a forecasting tool to overcome the limitations of traditional forecasting methods, especially in situations where established scientific theories were lacking or uncertainty was high.
• 1960s: The Delphi method gains more recognition as important researchers of the time use it for technology forecasting.
• 1970s–1990s: The Delphi method expands across more disciplines, such as education planning, economic forecasting, and health care decision making.
• 2000s–Present: The method continues to evolve, with online and digital tools being integrated into the process. This makes it easier to engage experts from around the world.

Today, the Delphi method is widely recognized as a great way to handle complex and uncertain problems by bringing in a diverse group of experts to assist in decision-making.

5 Steps of the Delphi Method in Research

Conducting Delphi studies involves following a series of structured steps to ensure methodological rigor and reliable results. There are five main steps in the Delphi research method. They are as follows:

1. Participant Selection: A panel of experts is chosen based on their knowledge and experience in the relevant field. Delphi panels are formed by selecting a diverse group of experts to ensure a broad range of perspectives.
2. Round-Based Surveys Begin: Experts are asked to answer a series of questions or rate statements in the first round. These are known as Delphi rounds. How many rounds are conducted depends on the study’s objectives, complexity, and available resources.
3. Feedback and Iteration Collected: After each round, controlled feedback is provided to participants, summarizing the group’s responses and key findings. In subsequent rounds, experts review this feedback and have the opportunity to refine or adjust their responses, moving the group closer to consensus.
4. Repetition Until Stability: The process of surveying, providing feedback, and revising responses is repeated until the responses stabilize or a consensus is reached.
5. Analyzing the Convergence of Opinions: The final responses are analyzed to determine the degree of agreement among the experts. Consensus building is achieved by synthesizing group opinion through the iterative process, ensuring that the collective judgment of the panel is robust and reliable.

Create your Delphi survey or questionnaire now!

1. Participant Selection

A group of experts within a particular subject area is selected for the study. They may be located anywhere in the world and often don’t know one another.

2. Round-Based Surveys Begin

Next, multiple rounds of surveys or questionnaires are conducted with each expert to gather their opinions, predictions, and rationales. The questions may be qualitative and quantitative in nature, and question types can be open-ended or structured.

3. Feedback and Iteration Collected

After each round of surveys, the responses are analyzed, summarized, and made anonymous. The aggregated results are then shared with the participants in the subsequent round. Participants are encouraged to revise their opinions or answers based on the collective insights from the group.

4. Round-Based Surveys Continue

Round-based surveys or questionnaires are again distributed to the experts, often several rounds, until a consensus or convergence of opinions is reached, or until a predetermined stopping point is reached.

5. Analyzing the Convergence of Opinions and Achieving Consensus

With the surveying rounds complete, researchers narrow down the range of opinions and identify areas of agreement or disagreement among the experts. The goal is to reach a more refined and informed perspective on the research topic.

Create your Delphi survey or questionnaire now!

Measuring Consensus in the Delphi Method

Measuring consensus is at the heart of the Delphi method, as it signals when the iterative process has successfully brought a group of experts to agreement—or when further rounds are needed. In the Delphi process, consensus is typically assessed using statistical group response techniques, such as calculating the mean, median, or interquartile range of expert opinions. These quantitative methods help researchers determine when the responses have stabilized and whether the group has achieved a reliable expert consensus.

In health research, the Delphi method is widely used to generate consensus among experts on complex topics like healthcare quality indicators, clinical practice guidelines, and research priorities. The process involves several rounds of questionnaires, with each round providing anonymous feedback to panel members. This allows experts to reconsider and refine their responses based on the collective intelligence of the group, all while expressing their opinions freely and without outside pressure.

The number of rounds required to achieve consensus can vary depending on the research question and the level of agreement needed. Most Delphi studies in healthcare research use two to four rounds, but the process is flexible and can be adapted to the needs of the study. The iterative nature of the Delphi method works to gradually narrow the range of expert opinions, making it easier to identify areas of strong agreement as well as points of ongoing debate.

One of the strengths of using the Delphi method is its ability to highlight not only where consensus among experts exists, but also where there is disagreement or uncertainty. By analyzing the responses from each round, researchers can pinpoint which healthcare quality indicators or clinical practices are widely supported, and which require further investigation or discussion.

Overall, the Delphi method provides a systematic and transparent approach to achieving consensus in healthcare research and beyond. Its use of multiple rounds, anonymous feedback, and statistical analysis ensures that the final results reflect a true group consensus, making it a valuable tool for developing guidelines, setting priorities, and informing decision-making in complex fields.

Create your Delphi survey or questionnaire now!

Pros and Cons of the Delphi Method

Like all types of surveys, there are pros and cons to the Delphi method. Here’s a look at each.

Advantages:

• Anonymity reduces the influence of dominant individuals.
• Iterative rounds allow for reflection and refinement of opinions.
• The Delphi method is valued for its ability to achieve reliable consensus among experts.

Disadvantages:

• The process can be time-consuming and resource-intensive.
• It may require specialized assessment tools and consensus measurement techniques to ensure methodological rigor.

Advantages of the Delphi Method

• Expert Consensus. By combining the collective expertise of a diverse group of experts, the Delphi method helps reduce individual biases and provides a more comprehensive view of the topic.
• Anonymity. Participants can remain anonymous, which reduces the potential for conflict or dominance by a few outspoken members of the group.
• Flexibility. The Delphi method works in various fields and disciplines, making it adaptable to a wide range of research contexts.
• Structured Process. Through round-based surveys, the Delphi process allows for refining and clarifying opinions over time, leading to increased accuracy and reliability.
• Objective Interpretation. Decisions based on experts are likely to be efficient.

Disadvantages of the Delphi Method

• Resource-Intensive. The Delphi method can be time-consuming and almost always involves several rounds of surveys and data analysis.
• Expert Availability. Securing a panel of qualified experts who are willing to participate can sometimes be a challenge.
• Expensive. It is often expensive to retain experts for the numerous rounds of surveys they will need to engage with.
• Subjective Interpretation. The process of reaching a consensus might involve the subjective judgment of the researcher in determining when to stop the survey rounds and consider an agreement as reached.

Delphi Method vs. Other Research Methods

So how does the Delphi method stack up against other types of survey methods? People often compare it to focus groups and interviews. While these methods do provide qualitative insights, the Delphi method allows for the aggregation and quantification of expert opinions, making it more suitable for reaching consensus. In addition, focus groups generally gather all participants in one room or on one call. This can lead to dominant personalities overtaking the conversation and others holding back, potentially skewing results.

Another consensus method often compared to Delphi is the nominal group technique. Unlike the Delphi method, which relies on anonymous, iterative rounds of surveys, the nominal group technique involves structured, face-to-face group discussions where participants first generate ideas individually and then discuss them as a group to reach consensus. This direct interaction can facilitate immediate clarification but may also introduce group dynamics that influence individual responses.

There are also variants of the Delphi method, such as the argument Delphi, which emphasizes structured argumentation and debate among participants rather than just seeking convergence of opinions. This approach allows for a more flexible and dynamic exploration of differing viewpoints within the consensus-building process.

Traditional surveys, whether conducted via mail or online, generally offer just a single round of data collection. The Delphi method, which uses round-based surveying, allows for refining and revising opinions over multiple surveys. This helps to arrive at a consensus.

The Delphi method is one type of consensus method, and consensus analysis is often used to evaluate the level of agreement and stability achieved among experts in these studies.

Uses of the Delphi Methodology

The Delphi methodology aids researchers in many different ways, serving as a valuable research methodology for those looking to conduct research in complex domains. It is often used as an experimental application in emerging fields, allowing experts to explore new or untested areas through structured consensus-building.

In healthcare and education, the Delphi method is widely used across health sciences to develop guidelines, inform policy decisions, and achieve expert consensus on complex issues.

For product innovation or risk assessment, the Delphi technique is frequently applied during project development to improve scope definition, risk assessment, and overall project success.

There are several adaptations of the Delphi method to suit specific research or policy needs. The modified Delphi and modified Delphi process involve iterative consensus-building, often with fewer rounds and active steering group participation, particularly in healthcare research. The disaggregative policy Delphi is a variation used in policy research, breaking down policy issues into smaller components for detailed examination and improved stakeholder engagement.

Create your Delphi survey or questionnaire now!

1. Decision-Making

The Delphi method can forecast future trends or outcomes. Experts provide their opinions on different scenarios, and through iterative rounds of feedback and discussion, a consensus is within reach. This helps organizations and others make more informed decisions.

2. Technology

Using the Delphi method, experts can provide insight into the potential impact of emerging technologies and innovations. This allows organizations to better plan and adapt for the future.

3. Risk Assessment

The Delphi method can help identify and assess the potential risks of a project, policy, or decision. Experts contribute their insights on various risks and their potential consequences, helping organizations develop risk management strategies.

4. Policy Development

Delphi helps with formulating and evaluating policy. Experts from relevant fields provide input on proposed policies, regulations, or initiatives. This helps ensure smart policies are put together from a wide range of perspectives.

5. Market Research

The Delphi method in market research efforts gathers expert opinions on consumer preferences, market trends, and emerging needs. By consulting marketing gurus, organizations can better understand their customers’ needs and wants and adjust their strategy. Delphi can also help with developing new products that consumers are looking for.

6. Healthcare

Delphi is applicable in healthcare, where expert opinion can help save lives. Surveys may ask medical professionals about medical diagnoses, treatment, and public health initiatives.

7. Product Innovation

When developing new products or technology, the Delphi method helps gather insights from experts about potential features, design considerations, and market viability. This helps ensure that products align with customer needs and industry trends.

8. Education

Delphi can aid in designing or updating educational curricula at any grade level. Experts offer their thoughts on subject matter, learning objectives, and teaching methods, with the goal of giving students well-rounded and effective education programs.

9. Public Opinion Analysis

After surveying the public through traditional surveys or online surveys, researchers analyze the results, looking at social attitudes, behaviors, and more. Next, they suggest ways organizations and officials can communicate with the public better.

10. Environmental Planning

Delphi is great for environmental planning and sustainability initiatives. Experts contribute their insights on environmental issues, potential solutions, and strategies for sustainable development.

Create your Delphi survey or questionnaire now!

Real-life Delphi Method Example

Here’s a look at the Delphi method in action. The Delphi method example uses a method in which a health care company wants to gather insights from a diverse group of medical experts to identify the most likely trends in medicine over the next 10 years.

To apply the Delphi method, the company first identifies medical experts in a variety of fields. These experts come directly from the health care field or from academia.

Round 1: The company designs a survey with open-ended questions, such as “Where do you think we’ll make the most significant medical progress in the next decade?” Experts submit their anonymous responses online.

Feedback and Iteration: The company compiles the responses, extracts key themes, and presents an anonymized summary of the experts’ opinions in a report. In the next round, experts are provided with the summary and are asked to reconsider and potentially revise their initial responses in light of the collective insights.

Round 2: The revised survey includes more questions based on the themes and ideas from the first round. Experts provide their opinions, and the agency again compiles and summarizes the results.

Convergence of Consensus: The agency continues this iterative process for several rounds (about 2 to 4 rounds) until the experts’ opinions start converging towards a consensus on the future trends in medicine. Each round refines the opinions and allows the experts to react to the aggregated insights from the group, with clear criteria for defining consensus.

Final Report: Once a consensus or a “convergence of opinions” is achieved, the company puts together the final report, outlining the identified trends, potential challenges, and key drivers that are likely to influence medicine in the next 10 years. The report shows areas of consensus as well as areas where expert opinions were different.

Published Delphi studies in healthcare often focus on selecting healthcare quality indicators, and systematic reviews and reporting guidelines help standardize the process. Reporting Delphi studies requires adherence to methodological guidelines and clear criteria for defining consensus. For example, the Delphi method is used in brain ultrasonography consensus and critical care group recommendations to develop skill recommendations and standardized practices. Theoretical or methodological guidelines are essential to ensure rigor and reproducibility in these studies.

Create your Delphi survey or questionnaire now!

Conclusion

When seeking expert opinions and wanting the opinions to be generally agreeable, the Delphi method is the best tool for the job. Another great tool for the job? SurveyLegend. Your experts can engage with our online surveys as many times as needed to reach a convergence of opinion. The surveys can be as fun or as professional as you wish, anonymous or not, and they’re highly secure, offering great protection for the often confidential information involved in a Delphi study.

Have you conducted a survey using the Delphi method in the past? Let us know how it went in the comments!