Just like any other kind of design, the process of designing a research project is creative, even artistic. And like other kinds of design, a good one works well and a bad one will just make a mess. Here are the steps to designing a good research project:
Step 1. Articulating Different Points of View about an Issue
The success of a research design project depends on knowing and understanding different points of view on the issue you plan to research. It is important to discover what evaluative criteria are important to you as well as understand how others may have different criteria by which to understand this issue. Your own issue criteria will be very important to your research design, but it will also be important to understand how people perceive this issue according to a different set of criteria. In your paper, you will need to explicitly state and articulate the different perspectives by which people understand your issue. You will begin your paper stating your research question and describing the research design you plan to use to gather data to answer your question. However, you will not write this introductory section yet. Instead, you need to gain an Area of Expertise and focus on the background of an issue. Then, in your literature review you will discuss the other ways people have looked at your issue according to the criteria they considered important.
Therefore, the first step in the research design process is to understand where this issue comes from (the "background"), see what people have said already about the general issue you want to research and to understand the criteria which they consider important in this conversation. Lab #1 will help you listen to and understand other people's perspectives and the criteria by which they perceive an issue. In some cases, what others have said may influence your own priority criteria (you might find their work convincing, in other words). In other cases, the work of others on this topic might anger you and focus you on arguing against their point of view (a little rage can be very productive at this point). Either way, you cannot ignore what others have to say. You need to discuss all points of view in the overall academic “conversation” on this topic.
The first rule of a literature review: the research for the data sources you use should also be well designed. That is, you should ascertain whether or not the source you are citing followed the design steps laid out here. Papers that are peer reviewed are more likely to be well designed. Papers coming from policy organizations with agendas may not be open about articulating those agendas, but they will provide you with a strong idea of how people argue from a particular point of view. Therefore, you should not shun advocacy pieces; they will be very good at viewing a particular issue according to a particular perspective. However, peer reviewed pieces should give you a better idea how to evaluate advocacy pieces. Major policy organizations depend on maintaining a reputation and may be more transparent in terms of their criteria. However, many papers that call themselves research papers are in fact advocacy pieces. It is good practice to learn how to write an advocacy piece where you clearly articulate and defend one opinion, and you may be asked to write such pieces based on your research. However, that is not the goal of this course.
Step 2. Defining your research question
After your criteria are set, and you have looked at the conversation surrounding your topic, you need to clearly define a research question. This question may change during the research process, but efficient and focused research requires that you always have a research question clearly in mind as you look for information and evidence. Your research question should be clearly linked to the research criteria you chose in Step 1.
The literature review and research question process is often iterative: once you formulate your research project, you may find that you may need to go back and read more of what others said, refine your criteria again, and re-state your research question in a somewhat different way. This will also mean that you may have to rework Step 3 as well.
Once you start gathering evidence to answer your research question, you are likely to get into the boundary problem. A good researcher is efficient and focused on gathering the information necessary to answer a specific research question. This involves making constant decisions about the boundaries of the issue being researched, asking, “Do I need to expand my gathering of data in order to answer my question, or is my field of data-gathering sufficient?” Or, “Am I trying to gather too much information to answer a focused question? Will I get lost in my research?”
Step 4. Designing your measurement strategy
Next you need to design your measurement strategy. Measurement design is like any other technological design, except that you are creating a “machine” -- a set of logical steps -- to collect data. Like any other technology, if you design it well, it will be efficient and will create a good product. Well-designed measurement machines are based on clearly laid out criteria, an understanding of the overall conversation about the topic in the literature, and a clear and focused research question. Poorly designed machines break down or produce something you can’t use to answer your research question.
Like any other kind of technological design, your how you go about measuring the world will have a lot to do with what criteria you have chosen to focus on (Step 1) and the type of research question you are asking (Step 2).
To design a good measurement strategy, you will need some tools. In particular, you need to understand how to use the concept of validity as a tool to make your measurement design work well.
Step 5: Carrying out your research and analyzing the results
If you used Steps 1-4 to design your research, this part should be straightforward. However, most good research leads to at least one Surprise – that is, results that you didn’t expect. An unexpected result is not the product of a bad research design. In fact, research that is designed to get expected results is usually not well-designed research. The best research designs always give you more knowledge than you know what to do with, and more than your criteria goals or research strategy set out to do.
[How you go about communicating your results in the design of your research paper is another kind of art, requiring another set of skills, which we will discuss elsewhere.]
Step 6. Understanding trade-offs of alternative designs
Research designs never satisfy all of the identified requirements necessary to answer a research question. In your final conclusion, you need to identify the limitations and comparative trade-offs associated with the research design you chose.
[There are sub-variations on research projects related to policy. If you are researching a policy question (Policy Analysis), this is where you introduce your policy proposal or policy reform. You also, at this point, discuss the tradeoffs involved in choosing this policy option over another. If you are evaluating a program (Program Evaluation) then you are asking whether the program works, looking at various criteria that defines "works" and creating a measurement machine that collects data according to those criteria.]
