Higher Education is faced with
multiple challenges such as tuition inflation, student retention, campus
climate, and time to degree. Specifically, graduate school, namely Ed.D. and
Ph.D. programs, continue to face challenges with student retention;
approximately 50% of doctoral students leave the program before attaining the
degree (Cassuto, 2013; Lovitts & Nelson, 2000; Reid, 2012) and most
students discontinue the program at the ABD stage (All But Dissertation), a
stage in which coursework is complete without successfully defending the
dissertation (Barnett, 2008). Before reaching the dissertation phase of the
doctoral program, many Ed.D. programs institute an end of the first year
assignment - i.e., students must qualify for the subsequent year by writing a
Qualifying Paper. This stage of the program is important in two ways. First,
this stage signifies a critical junction in which the student demonstrates the
ability to write a coherent literature review. Second, this stage signifies how
well the Qualifying Paper Course prepared and supported the student to move
forward in the doctoral program.
Researchers have begun analyzing
data trails as means to obtain insight into various academic individualization,
prediction, intervention, and adaptation. Analytics has captured meaningful
attention in institutional research and business (MacNeill, 2012), and is a
relatively new area of research in higher education (Barneveld, Arnold, &
Campbell, 2012; Bichsel, 2012). It can also support innovative and meaningful
ways of improving students’ performance and success (MacNeill, 2012). Learning
analytics enables us to generate specific learning processes and images of
students’ performance in a way that the two can be compared to the overall
performance of a course (Gaviria, Glahn, Drachsler, Specht, & Gesa, 2011).
Early investigation on academic analytics predict students’ academic difficulty
in order to help faculty members generate individualized learning instructions
tailored to the student’s learning needs (Campbell, DeBlois, & Oblinger,
2007). Broadly speaking, schools or other programs that use analytics evaluate
data to help make decisions with information that can determine the best course
of action to improve student learning (Hawkins, 2008; Long & Siemens, 2011;
Norris, Baer, & Offerman, 2009), and improve teaching and learning
(Campbell, 2007; Baepler & Murdoch, 2010).
In an effort to gain a different
perspective of writing a Qualifying Paper in the student’s doctoral journey, we
proposed that one class of the Qualifying
Paper Preparation Course be conducted online to see how students engaged in
dialogue about on-becoming a critical consumer of research. We analyzed data
from this online activity to garner insight in information-giving while
individual and collective learning developed among three groups of four to five
doctoral students. We saw data related to students’ early understanding of the
assignment develop in real-time. This digital footprint compiled evidence to
consider creative ways of action in teaching and students’ learning development
in this Qualifying Paper Preparation
Course. We were interested in how online analytics can unveil the beginning
stages of student learning and affect teaching strategies. We used Google Drive
as the platform for the online shared-activity because it is a familiar
technology. We collected data over a three-hour course that took place in the
third week of the quarter. Participants were randomly assigned to three groups
of four or five students. Each group navigated through four modules that
explored different approaches to becoming a critical consumer of research. The
following is a snapshot of the participants’ reflection on their experiences.
The first module asked participants
to think back to assigned course readings and share what it meant to be a
critical consumer of research. Building on this premise, the second module
began with a video that outlined the process of writing a literature review
followed by describing three strategies from the video they would be
incorporating in their upcoming literature review for the Qualifying Paper
milestone in the Ed.D. program. The third module asked if any new knowledge was
gained during this activity and if the participant’s initial definition of a
critical consumer of research has changed. After each module, participants also
wrote a brief reflection. Finally, the fourth module was an overall reflection
and opportunity to share general feedback about their online experience.
At the onset of examining our data,
we identified and used key adjectives linked to individual knowledge such as
learn, explore, understand, find, and realize. We coded instances of collective
learning and interactive conversations that showed collaboration. Responses
that suggested individual or collective learning were closely reviewed for
evidence of learning during the online activity. The first module focused on
defining what it meant to be a critical consumer of research. Group-One had
noticeably less interaction than the other groups. This group consistently
shared resources and information, but gave less insight into where they were in
their learning as a group. There was evidence of some contribution to learning
with words like “additionally” and “also” after acknowledging an agreement.
Group-Two illustrated greater collective understanding of shared learning
experiences and challenges by providing interpretations of each other’s
responses and using words like “agree” and “exactly.” Overall, there was a
great deal of agreement and key words similar to “learning” were used sparingly
in this module. Group-Three showed evidence of interaction and illustrated an
awareness of changing current practice as they explored the definition of critical
research consumption. This group conversation was deeper and showed a balance
of individual and collective learning. For example, one of the exchanges in
Group-Three exemplified a thoughtful interaction that included further
questioning. Student-One described her interpretation of being a critical
consumer of research and then Student-Two responded “I would agree with
everything [name of student] said about critical consumption of research ...I
would also add that just because it [article] does pertain, does not mean it
should be included.” This point was affirmed by Student-One and then she
followed with, “I would think that identifying why it [article] doesn’t fit can
begin to expose the gaps. Is that your thinking?” The exchange continued and
showed a confident display of understanding and deepening of thinking
surrounding on-becoming a critical consumer.
The second module shared a video
synopsis of how to construct a sound literature review. Group-One and Group-Two
shared a level of discomfort with the scholarly process and moving forward with
research. As one student in Group-One stated, “I am still questioning
everything I write in my paper.” Other students agreed, “I have moments of
uncertainty...it is helpful and a bit of validation that this is not an easy
process.” Group-Two demonstrated a similar tone of empathy and appreciation for
one another, but more acceptance toward the general challenges and ambiguities
of the research process. Student-One in Group-Two described “Be comfortable in not
being comfortable right now. I need to trust the process and realize that
things will start to come together and make sense.” Group-Three showed a high
level of interaction as well, and focused more on answering the prompt and
sharing strategies with one another. For example, when one student shared a
chart on how to organize literature, she received praise and appreciation from
her peers.
The third module asked to share any
shifts in thinking from their initial definition of becoming a critical
consumer of research. Overall, Group-One’s definition of a critical consumer
broadened or changed. As one student stated, “My new knowledge has come from
the video and the dialogue between my team members.” Statements like this were
heard repeatedly from Group-One and there was a particular emphasis in gaining
knowledge from the video. Group-Two and Group-Three by-in-large expressed that
their definition of a critical consumer of research did not change. Group-Three
participants stated: “The discussions tonight helped solidify my existing
understanding and allowed me the time to reflect on my own practices as a
critical consumer of research.” Statements like this captured students
experience in Group-Three. While Group-Two’s and Group-Three’s original
definition remained unchanged, Group-Two also shared their preference for
in-person (classroom) learning. This type of reflection did not surface in
Group-Three where insight and comfort were the more prominent themes.
The final module was a reflection on
the entire online activity and an opportunity to provide feedback on the
experience. There was a high level of interaction among all three groups in
this module and some variations on where they focused their reflections.
Group-One delved deeply into practices of online platforms, strengths,
weaknesses, and purposes. Group-Two emphasized their appreciation of being able
to work remotely with constant reference to the value of time. Group-Three did
not have many criticisms of the online platform. They seemed to have navigated
through it with a level of ease not as apparent in Group-One and Group-Two.
Group-Three saw value in this online activity earlier in their research
journey.
Overall, the three groups
demonstrated different characteristics to help us assess their understanding of
and progress with the course assignment. Group-One constantly shared
information and had moments of individual learning, but the collective
interaction and knowledge-gaining attributes were less visible. There was a
sense of community, but threads of dialogue were surficial at times. Group-Two
had a higher level of comfort with their own skill set and showed an
understanding of what it meant to be a critical consumer of research.
Group-Three demonstrated a high level of competency with learning together and,
much like Group-Two, their conversations pushed one another’s thinking.
Group-Three also showed a healthy balance of individual and collective learning
throughout the different modules. All of the groups conveyed instances of
individual and collective learning, and their unique approaches have been
enlightening as we learned about their needs as graduate students and emerging
researchers.
In brief, this online activity was
significant in that it made clear how the students defined their learning
progress related to becoming a critical consumer of research and affirmed and
supported each other’s success and struggles with the task of writing the
Qualifying Paper. Many of the students felt overwhelmed and lost by the task of
writing a 20-25 page Qualifying Paper which also meant undergoing critical
examination of existing research; an intimidating academic undertaking. These
uncertainties were masked before their participation in the online activity and
we believe these uncertainties resembled syndromes of an impostor. The impostor
syndrome is defined and redefined in a few ways since it was first presented in
1978 by psychologists Pauline Rose Clance and Suzanne Imes. The impostor
syndrome includes characteristics of fraud, fear of discovery, and difficulty
of internalizing actual successes (Craddock, Birnbaum, Rodriguez, Cobb, &
Zeeh, 2011; Gravois, 2007). The students who may have felt like they were
“faking their way through the course” no longer felt alone once they became
vulnerable in the online open forum. Upon outing their vulnerability, they
validated one another and provided words of support to help overcome fears or
struggles.
Simultaneously, the online
interaction deepened our understanding of the students’ learning and progress
with the Qualifying Paper. We gained insight in (1) areas in which to spend
class time to review developing a literature review, (2) some of the students
who had unique struggles in developing the literature review, and (3) students
who were doing well with the assignment. Therefore, the online activity served
as an intervention tool to learn students’ progress in the Qualifying Paper Preparation Course. In other words, the evidence
from the online interaction required us to be flexible in and adaptive to the
different levels of learning occurring among the students and respond to their
unique needs.
The differences in individual and
group learning made it clear that students don’t learn the same way and/or at
the same rate, even though they are provided the same information in the same
classroom. Although the online learning analytics were ephemeral, it is evident
that data has provided information to have discussions about ways to be
creative with learning, improve learning and possibly approach the entire
course differently for subsequent cohorts.
References
Baepler,
P., & Murdoch, C. J. (2010). Academic analytics and data mining in higher
education. International Journal for the
Scholarship of Teaching and Learning, 4(2), Article 17. DOI= http://digitalcommons.georgiasouthern.edu/ij-sotl/vol4/iss2/17.
Barneveld,
A. V., Arnold, K. L., & Campbell, J. P. (2012). Analytics in Higher
Education: DOI= http://net.educause.edu/ir/library/pdf/MWR07085.pdf.
Barnett, D. L. (2008).
Experiences influencing
degree completion articulated by doctoral students in education administration (Doctoral dissertation). Retrieved from http://ezproxy.csusm.edu/
Bichsel,
J. (2012). Analytics in Higher Education:
Benefits, Barriers, Progress, and Recommendations. Research Report.
Louisville, CO: EDUCAUSE Center for Applied Research. DOI= http://www.educause.edu/ecar.
Campbell,
J. P. (2007). The Grand Challenge: Using
Analytics to Predict Student Success. Presentation at the 2007 EDUCAUSE
Midwest Regional Conference (Oct. 2007). DOI=
http://net.educause.edu/ir/library/pdf/MWR07085.pdf.
Campbell,
J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new
tool for a new era. EDUCAUSE Review (July/August 2007), 41–57. DOI=
http://net.educause.edu/ir/library/pdf/erm0742.pdf.
Cassuto,
L. (2013). Ph.D. Attrition: How Much Is Too Much? Chronicle of Higher
Education. DOI= http://chronicle.com/article/PhD-Attrition-How-Much-Is/140045.
Craddock,
S., Birnbaum, M., Rodriguez, K., & Zeeh, S. (2011). Ph.D. Doctoral students
and the impostor phenomenon: Am I smart enough to be here? Journal of Student Affairs Research & Practice. 48, 4 (2011),
429-442.
Gaviria,
F., Glahn , C., Drachsler, H. Specht, M., & Gesa, R. F. (2011).
Activity-based learner-models for learner monitoring and recommendations in
Moodle. In C. Delgado Kloos, D. Gillet, R. M. Crespo García, F. Wild, and M.
Wolpers, M. (Eds.) Proceedings of the 6th
European Conference on Technology-Enhanced Learning (EC-TEL 2011, Palermo,
Italy, September 20-23, 2011). Heidelberg, Berlin: Springer-Verlag.
Gravois,
J. (2007). You’re not fooling anyone. Chronicle
of Higher Education. 54, 11, A1.
MacNeill,
S. (2012). What is Changing and Why Does
it Matter? A Briefing Paper. CETIS Analytics Series. 1, 1. DOI=
http://publications.cetis.org.uk/wp-content/uploads/2012/11/Analytics-Vol1-No1-Briefing-Paper-online.pdf.
Long,
P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and
education. EDUCAUSE review, 46(5),
30-40
Lovitts,
B. E., & Nelson, C. (2000). The hidden crisis in graduate school. Attrition
from Ph.D. programs. Academe, 86, 6, 44-50.
Norris,
D., Baer L., & Offerman, M. (2009). A National Agenda for Action Analytics.
White Paper. Presented at the National Symposium on Action Analytics. (St.
Paul, Minnesota, September 21-23, 2009). DOI= http://lindabaer.efoliomn.com/uploads/settinganationalagendaforactionanalytics101509.pdf
Reid,
C. D. (2012). Joyful obligation:
Listening to Black doctoral students in the academy. Doctoral Thesis. UMI
Order Number: UMI Order No. 3493868., New York University.
