The State of Critical Thinking:

EXECUTIVE SUMMARY

Critical thinking has always been an asset. But in today’s increasingly digital and globalized world, robust forms of thinking have become a necessity. 

With more and more information at our fingertips, we have to be far more discerning about our choices and judgments. Just consider that in October 2018 alone, Twitter took down dozens of accounts that falsely posed as lawmakers.(1)

But despite the need for more critical thinking, our institutions have not done nearly enough to give students richer thinking tools. In too many schools, critical thinking is not taught to young people. At workplaces, employers don’t do enough to prioritize richer forms of reasoning.

This happens despite the fact that strong reasoning skills have become increasingly key to navigating everyday life, and a growing body of research shows that thinking critically runs in lockstep with life outcomes. Researcher Heather Butler recently conducted a study that found “critical thinkers experience fewer bad things in life.”(2)

According to Butler, good critical thinkers are far less likely to foreclose on a home or carry large credit card balances, while those without strong critical thinking skills are more likely to have an extramarital affair and drink while driving.(3)

What’s more, there’s plenty of evidence that our democracy is fraying because of a lack of reflective thought. Politicians around the world are taking advantage of new technologies to push a political agenda that divides nations instead of uniting them, and there have been sharp upticks in reports of everything from racism to fascism.(4)* 

The Reboot Foundation is dedicated to promoting richer forms of thought and to better understanding the state of critical thinking today. With that in mind, the foundation recently commissioned a survey, which will be conducted each year in an attempt to better understand shifts in the public’s views on critical thinking and what it means for the future of society.

The Foundation surveyed more than 1,000 people using an online platform, and we weighted the results along demographic lines.

Our data uncovers a number of key findings.

While the public believes that critical thinking is crucial, most people believe that schools do not do enough to prepare young people to think more effectively. Across just about every demographic variable, people support more critical thinking, and nearly all respondents (95 percent) say critical thinking skills are necessary in today’s world.

Still, people worry that our schools do not teach robust forms of thinking, and about 80 percent of respondents say that young people lack the ability to engage in critical thinking. Only 29 percent of respondents say that they definitively studied critical thinking in school themselves.

There’s a lack of clarity about when, where, and even how critical thinking should be taught. About 48 percent of parents surveyed say that they (the parents) should be responsible for teaching critical thinking. Another 41 percent believe that educators should be responsible for teaching young people about how to think critically. And still another 22 percent believe that children themselves should be responsible.

While it’s encouraging that many feel critical thinking is a shared responsibility, this lack of consensus helps explain why people often don’t acquire better thinking skills: the teaching of the skill seems to simply fall through societal cracks.

While parents say that they know how to teach their kids critical thinking, they don’t generally practice these skills with them. In our survey, 72 percent of parents say that they know how to help their kids gain critical thinking skills, and 96 percent say that critical thinking is an important skill to teach to their children. 

But upon closer examination, we found that, on the whole, parents often fall short of teaching their children basic critical thinking skills. For instance, only 20 percent of parents frequently or daily ask their children to take an opposing view. Only a third of parents have their children regularly discuss issues without a right or wrong answer. 

The Reboot Foundation
Founded by Helen Lee Bouygues, The Reboot Foundation supports efforts to better integrate critical thinking into the daily lives of people around the world. It conducts surveys and opinion polls, leads its own research, and supports the work of university-affiliated scholars.

The Foundation also develops practical tools for parents, teachers, employers and others interested in cultivating a capacity for critical thinking.

Members of the public say they practice critical thinking, but their behaviors often suggest otherwise.The vast majority of respondents report that they have solid critical thinking skills, and about 67 percent of respondents say their reasoning skills have improved over time.

But it seems that there’s a reality gap, and people are simply overstating their reasoning skills. Many respondents report engaging in practices that don’t show much critical thinking. For instance, we found that 47 percent of them don’t typically plan where they will obtain information while doing research. And around 27 percent use only one source of information while making a decision.


The lack of critical thinking skills is particularly apparent online. For example, we found that over one-third of respondents consider Wikipedia, a crowd-sourced website, to be the equivalent of a thoroughly vetted encyclopedia. What’s more, people rely on Wikipedia almost as much as they rely on government websites for factual research, according to our study.

Many do not do enough to question the accuracy of social media. People believe the accuracy of more than a third of what they read on Twitter and Facebook, for instance. Respondents are also far more likely to engage with informal, non-vetted sources for information, and just under 40 percent say they regularly read blogs instead of institutional publications like newspapers.

The public says they engage opposing views, but they rarely do. Nearly 87 percent of respondents say that considering an opposing view is an important and useful exercise.

But few engage in the practice, and less than a quarter of respondents actually seek out views that challenge their own. For instance, 24 percent of respondents say they avoid people with opposing views. Another 25 percent rarely or never seek out people who have different views than theirs.

In other words, many people claim they solicit the views of others. But, in practice, they don’t do nearly enough to “stress test” their opinions, despite the wealth of evidence showing that engaging in opposing views is crucial to richer forms of critical thinking.(5)*

What is critical thinking?
We define critical thinking broadly, and we believe it is a type of reflective throught that requires reasoning, logic and analysis to make choices and understand problems. Key elements of critical thinking include seeking out opposing viewpoints, using evidence, and engaging in debate.

INTRODUCTION AND BACKGROUND

Critical thinking is not new. Nor are claims about its importance. The philosopher Socrates is credited with saying, “The unexamined life is not worth living.” For Socrates and many other ancient philosophers, reflective thinking was the ultimate human pursuit, the most important endeavor of any meaningful life.

In some ways, things have not changed at all since the time of Socrates. The unexamined life is still not worth living. But at the same time, critical thinking has become far more consequential —  and far more urgent. Today, reasoning is at the center of a 21st century society, the engine of the modern world.

Technology is driving much of the need for deeper critical thinking skills. It is the primary force behind our changing economy, in which richer forms of reasoning have become some of the best predictors of economic success. Technology is also driving shifts in our social and political worlds, from the debate over alleged “fake news” to the algorithms that track our every move online.

While the Internet has provided many benefits, it has made it harder to figure out fact from fiction. In more traditional forms of media, such as newspapers, there have long been clear demarcations that separate opinion pieces from reported articles. Online, however thoroughly-reported news items, op-eds, and totally unverified information are often promoted in similar ways without much distinction among them.

Social media makes this problem far worse. It is now fairly easy to push out maliciously false information online, and many sites and bots aim to spread information with questionable sources. Recently, Facebook removed almost 600 pages that continually posted misleading information.(6) One of those pages had more than 100,000 followers.(7)

Social media also pushes people to live in an echo chamber. According to Harvard University law professor Cass Sunstein, sites like Twitter and Facebook encourage people to engage only with claims that align with their own views, fostering a type of societal myopia. “I wouldn’t say that we are now more isolated from diversity; there’s a lot of diversity out there, in terms of how isolated people are from diversity,” Sunstein once explained. “But many people do like to isolate themselves, and that’s a big problem.”

At the same time, technology has eroded critical thinking. Our devices are making us less able to reflect and rationalize. Patricia Greenfield, a psychology professor and director of the University of California, Los Angeles Children’s Media Center, has found that as our reliance on technology has grown, our critical thinking skills have declined. We read less and consume more visual media, which does not allow for the analysis and reflection required of critical thinking.(9)

As if that weren’t enough, the democratization of the media in recent decades has put more and more responsibility on individuals to ferret out truth from fiction. While institutions have taken some steps to limit falsehoods, individuals increasingly must take steps to avoid becoming prey to dishonest

In this sense, the recent crisis over so-called “fake news” is really a crisis of our own making. Jim VandeHei, co-founder and CEO of the news site Axios, recently wrote, for example, that “each of us is very much to blame” for the phenomenon of fake news. He implored news consumers to think critically online. “Quit sharing stories without even reading them. Spend a few minutes to verify the trustworthiness of what you read,” he wrote.(10)

But, too often, people aren’t provided enough training in robust critical thinking to be able to do that. Our schools, in particular, fall short of empowering students with better reasoning skills. This is particularly evident online. One recent Stanford University study revealed that 93 percent of college students did not know that a lobbyist website was one-sided. Fewer than 20 percent of high-schoolers were aware that just one online photo does not prove something took place.(11)*

A large chunk of the public is also unskilled in using social media, often passing along “information” they’ve found online without doing their homework — that is, checking the original sources. One recent study, conducted by Columbia University, revealed that close to 60 percent of people share news-related pieces on Twitter that they have not clicked on to read at length. In other words, the headline alone was enough to confirm its legitimacy, then pass it along.(12)

Problems of critical thinking are not new, of course. Long before social media, philosophers argued for better ways to challenge the unjustifiably self-assured. The most notable is the Socratic method, a still-popular instructional technique. A recent summary of the method makes its application still highly relevant: “We can consider alternative interpretations of the data and information. We can analyze key concepts and ideas. We can question assumptions being made.”(13)

Ancient philosophers, then, offer both a warning and a solution. More exactly, they remind us that we need to do more to question our assumptions and to consider alternative interpretations. Data must be more at the center of our reasoning, and no doubt, the stakes are higher than ever. To inelegantly paraphrase Socrates, an unexamined society will not survive.

METHODOLOGY AND DESIGN

As part of our research, we surveyed more than 1,100 adults using Amazon’s Mechanical Turk (MTurk) platform from September 19, 2018, to September 25, 2018. A crowdsourcing tool, Mechanical Turk has increasingly been used for surveys and other experiments, and generally researchers praise the use of the platform. “Mturk is a fast and cost-effective way to collect nonprobability samples that are more diverse than those typically used by psychologists,” noted one recent research paper.(14)*

We used Mturk because of its speed and convenience. The platform also allowed us to include some items on the survey that were experimental in nature, like the “heat map” question related to search results.

Mturk-based surveys have limitations, to be sure. Like many online surveys, they provide convenience samples, and people using the Mturk site are younger and whiter than the population at large.(15)

To make our findings generalizable, we weighed our sample data with survey weights generated by doing iterative post-stratification on our data so that the marginal sample distributions on gender, income, and age match the corresponding marginal population distributions as reported by the American Community Survey for the year 2017.(16)

For the survey questions regarding critical thinking in daily life, we relied on items from the Youth Life Skills Survey. We first uncovered the series of survey items in “A Study of Critical Thinking Skills in International Baccalaureate Middle Years Programme.”(17 ) The items were developed by Claudia Mincemoyer, Daniel Perkins, and Catherine Munyua of Penn State.(18)

The data on parents came from a subsection of the survey that only asked questions of adults who have children. To examine demographic data, we conducted crosstabs across age, income, and gender. 

Two experts in survey design and implementation provided technical advice. They are Joe McFall of The State University of New York, Fredonia and Srikant Vadali of St. Anselm College. They are not responsible for any of the interpretations of the data contained in this document

For the full data results, a copy of the survey instrument or any other survey-related questions, please email Reboot Foundation advisor Ulrich Boser. He can be reached at ulrich@reboot-foundation.org.

MAJOR FINDING

The public thinks critical thinking is crucial but believes young people lack such skills. In the study, nearly all respondents (more than 95 percent) say critical thinking skills are necessary in today’s world, and nearly the same percentage believe we should think more critically in our everyday lives. This opinion crossed demographic lines – men and women, rich and poor, old and young. They all agreed that critical thinking is important, and we should do more of it

But respondents are deeply concerned that schools do not teach critical thinking. Only half of survey respondents say their experience in school gave them strong critical thinking skills. Men are 8 percentage points more likely than women to believe that their schools gave them strong critical thinking skills (50 percent for men vs. 42 percent for women).

In addition, more than 80 percent of respondents believe that critical thinking skills are lacking in today’s youth, and in the survey, people point to a range of reasons for the lack of critical thinking. Some 27 percent of respondents believe that modern technology inhibits critical thinking; interestingly, women are 12 percent more likely than men to think modern technology is at fault. Another 30 percent believe that society devalues critical thinking skills.

Notably, 26 percent of respondents say that critical thinking skills are lacking because of a flawed educational system. Young people are more likely to feel this way than those in older demographics, and in the 18-to-40-year-old category, 41 percent of respondents think schools are to blame. In contrast, just 28 percent of people in the 61-to-81-year-old group believe that schools are culpable.

Not surprisingly, older respondents are more likely to blame technology for a lack of critical thinking. Those in the 18-to-40 age range are less critical of modern tools, with only 21 percent saying they are the cause of poor thinking. In contrast, 33 percent of 41-to-60-year-olds blame modern technology on today’s lack of critical thinking skills. There’s a gender gap as well, and women are 12 percent more likely than men to think modern technology is at fault for the crisis in critical thinking.

Whatever the demographic differences, though, these findings suggest that there is a growing awareness that the modern world has deeply complicated critical thinking. Across lines of age, gender, and income, people believe that critical thinking is more important than ever. This is good news. After all, when people are aware of a problem, they’re more willing to address it.

There’s a lack of clarity about when and where critical thinking should be taught. Despite the public’s enthusiasm for critical thinking skills, respondents are split over what age is appropriate for developing such skills.

In our survey, 20 percent say critical thinking skills develop best in early childhood, or ages 5 and under. Another 35 percent say critical thinking is best developed during ages 6 to 12, and another 27 percent think ages 13 to 18 are best. About 13 percent say any age is good for developing critical thinking skills.

There are differences along demographics lines. Women are more likely than men to favor teaching critical thinking skills during the early years. For example,  24 percent of our survey’s female respondents believe in teaching critical thinking skills to children 5 and under, whereas just 17 percent of male respondents  share that belief.

There are also differences among income groups. Higher-income respondents are more likely to believe that parents should teach critical thinking during the early years. For instance, 29 percent of people in the $100,000-and-above category believe that critical thinking should be taught to children younger than 6 years of age. But just over 15 percent of those making less than $50,000 per year think that critical thinking should be taught to children younger than 6 years of age.

There is also a lack of clarity about who should be responsible for teaching critical thinking. About 74 percent of the parents surveyed say educators should be at least partially responsible for teaching young people how to think critically. Another 89 percent say they — the parents — should be responsible.

Perhaps most surprising, 22 percent of respondents believe that children themselves should be responsible for learning how to think critically. The respondents believed this idea,  despite the fact that most experts argue that parents, educators, and others can help improve critical thinking among young people.
(19)

When it comes to teaching critical thinking, the public believes that schools should play an important role. About 92 percent of respondents say that K-12 schools should require courses that develop those skills. Another 90 percent of respondents think critical thinking courses should be required in colleges and universities. 

While it is encouraging that many Americans believe that critical thinking is a shared responsibility, the lack of consensus over what inhibits it as well as how and when to teach it helps explain why people often don’t acquire better thinking skills. It is a problem of too many cooks in the critical thinking kitchen: with everyone in charge, no one is in charge.

Parents also do not typically help their children develop other important critical thinking skills. For instance, only a third of parents have their children regularly discuss issues without a right or wrong answer, despite evidence supporting the practice.(20) What’s more, only 26 percent of parents frequently help their children evaluate evidence, which is a key skill when it comes to better reasoning.

When it comes to parents and critical thinking, there are important differences along gender lines. For instance, women report doing more critical thinking skill development with their children than men do. For instance, women are about 6 percentage points more likely than men to report that they help children evaluate evidence and arguments every day (12 percent for women, 4 percent for men).

This gender split can likely be attributed to the fact that, historically, women have been the primary caregivers of children and are, on average, at home more often. While there is room for improvement for all parents in teaching critical thinking skills, it seems that male parents in particular have the most ground to make up.

While a majority of respondents say that their critical thinking skills have improved over the years, they often don’t engage in robust critical thinking practices. When it comes to critical thinking, there’s a large gap between what people believe and how they behave.   

For instance, 67 percent of our survey respondents say they have improved their reasoning skills since graduating high school. But many respondents also report making use of specific practices that reveal weak critical thinking.

We discovered, for example, that almost 50 percent of people do not typically plan where they will obtain information before engaging in research. Our survey also reveals that around one-third of respondents will use only one source of information when making a decision. 

Again, demographics make a difference. Older people, for instance, are more likely to use more than one source of information before making a decision. Case in point: people older than 60 are 19 percentage points more likely to always use more than one source than people younger than 40 (51 percent for the older group vs. 32 for the younger group).

The lack of highly developed critical thinking skills is particularly apparent when people are online. For example, we found that over one-third of people consider Wikipedia, a crowd-sourced website, to be the equivalent of a thoroughly vetted encyclopedia.

Income and age have a bearing on the perception of Wikipedia as a modern day encyclopedia. Fifty percent of respondents making $50,000 or less annually say that Wikipedia is a modern version of an encyclopedia. In contrast, just 16 percent of people making $100,000 or more share that belief.

Similarly, 48 percent of those 18 to 40 years old think that Wikipedia is a modern day encyclopedia. In contrast, just 25 percent of those in the 41-to-60-year age range think the technology as a robust as an encyclopedia.

Social media practices also suggest a lack of critical thinking. For instance, we found that more than 40 percent of people’s online reading is made up of blogs and other informal news sources. The other 60 percent consists of material from institutional sources, like a newspaper or traditional media outlet.

Not too astonishing, most younger people are more likely to read blogs. Respondents 18 to 40 years of age, for example, report that about 41 percent of what they read online tend to be blog items, whereas people in the 61-to-81-year range report their blog intake at an average of 11 percentage points less.

Our results also showed that people simply don’t look at enough sources while doing online research. According to our survey, only 33 percent of respondents examine more than 5 results during an Internet search. This means that two-thirds of people rely on very limited number of sources while doing online research. 

How Young Is Too Young?
At What Age Can Children Start to Engage in Critical Thinking?


At first glance it may appear that young children do not have the capacity to think critically. After all, most 3-year-olds struggle to even tie their shoes.

But there’s growing evidence showing that very young children have rich thinking skills. One study released this year found that preschoolers can engage in causal reasoning.(21)* Research also shows that children as young as 3 start to realize that some beliefs don’t necessarily jibe with reality.(22)*

Another study found that between the ages of 3 and 5, children begin to understand that what another person says is not necessarily “true” but is often more a reflection of his or her beliefs. For instance, most young children know that a statement like “the best dessert in the whole world is ice cream” is a belief, not a fact.

Recent evidence suggests that different teaching methods can promote more critical thinking in young children, especially when the strategies take advantage of changes in brain development. For instance, prior to age 10, a child’s emotional intelligence takes precedence over the intellectual.

So teachers and parents should allow a child to explain how she came to a conclusion without insisting she use facts to support it. This helps build self-esteem and teaches the child, at an early age, that no one person, agency or institution holds the key to the “truth,” according to researchers like Sebastian Dieguez at the University of Fribourg.

During later stages, ranging from pre-adolescence to the mid-teens, teaching critical thinking is a bit trickier. Children’s brains are constantly in flux, both physically and in the ways they receive information: in class, from friends, and on social media. At these stages, research says that it is important to equip children with the skills necessary to navigate this constant, often muddied, river of information. 


Giving young people effective thinking strategies can help. For example, one useful metaphor is telling children that possessing knowledge is like being in the driver’s seat of your own car. You, no one else, is in control. Research has also shown that giving young people thorough instruction in better thinking can yield very positive results; it makes for better students and higher grades.(23)*

While the public claims that they engage opposing views, they don’t actually engage other views in practice. Nearly 87 percent of respondents say that considering an opposing view is an important and useful exercise.

This is an encouraging finding, given the large body of evidence that shows that considering opposing views improves problem-solving. For instance, Scott Page at the University of Michigan has studied diversity of opinion and concluded that exposure to others’ perspectives leads to better outcomes. In fact, he found that diversity is more important than ability when it comes to problem-solving.(24)*

But when asked to detail how they engage in such practices in their daily lives, only 25 percent are willing to regularly have debates with people who disagree with them. A surprising 24 percent of respondents say they regularly avoid talking to people with opposing views. In other words, people might say that they want to engage other views in theory, but they rarely do so in practice.

Research helps explain this gap. Decades’ worth of studies show that people prefer to socialize with those who have similar backgrounds and beliefs. The scientific term is “homophily,” or, as one study puts it, the principle that “contact between similar people occurs at a higher rate than among dissimilar people.”(25) 

Because of these homophilic tendencies, many people are uncomfortable engaging with individuals whose views differ significantly from their own. They live in something of a bubble, where they continually reinforce their own beliefs, including incorrect information about people unlike themselves, without being challenged.

The results of homophily are clear in our politics. One recent study found that half of the Republicans and Democrats surveyed found talking politics with their rivals “stressful and frustrating.”26 And even more (65 percent Republican, 63 percent Democratic) said that, when speaking with their counterparts, they discover they have less in common politically than previously thought.

In our study, men in particular are unwilling to engage in critical discussions. They are roughly 20 percentage points more likely than women to avoid people with whom they disagree (33 percent vs. 13 percent). Along income lines, the difference is comparable: respondents in the lowest income bracket are at least 20 percentage points more likely than those in the highest income bracket to do the same (66 percent vs. 54 percent).

In the end, our data suggests the public overestimates its willingness to engage views that are different than its own, a crucial part of being a good critical thinker. Without these critical thinking skills, we risk becoming bad choosers. When we don’t consider the available evidence, when we settle for what is ideologically comfortable, we make incomplete decisions and we risk polarization. 

Where do you click? 
An experimental approach to measuring critical thinking online.

As part of our research into critical thinking, we relied on a more experimental approach to measuring how people engage with online sources, and we created a simulation of a real-life scenario to see what links people might click on while doing online research. 

Specifically, we asked respondents: “Imagine you are helping a child with a school research project about the U.S. Capitol. You have just conducted an online search through a search provider. Where would you click next?” 

We used technology to measure people’s clicks similar to a “heat map,” and as shown in the image below, we found that people are almost just as likely to click on Wikipedia as they are to click on the government’s actual website. 

On the positive side, respondents avoided the Capitol’s Twitter social media handle, which appears to provide the least relevant as well as least accurate set of results. (Note that the color red in the image below indicates more clicks. Green indicates fewer clicks.)

CONCLUSION

But without robust approaches to thinking, we risk deepening our own biases. We risk becoming susceptible to “fake news,” conspiracy theories and phishing scams. We risk increasing polarization, partisanship and infighting among the biggest challenges we face as a nation.

Looking forward, we must become better teachers, and better students, of critical thinking in order to fortify ourselves against such risks. It begins with encouraging our schools and teachers to better incorporate critical thinking exercises into curricula, at the primary, secondary and college levels. Parents, too, must support this effort by talking children through complex problems and encouraging them to be critical in their thought processes. We must also diversify our workplaces and social circles, and create spaces for thoughtful and robust discussions. Doing so will lead us to bigger breakthroughs and help spur greater inventions. Thinking critically will not only help us raise well-rounded children and break our own biases, but also grow our economy and strengthen our democracy. In the end, critical thinking is perhaps the most valuable tool in building a better nation, and as a society, we need to do far more to cultivate better forms of reasoning for our children and our future. 
Upcoming Products

Over the coming year, the Reboot Foundation plans to release a number of studies, papers, as well as some hands-on tools to improve critical thinking.

(1)* Sheera Frenkel, “Facebook Tackles Rising Threat: Americans Aping Russian Schemes to Deceive,”
New York Times, October 11, 2018.

(2)* Heather A. Butler, Christopher Pentoney., Mabelle P. Bong, “Why Do Smart People Do Foolish Things?” Scientific American, Springer Nature America, Inc., October 3, 2017,
https://www.scientificamerican.com/article/why-do-smart-people-do-foolish-things/

(3)* Heather Butler, “Predicting real-world outcomes: Critical thinking ability is a better predictor of life decisions than intelligence,” ScienceDirect, Thinking Skills and Creativity. Volume 25, September 2017, https://www.sciencedirect.com.

(4)* Yuva Noah Harari, “Why Technology Favors Tyranny,” The Atlantic, Oct. 2018,
https://www.theatlantic.com/magazine/archive/2018/10/yuval-noah-harari-technology-tyranny/568330/

(5)* .Lu Hong and Scott E. Page “Groups of diverse problem-solvers can outperform groups of high-ability problem-solvers,” PNAS, 101, 46 (2004): 16385–16389, https://sites.lsa.umich.edu.


(6)* .Sheera Frenkel, “Facebook Tackles Rising Threat: Americans Aping Russian Schemes to Deceive,”
New York Times, October 11, 2019

(8)* S Cass Sunstein, “Danger in the Internet Echo Chamber,” Harvard Law Today, March 24, 2017,
https://today.law.harvard.edu.

(9)* PM Greenfield “Technology and informal education: what is taught, what is learned,” Science, 323 (5910), (2009): 69-71, https://www.ncbi.nlm.nih.gov.

(10)* Jim VandeHei, “4 ways to fix ‘fake news,” Axios Media Inc., October 21 2018, https://www.axios.com.

(11)* Wineburg, Sam and McGrew, Sarah and Breakstone, Joel and Ortega, Teresa. (2016). “Evaluating Information: The Cornerstone of Civic Online Reasoning. Stanford Digital Repository,” 
http://purl.stanford.edu/fv751yt5934


(12)* Maksym Gabielkov et al., “Social Clicks: What and Who Gets Read on Twitter?” ACM SIGMETRICS / IFIP Performance 2016, (2016), Antibes Juan-les-Pins, France, (2016), https://hal.inria.fr
https://hal.inria.fr/hal-01281190

(13)* Richard Paul and Linda Elder, “Socratic Thinking,” The Foundation for Critical Thinking, (1997), 
http://www.criticalthinking.org.

(14)* Jesse Chandler and Danielle Shapiro “Conducting Clinical Research Using Crowdsourced Convenience Samples,” Annual Review of Clinical Psychology, 12, (2016): 53-81.  
https://www.annualreviews.org.

(15)* Ibid., 53-81.

(16)* United States Census Bureau, Surveys and Programs, “American Community Survey (ACS),” United States Census Bureau, https://www.census.gov.

(17)* Julie Wade, Natalie Wolanin, and Trisha McGaughey, “A Study of Critical Thinking Skills in the International Baccalaureate Middle Years Programme,” International Baccalaureate, (2015),
https://www.ibo.org.

(18)* Human Service Research, “Youthful Life Skills Evaluation,” Human Service Research Inc., 
http://www.humanserviceresearch.com.

(19)* Abrami, Philip C., Robert M. Bernard, Evgueni Borokhovski, Anne Wade, Michael A. Surkes, Rana Tamim, and Dai Zhang. “Instructional Interventions Affecting Critical Thinking Skills and Dispositions: A Stage 1 Meta-Analysis.” Review of Educational Research 78, no. 4 (December 2008): 1102–34. doi:10.3102/0034654308326084.1102–34. doi:10.3102/0034654308326084. doi:10.3102/0034654308326084.


(20)* Schommer, Marlene. (1990). Effects of Beliefs About the Nature of Knowledge on Comprehension. Journal of Educational Psychology. 82. 498-. 10.1037/0022-0663.82.3.498.

(21)* Mariel K. Goddu & Alison Gopnik, “Young Children rationally use evidence to select causally relevant variables for intervention”, (University of California, Berkeley, 2018).

(22)* Kuhn, Deanna. “A Developmental Model of Critical Thinking.” Educational Researcher 28, no. 2 (1999): 16-46. http://www.jstor.org/stable/1177186.

(23)* John Perry, David Lundie & Gill Golder “Metacognition in schools: what does the literature suggest about the effectiveness of teaching metacognition in schools?” Educational Review, (2018), DOI: 10.1080/00131911.2018.1441127.

(24)* Lu Hong and Scott E. Page “Groups of diverse problem-solvers can outperform groups of high-ability problem-solvers,” PNAS, 101, 46 (2004): 16385–16389, https://sites.lsa.umich.edu.

(25)* Miller McPherson, Lynn Smith-Lovin, and James M Cook, “Birds of a Feather: Homophily in Social Networks,” Annual Reviews of Sociology, 27 (2001): 415–44, http://aris.ss.uci.edu.