All Articles Education Voice of the Educator 5 questions to ask when evaluating AI edtech tools

5 questions to ask when evaluating AI edtech tools

How teachers can magnify AI edtech tools' power by focusing on their needs and possibilities when selecting them.

7 min read

EducationVoice of the Educator

Artificial Intelligence technology and Chatbot Customer Service of Ai Concept. Futuristic technology transformation

(Vithun Khamsong via Getty Images)

Many have hailed 2023 as generative AI’s breakout year. Doomsday scenarios that were previously the province of researchers, ethicists and technologists went mainstream. The boardroom dramas of AI companies were above-the-fold news. Commentators wondered how the acceleration of AI technologies would impact work, environment, education and a host of other areas. Meanwhile, some of us played around with new AI edtech tools and prompts — by turns delighted, frustrated, excited and scared by what we discovered. 

That was particularly true in education, where teachers, students, staff and administrators alike experimented with AI-powered tools for a range of tasks, including organization, writing, research and form completion. Some of these approaches were productive, while others produced concerning results. Meanwhile, it seems that every edtech product now positions itself as “AI-powered,” seeking to capitalize on AI’s big moment.

Close up stock photograph of a mature man studying a see-through computer monitor that’s displaying text provided by an AI (artificial intelligence) chatbot. for article on AI edtech tools
(Laurence Dutton via Getty Images)

At the same time, many hyped predictions for AI in education never came to fruition: School districts’ initial prohibitions on the use of ChatGPT on school-owned devices gradually gave way to guidance for using AI tools. Regular usage of AI tools by educators and students was more limited than many commentators expected. There was no sizable increase in cheating behaviors among students, even with the wide availability of AI tools. 

As we begin a new year, it’s time to think less about what AI could mean for education and more about how educators can harness AI technologies for education right now. There are a host of possibilities. Some involve direct interaction between educators and students, and others assist with back-end functions such as record-keeping and lesson planning. But whatever the case, we need to focus less on the technology itself and more on how the technology can help empower quality instruction — either by helping teachers in core instructional areas, or in freeing up time for them to concentrate on providing impactful instruction.

AI edtech tools can provide power boosts to instruction

With AI seemingly everywhere, and educators exhausted, overworked and often unclear how to evaluate the usefulness of new tools, it’s helpful to think about AI trends through the lens of “technology-enabled instruction,” an emerging term that we’ve explored. Tech-enabled instruction looks beyond technology integration (whether technology is used in the classroom) to when and how teachers use technology in their instructional practices in ways that research shows improve learning outcomes. 

How can educators and administrators understand the potential value of an AI tool for promoting tech-enabled instruction? Researchers have identified five power boosts that technology in instruction can provide: 

  • Personalization, differentiation and customization to address learner needs.
  • Curation, availability, accommodation and accessibility of vetted educational materials and learning environments.
  • Student engagement, interest and motivation.
  • Communication, collaboration and relationship-building.
  • Learning analytics.

Some AI tools are well-designed to harness these areas of instructional impact, while others are not. Most useful tools will not align to all areas; indeed, some might provide value in a single area or may free up teacher time to concentrate on these areas. Such alignment must be a key consideration when educators and administrators make decisions about when and how to use AI-powered tools. 

What questions should educators ask when evaluating alignment between an AI-powered edtech tool and power boost areas for impactful technology-enabled instructional practice? We break down five critical questions.

1. Does the tool allow teachers to differentiate?

Personalization to improve outcomes in learning predates modern edtech and does not need to involve technology. However, researchers have identified several ways that teachers’ use of technology for differentiation is uniquely well-positioned to advance student learning: scaffolding, multiple ways of assessing learners, opportunities for immediate feedback, opportunities to free up teacher time and more. 

AI tools can make differentiation and customization easier for improved learning, even if significant questions about their effectiveness remain. For example, Khan Academy’s chatbot, Khanmigo, allows students to receive responses that correspond directly to their questions rather than having to sift through reams of material that may or may not be appropriate for them. AI tools may also potentially realize the contextual factors impacting all learners in ways that traditional adaptive systems cannot.

2. Does the tool offer access to vetted educational materials?

The availability of vetted educational materials through edtech platforms frees teachers from the burden of having to compile materials from disparate — and often unvetted — sources. Researchers have shown that this compilation plays an important role in advancing student learning in two critical ways: supporting learner variability and making learning relevant

While curation is possible through nontechnological means, technology is a clear accelerant; it can help learners access materials that might be difficult to locate in print form, and it can allow multiple ways for learners to interact with materials. AI tools have the potential to build upon the power of open educational resource collections by customizing collections for different teaching use cases. For example, ISTE + ASCD’s walled garden chatbot draws only on materials developed by the learning organization. In evaluating any such tools, it will be important for educators to understand how the technology curates, vets and authenticates customized collections.

Polygonal brain shape of an artificial intelligence with various icon of smart city Internet of Things Technology over Asian young Student in casual suit reading the book in library of university
(Tzido via Getty Images)

3. Does the tool help teachers promote student interest? 

Research shows that when students are interested in a topic, they are more likely to perform well academically in that area. Teachers can employ a variety of nontechnological approaches to promoting student interest, but technology often makes that task much easier. For example, when guided by skilled educators, students can use particular databases or the internet to conduct research on topics of their own interest, using sources that would not be available to them without technology. 

So-called intelligent tutoring systems may have the potential to make this process easier, offering guidance to students about how to evaluate sources of interest to them and freeing teachers to foster deeper student interest. Of course, accuracy, bias and ease of use will be important considerations as educators evaluate such tools.

4. Does the tool offer opportunities for relationship-building? 

Strong teacher-student relationships lead to better learning outcomes in the long term, according to a report from Search Institute. Similarly, when educators form strong relationships with families, students are more likely to make positive gains in the long term.

At first, it may seem counterintuitive that technology could help strengthen relationships between people. But imagine a video platform that provides automatic translation services in a meeting between a teacher and a parent about a student’s progress. Or imagine a technological tool that allows students to share how they are feeling, flagging opportunities for follow-up with teachers. 

AI tools can extend the value of such tools even further. For example, AI-powered text messages from tools such as AllHere and Family Engagement Lab can allow educators to more effectively reach and develop relationships with historically underserved families. 

5. Does the tool offer impactful learning analytics? 

The possibility of data generation and data analysis for learning underlies many benefits of using technology for instructional purposes. Of course, technology’s ability to deliver on its promise requires that the data generated by a program are both useful and provided in a timely manner that positions educators to take action. For example, a teacher may analyze data generated by an adaptive learning system to pinpoint areas of student struggle and identify appropriate interventions. But the simple data point of how many minutes a student spent on the program would be much less useful.

AI technologies promise to transform learning analytics — not just what is taught and to whom, but how instruction itself takes place. For example, AI-powered tools such as TeachFX provide teacher feedback at scale, offering opportunities for improved instructional practice. 

What’s Next?

Decisions about which edtech tools to adopt and in what ways are notoriously difficult. AI has expanded the edtech field, making adoption decisions more challenging. Using the lens of technology-enabled instruction and technology’s power boost potential in teaching is not a panacea for this challenge. But asking the five questions above is an important first step in separating the chaff from the wheat in harnessing AI-powered tools’ full potential.


Opinions expressed by SmartBrief contributors are their own. 


Subscribe to SmartBrief’s FREE email newsletter to see the latest hot topics on edtech. It’s among SmartBrief’s more than 250 industry-focused newsletters.