ÌÇÐĶ¯Âþvlog

Get stories like this delivered straight to your inbox. Sign up for The 74 Newsletter

The end of the fiscal year is near. For many school administrators, that means scrambling to decide whether to spend more money on artificial intelligence-driven ed tech products that promise everything from letting teachers operate on autopilot to ensuring that all students receive exactly what they need, minute-by-minute. 

Principals, superintendents and other school leaders are being subjected to sleek demos from presenters who know all the right buzzwords. Salespeople are showcasing technologies and hinting that, should administrators choose not to buy their product, students will be left behind. That the digital divide will grow — on their watch. 

This intense pressure will cause some school leaders to sign agreements that will not serve their students well. While most mistakes won’t rise to the level of prominent school districts supporting a failed startup, some administrators will nonetheless let anxiety rule the day. They will spend money on tried-and-true methods that could do more good than the shiny new thing. 

What can prevent this? Asking the right questions. 

I have spent years on both sides of the table. As a teacher, I used both good and bad ed tech, so I have seen what actually works in the classroom. As an employee of an ed tech startup I designed technologies that were sold to schools, so I know when promises are inflated to help the bottom line. As a professor of educational technologies and leader of the University of Southern California’s Center for Generative AI and Society, I study these technologies to know what works — and what doesn’t — when it comes to AI’s influence on education.

That is why, with the support from my colleagues, I have written a titled “Beyond the Salespitch: A Practical Guide to Questioning EdTech Vendors.” Below are some key questions every school leader must consider before signing on the dotted line and committing limited resources to AI-powered ed tech.

Does the Tool Really Help Teachers in the Classroom?

• What specific cognitive skills does the product aim to develop (e.g., argument construction, data interpretation, metacognitive reflection)?
  
• How does it supplement learning rather than replace it? Ask for a step-by-step walk-through showing where student input is required and how feedback loops are built in.
  
• What evidence exists that students improve on the targeted skills? Request pilot study results, raw performance data or an independent evaluation report — nothing less rigorous will do. 
  
  

Does One Size Really Fit All?

• Which student populations were included in testing the product? Look for data on age, language proficiency, special education status and socioeconomic background. Biased data means biased performance.
  
• How does the system adapt to individual students’ profiles? Ask for a live demo of the personalization engine that shows adjustments based on prior performance and teacher input. 
  
• What safeguards are there for students who might become over-reliant on the tool? Ask how the product detects and flags excessive automation, prompting teacher intervention.

Who Owns the Content?

• Where is student data stored and processed? Identify the physical location of servers (domestic versus overseas) and any third-party processors involved. Ask how data breaches are handled and communicated.
  
• Who owns the text or media students generate using the tool? Ideally, the contract must state that all user-created content remains the property of the school district. Otherwise, your school becomes another content generator for a company. 

• Will our data be used to train future versions of the AI? If yes, require an opt-out clause and a clear explanation of how opting out affects service quality. 
  
• What happens if protected information (FERPA/HIPAA) is entered by mistake? Insist on a documented protocol for immediate deletion and breach notification. Consider clauses that allow you to exit agreements when large enough mistakes occur. 

Are Students Protected From Bias and Misinformation?

• What filters prevent misinformation or harmful content from reaching children? Get access to a live example of the system handling ambiguous queries that could lead to falsehoods. Stress-test it!
  
• Who is responsible when the AI produces inappropriate output? The contract should assign liability to the vendor for any harm caused by the tool’s autonomous decisions.

Can You Walk Away From the Contract?

• What are the cancellation terms? Look for a no-penalty opt-out clause that allows the district to discontinue use if outcomes aren’t met. 
  
• Is there an outcomes-based pricing model? Vendors that are willing to tie fees to measurable learning gains demonstrate confidence in their product.
  
• What support and training are included? Clarify the depth of professional development, response time for technical issues and availability of on-site experts.  

While AI has the potential to be a powerful tutor, a strong assistant and an equalizer in under-resourced classrooms, it can do so only if educators adopt tools with full information. School administrators should demand transparency, evidence and ethical safeguards from the companies that build them. 

District leaders should take these questions back to their board, curriculum team and legal counsel. Let them, not the slickness of a demo, drive the conversation. This can turn AI from a tempting shortcut into a responsible ally in education.

Did you use this article in your work?

We’d love to hear how The 74’s reporting is helping educators, researchers, and policymakers.