The Los Angeles Unified School District recently announced that a chatbot named “Ed” for students and families driven by artificial intelligence would not be delivered as promised because the start-up the district bought it from was defunct, The New York Times reports.
Los Angeles agreed to pay a start-up company, AllHere, up to $6 million to develop Ed, a chatbot that would be a friend to students and their families. By analyzing vast amounts of student data on test scores, mental health, physical health, special education accommodations, and family socioeconomic status, the chatbot would supposedly be able to answer questions like “What grade does my child have in math?” and respond to emotions such as hostility, happiness, and sadness.
But just two months after the district’s superintendent, Alberto Carvalho, made a glitzy presentation at a tech conference, AllHere’s founder and chief executive left her role, and the company furloughed most of its staff. AllHere posted on its website that the furloughs were because of “our current financial position,” the Times reported.
Trouble at AllHere was noted by The 74 a week ago, following the furlough announcement. An LAUSD spokesperson said the company is no longer on the job.
Editorial
Some paint this as an AI issue, but it is ultimately a start-up issue. And in AI, everything’s a start-up at this point. Companies and potential developers of AI products that might be useful in schools have no track record.
For example, a former AllHere employee told The 74 that the company failed to protect students’ personally identifiable information (PII). Leaky data practices that have been reported would doom an education service provider.
“Personally identifiable information should be considered acid in a company, and you should only touch it if you have to because acid is dangerous,” The 74 quoted Chris Whiteley, a former senior director of software engineering at AllHere, as saying. “The errors that were made were so egregious around PII. You should not be in education if you don’t think PII is acid.”
Whether or not PII was exposed, school districts rarely have the expertise to evaluate technology such as AI fully. It’s questionable if they have sufficient expertise in-house to evaluate cybersecurity or assess the risks involved with any product like Ed. Moreover, there are no referrals from other districts on which an evaluation of the company’s products could be based.
AI will find its way into schools soon enough, but for now, some large districts, including Prince George’s County Public Schools in Maryland, another AllHere purchaser, are quickly realizing that while it’s good to devote screen time to educational tools, extreme caution is advised when evaluating products that will have access to student or family information.
