Small Language Models Could Let Schools Run AI Entirely Behind Their Own Walls

Artificial intelligence is sweeping through education, but the systems making headlines—ChatGPT, Gemini, Claude—are cloud giants that quietly shuttle every student keystroke to remote data centers. For primary schools charged with protecting children as young as five, that arrangement is increasingly untenable. A new conceptual analysis published in Frontiers in the Internet of Things argues that the answer may lie not in bigger models but in smaller ones: compact language models hosted on cheap hardware inside the school building itself, where no child’s data ever leaves the premises.

The paper, authored by Misbah Zulfiqar, an independent researcher in New South Wales, Australia, and Umair Iqbal of the University of Newcastle’s School of Engineering, does not report a working deployment. Instead, it builds a theoretically grounded framework for what the authors call locally hosted small language models, or SLMs—neural networks with roughly 1 billion to 13 billion parameters, in contrast to frontier systems like GPT-4 that pack hundreds of billions. The authors’ central claim is provocative: in primary education, a model that is smaller but controllable may be institutionally more valuable than one that is larger but opaque.

The technical logic is straightforward. SLMs such as Microsoft’s Phi-3, Google’s Gemma, Meta’s smaller Llama variants, and Mistral 7B can run on consumer-grade graphics cards or even optimized CPUs, operate offline after an initial download, and cost dramatically less per inference than cloud application programming interfaces. They also consume less energy and allow schools to inspect and fine-tune the model itself. The trade-offs are real: reduced reasoning depth, smaller context windows, narrower general knowledge, and higher rates of hallucination—the confident generation of plausible but false information. A small model might, for instance, insist that the capital of Australia is Melbourne.

Yet the authors argue that primary school tasks are precisely the kind that neutralize these weaknesses. Learning activities for children aged five to twelve tend to be domain-specific, repetitive, heavily scaffolded by teachers, expressed in simplified language, and always mediated by an adult who evaluates outputs before students see them. Under those conditions, the gaps between a 7-billion-parameter model and a frontier giant shrink in practical importance, while the advantages in cost, governance, and deployability grow. The choice between local and cloud AI, the paper stresses, is not a binary of superior versus inferior technology but a contextual decision shaped by each school’s regulatory, financial, and organizational circumstances.

The proposed architecture is deliberately modest. A school might deploy a refurbished desktop workstation with a mid-range GPU, or a mini PC with an entry-level graphics card, at a one-off cost ranging from several hundred to several thousand Australian dollars per server. Open-source frameworks such as Ollama or LM Studio handle model deployment without requiring deep expertise in machine learning libraries. Students connect through the school’s internal local area network only, and the server is configured without outbound internet connectivity—an architectural prohibition on data exfiltration rather than a policy promise. A teacher dashboard with interaction logs would support both pedagogy and safeguarding audits.

That design directly confronts what the authors term a surveillance risk at the institutional level. When student interactions flow through cloud platforms, schools lose control over collection, retention, and secondary use of the data, and growing evidence suggests some education technology providers monetize student information for commercial purposes beyond its original intent. With an on-premises server, prompts and responses never leave school infrastructure. Schools can set automatic deletion schedules to satisfy the European Union’s General Data Protection Regulation and similar child-data regimes such as the Children’s Online Privacy Protection Act in North America, audit access logs, and explain to parents exactly what is collected, where it is stored, and who can see it—transparency that is effectively impossible with proprietary cloud systems whose training data and data-handling practices are closed.

Offline capability adds a resilience dividend that cloud systems cannot match. A school-hosted model keeps working during internet outages, a decisive advantage for rural schools with unreliable broadband, and downloaded models could even accompany students on excursions. The authors also point to emerging computational linguistics research suggesting that small models can reproduce features of child language development, including overregularization errors like saying “goed” instead of “went,” and approximate aspects of bilingual acquisition. They caution, however, that these findings come from simulations and textual analysis, not classroom trials, and should be interpreted cautiously.

The paper is candid about the burdens local deployment creates. Schools inherit full responsibility for cybersecurity—access controls, encryption, network segmentation, patching, backups, and incident response—at a time when attacks on educational institutions are rising. Teachers need professional development to recognize hallucinations and biased outputs, and students must be taught to treat AI-generated information as provisional rather than authoritative. Scalability poses another challenge: a single server may serve a pilot classroom, but school-wide adoption could demand multiple servers, load balancing, and periodic hardware upgrades as models evolve. The authors also invoke total cost of ownership, noting that electricity, maintenance, training, and downtime must be weighed against cloud subscription fees—and that no empirical lifecycle cost comparison yet exists.

On the pedagogical side, the framework leans on constructivist theory. Drawing on Piaget, Papert, and Vygotsky’s concept of the Zone of Proximal Development, the authors position SLMs as scaffolding tools that support inquiry and explanation while the teacher remains the pedagogical authority. Illustrative use cases include literacy support with vocabulary explanations and sentence starters, differentiated worksheet generation for teachers, guided question-and-answer sessions during science units, and personalized content for students with learning difficulties or English as an additional language—all under supervision, and none yet empirically validated in real classrooms.

Ultimately, the paper’s contribution is a reframing rather than a breakthrough. It argues that educational AI should be judged not by raw intelligence but by institutional appropriateness: the capacity to deploy and maintain a system, the authority to govern student data, regulatory compliance, and alignment with a school’s values. The authors call for pilot studies, comparative outcome research, stakeholder evaluations, and multi-year economic analyses, and they urge policymakers to reconsider procurement policies that push schools toward vendor dependency. Whether locally hosted SLMs deliver on their promise remains an open empirical question—but the framework offers primary schools something increasingly rare in the edtech landscape: a way to adopt AI while keeping the answers to “what happens to our children’s data?” inside their own walls.

Subject of Research: Locally hosted small language models as a privacy-preserving AI infrastructure for primary education

Article Title: Conceptualising local AI infrastructure using small language models (SLMs) for primary education

Article References: Conceptualising local AI infrastructure using small language models (SLMs) for primary education. (n.d.). https://doi.org/10.3389/friot.2026.1886609

Image Credits: AI Generated

DOI: 10.3389/friot.2026.1886609

Keywords: small language models, primary education, data privacy, local AI infrastructure, educational AI, data sovereignty, cloud-based LLMs, child data protection, open-source software, digital sovereignty, AI governance, on-premises deployment