LLMs as Knowledge Workers NeSy, SyNe, and trustworthy AI: AAAI-MAKE 2026 symposium talk

AAAI-MAKE 2026 · Spring Symposium Series · Burlingame, CA

Authors withheld for double-blind review

Sergei Nirenburg¹ Marjorie McShane¹ Sanjay Oruganti¹ Jesse English¹ Kwan Srijomkwan¹

¹The LEIA Lab, Rensselaer Polytechnic Institute

AAAI-MAKE 2026 LEIA Lab

BibTeX

@misc{nirenburg2026llmsknowledge,
      title={{LLMs as Knowledge Workers}},
      author={Nirenburg, Sergei and McShane, Marjorie and Oruganti, Sanjay and English, Jesse and Skijomkwan, Kwan},
      year={2026},
      howpublished={Talk at AAAI-MAKE 2026 Spring Symposium, Burlingame, CA},
      note={NeSy vs SyNe; symbolic supervision of LLMs as knowledge workers},
      url={https://www.leia-lab.com/},
}

Abstract

Trustworthy AI requires mitigating hallucination, opacity, and failures that resist diagnosis. This page accompanies a symposium talk arguing that scaling alone, and even strong symbolic verification bolted onto an LLM-centric core (typical NeSy), leaves a structural ceiling: when the neural model owns interpretation, formal methods only see what the model formalizes. We revisit Bar Hillel’s distinction between reliable formal inference and hard semantic interpretation[4], introduce SyNe (symbolic core, neural tools) as a contrasting pattern to NeSy (neural core, symbolic add-ons), and summarize properties that follow from putting symbolic analysis in charge. We describe a large-scale OntoAgent experiment in which LLMs propose lexicon and ontology extensions under automatic validation[8], outline how OntoAgent instantiates SyNe across the perception–reasoning–action pipeline, cite peer architectures with a similar division of labor[10][11][12][13], and close with assessment and claims about scaling knowledge-rich AI with LLMs as supervised knowledge workers.

Framing

Our common objective

We are here because we want to develop AI systems that can be trusted: eliminate hallucinations, opacity, the inability to explain or correct errors. The objective is stable across our different domains and tools.

Proceedings paper. Our team’s paper in the proceedings reports an experiment in applying LLMs to the task of extending knowledge resources — with automatic symbolic validation[8]. Please read the paper for experimental detail; this page follows the arc of the oral presentation.

This companion. What we instead want to present is a methodological convergence happening in this community that deserves a name and further discussion.

Recent evidence

The consensus: LLMs are not enough

Two recent lines of work illustrate why “prompt harder” is not a substitute for architectural change.

Song et al. [1] The definitive survey: systematic LLM reasoning failures across every cognitive dimension.

First unified taxonomy of LLM reasoning failures (67 pp.), including the reversal curse, compositional breakdown, working memory deficits, Theory of Mind failures, and metacognitive blindness. Root cause: architecture, not an engineering gap. Prescription: architectural innovation, not better prompting.

Shojaee et al. [2] “The Illusion of Thinking”: controlled experiments show complete LLM collapse beyond a complexity threshold.

Controlled experiments on frontier reasoning models (o3-mini, DeepSeek-R1, Claude Thinking). Three regimes: LLMs suffice at low complexity; reasoning models help at medium; both collapse at high. Providing the optimal algorithm in the prompt doesn't help — models cannot execute prescribed logical steps. The LLM is not a reasoning engine. It's a pattern completer that looks like one.

The obvious response: combine neural and symbolic. But even the best-engineered NeSy system hits a ceiling.

NeSy

NeSy and its best-engineered application

NeSy (neural + symbolic) adds symbolic structure to the LLM. A genuine improvement, but who is in charge? Most NeSy systems rely on the LLM for everything: interpretation, reasoning, decisions, with symbolic components as post-hoc filters. Even when a symbolic reasoner delivers mathematical guarantees, a ceiling remains.

ARC (Automated Reasoning Checks). [3] Bayless, Rakamarić et al., Amazon.

The LLM translates natural language into logic; an SMT solver verifies. The solver achieves >99% soundness, a genuine engineering achievement. At the highest soundness setting, only 16% of inputs survive. The LLM-based interpreter cannot produce confident formalizations for the other 84% and “declines” them, and cannot explain why. It can also “accept” incorrect translations, producing wrong answers downstream.

Quality ceiling. Excellent formal inference capability — but input interpretation performed by a system that does not understand what the input means. This defines a quality and trustworthiness ceiling for the system.

Generality. Any NeSy architecture that delegates interpretation to an LLM inherits this pattern.

This ceiling was diagnosed sixty-five years ago.

Historical note

Bar Hillel (1960)

Yehoshua Bar Hillel
(1915–1975)
Philosopher, logician;
pioneer of MT research

“The community devotes enormous effort to the rules of inference, which are mechanical once the formulas are in hand, while almost entirely neglecting the translation from natural language into formal expressions, where all the difficulty lives.”

The Present Status of Automatic Translation of Languages (1960)[4].

Note for 2026. Pattern-matching natural language onto formal-looking structures is not formalization. Bar Hillel’s hard problem is semantic and pragmatic: resolving ambiguity, recovering implicit meaning, and grounding language in an ontological world model.

Sixty-five years later, formal reasoning (Bar Hillel’s “easy” problem) can be done reliably by symbolic methods. Interpretation (his “hard” problem) is what much of the mainstream now delegates to the LLM.

Bridge. What if the symbolic system controlled interpretation, with LLMs assisting only in bounded roles?

Terminology

From NeSy to SyNe

SyNe (tentative term — suggestions welcome): the symbolic system controls the cognitive work; the LLM does the grunt work. This is not merely opposite control flow but a different assignment of who performs which cognitive functions.

NeSy: neural core + symbolic add-ons

LLM does the cognitive work

1LLM interprets input and generates formal structures.

2Reasoning follows (by LLM or symbolic module), but only from the LLM’s interpretation.

3Symbolic system validates after the fact.

4When any stage fails: no diagnosis of why, no learning from it, no targeted recovery.

Interpretation is opaque: there is no inspecting why the LLM chose a reading, and reasoning often is opaque as well.

SyNe: symbolic core + neural tools

Symbolic system controls the cognitive work

1Symbolic system interprets and builds meaning representations.

Typed properties and relations, compositional semantics, ontology-grounded (not mere pattern matching).

2Symbolic system reasons and decides from goals and world model.

Over persistent, inspectable structures.

3LLM assists in bounded roles (proposes, generates, retrieves); it never decides.

Hypotheses only; validated before use.

4Automatic validation against ontological constraints is always on.

Built into the system; no proposal is accepted without validation.

5Human teammate is final authority.

In learning, the human reviews proposals against constraints. In operation, the human is team leader: the agent consults, defers, and asks targeted questions. The system tracks what it does not know.

While both NeSy and SyNe face uncertainty, the former at best refuses to decide; the latter continues to function under uncertainty: diagnoses gaps, asks for help, defers, or learns. Like people do.

Comparison

What the pivot to SyNe brings about

SyNe properties follow from putting the symbolic system in charge. Does your work share them?

Property	Pure LLM	NeSy	SyNe	Why it matters
Who interprets input?	LLM (opaque)	LLM (opaque)	Symbolic analyzer (inspectable) with LLM support	Interpretive process is transparent and inspectable
Metacognitive self-monitoring?	Generates plausible text with no introspection	Can detect failure but cannot diagnose it	Detects gaps, diagnoses them, acts: asks for help, defers, or learns	Fails visibly and productively, not silently
Compositional reasoning?	Unreliable at multi-step composition	Formal step only; input interpretation can’t be trusted	Via structured MRs with typed properties and relations	Multi-step inference follows from representation
Error recovery?	Retrain or re-prompt	Rules fixable; LLM errors are not	Surgical: find the error, fix it, done	Targeted repair, no retraining
Persistent knowledge?	Parameters only	Knowledge graphs (nodes labeled with text, not concepts)	Ontological world model + lexicon + episodic memory	No re-interpretation cost; new knowledge builds on old
Inspectable reasoning?	Post-hoc rationalization	On the formal inference, not input interpretation	Full trace: input → MR → reasoning → action	Explains what it did, not what sounds plausible
Knows what it doesn’t know?	Uncertainty language decoupled from actual knowledge state	On the solver’s step only; blind to the LLM’s uncertainty	Grounded: computed from representation, not language patterns	Acts on what it does not know: asks, defers, or learns
Human role (learning)?	Prompter	Rule-writer	Symbolic system auto-validates; human is final arbiter on hard cases	Human as final arbiter, not acquirer
Human role (functioning)?	Prompter	Monitor / corrector	Team leader: instructs, monitors, overrides	Agent assists; human retains authority

Case study

Experiment: SyNe in use

A concrete instance uses the OntoAgent / HARMONIC knowledge stack:

~9,000

ontological concepts

~30,000

lexical senses

1,780

validated meaning representations

Built over 30+ years · operational in OntoAgent / HARMONIC

Task. Given an English text, the LLM proposed lexicon entries for unknown words and word senses, and new ontological concepts to represent their meanings when existing ones could not be used.

Validation. Every proposal was automatically validated against formal ontological constraints.

Human loop. When the automatic validator could not confidently accept or reject LLM results, human validators were consulted. The human became a final validator rather than acquirer.

Key line. That's how you scale AI without sacrificing trust.

This pattern generalizes across the entire AI agent pipeline. Throughout, the question is: how do I use an LLM without having to trust it?

OntoAgent

SyNe across the AI agent pipeline

In each module below, the symbolic system leads; LLM use is bounded and instrumented. (Metacognition is LLM-excluded by design in this architecture.)

Module	Symbolic system	LLM role	References
Perception	OntoSem analyzer builds typed meaning representations (XMRs) from text, speech, vision	Presemantic processing; paraphrase when the analyzer cannot parse input	McShane & Nirenburg, 2021[5]; McShane et al., 2024, chs. 4–7[6]
Reasoning	Plans from goals + world model; attention; diagnosis from domain knowledge	LLM outputs as parameters in heuristic functions	McShane et al., 2024, chs. 5–6[6]
Metacognition	Gap detection, diagnosis, actionability, help-seeking; collaborator models	Excluded by design	McShane et al., 2025[7]
Action	AMR → procedure calls (verbal, physical); HARMONIC extension	Assists at NLG stages	McShane et al., 2024, Sec. 4.3[6]; Oruganti et al., 2024[9]
Explanation	Full trace input→MR→decision; reasoning builds GMR for NLG	NLG for explanations; optional corpus retrieval for evidence	McShane et al., 2024, ch. 8[6]; McShane et al., 2025[7]
Learning	Validates proposals; human review below confidence threshold	Proposes lexicon and ontology extensions	English, 2010; McShane et al., 2024, chs. 7, 9[6]; Nirenburg et al., 2026[8]

Each module is operational and uses LLM tools. Work ongoing on expanding LLM tools' roles.

Landscape

Explicitly SyNe-oriented architectures

Four different mechanisms, a clear family resemblance: the symbolic system leads, the LLM serves.

Architecture	SyNe mechanism	Reference
Soar	LLMs extract goal descriptions; STARS vets outputs; validation by task completion (~94%). LLM is a tool; Soar stays in charge.	Kirk et al., 2024[10]
ACT-R	Decision traces from ACT-R fine-tune LLMs to replicate them; knowledge flows architecture → LLM, not the reverse.	Wu et al., 2025[11]
Companion	LLMs augment symbolic NLU; analogical consistency against NextKB. Goal: “knowledge autonomy.”	Nakos & Forbus, 2025[12]
DIARC	Symbolic control with deep NL integration; language as teaching interface.	Scheutz et al., 2019[13]

AAAI-MAKE 2026

Themes from the symposium conversation

Molière’s M. Jourdain discovered he had been speaking prose his whole life without knowing it. May it be that many of us have been “speaking” SyNe without naming it? Based on published abstracts from this symposium, three clusters follow (illustrative, not exhaustive).

The NeSy ceiling is real

The quality ceiling was identified independently: without symbolic interpretation, the LLM’s limitations propagate through the system.

Theme I: contributors and abstract-level findings on the NeSy ceiling
Contributors	Findingfrom abstract
Lima et al.	LLM-based robot planning is “flexible but still fragile” without a structured world model.
Shahid & Rothe	Grounding (interpretation) and compositionality in NeSy systems interfere rather than complement.
Clark / AI2	Long-horizon agents meander without structured knowledge. Right diagnosis; LLM still does the cognitive work.
Rakamarić / Amazon	ARC: >99% soundness on 16% of inputs; Bar Hillel’s warning, quantified.

SyNe-aligned desiderata

What trustworthy AI requires is articulated here: editable knowledge, interpretable representations, finer distinctions than NeSy allows.

Theme II: contributors and desiderata aligned with SyNe
Contributors	Findingfrom abstract
Khandelwal et al.	Reinforcement learning agents need editable, auditable knowledge; retraining opaque parameters is “costly and difficult to audit.”
Fang Li	Builds compositional function networks as inherently interpretable alternatives to opaque DNNs.
Lixing Li	LLM provers fail when surface patterns are obfuscated, even though the logical structure is unchanged.
Habibi et al.	Evaluation itself must be symbolic-mechanistic, not just accuracy-based.
Schmid	The NeSy label is too coarse: the integration pattern matters. The SyNe distinction is one answer.

Symbolic AI already in charge

The inversion has already been made: symbolic structures are the primary representation, not a constraint bolted on afterward.

Theme III: contributors and findings where symbolic structures lead
Contributors	Findingfrom abstract
Günther et al.	Scene graph as foundational backend of perception: the primary representation, not a derivative layer.
Millison et al.	State machines (explicit symbolic control) structure physical science reasoning agents.
Galitsky	Logical argumentation frameworks verify LLM output. Symbolic validates neural.
Shyalik et al.	Causal domain models drive diagnostics in manufacturing; LLMs assist (SyNe in an industrial setting).
Amblard et al.	Reject neural methods entirely for safety-critical chemical processes. Full symbolic for trust.

Abstract summaries are interpretive; consult the symposium proceedings for authoritative statements.

Synthesis

Assessment and claims

Generative AI alone. Generative AI will not get us to trustworthy AI by itself. Not by scaling, nor by formal verification bolted on afterward.

Knowledge-based AI. It has always been the path to trustworthy AI. The primary obstacle was cost, not the soundness of the approach. That cost was real; we have spent decades building the infrastructure.

LLMs as knowledge workers. The advent of LLMs dramatically reduces the cost of continuously extending the knowledge resources, though to yield adequate quality of results, LLMs must be employed as knowledge workers operating under the supervision of symbolic — explainable, transparent, trustworthy — AI.

Division of labor. The LLM proposes; the symbolic system validates; so when something is wrong, you can find it, explain it, and fix it.

The human. The human remains the ultimate arbiter and team leader.

But. The human is absolved from the task of manual knowledge acquisition.

SyNe as principle. SyNe is an architectural principle, not a single system. OntoAgent is one instance; others appear in this community.

And every group that uses symbolic components — however deep, however partial — has to make the choice between NeSy (generative AI core with varying contributions from symbolic AI) and SyNe (symbolic AI with LLMs doing the grunt work).

Claims:

To attain trustworthy AI, we need SyNe.
To attain broad-coverage AI, we need SyNe to scale.
The use of LLMs as knowledge workers will enable SyNe-based AI systems to scale, just as the use of GPUs enabled neural methods to scale.

Contact: nirens@rpi.edu · leia-lab.com

This convergence deserves a name, a community, and a research agenda during the symposium and after it.

Citations

References

Key references cited in the talk

Y. Song, S. Xie, S. Yuan, and C. Zhang, “Large Language Model Reasoning Failures: A Survey,” Trans. Mach. Learn. Res., 2026.
P. Shojaee et al., “The Illusion of Thinking,” Apple ML Research, arXiv:2506.06941, 2025.
J. Bayless, Z. Rakamarić et al., “Automated Reasoning Checks,” Amazon, arXiv:2511.09008, 2025.
Y. Bar Hillel, “The Present Status of Automatic Translation of Languages,” in Advances in Computers, vol. 1, 1960.
M. McShane and S. Nirenburg, Linguistics for the Age of AI. Cambridge, MA, USA: MIT Press, 2021.
M. McShane, S. Nirenburg, and J. English, Agents in the Long Game of AI: Computational Cognitive Modeling for Trustworthy, Hybrid AI. Cambridge, MA, USA: MIT Press, 2024.
M. McShane et al., metacognition and OntoAgent line, LEIA Lab publications, 2025.
S. Nirenburg et al., “LLM-assisted knowledge extension with validation,” in Proc. AAAI-MAKE Spring Symp. Ser., 2026.
S. Oruganti et al., HARMONIC cognitive-robotic architecture extensions, 2024.
J. Kirk, R. Wray, P. Lindes, and J. Laird, Soar with LLM tool use and STARS validation, in Proc. AAAI, 2024.
C. Wu, A. Oltramari, J. Francis, C. L. Giles, and F. Ritter, ACT-R decision traces and LLM fine-tuning, in Proc. AAAI-MAKE, 2025.
A. Nakos and K. Forbus, Companion cognitive architecture with NextKB, Cogn. Syst. Res., vol. 92, 2025.
M. Scheutz et al., DIARC, in Common Model of Cognition, Springer, 2019.