Your well written, insightful essay addressed many profound questions, including the meaning of life, the pursuit of knowledge, and the role of science in society. At a practical level you address how a professor adapts to large language models in his mentoring of students. I no longer teach but can imagine the dilemma of encouraging students to use the latest tools, while also wanting them to learn how to answer unaided. Your essay included a subtheme about the history of AI and sarcastically critiqued your own involvement as caught in a local optimum. You said, "I am hopelessly old school. My thesis was on knowledge representation. Symbolic AI, as it has been called, hit a wall." You sold yourself short in that part of the essay which is my intention to redress here.
I went to Google Scholar and searched for my profile. My most cited paper by two orders of magnitude was the one co-authored with you titled "Development and application of a metric on semantic nets". Another well cited paper we co-authored was "Merging thesauri: principles and evaluation". After you left me, as good PhD students must do, my ability to author cited papers collapsed!
Did those two papers -- "a metric on semantic nets" and "merging thesauri" -- go the way of the dinosaurs? ️ Why did we want a metric on semantic nets? We were automatically mapping queries and documents to a semantic net and using the metric to find the nearest document to a query. Neural networks might be viewed as doing that. Why did we merge thesauri? Thesaurus is another term for semantic net or knowledge base, and merging thesauri was a machine learning method to build knowledge atop knowledge. We took knowledge bases, such as the International Classification of Diseases and the Medical Subject Headings, and automatically merged them to produce a bigger knowledge base, as demonstrated by being able to retrieve documents better with the bigger knowledge base.
What comes after trillion parameter neural nets trained on all documents? Can we create an artificial system that has a purpose of growing, consumes resources in the process, and must compete with other systems for those resources? How do we respond to our students using deep learning systems to answer our assignments? We turn from mentoring our students on how to read and produce documents and turn to mentoring them on how to generate the next generation of artificial intelligence systems which become living systems and replace us!
Your essay portrays the struggles of a scientist as an artist who wonders whether Zipf's law is all there is, while wishing to be more like the author of 'One Hundred Years of Solitude'. You are that kind of creative scientist who goes beyond the routine problem solving of the craftsman. You conclude with a dialectic in your final two sentences: "Are my misgivings the last convulsions of an artisan concerned about the industrialization of their craft? Or, are we running the risk of hitting a creativity fixed-point by increasingly relying on LLM-inspired tools for coding, writing, and researching?" If restricting the use of LLMs is not a viable, long-term solution, you currently have no resolution to your dialectic. Your problem has no apparent solution within our existing paradigm. In Kuhn's "Structure of Scientific Revolutions" identifying a problem is a necessary step in a paradigm shift. The other step is to propose a new paradigm in which the problem can be solved. Do you think that a solution to your dilemma might require a new paradigm? If yes, then are you interested in first developing that paradigm but, more crucially, persuading your community to accept that new paradigm?
Hafedh felt he had hit a wall and likened that to the impact of Generative AI. Instead, we need a new paradigm.