Pāṇinian Grammar: Part 3 - Generative Grammar
This series of articles is dedicated to Pāṇinian grammar of the Saṃskṛta language, as presented in his treatise “Aṣṭādhyāyi”. In trying to characterize the grammar of the Saṃskṛta language spoken in his day, Pāṇini Maharṣi manufactures legitimate and only legitimate words of the Saṃskṛta language, by designing a set of 4000 rules in a highly compressed language of the “sūtra” that operates on the sounds, verb roots, prefixes and suffixes, to generate all the words of Saṃskṛta language.
In this process, Pāṇini uses techniques from modern computing (especially the theory of formal languages that are used in analysis and compiler design for programming languages) and anticipates them even though it would be 2500 years from his time when the first actual computer would be invented. Techniques like optimal ordering of data, information compression, metalanguage, metarules, use of non-terminal symbols, light before heavy are all used in this masterpiece. To study the Aṣṭādhyāyi, it is essential to know the Saṃskṛta language, as that is the language for which the grammar is designed. However in this series, my aim is to introduce how Pāṇini uses these techniques in his treatise for people who don’t have any knowledge of the Saṃskṛta language, or formal language theory in computer science. I also present the rich developments in the philosophy of language that Pāṇini’s work inspired in his successors and compare it with Western philosophies of language like that of Russell, Wittgenstein, de Saussure, etc.
Language was an important concern in ancient Indian philosophy, especially for the Āstika schools, as they believed that the efficacy of the Vedic rituals directly depended on the sounds of the chanted mantras. They wanted to justify how this efficacy is brought about, and in the process, they had to analyze how the sounds of a language relate to meaning and ultimately reality itself. By the end of this series, you will have a deep appreciation of the importance, depth and scope of Aṣtādhyāyi and the Indian views on language.
A Bookstore
In Part 1, we went to a fruit shop to demonstrate the concept of optimal ordering. This time, we will consider shopping in a bookstore. We will see that managing a bookstore efficiently requires more than optimal ordering. Imagine that you are the owner of a bookstore that sells a particular genre of books - say literature. Imagine that your bookstore is a bunch of linear arrangements of books in just one direction - left to right. For some reason, your shop has a lot of length but only a small width - so you can afford only a single large shelf that goes from left to right. This may be weird, but it will be most helpful when you later connect it to computer science (arrays) and Pāṇinian grammar. To be more precise, imagine all of the books that you want to sell are arranged in an array - one after the other, from left to right, as shown below.
Book 1 - Book 2 - Book 3 - Book 4 - Book 5 - and so on.
For an efficient shopping experience for your customers, you do realize that there should be an efficient ordering of the books involved - as in the fruit shop case. But the problem is that books carry more information than a fruit. There are so many criteria that one can use to efficiently order the books. Some of them are enumerated below.
One can arrange the books in alphabetical order of their authors. This will enable people who are shopping for books of specific authors to find what they want easily.
One can arrange the books starting with the most recent releases, to the book that is released the oldest. This can enable people who are exploring for new releases and selecting on the spot very much. This will also enable people who are exploring to randomly explore and select vintage books.
One can arrange the books in terms of increasing price. This can enable people who are shopping randomly and are open to any book but come with specific budget constraints.
One can arrange books by genre: fiction, fantasy, classic, self-help, etc. This is also another useful idea. Within a given genre, another criteria from the above options can be used to order the books within that genre.
We see that number 4 is the most efficient way; but that still leaves a question, how to order the books within a given genre? Should we follow 1, 2, or 3? Whatever we do, we realize that we can choose only one and cannot make everyone happy. Within a given genre, if we arrange by the alphabetical order of the author’s name, then people who shop for specific author’s books will be very happy to search, but people shopping by other criteria like recentness and budget constraints will be sad.
What do we do then? We could adopt this solution: We can do the arrangement via author’s name; but to satisfy the budget customers, and customers who have the recentness of the books they shop for in their mind, we can attach a tag from every book that has the price info and the month and year of release. For example, we can attach a big tag from every book that displays something like:
02-2023, Rs 435
This can easily convey that the book was released in Feb 2023 and costs Rs 435/-. A budget customer can keep looking at these tags for the price, instead of taking out each book and looking inside for the price. So he can take out and explore only those books within his price range. This will save some time for him (although not as ideal as the option number 3).
Now, there is an important thing. This tag is useful only until you select the book. This is only to facilitate shopping and selection. The information in the tag is not a part of the book. Once you purchase the book, you will throw it away (like you throw away the tags in your clothes once you purchase them - the tags are not a part of a cloth and are there only to facilitate the shopping experience). So notice that the tag is not a part of the book.
A Little Computer Science
I myself am not a computer programmer - do not worry if you also are not one. I will just give a barebones introduction to an extremely simple computational problem in the area of formal language theory, in an easily understandable manner so that one can appreciate what Pāṇini did, which will come up in the next section. Let’s say, someone states that you should display on your screen the following list of strings (a string is a collection of letters that need not have any meaning in any language):
AC AACC AAACCC AAAACCCC AAAAACCCCC upto 10,000 iterations.
You might ask initially, what is the point of this exercise? I have chosen a very simple problem to convey a point here, even though it might not seem that practically relevant. Obviously I have to generate 10,000 strings of this type, made of equal numbers of the letters A,C in equal numbers. One way to do this is to type these strings one-by-one manually onto the screen. But come on, you will realize that is inefficient, straightaway. Now, let me write an algorithm for a computer to follow:
Step 1: Start with S
From i=1 to i=10,000
Step 2: Rewrite S = ASC “i times”
After doing so for each “i”,
Step 3: Rewrite S = BStep 4: Erase BStep 5: Send the final string to screen
So, we see that this code can generate the required list of strings we want. For example, taking i=4, we have ‘Rewrite S = ASC’
This gives us the following derivation: S = ASC = AASCC = AAASCCC = AAAASCCCC (4th time, i=4)
Then, we have: AAAASCCCC = AAAABCCCC (Step 3: Replace S by B) = AAAACCCC (Step 4: Erase B)
So, we see that this set of instructions, when blindly repeated 10,000 times (counted by “i”), can generate all the strings that we want on the screen.
But look what has happened here! Here, the strings that I have to generate have only the letters A and C! But for coding and manufacturing them, I have used extra symbols in bold - S and B. These letters were used only as intermediaries for keeping track in the iterative process and they disappear in the final stage when the word is produced. So you realize that to generate a set of strings from an alphabet set computationally, one finds the need to use extra symbols apart from the alphabet for book-keeping purposes. Here, the extra letter S was used as a kind of counter to keep producing repeatedly more sets of A and C that we want (through the iteration with the variable “i”). The letter B was used as an intermediary to erase the S to produce the final string that is bereft of any letters other than A and C.
Now, the letters S, B - they are used just as intermediaries, facilitate the iterative generation of words, and disappear in the final stage. This must remind you of the tags in the bookstore that are not a part of the book, are there only to facilitate the shopping experience, and are removed after the book is selected and purchased! In the area of computer science called “Formal Language Theory” - which deals with algorithmic generation of a given set of words over an alphabet - there is a technical name to these two kinds of letters.
The letters that are a part of the words of the final intended language that is to be generated are called terminal symbols. The set of all such terminal symbols is called the terminal alphabet. The letters that are not a part of the words of the final intended language but are used by the algorithm to just act as intermediaries to facilitate the generation of the final word and disappear in the final stage when a word is created are called non-terminal symbols. The set of all such non-terminal symbols is called the non-terminal alphabet.
In this definition, the tag in the book is a non-terminal!
Now, what Pāṇini did is essentially this. He was concerned not with generating a bunch of simple strings of only two letters. His aim was to generate all Saṃskṛta words algorithmically! His problem was not as simple as generating AC, AACC, AAACCC etc. It was basically to generate all the words of a complicated natural language based on the sacred Vedas! He wanted to design an algorithm so that there was a way to derive any Saṃskṛta word through his algorithm, so that people could check whether what they uttered was a legitimate Saṃskṛta word or not.
Remember at the time of Pāṇini, the Vedic language was already becoming archaic and difficult to understand because Vedic Saṃskṛta had changed and evolved a lot into the various Prākṛts. So, at Pāṇini’s time, the language of ordinary discourse and the native tongue of everyone was one of the Prākṛts with Saṃskṛta, being reserved only for intellectual discourse by the people who preserved the Veda. Now, for a person whose native language is a Prākṛt, and with Saṃskṛta being learnt later and used only in formal communication, there is a high chance that he might mix the two up and make mistakes in his Saṃskṛta pronunciation or mix up Saṃskṛta pronunciation with a Prākṛtic pronunciation. But this should never happen to the Veda - the sacred texts were transmitted for thousands of years orally, and the efficacy of its hymns depended on accurate pronunciation of its words. This is where Pāṇini enters the scene - he thought ‘what if’ in future - two people are pronouncing the same word in a Vedic chant differently, how will they resolve which one of their pronunciation is correct, and hence which one of their words is a legitimate Saṃskṛta word? So, it is in this scenario that he presents his grammar. Through his sūtras, Pāṇini says:
A word is a legitimate Saṃskṛta word if and only if it can be derived from the repeated and correct application of some subset of my sūtras.
In other words, he is giving us an algorithm (a generation process, like we saw for the language consisting of the words ‘AC, AACC, AAACCC’) but for the natural language of Saṃskṛta consisting of all Saṃskṛta words! Pāṇini’s sūtras are the transformation steps like seen in the simple A&C language like “ Rewrite S = ASC” or “Rewrite S = B” or “Erase B”.
This technique of using tags or non-terminals or auxiliary symbols was rediscovered in Europe only in the last century.
Pāṇini used the method of auxiliary symbols (non-terminal symbols), in which new affixes are designated to mark syntactic categories and the control of grammatical derivations. This technique, rediscovered by the logician Emil Post, became a standard method in the design of computer programming languages.(1)
Noam Chomsky, the person who laid most of the foundations of modern linguistic theory and formal language theory, also acknowledges Pāṇini as his antecedent. At a speech he delivered in Kolkata in 2001, he said, “The first generative grammar in the modern sense was Pāṇini’s grammar”.
The idea of describing the structure of language with rewriting rules can be traced back to at least the work of Pāṇini, who used it in his description of Saṃskṛta word structure – hence, some suggest to rename Backus-Naur Form (BNF) to Pāṇini–Backus Form.(2)
References
- Kadvany, John (2007), “Positional Value and Linguistic Recursion”, Journal of Indian Philosophy 35: 587–520.
- P.Z. Ingerman (1967)
Authored by Rama Seshan Chandra Sekaran. Published by Brnat.