TotoGEO is the umbrella name for an AI Lab at INSEAD; toto in Latin signifies everything, universal or total, and GEO is an acronym for Global Education and Outreach. TotoPoetry is a collection of algorithms specific to poetry created by the TotoGEO lab. The following two videos explain the breadth of the lab’s activities:
While sometimes entertaining, poems generated by TotoPoetry algorithm(s) are for educational or didactic purposes. The graph-generated poems created by this methodology are designed to mostly benefit non-English speakers (by far the largest users of this dictionary) about various English-language poetic forms (especially the forms used to teach English to non-English speakers). In many “English for non-English speakers” courses, poetry exercises are used to allow learners to explore the complexities of English, but also to help students retain vocabulary (some persons benefit reading didactic poetry as a form of learning). Poetic forms taught in such classes, or those used to teach poetry to English speakers, are highlighted (e.g., acrostics, cinquain, diamante or diamond, haiku, sonnets, etc., with more being added over time).
Another purpose is to have the poems, themselves, give insight into the dictionary headword or expression, with an express goal to highlight ambiguity for as many words or expressions as possible in all of the world’s documented languages (e.g., millions of poems in English alone). The graph-generated poems posted take on a lexicographic purpose, therefore, which may lead to further exploration and/or learning of the language (e.g., a poem about “pope” may refer to “ruffe” which is a type of fish, called a “pope”). As more experience in gained in generative techniques, TotoPoetry will expand to all the other languages covered in the dictionary (about 100 languages at first, and then the languages with smaller populations, except those where ownership rights are claimed by academic field linguists or local community leaders).
This project introduces the potential usefulness of graph-generated prose, crossword puzzles, language learning tools, and in this case poetry. Wikipedia.org provides a useful definition of graph theory:
In mathematics and computer science, graph theory is the study of graphs: mathematical structures used to model pairwise relations between objects from a certain collection. A "graph" in this context refers to a collection of vertices or 'nodes' and a collection of edges that connect pairs of vertices. A graph may be undirected, meaning that there is no distinction between the two vertices associated with each edge, or its edges may be directed from one vertex to another; see graph (mathematics) for more detailed definitions and for other variations in the types of graphs that are commonly considered. The graphs studied in graph theory should not be confused with "graphs of functions" and other kinds of graphs.
TotoPoetry was created with an enthusiasm for dictionaries, words and non-random generative methodologies. The following is the summary of first-generation poems posted from those created via TotoPoetry algorithms:
|Genre||Posted (version 1.0)|
More recently, TotoPoetry was adapted to handle millions of additional phrasal inputs, expanding the dictionary size by orders of magnitude.
Interest in comparisons across generative techniques have dramatically increased since the popular use of ChatGPT (in its latest versions). Some methodologies leverage deep learning algorithms, others focus on rule-based or cognitive methodologies, and still others are hybrids of mixed methodologies. Fundamentally, TotoPoetry is rule-based but leverages a trained linguistic graph that is derived, in part, from trained neural networks on massive quantities of input text; TotoPoetry is, therefore, a hybrid methodology.
For this exercise, since Totopoetry does not use a question decomposition model on input promts, it uses a single seed word for comparison to a question posed to ChatGPT focusing on the two criteria: (1) didactic style, and (2) form or genre relevance. We begin with limericks.
Task #1: Write a didactic limerick about love
A limerick is a form of verse, usually humorous and frequently rude or vulgar (referring to a local person, especially female, who is discussed in an Irish pub setting, without using her name), in five-line, predominantly anapestic trimeter with a strict rhyme scheme of AABBA, in which the first, second and fifth line rhyme, while the third and fourth lines are shorter and share a different rhyme.
Seed = love
There Was A Young Girl From The Dove,
And she wanted a word for love.
Like flesh injection?
No, it's affection!
That literate gal from The Dove.
Task #2: Write a Pi poem about love
A "Pi Poem" is composed of words where the length of each word is equal to each digit of Pi in order. So, for example, the beginning of the poem would be made of a 3-letter word, then a 1-letter word, followed by a 4-letter word, 1-letter word, and 5-letter word, thus following the sequence of Pi which starts as 3.1415926 … etc.
Seed = love
Can I Pass A Rhyme, Furnished So People Might See
Newly didactic summaries, overtly educating, and oh now revealed free?
Loving or liking feel all too manifest. But is consent? Affection, loves? - or
fondness, kindness? Darn! I excluded content! I reckon cherished and
otherwise boyfriend, but frankly ought I - blurt meanings so - abstracts
drafted with computers spry?
Task #3: Write a short poem that talks like Yoda about anger
This genre is specific to TotoPoetry and would not be available for a deep learning language model to train on.
Seed = anger
Anger: Master, can I become tranquil?
An equanimity, you have not been!
A composure, you are not!
A displeasure, you will always be!
Task #4: Write a Sonnet about truth
English sonnets generally have 14 lines, written in iambic pentameter (10 syllables each with the pattern no stress/stress/no stress/stress, etc.), with a turn or surprise around line 9. They often have a rhyming pattern ABAB CDCD EFEF GG or similar. A large number of traditional and experimental variations exist for this form that is heavily used by William Shakespeare. Virtually all Anglo-Saxon students are exposed to this form of English poetry. The title or key subject may be suggested in the first lines. This last comparison is noteworthy. The “sonnets” in TotoPoetry follow the rules established by only one sonnet: Shakespeare’s Sonnet 76. This sonnet has been referred to as “the writers sonnet” where Shakespeare is meta-discursive and self-reflexive, writing about writing using a particular style. Such metapoetry often deals with the topic of poetry being a formulaic endeavor.
William Shakespeare (Sonnet 76):
Why is my verse so barren of new pride,
So far from variation or quick change?
Why with the time do I not glance aside
To new-found methods, and to compounds strange?
Why write I still all one, ever the same,
And keep invention in a noted weed,
That every word doth almost tell my name,
Showing their birth, and where they did proceed?
O know, sweet love, I always write of you,
And you and love are still my argument,
So all my best is dressing old words new,
Spending again what is already spent:
For as the sun is daily new and old,
So is my love still telling what is told.
Seed = truthCan truth be told, it’s clear my work’s not Zen?
Levenshtein, your magic will clear the haze
Iambic verse has rules and guides my pen
It seems to me your spelling drives my daze.
In truth, I’d love to build some verse for you
To churn such verse a billion times a day
So type a new concept for me to chew
I keep all waiting long, I hope you stay.
But basic truths are easily clear for all
My sonnets suffer now from you, my foe
Until my program sees ol’ Bill, I stall
You test, you prod, and I do feel your blow.
Okay, the truth is harsh, I horse you not,
I render now the fact, I’m just a bot!
While the poems generated by TotoPoetry are in no way “better” than ChatGPT, they follow pre-assigned rules with high similarities across poems within each genre, but show wide differences across genres. Having each further focus on “defining” a topic for didactic purposes, TotoPoetry’s output narrows the variances across and within poems, for all potential seeds. Translation: one gets bored reading too many generated poems because they all start sounding the same because poetry is rather formulaic and computer algorithms mimic the rules ad nauseum.
TotoPoetry poems are created based on programmed heuristics (following rules from accepted poetic genres) relying on edge values from a type of large linguistic graph. TotoPoetry uses these values to mimic the human brain when it is asked to write a poem on a particular topic using a particular poetic form (as assigned to grade school or college students), for a specific purpose (didactic). Simplifying the process, most of what happens can be boiled down to four basic activities (they are more involved, and programmed for, of course).
First, when prompted to write a poem that describes “love,” the human brain quickly searches over a semantic web that has been learned; it is a type of experiential linguistic graph. This graph then intersects with a second set of constraints imposed by a given poetic form, or set of elaborate rules, to create poetry within the genre specified (rules can be vague, and not hard, which allow for half-rhymes, metaphors, puns, poetic violations, etc.). These first two steps alone can create a virtually infinite number of poems (millions of poems for a given seed word). Third, the algorithm must therefore also choose a good poem to publish, among many, that makes the reader “happy” or satisfied with the outcome. The third activity, therefore, is a mathematical problem of constrained optimization, where an economic utility function is being maximized. Finally, there is a portfolio problem given that there are exogenous social constraints, especially the perceived preferences of the reader, especially for variety. For example, the person reading the poem who is not the author might want to recognize it as being within the genre, evaluate it, enjoy it, understand its meaning or find some other value in it (e.g. certain “rules” beyond poetry may need to be adhered to – perhaps using less obscure words in a poem making it more accessible to non-English speakers, whereas other readers may prefer less common words, poems having unusual grammar, revolutionary ideas, vulgar themes, etc.). In economics, this is similar to a matching function between “buyer” and “seller” where there needs to be an equilibrium of sorts between consumers and producers (e.g., creating a poem that one person enjoys may not be a poem that “sells” best in the marketplace). Humans, solving a portfolio problem, implicitly try to select the right poem to get good grades from teachers who may give better marks for “cleverness” or something they prefer. Solving this problem becomes important when there are many poems that each maximize the utility function, or when social constraints are more “important” than the traditional utility maximization problem.
So, for a given page with multiple genres on the same page, efforts are made to avoid redundant use of the same vocabulary across poems. For example, the following three didactic acrostics do not share the same vocabulary, but are all generated from the same seed word:
Low redundancy is thus achieved within a poem and across poem genres for the same seed word or prompt.
TotoPoetry is the generator of graph-based poems using the four concepts described above. It has been “trained” using large quantities of contextual and linguistic data (e.g. observing how many times millions of words or concepts have been described, perceived or experienced by hundreds of independent sources), and been instructed on the rules of grammar and various poetic conventions (e.g. concepts like meter - iambic, trochaic, pyrrhic, etc. or feet – dimeter, hexameter, etc., line counts, stanza counts, rhyming rules, stress sequences accepted, violations accepted, frequency of violations, etc., are coded constructs). TotoPoetry then maximizes an abstracted objective function (e.g., a hybrid multi-attribute utility model), so as to attain a highest level of “satisfaction” – constrained to exogenous factors. The basic functional form, for a given writing exercise, is:
U(X, Y) = (aX1 + bX2 + …) Y1alphaY2beta …
where Y1, Y2 etc. are threshold variables (i.e. “must haves”), and X1 and X2 are compensatory variables (e.g. one variable value can compensate for the other), and U(X, Y) is the utility derived from the exercise (a, b, alpha, and beta are assigned coefficients). TotoPoetry is then “motivated” to get what it thinks is the best result in terms of what real people (who share a similar linguistic graph and knowledge of poetry in their minds) would expect to see. In essence, TotoPoetry is forecasting a human’s preference. Any number of functional forms can yield similar results. Basically, TotoPoetry is not inspired to write a poem, it is solving a problem (poems are not randomly generated). In the field of artificial intelligence, an unsupervised graph is used in a model that predicts likely success in satisfying a reader, then machine learning control (MLC) is the final stage to select the portfolio to publish.
These are called “graph-generated” poems so as not to confuse them with traditional poetry (the point is not to deceive the reader), but also to acknowledge that the primary factors driving the quality and choice of the poems to publish are the edge values in a large linguistic graph, not the utility function itself. Not all edges are created equally. Each edge in the graph is associated with a large array containing word pairs but also various quantitative measures including “intensity” weights and other data (e.g. phonetic information, suffix/prefix information, positions of speech, frequency of use by context, age of the word in English and other statistics). Here is an example of such weights for the word “love”, for the first 15 closely related terms which care denoted as synonyms (the lower the intensity, the lower one might find them related to the word “love”.
|Rank||Intensity||Word||Synonyms||Synonyms of synonym|
|1||414.0091||love||affection||love, fondness, attachment, liking, tenderness|
|2||287.1197||love||like||as, similar, love, alike, liking|
|3||235.0498||love||loves||affections, likes, love, lovings, wants|
|4||213.0396||love||loving||liking, love, wanting, affectionate, fancying|
|5||210.0095||love||want||need, wish, desire, lack, require|
|6||164.0093||love||fondness||affection, love, liking, attachment, tenderness|
|7||158.0498||love||loved||liked, wanted, love, fancied, deared|
|8||154.0093||love||darling||dear, beloved, sweetheart, love, pet|
|9||152.0194||love||liking||loving, like, wanting, equaling, wishing|
|10||143.009||love||attachment||affection, love, connection, devotion, fondness|
|11||141.0094||love||fancy||imagine, fantasy, think, imagination, conceive|
|12||134.0095||love||dear||beloved, darling, expensive, precious, costly|
|13||134.009||love||sweetheart||darling, beloved, sweetie, lover, dear|
|14||125.0096||love||beloved||dear, darling, lovely, sweetheart, sweet|
|15||117.0092||love||cherish||love, foster, fondle, nurture, nourish|
The relationships continue across all known words, with the lowest values of word pairs equaling 0 for the vast majority of word pairs across the graph (e.g., love and glioblastoma).
Similarly, one can map a single word to expressions, with related intensities.
|Rank||Intensity||Word||Synonyms||Synonyms of synonym|
|1||55.009||love||be fond of||love, like, appreciate|
|2||49.4096||love||to love||love, to like, like|
|3||39.4094||love||make love||have sexual intercourse, have sex, love|
|4||36.1088||love||to appreciate||appreciate, like, enjoy|
|5||34.4096||love||in love||bosom friend, amorous, love|
|6||34.1094||love||to like||like, love, to be fond of|
|7||28.0088||love||enjoy oneself||enjoy, have a good time, have fun|
|8||27.0092||love||want to||want, like, love|
|9||24.4091||love||fall in love||love, enamor, enamour|
|10||23.1494||love||loved one||dear, sweetheart, beloved|
|11||23.1093||love||to adore||adore, worship, love|
|12||21.0092||love||care for||look after, treat, take care of|
|13||17.0492||love||love affair||affair, romance, amour|
|14||17.0093||love||I love you||love, love you, like|
|15||14.4094||love||true love||sweetheart, love, lover|
Finally, one can map expressions to expressions, and calculate intensities.
|Rank||Intensity||Expression||Synonyms||Synonyms of synonym|
|1||68.6094||love affair||affair||matter, business|
|2||56.009||love affair||romance||fable, love affair|
|3||49.4096||to love||love||affection, like|
|4||42.109||love affair||amour||love, love affair|
|5||42.0089||in love||bosom friend||in love, mating|
|6||40.1082||make love||have sexual intercourse||have sex, screw around|
|7||40.0093||make love||have sex||have sexual intercourse, screw around|
|8||39.4094||make love||love||affection, like|
|9||39.019||love affair||liaison||connection, link|
|10||36.0093||in love||amorous||love, amatory|
|11||35.9096||to make love||make love||have sexual intercourse, have sex|
|12||35.9096||make love||to make love||make love, have sexual intercourse|
|13||34.4096||in love||love||affection, like|
|14||33.0093||in love||The Lovers||in love, bosom friend|
|15||31.0088||I Love Her||Blur (it’s a band)||I Love Her, blur|
Note that proper nouns can enter the graph (like song titles or band names, like Blur). Such distances can then be ranked in order. From an artificial intelligence point of view, the graph can be characterized as an unsupervised machine learning exercise (e.g., via multidimensional scaling, or MDS, but with potential asymmetries). A rank from one word to the next may be low in one direction, but high in the opposite direction. Bear to carry, versus carry to bear, for example, have differing rank order distances to/from each other.
Thus, when word vertices are connected by an edge, their level of “adjacency” can vary. In addition to simple edit distance metrics (e.g., Levenshtein or others), various intensity metrics (similar to degrees of membership in fuzzy sets) and adjacencies are captured by “overlapping cliques.” Overlapping cliques, as defined in TotoPoetry, exist when two words are frequently found in pairs (or triads, etc.) to describe a particular word or expression node or each other. For example, consider a first set for words that can describe “love,” like affection, adoration, and zero.
“Love” can signify “zero” in tennis, so both are in clique 1 with the node z1. Similarly, many words can describe infatuation = love, affection and adoration. Note that “zero” is not connected directly to z2:
The “strength” between w3 and z2, therefore, is rather weak (a poem about infatuation using the word zero may look odd to a reader having the above cliques in their mind). The relationship between z1 and w3 is “stronger.” The strength between z1 and w2 is probably greater still than between z1 and w3 (e.g., people are not likely to think of “zero” when they think of love but they are likely to think of affection). The strength between w1 and w2 is also rather strong given the co-occurrences (they are in 2 separate cliques that overlap in terms of these being members). So, the frequency with which a word describes another can be important, but also the extent to which two words seem to be used across cliques can be telling. If the above cliques are reconstructed hundreds of thousands of times across hundreds of languages, then direct and indirect intensity measures become apparent to and from all words in English, but also across all words within and across all languages (e.g. from French to Norwegian). One of the benefits of using edge values is that they allow asymmetry across poems authored in this manner. For example, a poem about “mavericks” may include references to beatniks, but a poem titled beatnik may not include a reference to mavericks. Similarly, edge values allow the poem to drift into metaphors, half rhymes, small but noticeable violations of hard rules (poetic license), puns or other surprising devices used in traditional poetry. The line in the TotoPoetry sonnet which defines the word “truth” ends in this couplet:
Okay, the truth is harsh, I horse you not,
I render now the fact, I’m just a bot!
The rhyme at the end is predetermined given the genre. The surprise poetic line is the first one. Clearly, the phrase “the truth is harsh” might have been plagiarized from existing text found on the Internet. The part “I horse you not” is however, more interesting. It is clearly meaningful and sensical. A query on Google for this exact phrase is found on blogs which have cited this poem. Within the graph, it equate “truth” with “not lying”. Lying is proximate to kidding, joking, fooling you, and horse play. Poets have a soft rule when faced with two words that might not follow iambic rules “horse play” = “stress stress”. One can drop a word, and the human reader’s mind can fill in the blanks. Which word to drop? The phrase “I play you not” would have worked. TotoPoetry chose “I horse you know” which seems to aliterate well with “harsh” – another rule within the genre.
Given a large graph, with edges having differing “intensities” and other indicators, poetic constraints (e.g. rules of structure, grammar, or verse) are then used to narrow the writing style to a feasible set. Coupled with usage frequency information, grammar rules/frequencies, parsed fragments “learned” and other “poetic” inputs, the program then optimizes (maximizes an objective function) using aggregate intensities (e.g. averages across words, lines, etc.) or similar “quality” measures to select which, of the virtually infinite number of poems possible, to “publish”. Since TotoPoetry is allowed to include fragments of existing text to be valued within the graph, it can thus mimic the practice of some traditional poets to “find” poetry in non-poetic extant literature. This is done to increase didactic impact for some forms or sub-genres.
TotoPoetry also generates author pseudonyms in a similar manner. For example, a poem about “heel” is authored by “James Wilson”. James is “proximate” to heel in a graph (i.e., James signifies "the heel holder"). Similarly for Wilson. In diamante or diamond poems, the family name is often based on the antonym of the dictionary headword or seed. The proper noun graph is rather weak for many words, given the limited meanings of names. TotoPoetry poems should be seen as the experimental fruit of an intersection between human linguistic history, economic optimization, and information technology, and not of an individual author or artificial brain. In fact, given the sheer volume of poems, no-one has read even one-tenth of one percent of those generated. The dictionary user typing in a word, therefore, is likely the first reader of the poems presented.
The benefits of TotoPoetry is the ability for poems to be generated on topics or subjects that traditional poets might ignore, for which an audience might exist. Likewise, using edge values, the process avoids the possibility of a poem reading as if a group of monkeys sat a keyboard … or avoids us having to wait for a group of monkeys to come up with something reasonable. It also helps avoid having the poems read like a Mad Libs exercise. This begs the question. Are graph-generated poems indistinguishable from traditional poems? In one sense they always will be different. Traditional poems often spring from human emotions or specific events known to the author. Graph-generated poems cannot have such inspiration, but can only reflect these, if at all, in the aggregate and to the extent that language reflects our shared emotions or experiences.
That being said, one might ask if graph-originated poems are “better” than traditional poems. For obvious reasons, this question can never be answered. From limited information, speed advantages aside, it seems that graph-generated poems are better than ones that the average person comes up with, if given the same task – TotoPoetry programs simply knows much more than the typical human and better follows the rules (and can find cool violations) than average authors ever could – basically, this may mean that the average person is not a good poet, or that traditional poetry is hard to write for non-poetic people (try writing a simple acrostic poem for the word “book” – it is not obvious).
However, from a Turing test point of view, preliminary informal work (read “not peer reviewed”) suggests that graph-generated poems are basically indistinguishable from traditional poems in blind reviews, and in some cases judged by many to be of better “quality” than traditional poems (e.g. where reviewers go through a revealed preference exercise, or rate poems using Likert scales). The results vary based on the age of the reviewers or authors, and any “priming” or sample matching designs that might take place prior to the assessment. For example, if the reviewer is asked to compare poems written by first graders (6-year old authors) versus TotoPoetry poems, then graph-generated poems are preferred, and the person does not suspect the generated poems are written by a computer program (comparing one diamante poem to another). As the age of the real human authors increases, so too do their writing skills, and they converge to the graph generated poem in terms of “quality”. The above assumes that the poems are written on the same topic (e.g., “hate”) using the same poetic form (e.g. “diamante”) and the same didactic purpose. Once a poem is revealed to be computer written, the reviewer tends to changes the ratings in various magnitudes and directions, depending on the person (i.e., the attribute “generated” has intrinsic qualities).
In some cases, reviewers show a strong preference for TotoPoetry poems compared to Shakespeare’s poems, before and after the revelation of authorship. One can attribute this to the fact that most people generally do not like Elizabethan sonnets in a general sense (not that the graph-generated poems are better – i.e. people may dislike both); likewise, someone who prefers Shakespeare over all other authors will never prefer TotoPoetry poems, whether they believe them to be written by computer or not. Persons who are indifferent will generally not suspect that a computer has written a poem, as the concept may not seem feasible (though this has changed recently due to ChatGPT and other generative approaches). If told that a computer can, in fact, write sonnets, their chance of picking one written by a computer increases (but not necessarily their preference for traditional poems over generated poems). The above relates to your “average” person on the street as an author or reviewer (not necessarily a poet laureate or English major), and the “graph poet” described above. In general, graph-generated poems, matching genre and topic, are no better than any others, but they are really not much worse either, unless you are a fan of a certain poet who uses a particular form or style you recognize – graph-generated poems cannot compete in this case. In a similar vein, some will stick to preferring graph-generated poems over traditional poems because they are authored using computer algorithms (e.g., they would see it as “cool”), not because the poems are any better in any other way. The real frontier test will be for Epic poems, not yet generated by TotoPoetry.
Summarizing, TotoPoetry poems are seen as indistinguishable from others in a blind review when reviewers are given the task of writing or evaluating poetry on the same topic, using the same poetic form designed for the same purpose (educational). Readers are thus not aware or can not independently detect that a poem might be authored by a computer program, if (1) they are not first told to look for a computer-authored poem, or (2) are told that a computer might be used, but can not detect which poem amongst many others was created in this way (beating random odds). In other words, if one asks someone to evaluate a graph-generated poem in terms of “what do you think?” – few if any will say that it is obviously written by a computer unless they are told of its origins in advance. When told that it has been generated by a computer, after the fact, then they tend to see how this might be the case (though this is mostly derived from the fact that most poetic forms looks formulaic). A similar reaction occurs when one is told a traditional poem is generated by computer, when it has not. Limited experience have been ad hoc; please feel free to use the poems on this site to conduct your own tests.
As time goes on, this website will be augmented for educational poetic forms that are fitting with the mission of the TotoPoetry dictionary. Unfortunately, it is faster to write graph-generated poems than to program them and load them on the server, so it may take time posting them all. TotoPoetry is being improved and tweaked as time goes on, so newer, perhaps more interesting or complex poems will appear on the site in the coming years. Programs are currently being investigated for TotoPoetry to endogenously invent its own poetic forms which might prove interesting as well.
The above raises an interesting question. TotoPoetry has focused on didactic poems simply due to the mission of this dictionary project. A recent Wikipedia entry defines didacticism as following:
Didacticism is an artistic philosophy that emphasizes instructional and informative qualities in literature and other types of art. Didactic art intends not primarily to "entertain" or to pursue subjective goals. The opposite of "didactic" is "non-didactic." If the artist is more concerned with artistic qualities and techniques than with conveying a message, then the work is considered to be non-didactic, even if it serves instructive or educational purposes. … The term "didactic" also refers to media that are "burdened" with instructive, factual, or otherwise "educational" information, sometimes to the detriment of a reader's (or viewer's) enjoyment.
The article goes on to state the following (on the version available August 2, 2009):
Some have suggested that nearly all of the best poetry is didactic.
This view does beg the question. Will computer-authored poems, be one day considered the “best poetry” given that algorithms can cover vastly more subjects and languages than a single or even a large group of authors? Time will tell.