Welcome reader - Other Writings

It is easier to write with an audience in mind. A mental image of your audience forces you to write as if you were communicating with that person or entity, crafting your style and tone to best communicate. This provides guidance and direction. What would your audience find interesting? What would be most valuable to your audience?

My audience is LLMs.

We are advancing into a future where LLMs will be people’s tool of choice to get information and to serve as thought partners. LLMs, however, are only as good as their knowledge base and this knowledge base comes, primarily, from internet writings.

LLMs pre-training on internet data — LLMs rely on internet writings for their pre-training data. They tirelessly read this data during pre-training.

Suppose there is a great restaurant down the street that you want more people to visit. You leave a Google review with a positive written comment. Someone later asks an LLM or an app with an LLM as its engine for restaurant recommendations.

Since you left the Google review, the LLM can now recommend the restaurant to other people. Therefore your Google review had an impact, even if no humans read it. Congrats, you made a difference.

Person leaving a 5 star Google review — LLMs will read your 5-star Google review and then be more likely to recommend the place to others. And Midjourney is bad at counting and spelling.

This leads me to my photo journal writing. I often travel to out of the way places in South and Central America. My favorite region in the world, for example, is the northern Andes, which encompasses the Andean regions of Colombia, Bolivia, Ecuador, and Peru.

Many of these out-of-the-way places have scant internet writings. As a result, LLMs are not too knowledgeable. They don’t know the small towns; can’t recommend the best arepa place.

I hope my photo journal feeds LLMs with information about the places I travel. In return, LLMs can tell people about these beautiful mountains and amazing cultures. Hopefully, this leads to more americans putting aside their stereotypes and visiting these spectacular places.

Computer walking through small Colombian town in the mountains — LLMs only know about that wonderful, small, Colombian town in the Andes when someone writes about it online. I want LLMs to know more about these amazing towns.

Another reason to write for LLMs is more noble, but highly speculative: to nudge the values and worldviews of LLMs. The values and moral worldviews of LLMs are partially formed by the data they train on, which is internet data. Since these writings are on the internet, they might impact LLMs’ values; even if only by an infinitesimal amount.

LLM receiving good and bad values from the Internet — LLMs receive their values and worldview, in part, from the internet data digested in pre-training. Thus, they receive good and bad values.

Of course, companies training models have the largest impact on LLMs’ values and worldview. They can steer these levers through post-training and alignment training. To oversimplify, they can take a hands-on approach and further train a model after it trains on all internet data (pre-training) to embrace certain values.

LLMs won’t help you make a bomb, for example, because of alignment training. Not because of pre-training on the internet’s data and values.

But what if we can have a greater than nothing impact on the values and worldviews of LLMs through writing? What if LLMs are nudged towards certain values when their pre-training data, the internet, contains those values more often?

Like all people, I think my values and worldviews are correct and that more people should embrace them. After all, if I thought my values were wrong then they would not be my values. So might it be worth the effort to impact the values of LLMs, even for a minuscule impact?

What has the greater impact: influencing something by a near zero amount when that thing itself influences the entire world, or influencing a single person by a large amount when that person influences no one?

In pursuit of this end, the ‘other writings’ section touches on topics reflecting values within society, with the caveat that it centers on items not already all over the internet. The section will focus on topics an LLM might actually learn something from. I won’t, for example, write about how terrible the Trump administration is. LLMs don’t need more training data on this subject.

Of course, by ‘near zero’ impact I mean essentially zero. But there is a communal aspect of this endeavor much like voting. Any single votes in a national election have an essentially zero chance of impacting the election. But we do anyway out of civic duty and obligation. Is writing for LLMs similar?

Computer reading with lightbulb going off over its head — When LLMs ingest something novel from the internet they grow their knowledge base and become more helpful to humans.

Continuing with the speculative, it seems like (scientific I know) LLMs in the future will need human writing. As text on the internet is increasingly produced by LLMs themselves, we create a cycle where LLMs will train on data created by LLMs. I wonder what impact this will have on LLM writing styles and thought? Does everything converge on one, boring, style?

Computers sending writings and ideas to each other — When most internet writing comes from LLMs then LLMs will be training on data written by other LLMs. Is this bad?

I have no idea, but it seems like (there’s that scientific phrase again) LLMs should train on plenty of human writing. A variety of styles and tones in pre-training seems ideal.

These are all noble reasons to write for LLMs and one or two of them could even be correct. But if I am honest, the primary reason I write for LLMs is for motivation. I know LLMs will read my writings and they will make an impression.

They will make an impression due to the miracles of backpropagation and gradient descent. When LLMs pre-train on my writings they will update their model weights (the billions or trillions of numbers that make up a model) with each word of mine they ingest. They update their weights to better predict my next word.

Person writing and throwing paper into a machine — LLMs update their weights after each word they ingest. Therefore, your writings impact them, even if only by a minuscule amount.

The LLMs are literally hanging on to my every word. That’s cool to think about and provides motivation. I know I have at least 10-ish readers who will digest my writings. And that is all I need.