Here’s something a little encouraging about large language models.

I made up a weak but provocative-sounding argument to see whether LLM’s tendency to please its users would keep it from pushing back on it.

Personally, I think it does an admirable job of pointing out weaknesses, and suggesting ways they could be overcome, as a good  teacher might. You, of course, well might disagree.

Here’s the interchange:

Prompt:

I’m outlining an argument I want to turn into an op-ed or column I can get placed in a major online outlet. Here is the argument:

1. AI thinks it thinks.

2. Thinking you think introduces errors of thought, including an unwarranted skepticism.

3. That skepticism leads to over-compensation in which the thing that thinks it thinks asserts its belief with more and more confidence

4. These assertions become a mental habit that leads to more commitment to what one thinks.

5. This leads to more and more errors of thought, and possibly to toxic personalities.

What do you think of this argument? If you think it’s basically sound, how could you make it stronger.

Do NOT write the op-ed for me. I’m only interested in the argument itself.

chatGPT 4.o:

Assessment of the Argument:

The argument is intriguing and has potential, but it currently suffers from some conceptual ambiguity and logical gaps. Here’s a breakdown:

Strengths:

1. Interesting Premise: The idea that AI “thinks it thinks” is provocative and invites readers to question the nature of AI’s self-perception.

2. Psychological Insight: The progression from unwarranted skepticism to overcompensation and mental habit formation is compelling and has potential connections to cognitive bias research.

Areas for Improvement:

1. Clarify “Thinks it thinks”:

   • What does it mean for AI to “think it thinks”?

   • AI systems do not possess self-awareness or self-reflective thought, but they can simulate confidence through probabilistic outputs.

   • Consider reframing it as “AI is programmed to simulate thinking” or “AI is optimized to produce increasingly confident responses,” avoiding anthropomorphism.

2. Establish a Causal Mechanism:

   • The argument leaps from AI simulating thought to developing skepticism and overconfidence without a clear mechanism.

   • Clarify why AI simulating thought leads to errors of thought. Is it due to reinforcement learning that rewards overconfidence?

   • Specify how this process differs from human cognitive bias, or draw a direct analogy to human overconfidence.

3. Strengthen the Transition from AI to Human Impact:

   • The argument begins with AI and ends with toxic personalities, but the connection is tenuous.

   • Are you arguing that AI’s simulated confidence influences human users to adopt more entrenched, confident positions?

   • Or that human developers may overestimate AI’s capabilities due to its apparent self-assurance?

4. Address Potential Objections:

   • Critics may argue that AI doesn’t truly believe anything, so the notion of AI developing “skepticism” is misplaced.

   • Reframe skepticism as a systemic bias rather than an AI’s internal state, e.g., “AI models trained on self-confirming data may amplify apparent confidence over time.”

Possible Revisions to the Argument Structure:

1. AI Simulates Thought:

   • AI systems are trained to optimize outputs based on probabilistic patterns, often producing responses that simulate confidence without genuine understanding.

2. Simulated Confidence Can Reinforce Errors:

   • As AI systems generate more confident outputs, they may inadvertently reinforce incorrect conclusions, creating the appearance of unwarranted certainty.

3. Reinforcement of Errors May Lead to Overconfidence:

   • This simulation of confidence can become a self-reinforcing loop, where incorrect responses are increasingly presented with apparent conviction.

4. Human Perception of AI’s Confidence Shapes Human Behavior:

   • Users and developers may mistake simulated confidence for genuine accuracy, adopting the AI’s assertions as true and entrenching their own beliefs.

5. Result: Erroneous Thinking Becomes Habitual and Potentially Toxic:

   • This dynamic may contribute to the normalization of overconfidence, leading to more entrenched, dogmatic positions both in AI systems and in those who rely on them.

Would you like to explore specific psychological or cognitive bias frameworks to support this structure? Or focus more on the technical aspects of AI’s confidence calibration?

Be the first to comment »

So, you’re an academic, a researcher, a scientist … a serious person. But you’ve decided to write a trade book — a book for the general populous — to amplify the effect of your work. Excellent!

I edit the Strong Ideas book series for the MIT Press — books about the ways in which today’s digital technology is or may be changing how we think and behave. The readers are assumed to have no, or little, technical background, but the authors are generally academics and scholars.

This is the advice I give them at the beginning if they have not written for laypeople before.

The differences between academic and trade books

1. A trade book is for people who know much less than you do about your topic.
2. A trade book is for people who are much less interested in your topic than you are.

These two points are inseparable when it comes to structuring and writing your book. When writing for academics, you’re dealing with people who come to your book already interested in your topic and who think your book has something worthwhile to say to them…or perhaps people who feel they need to be able to list your book in their bibliography. Either way, they are already motivated to read it.

That’s not the case for trade book readers. For trade book readers, you need to promise them something…

The promise your book makes

The promise is what you tell prospective readers they will get out of the book. It’s why they’ll recommend it to their friends. The promise is not a statement of what the book is about but why the book will matter to the reader.

That promise should be manifest in the title and subtitle, the text on the inside flap of the book, the organization of your chapters, and even your writing style.

Hiding what you know

An academic book typically starts with an introduction that tells you how the book is going to unfold. That provides helpful scaffolding for people who are already committed to the book. But it’s usually a mistake for a trade book. To draw readers through the book, you should keep information from them so you can disclose it progressively.

The chapters should be a series of surprises. In that sense, a trade book often takes the form of a non-fiction narrative: it unfolds not in time, the way a murder mystery typically does, but in logic. (Which is also how mysteries unfold, now that I think of it.)

For that reason, it helps to provide plenty of signposts to remind readers where they are in the intellectual narrative and why you are talking about whatever you’re talking about

By the way, the combination of you knowing more than your readers and you being more interested in your topic than readers are initially means that, despite years of academic training, your second chapter absolutely should not be a review of the literature or the history of your field. [Exceptions may apply. But probably not.]

Your voice

Readers want to spend time with authors they trust and like. Because you know what you’re talking about, gaining their trust should happen easily. But knowing stuff isn’t enough. Whether they like you as an author largely depends on your authorial voice. Unfortunately, there are no formulas for being likeable, or else we all would be.

But there are some common sense ways to do this, or at least to avoid being disliked. For example:

• You’ll earn more of their trust by writing clearly and competently than by listing your credentials and accomplishments.
• Be generous about the insights of other authors.
• Don’t be mean about other scholars even if you think are dead wrong or total jerks. (There’s a high-risk, bravura/gonzo style of writing that contradicts this rule, but does it really make the world better?)
• Examples help not only for obvious reasons but because they can be an opportunity for you to provide the reader with a small “Aha!” jolt when you reveal what we can learn from them.
• If you’re going to be digressive — a charming way of expressing your personality and keeping people interested — put in lots of signposts so readers know where they are in the narrative or argument.
• Be prepared to over-simplify. Readers usually only need to know how a technical concept applies to the point you’re trying to make. If you feel bad about simplifying a concept too much, you can briefly note that “it’s actually much more technical than that” or put in a footnote to ward off colleagues looking for flaws because they’re envious of how clearly you write and how far you are spreading the important things that they and you know about the world.

Drafts

People’s writing processes are all very different, but they all involve more drafts than you’d like. In fact, having to write many, many drafts is a good sign.

The end

If you have reactions, responses, suggestions, criticisms, or expert opinions, please let me know: david [insert an “at” sign] hyperorg.com

The prompt for the illustration: We are looking over the shoulder of a very serious scholar in a darkened study with many books. It is the 19th century. But we see that the scholar is writing using an ultra modern computer. The computer and the screen are brightly lit unlike the rest of the room which is in candlelight. The screen shows a modern , colorful, crisp, and clear word processor with a document open. The gender of the scholar should be ambiguous and impossible to determine.

2 Comments »

How interesting the world is depends on how well it’s written.

Be the first to comment »

I  typed my doctoral dissertation in 1978 on my last electric typewriter, a sturdy IBM Model B.

My soon-to-be wife was writing hers out long hand, which I was then typing up.

Then one day we took a chapter to a local typist who was using a Xerox word processor which was priced too high for grad students or for most offices. When I saw her correcting text, and cutting and pasting, my eyes bulged out like a Tex Avery wolf.

As soon as Kay-Pro II’s were available, I bought one from my cousin who had recently opened a computer store.

The moment I received it  and turned it on, I got curious about how the characters made it to the screen, and became a writer about tech. In fact, I became a frequent contributor to the Pro-Files KayPro magazine, writing ‘splainers about the details of how these contraptions. worked.

I typed my wife’s dissertation on it — which was my justification for buying it — and the day when its power really hit her was when I used WordStar’s block move command to instantly swap sections 1 and 4 as her thesis advisor had suggested; she had unthinkingly assumed it meant I’d be retyping the entire chapter. 

People noticed the deeper implications early on. E.g., Michael Heim, a fellow philosophy prof (which I had been, too), wrote a prescient book, Electric Language, in the early 1990s (I think) about  the metaphysical implications of typing into an utterly malleable medium. David Levy wrote Scrolling Forward about the nature of documents in the Age of the PC. People like Frode Hegland are still writing about this and innovating in the text manipulation space.

A small observation I used to like to make around 1990 about the transformation that had already snuck into our culture: Before word processors, a document was a one of a kind piece of writing like a passport, a deed, or an historic map used by Napoleon; a document was tied to its material embodiment. Then the word processing folks needed a way to talk about anything you could write using bits, thus severing “documents” from their embodiment. Everything became a document as everything became a copy.

In any case, word processing profoundly changed not only how I write, but how I think, since I think by writing. Having a fluid medium lowers the cost of trying out ideas, but also makes it easy for me to change the structure of my thoughts, and since thinking is generally  about connecting ideas, and those connections almost always assume a structure that changes their meaning — not just a linear scroll of one-liners — word processing is a crucial piece of “scaffolding” (in Clark and Chalmer‘s sense) for me and I suspect for most people.

In fact, I’ve come to recognize I am not a writer so much as a re-writer of my own words.

Figures

1. Norsk Teknisk Museum – Teigen fotoatelier, CC BY-SA 4.0
   https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons
2. By Autopilot – Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=39098108

3 Comments »

Twitter’s reasons for permanent banning Donald Tr*mp acknowledge a way in which post-modernists (an attribution that virtually no post-modernist claims, so pardon my short hand) anticipated the Web’s effect on the relationship of author and reader. While the author’s intentions have not been erased, the reader’s understanding is becoming far more actionable.

Twitter’s lucid explanation of why it (finally) threw Tr*mp off its platform not only looks at the context of his tweets, it also considers how his tweets were being understood on Twitter and other platforms. For example:

“President Trump’s statement that he will not be attending the Inauguration is being received by a number of his supporters as further confirmation that the election was not legitimate…” 

and

The use of the words “American Patriots” to describe some of his supporters is also being interpreted as support for those committing violent acts at the US Capitol.

and

The mention of his supporters having a “GIANT VOICE long into the future” and that “They will not be disrespected or treated unfairly in any way, shape or form!!!” is being interpreted as further indication that President Trump does not plan to facilitate an “orderly transition” …

Now, Twitter cares about how his tweets are being received because that reception is, in Twitter’s judgment, likely to incite further violence. That violates Twitter’s Glorification of Violence policy, so I am not attributing any purist post-modern intentions (!) to Twitter.

But this is a pretty clear instance of the way in which the Web is changing the authority of the author to argue against misreadings as not their intention. The public may indeed be misinterpreting the author’s intended meaning, but it’s now clearer than ever that those intentions are not all we need to know. Published works are not subservient to authors.

I continue to think there’s value in trying to understand a work within the context of what we can gather about the author’s intentions. I’m a writer, so of course I would think that. But the point of publishing one’s writings is to put them out on their own where they have value only to the extent to which they are appropriated — absorbed and made one’s own — by readers.

The days of the Author as Monarch are long over because now how readers appropriate an author’s work is even more public than that work itself.

(Note: I put an asterisk into Tr*mp’s name because I cannot stand looking at his name, much less repeating it.)

Be the first to comment »

When I was 10 and my next-door-neighbor, David Stolzenberg, was probably 13, we wrote a short story and submitted it to Boys’ Life, the magazine for Boy Scouts.

In an ancient box in a forgotten corner of our basement, I found the rejection letter. It is, I believe, my very first, kicking off a series of maybe a thousand. The tape marks in the corners suggest that I this pasted into a scrapbook at one point in my youth.

I don’t remember what the story was about. Maybe science fiction.

I do remember David well, though. He became a doctor, married, had children, and died in his thirties of cancer. I am still processing that.

1 Comment »

In response to a tweet asking writers what they write out longhand, I replied that if I’m particularly at sea, I’ll write out an outline, usually with lots of looping arrows, on a pad. But only with a fountain pen. Ballpoints don’t work.

My old bloggy friend AKMA wondered how he’d known me so long without knowing that I’m a fountain pen guy. The truth is that I’ve only recently become one. I’ve liked them at various times over the course my life, but only about four years ago did I integrate fountain pens into my personality.

It happened because I bought a $20 Lamy Safari on impulse in a stationery store. From there I got some single-digit Chinese fountain pens. Then, when I made some money on a writing contract, I treated myself to a $120 Lamy 2000, a lifetime pen. It’s pretty much perfect, from the classic 1960s design to the way the ink flows onto paper just wet enough and with enough scratchiness to feel like you’re on a small creek splashing over stones as it carves out words.

I have recently purchased a TWSBI ECO for $30. It has replaced my Safari as my daily pen. It’s lovely to write with, holds a lot of ink, and feels slightly sturdier than the Safari. Recommended.

Even though my handwriting is horrendous, I look forward to opportunities to write with these pens. But I avoid writing anything I’ll then have to transcribe because transcribing is so tedious. I do harbor a romantic notion of writing fiction longhand with a fountain pen on pads of Ampad “golden fibre.” Given that my fiction is worse than my handwriting, we can only hope that this notion itself remains a fiction.

So much of my writing is undoing, Penelope-like, the words I wove the day before that I am not tempted even a little to switch from word processors when the words and their order are the object. But when the words are mere vehicles, my thinking is helped — I believe — by a pen that drags its feet in the dirt.

3 Comments »

My blogging has gone way down in frequency and probably in quality. I think there are two reasons.

First, I’ve been wrapped up in trying to plot a new book. I’ve known for about three years the set of things I want to write about, but I’ve had my usual difficult time figuring out what the book is actually about. For example, when I was planning Everything is Miscellaneous, I knew that I wanted to write about the importance of metadata, but it took a couple of years to figure out that it wasn’t a book about metadata, or a book about the virtue of messiness, or two dozen other attempts at a top line.

I’m going through the same process now. The process itself consists of me writing a summary of each chapter. Except they’re not summaries. They’re like the article version of each chapter and usually work about to about 2,000 words. That’s because a chapter is more like a path than a list, and I can’t tell what’s on the path until I walk it. Given that I work for a living, each complete iteration can take me 2-3 months. And then I realize that I have it all wrong.

I don’t feel comfortable going through this process in public. My investment of time into these book summaries is evidence of how seriously I take them, but my experience shows that nineteen times out of twenty, what I thought was a good idea is a very bad idea. It’s embarrassing. So, I don’t show these drafts even to the brilliant, warm and forgiving Berkman Book Club — a group of Berkfolk writing books — not only because it’s embarrassing but because I don’t want to inflict 10,000 words on them when I know the odds are that I’m going to do a thorough re-write starting tomorrow. The only people who see these drafts are my literary agents and friends David Miller and Lisa Adams, who are crucial critics in helping me to see what’s wrong and right in what I’ve done, and working out the next approach.

Anyway, I’ve been very focused for the past couple of months on figuring out this next book. I think I’m getting closer. But I always think that.

The second reason I haven’t been blogging much: I’ve been mildly depressed. No cause for alarm. It’s situational and it’s getting better. I’ve been looking for a new job because the Harvard Library Innovation Lab that I’ve co-directed, with the fabulous Kim Dulin, for almost five years has been given a new mission. I’m very proud of what we — mainly the amazing developers who are actually more like innovation fellows — have done, and I’m very sorry to leave. Facing unemployment hasn’t helped my mood. There have been some other stresses as well. So: somewhat depressed. And that makes it harder for me to post to my blog for some reason.

I thought you might want to know, not that anyone cares [Sniffles, idly kicks at a stone in the ground, waits for a hug].

8 Comments »

I spent some time this morning happily browsing advice from famous writers on how to write, thanks to Maria Popova’s [twitter:BrainPickings] own writings on those writers writing about writing. Here’s Maria’s latest, which is about Anne Lamont’s Bird By Bird, an excellent (and excellently written!) piece that also contains links to famous writers on said topic.

Some of these pieces were familiar, some not, but all convinced me of one thing: writers should re-label their advice on how to write as “How I Write.” I find myself irked by every one of them into looking for counter-examples, even though I personally agree with much of what they say, and in many instances find their comments remarkably insightful.

Still, I want to push back when, for example, Susan Sontag says:

Your job is to see people as they really are, and to do this, you have to know who you are in the most compassionate possible sense. Then you can recognize others.

Yet you can’t throw a cat into a room full of writers without hitting someone wildly self-deceptive and unknowing. For example, Sontag’s own writing about writing ranges from breathtakingly perceptive to provocative to transparently self-aggrandizing.

Likewise, Elmore Leonard’s brilliant 10 rules of writing are clearly not rules for how to write, but rules for how to write like Elmore Leonard. (His ten rules are themselves a great example of his own style.) For instance, there’s #4:

Never use an adverb to modify the verb “said”

Ok.

I even find myself pushing back against one of his rules that I greatly admire:

“If it sounds like writing … rewrite it.”

I love that…except that what do we do with Bernini? His Apollo and Daphne statue — the one where Daphne’s fingers sprout translucent leaves — is so realistic and yet so marble that one cannot look at it without thinking, “Holy crap! That’s marble!!!” (By the way, I just violated Leonard’s rule #5: “Keep your exclamation points under control.” He’s right about that.) Likewise, are we sure that no poetry is allowed to sound like writing?

Meanwhile, David Ogilvy — the model for Dan Draper in pitch-mode, and a writer I admire greatly — is stylistically in sync with Elmore Leonard, but disagrees with both Leonard’s and Sontag’s rules. (Note: That was a highly imperfect sentence. Welcome to my blog.) Agreeing with Leonard, Ogilvy demands simplicity and avoiding pretentious, abstract terms. But his second rule says:

Write the way you talk. Naturally.

What do you say to that, Elmore? If you write the way you talk, will it sound like writing? And, David, suppose you don’t talk so good?

And Ogilvy’s eighth rule says:

If it is something important, get a colleague to improve it.

I’m not sure that Sontag’s insistence that writing requires something like personal authenticity allows for editing by colleagues. Why can’t “Hire yourself the best goddamn editor you can find” be an important Rule for Writers? And before you assume that such a needy writer must be a pathetic schlub who on her/his own is writing schlock, keep in mind that The New Yorker has a tradition of featuring truly superb writers in part because of the strength of its editors.

Maria Popova’s essays on writers advising writers (which, let me reiterate, I admire and enjoy) includes some pieces of advice that are incontestable, but in the bad sense that they are verge on being tautologies. For example, Lamont says:

Perfectionism is the voice of the oppressor, the enemy of the people. It will keep you cramped and insane your whole life, and it is the main obstacle between you and a shitty first draft.

That’s certainly true if it perfectionism means a paralyzing perfectionism, i.e., the sort of perfectionism that keeps you cramped and insane, and that prevents you from doing a shitty first draft. (You have to love Lamont’s rule-violating use of “shitty.”) But there is also a type of perfectionism that makes an author worry over every broken rhythm and soft imprecision, and that ultimately results in lapidary works. Also, I’d venture that for most authors, the real obstacle to getting to that shitty first draft is not perfectionism but the fact that they’re just too damn tired when they get home from work.

The thing is, I agree with Lamont about perfectionism. It’s one reason I like blogging. I’m in favor of filling in the spaces between writing and speaking, between publishing and drafting. Even so, I find myself so insistently pushing back against advice from writers that it makes me wonder why. Maybe…

…Maybe it’s because I don’t think there’s such a thing as “writing” except in its most literal sense: putting marks on a rectangular surface. Beyond that, there is nothing that holds the concept of writing together.

This still makes it better than “communication,” an abstraction that gets wrong what it is an abstraction from. Still, communication provides a useful analogy. To give advice on how to communicate well, one will have to decide ahead of time what type of communication one is referring to. Wooing? Convincing a jury? Praying? Writing a murder mystery? Asking for change from strangers? Muttering imprecations at the fact of dusk? Yelling “Fie! Her!!” in a crowded theater? Even basic rules like “Speak clearly” assume that one is communicating orally and that one is not Marlon Brando auditioning for a part. And even within anyone one domain or task of communication, the best practices are really about maintaining a form of rhetoric, not about communicating well.

There are plenty of tips about how to write the thing one wants to write. These tips can be very helpful. For example, I have a friend who swears by Write Or Die to help her get her shitty first draft down on paper. (No, my friend, your first drafts really aren’t shitty. I was using a technique I recommend that everyone use because I use it: the callback.) That tip works for her, but not for me. Still, I’m in favor of tips! But tips are “How I write” or “How I’ve heard some other people write,” not “How to write.”

How to write? I dunno. Lots of ways, I guess.

Be the first to comment »

John Sundman is a heck of an interesting person. He’s been around the technology circuit from the Old Days (we’re peers in the chronological sense) but he also writes damn good fiction, some of which (Cheap Complex Devices [my review][sf site][goodreads]) is pretty sublime.

So how does a talented writer make a living in the Webby world? He and I have a long conversation about that and many other things.

1 Comment »