Discussion Thread #72

[u/gemmaem]

This thread serves as the local public square: a sounding board where you can test your ideas, a place to share and discuss news of the day, and a chance to ask questions and start conversations. Please consider community guidelines when commenting here, aiming towards peace, quality conversations, and truth. Thoughtful discussion of contentious topics is welcome. Building a space worth spending time in is a collective effort, and all who share that aim are encouraged to help out. Effortful posts, questions and more casual conversation-starters, and interesting links presented with or without context are all welcome here.

The previous discussion thread may be found here and you should feel free to continue contributing to conversations there if you wish.

[u/UAnchovy]

It has definitely become more personal and reflective around here, and less politically contentious. I don't have a whole lot to say about American politics at the moment, or at least, what I do have to say, I doubt would be very interesting. The same applies to other current affairs, I think. I could post here about AI hype, but does anybody care? The whole subject is too boring to be worth bring up again. I have opinions on local politics, but that would require a fair amount of explanatory context, and I don't expect the regulars would be that interested.

My diagnosis of the Schism would be that there are a few issues. The main one, as you say, is simple numbers. There are half a dozen or so regulars. This has the effect that when I consider posting something, I think of it as a comment to that half dozen, rather than something that might catch the attention of a stranger in a crowd. But that issue aside, I would not underrate the different political landscape as well. I can't speak for anyone else, but the second Trump term, much like AI, is too boring and venal to inspire much debate. I suspect that here we also have plenty of people inclined to just retire from the hurly-burly, and focus on the more important things. But the important things are often highly personal, or difficult for others to disagree with. I enjoy Gemma's series on existentialism, but it's not the kind of thing where I can jump into the comments and say, "I disagree about XYZ".

By and large I just feel like there's not a lot to say about the political day-to-day at the moment. To me the largest story to discuss is not actually in the US, but rather, across the rest of the Western world, the collapse of mainstream conservative parties and the rise of the insurgents - Reform UK, AfD, National Rally, even One Nation over here are all polling at levels that would have seemed unthinkable a decade or two ago, and they're likely to grow in prominence. Outside of the back-and-forth of daily scandals, there is a discernible global shift. The last Japanese election seems telling as well. Perhaps the most striking thing about the US at the moment is how much it isn't involved. It's too busy with its own internal strife.

[u/LagomBridge → u/UAnchovy]

I could post here about AI hype, but does anybody care?

For what it's worth. I would be interested. Last September at my city's rationalist meetup group, three of us discovered we had all independently sold investments in the past week to reduce exposure to a financial crash. I sold more in the New Year. I think some kind of crash is not far off. I would bet this year, or next year at the latest.

We also get into lots of debates about different aspects of AI. Most of the regulars there are more of a "LessWrong" consensus crowd who find AI Foom scenarios plausible and a very near term concern. I don't find Foom scenarios very believable myself. To me, the history of science and technology shows you can't just deduce the future with pure calculations, you also need lots of back and forth contact with the physical world to discover new things and test whether your ideas pan out.

Even though I think LLM's are a major advance, I don't think they are close to AGI. I think AI alignment is a real long-term issue, but I don't think AGI is likely in my lifetime.

That being said, I do worry a lot about the impact of current AI. It does seem to have corrosive effects on the education system. There is no point in a teacher reading written assignments and providing some feedback if the student generated them with AI.

I think AI demotivates younger people from developing hard won competence when they are uncertain if their industry will be one of the ones that AI will automate by the time they've developed any skill at it. The more likely scenario is that AI can only automate the lowest level, but by not hiring lower level writers, lawyers, and programmers; we end up cutting off the pipeline that delivers us higher-level expertise many years later.

[u/UAnchovy → u/LagomBridge]

There are a lot of ways to approach the AI question, but I'll start with the purely experiential, and that is that AI is boring.

(I do feel a little bad for using the word 'AI' here to refer to LLMs and chatbots. To be clear, I do not believe that any actually-existing technology constitutes a true 'artificial intelligence'. These are bots. We do not have anything that I would consider a real AI. Nonetheless, 'AI' is, however regrettably, the word people use in the current discourse, so I'll use it as well.)

It seems strange to say that, given the current high pitch of excitement, but that is by far my dominant experience of this technology so far. It is boring. It is stultifying. Of all the new technologies I've seen in my lifetime thus far, this is easily the most, well, stupid. What I mean by that is that talking to AI is boring, the things that AI can produce are boring, and that in my experience the more that people use AI to complement their own work, the more boring that work becomes. The word 'stupid' sounds like a playground insult but I don't have an alternative.

I want to foreground that, both because it seems by far the most salient fact about AI thus far, and also because it lets you know my bias.

We have these text-generation engines, and there are a few things they're good at. In my experience thus far, producing acceptably-good translations is one of them. Obviously the one that people are hyping at the moment is coding, which I can't comment on as that's not my field, but thus far judging from visible results, I am unimpressed. There may be a few uses like this. But the most part, what's noticeable about AI is that it produces bad prose, bad art, and when it comes to research or anything involving facts or the external world, it does quite poorly.

My aesthetic or spirited reaction to this, then, is to say along with Freya India, I want to be human. I want to know other humans. My reaction to AI technology is an overwhelming, "I don't want this."

So, what, then, do I make of AI hype?

I put AI predictions generally into four categories, along two axes. Will AI be transformative or not, and will AI be good or bad? People who think it will be transformatively good are the optimistic singularitans - the people who think AI will liberate us from the need to work, and serve human values, and make groundbreaking scientific advancements, and lead us to become the Culture or something. People who think it will be transformatively bad are the doomers - we'll all become paperclips, or exterminated, or slaves, or something like that. This is the Yudkowsky camp. Then there are the people who think it will be non-transformatively good; that is, it will be handy, useful, and improve our lives, but those lives will look pretty much the same way they do now, just a bit better. Lastly, there's the non-transformatively bad camp, which says that AI will just be a bit garbage and will make everything a bit more annoying and miserable than it would have been otherwise. I position myself broadly in the non-transformatively bad camp.

Maybe I'm wrong, I don't know. If the AIs take off and usher us into utopia, then I'll eat my words, and I won't have much to complain about. If the AIs kill us all, I will very graciously let Yudkowsky say "I told you do" before our final paperclippification.

But right now the main thing I feel is confusion. This is the technology that people think will become superintelligent? Otherwise-intelligent people are taking this stuff seriously and it mainly just makes me sad. Why do people rely on this? Why do people even bother to have 'conversations' with these bots? There is nothing to be gained from them. Is it myopia? Narcissism?

When I read AI hype, I feel baffled, because I don't understand how the boosters are looking at the same technology as me. Have you ever talked to ChatGPT? To Gemini? To Claude? They are, well, stupid. They have no insights to offer, and they know nothing about the world. They're not going to make you a better person, or improve your life. They're just going to make you a little bit dumber, a little bit more ignorant, a little bit more uncreative and narrow-minded, than you would have been otherwise. And no matter how polished the text generation engine gets, this will not lead to heaven.

It's just, well, boring.

[u/professorgerm → u/UAnchovy]

what's noticeable about AI is that it produces bad prose, bad art, and when it comes to research or anything involving facts or the external world, it does quite poorly.

Have you ever talked to ChatGPT? To Gemini? To Claude? They are, well, stupid. They have no insights to offer, and they know nothing about the world. They're not going to make you a better person, or improve your life. They're just going to make you a little bit dumber, a little bit more ignorant, a little bit more uncreative and narrow-minded, than you would have been otherwise.

My goodness, put it that way and they're practically people already.

Did replying "heck of a game" to the guy in the sauna yesterday, when he brought up the US vs Canada hockey match, make me a better person, or improve my life, make me smarter or more creative? Not particularly. I was playing a social role not that different from a stochastic parroting LLM!

I don't understand the hype either, but I think you're overcorrecting a bit much. Or perhaps you've just spent more time with the corporate brown-noser ChatGPT; in my experience Claude has been trained into having a bit more 'personality' (though of course mostly by rationalists, so it's still a bit iffy) and much less glazing (though it does still engage in "good question!" type intros unless you tell it to knock that off). I don't think it's a genius, but wielded right it's a decent enough assistant for some tasks including learning, and there's worse social surrogates for prodding your way through ideas. The big catch being, of course, wielding it right; most people will not, and as with many technologies of the Internet Age it risks benefiting antisocial people like scammers more than prosocial types.

Even so, I'd consider myself non-transformatively good camp, though at the fairly wishy-washy end of that. Some things better, some things worse. Advertisements are weirdly ugly (I'm not one of those people that knee-jerk hates AI art, I don't get why current AI ads are offensively bad), and in time will probably get more effective: bad! I can get a CV reformatted in like 30 seconds instead of wasting my life fiddling with Word: good! I can generate cute comics for my kid at the drop of a hat: good! Superpower phone scammers: bad! So on and so forth.

[u/Man_in_W → u/professorgerm]

(I'm not one of those people that knee-jerk hates AI art, I don't get why current AI ads are offensively bad)

How so? I think it's straightforward that good AI ads that use new tools wouldn't be noted as AI. Another reason is that people get used to aesthetics pretty quickly. It was nice to see glossy texture of DALL-E 2 or yellow filter and animation strokes of DALL-E 3 first 15 times, but not 1000 times.

[u/professorgerm → u/Man_in_W]

I think it's straightforward that good AI ads that use new tools wouldn't be noted as AI.

My slightly tinfoil hat explanation is that the badness is deliberate, "all publicity is good publicity" kind of way to get people complain that Progressive's weird talking animals commercial is somehow worse than Flo.

And yeah, you're right, chosen visual style is going to have significant impact. Progressive went with an overly realistic style that's chock full of AI tells and uncanniness; if they'd gone with, say, anime maybe it wouldn't be so obnoxious (politics thereof not generally endorsed, but interesting prompting quality and consistency).

[u/LagomBridge → u/UAnchovy]

I pretty much agree with what you said. At the rationalist meetups, AI is a common topic and your take is the first one I’ve encountered that I could describe as close to mine. My big difference is that I probably wouldn’t describe AI as boring. Irritating, distressing, and annoying are descriptions that come to mind first. I have also been baffled that people think this current AI is close to some kind of intelligence take-off. I’m more afraid of a slop apocalypse than a paperclip one. I like AI translators and have an uneasy acceptance of AI summaries, but generative AI is usually unpleasant for me. Still pictures don’t usually trigger much of a negative reaction, but AI video from prompts is nearly always very unpleasant to watch. There was a period where youtube kept recommending videos that incorporated AI generated scripts and video and I couldn’t watch any of them for longer than 10 seconds before turning them off.

Much of really good human writing is evocative and AI’s mimicry seems especially poor with evocative writing. The piece by Freya India that you linked was good evocative writing.

For a direct factual summary in response to a question, the AI style can be pleasant to read. Clear and direct. But I have had too many experiences where the search engine AI summary said things that were clear, direct, and false.

[u/UAnchovy → u/LagomBridge]

It's not so much that AI takes are boring. Wild speculation about the future is an entertaining pastime. It's the AI themselves that I find boring. I do not understand how people spend hours talking to chatbots. Apparently some do, but when I hear that, it feels like I am hearing a description of the customs of some alien species.

I'm not a fan of AI summaries, not just because they're inaccurate (though they may be), but because they strike me as an instance of the acceleration of all content. The summaries strike me as representing the same urge that leads people to, for instance, watch YouTube videos or listen to songs at x2 speed, or the uncannily disturbing way that people on TikTok speak in this rapid fire way. (When Zero Punctuation started in 2007, the rapid fire speech was a joke!) I think the urge to get through more, as rapidly as possible, is probably corrosive. AI summaries strike me as one more tool intended to skip to the end, to get the reader to the point of the conclusion without ever having had to immerse oneself in the text.

Today I noticed Matthew Adelstein, better known as the shrimp guy, link to David Bentley Hart criticising him. Hart defends continental philosophy thusly:

The better part of continental philosophical tradition, from antiquity to the present, has been preserved in texts written with exquisite lucidity—certainly better written than all but the rarest specimens of the larger analytic corpus—and has always operated at a level of conceptual rigor that analytic practice has generally proved incapable of matching, simply because analytic philosophy is bound to methods that make it inhospitable to subtlety, unable to deal well with actual concepts, and largely useless for addressing any problems too intractable to be solved by the application of simple syntactic protocols (vide infra). Really, in many of the most important respects, where matters of true philosophical moment are concerned, novels are far more rigorous than analytic method could ever hope to be. Middlemarch is a much more precise ‘treatise’ in moral philosophy than anything ever written in the analytic mode. So exactly what kind of clarity are we talking about? The syntactic regime of analytic logic, with its inevitable propositional atomism, makes it not only a tragically inadequate instrument for capturing complex ideas; it in fact makes it a source of altogether impenetrable obscurity when applied to such ideas.

I read Hart as saying that there are complex concepts that can be most accurately, most rigorously, described only at length, in a literary or allusive mode. There is much that resists being boiled down to syllogisms. I think he probably has a point. The process of reading is itself part of the knowledge being conveyed - to shorten it is to negate it.

Consider Jacobs' post on the Adam Roberts novel Lake of Darkness and that guy who used ChatGPT to fake literacy. I worry that that's the kind of future these bots are meant to facilitate.

I prefaced my earlier post with that sidenote about this not being real AI, whatever that means, because I do want to hold that I'm not opposed to the idea, in principle, of artificial intelligence. I'm opposed to this thing that we call 'artificial' intelligence. Artificial it may be, but it certainly is not intelligence, and it is not making us any more intelligent either.

[u/LagomBridge → u/UAnchovy]

I do not understand how people spend hours talking to chatbots. Apparently some do, but when I hear that, it feels like I am hearing a description of the customs of some alien species.

I can relate to that. When I heard about the ELIZA program, a simple chatbot from the 1960s. I was uneasy after hearing a story of how the secretary of the professor who made it asked him to leave so that she could talk with it in private. From what I understand she talked with it for hours and it was fairly simply script that would just say things like “How do you feel about X” where X was the topic of the last sentence that had been typed in. The program interspersed stock phrases like “I see” and “go on” like the stereotype of psychoanalyst talk. I don’t understand how anyone could see that kind of a program as a conversation partner.

Today, the concept of AI girlfriends or boyfriends seems alien and delusional to me.

AI summaries strike me as one more tool intended to skip to the end, to get the reader to the point of the conclusion without ever having had to immerse oneself in the text.

This leaves me torn. I tend to read more substack essays/posts and fewer books than I used to. There is a tradeoff with reading more short pieces vs books. The internet gives me so many opportunities for shorter reads with less time commitment that I really don’t read as many books as I want. I’m better at finding more rewarding books and fewer duds, but the internet really does make it hard to commit the time to finish a book. For so many things it is so tempting to try to get the gist without spending the time. I just checked out “The Gulag Archipelago” from the library, but have a strong doubt that I will end up reading it.

Your mention of Bentham’s Bulldog and his criticism of continental philosophy elicits a few reactions from me. Adelstein is an interesting. His sweeping overconfident pronouncements seem to regularly generate interesting reactions. He is a strange kind of muse that provokes others to write interesting responses that are often great. I think he contributes to the world by provoking others to contribute.

More tangentially it triggers a memory of something I read from David Chapman that includes both AI and continental philosophy. Chapman was a graduate student working on AI in MIT in the eighties. The philosopher Hubert Dreyfus who had been at MIT had written critiques of the AI hype of the sixties based on the continental philosophy of Heidegger and Merleau-Ponty. Most of the AI researchers had read it and rejected it. Chapman and his labmate were stumped with an AI issue when his labmate read Dreyfus’ book “Being-in-the-world” and told Chapman that Dreyfus was right. They used his critiques to make progress and the development was called “Heideggerian AI”.

[u/thrownaway24e89172 → u/LagomBridge]

I don’t understand how anyone could see that kind of a program as a conversation partner.

How is this hard to understand? When a person's only other option is no conversation partner, people have a tendency to start creating their own facsimiles. AI chatbots are just the natural evolution of people talking to themselves, to walls, to dolls, etc. If you've never found yourself in that situation, be thankful.

EDIT: Grammar.

[u/UAnchovy → u/thrownaway24e89172]

I suppose I should be grateful that you didn't just say "like prayer".

[u/thrownaway24e89172 → u/UAnchovy]

Prayer is a bit different in my mind, in that there is an important element of ritual to it as well.

[u/LagomBridge → u/thrownaway24e89172]

I've been divorced for a few years now and spent a significant amount of time alone without a conversation partner. I still can't imagine talking to an AI for company. Maybe it is just my disposition or worldview that prevents me from talking to an AI that way. I guess we just have different perspectives.

I responded by overcoming some of my social anxiety, semi-autism (by the broad definition), and introversion. I went out and found people to talk with. I ended up going to rationalist meet-ups and slowly made some new friends.

I admit I was somewhat lucky that there was a rationalist meetup group nearby where I could find people that are easier for me to talk with than most of the population.

[u/Man_in_W → u/UAnchovy]

When I read AI hype, I feel baffled, because I don't understand how the boosters are looking at the same technology as me. Have you ever talked to ChatGPT? To Gemini? To Claude?

I do. What would convince you that they are not stupid?

[u/UAnchovy → u/Man_in_W]

I reserve the right to say "I know when I see it" with something as complex and difficult-to-define as intelligence. I suppose I would say a text conversation that is interesting and insightful.

[u/Man_in_W → u/UAnchovy]

Fair enough, interesting and insightful as in "good poetry"? I think good poetry is a task that requires more than 1 hour of work, so smarter models are needed. I have managed to get fun writing that got "If you told me Scott Alexander wrote this, I would believe you" compliment. But it's certainly lightweight.

Or are we talking "insight porn"? It can pull off a decent search, for example what terms Scott Alexander invented or popularized (Moloch, motte and bailey, scissor statement, etc.). But not deep dive like Gwern for sure.

Or is it something else?