Algorithm Rebels Club #2 Agency. Look Inside Your Head, Not the Search Bar

Translated from Korean

Philosophy of Technology

We held the second session of the Netflix Love Song Gathering Algorithm Rebellion Club. This gathering's theme was 'Autonomy: Look Inside Your Own Mind Before the Search Bar'.

The content covered in the gathering was the following three items:

• Netflix documentary The Social Dilemma (Jeff Orlowski, 2020)
• Book: The Steal of the Century (Johan Hari, 2023)
• Book: Don't Dim My Light (James Williams, 2023)

Preparing for the second session involved a lot of deliberation. My biggest concern was: How could we address this topic while ensuring participants felt neither 'uncomfortable' nor 'afraid,' allowing for a refreshing discussion? And how could they find practical takeaways to apply to themselves amidst that refreshing atmosphere? That was my primary worry.

I also hesitated because I felt the phrase 'Big Tech companies are actually manipulating your minds this much!'!!' wouldn't resonate deeply with people.

Carrying these concerns, I revisited books and documentaries I'd read and watched before, scrutinizing them carefully. Through this process, the theme gradually took shape. I'll share the discussion from the meeting slowly.

1. The Curse Doll That Becomes Like You

• This is also a core theme highlighted in the Netflix documentary The Social Dilemma. It posits that each of us is like a stuffed doll. Big Tech companies feed us various information to test what "takes hold," gradually turning us into dolls that resemble ourselves. We then spend our time endlessly scrolling through information under their control. We opened the meeting with this story.

  > "Imagine there's a little cursed doll inside Facebook and Google's servers, modeled after us. At first, this doll isn't much like us. It's like a standard human model. But from that moment, they start collecting our click trails and everything about our online lives. They reconstruct all the metadata we don't think is important, and the cursed doll gradually becomes more like us."

  > "When we show up on YouTube, they wake up that doll and test it with hundreds of thousands of videos to see which ones it reacts to and moves on." — Aja Raskin (Infinite Scroll developer), The Attention Heist, p. 195

• But does this immediately make you think, 'Aha! Big Tech bastards are tormenting me too!'? I took the question posed in the documentary and threw it back at the members.

  • Question: So what exactly is dangerous? Scrolling, clicking. What's dangerous about making us watch more videos?

2. We Pay for Attention with Our Lives

• This is where technology ethicist James Williams' book comes in. In it, he quotes Bob Dylan to discuss why 'attention' is so crucial in our lives.

  > "If you wake up in the morning, go to sleep at night, and do what you want in between, you're a successful person."" — Bob Dylan

  > "Technology sometimes helps us do what we want, but sometimes it doesn't. When technology fails to help us, it destroys the 'focus' of attention. This destruction causes functional distraction, which in turn distances us from information or actions relevant to the task or goal at hand." — Don't Shine My Light, p. 87

• Ultimately, we encounter the crucial sentence: "We pay for the life we could have experienced with our attention."
• Up to this point, you might think, "Oh, yeah, I can see that perspective." Now, let's take it one step further.

3. Is This Not an Age of Attention Rather Than an Information Age?

• Before delving deeper into 'attention,' let's first discuss 'information.' We call our current era the 'Information Age.' Why is that?

  > "People living in specific eras like the Bronze Age or Iron Age probably didn't call their times by the names we use today. They likely used different names rooted in the fundamental assumptions of their era—assumptions they believed would never change." "In this sense, the fact that we call our era the Information Age somewhat baffles me. Information is the water we swim in. We consider information the raw material of human experience."

  > "In a world overflowing with information, the abundance of information signifies a scarcity of something else. Namely, whatever information consumes becomes scarce. That object is obvious. Information consumes the attention of the subject who receives it." "Therefore, the abundance of information creates a scarcity of attention, and the challenge of efficiently allocating attention among the wealth of information." — Don't Block My Light, pp. 38-40

• Exactly. Did people in the Bronze/Iron Age really call it that because of the abundance of bronze or iron? No. It was because bronze and iron were the defining core technologies and resources of that era. Who possessed that technology determined power. Today, what we call the 'Information Age' is overflowing with information. But is information truly the core resource of our era? James Williams uses the analogy of the 'Tetris game' to explain this.

> "If we can respond quickly enough, we can play this (Tetris) game indefinitely. What matters isn't abundance or infinity. The challenge of this game, and what we must ultimately contend with, is the increasingly rapid speed at which the blocks fall." "Similarly, the sheer volume of information only matters when we can keep pace with the speed at which it arrives. No matter how much information there is, if it comes at an extremely fast pace, it is meaningless." "In this sense, the primary danger posed by the abundance of information is not that information seizes or depletes our attention (as if our attention were a quantifiable finite resource), but that we lose control in the process." "In other words, the true crisis in Tetris doesn't occur when you place a block in the wrong spot (though that certainly makes things worse), but when you lose control over your ability to determine the block's orientation, rotate it, and stack it." — Don't Block My Light, p. 42

> "We call today's world the Information Age, but it might be more accurate to call it the Age of Attention."

• Like Tetris blocks falling endlessly, information is already abundant. Search it and it pours out; algorithms constantly push it at us. What's truly scarce, what the real competition is over, is attention. YouTube, Instagram, TikTok… These platforms aren't fighting over 'who provides more information.' It's 'who can hold your attention longer.' Just as the Bronze Age was named for bronze, wouldn't naming our era after the resource where real power lies make "Age of Attention" far more accurate?

• After discussing this far, I posed a question to the members: "In the past week, when was the moment you felt the biggest gap between what you 'intended to do' and what you 'actually did'? What were you doing then?"

• Various stories emerged. One shared how, despite wanting to read or do something productive after work as a reward for their fatigue, they ended up just scrolling on their phone until they fell asleep, regretting it the next morning. Another mentioned trying to avoid shopping but buying something they wanted anyway, using YouTube recommendations and year-end tax refunds as excuses. - We shared stories like wanting to watch the documentary 'The Social Dilemma' and read the book 'Don't Dim My Light,' but falling asleep scared by the documentary, or trying to read the book only to get sucked into other parts of the phone and never finish.

• So what's really happening? We discussed this in more detail.

4. Hostile GPS - Where is it taking us?

• If a car navigation system kept directing us to some random place instead of our intended destination, we'd get angry and replace the device immediately. But we shared the observation that with the navigation systems of the internet and smartphones, even though they constantly steer us away from our original task (destination) toward ads or sensational content (strange places), we don't get angry about it. Instead, we display astonishing patience and just let ourselves be led around.

  > "In a way, information technology should be the GPS guiding our lives. (Of course, in life, there are times when we don't know exactly where we want to go. That's when technology's role is to help us reach our destination in the way we want.)" "But imagine if the GPS device we use is hostile to us. You just bought a new GPS and installed it in your car. The first time you used it, it guided you efficiently to the correct location. But on your second trip, it took you to a spot slightly off from your intended destination." — Don't Block My Light, p. 32

• That thought: 'Huh, this isn't where I meant to go…' Could it be that we're walking along paths guided by a 'hostile GPS'?

  > "Yet we routinely endure the confusion caused by the technology guiding our journeys through informational space. In other words, we demonstrate astonishing patience toward the faulty navigation of the GPS guiding our lives—the information and communication systems that steer so much of our thinking and behavior today." — Don't Block My Light, p. 33

• Here we looked at the famous quote from The Social Dilemma together.

• In exchange for the free services we use (KakaoTalk, SNS, etc.), companies sell our attention to advertisers. Technology is not a neutral tool, but a 'persuasive technology' meticulously designed to steer our thoughts and actions in the direction corporations desire. Companies collect our data to create digital avatars (cursed dolls) of ourselves in the digital world, using them to predict and manipulate how we respond to stimuli.

  > "The tech industry doesn't design products; it designs users. This magical, universal goal system is never a neutral tool. It is a goal-based navigation system guiding the lives of living humans. It is an extension of our attention." — Don't Dim My Light, p. 34

> "In the broadest sense, all technological design is persuasive. Every design steers thoughts or actions in one direction or another. There is no such thing as neutral technology. Every design reveals specific goals and values, shaping the world in particular ways." "A neutral steering wheel is actually a contradictory concept. While holding the wheel straight may help maintain course, it doesn't steer the ship anywhere. The same applies to technology." — Don't Dim My Light, p. 58

• So, in truth, products from the tech industry that seem 'designed for us' are designed not for us or our lives, but (obviously) to increase 'user engagement'. Everyone working in the IT industry understands this. Therefore, it might be a natural consequence that we often feel, 'Huh? This isn't what I meant to do?' Because that's how it was designed.
• In the documentary, Tristan Harris asks, "How can you wake up from the Matrix if you're not even aware you're in it?" He suggests we might all be living in our own Truman Shows.

• Later, I asked members about moments when their patience ran out with this shoddy navigation, or when they thought, "Why am I here?" I compared technology leading us to unintended destinations (ads/addiction) rather than our desired ones (information/action) to a "broken navigation system." This question was similar to the first but delved a bit deeper. Members shared various stories. It became clear that it's not just social networking services anymore; with AI features now integrated, our GPS is gradually breaking down.
  • (Myeongin) I went on Instagram curious about a friend's updates, but instead of their posts, I was flooded with ads and irrelevant content, which made me furious. I ended up deleting the app from my home screen and now only access it by searching for their ID when needed.
  • (Yuna) My patience runs out when using ChatGPT. I want precise, machine-like answers, but it gets frustrating when the AI overinterprets context or arbitrarily assumes user intent to offer advice (pretending to have a mind of its own).
  • (Mijeong) The YouTube Music algorithm knows me too well. I felt trapped in a cycle where I wanted to hear new songs, but it kept recommending only songs I'd listened to before or similar tastes, which was frustrating.
  • (Doo-seung) After wasting 5-6 hours bingeing on TikTok in the past, I deleted the app. Knowing I'm vulnerable to short-form video addiction, I deliberately install it only on my tablet to avoid it.
  • (Hyo-na) I didn't use Instagram until I started for networking, but following notifications keeps me scrolling. I also waste time on Naver Search or YouTube, forgetting my original purpose and just watching recommended content.
  • (Minsuk) He expressed strong dislike for Naver's AI-based recommendation feature that pops up suggested content when you type a search term. He feels exhausted by its design that makes you forget your original purpose and keeps you scrolling.
  • (Taehoon) countered, "Isn't it precisely because we're unaware we're exercising patience that we're actually demonstrating remarkable patience right now?"
• In other words, he made the sharp observation that we don't feel we're 'enduring' technology's inconveniences or guided behaviors; we mistakenly believe this is the natural state, which paradoxically allows us to maintain astonishing patience.

5. Attempts to Find Light, and the System's Victory

• So what should we do now? There are attempts to resist algorithms, like turning off YouTube viewing history or doing digital detoxes. But perhaps more fundamental is realizing what we are losing. Earlier, we said, "We pay for the life we could have experienced with our attention." Therefore, reflecting more on 'the life we could have experienced' could be an opportunity to ensure we use our precious 'attention' not to be dragged along by a broken GPS, but to steer our lives in the direction we truly desire.
• This conclusion might sound somewhat self-help-ish, but it's less about some imperative to "achieve the life I want" and more about a journey of reclaiming agency—a quest to discover what light we are losing.
• In Don't Dim My Light, tech ethicist James Williams introduces the story of Alexander and Diogenes, drawing parallels to the modern 'attention economy'.

• The great ancient Greek philosopher Diogenes lived in a barrel, practicing non-possession. One day, Alexander the Great, the most powerful ruler of his time who had conquered the world, came to visit Diogenes. Respecting Diogenes's reputation, Alexander asked:

  > "I will grant you anything you desire. Just name it."

• Sunbathing at the time, Diogenes replied with apparent annoyance:

  > "Then step out of my sunlight (Stand out of my light)."

• If we liken 'Alexander the Great' to a giant tech corporation or digital technology, we could see it like this. Possessing power and resources formidable enough to conquer the world, it approaches us pretending to be kind, saying, "Search for anything and I'll show you everything," "I'll give you all the information you want."
• Then, the 'sunlight' could be seen as our attention and agency, right? For Diogenes, sunlight symbolized not just warmth, but the 'light of truth' that allows us to see the world clearly and think for ourselves, and 'my attention'.
• We can interpret Diogenes' demand to Alexander the Great to "step aside" this way: No matter how convenient the features and information tech companies provide, if they ultimately block and interfere with what I should truly focus on (my life's purpose, my power of thought), then what we must tell them is, "Just don't block my view—step aside."
• Ultimately, the core message this book conveys is this: "We must reclaim our light against technologies that pretend to offer immense information or convenience while blocking our most vital resource—our 'attention (light)'—the light of our lives."
• James Williams introduces three forms of focus we must understand to avoid losing our light in Don't Block My Light:
  • 🔦 Spotlight: The Light of Focus [Action] "I'm going to the kitchen right now to make coffee." : If the spotlight is scattered or interrupted, we cannot perform short-term actions. It is the direct ability that propels our awareness and actions toward a task, making us do what we want to do.
  • ⭐ Starlight: The starlight of long-term goals [Orientation] "I'm going to start a business. I'm going to be a good person." : Losing Starlight means losing sight of long-term goals and beginning to forget where you were headed. The overarching ability that guides our lives 'by the stars' toward higher goals and values, making us become the person we aspire to be.
  • ☀️ Daylight, the sunlight of daily self-awareness [Metacognition] The sense of clearly observing one's surroundings: Losing daylight means failing to grasp who you are, what you wanted to do, or even where you wish to go. The fundamental ability (reflection, metacognition, reason, intelligence) that enables us to define our goals and values in the first place. It makes us 'want what we want'.

> "As a result of the feature he invented, over 200,000 human lifetimes—every moment from birth to death—are spent daily scrolling screens. This time would have been used for other activities without infinite scroll."

> "This time just vanishes. An entire life just vanishes. This time could have been spent addressing the climate crisis, being with family, or strengthening social bonds. Whatever it was, it could have been used to live a better life." — Aja Raskin (Infinite Scroll developer), The Attention Heist, p. 185

• Unbeknownst to us, we are losing these three lights (Spotlight, Starlight, Daylight). But fortunately, there were people who first sensed 'something is wrong' in this darkness and lit candles. They were within the army of Alexander the Great—that is, inside the giant tech companies. Like Aja Raskin, mentioned earlier. They realized the algorithms they created were obscuring people's 'light' and began fighting from within the company.

• The middle section of ⟨The Attention Trap⟩ (Chapters 6-9) and ⟨The Social Dilemma⟩ deal precisely with the struggle of these 'whistleblowers'. Let's explore what voices emerged within Silicon Valley against the massive wave of technology, and why they failed or became disheartened.

• Google — Tristan Harris (2013)
  • While working at Google, Harris created a 141-slide presentation titled "A Call to Minimize Distraction & Respect Users' Attention." It went viral internally at Google, viewed by tens of thousands of employees.
  • He argued that Google, Apple, and Facebook bear an "enormous responsibility" for people becoming immersed in their smartphones.
  • The result? He was given the title "Design Ethicist," but failed to drive meaningful change. He left in 2015 to found the Center for Humane Technology.
• Facebook — Chris Cox (2017-2018)
  • Chris Cox (then CPO) personally formed a task force in 2017 to study whether maximizing engagement contributed to political polarization.
  • They found a correlation with polarization and confirmed that reducing it would negatively impact engagement.
  • Cox fought internally for integrity ranking and Common Ground changes but lost most battles, ultimately leaving the company. With his departure, remaining project participants lost their champion, and the team disbanded.
• Twitter — META Team (2021-2022)
  • Twitter's Machine Learning, Ethics, Transparency and Accountability (META) team served as an internal watchdog auditing algorithmic bias and harm.
  • They achieved tangible results, such as halting an auto-crop algorithm found to exhibit racial bias.
  • However, in November 2022, shortly after Elon Musk's acquisition, all but one member were fired. On the same day, the entire Human Rights team was also fired, and the Accessibility team was disbanded.
• Meta — Massive Trust and Safety Layoffs (2023)
  • Meta also eliminated over 16 positions in the Instagram well-being group and over 100 positions related to trust, integrity, and responsibility.
• While several attempts were made within Silicon Valley companies, most ended in failure. Five years have passed since the release of the Social Dilemma documentary, yet the problems continue to worsen. Recently, the advancement of AI has even posed a serious threat. However, we must acknowledge that "ultimately, there is no one to blame." Because the "flaw" lies not in individuals' internal decision-making structures, but in the new architecture of complex, multi-acting systems.

> "Quality management guru William Edwards Deming stated: 'A weak system will always overwhelm good people.'" — Don't Dim My Light, p. 156

Finally, we concluded the second meeting by sharing our three personal lights with the members. Though everyone's were slightly different, it was an interesting time with diverse insights like 'thinking with my own strength' and 'becoming still.'

We concluded the meeting hoping today's discussion would serve as an opportunity to reflect on my relationship with technology, rather than causing discomfort or unpleasantness. Next time, we'll focus on the theme of 'relationship' and delve deeper into discussing my relationship with AI. It promises to be another interesting session.