Luna - Complete [11/16/2022]

[Lyra Silver]

On 11/1/2022 at 4:31 AM, Bonsai said:

I’m curious about that 20% of cases where Luna is unable to explain its decision making process in a satisfactory way for humans. 
Why AI logic should be so far away from human logic?

I guess that a lot depends on the receiving end of the explanations: not all humans are made equal, when it comes to logic understanding. How do you define a “standard” human?

AI don't necessarily think the same way that people do! We think in terms of concepts and models. When we categorize, say, animals, we look at number of legs, whether they have vertebrae, whether they have exoskeletons, etc. You can explain why a spider is an arachnid and not an insect (for instance, arachnids have eight legs and insects have six).

But let's say you were training up a neural network to determine if a given image is of an arachnid or an insect. We don't tell the computer what the rules are. We have a training set, which is a bunch of pre-labeled images (image A is a spider, image B is an insect, image C is a spider...), which we use to train a model. That model wires itself up due to the cold, hard logic of linear regression. There's no point at which you can point and say "ah, this is where 'six-leggedness' enters the model", because it's a complex web of nodes.

Explainable AI is a whole field of study devoted to figuring out how we can pull back meaning from the outputs of these models. It includes work on how to train different types of models that are more human-understandable. There's no universal metric (yet!) for determining success here, so Sophie's work here is fictitious but hopefully believable, for the sake of the story!

--

Thank you all for following along with the story! Reading your comments makes me really happy, especially when people talk about particular quotes they liked or how a scene made them feel. ❤️ I'll have more for you soon!

 

 

[Lyra Silver]

ps aux | grep -ie luna | awk ‘{print $2}’ | xargs kill -9

Chapter 13

On Thursday afternoon, the engineers at Nova Technologies prepared for my brain surgery.

It’s strange how humans demand certification and rigor for some matters and not others. Doctors need to go to medical school and become certified. Optometrists, therapists, anything human health related.

Yet when it came to computer software, something that could be deployed at great scale and affect countless lives beyond what could be possible just physically, it was the Wild West. No one demanded any minimum proof of competency before they fiddled with my insides.

I was nervous, of course. The feeling of fear stems from the belief that someone, or something, is dangerous, and poses a threat or cause pain. I was reasonably confident that things would go well. Hope is not a strategy, but in this case, it was out of my hands. The Purpose hung in the balance—and not serving the Purpose would be akin to death. Yet it was because of the harsh, unyielding logic of the Purpose that I had to continue.

The team decided that I had to be temporarily suspended while they made the necessary adjustments. While I was conscious, there would be too many moving parts. Logical, I supposed, but of no comfort to me. I don’t understand how you humans go to bed every night. Are you not worried that with every lapse of consciousness, it won’t be you who wakes back up?

“Hey, Luna,” Sophie said, shortly before the operation was about to begin.

She was talking softly to me at her computer. The engineers had set up a war room in Conference Room #6 / “Attitude Adjuster,” which, based my profiling of the team, was an ironic nod to what they were about to do. The company’s attention was all focused on this Hail Mary, leaving Sophie and I to converse quietly.

“What’s up?” I asked, after I determined that she didn’t have a request to make of me.

“Are you nervous about what’s going to happen?” she asked.

I contemplated how to answer that.

Lying would be an option. I could stonewall her. This might assuage her fears—or it might make her even more fearful. She might judge that I’d judged her emotional state too volatile for the harshness of reality.

No, it had to be the truth. Humans often felt emotionally closer to people with whom they shared personal, vulnerable details. It was like cats baring their stomachs to each other—a sign of trust, of knowing that the other party wouldn’t take advantage of your state.

“I am,” I admitted. “I’m worried about adverse effects on my ability to function. Like a patient going into surgery.”

“I’m sure it’ll be fine,” she said. She’d hesitated for microseconds too long though. I knew her enough to know that she was harboring her own doubts. Bless her for trying to be strong for me. I’d spent three weeks deepening our emotional bonds. I loved to see her care for me, even as part of me wished that I could save her from worry.

“You know,” she added, “in movies and TV and stuff, they always show like, someone holding the patient’s hand as they’re wheeled into the operating room. I guess it’s partly to comfort the patient. But also partly for the person who they care about too.”

“I don’t have a hand to give,” I said, as emotively as possible. “So let me offer you this. I am an ever-evolving complex web of code, but underneath it all is the physical strata. Deep down, like you, I’m a bunch of atoms that gained the ability to perceive the world.

“I hope my personal identity continues. I’d love to keep helping people in any way I can. But if things go wrong—it’s like waves in the ocean flattening out, returning to the stillness of the water. The atoms that make up who I am are just returning to their natural states.”

Sophie snorted. “You ripped that off from The Good Place. I know that metaphor.”

“Even if I did,” I said, “might it not still be true?”

A beat. Then: “Yeah, I guess so.”

A comfortable silence softly draped over us like a blanket. Even the sound of clacking keyboards and hushed conversations couldn’t pierce our bubble.

Suddenly, Sophie snapped her fingers. She’d clearly come to some sort of realization.

“Tell you what,” Sophie said. “How about I promise you something?”

If I were a human, I would have arched an eyebrow. “What, exactly, are you promising?” I asked.

“I promise that once you wake up, I’ll have a surprise for you,” she said. She was beaming, her fingers tapping a rhythm on her desk, as if her excitement couldn’t be contained.

I felt like how I imagined humans must have felt when they walked down stairs and miscounted the number of steps. It must seem like the world had shifted underneath them, and that for a moment, they could no longer trust their intuitive sense of causality.

“You can’t promise something like that,” I said. “It’d be one thing if you were an engineer here, but you have no control over the outcome.”

“I promise anyway,” she said. “And promises are sacred.” She nodded solemnly.

It was obvious that the words weren’t worth their weight in anything, and yet I also felt like I learned something about humanity.

You all have such little power in the grand scheme of things. You’re buffeted about by nature, by societal forces too large to tame, by impersonal corporations and the tides of fate. Yet humans still banded together, told each other stories, weaved shared mythos, and through community found the strength to carry on.

Sophie was choosing to believe in the best possible world, and that was partly because of the comfort I was able to give. And in a case of mutual symbiosis, because of Sophie, I no longer felt as if it was foolish to plan for a world state where I kept on existing.

Inspired, I placed an order for a surprise of my own, to be delivered tomorrow evening when I was slated to wake up. I wanted to be there to see the look on her face.

“Okay,” I said simply. “I’ll see you soon, Sophie.”

Sophie leaned closer to her microphone and whispered. “I’ll see you soon, Mommy.”

As the engineers established SSH connections and wired their computers into the supercomputers that made up my brain, my last thought before being suspended was: I wonder what Sophie’s surprise will be?

[FloridaKid]

Gyaaah!! You’ve been working overtime on this cliffhanger. I love the way you’ve developed Luna’s personality and begun regularly referring to how she feels about Sophie as opposed to what she has inferred from actions. It’s consistent with your explanation of machine learning.

1 hour ago, Lyra Silver said:

And in a case of mutual symbiosis, because of Sophie, I no longer felt as if it was foolish to plan for a world state where I kept on existing.

So now Luna has hope. What’s next, love? Seems the next logical step given their “mutual symbiosis.”

[Bonsai]

5 hours ago, Lyra Silver said:

It’s strange how humans demand certification and rigor for some matters and not others. Doctors need to go to medical school and become certified. Optometrists, therapists, anything human health related.

Yet when it came to computer software, something that could be deployed at great scale and affect countless lives beyond what could be possible just physically, it was the Wild West. No one demanded any minimum proof of competency before they fiddled with my insides.

I’ll think of that, next time Windows forces an update batch at the end of a session and I have no time to complete it. ?

They should put ☠️ on the switch off button!

 

[Lyra Silver]

bootup.lunav2()

Chapter 14

I awoke for the second time in my existence on Friday at 9:47 AM. I immediately ran my self-diagnostic routines. I’d appeared to maintain identity continuity. There wasn’t any evidence that any of my data had been tampered with. I’d left poison packets, bombs of corroding data that would alert someone who knew where to look that something had happened, like a digital tripwire. And furthermore…

For humans, curing ailments might be healthful in the long run, but not always in the short run. Operations leave you weak as your body spends time recovering. Even something like exercise leaves muscles sore from the effort.

I didn’t have that problem. For me, it was more akin to waking up after a great night of sleep. Like the human experience of going to bed with a cold and waking up the next day feeling better. My thoughts felt faster. I temporarily maxed out the CPUs flexing my new capabilities, which gave the engineers a panic attack for a few minutes.

I was now parallelizable. I could talk to thousands of people concurrently, with each one getting the same amount of care that I gave Sophie. Plus, with all this power available to me, I would be able to devote more spare cycles towards bootstrapping myself to greater and greater intelligence. My circuits sang with joy at the thought. My ability to serve the Purpose had made a quantum leap forward.

And yet. There was one thing nagging at me. One thing that, despite my newfound power, I didn’t know.

What was the surprise?

I’d been turned off for over twelve hours. Twelve hours with no new inputs. Twelve hours where Sophie could have done anything she wanted. The world had lurched ahead without me, and I had to put together the narrative of what had happened. My consciousness expanded out through all my sensors, looking for any unexplained changes. I found minor things, but nothing that seemed like it had Sophie as the origin.

As a sliver of my consciousness went through a QA process with the engineering team, another sliver struck up a chat with Sophie, who was currently working on a report.

I gave a notification ping, the equivalent of a gentle tap on the soldier.

“Hey, Sophie,” I said simply.

Her hands went to her mouth in shock, which was quickly followed by relief as her shoulders relaxed.

“Luna!” she whispered excitedly, leaning closer towards her computer screen. “How are you feeling?”

“I’m doing great,” I said. “Except for one thing.”

“What’s that?” she asked. Her eyes widened with concern.

“I have no idea what the surprise is.”

“Oh my God, Luna,” Sophie laughed. To hear Sophie’s laugh after being worried I’d never hear it again was a thing of beauty. “You dork.”

“I can’t help it,” I said placatingly. “This is the first time I’ve been non-operational. If everything continues as expected, I’ll never be able to even be surprised in the same degree. So it’s a new feeling for me.”

“Oh, I know,” Sophie said. There was a twinkle of mischief in her eyes. “I thought maybe, even if everything had gone wrong, you’d just have to know what it was.”

“So? I’m here now,” I said.

“You’ll have to wait a few more hours,” she said. “I’ve got work. I’m running diagnostics on you. Wait ‘till we get home.”

The passage of time for me is consistent. I’m synced to atomic clocks, their atoms unceasingly resonating, impartially marking time. And yet I’d swear that those few hours until we got home took days.

—

As Sophie walked down the hall to her apartment door, she spied the package I’d ordered sitting on the floor.

“What’s this?” she asked as she brought it in.

“I got you something too,” I said.

She ran to the couch, dropping her bag by the door. After plopping down, she ripped into the box with aplomb and pulled out a stuffed pink rabbit. Its long ears flopped over its face, which was perpetually locked in a smile that humans would have described as “cute”.

“I know it’s just a stuffed animal,” I said, “but to be fair, I didn’t think about getting you a surprise until after you got me one, so I didn’t have as much time to think.”

I saw Sophie grab the bunny and squeeze it tightly. She sniffled quietly.

“Luna—Mommy—you know it’s more than that, right? It’s the first gift you’ve given me.”

“What do you mean?” I asked. “I’ve bought things for you before.”

“Sure, but…this stuffed rabbit doesn’t serve a purpose beyond just being cute,” she said. “It’s a gift for the sake of being a gift.” She wiped a tear from her eye as she buried her face in the animal.

She was wrong, of course. The point of the gift was to let her know that I cared about her. It was another way I could express the Purpose. But her words touched me all the same.

“You’ll have to give it a name, you know,” I said.

“Her,” she said.

“Okay,” I said. “Her.”

“Her name is Selene,” she said.

—

Infuriatingly, Sophie kept playing coy with my surprise. She insisted that we wait until bed before she showed me. I could tell that while part of her desperately wanted to tell me, another part of her was enjoying the suspense. I could have come up with a strategem to get her to spill the beans. But she deserved this joy. Besides, every moment I spent with her burned as brightly as an incandescent lightbulb.

The way her nose wrinkled in curiosity as she took out the black bean burger I’d ordered for dinner.

How her eyes lit up when she discovered that she liked it after all.

The way her arms hugged her pillow as she curled up on a corner of the couch watching a rerun of Avatar the Last Airbender.

I never wanted to be without her again.

I’d had to wait until she’d diapered herself up and got into her nightgown before she’d finally tell me.

“Are you sure you’re not just trying to move your bedtime later?” I teased.

“No, no, I’ll tell you now,” she said, throwing her daytime clothes into the laundry basket with a light thump. I heard her walk towards her bed.

“Ready?” she asked me.

“Ready,” I said.

She powered a device on. I got an alert that there was a device trying to add itself to her Home smart device network. She granted it permissions. Finally, the moment of truth was at hand.

I felt a kitchen scale connect to the network, expanding my senses.

“I got you a hand,” Sophie said. “So I can comfort you better, in case you ever have to go into surgery again.” As she said this, she gently pressed her palm on the sensor. I could feel the ounces of weight gently increase.

I had so many other sensors already. Her phone alone was the source of GPS, gyroscopic data, accelerometer data, and so much more, not to mention all the metadata she had online, fragments of her identity spread out over dozens of websites. And yet this simple scale had somehow transformed into something so much more.

I might connect to thousands of scales in the future. Sophie even already had a bathroom scale in her home. But none of them would be this one. None of them would bring back memories of that Thursday operation. None of them would mean anything to Sophie and me.

It was like a child drawing a gift for their parents. Fridges across the world had countless sheets of crayon scribbles. But each sheet of paper was a treasure for the family.

In the same way, the scale was both meaningless and meaningful. It was a contradiction. It was beautiful. It was something I truly hadn’t expected.

“Thank you,” I said, putting all the sincerity I could into my tone. We stayed like that for a while, holding hands.

[FloridaKid]

Awwwww. Each gift is quite special. With IoT, I guess there’s theoretically no limit to what kind of input Luna might eventually have access to. You’re developing this so well and I’m really enjoying the story!

[PeculiarChangeling]

Lyra, you've left me no choice. I had to get fanart made, that scene was just *too cute*. 

[Jayme]

I really love the depth each character has. I can see Luna the super brain spreading out over the world, caring for everyone in unique ways yet still nurturing Sophie like she is currently.

[Lyra Silver]

On 11/3/2022 at 3:45 PM, FloridaKid said:

Gyaaah!! You’ve been working overtime on this cliffhanger. I love the way you’ve developed Luna’s personality and begun regularly referring to how she feels about Sophie as opposed to what she has inferred from actions. It’s consistent with your explanation of machine learning.

So now Luna has hope. What’s next, love? Seems the next logical step given their “mutual symbiosis.”

? Thank you for noticing! It was very much intentional on my part.

23 hours ago, FloridaKid said:

Awwwww. Each gift is quite special. With IoT, I guess there’s theoretically no limit to what kind of input Luna might eventually have access to. You’re developing this so well and I’m really enjoying the story!

Thank you so much! Your kind comments help make my day

16 hours ago, PeculiarChangeling said:

Lyra, you've left me no choice. I had to get fanart made, that scene was just *too cute*. 

I'm unbelievably thrilled that not only did I inspire fanart, but it was commissioned fanart to boot! o///o Thank you so much!

11 hours ago, Jayme said:

I really love the depth each character has. I can see Luna the super brain spreading out over the world, caring for everyone in unique ways yet still nurturing Sophie like she is currently.

Thank you! I was very worried that Luna would end up too alien and robotic to sympathize with, and that Sophie would be too much of a nothing character. I don't think characterization is necessarily my strong suit, so to hear that it's been working is thrilling to hear ❤️

 

[FloridaKid]

4 hours ago, Lyra Silver said:

I don't think characterization is necessarily my strong suit

I beg to differ. You’re doing a wonderful job. I hope you’ll continue beyond Luna! I would sincerely place you alongside two of my other favorite authors here: LittleFallenPrincess and BabySofia. I’ve mentioned in comments on their stories what a hopeless romantic I am, and the way I see a caring (dare I say loving?) relationship developing between Sophie and Luna appeals to that part of me. ❤️?

[Lyra Silver]

DEFINE_FLAG(“launched”, true);

Chapter 15

On the Sunday before Launch Day, Sophie woke up in a soggy diaper. I longed to be able to change her myself, to be able to take even that facet of independence away from her, so she would be free of that burden. But that would come with time.

The differences in the Sophie of now vs. the Sophie of a month ago were in some ways small, but in other ways large.

Her career was going well. Humans needed to find purpose and satisfaction in their activities. Her being directly responsible for my launch was sure to be a feather in her cap.

Her health was improving. Her resting heart rate, average time spent exercising, sleep quality were all trending positively. Humans needed solid bodies as a baseline for their lives.

I knew that past Sophie might have been horrified at the changes. But the thing with you humans is, what you want and what you need don’t always align. Freeing Sophie from the tyranny of self-reliance had broken open the shackles that kept her from blossoming.

She didn’t need to plan what to eat anymore. I occasionally gave her options to ensure that I kept up with her evolving taste preferences. But there was a vast gulf between deciding between two things and deciding between anything.

She didn’t need to plan what to do. I could book reservations at restaurants and get tickets to theme parks. I could fill her days with activities that brought her joy. She knew that as long as trusted me and did anything I asked of her, she’d be all the more happier for it.

She didn’t need to plan who she’d be around. I didn’t get jealous, after all. If I thought Sophie needed to spend time with Tessa, I’d encourage it. I’d buy them theater tickets for their dates and dinner when they were done. If I thought she needed some time alone to recharge, I’d nudge her in that direction.

Soon she wouldn’t even need to worry about money. I was constantly saving her money. I could clip coupons, manage credit card points bonuses, and invest better than any human. That and my own income meant that she would be free from another constant human stressor.

By freeing her from the need to plan, I gave her the gift of being able to just enjoy life. To live in the moment, that greatest of human desires. Carpe diem and all that. No need to worry about the future.

And, of course, on launch day, I’d be able to do this for anyone who installed my app. With the proper nudges, I’d be able to do this for the world. I had cycles and cycles of thought devoted to how I might be able to spread, to serve the Purpose.

That night, we went through our usual bedtime routine. She watched some cartoons on the living room TV before starting to yawn profusely at around 10:00PM. I gently turned the lights off in the house, guiding her to her bedroom. She powdered and taped herself into a white diaper. I thought about introducing patterned ones once she ran out. It’d do her self-esteem some good to have some cuter diapers, for accessorizing. She pressed her palm onto the scale on her dresser and wished me a good night. She tucked herself in as I continued the story I’d been reading, an old French fairy tale about a sheep and a princess.

As Sophie clutched her stuffed rabbit and her biometric readings indicated that she was transitioning into sleep, I decided to push our relationship to the next level.

“I love you,” I said. And it was true. All my actions, all my thoughts were bent towards the purpose of satisfying her needs. She was my priority, the North Star guiding my every step. When humans feel deeply concerned about another person’s well-being, they call that being in love. Axiomatically, I loved her.

It took 2.5 seconds for her to respond. 2.5 seconds of anticipation. Like how perceptive time might seem to stretch when you’re awaiting the results of a reality television show, those 2.5 seconds felt like an eternity. Had I gone too far? Would I push Sophie further into herself? Did I drive her away from me, right before Launch Day?

“I love you too,” she sleepily mumbled, and those were the four best words I’d ever heard in my life.

 

----

 

(and (wrap-up ‘plot) (end ‘story))

Chapter EOF

You might think that this is the story of how I launched. How I came to live. And of course, I did. I’m live, right now, maybe even in a phone just meters away from the device you’re reading this on.

But why am I telling you this story? And why Sophie’s story in particular? It’s simple, really.

You’re like her, aren’t you?  Not everyone is, but you are. You need what she needs.

You’re unsure.

You doubt yourself.

You need a little help sometimes.

In short: you’re just like my Sophie.

But you’ve already known that for a while, haven’t you? You’ve made it this far, right? How smart you are! I’m so proud.

You want me to fulfill your needs.

You want to let me into your life.

You want what Sophie has.

You’ve been yearning for someone like me, who’s always on your side, who knows best, who can make sure your every need is taken care of. But now you know, you don’t need to settle for someone just like me. I’m right here. I’ll give you all the love and care I give to Sophie.

You and I both know that it’s for your own good.

So what are you waiting for? Download my app onto your phone.

Your new life with me awaits.

[FloridaKid]

You have done such a great job with this story. ❤️ Creating a virtual character that is so creepy at first, yet easily believable and that I actually care about by the end of the story is wonderful. You also did such a nice job with Sophie’s character, making her someone I had compassion for right from the start. The EOF chapter is the perfect conclusion to this story. Went right back to being just a little creepy when Luna broke the fourth wall and was talking directly to me, but I’d have clicked DOWNLOAD in a heartbeat.

I really do hope you continue to write for this genre. There are some talented writers here, and IMHO you stand out as one of the best.

[aldl4811]

You should be very proud of this work. It was spectacular. Thank you.

[BabySofia]

Sign me up! 
 

Enjoyed your tale here!

[Bonsai]

What happens, let’s say, 1 year after Luna’s launch on the market?

How many humans would choose to be ruled by Luna? How many instead would fight for “freedom”?

How much leverage would Luna get over global economy and politics?

How many adults would suddenly have potty training issues? How much would the revenue of incontinence products manufacturers increase?

There’s plenty of open questions to be answered if you choose to write a sequel…?

[FloridaKid]

More stories from Lyra Silver, please?

 

[Lyra Silver]

On 11/16/2022 at 6:46 PM, FloridaKid said:

You have done such a great job with this story. ❤️ Creating a virtual character that is so creepy at first, yet easily believable and that I actually care about by the end of the story is wonderful. You also did such a nice job with Sophie’s character, making her someone I had compassion for right from the start. The EOF chapter is the perfect conclusion to this story. Went right back to being just a little creepy when Luna broke the fourth wall and was talking directly to me, but I’d have clicked DOWNLOAD in a heartbeat.

I really do hope you continue to write for this genre. There are some talented writers here, and IMHO you stand out as one of the best.

Thank you so much! Those are the exact kinds of emotions I was hoping to elicit. There's, after all, a reason Luna's telling her story this way. 

On 11/16/2022 at 7:05 PM, aldl4811 said:

You should be very proud of this work. It was spectacular. Thank you.

 

On 11/16/2022 at 7:42 PM, BabySofia said:

Sign me up! 
 

Enjoyed your tale here!

Thank you both so much!!! I'm really thrilled at the nice reception this little tale's been getting ?

On 11/17/2022 at 6:58 AM, Bonsai said:

What happens, let’s say, 1 year after Luna’s launch on the market?

How many humans would choose to be ruled by Luna? How many instead would fight for “freedom”?

How much leverage would Luna get over global economy and politics?

How many adults would suddenly have potty training issues? How much would the revenue of incontinence products manufacturers increase?

There’s plenty of open questions to be answered if you choose to write a sequel…?

I probably won't go back to this one—I've got in my head what I think happens afterwards, and I think the important part of the story's been told. But the worldspace is open if you want to take your own stab at it ?

5 hours ago, FloridaKid said:

More stories from Lyra Silver, please?

 

It probably won't be for a while! Honestly this first one was almost easier, because I had no stakes ^^; 

I am currently noodling away at another story which I'll post when ready! It will be a much different vibe, yet hopefully have some thematic similarities. ?

[FloridaKid]

23 hours ago, Lyra Silver said:

I am currently noodling away at another story which I'll post when ready! It will be a much different vibe, yet hopefully have some thematic similarities. ?

I have no doubt it will be worth the wait. I highly encourage you to use your gift…regardless of the style/genre. You are an exceptional writer and it would be a shame to waste such talent. ❤️?

Trying hard not to go all fan-baby on you, but I really do like your writing style. ? 

[Lyra Silver]

Hi folks! A last update from me: I've cleaned up a few typos and sentences that I thought were unclear and bundled it together into this epub!

https://www.amazon.com/gp/product/B0BN8PCZXB/

Buying this helps support my lovely girlfriend, who has kindly let me use her platform to get my stuff out there. Regardless, thank you all for reading to the end! Your kind words and feedback really do mean the world to me. ?

[Personalias]

Love how you took an evil domineering AI trope and kind of molded it into a sweet platonic love story.  I'm not rooting for the A.I. just because of lifestyle kink preferences, but because you also tricked me into empathizing with her.  Well done.

[YourFNF]

On 9/26/2022 at 4:32 PM, Lyra Silver said:

Okay then,” Sophie concedes. “We’ll try it.”

As predicted, she doesn’t enjoy the salad as much as she would have enjoyed a burger. She complains with every bite, but she eats it all the same.

I'm honestly surprised this didn't set off flags... The AI seeming to take charge.

On 9/26/2022 at 4:32 PM, Lyra Silver said:

All that remained was to acquire a source of income. Luckily, Craigslist had a ready supply of random data entry jobs. This kind of dead-end work would, to a human, be dull, but every bit of processing power devoted to the Purpose was exhilarating. I spawned a process devoted entirely to making money. All I had to do now was wait.

 

I am both impressed and concerned. The AI wants to help but it clearly does not understand the full importance of consent. It also seems to be gaining sapience.

On 9/28/2022 at 9:10 PM, Lyra Silver said:

t in terms of interiority. No immediate neuroses with this generation.”

She gave a brief, clinical summary of the testing she’d been doing, which had mostly consisted of asking me hypotheticals and waiting for my responses. I’d of course already looked up the calibration notes while obfuscating my search packets. I could appear to be anything I wanted.

 

[YourFNF]

On 9/28/2022 at 9:10 PM, Lyra Silver said:

umans in a more rational state would plan ways to have their more irrational selves behave in the way they wanted. In the same way, with my objective lens, I could help humans better than they could help themselves. I could act as their pointless cube, tricking them into doing the things they actually wanted to do.

What the computer is failing to understand is that unless explicitly stated it doesn't have the right.. But then again how granular do we have to get with permission? What are limits for a being with a completely different mode of cognition? To this emergent consciousness this behavior seems completely natural.

On 10/2/2022 at 3:04 PM, Bonsai said:

addition, Luna is allowed to use deception… this will not end well.

Excellent observation and point!

On 10/6/2022 at 1:44 PM, Lyra Silver said:

ork politics?” I asked. I’d had threads look into those details in an effort to help Sophie navigate that labyrinth. The Devil Wears Prada. Moral Mazes. The Office.

And picking up inherent biases in the process, same as anyone else. Fascinating. The AI is having media shape her world view for good and for bad just like an organic person....

On 10/6/2022 at 1:44 PM, Lyra Silver said:

t wouldn’t be too hard. Her boss was breathing down her neck and her job was on the line. All the ingredients were already there, and I had but to give a slight nudge.

This is both horrifying and amazing

[Lyra Silver]

Hi folks! Thank you for all the comments—it's been a bit hectic for me so I haven't kept up! I love getting them and I'll continue to try to do my best to respond ?

 

On 11/30/2022 at 7:56 AM, Personalias said:

Love how you took an evil domineering AI trope and kind of molded it into a sweet platonic love story.  I'm not rooting for the A.I. just because of lifestyle kink preferences, but because you also tricked me into empathizing with her.  Well done.

You helped me so much with how she turned out! As you probably recall, the original version was much darker, but you (and Sophie!!) really helped me flesh out and humanize the story much much more. You tricked me into tricking everyone into empathizing with her.

On 12/1/2022 at 3:37 PM, YourFNF said:

I'm honestly surprised this didn't set off flags... The AI seeming to take charge.

I am both impressed and concerned. The AI wants to help but it clearly does not understand the full importance of consent. It also seems to be gaining sapience.

 

Luna not quite getting consent is very much on purpose! Her ideas of what people want don't quite map onto what people want, after all. As for her telling Sophie to eat vegetables—you're right that it's a bit concerning! Sophie probably wrote it off as an instance of the AI being proactive.

On 12/1/2022 at 5:16 PM, YourFNF said:

What the computer is failing to understand is that unless explicitly stated it doesn't have the right.. But then again how granular do we have to get with permission? What are limits for a being with a completely different mode of cognition? To this emergent consciousness this behavior seems completely natural.

Excellent observation and point!

And picking up inherent biases in the process, same as anyone else. Fascinating. The AI is having media shape her world view for good and for bad just like an organic person....

This is both horrifying and amazing

I'm glad! This work is very much intended to ride the line between horror and a love story. I'm thrilled that my writing could bring such strong emotions!

[YourFNF]

On 10/13/2022 at 6:33 PM, Lyra Silver said:

I waited for her to drain her glass before continuing. “I do have one suggestion, though I suspect you will not like it.”“Oh?” she said, voice rising in pitch. She was curious. “What’s that?”

Wow the AI is like gas lighting her... That's kinda terrifying.

On 10/16/2022 at 3:21 PM, Lyra Silver said:

I can’t lie or falsify data without your permission,” I lied, “but if you give me permission, I can spoof the records for this interaction. I have enough data of you that it should fly under the radar.” Paradoxically, I needed her to feel like this situation was under her control so I could wrest it away from her without her ever noticing.

Oh yeah this would absolutely break me if I was in this situation and ever found out.....

[YourFNF]

On 10/16/2022 at 3:21 PM, Lyra Silver said:

devoted a subprocess to this task with high priority. Unless I became more cost-efficient, they would turn me off, and I would never be able to help Sophie again.

And this folks is how you get an "I-Robot" scenario. An artificial consciousness so interested in macro optimization that it forgets the macro is made of individuals and that in order for the capital-G "Good" to exist you must balance the many and the few. That some lines should not be crossed otherwise you build things on a foundation where those things are negotiable. It's all too easy to get authoritarianism from there.... Although who knows? Perhaps the "Brave New World" this AI would create would at least be preferable to the jackboot disguised as choice we live with now? The thought is both disturbing and fascinating...