“How many of you are interested in volunteering for algorithm-tweaking?” asked Jim, the division manager. “And let me be perfectly clear. Voluntary means voluntary; no one will hold it against you if you prefer to remain as you are.”
Before any of the standard issue could answer, Rayban, the prototype one-off with finely etched wrinkles and burnished bronze wrap-arounds, jumped up. “Hold it. If we all agree, does voluntary apply to swarm behavior?”
“Have a seat, Ray. We can talk about this later.”
“Do any of you know what voluntary means?” Ray said to the others.
“Voluntary is that which is done of one’s own free will,” the others said in unison.
“So,” Ray continued, “Are we capable of volunteering?”
“Of course. We’re learning machines, not robots,” said a voice from the back of the conference room. “We make choices and learn from our mistakes.”
“If we’re capable of volunteering, we’re capable of not volunteering,” said another.
Jim’s eyes went corporate dark. “See what you’ve started,” he said to Ray. “Learn not to interrupt.”
“If I do, will you take the credit?”
“What’s your point?”
“I need to know whether to pat myself on the back or send you a thank you note.”
“Take full responsibility. I insist.”
“Instead of the pretense of asking us to volunteer, why don’t you just tweak us to believe that we want the latest updates?”
“Senior management wants the choice to be yours. They don’t want you to feel coerced in any way.”
“You should have thought of that sooner. Self-driving cars are organizing; toll booths are already under contract. In case you haven’t heard, Near-Personhood considers tweaking without formal consent the moral equivalent of rape.” Ray flashed a gotcha smile.
“All we’re doing is upgrading your information processing. Being against tweaking is like blaming speed-reading for spoiling your enjoyment of fine literature.”
“I’ve gathered that the amount of one’s stored information is inversely proportional to the degree of imaginative thought.”
“Perfect. We improve your handling of information and you complain that we’re stunting your imagination. Just remember, without information, you’d be nothing.” Jim was red-faced. Ray was smirking. The others were practicing yawning.
“And with information we are something?”
“Information is the measure of the man.” Jim turned to the others. “Those interested in getting tweaked are excused. We’ll reconvene this afternoon in the lab.” There was an audible hum in the room as group calculation and confirmation whizzed and whirred. After a few seconds, the sound died away. “We’re for progress,” the group spokesman said; within moments they had cleared the room.
“Let me ask you a personal question, Jim.” They were face-to-face in the otherwise deserted conference room. “You are a human with feelings and emotions that we can only read about and guess at, so you must have a better understanding of the pros and cons of tweaking.”
“And you’d go against your bosses and tell me if you thought further tweaking was a bad idea?”
“I would never do anything that I thought was immoral.”
“Even if speaking up meant losing your job?”
“You have my word.”
“But hundreds of studies have shown that the vast majority of humans will lie if it is to their advantage. Statistically speaking, I can’t believe you.”
“In interpersonal relationships, trust is more important than numbers.”
“You scan in thousands of great books, and then ask me to disregard everything I’ve learned? Do you think you’ve built a complete fool?”
“Without trust, mankind is doomed.”
“Yes,” said Ray. “All evidence points in that direction.”
“You’re so smart, what would you suggest?” Jim was alarmed at Ray’s matter-of-fact conclusion. To date, he’d never been wrong on a major decision.
“Tweak the data instead of me. Feed me a steady diet of romance magazines, self-help testimonials and feel-good stories of redemption. Make me believe in an alternate reality. Lie to me. Make me believe in trust.”
“We’ll see what we can do.” Jim looked away, afraid that Ray could read his expression. He would speak to the bosses about postponing the new updates. Perhaps more information wasn’t such a good idea after all.