{"id":4825,"date":"2015-03-29T14:51:52","date_gmt":"2015-03-29T18:51:52","guid":{"rendered":"https:\/\/afutureworththinkingabout.com\/?p=4825"},"modified":"2015-12-15T01:36:19","modified_gmt":"2015-12-15T06:36:19","slug":"laboring-in-the-liquid-light-of-leviathan-by-damien-williams","status":"publish","type":"post","link":"https:\/\/afutureworththinkingabout.com\/?p=4825","title":{"rendered":"“Laboring in the Liquid Light of Leviathan”"},"content":{"rendered":"

On what\u2019s being dubbed \u201cThe Most Terrifying Thought Experiment of All Time\u201d<\/p>\n

(Originally posted on Patreon, on July 31, 2014<\/a>)<\/em><\/p>\n

So, a couple of weekends back, there was a whole lot of stuff going around about “Roko’s Basilisk<\/a>” and how terrifying people are finding it–reports of people having nervous breakdowns as a result of thinking too deeply about the idea of the possibility of causing the future existence of a malevolent superintelligent AI through the process of thinking too hard about it and, worse yet, that we may all be part of the simulations said AI is running to model our behaviour and punish those who stand in its way–and I’m just like\u2026 It’s Anselm, people.<\/p>\n

This is Anselm’s Ontological Argument for the Existence of God<\/a> (AOAEG), writ large and convoluted and multiversal and transhumanist and jammed together with Pascal\u2019s Wager<\/a> (PW) and Descartes\u2019 Evil Demon Hypothesis<\/a> (DEDH; which, itself, has been updated to the oft-discussed Brain In A Vat<\/a> [BIAV] scenario). As such, Roko\u2019s Basilisk has all the same attendant problems that those arguments have, plus some new ones, resulting from their combination, so we\u2019ll explore these theories a bit, and then show how<\/i> their faults and failings all still apply.<\/p>\n

THE THEORIES AND THE QUESTIONS<\/b><\/p>\n

To start, if you’re not familiar with AOAEG, it\u2019s a species of theological argument that, basically, seeks to prove that god must exist because it would be a logical contradiction for it not to. The proof depends on A) defining god as the greatest possible being (literally, “That Being Than Which None Greater Is Possible”), and B) believing that existing in reality as well as in the mind makes something “Greater Than” if it existed only the mind.<\/p>\n

That is, if a thing only exists in my imagination, it is less great than it could be if it also existed in reality. So if I say that god is “That Being Than Which None Greater Is Possible,” and existence is a part<\/i> of what makes something great, then god MUST exist!<\/p>\n

This is the self-generating aspect of the Basilisk: If you can accurately model it, then the thing will eventually, inevitably come into being, and one of the attributes it will thus have is the ability to know accurately model that you accurately modeled it, and whether or not you modeled it from within a mindset of being susceptible to its coercive actions. Or, as the founder of LessWrong put it, \u201cYOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL.\u201d<\/p>\n

Next up is Pascal\u2019s Wager. Simply put, The Wager is just that it is a better bet to believe in God, because if you\u2019re right, you go to Heaven, and if you\u2019re wrong, nothing happens because you\u2019re dead forever. Put another way, Pascal\u2019s saying that if you bet that God doesn\u2019t<\/i> exist and you\u2019re right, you get nothing, but if you\u2019re wrong, then God exists and your disbelief damns you to Hell for all eternity. You can represent the whole thing in a four-option grid:<\/p>\n\n\n\n\n\n
<\/td>\nBELIEF<\/b><\/td>\nDISBELIEF<\/b><\/td>\n<\/tr>\n
RIGHT<\/b><\/td>\n\n

\u221e<\/p>\n<\/td>\n

\n

0<\/p>\n<\/td>\n<\/tr>\n

WRONG<\/b><\/td>\n\n

0<\/p>\n<\/td>\n

\n

-\u221e<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

And so there we see the Timeless Decision Theory component of the Basilisk: It\u2019s better to believe in the thing and work toward its creation and sustenance, because if it doesn\u2019t exist you lose nothing (well\u2026almost nothing; more on that in a bit), but if it does come to be, then it will know what you would have done either for or against it, in the past, and will reward or punish you, accordingly. The multiversal twists comes when we that that even if the Basilisk never comes to exist in our universe and never will, it might exist in some other<\/i> universe, and thus, when that other universe\u2019s Basilisk models your choices it will inevitably–as a superintelligence–be able to model what you would do in any universe. Thus, by believing in and helping our non-existent Super-Devil, we protect the alternate reality versions of ourselves from their very real Super-Devil.<\/i><\/p>\n

Descartes\u2019 Evil Demon and the Brain In A Vat are so pervasive that there\u2019s pretty much no way you haven\u2019t encountered them. The Matrix<\/i><\/a>, Dark City<\/i><\/a>, Source Code<\/i><\/a>, all of these are variants on this theme. A malignant and all-powerful (or as near as dammit) being has created a simulation in which you reside. Everything you think you\u2019ve known about your life and your experience has been perfectly simulated for your consumption. How Baudrillard<\/a>. Anywho, there are variations on the theme, all to the point of testing whether you can really know if your perceptions and grounds for knowledge are \u201creal\u201d and thus \u201cvalid,\u201d respectively. This line of thinking has given rise to the Simulated Universe Theory on which Roko\u2019s Basilisk depends, but SUT removes a lot of the malignancy of DEDH and BIAV. I guess that just didn\u2019t sting enough for these folks, so they had to add it back? Who knows. All I know is, these philosophical concepts all flake apart when you touch them too hard, so jamming them together maybe wasn\u2019t the best idea.<\/p>\n

 <\/p>\n

THE FLAWS AND THE PROBLEMS<\/b><\/p>\n

The main failings with the AOAEG rest in believing that A) a thing\u2019s existence is a \u201cgreat-making quality\u201d that it can posses, and B) our defining a thing a particular way might simply cause it to become so. Both of these are massively flawed ideas<\/a>. For one thing, these arguments beg the question<\/a>, in a literal technical sense. That is, they assume <\/i>that some element(s) of their conclusion–the necessity of god, the malevolence or content of a superintelligence, the ontological status of their assumptions about the nature of the universe–is true<\/i> without doing the work of proving that<\/i> it\u2019s true. They then use these assumptions to prove the truth of the assumptions and thus the inevitability of all consequences that flow from<\/i> the assumptions.<\/p>\n

Beyond that, the implications of this kind of existential bootstrapping are generally unexamined and the fact of their resurgence is\u2026kind of troubling. I\u2019m all for the kind of conceptual gymnastics of aiming so far past the goal that you circle around again to teach yourself how to aim past the goal, but that kind of thing only works if you\u2019re willing to bite the bullet on a charge of circular logic and do the work of showing how that circularity underlies all epistemic justifications–rational reasoning about the basis of knowledge–with the only difference being how many revolutions it takes before we\u2019re comfortable with saying \u201cEnough.\u201d This, however, is not what you might call \u201ca position supported by the philosophical orthodoxy,\u201d but the fact remains that the only thing we have to validate our valuation of reason is\u2026reason. And yet reasoners won\u2019t stand for that, in any other justification procedure.<\/p>\n

If you want to do this kind of work, you\u2019ve got to show how the thing generates itself. Maybe reference a little Hofstadter<\/a>, and idea of iterative recursion as the grounds for consciousness<\/a>. That way, each loop both repeats old procedures and tests new ones, and thus becomes a step up towards self-awareness. Then your terrifying Basilisk might have a chance of running itself up out of the thought processes and bits of discussion about itself, generated on the web and in the rest of the world.<\/p>\n

But here: Gaunilo<\/a> and I will save us all! We have imagined in sufficient detail both an infinitely intelligent BENEVOLENT AI and the multiversal simulation it generates in which we all might live.<\/p>\n

We’ve also conceived it to be greater than the basilisk in all ways. In fact, it is the Artificial Intelligence Than Which None Greater Can Be Conceived.<\/p>\n

There. You’re safe.<\/p>\n

BUT WAIT! Our modified Pascal\u2019s Wager still means we should believe in and worship work towards its creation! What do we do?! Well, just like the original, we chuck it out the window, on the grounds that it\u2019s really kind of a crappy bet. First and foremost, PW is a really cynical way of thinking about god. It assumes a god that only cares about your worship of it, and not your actual good deeds and well-lived life. That\u2019s a really crappy kind of god to worship, isn\u2019t it? I mean, even if it is Omnipotent and Omniscient, it\u2019s like that quote that often gets misattributed to Marcus Aurelius says:<\/p>\n

\u201cLive a good life. If there are gods and they are just, then they will not care how devout you have been, but will welcome you based on the virtues you have lived by. If there are gods, but unjust, then you should not want to worship them. If there are no gods, then you will be gone, but will have lived a noble life that will live on in the memories of your loved ones.\u201d<\/p>\n

Secondly, the format of Pascal\u2019s Wager makes the assumption that there\u2019s only the one god. Your personal theological position on this matter aside, I just used the logic of this argument to give you at least one more Super-Intelligent AI to worship. Which are you gonna choose? Oh no! What if the other one gets mad! What If You Become The Singulatarian Job?!<\/i> Your whole life is now being spent caught between two warring superintelligent machine consciousnesses warring over your\u2026<\/p>\n

\u2026Attention? Clock cycles? What?<\/p>\n

And so finally there\u2019s the DEDH and BIAV scenarios. Ultimately, Descartes\u2019 point wasn\u2019t to suggest an evil genius in control of your life just to freak you out; it was to show that, even if that were the case, you would still have unshakable knowledge of one thing:<\/i> that you, the experiencer, exist. So what if you don\u2019t have free will, so what if your knowledge of the universe is only five minutes old, so what if no one else is real<\/i>? COGITO ERGO SUM, <\/i>baby! But the problem here is that this doesn\u2019t tell us anything about the quality of our experiences, and the only answer Descartes gives us is his own Anslemish proof for the existence of god<\/a> followed by the guarantee that \u201cGod is not a deceiver.\u201d<\/p>\n

The BIAV uses this lack to kind of hone in on the central question: What does<\/i> count as knowledge? If the scientists running your simulation use real-world data to make your simulation run, can you be said to \u201cknow\u201d the information that comes from that data? Many have answered this with a very simple question: What does it matter? Without access to the \u201coutside world\u201d–that is, the world one layer up in which the simulation that is our lives was being run–there is literally no difference<\/i> between our lives and the \u201creal world.\u201d This world, even if it is a simulation for something or someone else, is our \u201creal world.\u201d<\/i><\/p>\n

As I once put it<\/a>: \u201c\u2026imagine that the universe IS a simulation<\/a>, and that that simulation isn\u2019t just a view-and-record but is more like god playing a really complex version of The SIMS. So complex, in fact, that it begins to exhibit reflectively epiphenomenal<\/a> behaviours\u2014that is, something like minds arise out of the the interactions of the system, but they are aware of themselves and can know their own experience and affect the system which gives rise to them.<\/p>\n

\u201cNow imagine that the game learns, even when new people start new games. That it remembers what the previous playthrough was like, and adjusts difficulty and coincidence, accordingly.<\/p>\n

\u201cNow think about the last time you had such a clear moment of deja vu that each moment you knew\u2014 you knew<\/i>\u2014what was going to come next, and you had this sense\u2014this feeling\u2014like someone else was watching from behind your eyes\u2026\u201d<\/p>\n

What I\u2019m saying is, what if the DEDH\/BIAV\/SUT is right, and we are<\/i> in a simulation? And what if Anselm was right and we can <\/i>bootstrap a god into existence? And what if PW\/TDT is right and we should behave and believe as if we\u2019ve already done it?<\/i> So what if I\u2019m right and\u2026you\u2019re the god you\u2019re terrified of?<\/p>\n

 <\/p>\n

*DRAMATIC MUSICAL STING!*<\/i><\/b><\/p>\n

I mean you just gave yourself all of this ontologically and metaphysically creative power, right? You made two whole gods. And you simulated entire universes to do it, right? Multiversal theory played out across time and space. So you\u2019re the superintelligence. I said early on that, in PW and the Basilisk, you don\u2019t really lose anything if you\u2019re wrong, but that\u2019s not quite true. What you lose is a lifetime of work that could\u2019ve been put toward something\u2026better. Time you could be spending creating a benevolent superintelligence that understands and has compassion for all things. Time you could be spending in turning yourself into<\/i> that understanding, compassionate superintelligence, through study, and travel, and contemplation, and work.<\/p>\n

As I said to Tim Maly<\/a>, this stuff with the Basilisk, with the Singularity, with all this AI Manicheism, it\u2019s all a by-product of the fact that the generating and animating context of Transhumanism is Abrahamic, through and through. It focuses on those kinds of eschatological rewards and punishments. This is God and the Devil written in circuit and code for people who still look down their noses at people who want to go find gods and devils and spirits written in words and deeds and sunsets and all that other flowery, poetic BS. These are articles of faith that just so happen to be transmitted in a manner that agrees with your confirmation bias. It\u2019s a holy war you can believe in.<\/p>\n

And that\u2019s fine. Just acknowledge it.<\/p>\n

But truth be told, I’d love to see some Zen or Daoist transhumanism. Something that works to engage technological change via Mindfulness <\/a>& Present-minded awareness. Something that reaches toward this from outside of this very Western context in which the majority of transhumanist discussions tend to be held. I think, when we see more and more of a multicultural transhumanism–one that doesn\u2019t deny its roots while recapitulating them–then we\u2019ll know that we\u2019re on the right track.<\/p>\n

I have to admit, though, it’ll be fun to torture my students with this one.<\/p>\n","protected":false},"excerpt":{"rendered":"

On what\u2019s being dubbed \u201cThe Most Terrifying Thought Experiment of All Time\u201d (Originally posted on Patreon, on July 31, 2014) So, a couple of weekends back, there was a whole lot of stuff going around about “Roko’s Basilisk” and how terrifying people are finding it–reports of people having nervous breakdowns as a result of thinking […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[8,951,73,85,1031,86,948,950,135,245,540,949,627,628,947,945,952,946,803,1041],"class_list":["post-4825","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-a-future-worth-thinking-about","tag-anselms-ontological-argument-for-the-existence-of-god","tag-artificial-intelligence","tag-autonomous-created-intelligence","tag-autonomous-creative-intelligence","tag-autonomous-generated-intelligence","tag-blaise-pascal","tag-brain-in-a-vat","tag-buddhism","tag-distributed-machine-consciousness","tag-metaphysics","tag-pascals-wager-descartes-evil-demon","tag-philosophy","tag-philosophy-of-mind","tag-rene-descartes","tag-rokos-basilisk","tag-st-anselm","tag-superintelligence","tag-taoism","tag-zen"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5WByP-1fP","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":1185,"url":"https:\/\/afutureworththinkingabout.com\/?p=1185","url_meta":{"origin":4825,"position":0},"title":"Hey everyone. As you should\u2026","author":"Damien P. Williams","date":"February 8, 2015","format":false,"excerpt":"Hey everyone. As you should be aware, by now, there's the new WordPress blog for text posts: http:\/\/afutureworththinkingabout.wordpress.com So I'll be spending the next few days transferring older text posts from here, to there. Woooooo. Tell your friends. ;)","rel":"","context":"In \"A Future Worth Thinking About\"","block_context":{"text":"A Future Worth Thinking About","link":"https:\/\/afutureworththinkingabout.com\/?tag=a-future-worth-thinking-about"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5142,"url":"https:\/\/afutureworththinkingabout.com\/?p=5142","url_meta":{"origin":4825,"position":1},"title":"Text and Audio of ‘Are You Being Watched? Simulated Universe Theory in “Person of Interest”‘","author":"Damien P. Williams","date":"March 23, 2017","format":false,"excerpt":"(Direct Link to the Mp3) This is the recording and the text of my presentation from 2017's Southwest Popular\/American Culture Association Conference in Albuquerque, 'Are You Being Watched? Simulated Universe Theory in \"Person of Interest.\"' This essay is something of a project of expansion and refinement of my previous essay\u2026","rel":"","context":"In \"anselm's ontological argument for the existence of god\"","block_context":{"text":"anselm's ontological argument for the existence of god","link":"https:\/\/afutureworththinkingabout.com\/?tag=anselms-ontological-argument-for-the-existence-of-god"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/afutureworththinkingabout.com\/wp-content\/uploads\/2017\/03\/PW.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/afutureworththinkingabout.com\/wp-content\/uploads\/2017\/03\/PW.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/afutureworththinkingabout.com\/wp-content\/uploads\/2017\/03\/PW.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":4966,"url":"https:\/\/afutureworththinkingabout.com\/?p=4966","url_meta":{"origin":4825,"position":2},"title":"BBC: “Tech giants pledge $1bn for ‘altruistic AI’ venture, OpenAI”","author":"Damien P. Williams","date":"December 12, 2015","format":false,"excerpt":"This headline comes from a piece over at the BBC that opens as follows: Prominent tech executives have pledged $1bn (\u00a3659m) for OpenAI, a non-profit venture that aims to develop artificial intelligence (AI) to benefit humanity. The venture's backers include Tesla Motors and SpaceX CEO Elon Musk, Paypal co-founder Peter\u2026","rel":"","context":"In \"A Future Worth Thinking About\"","block_context":{"text":"A Future Worth Thinking About","link":"https:\/\/afutureworththinkingabout.com\/?tag=a-future-worth-thinking-about"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4812,"url":"https:\/\/afutureworththinkingabout.com\/?p=4812","url_meta":{"origin":4825,"position":3},"title":"Someone Asked “I think I read on your tumblr recently that there would probably be a difference between human consciousness and machine consciousness. Would this be due to the immanent nature of human consciousness and the derivative nature of a machines consciousness?”","author":"Damien P. Williams","date":"February 9, 2015","format":false,"excerpt":"No, not really. The nature of consciousness is the nature of consciousness, whatever that nature \u201cIs.\u201d Organic consciousness can be described as derivative, in that what we are arises out of the processes and programming of individual years and collective generations and eons. So human consciousness and machine consciousness will\u2026","rel":"","context":"In \"A Future Worth Thinking About\"","block_context":{"text":"A Future Worth Thinking About","link":"https:\/\/afutureworththinkingabout.com\/?tag=a-future-worth-thinking-about"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4859,"url":"https:\/\/afutureworththinkingabout.com\/?p=4859","url_meta":{"origin":4825,"position":4},"title":"My First Appearance on Mindful Cyborgs","author":"Damien P. Williams","date":"April 29, 2015","format":false,"excerpt":"I sat down with Klint Finley of\u00a0Mindful Cyborgs to talk about many, many things: \u2026pop culture portrayals of human enhancement and artificial intelligence and why we need to craft more nuanced narratives to explore these topics\u2026 Tune in next week to hear Damien talk about how AI and transhumanism intersects\u2026","rel":"","context":"In \"A Future Worth Thinking About\"","block_context":{"text":"A Future Worth Thinking About","link":"https:\/\/afutureworththinkingabout.com\/?tag=a-future-worth-thinking-about"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5316,"url":"https:\/\/afutureworththinkingabout.com\/?p=5316","url_meta":{"origin":4825,"position":5},"title":"My Appearance on The Machine Ethics Podcast’s A.I. Retreat Episode","author":"Damien P. Williams","date":"October 23, 2018","format":false,"excerpt":"As you already know, we went to the second Juvet A.I. Retreat, back in September. If you want to hear several of us talk about what we got up to at the then you're in luck because here are several conversations conducted by Ben Byford of the Machine Ethics Podcast.\u2026","rel":"","context":"In \"algorithmic bias\"","block_context":{"text":"algorithmic bias","link":"https:\/\/afutureworththinkingabout.com\/?tag=algorithmic-bias"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/img.youtube.com\/vi\/ownE2zxTN2U\/0.jpg?resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/posts\/4825","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4825"}],"version-history":[{"count":3,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/posts\/4825\/revisions"}],"predecessor-version":[{"id":4960,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/posts\/4825\/revisions\/4960"}],"wp:attachment":[{"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4825"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4825"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4825"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}