{"id":5345,"date":"2019-01-08T10:22:44","date_gmt":"2019-01-08T15:22:44","guid":{"rendered":"https:\/\/afutureworththinkingabout.com\/?p=5345"},"modified":"2019-01-15T00:47:12","modified_gmt":"2019-01-15T05:47:12","slug":"of-predictive-algorithms-and-emojis","status":"publish","type":"post","link":"https:\/\/afutureworththinkingabout.com\/?p=5345","title":{"rendered":"Of Predictive Algorithms and Emojis"},"content":{"rendered":"

We do a lot of work and have a lot of conversations around here with people working on the social implications of technology, but some folx sometimes still don’t quite get what I mean when I say that our values get embedded in our technological systems, and that the values of most internet companies, right now, are capitalist brand engagement and marketing. To that end, I want to take a minute to talk to you about something that happened, this week and just a heads-up, this conversation is going to mention sexual assault and the sexual predatory behaviour of men toward young girls.
\n
\nSo here’s the fucking thing: An increasing number of text-integrated platforms have begun incorporating a “Suggested Responses” feature, whether that means Gmail’s autocomplete suggestions (or even their whole-cloth short sentence responses) or Facebook’s “Suggested Emoji’s” on mobile. Thing of it is, the algorithms that run these are, yes, learning, but certainly not yet robust or well-formed, especially in Facebook’s case. I say especially Facebook because, while Gmail has figured out how to achieve something like a suitable range of tone and tenor for their preloaded email responses, Facebook’s suggested range of emojis just offered me a range of heart-eyes, heart, fire, and other fanatically supportive options on a story about the sexual assault of an 17 year-old girl.<\/p>\n

\"\"

[Two rows of emojis (not the original inciting options) above respective comment entry fields on FB Mobile: Top Row Emojis: Blue Heart; Cry Laugh; Hysterical Cry Laugh; Heart; Fire; Black Heart Bottom Row Emojis: Cry Laugh; Hysterical Cry Laugh; Weeping; Hmmm; Eyes, Fire]<\/p><\/div><\/dt>\n<\/dl>\n

And why did it give me this range, you might ask? Well, I don’t have direct access to Facebook’s algorithms (and if they have their way<\/a>, i likely never will), but knowing what we tend to know, I can guess that it suggested these emojis because the Perpetrator of said assault was a famous person whose name is very frequently responded to with that range and tone of emojis, and the assault in question took place at one of his concerts.<\/p>\n

No, not that guy<\/a>. The other guy<\/a>.<\/p>\n

To clarify, the fame and adoration of the perpetrator<\/strong><\/em>\u2014the brand<\/strong><\/em>\u2014is given greater weight and precedence in the algorithm over and above the the fact that the story was about his assault of a 17 year-old girl<\/strong><\/em>. The Facebook emoji suggestion algorithm learned that “Drake+Concert=Extreme Joy And Adoration Emojis” even when in context with “Story About Sexual Assault.” And if you think about Facebook’s reinforcement techniques for your timeline, wherein you are increasingly likely to see the kind of content you’ve engaged with, before, then this makes perfect sense.<\/p>\n

Facebook wants you to click and like and fire emoji and heart-eyes emoji or whatever it is that makes you more likely to engage with their advertising partners and generate revenue for Facebook. I’ve got a book chapter coming out soon where I put it like this:<\/p>\n

And how was it trained? By exposure to Facebook\u2019s network patterns and the behavior of people in them\u2014networks which Facebook has been notoriously reticent to moderate for racist, disableist, sexist, xenophobic, transphobic, homophobic, or otherwise bigoted content. When humans train a system via uncritical pattern-recognition protocols applied to that system\u2019s continual exposure to bigoted datasets, humans will get a bigoted system.\u00a0 When humans then teach that bigoted system to weight its outputs for use in a zero-sum system like capitalist marketing and advertising, where every engagement is a \u201cgood\u201d engagement, the system will then exacerbate that bigotry in an attempt to generate the most<\/em><\/strong> engagement. Once that\u2019s done, the capitalist advertising-driven system made of the biases and bigotries it learned from humans will sell the amplified iterations of those biases and bigotries back to those humans<\/em><\/strong>. Then, the assessment of how those humans engage with those iterations (remembering, of course, that any click is a good click) will, again, inform the system\u2019s next modulations. And then the cycle begins again.<\/p><\/blockquote>\n

That is what i mean, and that<\/strong> <\/em>is why all of this is important.<\/p>\n","protected":false},"excerpt":{"rendered":"

We do a lot of work and have a lot of conversations around here with people working on the social implications of technology, but some folx sometimes still don’t quite get what I mean when I say that our values get embedded in our technological systems, and that the values of most internet companies, right […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[1081,1300,959,101,1363,278,1111,1002,1088,1112,418,419,1230,1362,1030,1158],"class_list":["post-5345","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-algorithmic-bias","tag-algorithmic-justice","tag-algorithmic-systems","tag-bias","tag-drake","tag-ethics","tag-facebook","tag-google","tag-implications","tag-implicit-bias","tag-invisible-architecture-of-bias","tag-invisible-architectures-of-bias","tag-science-technology-and-society","tag-social-construction-of-technology","tag-technological-ethics","tag-values"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5WByP-1od","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":5249,"url":"https:\/\/afutureworththinkingabout.com\/?p=5249","url_meta":{"origin":5345,"position":0},"title":"“We Built Them From Us”: My Appearance on the TEAM HUMAN Podcast","author":"Damien P. Williams","date":"February 22, 2018","format":false,"excerpt":"Earlier this month I was honoured to have the opportunity to sit and talk to Douglas Rushkoff on his TEAM HUMAN podcast. If you know me at all, you know this isn't by any means the only team for which I play, or even the only way I think about\u2026","rel":"","context":"In \"algorithmic bias\"","block_context":{"text":"algorithmic bias","link":"https:\/\/afutureworththinkingabout.com\/?tag=algorithmic-bias"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5295,"url":"https:\/\/afutureworththinkingabout.com\/?p=5295","url_meta":{"origin":5345,"position":1},"title":"At HPE: “4 obstacles to ethical AI (and how to address them)”","author":"Damien P. Williams","date":"July 8, 2018","format":false,"excerpt":"I talked with Hewlett Packard Enterprise's Curt Hopkins, for their article\u00a0\"4 obstacles to ethical AI (and how to address them).\" We spoke about the kinds of specific tools and techniques by which people who populate or manage artificial intelligence design teams can incorporate expertise from the humanities and social sciences.\u2026","rel":"","context":"In \"A Future Worth Thinking About\"","block_context":{"text":"A Future Worth Thinking About","link":"https:\/\/afutureworththinkingabout.com\/?tag=a-future-worth-thinking-about"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/web.archive.org\/web\/20201109005732\/https%3A\/\/www.hpe.com\/content\/dam\/hpe\/insights\/articles\/2018\/07\/4-obstacles-to-ethical-ai-and-how-to-address-them\/featuredStory\/How-to-fix-AI.jpg.transform\/nxt-1043x496-crop\/image.jpeg?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/web.archive.org\/web\/20201109005732\/https%3A\/\/www.hpe.com\/content\/dam\/hpe\/insights\/articles\/2018\/07\/4-obstacles-to-ethical-ai-and-how-to-address-them\/featuredStory\/How-to-fix-AI.jpg.transform\/nxt-1043x496-crop\/image.jpeg?resize=350%2C200 1x, https:\/\/i0.wp.com\/web.archive.org\/web\/20201109005732\/https%3A\/\/www.hpe.com\/content\/dam\/hpe\/insights\/articles\/2018\/07\/4-obstacles-to-ethical-ai-and-how-to-address-them\/featuredStory\/How-to-fix-AI.jpg.transform\/nxt-1043x496-crop\/image.jpeg?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/web.archive.org\/web\/20201109005732\/https%3A\/\/www.hpe.com\/content\/dam\/hpe\/insights\/articles\/2018\/07\/4-obstacles-to-ethical-ai-and-how-to-address-them\/featuredStory\/How-to-fix-AI.jpg.transform\/nxt-1043x496-crop\/image.jpeg?resize=700%2C400 2x"},"classes":[]},{"id":5082,"url":"https:\/\/afutureworththinkingabout.com\/?p=5082","url_meta":{"origin":5345,"position":2},"title":"From WIRED: “Tech Giants Team Up to Keep AI From Getting Out of Hand”","author":"Damien P. Williams","date":"September 28, 2016","format":false,"excerpt":"I spoke with Klint Finley over at WIRED about Amazon, Facebook, Google, IBM, and Microsoft's new joint ethics and oversight venture, which they've dubbed the \"Partnership on Artificial Intelligence to Benefit People and Society.\" They held a joint press briefing, today, in which Yann LeCun, Facebook's director of AI, and\u2026","rel":"","context":"In \"A Future Worth Thinking About\"","block_context":{"text":"A Future Worth Thinking About","link":"https:\/\/afutureworththinkingabout.com\/?tag=a-future-worth-thinking-about"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5227,"url":"https:\/\/afutureworththinkingabout.com\/?p=5227","url_meta":{"origin":5345,"position":3},"title":"Appearance on the You Are Not So Smart Podcast","author":"Damien P. Williams","date":"December 4, 2017","format":false,"excerpt":"A few weeks ago I had a conversation with David McRaney of the You Are Not So Smart podcast, for his episode on Machine Bias. As he says on the blog: Now that algorithms are everywhere, helping us to both run and make sense of the world, a strange question\u2026","rel":"","context":"In \"A Future Worth Thinking About\"","block_context":{"text":"A Future Worth Thinking About","link":"https:\/\/afutureworththinkingabout.com\/?tag=a-future-worth-thinking-about"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5899,"url":"https:\/\/afutureworththinkingabout.com\/?p=5899","url_meta":{"origin":5345,"position":4},"title":"ChatGPT is Actively Marketing to Students During University Finals Season","author":"Damien P. Williams","date":"April 4, 2025","format":false,"excerpt":"It's really disheartening and honestly kind of telling that in spite of everything, ChatGPT is actively marketing itself to students in the run-up to college finals season. We've talked many (many) times before about the kinds of harm that can come from giving over too much epistemic and heuristic authority\u2026","rel":"","context":"In \"A Future Worth Thinking About\"","block_context":{"text":"A Future Worth Thinking About","link":"https:\/\/afutureworththinkingabout.com\/?tag=a-future-worth-thinking-about"},"img":{"alt_text":"Screenshot of ChatpGPT page:ChaptGPT Promo: 2 months free for students ChatGPT Plus is now free for college students through May Offer valid for students in the US and Canada [Buttons reading \"Claim offer\" and \"learn more\" An image of a pencil scrawling a scribbly and looping line] ChatGPT Plus is here to help you through finals","src":"https:\/\/cdn.bsky.app\/img\/feed_fullsize\/plain\/did:plc:ybkylffhwhn2an2ic2lxh76k\/bafkreidh6mhffosfxhbgnxx6aybjycvgj3c2ygzto2xhzvsohdsv3g6evm@jpeg","width":350,"height":200,"srcset":"https:\/\/cdn.bsky.app\/img\/feed_fullsize\/plain\/did:plc:ybkylffhwhn2an2ic2lxh76k\/bafkreidh6mhffosfxhbgnxx6aybjycvgj3c2ygzto2xhzvsohdsv3g6evm@jpeg 1x, https:\/\/cdn.bsky.app\/img\/feed_fullsize\/plain\/did:plc:ybkylffhwhn2an2ic2lxh76k\/bafkreidh6mhffosfxhbgnxx6aybjycvgj3c2ygzto2xhzvsohdsv3g6evm@jpeg 1.5x, https:\/\/cdn.bsky.app\/img\/feed_fullsize\/plain\/did:plc:ybkylffhwhn2an2ic2lxh76k\/bafkreidh6mhffosfxhbgnxx6aybjycvgj3c2ygzto2xhzvsohdsv3g6evm@jpeg 2x"},"classes":[]},{"id":5316,"url":"https:\/\/afutureworththinkingabout.com\/?p=5316","url_meta":{"origin":5345,"position":5},"title":"My Appearance on The Machine Ethics Podcast’s A.I. Retreat Episode","author":"Damien P. Williams","date":"October 23, 2018","format":false,"excerpt":"As you already know, we went to the second Juvet A.I. Retreat, back in September. If you want to hear several of us talk about what we got up to at the then you're in luck because here are several conversations conducted by Ben Byford of the Machine Ethics Podcast.\u2026","rel":"","context":"In \"algorithmic bias\"","block_context":{"text":"algorithmic bias","link":"https:\/\/afutureworththinkingabout.com\/?tag=algorithmic-bias"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/img.youtube.com\/vi\/ownE2zxTN2U\/0.jpg?resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/posts\/5345","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5345"}],"version-history":[{"count":3,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/posts\/5345\/revisions"}],"predecessor-version":[{"id":5351,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=\/wp\/v2\/posts\/5345\/revisions\/5351"}],"wp:attachment":[{"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5345"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5345"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/afutureworththinkingabout.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5345"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}