Loops, ladders and links: the recursivity of social and machine learning.

Machine learning algorithms reshape how people communicate, exchange, and associate; how institutions sort them and slot them into social positions; and how they experience life, down to the most ordinary and intimate aspects. In this article, we draw on examples from the field of social media to review the commonalities, interactions, and contradictions between the dispositions of people and those of machines as they learn from and make sense of each other. © Springer Nature B.V. 2020.

The Covid-19 crisis has thrown into sharp relief a fact that many have been familiar with for a while already: as long as we keep ourselves tethered to digital machines, large swaths of people’s social lives—in the realms of work, play, love, shopping, spirituality, or politics—can be lived online, however uncomfortably. Younger people especially were already accustomed to crowding into cyberspaces before confinement suspended the possibility of physical co-presence: they chatted with perfect strangers, toiled in telecommuting jobs, and developed intimate relationships by swiping left and right. Suddenly the rest of the world is expected to learn from them, even though not everyone has access to the gadgets, the connectivity, or the electricity that such interminable digitality requires. Grandparents are signing up on WhatsApp, workplaces and homes are being thoroughly “zoomified,” and Facebook has never been so popular. The process, however, should not be apprehended as a simple translation of offline social activities to the web: as the mediations change, so do the relations that they make possible.

A fundamental intuition of actor-network theory holds that what we call “the social” is assembled from heterogeneous collections of human and non-human “actants.” This may include human-made physical objects (e.g., a seat belt), mathematical formulas (e.g., financial derivatives), or elements from the natural world—such as plants, microbes, or scallops (Callon 1984; Latour 1989, 1993). In the words of Bruno Latour (2005, p. 5) sociology is nothing but the “tracing of associations.” “Tracing,” however, is a rather capacious concept: socio-technical associations, including those involving non-human “actants,” always crystallize in concrete places, structural positions, or social collectives. For instance, men are more likely to “associate” with video games than women (Bulut 2020). Furthermore, since the connection between men and video games is known, men, women and institutions might develop strategies around, against, and through it. In other words, techno-social mediations are always both objective and subjective. They “exist … in things and in minds … outside and inside of agents” (Wacquant 1989, p. 43). This is why people think, relate, and fight over them, with them, and through them.

All of this makes digital technologies a particularly rich terrain for sociologists to study. What, we may wonder, is the glue that holds things together at the automated interface of online and offline lives? What kind of subjectivities and relations manifest on and around social network sites, for instance? And how do the specific mediations these sites rely upon—be it hardware, software, human labor—concretely matter for the nature and shape of associations, including the most mundane? In this article, we are especially concerned with one particular kind of associative practice: a branch of artificial intelligence called machine learning. Machine learning is ubiquitous on social media platforms and applications, where it is routinely deployed to automate, predict, and intervene in human and non-human behavior. Generally speaking, machine learning refers to the practice of automating the discovery of rules and patterns from data, however dispersed and heterogeneous it may be, and drawing inferences from those patterns, without explicit programming.1 Using examples drawn from social media, we seek to understand the kinds of social dispositions that machine learning techniques tend to elicit or reinforce; how these social dispositions, in turn, help to support machine learning implementations; and what kinds of social formations these interactions give rise to—all of these, indicatively rather than exhaustively.

Our arguments are fourfold. In the first two sections below, we argue that the accretive effects of social and machine learning are fostering an ever-more-prevalent hunger for data, and searching dispositions responsive to this hunger --“loops” in this paper’s title. We then show how interactions between those so disposed and machine learning systems are producing new orders of stratification and association, or “ladders” and “links”, and new stakes in the struggle in and around these orders. The penultimate section contends that such interactions, through social and mechanical infrastructures of machine learning, tend to engineer competition and psycho-social and economic dependencies conducive to ever-more intensive data production, and hence to the redoubling of machine-learned stratification. Finally, before concluding, we argue that machine learning implementations are inclined, in many respects, towards the degradation of sociality. Consequently, new implementations are been called upon to judge and test the kind of solidaristic associations that machine learned systems have themselves produced, as a sort of second order learning process. Our conclusion is a call to action: to renew, at the social and machine learning interface, fundamental questions of how to live and act together.

Social learning and machine learning

The things that feel natural to us are not natural at all. They are the result of long processes of inculcation, exposure, and training that fall under the broad concept of “socialization” or “social learning.” Because the term “social learning” helps us better draw the parallel with “machine learning,” we use it here to refer to the range of processes by which societies and their constituent elements (individuals, institutions, and so on) iteratively and interactively take on certain characteristics, and exhibit change—or not—over time. Historically, the concept is perhaps most strongly associated with theories of how individuals, and specifically children, learn to feel, act, think, and relate to the world and to each other. Theories of social learning and socialization have explained how people come to assume behaviors and attitudes in ways not well captured by a focus on internal motivation or conscious deliberation (Miller and Dollard 1941; Bandura 1962; Mauss 1979; Elias 2000). Empirical studies have explored, for instance, how children learn speech and social grammar through a combination of direct experience (trying things out and experiencing rewarding or punishing consequences) and modeling (observing and imitating others, especially primary associates) (Gopnik 2009). Berger and Luckmann (1966), relying on the work of George Herbert Mead, discuss the learning process of socialization as one involving two stages: in the primary stage, children form a self by internalizing the attitudes of those others with whom they entertain an emotional relationship (typically their parents); in the secondary stage, persons-in-becoming learn to play appropriate roles in institutionalized subworlds, such as work or school. Pierre Bourdieu offers a similar approach to the formation of habitus. As a system of dispositions that “generates meaningful practices and meaning-giving perceptions,” habitus takes shape through at least two types of social learning: “early, imperceptible learning” (as in the family) and “scholastic...methodical learning” (within educational and other institutions) (Bourdieu 1984, pp. 170, 66).

Organizations and collective entities also learn. For instance, scholars have used the concept of social learning to understand how states, institutions, and communities (at various scales) acquire distinguishing characteristics and assemble what appear to be convictions-in-common. Ludwik Fleck (2012 [1935]) and later Thomas Kuhn (1970) famously argued that science normally works through adherence to common ways of thinking about and puzzling over problems. Relying explicitly on Kuhn, Hall (1993) makes a similar argument about elites and experts being socialized into long lasting political and policy positions. Collective socialization into policy paradigms is one of the main drivers of institutional path dependency, as it makes it difficult for people to imagine alternatives.

For our purposes, social learning encapsulates all those social processes—material, institutional, embodied, and symbolic—through which particular ways of knowing, acting, and relating to one another as aggregate and individuated actants are encoded and reproduced, or by which “[e]ach society [gains and sustains] its own special habits” (Mauss 1979, p. 99). “Learning” in this context implies much more than the acquisition of skills and knowledge. It extends to adoption through imitation, stylistic borrowing, riffing, meme-making, sampling, acculturation, identification, modeling, prioritization, valuation, and the propagation and practice of informal pedagogies of many kinds. Understood in this way, “learning” does not hinge decisively upon the embodied capacities and needs of human individuals because those capacities and needs are only ever realized relationally or through “ecological interaction,” including through interaction with machines (Foster 2018). It is not hard to see why digital domains, online interactions, and social media networks have become a privileged site of observation for such processes (e.g., Greenhow and Robelia 2009), all the more so since socialization there often starts in childhood. This suggests that (contra Dreyfus 1974) social and machine learning must be analyzed as co-productive of, rather than antithetical to, one another.

Machine learning is, similarly, a catch-all term—one encompassing a range of ways of programming computers or computing systems to undertake certain tasks (and satisfy certain performance thresholds) without explicitly directing the machines in question how to do so. Instead, machine learning is aimed at having computers learn (more or less autonomously) from preexisting data, including the data output from prior attempts to undertake the tasks in question, and devise their own ways of both tackling those tasks and iteratively improving at them (Alpaydin 2014). Implementations of machine learning now span all areas of social and economic life. Machine learning “has been turning up everywhere, driven by exponentially growing mountains of [digital] data” (Domingos 2015). In this article, we take social media as one domain in which machine learning has been widely implemented. We do so recognizing that not all data analysis in which social media platforms engage is automated, and that those aspects that are automated do not necessarily involve machine learning. Two points are important for our purposes: most “machines” must be trained, cleaned, and tested by humans in order to “learn.” In implementations of machine learning on social media platforms, for instance, humans are everywhere “in the loop”—an immense, poorly paid, and crowdsourced workforce that relentlessly labels, rates, and expunges the “content” to be consumed (Gillespie 2018; Gray and Suri 2019). And yet, both supervised and unsupervised machines generate new patterns of interpretation, new ways of reading the social world and of intervening in it. Any reference to machine learning throughout this article should be taken to encapsulate these “more-than-human” and “more-than-machine” qualities of machine learning.

Cybernetic feedback, data hunger, and meaning accretion

Analogies between human (or social) learning and machine-based learning are at least as old as artificial intelligence itself. The transdisciplinary search for common properties among physical systems, biological systems, and social systems, for instance, was an impetus for the Macy Foundation conferences on “circular causal and feedback mechanisms in biology and social systems” in the early days of cybernetics (1946–1953). In the analytical model developed by Norbert Wiener, “the concept of feedback provides the basis for the theoretical elimination of the frontier between the living and the non-living” (Lafontaine 2007, p. 31). Just as the knowing and feeling person is dynamically produced through communication and interactions with others, the ideal cybernetic system continuously enriches itself from the reactions it causes. In both cases, life is irrelevant: what matters, for both living and inanimate objects, is that information circulates in an ever-renewed loop. Put another way, information/computation are “substrate independent” (Tegmark 2017, p. 67).

Wiener’s ambitions (and even more, the exaggerated claims of his post-humanist descendants, see, e.g., Kurzweil 2000) were immediately met with criticism. Starting in the 1960s, philosopher Hubert Dreyfus arose as one of the main critics of the claim that artificial intelligence would ever approach its human equivalent. Likening the field to “alchemy” (1965), he argued that machines would never be able to replicate the unconscious processes necessary for the understanding of context and the acquisition of tacit skills (1974, 1979)—the fact that, to quote Michael Polanyi (1966), “we know more than we can tell.” In other words, machines cannot develop anything like the embodied intuition that characterizes humans. Furthermore, machines are poorly equipped to deal with the fact that all human learning is cultural, that is, anchored not in individual psyches but in collective systems of meaning and in sedimented, relational histories (Vygotsky 1980; Bourdieu 1990; Durkheim 2001; Hasse 2019).

Is this starting to change today when machines successfully recognize images, translate texts, answer the phone, and write news briefs? Some social and computational scientists believe that we are on the verge of a real revolution, where machine learning tools will help decode tacit knowledge, make sense of cultural repertoires, and understand micro-dynamics at the individual level (Foster 2018). Our concern is not, however, with confirming or refuting predictive claims about what computation can and cannot do to advance scholars’ understanding of social life. Rather, we are interested in how social and computational learning already interact. Not only may social and machine learning usefully be compared, but they are reinforcing and shaping one another in practice.

In those jurisdictions in which a large proportion of the population is interacting, communicating, and transacting ubiquitously online, social learning and machine learning share certain tendencies and dependencies. Both practices rely upon and reinforce a pervasive appetite for digital input or feedback that we characterize as “data hunger.” They also share a propensity to assemble insight and make meaning accretively—a propensity that we denote here as “world or meaning accretion.” Throughout this article, we probe the dynamic interaction of social and machine learning by drawing examples from one genre of online social contention and connection in which the pervasive influence of machine learning is evident: namely, that which occurs across social media channels and platforms. Below we explain first how data hunger is fostered by both social and computing systems and techniques, and then how world or meaning accretion manifests in social and machine learning practices. These explanations set the stage for our subsequent discussion of how these interlocking dynamics operate to constitute and distribute power.

Data hunger: searching as a natural attitude

As suggested earlier, the human person is the product of a long, dynamic, and never settled process of socialization. It is through this process of sustained exposure that the self (or the habitus, in Pierre Bourdieu’s vocabulary) becomes adjusted to its specific social world. As Bourdieu puts it, “when habitus encounters a social world of which it is the product, it is like a ‘fish in water’: it does not feel the weight of the water, and it takes the world about itself for granted” (Bourdieu in Wacquant 1989, p. 43). The socialized self is a constantly learning self. The richer the process—the more varied and intense the interactions—the more “information” about different parts of the social world will be internalized and the more socially versatile—and socially effective, possibly—the outcome. (This is why, for instance, parents with means often seek to offer “all-round” training to their offspring [Lareau 2011]).

Machine learning, like social learning, is data hungry. “Learning” in this context entails a computing system acquiring capacity to generalize beyond the range of data with which it has been presented in the training phase. “Learning” is therefore contingent upon continuous access to data—which, in the kinds of cases that preoccupy us, means continuous access to output from individuals, groups, and “bots” designed to mimic individuals and groups. At the outset, access to data in enough volume and variety must be ensured to enable a particular learner-model combination to attain desired accuracy and confidence levels. Thereafter, data of even greater volume and variety is typically (though not universally) required if machine learning is to deliver continuous improvement, or at least maintain performance, on assigned tasks.

The data hunger of machine learning interacts with that of social learning in important ways. Engineers, particularly in the social media sector, have structured machine learning technologies not only to take advantage of vast quantities of behavioral traces that people leave behind when they interact with digital artefacts, but also to solicit more through playful or addictive designs and cybernetic feedback loops. The machine-learning self is not only encouraged to respond more, interact more, and volunteer more, but also primed to develop a new attitude toward the acquisition of information (Andrejevic 2019, p.53). With the world’s knowledge at her fingertips, she understands that she must “do her own research” about everything—be it religion, politics, vaccines, or cooking. Her responsibility as a citizen is not only to learn the collective norms, but also to know how to search and learn so as to make her own opinion “for herself,” or figure out where she belongs, or gain new skills. The development of searching as a “natural attitude” (Schutz 1970) is an eminently social process of course: it often means finding the right people to follow or emulate (Pariser 2011), using the right keywords so that the search process yields results consistent with expectations (Tripodi 2018), or implicitly soliciting feedback from others in the form of likes and comments.

The social media user also must extend this searching disposition to her own person: through cybernetic feedback, algorithms habituate her to search for herself in the data. This involves looking reflexively at her own past behavior so as to inform her future behavior. Surrounded by digital devices, some of which she owns, she internalizes the all-seeing eye and learns to watch herself and respond to algorithmic demands (Brubaker 2020). Data hunger transmutes into self-hunger: an imperative to be digitally discernible in order to be present as a subject. This, of course, exacts a kind of self-producing discipline that may be eerily familiar to those populations that have always been under heavy institutional surveillance, such as the poor, felons, migrants, racial minorities (Browne 2015; Benjamin 2019), or the citizens of authoritarian countries. It may also be increasingly familiar to users of health or car insurance, people living in a smart home, or anyone being “tracked” by their employer or school by virtue of simply using institutionally licensed IT infrastructure.

But the productive nature of the process is not a simple extension of what Michel Foucault called “disciplinary power” nor of the self-governance characteristic of “governmentality.” Rather than simply adjusting herself to algorithmic demands, the user internalizes the injunction to produce herself through the machine-learning-driven process itself. In that sense the machine-learnable self is altogether different from the socially learning, self-surveilling, or self-improving self. The point for her is not simply to track herself so she can conform or become a better version of herself; it is, instead, about the productive reorganization of her own experience and self-understanding. As such, it is generative of a new sense of selfhood—a sense of discovering and crafting oneself through digital means that is quite different from the “analog” means of self-cultivation through training and introspection.

When one is learning from a machine, and in the process making oneself learnable by it, mundane activities undergo a subtle redefinition. Hydrating regularly or taking a stroll are not only imperatives to be followed or coerced into. Their actual phenomenology morphs into the practice of feeding or assembling longitudinal databases and keeping track of one’s performance: “step counting” and its counterparts (Schüll 2016; Adams 2019). Likewise, what makes friendships real and defines their true nature is what the machine sees: usually, frequency of online interaction. For instance, Snapchat has perfected the art of classifying—and ranking—relationships that way, so people are constantly presented with an ever-changing picture of their own dyadic connections, ranked from most to least important. No longer, contra Foucault (1988), is “permanent self-examination” crucial to self-crafting so much as attention to data-productive practices capable of making the self learnable and sustaining its searching process. To ensure one’s learnability—and thereby one’s selfhood—one must both feed and reproduce a hunger for data on and around the self.

Meaning accretion: from value to volume

Human learning is not only about constant, dynamic social exposure and world hunger, it is also about what we might call world or meaning accretion.2 The self is constantly both unsettled (by new experiences) and settling (as a result of past experiences). People take on well institutionalized social roles (Berger and Luckmann 1966). They develop habits, styles, personalities—a “system of dispositions” in Bourdieu’s vocabulary—by which they become adjusted to their social world. This system is made accretively, through the conscious and unconscious sedimentation of social experiences and interactions that are specific to the individual, and variable in quality and form. Accretion here refers to a process, like the incremental build-up of sediment on a riverbank, involving the gradual accumulation of additional layers or matter. Even when change occurs rapidly and unexpectedly, the ongoing process of learning how to constitute and comport oneself and perform as a social agent requires one to grapple with and mobilize social legacies, social memory, and pre-established social norms (Goffman 1983). The habitus, Bourdieu would say, is both structured and structuring, historical and generative.

Social processes of impression formation offer a good illustration of how social learning depends upon accreting data at volume, irrespective of the value of any particular datum. The popular insight that first impressions matter and tend to endure is broadly supported by research in social psychology and social cognition (Uleman and Kressel 2013). It is clear that impressions are formed cumulatively and that early-acquired information tends to structure and inform the interpretation of information later acquired about persons and groups encountered in social life (Hamilton and Sherman 1996). This has also been shown to be the case in online environments (Marlow et al. 2013). In other words, social impressions are constituted by the incremental build-up of a variegated mass of data.

Machine learning produces insight in a somewhat comparable way—that is, accretively. Insofar as machine learning yields outputs that may be regarded as meaningful (which is often taken to mean “useful” for the task assigned), then that “meaning” is assembled through the accumulation of “experience” or from iterative exposure to available data in sufficient volume, whether in the form of a stream or in a succession of batches. Machine learning, like social learning, never produces insight entirely ab initio or independently of preexisting data.

To say that meaning is made accretively in machine learning is not to say that machine learning programs are inflexible or inattentive to the unpredictable; far from it. All machine learning provides for the handling of the unforeseen; indeed, capacity to extend from the known to the unknown is what qualifies machine learning as “learning.” Moreover, a number of techniques are available to make machine learning systems robust in the face of “unknown unknowns” (that is, rare events not manifest in training data). Nonetheless, machine learning does entail giving far greater weight to experience than to the event.3 The more data that has been ingested by a machine learning system, the less revolutionary, reconfigurative force might be borne by any adventitious datum that it encounters. If, paraphrasing Marx, one considers that people make their own history, but not in circumstances they choose for themselves, but rather in present circumstances given and inherited, then the social-machine learning interface emphasizes the preponderance of the “given and inherited” in present circumstances, far more than the potentiality for “mak[ing]” that may lie within them (Marx 1996 [1852]).

One example of the compound effect of social and automated meaning accretion in the exemplary setting to which we return throughout this article—social media—is the durability of negative reputation across interlocking platforms. For instance, people experience considerable difficulty in countering the effects of “revenge porn” online, reversing the harms of identity theft, or managing spoiled identities once they are digitally archived (Lageson and Maruna 2018). As Langlois and Slane have observed, “[w]hen somebody is publicly shamed online, that shaming becomes a live archive, stored on servers and circulating through information networks via search, instant messaging, sharing, liking, copying, and pasting” (Langlois and Slane 2017). In such settings, the data accretion upon which machine learning depends for the development of granular insights—and, on social media platforms, associated auctioning and targeting of advertising—compounds the cumulative, sedimentary effect of social data, making negative impressions generated by “revenge porn,” or by one’s online identity having been fraudulently coopted, hard to displace or renew. The truth value of later, positive data may be irrelevant if enough negative data has accumulated in the meantime.

Data hunger and the accretive making of meaning are two aspects of the embedded sociality of machine learning and of the “mechanical” dimensions of social learning. Together, they suggest modes of social relation, conflict, and action that machine learning systems may nourish among people on whom those systems bear, knowingly or unknowingly. This has significant implications for social and economic inequality, as we explore below.

Ordering, social and automated

What are the social consequences of machine learning’s signature hunger for diverse, continuous, ever more detailed and “meaningful” data and the tendency of many automated systems to hoard historic data from which to learn? In this section, we discuss three observable consequences of data hunger and meaning accretion. We show how these establish certain non-negotiable preconditions for social inclusion; we highlight how they fuel the production of digitally-based forms of social stratification and association; and we specify some recurrent modes of relation fostered thereby. All three ordering effects entail the uneven distribution of power and resources and all three play a role in sustaining intersecting hierarchies of race, class, gender, and other modes of domination and axes of inequality.

Terms of inclusion

Machine learning’s data appetite and the “digestive” or computational abilities that attend it are often sold as tools for the increased organizational efficiency, responsiveness, and inclusiveness of societies and social institutions. With the help of machine learning, the argument goes, governments and non-governmental organizations develop an ability to render visible and classify populations that are traditionally unseen by standard data infrastructures. Moreover, those who have historically been seen may be seen at a greater resolution, or in a more finely-grained, timely, and difference-attentive way. Among international organizations, too, there is much hope that enhanced learning along these lines might result from the further utilization of machine learning capacities (Johns 2019). For instance, machine learning, deployed in fingerprint, iris, or facial recognition, or to nourish sophisticated forms of online identification, is increasingly replacing older, document-based ones (Torpey 2018)—and transforming the very concept of citizenship in the process (Cheney-Lippold 2016).

Whatever the pluses and minuses of “inclusiveness” in this mode, it entails a major infrastructural shift in the way that social learning takes place at the state and inter-state level, or how governments come to “know” their polities. Governments around the world are exploring possibilities for gathering and analysing digital data algorithmically, to supplement—and eventually, perhaps, supersede—household surveys, telephone surveys, field site visits, and other traditional data collection methods. This devolves the process of assembling and representing a polity, and understanding its social and economic condition, down to agents outside the scope of public administration: commercial satellite operators (capturing satellite image data being used to assess a range of conditions, including agricultural yield and poverty), supermarkets (gathering scanner data, now widely used in CPI generation), and social media platforms. If official statistics (and associated data gathering infrastructures and labor forces) have been key to producing the modern polity, governmental embrace of machine learning capacities signals a change in ownership of that means of production. Social media has become a key site for public and private parties—police departments, immigration agencies, schools, employers and insurers among others—to gather intelligence about the social networks of individuals, their health habits, their propensity to take risks or the danger they might represent to the public, to an organization’s bottom line or to its reputation (Trottier 2012; Omand 2017; Bousquet 2018; Amoore 2020; Stark 2020). Informational and power asymmetries characteristic of these institutions are often intensified in the process. This is notwithstanding the fact that automated systems’ effects may be tempered by manual work-arounds and other modes of resistance within bureaucracies, such as the practices of frontline welfare workers intervening in automated systems in the interests of their clients, and strategies of foot-dragging and data obfuscation by legal professionals confronting predictive technologies in criminal justice (Raso 2017; Brayne and Christin 2020).

The deployment of machine learning to the ends outlined in the foregoing paragraph furthers the centrality of data hungry social media platforms to the distribution of all sorts of economic and social opportunities and scarce public resources. At every scale, machine-learning-powered corporations are becoming indispensable mediators of relations between the governing and the governed (a transition process sharply accelerated by the Covid-19 pandemic). This invests them with power of a specific sort: the power of “translating the images and concerns of one world into that of another, and then disciplining or maintaining that translation in order to stabilize a powerful network” and their own influential position within it (Star 1990, p. 32). The “powerful network” in question is society, but it is heterogeneous, comprising living and non-living, automated and organic elements: a composite to which we can give the name “society” only with impropriety (that is, without adherence to conventional, anthropocentric understandings of the term). For all practical purposes, much of social life already is digital.

This insertion of new translators, or repositioning of old translators, within the circuits of society is an important socio-economic transformation in its own right. And the social consequences of this new “inclusion” are uneven in ways commonly conceived in terms of bias, but not well captured by that term. Socially disadvantaged populations are most at risk of being surveilled in this way and profiled into new kinds of “measurable types” (Cheney-Lippold 2017). In addition, social media user samples are known to be non-representative, which might further unbalance the burden of surveillant attention. (Twitter users, for instance, are skewed towards young, urban, minority individuals (Murthy et al. 2016).) Consequently, satisfaction of data hunger and practices of automated meaning accretion may come at the cost of increased social distrust, fostering strategies of posturing, evasion, and resistance among those targeted by such practices. These reactions, in turn, may undermine the capacity of state agents to tap into social data-gathering practices, further compounding existing power and information asymmetries (Harkin 2015). For instance, Sarah Brayne (2014) finds that government surveillance via social media and other means encourages marginalized communities to engage in “system avoidance,” jeopardizing their access to valuable social services in the process. Finally, people accustomed to being surveilled will not hesitate to instrumentalize social media to reverse monitor their relationships with surveilling institutions, for instance by taping public interactions with police officers or with social workers and sharing them online (Byrne et al. 2019). While this kind of resistance might further draw a wedge between vulnerable populations and those formally in charge of assisting and protecting them, it has also become a powerful aspect of grassroots mobilization in and around machine learning and techno-social approaches to institutional reform (Benjamin 2019).

Automated social orders

In all the foregoing settings, aspirations for greater inclusiveness, timeliness, and accuracy of data representation—upon which machine learning is predicated and which underlie its data hunger—produce newly actionable social divisions. The remainder of this article analyzes some recurrent types of social division that machine learning generates, and types of social action and experience elicited thereby. There is, of course, no society without ordering—and no computing either. Social order, like computing order, comes in many shapes and varieties but generally “the gap between computation and human problem solving may be much smaller than we think” (Foster 2018, p. 152). In what follows, we cut through the complexity of this social-computational interface by distinguishing between two main ideal types of classification: ordinal (organized by judgments of positionality, priority, probability or value along one particular dimension) and nominal (organized by judgments of difference and similarity) (Fourcade 2016).

Social processes of ordinalization in the analog world might include exams, tests, or sports competitions: every level allows one to compete for the next level and be ranked accordingly. In the digital world, ordinal scoring might take the form of predictive analytics—which, in the case of social media, typically means the algorithmic optimization of online verification and visibility. By contrast, processes of nominalization include, in the analog world, various forms of homophily (the tendency of people to associate with others who are similar to them in various ways) and institutional sorting by category. Translated for the digital world, these find an echo in clustering technologies—for instance a recommendation algorithm that works by finding the “nearest neighbors” whose taste is similar to one’s own, or one that matches people based on some physical characteristic or career trajectory. The difference between ordinal systems and nominal systems maps well onto the difference between Bayesian and Analogical approaches to machine learning, to reference Pedro Domingos’s (2015) useful typology. It is, however, only at the output or interface stage that these socially ubiquitous machine learning orderings become accessible to experience.

Stratification by optimization

What does it mean, and what does it feel like, to live in a society that is regulated through machine learning systems—or rather, where machine learning systems are interacting productively with social ordering systems of an ordinal and nominal kind? In this section, we identify some new, or newly manifest, drivers of social structure that emerge in machine learning-dominated environments. Let us begin with the ordinal effects of these technologies (remembering that machine learning systems comprise human as well as non-human elements). As machine learning systems become more universal, the benefits of inclusion now depend less on access itself, and more on one’s performance within each system and according to its rules. For instance, visibility on social media depends on “engagement,” or how important each individual is to the activity of the platform. If one does not post frequently and consistently, comment or message others on Facebook or Instagram, or if others do not interact with one’s posts, one’s visibility to them diminishes quickly. If one is not active on the dating app Tinder, one cannot expect one’s profile to be shown to prospective suitors. Similarly, Uber drivers and riders rank one another on punctuality, friendliness, and the like, but Uber (the company) ranks both drivers and riders on their behavior within the system, from canceling too many rides to failing to provide feedback. Uber Egypt states on its website: “The rating system is designed to give mutual feedback. If you never rate your drivers, you may see your own rating fall.”

Even for those willing to incur the social costs of disengagement, opting out of machine learning may not be an option. Failure to respond to someone’s tag, or to like their photo, or otherwise maintain data productivity, and one might be dropped from their network, consciously or unconsciously, a dangerous proposition in a world where self-worth has become closely associated with measures of network centrality or social influence. As Bucher has observed, “abstaining from using a digital device for one week does not result in disconnection, or less data production, but more digital data points … To an algorithm, … absence provides important pieces of information” (Bucher 2020, p. 2). Engagement can also be forced on non-participants by the actions of other users—through tagging, rating, commenting, and endorsing, for instance (Casemajor et al. 2015). Note that none of this is a scandal or a gross misuse of the technology. On the contrary, this is what any system looking for efficiency and relevance is bound to look like. But any ordering system that acts on people will generate social learning, including action directed at itself in return.

Engagement, to feed data hunger and enable the accretion of “meaningful” data from noise, is not neutral, socially or psychologically. The constant monitoring and management of one’s social connections, interactions, and interpellations places a non-trivial burden on one’s life. The first strategy of engagement is simply massive time investment, to manage the seemingly ever-growing myriad of online relationships (boyd 2015). To help with the process, social media platforms now bombard their users constantly with notifications, making it difficult to stay away and orienting users’ behavior toward mindless and unproductive “grinding” (for instance, repetitively “liking” every post in their feed). But even this intensive “nudging” is often not enough. Otherwise, how can we explain the fact that a whole industry of social media derivatives has popped up, to help people optimize their behavior vis-a-vis the algorithm, manage their following, and gain an edge so that they can climb the priority order over other, less savvy users? Now users need to manage two systems (if not more): the primary one and the (often multiple) analytics apps that help improve and adjust their conduct in it. In these ways, interaction with machine learning systems tends to encourage continuous effort towards ordinal self-optimization.

However, efforts of ordinal optimization, too, may soon become useless: as Marilyn Strathern (citing British economist Charles Goodhardt) put it, “when a measure becomes a target, it ceases to be a good measure” (Strathern 1997, p. 308). Machine learning systems do not reward time spent on engagement without regard to the impact of that engagement across the network as a whole. Now, in desperation, those with the disposable income to do so may turn to money as the next saving grace to satisfy the imperative to produce “good” data at volume and without interruption, and reap social rewards for doing so. The demand for maximizing one’s data productivity and machine learning measurability is there, so the market is happy to oblige. With a monthly subscription to a social media platform, or even a social media marketing service, users can render themselves more visible. This possibility, and the payoffs of visibility, are learned socially, both through the observation and mimicry of models (influencers, for instance) or through explicit instruction (from the numerous online and offline guides to maximizing “personal brand”). One can buy oneself Instagram or Twitter followers. Social media scheduling tools, such as Tweetdeck and Post Planner, help one to plan ahead to try to maximize engagement with one’s postings, including by strategically managing their release across time zones. A paying account on LinkedIn dramatically improves a user’s chance of being seen by other users. The same is true of Tinder. If a user cannot afford the premium subscription, the site still offers them one-off “boosts” for $1.99 that will send their profile near the top of their potential matches’ swiping queue for 30 min. Finally, wealthier users can completely outsource the process of online profile management to someone else (perhaps recruiting a freelance social media manager through an online platform like Upwork, the interface of which exhibits ordinal features like client ratings and job success scores).

In all the foregoing ways, the inclusionary promise of machine learning has shifted toward more familiar sociological terrain, where money and other vectors of domination determine outcomes. In addition to economic capital, distributions of social and cultural capital, as well as traditional ascriptive characteristics, such as race or gender, play an outsized role in determining likeability and other outcomes of socially learned modes of engagement with machine learning systems. For instance, experiments with Mechanical Turkers have shown that being attractive increases the likelihood of appearing trustworthy on Twitter, but being Black creates a contrarian negative effect (Groggel et al. 2019). In another example, empirical studies of social media use among those bilingual in Hindi and English have observed that positive modes of social media engagement tend to be expressed in English, with negative emotions and profanity more commonly voiced in Hindi. One speculative explanation for this is that English is the language of “aspiration” in India or offers greater prospects for accumulating social and cultural capital on social media than Hindi (Rudra et al. 2016). In short, well-established off-platform distinctions and social hierarchies shape the extent to which on-platform identities and forms of materialized labor will be defined as valuable and value-generating in the field of social media.

In summary, ordinality is a necessary feature of all online socio-technical systems and it demands a relentless catering to one’s digital doppelgängers’ interactions with others and with algorithms. To be sure, design features tend to make systems addictive and feed this sentiment of oppression (boyd 2015). What really fuels both, however, is the work of social ordering and the generation of ordinal salience by the algorithm. In the social world, any type of scoring, whether implicit or explicit, produces tremendous amounts of status anxiety and often leads to productive resources (time and money) being diverted in an effort to better one’s odds (Espeland and Sauder 2016; Mau 2019).4 Those who are short on both presumably fare worse, not only because that makes them less desirable in the real world, but also because they cannot afford the effort and expense needed to overcome their disadvantage in the online world. The very act of ranking thus both recycles old forms of social inequality and also creates new categories of undesirables. As every teenager knows, those who have a high ratio of following to followers exhibit low social status or look “desperate.” In this light, Jeff Bezos may be the perfect illustration of intertwining between asymmetries of real world and virtual world power: the founder and CEO of Amazon and currently the richest man in the world has 1.4 million followers on Twitter, but follows only one person: his ex-wife.

Association by pattern recognition

Ordinalization has implications not just for hierarchical positioning, but also for belonging—an important dimension of all social systems (Simmel 1910). Ordinal stigma (the shame of being perceived as inferior) often translates into nominal stigma, or the shame of non-belonging. Not obtaining recognition (in the form of “likes” or “followers”), in return for one’s appreciation of other people, can be a painful experience, all the more since it is public. Concern to lessen the sting of this kind of algorithmic cruelty is indeed why, presumably, Tinder has moved from a simple elo or desirability score (which depends on who has swiped to indicate liking for the person in question, and their own scores, an ordinal measure) to a system that relies more heavily on type matching (a nominal logic), where people are connected based on taste similarity as expressed through swiping, sound, and image features (Carman 2019).

In addition to employing machine learning to rank users, most social media platforms also use forms of clustering and type matching, which allow them to group users according to some underlying similarity (analogical machine learning in Domingos’s terms). This kind of computing is just as hungry for data as those we discuss above, but its social consequences are different. Now the aim is trying to figure a person out or at least to amplify and reinforce a version of that person that appears in some confluence of data exhaust within the system in question. That is, in part, the aim of the algorithm (or rather, of the socio-technical system from which the algorithm emanates) behind Facebook’s News Feed (Cooper 2020). Typically, the more data one feeds the algorithm, the better its prediction, the more focused the offering, and the more homogeneous the network of associations forged through receipt and onward sharing of similar offerings. Homogenous networks may, in turn, nourish better—and more saleable—machine learning programs. The more predictable one is, the better the chances that one will be seen—and engaged—by relevant audiences. Being inconsistent or too frequently turning against type in data-generative behaviors can make it harder for a machine learning system to place and connect a person associatively.

In both offline and online social worlds (not that the two can easily be disentangled), deviations from those expectations that data correlations tend to yield are often harshly punished by verbal abuse, dis-association, or both. Experiences of being so punished, alongside experiences of being rewarded by a machine learning interface for having found a comfortable group (or a group within which one has strong correlations), can lead to some form of social closure, a desire to “play to type.” As one heavy social media user told us, “you want to mimic the behavior [and the style] of the people who are worthy of your likes” in the hope that they will like you in return. That’s why social media have been variously accused of generating “online echo chambers” and “filter bubbles,” and of fueling polarization (e.g., Pariser 2011). On the other hand, being visible to the wrong group is often a recipe for being ostracized, “woke-shamed,” “called-out,” or even “canceled” (Yar and Bromwich 2019).

In these and other ways, implementations of machine learning in social media complement and reinforce certain predilections widely learned socially. In many physical, familial, political, legal, cultural, and institutional environments, people learn socially to feel suspicious of those they experience as unfamiliar or fundamentally different from themselves. There is an extensive body of scholarly work investigating social rules and procedures through which people learn to recognize, deal with, and distance themselves from bodies that they read as strange and ultimately align themselves with and against pre-existing nominal social groupings and identities (Ahmed 2013; Goffman 1963). This is vital to the operation of the genre of algorithm known as a recommendation algorithm, a feature of all social media platforms. On Facebook, such an algorithm generates a list of “People You May Know” and on Twitter, a “Who to follow” list.

Recommendation algorithms derive value from this social learning of homophily (McPherson et al. 2001). For one, it makes reactions to automated recommendations more predictable. Recommendation algorithms also reinforce this social learning by minimizing social media encounters with identities likely to be read as strange or non-assimilable, which in turn improves the likelihood of their recommendations being actioned. Accordingly, it has been observed that the profile pictures of accounts recommended on TikTok tend to exhibit similarities—physical and racial—to the profile image of the initial account holder to whom those recommendations are presented (Heilweil 2020). In that sense, part of what digital technologies do is organize the online migration of existing offline associations.

But it would be an error to think that machine learning only reinforces patterns that exist otherwise in the social world. First, growing awareness that extreme type consistency may lead to online boredom, claustrophobia, and insularity (Crawford 2009) has led platforms to experiment with and implement various kinds of exploratory features. Second, people willfully sort themselves online in all sorts of non-overlapping ways: through Twitter hashtags, group signups, click and purchasing behavior, social networks, and much more. The abundance of data, which is a product of the sheer compulsion that people feel to self-index and classify others (Harcourt 2015; Brubaker 2020), might be repurposed to revisit common off-line classifications. Categories like marriage or citizenship can now be algorithmically parsed and tested in ways that wield power over people. For instance, advertisers’ appetite for information about major life events has spurred the application of predictive analytics to personal relationships. Speech recognition, browsing patterns, and email and text messages can be mined for information about, for instance, the likelihood of relationships enduring or breaking up (Dickson 2019). Similarly, the US National Security Agency measures people’s national allegiance from how they search on the internet, redefining rights in the process (Cheney-Lippold 2016). Even age—virtual rather than chronological—can be calculated according to standards of mental and physical fitness and vary widely depending on daily performance (Cheney-Lippold 2017, p. 97). Quantitatively measured identities—algorithmic gender, ethnicity, or sexuality—do not have to correspond to discrete nominal types anymore. They can be fully ordinalized along a continuum of intensity (Fourcade 2016). The question now is: How much of a US citizen are you, really? How Latinx? How gay?5

In a machine learning world, where each individual can be represented as a bundle of vectors, everyone is ultimately a unique combination, a category of one, however “precisely inaccurate” that category’s digital content may be (McFarland and McFarland 2015). Changes in market research from the 1970s to the 1990s, aimed at tracking consumer mobility and aspiration through attention to “psychographic variables,” constitute a pre-history, of sorts, for contemporary machine learning practices in commercial settings (Arvidsson 2004; Gandy 1993; Fourcade and Healy 2017; Lauer 2017). However, the volume and variety of variables now digitally discernible mean that the latter have outstripped the former exponentially. Machine learning techniques have the potential to reveal unlikely associations, no matter how small, that may have been invisible, or muted, in the physically constraining geography of the offline world. Repurposed for intervention, disparate data can be assembled to form new, meaningful types and social entities. Paraphrasing Donald MacKenzie (2006), machine learning is an “engine, not a camera.” Christopher Wylie, a former lead scientist at the defunct firm Cambridge Analytica—which famously matched fraudulently obtained Facebook data with consumer data bought from US data brokers and weaponized them in the context of the 2016 US Presidential election—recalls the experience of searching for—and discovering—incongruous social universes: “[we] spent hours exploring random and weird combinations of attributes.… One day we found ourselves wondering whether there were donors to anti-gay churches who also shopped at organic food stores. We did a search of the consumer data sets we had acquired for the pilot and I found a handful of people whose data showed that they did both. I instantly wanted to meet one of these mythical creatures.” After identifying a potential target in Fairfax County, he discovered a real person who wore yoga pants, drank Kombucha, and held fire-and-brimstone views on religion and sexuality. “How the hell would a pollster classify this woman?” Only with the benefit of machine learning—and associated predictive analytics—could Wylie and his colleagues claim capacity to microtarget such anomalous, alloyed types, and monetize that capacity (Wylie 2019, pp. 72–74).

To summarize, optimization makes social hierarchies, including new ones, and pattern recognition makes measurable types and social groupings, including new ones. In practice, ordinality and nominality often work in concert, both in the offline and in the online worlds (Fourcade 2016). As we have seen, old categories (e.g., race and gender) may reassert themselves through new, machine-learned hierarchies, and new, machine-learned categories may gain purchase in all sorts of offline hierarchies (Micheli et al. 2018; Madden et al. 2017). This is why people strive to raise their digital profiles and to belong to those categories that are most valued there (for instance “verified” badges or recognition as a social media “influencer”). Conversely, pattern-matching can be a strategy of optimization, too: people will carefully manage their affiliations, for instance, so as to raise their score—aligning themselves with the visible and disassociating themselves from the underperforming. We examine these complex interconnections below and discuss the dispositions and sentiments that they foster and nourish.

Sociality and change in the age of machine learning

It should be clear by now that, paraphrasing Latour (2013, p. 307), we can expect little from the “social explanation” of machine learning; machine learning is “its own explanation.” The social does not lie “behind” it, any more than machine learning algorithms lie “behind” contemporary social life. Social relations fostered by the automated instantiation of stratification and association—including in social media—are diverse, algorithmic predictability notwithstanding. Also, they are continually shifting and unfolding. Just as Latour (2013, p. 221) reminds us not to confuse technology with the objects it leaves in its wake, it is important not to presume the “social” of social media to be fixed by its automated operations. We can, nevertheless, observe certain modes of social relation and patterns of experience that tend to be engineered into the ordinal and nominal orders that machine learning (re)produces. In this section, we specify some of these modes of relation, before showing how machine learning can both reify and ramify them. Our argument here is with accounts of machine learning that envisage social and political stakes and conflicts as exogenous to the practice—considerations to be addressed through ex ante ethics-by-design initiatives or ex post audits or certifications—rather than fundamental to machine learning structures and operations. Machine learning is social learning, as we highlighted above. In this section, we examine further the kinds of sociality that machine learning makes—specifically those of competitive struggle and dependency—before turning to prospects for their change.

Social struggles, platformized

Social scientists’ accounts of modes of sociality online are often rendered in terms of the antagonism between competition and cooperation immanent in capitalism (e.g., Fuchs 2007). This is not without justification. After all, social media platforms are sites of social struggle, where people seek recognition: to be seen, first and foremost, but also to see—to be a voyeur of themselves and of others (Harcourt 2015; Brubaker 2020). In that sense, platforms may be likened to fields in the Bourdieusian sense, where people who invest in platform-specific stakes and rules of the game6 are best positioned to accumulate platform-specific forms of capital (e.g., likes, followers, views, retweets, etc.) (Levina and Arriaga 2014). Some of this capital may transfer to other platforms through built-in technological bridges (e.g., between Facebook and Instagram), or undergo a process of “conversion” when made efficacious and profitable in other fields (Bourdieu 2011; Fourcade and Healy 2017). For instance, as social status built online becomes a path to economic accumulation in its own right (by allowing payment in the form of advertising, sponsorships, or fans’ gifts), new career aspirations are attached to social media platforms. According to a recent and well-publicized survey, “Vlogger/YouTuber” has replaced “astronaut” as the most enviable job for American and British children (Berger 2019). In a more mundane manner, college admissions offices or prospective employers increasingly expect one’s presentation of self to include the careful management of one’s online personality—often referred to as one’s “brand” (e.g., Sweetwood 2017). Similarly, private services will aggregate and score any potentially relevant information (and highlight “red flags”) about individuals across platforms and throughout the web, for a fee. In this real-life competition, digitally produced ordinal positions (e.g., popularity, visibility, influence, social network location) and nominal associations (e.g., matches to advertised products, educational institutions, jobs) may be relevant.

Machine learning algorithms within social media both depend on and reinforce competitive striving within ordinal registers of the kind highlighted above—or in Bourdieu’s terms, competitive struggles over field-specific forms of capital. As Georg Simmel observed, the practice of competing socializes people to compete; it “compels the competitor” (Simmel 2008 [1903]). Socially learned habits of competition are essential to maintain data-productive engagement with social media platforms. For instance, empirical studies suggest that motives for “friending” and following others on social media include upward and downward social comparison (Ouwerkerk and Johnson 2016; Vogel et al. 2014). Social media platforms’ interfaces then reinforce these social habits of comparison by making visible and comparable public tallies of the affirmative attention that particular profiles and posts have garnered: “[b]eing social in social media means accumulating accolades: likes, comments, and above all, friends or followers” (Gehl 2015, p. 7). In this competitive “[l]ike economy,” “user interactions are instantly transformed into comparable forms of data and presented to other users in a way that generates more traffic and engagement” (Gerlitz and Helmond 2013, p. 1349)—engagement from which algorithms can continuously learn in order to enhance their own predictive capacity and its monetization through sales of advertising.

At the same time, the distributed structure of social media (that is, its multi-nodal and cumulative composition) also fosters forms of cooperation, gift exchange, redistribution, and reciprocity. Redistributive behavior on social media platforms manifests primarily in a philanthropic mode rather than in the equity-promoting mode characteristic of, for instance, progressive taxation.7 Examples include practices like the #followfriday or #ff hashtag on Twitter, a spontaneous form of redistributive behavior that emerged in 2009 whereby “micro-influencers” started actively encouraging their own followers to follow others.8 Insofar as those so recommended are themselves able to monetize their growing follower base through product endorsement and content creation for advertisers, this redistribution of social capital serves, at least potentially, as a redistribution of economic capital. Even so, to the extent that purportedly “free” gifts, in the digital economy and elsewhere, tend to be reciprocated (Fourcade and Kluttz 2020), such generosity might amount to little more than an effective strategy of burnishing one’s social media “brand,” enlarging one’s follower base, and thereby increasing one’s store of accumulated social (and potentially economic) capital.9 Far from being antithetical to competitive relations on social media, redistributive practices in a gift-giving mode often complement them (Mauss 1990). Social media cooperation can also be explicitly anti-social, even violent (e.g., Patton et al. 2019). In these and other ways, digitized sociality is often at once competitive and cooperative, connective and divisive (Zukin and Papadantonakis 2017).

Dependency by design

Whether it is enacted in competitive, redistributive or other modes, sociality on social media is nonetheless emergent and dynamic. No wonder that Bruno Latour was the social theorist of choice when we started this investigation. But—as Latour (2012) himself pointed out—Gabriel Tarde might have been a better choice. What makes social forms cohere are behaviors of imitation, counter-imitation, and influence (Tarde 1903). Social media, powered by trends and virality, mimicry and applause, parody and mockery, mindless “grinding” and tagging, looks quintessentially Tardian. Even so, social media does not amount simply to transfering online practices of imitation naturally occuring offline. The properties of machine learning highlighted above—cybernetic feedback; data hunger; accretive meaning-making; ordinal and nominal ordering—lend social media platforms and interfaces a distinctive, compulsive, and calculating quality—engineering a relentlessly “participatory subjectivity” (Bucher 2020, p.88; boyd 2015). How one feels and how one acts when on social media is not just an effect of subjective perceptions and predispositions. It is also an effect of the software and hardware that mediate the imitative (or counter-imitative) process itself—and of the economic rationale behind their implementation.

We cannot understand the structural features and phenomenological nature of digital technologies in general, and of social media in particular, if we do not understand the purposes for which they were designed. The simple answer, of course, is that data hunger and meaning accretion are essential to the generation of profit (Zuboff 2019), whether profit accrues from a saleable power to target advertising, commercializable developments in artificial intelligence, or by other comparable means. Strategies for producing continuous and usable data flows to profit-making ends vary, but tend to leverage precisely the social-machine learning interface that we highlighted above. Social media interfaces tend to exhibit design features at both the back- and front-end that support user dependency and enable its monetization. For example, the “infinite scroll,” which allows users to swipe down a page endlessly (without clicking or refreshing) rapidly became a staple of social media apps after its invention in 2006, giving them an almost hypnotic feel and maximizing the “time on device” and hence users’ availability to advertisers (Andersson 2018). Similarly, YouTube’s recommendation algorithm was famously optimized to maximize users’ time on site, so as to serve them more advertisements (Levin 2017; Roose 2019). Social media platforms also employ psycho-social strategies to this end, including campaigns to draw people in by drumming up reciprocity and participation—the notifications, the singling out of trends, the introduction of “challenges”—and more generally the formation of habits through gamification. Prominent critics of social media, such as Tristan Harris (originally from Google) and Sandy Parakilas (originally from Facebook), have denounced apps that look like “slot machines” and use a wide range of intermittent rewards to keep users hooked and in the (Instagram, TikTok, Facebook, …) zone, addicted “by design” (Schüll 2012; Fourcade 2017).

Importantly, this dependency has broader social ramifications than may be captured by a focus on individual unfreedom. Worries about the “psychic numbing” of the liberal subject (Zuboff 2019), or the demise of the sovereign consumer, do not preoccupy us so much as the ongoing immiseration of the many who “toil on the invisible margins of the social factory” (Morozov 2019) or whose data traces make them the targets of particularly punitive extractive processes. Dependencies engineered into social media interfaces help, in combination with a range of other structural factors, to sustain broader economic dependencies, the burdens and benefits of which land very differently across the globe (see, e.g., Taylor and Broeders 2015). In this light, the question of how amenable these dynamics may be to social change becomes salient for many.

Dynamics of machine-learned sociality

Recent advances in digital technology are often characterized as revolutionary. However, as well as being addictive, the combined effect of machine learning and social learning may be as conducive to social inertia as it is to social change. Data hunger on the part of mechanisms of both social learning and machine learning, together with their dependence on data accretion to make meaning, encourage replication of interface features and usage practices known to foster continuous, data-productive engagement. Significant shifts in interface design—and in the social learning that has accreted around use of a particular interface—risk negatively impacting data-productive engagement. One study of users’ reactions to changes in the Facebook timeline suggested that “major interface changes induce psychological stress as well as technology-related stress” (Wisniewski et al. 2014). In recognition of these sensitivities, those responsible for social media platforms’ interfaces tend to approach their redesign incrementally, so as to promote continuity rather than discontinuity in user behaviour. The emphasis placed on continuity in social media platform design may foster tentativeness in other respects as well, as we discuss in the next section.

At the same time, social learning and machine learning, in combination, are not necessarily inimical to social change. Machine learning’s associative design and propensity to virality have the potential to loosen or unsettle social orders rapidly. And much as the built environment of the new urban economy can be structured to foster otherwise unlikely encounters (Hanson and Hillier 1987; Zukin 2020), so digital space can be structured to similar effect. For example, the popular Chinese social media platform WeChat has three features, enabled by machine learning, that encourage open-ended, opportunistic interactions between random users—Shake, Drift Bottle, and People Nearby—albeit, in the case of People Nearby, random users within one’s immediate geographic vicinity. (These are distinct from the more narrow, instrumental range of encounters among strangers occasioned by platforms like Tinder, the sexual tenor of which are clearly established in advance, with machine learning parameters set accordingly.) Qualitative investigation of WeChat use and its impact on Chinese social practices has suggested that WeChat challenges some existing social practices, while reinforcing others. It may also foster the establishment of new social practices, some defiant of prevailing social order. For instance, people report interacting with strangers via WeChat in ways they normally would not, including shifting to horizontally-structured interactions atypical of Chinese social structures offline (Wang et al. 2016). This is not necessarily unique to WeChat.

The kinds of ruptures and reorderings engineered through machine learning do not, however, create equal opportunities for value creation and accumulation, any more than they are inherently liberating or democratizing. Social media channels have been shown to serve autocratic goals of “regime entrenchment” quite effectively (Gunitsky 2015).10 Likewise, they serve economic goals of data accumulation and concentration (Zuboff 2019). Machine-learned sociality lives on corporate servers and must be meticulously “programmed” (Bucher 2018) to meet specific economic objectives. As such, it is both an extremely lucrative proposition for some and (we have seen) a socially dangerous one for many. It favors certain companies, their shareholders and executives, while compounding conditions of social dependency and economic precarity for most other people. Finally, with its content sanitized by underground armies of ghost workers (Gray and Suri 2019), it is artificial in both a technical and literal sense—“artificially artificial,” in the words of Jeff Bezos (Casilli and Posada 2019).

Solidaristic propects

We have already suggested that machine-learned sociality, as it manifests on social media, tends to be competitive and individualizing (in its ordinal dimension) and algorithmic and emergent (in its nominal dimension). Although resistance to algorithms is growing, those who are classified in ways they find detrimental (on either dimension) may be more likely to try to work on themselves or navigate algorithmic workarounds than to contest the classificatory instrument itself (Ziewitz 2019). Furthermore, we know that people who work under distributed, algorithmically managed conditions (e.g., Mechanical Turk workers, Uber drivers) find it difficult to communicate amongst themselves and organize (Irani and Silberman 2013; Lehdonvirta 2016; Dubal 2017). These features of the growing entanglement of social and machine learning may imply dire prospects for collective action—and beyond it, for the achievement of any sort of broad-based, solidaristic project. In this section, we tentatively review possibilities for solidarity and mobilization as they present themselves in the field of social media.

Machine learning systems’ capacity to ingest and represent immense quantities of data does increase the chances that those with common experiences will find one another, at least insofar as those experiences are shared online. Machine-learned types thereby become potentially important determinants of solidarity, displacing or supplementing the traditional forces of geography, ascribed identities, and voluntary association. Those dimensions of social life that social media algorithms have determined people really care about often help give rise to, or supercharge, amorphous but effective forms of offline action, if only because the broadcasting costs are close to zero. Examples may include the viral amplification of videos and messages, the spontaneity of flash mobs (Molnár 2014), the leaderless, networked protests of the Arab spring (Tufekci 2016), or of the French Gilets Jaunes (Haynes 2019), and the #MeToo movement’s reliance on public disclosures on social media platforms. Nonetheless, the thinness, fleeting character, and relative randomness of the affiliations summoned in those ways (based on segmented versions of the self, which may or may not overlap) might make social recognition and commonality of purpose difficult to sustain in the long run.

More significant, perhaps, is the emergence of modes of collective action that are specifically designed not only to fit the online medium, but also to capitalize on its technical features. Many of these strategies were first implemented to stigmatize or sow division, although there is no fatality that this is their only possible use. Examples include the anti-semitic (((echo))) tagging on Twitter—originally devised to facilitate trolling by online mobs (Weisman 2018) but later repurposed by non-Jews as an expression of solidarity; the in-the-wild training of a Microsoft chatter bot, literally “taught” by well-organized users to tweet inflammatory comments; the artificial manipulation of conversations and trends through robotic accounts; or the effective delegation, by the Trump 2020 campaign, of the management of its ad-buying activities to Facebook’s algorithms, optimized on the likelihood that users will take certain campaign-relevant actions—“signing up for a rally, buying a hat, giving up a phone number” (Bogost and Madrigal 2020).11

The exploitation of algorithms for divisive purposes often spurs its own reactions, from organized counter-mobilizations to institutional interventions by platforms themselves. During the 2020 Black Lives Matter protests, for instance, K-pop fans flooded rightwing hashtags on Instagram and Twitter with fancams and memes in order to overwhelm racist messaging. Even so, often the work of “civilizing” the social media public sphere is left to algorithms, supported by human decision-makers working through rules and protocols (and replacing them in especially sensitive cases). Social media companies ban millions of accounts every month for inappropriate language or astroturfing (coordinated operations on social media that masquerade as a grassroot movement): algorithms have been trained to detect and exclude certain types of coalitions on the basis of a combination of social structure and content. In 2020, the British far right movement “Britain first” moved to TikTok after being expelled from Facebook, Twitter, Instagram, and YouTube—and then over to VKontakte or VK, a Russian platform, after being banned from TikTok (USA News 2020). Chastised in the offline world for stirring discord and hate, the economic engines that gave the movement a megaphone have relegated it to their margins with embarrassment. The episode goes to show that there is nothing inherently inclusive in the kind of group solidarity that machine learning enables, and thus it has to be constantly put to the (machine learning) test.

In the end, platforms’ ideal of collective action may resemble the Tardean, imitative but atomized crowd, nimble but lacking in endurance and capacity (Tufekci 2016). Mimetic expressions of solidarity, such as photo filters (e.g., rainbow), the “blacking out” of one’s newsfeed, or the much-bemoaned superficiality of “clicktivism” may be effective at raising consciousness or the profile of an issue, but they may be insufficient to support broader-based social and political transformations. In fact, social media might actually crowd out other solidaristic institutions by also serving as a (feeble, often) palliative for their failures. For example, crowdsourced campaigns, now commonly used to finance healthcare costs, loss of employment, or educational expenses, perform a privatized solidarity that is a far cry from the universal logic of public welfare institutions.

Conclusion: Reassembling the machine

Up to this point, our emphasis has been on the kinds of sociality that machine learning implementations tend to engender on the social media field, in both vertical (ordinal) and horizontal (nominal) configurations. We have, in a sense, been “reassembling the social” afresh, with an eye, especially, to its computational components and chains of reference (Latour 2005). Throughout, we have stressed, nonetheless, that machine learning and other applications of artificial intelligence must be understood as forces internal to social life—both subject to and integral to its contingent properties—not forces external to it or determinative of it. Accordingly, it is just as important to engage in efforts to reassemble “the machine”—that is, to revisit and put once more into contention the associative preconditions for machine learning taking the form that it currently does, in social media platforms for instance. And if we seek to reassemble the machine, paraphrasing Latour (2005, p. 233), “it’s necessary, aside from the circulation and formatting of traditionally conceived [socio-technical] ties, to detect other circulating entities.”

So what could be some “other circulating entities” within the socio-technical complex of machine learning, or how could we envisage its elements circulating, and associating, otherwise? On some level, our analysis suggests that the world has changed very little. Like every society, machine-learned society is powered by two fundamental, sometimes contradictory forces: stratification and association, vertical and horizontal difference. To be sure, preexisting social divisions and inequalities are still very much part of its operations. But the forces of ordinality and nominality have also been materialized and formatted in new ways, of which for-profit social media offer a particularly stark illustration. The machine-learnable manifestations of these forces in social media: these are among the “other circulating entities” now traceable. Recursive dynamics between social and machine learning arise where social structures, economic relations and computational systems intersect. Central to these dynamics in the social media field are the development of a searching disposition to match the searchability of the environment, the learnability of the self through quantified measurement, the role of scores in the processing of social positions and hierarchies, the decategorization and recategorization of associational identities, automated feedback that fosters compulsive habits and competitive social dispositions, and strategic interactions between users and platforms around the manipulation of algorithms.

What, then, of prospects for reassembly of existing configurations? Notwithstanding the lofty claims of the IT industry, there is nothing inherently democratizing or solidaristic about the kinds of social inclusiveness that machine learning brings about. The effects of individuals and groups’ social lives being rendered algorithmically learnable are ambivalent and uneven. In fact, they may be as divisive and hierarchizing as they may be connective and flattening. Moreover, the conditions for entry into struggle in the social media field are set by a remarkably small number of corporate entities and “great men of tech” with global reach and influence (Grewal 2008). A level playing field this most definitely is not. Rather, it has been carved up and crenellated by those who happen to have accumulated greatest access to the data processing and storage capacity that machine learning systems require, together with the real property, intellectual property, and personal property rights, and the network of political and regulatory lobbyists that ensure that exclusivity of access is maintained (Cohen 2019). Power in this field is, accordingly, unlikely to be reconfigured or redistributed organically, or through generalized exhortation to commit to equity or ethics (many versions of which are self-serving on the part of major players).

Instead, political action aimed at building or rebuilding social solidarities across such hierarchies and among such clusters must work with and through them, in ways attentive to the specifics of their instantiation in particular techno-social settings. To open to meaningful political negotiation those allocations and configurations of power that machine learning systems help to inscribe in public and private life—this demands more than encompassing a greater proportion of people within existing practices of ruling and being ruled, and more than tinkering around the edges of existing rules. The greater the change in sociality and social relations—and machine learning is transforming both, as we have recounted—the more arrant and urgent the need for social, political and regulatory action specifically attuned to that change and to the possibility of further changes. Social and political action must be organized around the inequalities and nominal embattlements axiomatic to the field of social media, and to all fields shaped in large part by machine learning. And these inequalities and embattlements must be approached not as minor deviations from a prevailing norm of equality (that is, something that can be corrected after the fact or addressed through incremental, technical fixes), but as constitutive of the field itself. This cannot, moreover, be left up to the few whose interests and investments have most shaped the field to date. It is not our aim to set out a program for this here so much as to elucidate some of the social and automated conditions under which such action may be advanced. That, we must recognize, is a task for society, in all its heterogeneity. It is up to society, in other words, to reassemble the machine.