Unreal Data

Thank you, Francis. Thank you, Hito. Thank you, Paul. And thanks, Constant. Looking forward, I think we'll take that series, kind of like the collaboration. Now joining meeting group with Nick. Yeah, I think our talk is not going to be as funny as his, but there is hope for the future when he joins our group. Next talk might be better. So just like as a reference to quickly before we start our talk to Constant about kind of like being a bot or not being a bot and how weird this feeling is at the moment where you're not really sure. But you need to constantly prove like AI crawlers are people. They are producing, really breaking the internet at the moment. All the big platforms are struggling with AI crawlers, scraping all the contents, producing massive amounts of traffic and non-human traffic. And so they kind of like this, are you a robot? Captcha keeps coming and coming and coming and nobody has a real solution. So there is like spoofing solutions now where they just like make garbage websites basically for AI bots specifically. So they kind of like have. Yeah. A new sort of reality for them.

And another tactics is to make kind of like execute scripts which produce a lot of waste and work. So kind of like the AI crawlers are stuck on this website just like executing commands which does not make any sense to kind of like slower the process. So we are just like in this in this strange fight also at the moment of kind of like AI crawlers versus kind of like the old web, which maybe still doesn't. I mean, like still exists, but it's kind of like disappearing. Which should I take? Yeah. Yes. Sorry. So during the pandemic, there was this work by Reuben, a Paris based arts collective, who was. I don't know. It's not really a. It's not really a. I guess it's not really a. All right. From designited to. That. What. Did. I, I mean, I think. So there is this test and you can pass kind of like to the content of the website only when you prove that you are a bot. So they suggest that you use VPNs, that you use kind of like strange tools which make you kind of like look botty, not move your mouse too much, all that stuff or kind of like provide weird behavior.

But actually what we also figured what you can do, kind of like identity enforcement or like on the web is very simple. It's all text-based. So normally like if you speak to a server, you tell him, hey, I'm a Firefox and I have this resolution and I can do this and this. I have this font. So if you can like, can you send me the website? So it's IP and it's this thing called user agent. And the biggest user agent in the world, the biggest user basically in the world was back in that time was the Google bot. So Google bot is the most influential surfer which is around. So it serves all the websites, scrapes all the content and then kind of like placed in a database, makes it sortable. So we can kind of like get to those results. So but you can also claim to be from Google. You can say, hey, I'm not like Firefox. It's not the Firefox coming now. We just changed one string. It's just like one line of code. We just overrides this. I'm a Firefox into, hey, I'm a Google bot. Give me a Google bot website.

Give me a Google. Give me a website which the Google bot understands. And this totally changes. Kind of like how the Internet works because we call this the gray web. It's not the clear web or the dark web, but it's the gray web. It's an Internet which is built for bots. So there is this browser extension we wrote, which is just like one line of code which you can install and. You can kind of like can you just take the mic? It's a bit. Yeah. I need to open a new tab now because there's like caching. There's a lot of issues. But if we do, we do the test now. Google thinks we are a bot, so we just like past kind of like bot test. And it also changes how the web kind of like works. So if we go to Google now, Google dot com, we get the reduced and old school version of Google. It's not the same anymore. So there's like much less ads for for bots than for humans. Because why would you render ads for bots? So if you serve the web as a bot, you get a different reality.

Also, a lot of websites are closed. Newspaper archives, for example, usually will give you a paywall if you come as a human. But if you come as Googlebot, they will want you to browse and index their archives. So they will open the archive to you. So it's also a nice extension. It works on and off with the New York Times archive, for example. With Süddeutsche, it works off and on. Depends a bit on the type of code they have running. So it's also this constant fight there. But it's just like it's a browsing experience. We push you to try it. It's really kind of like a weird space. It's also some sort of Latin space. Cool. Yeah. So this Unreal data comes out of our interest specifically in algorithmic systems when they start making decisions for us. So this was kind of our focus for… research project we did together with Felix Stalder, who was also here yesterday, called Latin Spaces Performing Ambiguous Data at the Institute for Contemporary Art Research in Zurich. And this performing ambiguous data kind of refers very much to how we see the arts interacting with these systems. We believe you need to kind of intervene into the systems directly.

You need to perform them. You need to understand how they work. Because many of these data-driven systems are black boxes. You do not… It's hard to reverse engineer them. They're really, really big. But you can probe the machine and you can kind of try and learn what it sees, how it works, and how you can best interact with them. And I think also what we do speaks very much to what James and Navin mentioned, that… …um… This is not about solutions for the future. It's kind of… For us, it's kind of a lot of times making the contemporary kind of livable or more livable or in a way more interesting and fun, but also seriously more politically viable. So with the shift to data-driven systems or data-driven societies, automated data collection has become an intrinsic component of most technological systems and devices. Our interactions with technology generate data that in turn influences our world. Our devices tell us how well we have slept, predict where our favorite restaurants will be, and what products we will like. But data systems also shape the news we read, the borders we are allowed to cross, the jobs we can successfully apply for.

Data is what drives the so-called AI systems. Far from being intelligent, these are automated sorting systems that use data to learn and categorize, divide the masses of people and of things into addressable, computable groups, according to predefined categories. And this to us was really interesting, that data is not information. There is a step when… When the machine receives the data for it to become information. And this space, these latent spaces usually have some sort of ambiguities. And the machine needs to collapse these ambiguities to make sense of something. And we, in this research project, started to ask ourselves, is this a space that could be useful for creating agency? Can we kind of… Go into this ambiguous space and try to understand how this collapsing of meaning works, and try to, you know, give the machine other data, or kind of change this system? Basically, kind of like you input false data streams. Data streams you produce, trying to influence the outcome of it. Trying to influence the process of decision. Yeah. And because the machine… And because the machine… And because the machine… And because the machine always represents real world events, we kind of came to something that we think is quite an interesting phenomenon, I guess.

This is the simple sabotage field manual, which you may have stumbled across online also. It's a World War II era document published in 1944 by the United States Office of Strategic Services, to provide guidance to resistance, to provide guidance to resistance groups on how to disrupt enemy operations through covert means. And interestingly enough, in January this year, when the Trump… Well, Trump was… Came into power again in the US, it was the most downloaded document within the bureaucracy of American government. And we thought this was really interesting, and it's one of many, since the beginning, documents, that we've been using to try to understand the role of the American government. But the main thing is that this is a document that is not a simple sabotage manual. It's a simple sabotage manual. I think simple sabotage just too quickly… The document frames it really nicely. It's a beautiful document to read. Distinguishes simple sabotage as something you don't need specific tools for. So the tools are in your environment. You don't need specific knowledge. You don't need to be a trained operative. You just need to be part of the machine. And you kind of work from within.

And you kind of work from within. So we brought some examples, of course. Simple sabotage. One thing they say is, mis-file essential documents. Work slowly. Be inefficient. Bring up irrelevant issues as frequently as possible. Hold conferences where there's more critical work to be done. Start to duplicate files. Make mistakes in quantities of material when you are copying orders. Confuse similar names. Use wrong addresses. Slow down the machine, basically. Yeah. And I think… Oh, sorry. I think Constant also went into this. This is very similar. It's one of the first examples of what we were looking for. This ambiguity of data being changed. That we encountered in real life. This was 2019. And it's an instance of spoofing of AIS data. We're really interested in infrastructural systems, always. But specifically in maritime systems. Because still 80% of the goods are transported on waterways. And these ships are huge. As you may remember from the ship that got stuck in the Suez Canal. And kind of disrupted a lot of the transport that came from there. The transport that came to Europe. These ships today don't navigate. Or already for a while have not been navigating on site.

They're too big for that. They rely on data. So every ship on international waters above a certain length needs to have an AIS system on board. Which basically broadcasts the direction, the speed, and the location of a ship in real time. This data is public. Like with the airplanes. You may know this from the airplanes. You can download an app. And you can just watch the ships moving around. And we went to Shanghai. We were invited to Shanghai in 2019. Right before the pandemic. Which has one of the busiest ports. And we realized standing on the Huangpu River. That what we saw in terms of the ships that were moving. Was not corresponding to the apps we have. So we do this always when we're at the seaside. And usually it's really very accurate. The ship comes. The ship moves on the app. Not in Shanghai. So it was just strange. We would see ships which kind of like did not exist in the app. And we would not see ships which were kind of like broadcasting areas. So it was just like this confusion. And also it's dangerous stuff which is happening there. Because Kamen Rider Raid said.

They're not like driving on site anymore. They're using kind of like these things to navigate these huge vessels. And we started to read up on this online. And it appeared that this spoofing in the port of Shanghai. Had been ongoing for a while. They called it the Shanghai crop circles. And something that people. I'm not sure anyone ever found out what it was. Something was spoofing ships onto land in perfect circles. These ships were still moving. They were moving around in circles. And it was just very absurd. It also led to a few accidents that were reported. And there was somebody from MIT technology review. Mark Harris. He put forward like three theories he has on. Maybe just like technically it was very hard for them to explain how it works. Like moving spoofing kind of like all ships a kilometer away is possible. You can do that. But place them kind of like algorithmically. Like for each signal. You need to kind of like this is modulated. This is a modulated radio signal basically you're sending out. So this was just like nobody knew technically how this is being done. So he. Yeah. So he has this theory that it was the Chinese government testing.

War related technologies. Or that it was the sand mafia which operates in this area and illegally digs up sand. I mean there it's a bit unclear whether they would have the capabilities. Our favorite theory of course is. That it's. A technology to obfuscate. Illegal transports that go through Shanghai port to North Korea which is embargoed. So there's in Germany there's always this question of how. How do the latest model Mercedes Benz arrive in North Korea like immediately after release. And this may be one of the ways this happens. One of the ways is kind of like you would send a ship to Shanghai. Then you would scramble the data. You would produce some sort of like fog change. The car would change shape. And then kind of like with a kind of like clear history go to North Korea. And this is how you would do it nowadays. And for us as artists of course. It reminded us very much of Jody's 2008 piece. Geo Go where they kind of used Google pointers to draw images on Google Maps. Yeah. Yeah. Okay. We then started to look into this question of how real is the data we have and came across another example of unreal data from Uber.

I don't know if you remember this story. This was a story that broke in 2017 and was kind of reported by the New York Times. Uber at the time was facing attempts by governments to regulate them. And what they did as a reaction to this is they created a shadow app, a shadow Google app. They called it Grayball. And they would, if they identified somebody as being from the government or from any institution that could regulate them or that was supposed to oversee them. They would send them a message. They would send them a message. They would send or install the Grayball app on their phone when the person would download the app instead of the actual Uber app, giving them access only to licensed cars and to much fewer cars, only licensed cars and only in areas that were kind of legal. So it only came out because two politicians in Brussels, EU politicians, parliamentarians, were standing next to each other, both trying to get an Uber. And one had a thousand cars in the city and the other one had 50. And they were like, what's going on here? What's happening here? So we see it kind of like it's like these techniques are being also misused by kind of like big tech companies to influence our political decisions, influence our politicians to kind of produce some sort of like fake reality for them.

So maybe we can say that. Yeah. Unreal Data or this this untethering of data from reality can become a space for agency. These were all examples of, you know, the companies themselves or the people with a lot of money and access to tools altering data. But can we also do it for our own data as a kind of to create spaces of agency? This is four. From last week, I don't know if everyone saw this, last week there was this funny instance of Google Maps on the day of a public holiday in Germany rendered all autobahns, all motorways around Ruhrgebiet, showed them as inaccessible, closed, and rerouted all traffic around. And there were like huge traffic jams all around on the small roads, and people were like very confused because the autobahns were in fact open and nobody was using them. And it's, as far as I know, still unclear what happened. Google says something went wrong with their data, but to us it's just really interesting because it produces, however it's done, it still produces an actual reality. This meant people didn't use certain streets and we have an example from an artist whose name is Simon Weckert from Berlin.

It's, this work is called Google Maps Hacks. It's from 2020, so he did this before Google did it last week, so to say. He walked around Berlin with a trolley, a little cart containing 99 smartphones, and he just walked along the street. And then of course for Google, Google is a very, very good example of that. And then of course for Google, And then of course for Google, if 99 smartphones all logged into Google accounts are moving very slowly along a street, this to Google signifies traffic jam. So also in this case he had the street to himself because all the other cars were routed around. Of course it produced this kind of beautiful potential of kind of like you're owning the streets again, like the cars are gone. But also for me it also raises questions like why is Google routing our local traffic? Yeah. I don't know if it's just like the traffic, I don't know if it's like the traffic, I don't think it's just like the traffic, like the traffic, like the traffic, but why is Google routing our local traffic? I don't know. I don't know. I don't know. I don't know.

I don't know. traffic in Berlin? Have we ever discussed this? I mean, there's other agencies which should be caring about that. There should at least be some sort of discourse around that, but that never happened. These interventions bottom up, so to speak, usually are very short term, so the people with access to the codes of the AI or the machine learning or the algorithmic systems usually try to then prevent these types of interferences, but nevertheless, for a while, they work. This is an example, a different example, regarding driverless cars in San Francisco, which are not very popular because I haven't been to San Francisco in a while, but as far as I understand, they kind of don't work very well. They kind of block streets and get stuck. People are also afraid when there is mass events in the streets, they're afraid of these cars, that they might just jump into crowds or whatever, so it's still a big distrust towards those autonomous cars. Yeah, and what people realized is that if they place a traffic cone on the hood of the car, the car will not move anymore. It actually needs somebody to come and restart it, because this is kind of a hard no-no, apparently, within the code, so they, activists practiced, you know, throwing these traffic cones onto, like, slowly moving cars to kind of get them to stop in areas that were okay.

And I think with this move into automation or algorithmic systems, and with many, many of the areas of our lives being governed, at least in part, by our algorithmic systems, our question is a bit, how do we develop this understanding of unreal data? How do we learn to see these latent spaces as spaces of potential or ambiguity we can intervene into? And we just, I mean, there would be, I mean, there are endless examples, but we're going to focus on something we research, because we realized that if we want to create more agency, we need to really learn more about these systems. And this is usually, the best way to do this is just to probe these systems. And during the pandemic, when so many of our work-related activities went online, we started looking at the home office as a place where a lot of software was being pushed that was also AI guided or AI aided, and tried to understand what was happening there. So one of the problems of the companies, having people or their workers in home office was, or this is what they also say, is that they lost control over the office. They lost the line of sight. So offices are built in a way that the boss can move around and you see people like in the room. And you can kind of like check for if they are here, what they are working, you can speak with them. But having people at home meant that they lost their control.

So software came as a solution of course. And we call that software bossware. So this is basically spy tools which were installed on kind of like people's computers and they were mandatory. Several dozens of companies just popped up in the end. Teams 365 won the battle mostly. Yeah. Sorry. You're jumping ahead. Yeah. Sorry. Just quickly I think, can you do the next one? I think for us what was really interesting looking at the history of these systems workplace surveillance has a long history and comes from a guy called Taylor. Here he is. He is Frederick Taylor, a US American mechanical engineer who worked as an efficiency consultant. So this whole idea of workplace surveillance comes from this idea of efficiency. And he, so Taylor would watch people work at machines at the you know, big turn of the 20th century and figure out how they could do whatever they were doing more efficiently. He wrote a book called The Principles of Scientific Management and the very basic quote, the very basic kind of idea it boils down to is the unobserved worker is an inefficient one. So there's no trust in the workplace. There's this, this idea that as soon as the boss turns away and doesn't like directly look at the worker, the worker will wander off and go do something else.

You know, there's like zero work ethic there. And it's really interesting that this claim has actually been disputed in psychology multiple times over the past, I don't know, 100 years. But it's still there and it's ingrained into every software that's been used. But it's still there and it's ingrained into every software that's been used. But it's still there and it's ingrained into every software that was pushed into home offices during the pandemic. So this is kind of a list of you don't need to be able to read that. Sorry, it's a bit small of kind of the boss where companies that cropped up and kind of the things they they offer. So one is kind of like just like I'm reading the point software monitoring software can be made invisible. So the user doesn't kind of like know that it's installed. It can remote take. Over the whole desktop key logging is installed screen monitoring automated screenshots in intervals, internet app monitoring, so checking what you're surfing or what kind of apps you're using. Besides work apps, call tapping, GPS tracking, tracking webcam, mic surveillance, audio recording, email monitoring, monitoring, mobile device access, user action alerts, time tracking.

Go to the next one. And these. Usually everything that involves a mic and a camera is aided at least. So there will be an AI running that checks whether, you know, for example, enforces clean desk. So it will analyze the image to see what is on your home desk or it will see whether there are private prohibited devices in the image or whether you're eating and drinking or on a phone call. Or. Facial suspicion is also facial recognition to understand how happy the worker is while working. So it goes quite far. And this is a tweet by the. What's the crystal. He's an advocate for. Digital rights activists. Digital rights activists. Thank you. Who. Wrote this tweet in the moment that Microsoft 365. I don't know if this university is also using this Microsoft teams. As a as an environment. So this now has these efficiency tools or productivity tools as they are called built in standard. You can't get rid of them. So this key logging this you know. How much time does who spend online logged into Microsoft teams. This is done automatically. And. This. Is. A. A. A. And. What he also talked about was kind of he calls them esoteric metrics and this for us was a really important point in the research project to realize.

That it really depends on the metrics you use to measure productivity or efficiency and. There's this. This is nice story of the the Cobra effect. Which is basically perverse. Incentives. And. The Cobra effect is what's described by the economist host. See but. And I'm going to quickly tell you the story because it's so nice. He based the name on a historic event. Of. India on the British rule. India suffered from or New Delhi suffered from a plague of phenomenon. Venomous. Cobras. And the English colonial administration. Offered. A bounty for every dead snake. Somebody would bring. This. Didn't lead to less Cobras. It led to. Because the metric was badly chosen. It led to. A large. Number of snakes being killed first. And the rewards being claimed. But then of course people started breeding the snakes. Breeding the snakes killing the snakes bringing the snakes to.