3 CONCERNS

I'm worried about three closely intertwined trends. So we are living in a world of data exploitation which means thousands of companies, entities, have unprecedented insights into your life and this data may or may not be correct in the first place. The second development is that this data is increasingly being fed into automated and complex systems that not just identify you or judge you but also make well informed decisions that are of great consequence. And the third trend which is not a tech trend but more of an ideology that is having a bit of a revision these days is the belief that people's life is determined, that you have an essence, that I can see from your face who you are basically. If you put the three together the things that I find terrifying our developments like the use or misuse of AI to determine whether you are a criminal. That's a worrying pseudo-scientific trend that we have to be very careful about and take a strong stand to make sure that this does not become mainstream. What keeps me up at night are two closely intertwined developments. One is the tendency that people are being identified, judged, and profiled, based on an opaque and automated system. One example would be if you pair a CCTV camera with facial recognition in order to make decisions identify people in public spaces but also to make judgments about them such as who looks angry who looks like a threat who could be homeless who could be an alcoholic. So this is one development but this is increasingly being paired with not a technological development but with an ideology that assumes that people's identities and characters are fixed and determined. And if you put the two together you have pseudo-scientific trends such as technology that tries to predict criminality from faces or technologies that judge your character based on your face as well and this combined is extremely worrying.

CIVIL SOCIETY

Civil society and nonprofits are important because their purpose is to represent the public interest. The questions we are dealing with involves some of the richest and wealthiest companies in the world who might have good intentions but who always have a profit interest and certain practices that might actually not be to the benefit of all of us. So that's why the involvement of civil society is really crucial. So just having someone in the room was not paid to say what they're saying really can make all the difference. But there's a second dimension in which civil society and NGO’s are needed. Data exploitation means there's a fundamental asymmetry between those that mine and profit from data and those about whom this data is. We need people to investigate opaque systems and we need people to uncover what is actually happening in the first place. Perhaps the most important take away from the Cambridge Analytica scandal is that even blatant violations of the law often require investigations to reach the light of day. And there are thousands of more scandals and worrying practices that need to be talked about, that need to be investigated, that need to be uncovered.