Rules for the Design of
Algorithmic Systems
Participate now
Strengthen competency

The function and potential effects of an algorithmic system must be understood.

Those who develop, operate and/or make decisions regarding the use of algorithmic systems must have the necessary expertise and appropriate-to-scale understanding of how the technology functions and its potential effects. Sharing individual and institutional knowledge as well as promoting interdisciplinary exchange across task areas are just as crucial as ensuring appropriate skills development. These approaches should be integrated into the education, training and onboarding of new employees. In addition, interdisciplinary exchange should be an ongoing endeavor that remains open to those who are interested and/or affected.
Define responsibilities

A natural or legal person must always be held responsible for the effects involved with the use of an algorithmic system.

Accountability must be clearly assigned. The accountable person must be aware of the responsibilities associated with their tasks. This also applies to responsibilities that are shared by several people or organizations. The allocation of responsibility must be fully documented and transparent for internal and external parties. Responsibility may not be transferred to the algorithmic system itself, users or people who are affected by the system.
Document goals and anticipated impact

The objectives and expected impact of the use of an algorithmic system must be documented and assessed prior to implementation.

The objectives of an algorithmic system must be clearly defined and information regarding its use must be documented. This includes the underlying data and calculation models. Before an algorithmic system is put to use, an impact assessment should be conducted and documented. Particularly in the case of machine-learning systems and in dynamic areas of application that are subject to frequent change, an impact assessment should be repeated at regular intervals. The risk of discrimination and other consequences affecting individuals and the common good must be taken into consideration. The objectives considered, their underlying values and the use of algorithmic systems must be documented.
Guarantee security

The security of an algorithmic system must be tested before and during its implementation.

The reliability and robustness of an algorithmic system as well as its underlying data with respect to attacks, access and manipulation must be guaranteed. Security must be built into the architecture of the algorithmic system (security by design). The system must be tested in a protected environment prior to implementation. Security precautions must be documented.
Provide labeling

The use of an algorithmic system must be identified as such.

People interacting with algorithmic systems must be able to identify that a decision or prediction is based on an algorithm. This is particularly important in cases where the system imitates a human being in how it interacts (e.g., through language or appearance).
Ensure intelligibility

The decision-making processes within an algorithmic system must always be comprehensible.

In order to question and review decisions resulting from an algorithmic system, people must be able to understand both direct and indirect effects of the system as well as how decisions are reached. Information about the data and models on which the system is based, its architecture and potential effects must be published in easily understood terms. In addition, it is important to check whether an objective can be achieved without a significant loss in quality through the use of a less complex algorithmic system that involves an easier to understand mode of operation.
Safeguard manageability

An algorithmic system must be manageable throughout the lifetime of its use.

In order for an algorithmic system to remain adaptable, everyone involved in its development and implementation must maintain joint control over the system. This involves ensuring broad oversight of the entire system, even when tasks are distributed across various departments within an organization and among several individuals. The complexity of a system’s operations must never exceed the capacity of human oversight and a person’s capacity to make changes to the system. This applies in particular to machine-learning systems. If this manageability cannot be guaranteed, the algorithmic system in question should not be used.
Monitor impact

The effects of an algorithmic system must be reviewed on a regular basis.

An algorithmic system must be subject to active monitoring in order to determine whether the targeted objectives are actually achieved, and the use of the system does not violate existing legislation. Taking the appropriate technological precautions, external bodies should be able to conduct an independent, comprehensive and effective audit of an algorithmic system without compromising legitimate concerns regarding business confidentiality. Should a negative impact be determined, the cause must be identified and the algorithmic system adapted accordingly.
Establish complaint mechanisms

If an algorithmic system results in a questionable decision or a decision that affects an individual’s rights, it must be possible to request an explanation and file a complaint.

The person or organization using an algorithmic system must provide an easily accessible means of contact. First, those affected must be able to request appropriate and detailed information regarding a specific decision and the considerations that have fed into it. This should be an option also for organizations acting in their legitimate interest and for situations in which an organization acts on the behalf of an individual. Second, there must be an easily accessible and effective way to lodge a complaint. Complaints and actions taken must also be documented.

Preamble

 

Why do we need Algo.Rules?

Algorithmic systems are being implemented in a growing number of areas and are being used to make decisions that have a profound impact on our lives. They involve opportunities as well as risks. It is up to us to ensure that algorithmic systems are designed for the benefit of society. The individual and collective freedoms and rights that comprise human rights should be strengthened, not undermined, by algorithmic systems. Regulations designed to protect these norms must remain enforceable. To achieve this objective, we’ve developed the following Algo.Rules together with a variety of experts and the interested public.

What are Algo.Rules?

The Algo.Rules are a catalogue of formal criteria for enabling the socially beneficial design and oversight of algorithmic systems. They provide the basis for ethical considerations as well as the implementation and enforcement of legal frameworks. These criteria should be integrated from the start in the development of any system and therefore be implemented by design. Given their interdependence on each other, the Algo.Rules should be treated as a composite unit. Interested stakeholders and experts are invited to join us in developing the Algo.Rules further and to adopt them, adapt them, expand them and, above all, explore opportunities to apply them in practice. Dynamic by design, the Algo.Rules should be fine-tuned, particularly in terms of their practical implementation.

 

What is an „algorithmic system“?

The term “algorithm” refers to a set of precise instructions or rules regarding actions to be taken in solving a predefined problem. An algorithmic system is a system comprised of one or more algorithms used in a software to collect and analyze data as well as draw conclusions as part of a process designed to solve a pre-defined problem. The system can involve machine learning or follow pre-programmed decision-making rules. Drawing on Algo.Rules to evaluate an algorithmic system includes taking into consideration the broader socio-technical context in which the software is embedded. This involves, for example, considering how results are interpreted and how this informs the user of a system’s decisions. The Algo.Rules apply to the entire process of algorithmic system development as well as their embeddedment within a social context.

The Algo.Rules focus on those algorithmic systems that have a significant impact on society or individual lives, regardless of whether this involves direct or indirect effects. It is therefore not a question of establishing a set of design rules for all algorithmic systems but, rather, for those that are socially relevant. In order to determine whether an algorithmic system is socially relevant, an impact assessment should be conducted before it is designed. The stronger the potential influence of an algorithmic system on society or people’s lives, the more carefully it should comply with the Algo.Rules.

 

 

Target group

 

Many people are involved with the design of algorithmic systems. We aim to engage everyone who is involved in the development process.

The Algo.rules address everyone who signficantly influences the creation, development, programming, implementation or the effects of an algorithmic system, as well as everyone who has commissioned the development or integration of such a system. We purposefully do not limit the scope of our attention to programmers. This is because the effects of such systems are influenced not only through their programming codes, but also by their objectives, training data, organizational context and the ways in which a system’s results are presented, interpreted and applied. A system’s goals are often determined by the client or management. Should the system help generate the largest possible profit, solve a problem as fast as possible or should societal interests be the primary consideration? Operators and software designers also have considerable influence on the effects brought about by an algorithmic system. They determine the presentation and application of a decision made by said system. The Algo.Rules establish a uniform set of guiding principles for all of these individuals. Ensuring their use in day-to-day practice requires that they be further specified. Our next step therefore involves personalizing the rules. The purpose of this exercise is to emulate specific rules and recommended actions for real-world situations.

The Algo.Rules address in particular (but not limited to) the following groups:

  • Researchers,
  • data collectors,
  • executives,
  • programmers,
  • developers,
  • software designers
  • and operators.

Approach

 

The development of the Algo.Rules involves an open, participatory and interdisciplinary process in which 400 people have thus far participated.

 In addition to the complexity and diversity of algorithmic systems, the Algo.Rules must consider the different stages of a process and the people involved along the way. In order to make sure the rules remain understandable to everyone, we have been as specific as possible and as general as necessary. It makes little sense to formulate such rules behind closed doors and without referring to any practical application. Therefore, we have had the quality criteria of a binding code of professional ethics such as the Hippocratic Oath and the Press Code analyzed. Findings show that participatory approaches to development are one of the most important factors of success with such codes. Our work therefore focuses on developing the rules together with a large number and diverse group of individuals and organizations who, as part of an open and dynamic process, contribute their knowledge. This includes IT experts and representatives of other disciplines such as the humanities and law, as well as individuals who work in politics, civil society and business. The effects of algorithmic systems concern us all. This is why, in addition to our workshops and consultations with experts, we have engaged with the broader public through an open online survey and by discussing the Algo.Rules on panels and at meetups. The Algo.Rules are a joint effort. The process by which the Algo.Rules are developed is coordinated by the Bertelsmann Stiftung and the think tank iRights.Lab. We have defined nine Algo.Rules. Currently, we are working on models for the practical application of each rule. This involves refining the Algo.Rules for three so-called focus groups and the context of their application: developers of algorithmic systems, executives in companies and the public sector. Our aim here is to clarify what the Algo.Rules mean for each group and how the rules could be implemented in practice.

Join our efforts and make your contribution to the further development of the Algo.Rules!

 

 

Participants

 

"Thanks to the large and diverse group of people and organizations involved in our process, the Algo.Rules are a joint effort."

The ways in which algorithmic systems effect society cannot be reduced to a matter of programming codes alone. Just as relevant are a system’s goals and their underlying values, the data flowing into the system, and how results are presented and interpreted. The application of algorithmic systems raises technical as well as ethical and legal issues. We have therefore taken care to integrate a diverse goup of actors and perspectives into the process. This includes representatives from:

  • Science and research
  • Companies and professional associations
  • Civil society and NGOs
  • Politics and administrative bodies

Our group of experts played an important role here as major contributors to the development of the Algo.Rules.

Participants of the first expert workshop on the substance of the Algo.Rules in May 2018.
Participants of the second expert workshop on the implementation of the Algo.Rules in October 2018.

Contact

The Algo.Rules are a dynamic collection of criteria. Currently, we are working to formulate the rules in more concrete terms and to render them more applicable in practice. We are therefore looking for developers and decision-makers as well as institutions and companies with an interest in such rules. We are also looking for real-life examples of applied algorithmic systems to test the Algo.Rules. Ideas? Questions? Curious? Please contact us:


Bertelsmann Stiftung
Carl-Bertelsmann-Str. 256
D-33311 Gütersloh
Fon: +49 5241-810
Fax: +49 5241-81681396
E-Mail: carla.hustedt (at) bertelsmann-stiftung.de
Web: www.algorithmenethik.de


iRights.Lab GmbH
Almstadtstr. 9/11
 D-10119 Berlin
Fon: +49 (0)30 89 37 01 03
Fax: +49 (0)30 91 68 18 49
E-Mail: otto@irights-lab.de
Web: www.irights-lab.de

 

 

Subject

FAQ

Frequently Asked Questions

Why develop the Algo.Rules? What’s their objective?

Algorithmic systems have been a part of our daily lives for quite some time. They make decisions for and about us by filtering job applications, delivering diagnoses regarding our health or assessing creditworthiness. It is therefore important for us to discuss how the use of this technology can present us all with more opportunities than it does risks. We believe the design of algorithmic systems must therefore follow certain rules. The Algo.Rules are our proposed set of principles. Our goal is to see the Algo.Rules applied in all relevant algorithmic systems.

Why are you – the Bertelsmann Stiftung and iRights.Lab – doing this?

Those of us involved with the “Ethics of Algorithms” project at the Bertelsmann Stiftung initiated the process for developing the Algo.Rules. We want to contribute to the design of algorithmic sytems that result in a more inclusive society. Social worthiness – not technical feasibility – should be the guiding principle here. One of the overriding objectives of the Stiftung’s work is to ensure that digital transformation serve the needs of society. We aim to strengthen the voice of civil society, build bridges between disciplines, foster collaboration between theory and practice, and take a proactive approach to solving problems. We believe that developing rules for the design of algorithmic systems is one of many promising strategies which, in combination with each other, can help ensure that algorithms serve the common good. The independent think tank iRights.Lab has been commissioned to manage the development process. iRights.Lab has several years of experience with taking a transdisciplinary approach to developing complex systems and structures with a diverse network and making these systems relevant for practical use. Together, we aim to drive the public debate further and make a positive contribution to society in this field.

How were the Algo.Rules developed?

The Algo.Rules were created in an interdisciplinary, multisectoral and open process. We work in an interdisciplinary manner because the effects of algorithmic systems can only be understood through the convergence of different perspectives. We incorporate a variety of perspectives because actors from academia, civil society, politics and the business sector should engage in more extensive dialogue with one another. We have also chosen to pursue a fundamentally open approach, because the future of our digitalized society concerns all of us.

The process was launched at a workshop in May 2018. The theoretical groundwork for the Algo.Rules was laid in the context of two studies (on success factors for codes of professional ethics, and on the strengths and weaknesses of existing compendia of quality criteria for the use of algorithms), as well as through the consideration of numerous other sets of principles for algorithmic systems. In parallel, we consulted with 40 experts from the political, business, civil society and academic sectors over the course of the summer, asking them for intensive feedback. A further workshop in the autumn addressed issues related to the implementation of the Algo.Rules. Finally, in December 2018, the general public was invited to take part and contribute ideas through an online participation process.

However, the process is not yet complete. Interested parties are still invited to work with us in developing the Algo.Rules further. We invite others to borrow, adapt and expand them, and above all to find ways to implement the rules in a real-world context. The Algo.Rules are dynamic and are intended to be further refined, particularly with regard to their practical form and content.

 

 

Why are the Algo.Rules aimed at such a diverse target group?

A professional ethic for programmers would not be enough, as they are not the only ones who play a crucial role in the process of shaping algorithmic systems. Many other professional groups, from company executives to the users themselves, are just as relevant. We do not want to look simply at the code of a program. Rather, we want to examine how the application is socially embedded, who it will affect and how it is implemented. The Algo.Rules are oriented toward the process of algorithmic systems. This is one of the strengths of our approach: that it addresses multiple target groups together. The target groups relevant in this regard are described in greater detail both here and in the preamble.

Should the Algo.Rules apply to all algorithmic systems? For example, should even my pocket calculator be labeled as an algorithmic system?

No, because the Algo.Rules do not relate to all algorithmic systems. They apply only to those that have a direct or indirect, but always significant, impact on people’s lives or on society. For example, this includes software used to filter job applications, make recommendations to the police or to courts, or make decisions in disease-diagnosis settings. Thus, our approach is not about establishing design rules for all algorithmic systems but only for those that are relevant in this way. A recent assessment of potential impact on social inclusion, carried out by Ben Wagner and Kilian Vieth on behalf of the Bertelsmann Stiftung, offers an example of criteria that could guide this kind of relevance evaluation. Among other factors, this depends on who is implementing the algorithmic system, whether it is making decisions about people, what kinds of larger processes and in what area the algorithmic system is embedded in, and what consequences the system’s decisions might have for individuals’ lives or within society more broadly. In this regard, we classify systems at different levels: The stronger the algorithmic system’s potential influence on people’s lives or on society, the more important it is that the Algo.Rules be followed, and the more stringently the system should be reviewed.

How do the Algo.Rules relate to the many other similar initiatives, compendia and sets of criteria, such as the ethical guidelines of the German Informatics Society?

We have looked closely at sets of principles and criteria that have previously been published or are currently in the development phase. Most of these were created in the USA. In a previous study, we analyzed three of these in greater detail, including the “Principles for Accountable Algorithms and a Social Impact Statement for Algorithms” produced by the FAT/ML Conference, the Future of Life Institute’s “Asilomar AI Principles,” and the “Principles for Algorithmic Transparency and Accountability” published by the ACM U.S. Public Policy Council. Our work additionally incorporated findings derived from analyses of additional compendia and from exchanges with numerous other projects. The Algo.Rules represent a complement to and a further development of existing initiatives. They differ from other compendia in two respects: First, they are not oriented solely toward programmers. Rather, they address all persons who are involved in the development and application of algorithmic systems. Second, they include formal design rules, while eschewing reference to moral norms. To a certain degree, this makes them universal.

Do the Algo.Rules relate to Germany or to Europe, or are they intended to have global application?

○ The Algo.Rules emerge from a European cultural context. However, they have also been influenced by international discussions regarding the ethics and value-oriented design of algorithmic decision-making systems and applications. The version presented here has thus far been drafted primarily by actors active in Germany. However, they are oriented toward a public beyond Germany, and are thus being published in an English version as well. As the Algo.Rules do not contain moral norms, they are to a certain degree universal, and applicable within a variety of cultural contexts.

How do you want the Algo.Rules to be implemented in practice?

Aside from the effort to address a diverse and wide-ranging target group, the present challenge facing the Algo.Rules rests in their actual implementation. Our analysis has shown that many of the other existing compendia of criteria have failed because their drafters failed to focus specifically on the practicalities of implementation, or because their implementation strategies never bore fruit. We stand now at this same point. As a next step, we will refine the Algo.Rules further for specific target groups. In our view, this is the first important step toward making the rules practically applicable. Possibilities for further implementation range from the integration of the Algo.Rules in computer-science curricula to public commitments on the part of companies or the establishment of an independent oversight agency for algorithmic systems.

What happens next with the Algo.Rules?

In their current form, the Algo.Rules are being offered to the public as a possible starting point for many exciting processes. The first step has thus been taken. The next is now their concrete implementation. To this end, we will further develop the Algo.Rules so as to address specific target groups. This is critical because the concrete meaning of these rules will differ for someone at the executive level of an IT company and a programmer conducting an impact assessment for the next app, for example. Therefore, we will more closely examine three so-called focus target groups and the context of their activities. These groups will include developers and executives in companies and the private sector. In workshops, bilateral meetings and consultations, we will work out what the Algo.Rules concretely mean for these three groups, and compile mechanisms by which the rules can be implemented. In addition to ensuring that the Algo.Rules are relevant to actual practice, the goal is to compile concrete proposals for implementation along with examples of the rules’ application. Anyone who wants to help us create more specific versions of the Algo.Rules, or who has an idea regarding applications that could be used to test the implementation of the rules, should reach out to us using our contact form.