Search results for keyword 'test'

1. Accountability and Transparency

...We have implemented audit and risk assessments to test our models as the baseline of our oversight methodologies....

Published by ADP

· Article 9: Diversity and inclusivity.

...Continually test and validate algorithms, so that they do not discriminate against users based on race, gender, nationality, age, religious beliefs, etc....

Published by Artificial Intelligence Industry Alliance (AIIA), China

1. Transparency and Explainability

...Some practices to demonstrate repeatability include conducting repeatability assessments to ensure deployments in live environments are repeatable and performing counterfactual fairness testing to ensure that the AI system’s decisions are the same in both the real world and in the counterfactual world....

Published by ASEAN

2. Fairness and Equity

...Deployers of AI systems should conduct regular testing of such systems to confirm if there is bias and where bias is confirmed, make the necessary adjustments to rectify imbalances to ensure equity....

Published by ASEAN

2. Fairness and Equity

...For example, thetraining and test dataset for an AI system used in the education sector should be adequately representative of the student population by including students of different genders and ethnicities....

Published by ASEAN

3. Security and Safety

...Before deploying AI systems, deployers should conduct risk assessments and relevant testing or certification and implement the appropriate level of human intervention to prevent harm when unsafe decisions take place....

Published by ASEAN

3. Security and Safety

...It is also important for deployers to make a minimum list of security testing (e.g....

Published by ASEAN

3. Security and Safety

...vulnerability assessment and penetration testing) and other applicable security testing tools....

Published by ASEAN

3. Security and Safety

...vulnerability assessment and penetration testing) and other applicable security testing tools....

Published by ASEAN

4. Human centricity

...One example of how deployers can do this is to test the AI system with a small group of internal users from varied backgrounds and demographics and incorporate their feedback in the AI system....

Published by ASEAN

7. Robustness and Reliability

...Deployers should conduct rigorous testing before deployment to ensure robustness and consistent results across a range of situations and environments....

Published by ASEAN

Accountability:

...Accountability also entails testing for and anticipating potential harms, taking preemptive steps to mitigate such harms, and maintaining systems to respond to unanticipated harmful outcomes....

Published by Adobe

Reliability and safety

...AI systems should be monitored and tested to ensure they continue to meet their intended purpose, and any identified problems should be addressed with ongoing risk management as appropriate....

Published by Department of Industry, Innovation and Science, Australian Government

Safety and Controllability

...These AI applications and services should follow the principles of prudence and precaution, should be adequately tested for accuracy and robustness, and should be with meaningful human control....

Published by International Research Center for AI Ethics and Governance, Instituteof Automation, Chinese Academy of Sciences，World Animal Protection Beijing Representative Office and other 7 entities

· Focus on humans

...Human control of AI should be mandatory and testable by regulators....

Published by Centre for International Governance Innovation (CIGI), Canada

· Fairness and inclusion

...Employers should be required to test AI in the workplace on a regular basis to ensure that the system is built for purpose and is not harmfully influenced by bias of any kind — gender, race, sexual orientation, age, religion, income, family status and so on....

Published by Centre for International Governance Innovation (CIGI), Canada

· Fairness and inclusion

...Workplace AI should be tested to ensure that it does not discriminate against vulnerable individuals or communities....

Published by Centre for International Governance Innovation (CIGI), Canada

· Reliability

...AI should be designed within explicit operational requirements and undergo exhaustive testing to ensure that it responds safely to unanticipated situations and does not evolve in unexpected ways....

Published by Centre for International Governance Innovation (CIGI), Canada

· Transparency

...While in some circumstances open source code will be helpful, what is more important are clear, complete and testable explanations of what the system is doing and why....

Published by Centre for International Governance Innovation (CIGI), Canada

· (7) Innovation

...This engineering includes methods for the development, testing and operation of AI....

Published by Cabinet Office, Government of Japan

We should promote the establishment of a testing and assessment system based on AI risk levels, implement agile governance, and carry out tiered and category based management for rapid and effective response. R&D entities should improve the explainability and predictability of AI, increase data authenticity and accuracy, ensure that AI always remains under human control, and build trustworthy AI technologies that can be reviewed, monitored, and traced.

... We should promote the establishment of a testing and assessment system based on AI risk levels, implement agile governance, and carry out tiered and category based management for rapid and effective response....

Published by Cyberspace Administration of China

4. Reliable.

...DoD AI systems should have an explicit, well defined domain of use, and the safety, security, and robustness of such systems should be tested and assured across their entire life cycle within that domain of use....

Published by Defense Innovation Board (DIB), Department of Defense (DoD), United States

I. Human agency and oversight

...All other things being equal, the less oversight a human can exercise over an AI system, the more extensive testing and stricter governance is required....

Published by European Commission

III. Privacy and Data Governance

...Processes and data sets used must be tested and documented at each step such as planning, training, testing and deployment....

Published by European Commission

III. Privacy and Data Governance

...Processes and data sets used must be tested and documented at each step such as planning, training, testing and deployment....

Published by European Commission

(g) Security, safety, bodily and mental integrity

...All dimensions of safety must be taken into account by AI developers and strictly tested before release in order to ensure that ‘autonomous’ systems do not infringe on the human right to bodily and mental integrity and a safe and secure environment....

Published by European Group on Ethics in Science and New Technologies, European Commission

· 2.3 Shaping an enabling policy environment for AI

...To this effect, they should consider using experimentation to provide a controlled environment in which AI systems can be tested, and scaled up, as appropriate....

Published by G20 Ministerial Meeting on Trade and Digital Economy

· 3. Be built and tested for safety.

...Be built and tested for safety....

Published by Google

· 3. Be built and tested for safety.

...In appropriate cases, we will test AI technologies in constrained environments and monitor their operation after deployment....

Published by Google

· 2. The Principle of Non maleficence: “Do no Harm”

...Therefore not only should AI be designed with the impact on various vulnerable demographics in mind but the above mentioned demographics should have a place in the design process (rather through testing, validating, or other)....

Published by The European Commission’s High-Level Expert Group on Artificial Intelligence

· 2. Data Governance

...In addition, it must be ensured that the proper division of the data which is being set into training, as well as validation and testing of those sets, is carefully conducted in order to achieve a realistic picture of the performance of the AI system....

Published by The European Commission’s High-Level Expert Group on Artificial Intelligence

· 2. Data Governance

...It must particularly be ensured that anonymisation of the data is done in a way that enables the division of the data into sets to make sure that a certain data – for instance, images from same persons – do not end up into both the training and test sets, as this would disqualify the latter....

Published by The European Commission’s High-Level Expert Group on Artificial Intelligence

· 4. Governance of AI Autonomy (Human oversight)

...All other things being equal, the greater degree of autonomy that is given to an AI system, the more extensive testing and stricter governance is required....

Published by The European Commission’s High-Level Expert Group on Artificial Intelligence

3. Given the non deterministic nature of intelligent systems, a system of constant tests and validations must be established, including the inputs made to the system and its overall behavior. The architecture of AI systems must establish behavioral limits.

...Given the non deterministic nature of intelligent systems, a system of constant tests and validations must be established, including the inputs made to the system and its overall behavior....

Published by IA Latam

· Measurement and Evaluation

...To ensure red line testing regimes keep pace with rapid AI development, we should invest in red teaming and automating model evaluation with appropriate human oversight....

Published by IDAIS (International Dialogues on AI Safety)

· Consensus Statement on AI Safety as a Global Public Good

...Thanks to these summits, states established AI Safety Institutes or similar institutions to advance testing, research and standards setting....

Published by IDAIS (International Dialogues on AI Safety)

· Safety Assurance Framework

...Models whose capabilities fall below early warning thresholds require only limited testing and evaluation, while more rigorous assurance mechanisms are needed for advanced AI systems exceeding these early warning thresholds....

Published by IDAIS (International Dialogues on AI Safety)

· Safety Assurance Framework

...Although testing can alert us to risks, it only gives us a coarse grained understanding of a model....

Published by IDAIS (International Dialogues on AI Safety)

· Safety Assurance Framework

...Pre deployment testing, evaluation and assurance are not sufficient....

Published by IDAIS (International Dialogues on AI Safety)

· Safety Assurance Framework

...States should mandate that developers conduct regular testing for concerning capabilities, with transparency provided through independent pre deployment audits by third parties granted sufficient access to developers’ staff, systems and records necessary to verify the developer’s claims....

Published by IDAIS (International Dialogues on AI Safety)

· Independent Global AI Safety and Verification Research

...To ensure global trust, it will be important to have international collaborations developing and stress testing verification methods....

Published by IDAIS (International Dialogues on AI Safety)

4. Principle 4 — Transparency

...Develop new standards* that describe measurable, testable levels of transparency, so that systems can be objectively assessed and levels of compliance determined....

Published by The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems

· 1.3 Robust and Representative Data

...To promote the responsible use of data and ensure its integrity at every stage, industry has a responsibility to understand the parameters and characteristics of the data, to demonstrate the recognition of potentially harmful bias, and to test for potential bias before and throughout the deployment of AI systems....

Published by Information Technology Industry Council (ITI)

• Foster Innovation and Open Development

...Governments should support the controlled testing of AI systems to help industry, academia, and other stakeholders improve the technology....

Published by Intel

• Foster Innovation and Open Development

...• Allow for experimentation: Governments should create the conditions necessary for the controlled testing and experimentation of AI in the real world, such as designating self driving test sites in cities....

Published by Intel

• Foster Innovation and Open Development

...• Allow for experimentation: Governments should create the conditions necessary for the controlled testing and experimentation of AI in the real world, such as designating self driving test sites in cities....

Published by Intel

• Liberate Data Responsibly

...• Support the creation of reliable data sets to test algorithms: Governments should explore non regulatory methods to encourage the development of testing data sets....

Published by Intel

• Liberate Data Responsibly

...• Support the creation of reliable data sets to test algorithms: Governments should explore non regulatory methods to encourage the development of testing data sets....

Published by Intel

• Rethink Privacy

...Privacy approaches like The Fair Information Practice Principles and Privacy by Design have withstood the test of time and the evolution of new technology....

Published by Intel

Responsible Deployment

...Make safety a priority: Any deployment of an autonomous system should be extensively tested beforehand to ensure the AI agent’s safe interaction with its environment (digital or physical) and that it functions as intended....

Published by Internet Society, "Artificial Intelligence and Machine Learning: Policy Paper"

Responsible Deployment

...Responsible disclosure: Security researchers acting in good faith should be able to responsibly test the security of AI systems without fear of prosecution or other legal action....

Published by Internet Society, "Artificial Intelligence and Machine Learning: Policy Paper"

3. Technical reliability, Safety and security

...Artificial intelligence solutions should be extensively tested, used with care and monitored....

Published by Megvii

8 PRUDENCE PRINCIPLE

...3) Before being placed on the market and whether they are offered for charge or for free, AIS must meet strict reliability, security, and integrity requirements and be subjected to tests that do not put people’s lives in danger, harm their quality of life, or negatively impact their reputation or psychological integrity....

Published by University of Montreal

8 PRUDENCE PRINCIPLE

...These tests must be open to the relevant public authorities and stakeholders....

Published by University of Montreal

D. Reliability:

...The safety, security, and robustness of such capabilities will be subject to testing and assurance within those use cases across their entire life cycle, including through established NATO and or national certification procedures....

Published by The North Atlantic Treaty Organization (NATO)

· 2.3 Shaping an enabling policy environment for AI

...To this effect, they should consider using experimentation to provide a controlled environment in which AI systems can be tested, and scaled up, as appropriate....

Published by The Organisation for Economic Co-operation and Development (OECD)

3. Safe

...This would include mechanisms related to system testing, piloting, scaling and human intervention as well as alternative processes in case a complete halt of system operations is required....

Published by Government of Ontario, Canada

Uphold high standards of scientific and technological excellence

...Rebellion is committed to scientific excellence as we advance the development, testing, and deployment of artificial intelligence and other technologies....

Published by Rebelliondefense

Design for human control, accountability, and intended use

...We achieve this through modern software engineering and rigorous testing on our entire systems including their constituent data and AI products, in isolation and in concert....

Published by Rebelliondefense

Ensure fairness

...Our teams are trained to identify and challenge biases in our own decision making and in the data we use to train and test our models....

Published by Rebelliondefense

Ensure fairness

...We execute statistical tests to look for imbalance and skewed datasets and include methods to augment datasets to combat these statistical biases....

Published by Rebelliondefense

Ensure fairness

...We pressure test our decisions by performing peer review of model design, execution, and outcomes; this includes peer review of model training and performance metrics....

Published by Rebelliondefense

Ensure fairness

...This review includes in sample and out of sample testing to mitigate the risk of model overfitting to the training data, and biased outcomes in production....

Published by Rebelliondefense

· 2. NEED FOR CONSCIOUS RESPONSIBILITY WHEN CREATING AND USING AI

...during any stage, including design, development, testing, implementation or...

Published by AI Alliance Russia

· 1. Foundation of the code action

...Such persons include, but are not limited to, the following: developers who create, train, or test AI models systems and develop or implement such models systems, software and or hardware systems and take responsibility for their design; customers (individuals or organizations) receiving a product; or a service; data providers and persons involved in the formation of datasets for their use in AISs; experts who measure and or evaluate the parameters of the developed models systems; manufacturers engaged in the production of AIS; AIS operators who legally own the relevant systems, use them for their intended purpose and directly implement the solution to the problems that arise from using AIS; operators (individuals or organizations) carrying out the work of the AIS; persons with a regulatory impact in the field of AI, including the developers of regulatory and technical documents, manuals, various regulations, requirements, and standards in the field of AI; and other persons whose actions can affect the results of the actions of an AIS or persons who make decisions on the use of AIS....

Published by AI Alliance Russia

· 2. RESPONSIBILITY MUST BE FULLY ACKNOWLEDGED WHEN CREATING AND USING AI

...Any use, including the design, development, testing, integration or operation of an AI system capable of purposefully causing harm to the environment, human life and or health, the property of citizens and legal entities, is prohibited....

Published by AI Alliance Russia

· 1. THE BASICS OF THE CODE

...• developers who create, train or test AI models systems and develop or implement such models systems, software and or hardware systems and take responsibility for their design;...

Published by AI Alliance Russia

5. We uphold quality and safety standards

...Our AI software undergoes thorough testing under real world scenarios to firmly validate they are fit for purpose and that the product specifications are met....

Published by SAP

· Build and Validate:

...1 At the build and validate stage of the AI System Lifecycle, it is essential to take into consideration implementation fairness as a common theme when building, testing, and implementing the AI system....

Published by SDAIA

· Build and Validate:

...3 The AI System should be tested to ensure that the combination of available data does not reveal the sensitive data or break the anonymity of the observation....

Published by SDAIA

· Plan and Design:

...2 The designers of the AI model should define how the AI system will align with fundamental human rights and KSA’s cultural values while designing, building, and testing the technology; as well as how the AI system and its outcomes will strive to achieve and positively contribute to augment and complement human skills and capabilities....

Published by SDAIA

Prepare Input Data:

...2 It is crucial for the build and validate step to test how the system behaves under outlier events, extreme parameters, etc....

Published by SDAIA

Prepare Input Data:

...In this step, stress test data should be prepared for extreme scenarios....

Published by SDAIA

· Build and Validate:

...1 To develop a sound and functional AI system that is both reliable and safe, the AI system’s technical construct should be accompanied by a comprehensive methodology to test the quality of the predictive data based systems and models according to standard policies and protocols....

Published by SDAIA

· Build and Validate:

...2 To ensure the technical robustness of an AI system rigorous testing, validation, and re assessment as well as the integration of adequate mechanisms of oversight and controls into its development is required....

Published by SDAIA

· Build and Validate:

...System integration test sign off should be done with relevant stakeholders to minimize risks and liability....

Published by SDAIA

· Deploy and Monitor:

...Periodic UI and UX testing should be conducted to avoid the risk of confusion, confirmation of biases, or cognitive fatigue of the AI system....

Published by SDAIA

· Build and Validate:

...3 The decisions should be supported with quantitative (performance measures on train test datasets, consistency of the performance on different sensitive groups, performance comparison for each set of hyperparameters, etc.)...

Published by SDAIA

· Build and Validate:

...4 The appropriate stakeholders and owners of the AI technology should review and sign off the model after successful testing and validation of user acceptance testing rounds have been conducted and completed before the AI models can be productionized....

Published by SDAIA

· Build and Validate:

...4 The appropriate stakeholders and owners of the AI technology should review and sign off the model after successful testing and validation of user acceptance testing rounds have been conducted and completed before the AI models can be productionized....

Published by SDAIA

3.1 Explainability and verifiability

...Clarity in the context of these Guidelines means that all processes: development, testing, commissioning, system monitoring and shutdown must be transparent....

Published by Republic of Serbia

3.1 Explainability and verifiability

...Verifiability includes the actions and procedures of checking artificial intelligence systems during testing and implementation, as well as checking the short term and long term impact that such a system has on humans....

Published by Republic of Serbia

· 1) Accountability:

...We are committed to confirming test standards, deployment processes and specifications, ensuring algorithms verifiable, and gradually improving the accountability and supervision mechanism of artificial intelligence systems....

Published by Youth Work Committee of Shanghai Computer Society

(Preamble)

...To contribute to that AI can extend and complement human abilities rather than lessen or restrict them, Telia Company provides the following Guiding Principles to its operations and employees for proactive design, implementation, testing, use and follow up of AI....

Published by Telia Company AB

6. Safe and secure

...Our solutions are built and tested to prevent possible misuse and reduce the risk of being compromised or causing harm....

Published by Telia Company AB

8. Fair and equal

...We aspire to embed the principles of fairness and equality in datasets and algorithms applied in all phases of AI design, implementation, testing and usage – fostering fairness and diversity and avoiding unfair bias both at the input and output levels of AI....

Published by Telia Company AB

· Test and validation

...· test and validation...

Published by Tencent Research Institute

· Test and validation

... Ensure AI systems go through rigorous test and validation to achieve reasonable expectations of performance...

Published by Tencent Research Institute

Fifth principle: Reliability

...Those parameters must be regularly reviewed and tested for reliability to be assured on an ongoing basis, particularly as AI enabled systems learn and evolve over time, or are deployed in new contexts....

Published by The Ministry of Defence (MOD), United Kingdom

5. Ensure safety for children

...Require testing of AI systems for safety, security and robustness....

Published by United Nations Children's Fund (UNICEF) and the Ministry of

(b) The United States must drive development of appropriate technical standards and reduce barriers to the safe testing and deployment of AI technologies in order to enable the creation of new AI related industries and the adoption of AI by today’s industries.

... (b) The United States must drive development of appropriate technical standards and reduce barriers to the safe testing and deployment of AI technologies in order to enable the creation of new AI related industries and the adoption of AI by today’s industries....

Published by The White House, United States

4. Reliable

...The department's AI capabilities will have explicit, well defined uses, and the safety, security and effectiveness of such capabilities will be subject to testing and assurance within those defined uses across their entire life cycles....

Published by Department of Defense (DoD), United States

7. Validation and Testing

...Validation and testing...

Published by ACM US Public Policy Council (USACM)

7. Validation and Testing

...In particular, they should routinely perform tests to assess and determine whether the model generates discriminatory harm....

Published by ACM US Public Policy Council (USACM)

7. Validation and Testing

...Institutions are encouraged to make the results of such tests public....

Published by ACM US Public Policy Council (USACM)

3 Ensure transparency, explainability and intelligibility

...All algorithms should be tested rigorously in the settings in which the technology will be used in order to ensure that it meets standards of safety and efficacy....

Published by World Health Organization (WHO)

3 Ensure transparency, explainability and intelligibility

...tests and evaluations should be regular, transparent and of sufficient breadth to cover differences in the performance of the algorithm according to race, ethnicity, gender, age and other relevant human characteristics....

Published by World Health Organization (WHO)

3 Ensure transparency, explainability and intelligibility

...There should be robust, independent oversight of such tests and evaluation to ensure that they are conducted safely and effectively....

Published by World Health Organization (WHO)

5 Ensure inclusiveness and equity

...These parties also have a duty to address potential bias and avoid introducing or exacerbating health care disparities, including when testing or deploying new AI technologies in vulnerable populations....

Published by World Health Organization (WHO)

Contact

Institute of Automation, Chinese Academy of Sciences
95 Zhongguancun East Road,
Haidian District, Beijing,
China 100190
Principal Investigator - Yi Zeng
contact@long-term-ai.cn

Contributors: Cunqing Huangfu, Enmeng Lu, Zizhe Ruan et al.

Copyright © 2018-
Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences
International Research Center for AI Ethics and Governance, Institute of Automation, Chinese Academy of Sciences
All Rights Reserved