Now that Giskard has proven himself with French clients, it’s time to give him global scale

Alex Combessie is co-founder and CEO of Giskard, a young French start-up specialized in auditing artificial intelligence systems.

JDN. What gave you the idea to create a suite of tools to audit AI models?

At Giskard, we are not newcomers to the field of AI. I previously held a technical position at Dataiku, where I worked on the design and implementation of AI systems for large companies. I daily observed quality issues in projects.

Basically, there are two scenarios: either we do not test the models, and then we encounter problems in production with lower than expected performances, as well as ethical biases or drifts. Or we try to test, but it takes a lot of time. I saw projects blocked for sometimes more than a year while conducting benchmarks, test reports, model comparisons, and building these test procedures from scratch. I therefore thought that a tool facilitating these tests and applicable to a wide variety of AI models would save a lot of time for developers, and also strengthen the confidence of users and decision-makers. Like the systematic quality controls that, for example, our food products or sanitary equipment benefit from, we are seeking to establish equivalent test standards for AI models.

Alex Combessie is co-founder and CEO of Giskard. © Groover

Who are your investors?

Among our investors are some well-known names in artificial intelligence, such as Julien Chaumond, the chief technology officer (CTO) of Hugging Face, Oscar Salazar, Uber’s first CTO, and Charles Gorintin, Alan’s CTO. We also count among our investors profiles that are somewhat less specialized in AI, like the founder of 360 Learning or Good In Tech. In total, we have gathered around forty investors including some big names in the AI sector.

Where does the name Giskard come from?

We are often asked where our name comes from. It is a reference to the science fiction writer Isaac Asimov who, in the 1970s, had imagined the concept of “the laws of robotics”. The robots described at the time in a way correspond to today’s artificial intelligences. Asimov explored the idea that laws could be integrated into the code of robots to regulate their behavior, much like humans are governed by laws. This concept was introduced through the character of Giskard in Asimov’s novels. We chose this name as a reference because it symbolizes the mission of our company. Giskard formulated the “zeroth law” stating that robots must not harm humanity. This is also our goal at Giskard: to develop AI in an ethical and beneficial manner.

What are the different tests carried out by Giskard and on what variety of AI models are they applied?

Our solution allows for auditing all types of artificial intelligence: language models, generative models, computer vision, fraud detection, tabular models, etc. We have a very holistic approach to cover all AI use cases. We also plan to work on auditing recommendation and time series models within a two-year horizon.

We offer tests covering different aspects: ethics, performance, security, robustness, etc. These tests are applied with logic adapted to the type of AI model. For example, for large language models (LLMs), we analyze specific vulnerabilities such as hallucinations. While for traditional machine learning (ML) models, the vulnerabilities are of a different nature. In any case, our goal is to comprehensively cover ethics, data leaks, and all potential risks, for any type of model. We adapt the test batteries to target the salient issues according to each AI technology.

Who are these audits for?

Our tests are primarily aimed at end users of AI models. We work, for example, with Mistral on upstream issues, but in general, quality issues are mainly related to the context of model application. We collaborate with companies like Axa on auditing HR applications, or iAdvize for chatbots deployed in retail. In each case, these are specific applications requiring tailored metrics. Our approach seeks to focus on real usage contexts rather than too generic tests. Global model creators do not always have all the expertise necessary on what matters in the specific context of a company’s business. For us, the risks stem more from how AI is used than from intrinsic vulnerabilities in the algorithms.

What solutions are available to companies that want to use Giskard to test and monitor their AI solutions?

We offer two types of solutions: ML Testing (Python library for coders) and AI Quality Hub (visual dashboards for non-technicians). Our core business is quality assurance. We therefore recommend conducting tests before production: diagnosis, QA, etc. But we also provide LLMon (in beta) to monitor a deployed LLM. In production, metrics are simplified because tests must be executed in less than a second. Complex checks (massive databases, adversarial tests, high computing power) are carried out in pre-production. Once the robust QA phase is completed, LLMon calculates fast metrics like toxicity alerts, allowing monitoring of a model already in production.

What are the next steps in your roadmap?

We are currently in full commercial expansion. We already have French clients who validate our technology, but we now want to expand internationally. I will also be spending two weeks in the United States soon. We are also considering a fundraising to accelerate. Our goal is to make our solution available to as many people as possible through open source, but also to expand our portfolio of paying customer companies. Now that the technology has been tried and tested with the first French clients, the aim is to give it a global scale and build a true community on both the open source and business aspects.

Leave a Reply

Your email address will not be published. Required fields are marked *