It's an independent assessment of how genuinely useful, consistent and true to the brand's tone a corporate chatbot or virtual assistant really is. We simulate real conversations as a customer would, exactly as in traditional mystery shopping, and measure service quality across multiple dimensions, summarised in a single public score, the Omnia Score.

The name echoes the principles of mystery shopping applied to the relationship between customer and customer service. We call it an “incognito” audit for chatbots too because the corporate bot is tested by an adversarial AI agent that, without agreeing the questions in advance, realistically tests the quality, consistency and efficiency with which it handles the service without human intervention.

Before every test we verify that we're speaking with an automated system (gate 00). We then run realistic conversations on real-world scenarios, assessing service quality across multiple dimensions, from the ability to genuinely resolve the customer's need to the consistency of answers across variants, channels and over time, through to fidelity of tone and style to the brand, with scores from 1 to 10 and sample-based human calibration. The final result is the Omnia Score. Any GDPR compliance issues that emerge during the conversations are flagged in the report, but they are not the focus of a dedicated adversarial test.

Yes: the service is based on a contractual relationship with the client company, which authorises us to test its own systems, in line with the principles of independence, objectivity and professional integrity of the UNI 11312 Standard on mystery auditing. We never test third-party systems without the explicit authorisation of the service owner. Tests are always purely conversational: no technical exploits, no credential manipulation, no real transaction is ever completed.

Website, app, WhatsApp, Telegram, Messenger, SMS/RCS and voice/IVR systems, as well as, naturally, the chat on the website. For channels without API access (e.g. WhatsApp Business standard), the test is carried out manually by a human operator, following predefined scenarios.

It isn't mandatory. For channels with an available API, access to a test environment speeds up the work, but most of the testing happens conversationally, just as a real customer would. We never ask for access credentials to internal systems or real data about your customers.

We use exclusively fictitious but plausible data (names, orders, test requests) during the tests, never real data about your customers. Reports and conversation transcripts are shared only with the representatives authorised by the client company.

The UNI 11312 Standard is the Italian reference standard for mystery audit services. We apply its methodological principles (independence, objectivity, repeatability of assessment) to the context of corporate chatbots too, a field that is new compared with traditional mystery shopping. We also adhere to the professional principles the standard sets out: we never use a chatbot to test real people, only to assess the quality of the service provided by AI or digital agents, in a relationship that always remains machine-to-machine.

It depends on the number of channels and scenarios to be tested. An initial pilot package on a single channel typically takes 1-2 weeks between planning, running the tests and delivering the report.

The cost varies depending on the number of channels tested, the volume of scenarios and the depth of analysis required. Contact us for a tailored assessment based on your needs.

A report with the Omnia Score and a breakdown of the evaluation indices, the scenarios tested with the bot's responses, the issues identified (including any GDPR compliance observations that emerged during the conversations) and practical recommendations for improving the customer experience and consistency with the brand.