Azati Software

Our Data Scientists have successfully implemented a prototype system into an already functioning calorie counting application, that can instantly estimate the calorie content of complex dishes by images analysis. Such a solution can be useful in such domains as agriculture, catering, sports or even for everyday life.

OBJECTIVE
In recent years, it has become possible to use deep learning to recognize objects in images with high accuracy. We realized Azati could apply new technologies to the problem of food quality estimation to simplify the process and provide the user with the fastest and most efficient result.

SOLUTION
While solving challenges we developed a small script written in Python. The prototype takes an image as the input and returns a set of frames where each component is circled with a square in which calories are indicated, and the total result of the whole dish is displayed. Check out the screenshots below to see how the results look.

RESULTS
We made it possible to process images of compound dishes and calculate their calorie content using machine learning and computer vision. The model recognizes each product quite accurately and distinguishes one component from another. For more complex object classification, the model requires additional data and extra training.

Objective

The primary purpose of the project was to develop an analog for ChatGPT by using an open-source language model framework. This venture provides employees with a very comfortable communication tool and at the same time keeps the confidentiality of the corporate data. The efficiency of our communication is improved; at the same time, the company is prepared to meet the new demands of the present-day business environment in a better way.

?hallenges

Challenge#1. High computational resource requirements:

A key challenge encountered in developing an alternative to GPT was meeting the computational resource demands.

Developing language models, like OpenAI GPT’s, demands an amount of computational power for training and optimal performance. Securing computing resources can pose a challenge for businesses operating on tight budgets, which may impede the smooth execution of the project.

Challenge#2. Identifying optimal models comparable in quality to OpenAI GPT’s:

It required research and testing of open-source LLMs, not just focusing on text generation quality but also on understanding context and user interaction in a corporate setting. This process could take some time due to the need to compare models.

Challenge#3. Ensuring the security of corporate data:

Creating a secure analog to the GPT also raised concerns about the security of corporate data. Considering the sensitivity of information exchanged in a corporate setting, mechanisms for data encryption and other security measures needed to be developed. Ensuring the security of employee communication and protecting corporate secrets became an integral part of the process of creating such a system.

Process

1. Choosing a Suitable Language Model (LLM):
   The process began with a review of open-source language models considering factors like performance, accuracy, and suitability for corporate use. Popular models such as BERT and GPT 2, among others, were evaluated to determine which one aligns best with the project objectives.
2. Fine-tuning using LoRA (Low-Rank Adaptation):
   After the base LLM is chosen, domain-tuning is the next step. This is accomplished via the expansion of the model to satisfy the needs of the application setting which include terminology, business needs, and other business characteristics.
3. Quantization:
   After fine-tuning, the quantization — an optimization technique that reduces the model’s memory footprint — was made. It was important since in an enterprise setting the computational resources might be very limited.
4. Enhancing the Retrieval-Augmented Generation (RAG) Approach:

   To optimize interactions enhancements were made to the RAG approach to methods:

   • Addressing questions: Enhancing response generation for queries.
   • Multiquering: Expanding capabilities to handle queries simultaneously.
   • Parent retrieval: Improving retrieving parent queries efficiently.
   • Hypothetical questions: Introducing questions to enrich content creation.
   • Keyword and topic extraction: Enhancing keyword and topic extraction procedures.
   • HyDE (Hybrid Data Enhancer): Employing a technique to enhance data quality.

   These measures not only tailored the model to fit requirements but also boosted its performance in query handling and content generation.

Solution

The solution to the task involved the following key steps:

Development of an Independent Internal Service:
By choosing an LLM and using techniques, like tuning, quantization, and an enhanced RAG method, an independent internal service was successfully developed. This platform offered a smart way of sharing information designed to meet the needs of the business world.

Ensuring the Security of Corporate Data:
To safeguard information, security protocols were established across different tiers. These measures encompassed encryption techniques, stringent access controls, and additional technologies aimed at guaranteeing defense against data breaches.

Integration with Confidential Data:
In the end, the developed service was smoothly connected with the organization’s databases allowing easy data exchange among staff members. The main goal in the meantime was to customize the service according to the organization’s data structures and specific needs.

Testing and Performance Evaluation:
Following the development and integration of the service, various tests were carried out to evaluate its performance and efficiency. These assessments included real-world usage scenarios, analysis of response times, and validation of adherence to security protocols.

Employee Training and Implementation:
To ensure an implementation of the service, comprehensive training sessions were conducted for employees. This involved getting acquainted with the interface, understanding its functionalities, and receiving guidance on usage practices.

Because of these steps, the successful creation and implementation of an independent internal service were achieved, completely replacing ChatGPT and ensuring the secure handling of confidential corporate data.

Azati helped a Canadian customer develop an AI-powered service for automatic data processing from meters that measured produced oil & gas resources using machine learning and computer vision technologies.

Customer

Petroleum products are the basic fuel for most types of transport, as well as raw materials for chemical production. Natural gas is one of the best types of fuel for domestic and industrial needs, polymers are made from gas, and helium is released, which is used in the production of high-precision equipment and in the space industry.

The oil & gas industry plays a leading role in the economy and is closely linked to other industries. This is a complex system, including the raw materials extraction, the production of fuel purification and its further processing. An important role in this redistribution is given to modern specialized technologies.

A Canadian oil & gas customer service company turned to Azati to automate reading data from meters.

Objective

Today, any industrial complex of the oil and gas industry must be fully automated. So numerous controllers, meters and block modules are created. Production automation leads not only to decrease the influence of the human factor but also to increase efficiency.

The customer turned to us to search and develop approaches to automate the data reading from graphs that are printed by equipment for accounting for extracted resources (meters).

The task included graphic information processing(recognition and reading of curves on the graph), printed data (stickers with printed text, graphics, such as barcodes), as well as handwritten data (dates, numbers, notes from equipment operators).

Challenges

01. Challenge

The equipment prints data on round discs, which are scanned and sent to the system for subsequent reading and data processing. Thus it was necessary to find models and solutions that could work with such incoming data.

To overcome this challenge we have developed an algorithm that can unfold a round disk and convert the image into a rectangular shape to make it possible to trace and read the coordinates (x, y) of the fixed curves.

02. Challenge

The equipment prints several curves indicators that characterize the conditions for resource extraction on one graph. Each curve has its own color. For some of them, the background color could overlap with the line color, so it was necessary to find a solution that would allow to highlight the curves on the graph, bypassing this extraneous “noise”.

We have done work on training the neural network with test materials, which could select curves of different colors on the incoming graph with sufficient accuracy, considering all the features and potential interference.

03. Challenge

The customer has many partners around the world, who use different equipment. The problem arose while processing various data formats.

We have trained a neural network that, based on the characteristics studied, could conclude that the input data belonged to one or another client, and, depending on this, send it to the necessary data processor.

04. Challenge

Another problem was related to handwriting recognition because of the human factors: excitement, haste, absent-mindedness, etc.

The problem was solved by searching for data regions. From a few regions (bounding box), an attempt was made to recognize handwritten data (dates and numbers) using a trained neural network based on Google Tesseract.

Process

We started the project by creating a successful pilot prototype for reading curves on a graph. This task turned out to be feasible, and we started development to recognize other aspects and details of the input data (graphs).

The Canadian side was involved in project management, prioritization of activities, and coordination of the delivery schedule. The client, in turn, decides how closely the potential result meets the project’s goals and is suitable for marketing analysis.

Solution

The project was done by a team of ML specialists.

The product is a set of services that receives a scan of the input data and the expected result on the output, recognized and calculated by the developed model based on artificial intelligence.

The services were integrated into the client’s infrastructure and launched in the cloud.

The developed services included the following functionality:

• 90% accuracy of barcodes processing
• Above 80% accuracy of line processing
• Processing of handwritten data (dates and numbers) varied greatly from the quality of the input data, from 30 to 70+%. Everything rested on the human factor, i.e. the accuracy of the data in the form fields, blots and corrections, handwriting features.

Customer

Health is the highest value, but unfortunately, only a few think so. Drug manufacturers, pharmacies, doctors have to make a lot of effort to promote the idea of ??health to create a desire to be healthy. You can effectively promote health only if you have a good understanding of people and their motivation. These are the problems that pharmaceutical marketing is working on.

Pharmaceutical products are pretty sophisticated for understanding by non-experts in Healthcare, so conventional marketing is hardly applicable to them. Therefore direct pharmaceutical marketing is more appropriate for such aims. Simple marketing is used when small earnings are needed, pharmaceutical companies are looking for ways to make much more profit and prefer advanced marketing.

Our customer is an entrepreneur with extensive experience in the marketing business who turned to us with a unique idea to help pharmaceutical companies increase the level of trust of ordinary buyers who know little about the drug’s composition.

Objective

Pharmaceutical marketing, in this case, is not just the sharing of information in different sources but also the quality and compliance with the declared one. And in healthcare, it is sometimes difficult for an ordinary person to understand which pharmaceutical composition will be better or more appropriate. Hence, people often buy “popular” medications that they hear about.

The customer turned to us with a unique idea, which had not yet been on the market. The essential purpose was to facilitate the search and comparison of the required product with the recommendations of doctors based on questionnaires and insights from professionals.

Our task was to develop an MVP using Artificial Intelligence and Machine Learning technologies to build assessment reports for pharmaceutical companies. Azati’s team studied models and tools for solving ML tasks, including speech-to-text, text mining, finding similar phrases and mismatches using NLP.

We also learned data visualization tools to present the resulting reports in an understandable form.

Challenges

01.Challenge

Since the task was new and sophisticated for our team, we faced the inefficiency of the available tools at some development stages.

At the customer’s initiative, we invite a third-party consultant – Doctor of Science and an expert in his specialty. He recommended several ways that we worked on and thus came out of the creative impasse.

02.Challenge

The generated reports had to be structured and easily readable by any user. We developed a custom system to build a clear report to evaluate the found phrases, classify them, and sort them according to the calculated ratings.

Process

Development process step by step:

Stage 1.

It was necessary to decide how to bring all the input data to a standard format with which the AI ??algorithm can work. We had to solve speech-to-text recognition issues, model training for specific terms, solve the nuances of punctuation and capitalization of subtasks within the framework of NLP.

Stage 2.

We solved problems directly related to NLP by analyzing input text data, finding common phrases, segmenting phrases by topics, assessing sentiment analysis, and calculating scores for found phrases.

Stage 3.

On this stage we solved the problems of grouping the obtained data into thematic reports that highlight one or another aspect of analyzing the respondents’ answers.

Stage 4.

At the final stage, we considered the means for visualizing the received reports and finding a simple and intuitive tool for sharing and demonstrating the results and main conclusions to potential customers.

Solution

As a result, we have developed an ML model that can analyze data from medical questionnaires, find insights, and build reports for the end-user within the existing pharmaceutical domain.

We have built an ML model that can automate the following tasks for marketing analysis:

• Analyze questionnaires from a group of doctors, and identify similar answers;
• Compare and correlate the aspects voiced by doctors with the offers in the marketing strategy of pharmaceutical companies;
• Generate reports where users can see what doctors-practitioners are talking about and what they value regarding the specified pharmaceutical product (effectiveness, safety, usage, etc.) and how the marketing strategy for this product can be improved.

Custom system for engineering drawings digitization powered by artificial intelligence to extract data from on-paper maps, schemes, and other technical documents.

Customer:

Together with our Strategic partner DIGATEX, we combined our software and data science skills and their domain knowledge of engineering data management to create DI-analytics, a unique solution to digitizing vast amounts of advanced documents for customers who own and operate complex assets such as oil refineries and offshore production facilities.

One of the first customers for this solution is a South East Asia corporation that explores and manufactures petrochemical products. The company is ranked among Fortune Global 500’s largest corporations in the world with business interests spanning 35 countries.

Due to specific business demands the customer regularly has to digitize vast amounts of advanced documents. The service was provided as an outsourced process comprising document processing, data extraction and collation.

Objective:

The objective was to build a solution for digitizing a large number of complex documents in the shortest terms. The majority of documents were pipeline layouts, industrial plans, manufacturing schemes and maps obtained from the third-party vendors and partners.

CHALLENGE #1:

All documents from a single vendor or partner can be divided into several groups, and each group has multiple document templates. So, hundreds of vendors lead to the thousands of templates.

It is very challenging for a human not only to remember all the templates but also to determine the right template that suits the specific document. The first concern we faced was to determine the document template, to know what kind of data to extract.

CHALLENGE #2:

Another challenge was to extract the data from the technical documents. Every template has it is own unique set of fields, custom abbreviations and unique symbols in addition to the flexible structure.

Our goal was to make the application to determine the zones and fields of the document automatically, without manual mapping. It is a very challenging process if we take into consideration the number of various templates.

CHALLENGE #3:

The majority of schemes and plans were autogenerated by other software applications in multiple steps. It was the usual situation when the information we need to extract lies under another element or abbreviation. It is challenging even for the human to read some schemes.

Our engineers decided to train machine learning to recognize complicated elements according to its previous experience and already extracted data.

Process:

After initial research, we figured out – that no existing technology could help us to overcome the customer challenges. Several companies provide similar services, but their products are entirely unsuitable for documents with flexible structures and industrial maps.

Our engineers DECIDED TO build a custom Optical Character Recognition (OCR) Engine powered by Artificial Intelligence.

AI was a good option – it acts like a human, and it uses the same algorithms and methods while searching the data patterns in the document as the human does.

The solid scientific background helped our engineers to build MVP in less than two weeks. We immediately requested the first documents from the customer and got a predictable result, that impressed the customer.

We processed about 10.000 documents in less than 8 hours with an average accuracy of 84%.

Since that moment, we have been tuning algorithms and improving the performance of the system.

Now the accuracy of extracted data is close to 97%.

Solution:

The final application is the entirely modular system, hosted in the secure enterprise cloud. All ongoing tuning and maintenance are entirely remote, which helps the customer avoid on-site personnel training and cut down maintenance costs.

We are proud to say, that a small group of neural networks powers every single module, and all the modules form a unique artificial intelligence that takes the document as the input and provides the accurately extracted data as the output.

As artificial intelligence is hosted in the cloud, it can be easily managed from any place. If the customer wants to process a considerable number of documents in the shortest terms, we can enable the required resources in several minutes and handle any number of documents.

Idea

The project idea revolves around enhancing the dining experience through innovative technology. Each table is equipped with a sophisticated sound system allowing customers to naturally issue commands like: “call the waiter”, “give the bill”, “bring the bread”, “provide the menu”, etc.

These commands seamlessly integrate into the control system, where they undergo interpretation, context analysis, and the extraction of any necessary supplementary details. Automatically, tasks are generated, assigned to the right staff member, and precise timers are set for performance tracking.

Once assigned, tasks are sent to the staff member’s device with voice commands for specific actions, creating a personalized to-do list. Reminders are sent when the timer runs out.

Conversely, the waitstaff also engage with the system through vocal interactions. For instance, they might say, “I’ve taken an order for table 7: one black coffee and one croissant,” or “Please arrange a taxi for table 3”. And the task will be created. Additionally, they provide timely status updates on completed tasks, such as “The bill has been settled for the guest at table 5,” triggering automatic task completion in the queue.

All information is duplicated on internal resources, dashboards, and screens. The system allows monitoring current processes, quickly finding bottlenecks, identifying and fixing problems.

Objective

The project aimed to develop and implement a system utilizing machine learning to recognize and analyze speech from both restaurant employees and customers. Key tasks involved converting speech into text, extracting commands and their attributes, and discerning when commands were completed.

Furthermore, the project successfully implemented the application architecture, established a task management system, and enabled seamless communication between the server and wireless devices for prompt command processing.

Part of the analytics and optimization includes analyzing the efficiency of all processes, identifying weaknesses and bottlenecks in the system, which allows improving its productivity and operational efficiency.

?hallenges

CHALLENGE#1:

Multilingualism is a key challenge encountered in this project.
The objective here was to ensure the system’s proficiency in accommodating multiple languages. This entails the incorporation of support for foundational languages such as English, Spanish, and French, along with their possible combinations, to meet the needs of multilingual customers. Achieving this goal required creating and integrating mechanisms specifically designed to adeptly recognize and process a wide array of language-based requests.

CHALLENGE#2:

Voice commands should not be formalized.
Customers and employees alike should be able to express their requests and instructions in natural language, without having to follow a strict format. The system must be able to automatically identify commands in their speech, determine the attributes of those commands, and create appropriate tasks for execution.

CHALLENGE#3:

Deploying the model on weak hardware.
Another significant problem in the project was the need to deploy the model on weak hardware. The customer expressed a desire to have a resource-efficient solution that could be deployed locally even on ordinary laptops. This posed challenges as the model had to be optimized to function under limited computing conditions.

Process

The journey of crafting a speech processing system involved numerous stages where our accumulated expertise proved invaluable. Commencing with speech recording and digitization, we progressed to the audio-to-text transformation, a pivotal phase in translating audio data into comprehensible information for the system. Subsequently, we delved into text content analysis, deciphering not just the words but also the emotions and intonations, enabling a profound grasp of the statements’ meanings and users’ needs. This meticulous, multi-step process served as the bedrock for the creation of a speech processing system that is both precise and highly responsive.

Solution

To implement the ML part of the project, we deployed and configured machine learning models. This process involved training the model to identify several dozen core commands, which required collecting and annotating a large amount of data. We then conducted preliminary testing and verification of the model, creating a Proof of Concept (POC) to ensure its functionality and effectiveness. Finally, we successfully demonstrated the developed ML system, showing its ability to accurately recognize and process a variety of commands, which confirmed its readiness for integration into the overall project architecture.

Results

We verified and showcased the functionality of the essential components within the machine learning domain: speech analysis, command identification, interpretation, attribute search, and task formulation. A proof of concept (POC) was meticulously prepared and presented to the client, complete with detailed calculations and a comprehensive commercial proposal.