Cloud-Based Manufacturing Optimisation: A Student's Story with Azure and AI

Have you ever tried to create something while simultaneously learning how to use the tools? My team and I got to experience that in our project with Azure services in our final semester to create a manufacturing optimisation solution.

We survived, learned a ton, and even managed to build something pretty cool. It would be nice to have a demo, but it's our client's. However, please strap in for the ride if you have time.

While the work was divided equally among team members, I took on the role of team leader where I provided a broad overview of the project, architecture design, set expectations, and orchestrated the team's efforts into a coherent product - in this case, a software module integrated into our client's existing system.

The Challenge: Making Machines Smarter

The Picture: You have manufacturing machines generating tons of data through sensors, but that data is just sitting there, not really doing much. So we wanted to create a software solution that could not just monitor these machines, but actually make sense of all that data and help engineers optimise operations.

The core idea was to build a system that could:

• Monitor machines through sensor data
• Store historical performance data
• Use generative AI to query data, create reports, recognize faults, and raise alerts
• Visualise everything in a user-friendly format

IoT Sensor Data Analysis, Anomaly Detection and LLM Integration System

1. Data Collection and Ingestion

Our system was designed to collect key parameters like temperature, speed, torque, and positional data from various sensors installed on the machines. Along with Raspberry Pi devices as edge computing units, we used OPC UA protocol via PLCs (Programmable Logic Controllers) to translate machine signals into readable data.

So the pipeline looks like this:

2. Data Storage and Processing

Once the data was collected, 2 databases were used:

• Azure Cosmos DB: Stores raw IoT messages
• Azure Blob Storage: Stores processed performance metrics (OEE, OT, OE) for LLM consumption

For data processing, we implement an Event Hub Trigger Azure Function with Python 3.11 runtime to:

• Parse incoming IoT telemetry
• Update Digital Twin properties
• Calculate statistical metrics
• Store processed data in Blob Storage

3. Anomaly Detection and LLM Integration

For anomaly detection, we used Azure Machine Learning to train a simple model that could predict machine failures based on historical data. The model was then deployed as a web service and integrated into the Azure Function App to raise alerts when anomalies were detected.

Workflow:

• The backend retrieves data from Cosmos DB
• Data is scaled using a pre-trained scaler
• The maintenance alert model predicts and raise result toward frontend dashboard

4. Digital Twin Integration

Digital Twin Setup

• Create Azure Digital Twin models for the manufacturing system (e.g., Gantry System)
• Define relationships between components:
  • Vacuum → Storage: have_access_to
  • ControlPad → Vacuum: controls
  • Conveyor → Sensors: linked_to

Real-time Monitoring

• IoT data updates Digital Twin properties via Function App
• The Digital Twin model provides a virtual representation of the physical system
• Use 3D visualization (Azure Digital Twins 3D Scenes) and in-house dashboard to monitor machinery status in real-time

5. LLM Integration for Log Processing and Optimization

Azure OpenAI Services Setup

• Deploy GPT-4o model in Azure OpenAI Services
• Set up text-embedding-ada-002 model for vector search capabilities
• Configure system message and parameters

Integrating Machine Data with LLM

• Process and store machine performance metrics in Azure Blob Storage
• Set up Azure AI Search to index this data
• Configure vector search for semantically relevant retrieval
• Connect the Azure OpenAI Service to search this indexed data

Optimization Suggestion Workflow

• Users interact with the LLM via a chat interface
• The LLM queries Azure AI Search for relevant machine data
• The system combines its knowledge with real-time and historical data
• It generates reports and optimization suggestions based on this integrated information

For an example prompt, it goes something like this:

"Analyse the raw data and statistics from the time period and generate a summary.
Your report should include the following sections:
1. Executive Summary
2. Variable Trends
3. Anomalies Detected
4. Potential Faults
5. Optimization Strategies
For each section, provide concise, data-driven insights. Use specific numbers and percentages where relevant."
            

Azure: A Maze

Working with Azure was like exploring a city where every service is its own neighborhood. Some neighborhoods are friendly and work great together, while others... let's just say they need better public transportation.

We used various Azure services for storing data, processing it, and running AI models. But here's where it gets interesting (and by interesting, I mean challenging 🎢).

Combining Azure services is a bit like following a recipe with missing instructions. Here are some fun challenges we faced:

1. Documentation: Sometimes Azure's documentation felt like a treasure hunt. "Oh, you want these two services to work together? Here's a cryptic clue in the docs!" While slightly exaggerated, the point stands. When designing infrastructure for clients, we couldn't say with confidence if things would work as intended. The amount of documentation diving needed to verify feature availability was too extensive for proposal phases.
2. The Integration: Getting different Azure services to work together was way harder than it should be (had to thank my teammates for figuring this out before I go ape-****). Often you have two Azure services that you expect to work together nicely, but they don't or require some roundabout way to integrate. For some services, I spent weeks trying to find ways for them to talk to each other, only to realize it wasn't documented anywhere and the only working solution was a roundabout approach using additional services. This problem might not be a big deal if you use the Azure native version of a common technology (like Azure SQL vs Postgres for Azure, Synapse vs Databricks, Cosmos vs other non-structured DBs), but then you hit point number 1 again - oops, that feature in open-source product A isn't in our version yet, it's on the road map or in preview mode.
3. The "It's Not You, It's Me" Moments: Azure services occasionally decided to take their sweet time responding, like waiting for a text reply that never comes (something like this -> ).

Lessons Learned (The Hard Way)

• Reading Documentation is an Art: Beyond just reading, you need to read between the lines. Often, what's missing is as important as what's included. To add to this, the number of things that are poorly or ambiguously documented across multiple locations is too high. Maybe it's just me.
• Integration Testing is Your Best Friend: When working with multiple services, test their interactions early. Trust me, finding out two services don't play nice together, have clunky interactions, occasional timeout errors, or bugs after building half your solution is not fun.
• Keep It Simple: Start with the basics and add complexity gradually. This is something that I have learned from my past mistakes and from my teammates as well. They are very good at breaking down the problem into smaller pieces and then solving them one by one. This is a good approach to avoid getting overwhelmed by the system's complexity.

Don't get me wrong, Azure has the most compelling storyline because it produces the top BI tool on the planet (PowerBI), the most common enterprise RDBMS (SqlServer), the most common business productivity suite (O365), and the most common authentication service (AAD). If I have to guess, probably MSFT doesn't pay as much attention to the DE or developers' side of the business, as many DEs are code-heavy.

The Silver Linings

Despite the challenges, we built something genuinely useful:

• A real-time monitoring system
• AI-powered insights for optimisation
• Interactive 3D visualisations
• A chat interface for querying system data

And the best part? Everything actually worked together (after some convincing).

Personal Takeaways

Looking back, this project was more of a proof of concept, a prototype for our client rather than an actual staging product, like solving a puzzle where the pieces keep changing shape. Frustrating? Sometimes. Educational? It taught me that in the real world, technical skills are just part of the equation. Problem-solving, persistence, and being able to shrug it off are equally important.

Want to see how it all came together? Check out our project presentation video where we demonstrate the system in action. Our teachers seemed impressed, so we must have done something right.