Lessons from Building an AI Chatbot That Actually Qualifies Leads

We recently built an AI-powered qualification chatbot for a client’s Facebook Messenger and Instagram channels. The project started as a proof of concept and turned into one of the most interesting builds we have done this year.

Here is what we learned, what went wrong, and what Gold Coast business owners should know before trying something similar.

The Problem: A Chatbot Stuck in 2018

Our client had a ManyChat chatbot handling enquiries from Facebook Messenger, Instagram, and TikTok. ManyChat is a solid platform. It lets you build automated conversation flows without writing code.

The issue was that their existing chatbot was built before large language models existed. It followed a rigid script. If someone said something unexpected or asked a follow-up question, the bot got confused. It could not handle the way people actually talk.

The client wanted something that felt like talking to a real person but still collected the specific information they needed to qualify leads. They wanted the best of both worlds: the natural conversation of ChatGPT with the structured data collection of a traditional form.

Why We Did Not Use ManyChat’s Built-In AI

ManyChat does have a ChatGPT integration. We looked at it. It was not great for what we needed.

The built-in integration lets you send a message to ChatGPT and store the full response. But it does not do structured extraction. It cannot pull out “first home buyer: yes” and “deposit: $40,000” from a conversational message and store them as separate fields. It just gives you back a blob of text.

For lead qualification, you need the AI to do two things at once. It needs to reply naturally to the user AND extract structured data from what they said. The built-in integration only does the first part.

So we built a middleware instead.

The ManyChat Flow Builder: when a new message arrives, it triggers an external request to our Azure middleware. The middleware returns the AI response and extracted fields, which ManyChat stores before replying.

The Architecture: A Simple Idea That Got Complicated

The concept was straightforward. ManyChat sends the user’s message to our middleware. The middleware sends it to an AI model along with the conversation history. The AI extracts the relevant information and generates a natural response. The middleware sends everything back to ManyChat, which updates the contact fields and replies to the user.

We built the middleware as an Azure Function. It is just a script that runs in the cloud and responds to web requests. The conversation history gets stored in Azure Table Storage, which is a cheap, simple database.

The AI model itself runs on Azure OpenAI, which is just ChatGPT hosted in a private environment. This matters for businesses handling sensitive customer data because nothing goes to a shared public API.

Simple on paper. In practice, we hit a lot of interesting problems.

The Azure Function App running in Australia East. The process_message function handles the core middleware logic with a 1.8-second average response time.

Lesson 1: Let the AI Do What AI Does Best

Early on, we made the mistake of trying to hard-code too much logic into the script. We had the script doing intent detection with keyword matching, deterministic field extraction, and rigid conversation routing.

It did not work well. People do not talk in keywords. Someone might say “this will be my first home” instead of “yes” when asked if they are a first home buyer. A hard-coded script cannot handle that. An AI model can.

The breakthrough came when we split the responsibilities clearly:

This hybrid approach worked much better than either extreme. The AI is brilliant at understanding messy human language. The script is reliable at following business rules that should never change based on how the AI is feeling that day.

Why this matters for your business

If you are thinking about adding AI to your customer interactions, this is the most important lesson. AI should handle the parts that need flexibility and understanding. Your business logic should stay deterministic. You do not want an AI deciding whether a customer qualifies for something. You want it extracting the data so your rules can make that decision.

Lesson 2: One Prompt to Rule Them All

We initially tried multiple AI calls per conversation turn. One call to detect intent, another to extract fields, another to generate the response. This was slow and expensive.

The solution was a single unified prompt that does everything at once. Each time a user sends a message, we send one request to the AI that includes:

• The full conversation history
• All fields collected so far
• The current question being asked
• A list of all possible fields that could be extracted
• Instructions on how to respond

The AI returns a single JSON response containing the extracted fields and a conversational reply. One call. Under two seconds.

This approach also made it possible for the AI to correct earlier extractions. If someone said their income was $2,000 a week in the first message and then corrected it to $2,500 later, the AI could update that field because it had the full context every time.

Azure OpenAI Studio showing the unified prompt approach. One API call handles both natural conversation and structured field extraction, returning a JSON response with all captured data.

Lesson 3: Plan for When AI Fails

AI models are not perfect. They have rate limits. They sometimes return unexpected responses. Content filters can block legitimate messages. Network issues happen.

Before we added proper error handling, a failed AI call would leave the user hanging with no response. That is worse than a bad response.

We built fallback mechanisms at every level:

• Rate limiting: If the AI is getting too many requests, the system pauses and retries rather than crashing
• Bad responses: If the AI returns something that does not match the expected format, the middleware asks it to try again or falls back to a safe default message
• Content filters: If a message gets blocked, the user gets a polite redirect rather than silence
• Complete failure: If everything goes wrong, the conversation data is preserved so nothing is lost

The ManyChat platform itself also had a built-in AI chatbot that could act as a safety net. If our middleware was unavailable, the conversation could fall back to the standard ManyChat flow. Not as good, but far better than nothing.

Why this matters for your business

Any AI solution that does not have a Plan B is not ready for production. Your customers do not care about your API rate limits. They care about getting a response. Build the failure modes first, then the happy path.

Lesson 4: Testing AI Conversations Is Different

Testing a traditional chatbot is straightforward. You send a specific input, you expect a specific output. Testing an AI chatbot is harder because the AI generates different responses every time.

We came up with an approach that worked well. We gave the AI model our qualification flowchart and asked it to generate test conversations with expected outcomes. This gave us dozens of realistic conversation scenarios that we could run through the system automatically.

We also added detailed logging to every AI call. Each time the middleware talks to the AI model, it records exactly what was sent and what came back. This made debugging much easier when something went wrong. Instead of guessing why the bot said something strange, we could look at the exact prompt and response.

Lesson 5: The Loop Problem

ManyChat works by triggering automations. When a user sends a message, it triggers a flow. That flow calls our middleware, gets a response, and replies to the user. When the user responds again, a new automation triggers.

The tricky part is that ManyChat was not designed to loop back to an external service repeatedly. We had to build a mechanism where the automation effectively called our middleware in a loop, continuing the conversation until the AI collected all the information it needed.

This caused some unexpected bugs. At one point, users who said they were from outside the service area got stuck in an infinite loop. The system would clear the location field, ask again, get the same out-of-area response, clear it again, and repeat forever.

The fix was not a logic error in the traditional sense. It was a field management issue. The system was clearing a field it should have been keeping, which caused the loop. Once we identified that, the fix was simple. But finding it took hours of debugging.

Lesson 6: AI and Extraction Do Not Mix Well (At First)

One of the trickiest issues we faced was combining field extraction with other tasks in the same AI call.

When we asked the AI to extract data AND detect whether the user wanted to exit the conversation, it started failing at extraction. The income field would come back empty. The deposit amount would not get captured. The AI was spending all its attention on figuring out if the user wanted to leave and not enough on actually pulling out the data.

The fix was to restructure the prompt. We gave the AI clear examples of how extraction should work and made the extraction the primary task with exit detection as secondary. We also referenced a previous version of the prompt that had worked well for extraction and asked the AI to apply those same methods.

This is something worth knowing if you are building AI into any workflow. When you ask an AI model to do multiple things at once, it will prioritise some tasks over others. You need to be explicit about what matters most.

Lesson 7: This Replaces Forms, Not People

The most important takeaway from this project is what the chatbot actually replaces. It does not replace a salesperson or a customer service agent. It replaces the form.

Think about the last time you filled out a contact form on a business website. Name, email, phone, what are you looking for, what is your budget. It is boring. A lot of people abandon forms halfway through. And even when they complete the form, you get bare minimum information.

An AI chatbot collects the same information through a conversation. It feels natural. It adapts to how people actually communicate. Someone might volunteer their income, location, and buying intent all in a single message, and the AI captures all of it without asking three separate questions.

For the client, this means leads arrive pre-qualified with all the information their team needs. No back-and-forth emails. No phone tag to collect basic details. The chatbot does that work 24/7 across every channel.

What This Means for Gold Coast Businesses

You do not need to be a tech company to benefit from this kind of AI integration. If your business qualifies leads, books appointments, or collects information from customers through online channels, this approach works.

The middleware pattern we used is not specific to any industry. It connects a conversation platform (ManyChat, or anything similar) to an AI model through a custom layer that you control. That custom layer is where your business rules live.

Here are some examples of where this approach could work:

• Real estate: Qualify buyers by budget, location, and timeline through Instagram DMs
• Trades: Collect job details, location, and urgency through Facebook Messenger before dispatching
• Professional services: Pre-screen potential clients by service type and budget through website chat
• Healthcare: Triage patient enquiries by symptoms and urgency before booking appointments
• E-commerce: Help customers find the right product by understanding their needs through conversation

The key is that the customer gets a better experience than filling out a form, and your team gets better data than a generic enquiry.

Mockup of what the customer sees: a natural Messenger conversation that feels like talking to a real person. All qualification data is collected without a single form field. Mockup of what the team sees in ManyChat: every qualification field extracted and stored automatically. The contact is tagged and marked as qualified before a human ever gets involved.

Should You Build This Yourself?

Honestly, probably not. The middleware approach requires someone who understands APIs, cloud hosting, prompt engineering, and conversation design. It is not a weekend project.

But you also do not need to spend $50,000 on a custom AI solution. We built this proof of concept for under $3,000, and the ongoing costs are minimal because AI API calls are cheap at this scale.

The important thing is to start with a clear use case (we wrote about a similar approach in our AI phone system project). Do not build an AI chatbot because AI is trendy. Build one because you have a specific problem: forms are not converting, leads are not qualified, or your team is spending hours on repetitive initial conversations.

If you can answer those questions, you have enough to start a conversation about whether this approach makes sense for your business.

What We Would Do Differently

If we started this project again, we would spend more time on the prompt before writing any code. The prompt is where 80% of the value lives. Getting the extraction right, the tone right, and the fallback behaviour right in the prompt saves weeks of debugging in the middleware.

We would also separate the testing environment from production earlier. Running tests on the same ManyChat automation that real users interact with is a recipe for confusion. Having a separate test flow from day one would have saved us a lot of time.

And we would document the decision tree before touching any code. We spent time building the middleware against an incomplete qualification flow and then had to rebuild parts of it when the full requirements came through. Getting sign-off on the complete business logic first is always faster in the end.

Frequently Asked Questions

The Bottom Line

AI chatbots that actually work are not magic. They are a combination of good prompt design, reliable middleware, solid error handling, and clear business rules. The AI handles the messy human conversation part. Your business logic handles everything else.

If your Gold Coast business is losing leads because your forms are boring, your phone lines are busy, or your team cannot respond to Messenger and Instagram enquiries fast enough, this kind of solution is now within reach.

We are happy to have a conversation about whether it makes sense for your specific situation. No forms required.