We gave our phone system the AI treatment

We recently replaced our phone system’s hold music with a track written by AI.  The result was a very jazzy, catchy and cringeworthy tune about cybersecurity, Burleigh Heads, and the Essential Eight. I thought it was lovely, but it got mixed reviews.

The song was easy to make with the Suno app and we had a great time generating a playlist of GCIT-related tunes. But it got us thinking, how can we apply some of other AI’s other strengths in summarisation, transcription, and sentiment analysis to the rest of our phone system?

Using AI to Improve Service Desk Operations

While AI’s encroachment on creative industries has a range of (many negative) implications, there is no denying its ability to help us do things we simply didn’t have the time to do before. For instance, imagine if you had a dedicated team member who could oversee all of your interactions with customers, summarise lengthy interactions and phone calls, analyse sentiment and provide suggested improvements?

We’ve built a simple system that:

• Transcribes every service desk call in near real-time
• Provides a summary of each conversation
• Scores the customer sentiment
• Updates a dashboard every 10 minutes

Caller details and call content are blurred for privacy.

Now, you might be thinking, all this stuff has been possible for a while, so -

What difference does using an AI model make?

Businesses that embrace automation have always had an advantage over ones that don’t. But there’s been some things that have been notoriously difficult to automate, especially for smaller organisations.

When it comes to tracking and improving the customer support experience, things like call transcription, summarisation and sentiment analyis have been possible for a long time, however they’ve usually been out of reach for most due to how difficult it can be to connect these systems. Having access to automated artificial intelligence via a single endpoint changes everything.

AI’s unbelievable ability to reliably understand relatively complex prompts, and then return reliably formatted, structured data cuts down on an incredible amount of extra development.

In our phone system’s AI analysis example, previously we’d have to send phone call data to a dedicated speech-to-text transcription service like Azure AI Speech, get the returned result, attempt to identify who said what, and then send that to the Azure AI Language service for sentiment analysis and opinion mining. And we’d still likely need an AI model of some sort to generate the summary.

Now we just process the calls using a private audio-optimised ChatGPT model hosted in Azure AI Foundry. And instead of stringing three systems together, we send it a phone call and the following prompt:

“Please provide a transcript of this call. Try to identify the callers by name based on who is in the call and add this to the transcript. At the end of the transcript, provide a summary of the call and a sentiment analysis based on tone etc. Here’s a JSON representation of the call details, use this to determine who is in the call and correct any spelling mistakes etc. [JSON representation of the phone call metadata from our phone system.]”

(Not sure why I always say please.)

We also provide it with a system prompt describing how we want the returned object to be formatted to make sure we can put it into our SharePoint. The script outputs its results into a SharePoint document library with extra metadata for the summaries, transcriptions and sentiment analysis. Which lets us interact with it using the built-in AI capabilities and automation workflows available in SharePoint.

# Excerpt from our Azure Function. Long prompt strings are shortened with ... here.
function Get-GACCallAnalysis {
    param (
        [Parameter(Mandatory=$true)][PsCustomObject]$CallObject
    )

    $returnSchema = [pscustomobject]@{
        Title = "Call From [Caller] To [Other Participants] (try to include company names if possible)"
        Summary = "CallSummary"
        Sentiment = "Positive/Negative/Neutral"
        SentimentScore = "0-10"
        Transcript = "Long String for the transcript, separate each line in the transcript with a new line. ..."
    } | ConvertTo-Json -Depth 100

    $systeminstructions = "Provide me only a json object that matches the following schema ..."
    $Prompt = "Please provide a transcript of this call. Try to identify the callers by name based on who is in the call ..."
    $audioPath = "C:\temp\$($callobject.id).mp3"
    Invoke-RestMethod $callobject.recording -OutFile $audioPath
    $result = New-ChatGPTAudioQuery -Prompt $Prompt -SystemInstructions $systeminstructions -AudioFilePath $audioPath
    $result = $result | ConvertFrom-Json
    Remove-Item $audioPath -Force
    return $result
}

What do we use the AI Phone Call Analysis for?

It gives our team visibility into our client experience without waiting for ticket reviews or listening to hours of recordings. We can identify unhappy customers faster, identify areas of improvement, and get a better idea of where we need to focus our training.

It’s worth noting that services similar to these are available already on the market. Our phone system provider, AirCall, has a relatively expensive AI transcription and summarisation feature, as does Dialpad and a few others. These systems have their advantages, however for the cost and extensibility improvements, we’re pretty happy with our DIY effort.

The Bigger Picture for Small Business AI

This kind of automation used to feel out of reach for smaller teams, but being able to automate against an AI model that accepts plain english requests, carries out complex tasks feels like a bit of super power.

And the opportunities to apply AI in businesses of any kind are only increasing - eg internal chatbots familiar with your company, internal documentation summaries, even personalised service recommendations based on support history.

If you’ve got an idea you’d like to bring to life with AI for your business, get in touch with us today.