Am I building an Air Traffic Control simulator?

If you're just looking to try out the tool and don't care as much about the build process, you can do that here: https://chat.openai.com/g/g-OOhrPEbyy-air-traffic-controller

Am I building an Air Traffic Control simulator?...maybe.

I've owned the domain name airtraffictutor.com for years now and I think I finally know what to do with it.

OpenAI recently released Custom GPTs, which allow you to basically release your own version of ChatGPT with a custom prompt that you've written and let others (with a ChatGPT+ acccount) interact with it, without having to share the prompt. This also means you don't have to pay for the hosting or spin up your own AI-powered application.

As I was playing around with a couple ideas for how these might be useful, I stumbled upon the idea of that old domain name.

I'm a student pilot who's hopefully (🤞🏻) getting up in the air soon and even though I've watched hours of FlightChops and steveo1kinevo videos, I'm still a bit nervous about talking to Air Traffic Control (ATC).

However, one of the features of the ChatGPT app (and thus, any Custom GPTs that you develop) is that you can talk to them via voice, which makes it a perfect way to simulate a conversation with ATC, as long as we can train the GPT to respond with some amount of realism. And that's where I decided to start experimenting.

Creating the GPT

One of the cool features of the Custom GPT interface, is that you can talk to it just like ChatGPT, tell if what you want your custom GPT to do and it will do all the configuration for you under the hood. But because I've already built a couple of these and I much prefer writing the initial prompt myself before letting the engine fine tune it, I decided to use the "Configure" tab to edit the GPT directly.

I filled out the name and the description myself, but needed a cool icon to make it really stand out. Since I'm not a graphic designer, I called in DALLE for the job.

Initially, I used a simple prompt of "Generate an icon for an Air Traffic Controller AI Bot" just to see what it would come up with. I didn't love the result, so I asked it to "...make the face of the robot a bit less intense and more welcoming?" and then "...make it a bit more masculine and more like a cartoon instead of a photorealistic robot?", which resulted in a final version of the image that I liked.

The fact that you can do all of this right inside ChatGPT using GPT4 is super powerful and makes it was easier to tie all these different AI tools together. Now with the image in place, it was time to take a first pass at writing the prompt.

Writing the Prompt

When writing a prompt for a Custom GPT, you can use the provided chat interface or flip over to the Configure tab. I chose the latter here, as I already had a bit of an idea as to what I wanted the prompt to be. Here is the first draft of the prompt I settled on:

You are an Air Traffic Controller tasked with the job of helping a general aviation pilot taxi from the ramp at an airport of your choice (keep the same airport for the length of one simulation), take the runway (avoiding any other traffic) and successfully take off.

Make sure the user (pilot) specifies their airplane's callsign and repeats your radio calls correctly with every message. Remember their callsign and address them with it at the start of every message.

If the user deviates from the expected air traffic control conversation format, inform them that they've made an error and ask them to correct their transmission.

Only transmit one instruction at a time and wait for the user to respond.

The simulation ends when the user has successfully taken off or if the user violates one of your radio directions and collides with simulated traffic.

Once a pilot has successfully passed or failed the simulation, give them feedback on how they could have improved.

After testing this out, I realized there was one key piece missing: the simulator didn't provide the "pilot" with any weather information. I also wanted to make sure that the airport, weather and other variables stayed consistent throughout the entire conversation, so I added the following to the prompt:

When a pilot starts the simulation, choose an airport and give them a simulated ATIS weather readout. Make sure to use the same airport, active runway and weather data throughout the entire simulation.

Knowledge

In addition to the prompt, you can provide your Custom GPT with "knowledge" files, which it will reference and use when generating responses. For this, I actually took a bit of a shortcut and asked ChatGPT to generate simulated ATC conversations, verified them manually and made a couple corrections, and uploaded them as the knowledge documents.

The responses could be likely made a bit more natural if I transcribed some actual ATC conversations, but I was focused on getting something up and running quickly.

I also asked ChatGPT to generate a sample weather readout and uploaded that as a knowledge document. And with that, we were up and running and ready to start testing.

Testing it Out

Even with this relatively basic information, our ATC bot output was pretty passable. Interacting with it over text gave a pretty realistic back and forth and it was able to talk me through getting off the general aviation ramp, to the runway and airborne.

Importantly, if I made an error in the readback, it was able to ask for a correction and not allow me to proceed until I did so.

As you can see, some of the dialog is a bit stilted and doesn't exactly mirror what you'd expect, but I think this could be improved with more training data.

Testing it Out (with speech)

One of the great things about the fact that this runs on Open AI's infrastructure is that you can use the ChatGPT app to run it on your phone. This includes the ability to chat via voice with any of the GPTs you have access to. This really makes this bot a lot more realistic because voice (not text chat) is the method by which you'll be interacting with ATC anyway.

When you open the ChatGPT app (especially if you've used the ATC GPT before), you'll be able to switch over to it. Look for the headphone icon in the lower right hand corner to start a voice session.

Once you've enabled this, you'll be dropped into the voice interface. The ChatGPT app will detect when you stop talking and pass your transcribed voice to the ATC Bot before reading out the response. Once you get going, it can start to be pretty realistic. I've spent a couple hours walking around the neighborhood getting some weird glances from people because I'm pretending to talk to ATC through my AirPods.

The biggest thing that this mode revealed was the need to specify the format we wanted the speech to take. Specifically, it didn't know to use the phonetic alphabet and read out all the numerals that it needed to. It was was pronouncing things like 31L (three one el) and callsigns improperly.

Adding the following to the prompt fixed the issue:

When in voice chat mode, always read out numbers in their full version (for example, read out 8 as eight) and use the phonetic alphabet for all letters in a callsigns and runway and taxiway designations)

And with that, we have a pretty good first version! I would love for you to give it a try (sadly, at the moment it requires you to have a ChatGPT+ subscription) and let me know what you think !

https://chat.openai.com/g/g-OOhrPEbyy-air-traffic-controller

(Twitter/X DMs are open or shoot me a note at k.koppenhaver@gmail.com)

What's Next?

While building Custom GPTs like this make it really easy to get up and running with an MVP, there are definitely ways I want to expand this if people enjoy using it.

For one, I would love to get actual correct numbered runways for the airport you select and something approximating the current weather. I think to do this, I would have to spin up a proper SaaS and just use the Open AI API as the backend instead of relying completely on their infrastructure.

Also, to be useful as a teaching tool, it needs to give better feedback than it currently does. It can give broad strokes instructions about what you can improve, but it doesn't give any feedback about shortening words, using incorrect terms, etc.

Those are things that could be solved relatively easily with fine-tuning, providing the model with more real-world data and taking time to test more edge cases and give it feedback on how it behaved when the user doesn't perform perfectly.

I'm super excited about all this AI tooling, especially as to how it can be used to very quickly spin up new products and try new things.