Azure OpenAI and its challenges
Azure OpenAI Service is a fully managed service that allows developers to easily integrate OpenAI models into their own applications. Azure OpenAI is simple to use, it offers a pre-trained model and it allows customization where you can enhance the model with your own private data. Most importantly, compared to OpenAI, Azure offers the highest level of data privacy protection. Azure assures that your training data is not shared with other customers, OpenAI or being used to improve OpenAI, Microsoft or any other products and services. In other words, your training data, your output and your model are yours to keep.
By default, Azure OpenAI API endpoint is public, that is, the API endpoint URL is associated with a public IP address. While this works functionally, it poses security risks as your OpenAI instance is exposed to the Internet, and as such it is subject to vulnerability exploits. It is also vulnerable to authentication-based attacks, Dos and DDos attacks that can slow or halt API services. Therefore it is a best practice to use Azure OpenAI private link to access the API privately and shut down Internet access routes.
The challenge is that developers and enterprises have been using AWS as their product development and launch site. There is no native way to connect to Azure OpenAI for private access.
Bumblebee Global Private Link for Azure OpenAI private access
Bumblebee Networks offers Global Private Link, a service that extends the capability of private link by cloud providers to multi-region, multi-cloud, on-prem and remote workers.
Below is the diagram how Bumblebee connects to Azure OpenAI privately.
As shown in the diagram above, accessing Azure OpenAI privately is supported from AWS VPCs, on-prem offices or remote developers.
When connecting VPC to Azure OpenAI, you use native AWS endpoint to connect to Bumblebee Networks and through which connect to Azure OpenAI, allowing private from AWS VPC to Azure OpenAI. The user experience is as if you are using AWS private link to connect to Azure OpenAI.
When connecting on-prem to Azure OpenAI, you deploy one or more on-prem Bumblebee endpoints and through the same mechanism to securely and privately access Azure OpenAI
When connecting remote developers to Azure OpenAI, you download Bumblebee agent and through which to access Azure OpenAI.
Bumblebee Global Private Link benefits
1. Fast
No waiting for months to get your dedicated private circuit provisioned. Secure private access to your application and data in minutes. No gateway instances to deploy and all cloud native construct on your end. DevOps and infrastructure engineers can build and manage the network.
2. Low Latency
If you have workloads in AWS VPCs that need access Azure OpenAI, connecting directly from AWS VPC to Azure OpenAI provides the shortest latency comparing to alternative of routing traffic to on-prem, through data center hops and then back to Azure.
3. Secure
The original default accessing route of publicly accessing Azure OpenAI is completely shut off. Even the Azure OpenAI private IP address is obfuscated through various IP address transformations and tunneling. This makes it impossible to attack your Azure OpenAI deployment.
While Bumblebee Networks is a SaaS solution, the data path is single tenant, i.e., other customer's data does not mingle with yours. In addition to mutual TLS authentication between the consumer end and the provider end, a request and approval process is required for additional authentication.
Application segmentation and isolation is built in and enforced automatically in the solution. In this case, only the traffic destined for the Azure OpenAI private endpoint IP address and on the specific port 443 is allowed to go through the encrypted tunnel between the consumer and provider.
4. Cloud scale & high performance
Unlike a traditional VPN gateway based solution that limits performance throughput to 1.5Gbps, Bumblebee Networks is built from the ground to scale out to handle any kind of network bandwidth requirement.
Start with one instantiation of Bumblebee endpoint or app service which supports 500mbps network throughput. To increase the throughput you simply resize the bandwidth to add another 500mbps to your app service or endpoint. You can do so without limitation. Similarly you can reduce the bandwidth by resizing down the deployment. The flexibility of scale as you go eliminates the need for planning and upfront wasteful cost.
5. Multi region failover
Traditional secure network connectivity for site-to-site is carried out by IPSec VPN protocol. IPSec protocol is symmetrical which does not lend itself to leverage modern load balancers to build out redundancy and scaling. The best redundancy system an IPSec VPN can create is an active and stand by deployment where you employ twice the resources but can only use one.
Bumblebee Networks uses the client server based TLS protocol as deployed in all Internet applications. The protocol is not only much simpler, but also allows a cluster of instantiation of endpoints and app services in all active modes to simultaneously handle traffic, creating orders of magnitude higher redundancy.
Bumblebee Networks improves your service availability by supporting multi-hosting, a feature that allows you to build policy based redundancy for applications or services registered on the platform. You specify an operation priority among multiple deployments of a service and should one site fail, Bumblebee Networks re-route traffic to the next site in the order you define.
For example, you launch an Azure OpenAI service in the West US 3 and launch another one in the East US as a backup service. Bumblebee Networks monitors the health of Azure OpenAI in all regions and normally routes traffic to the West US 3. When it detects failure, it re-routes traffic to East US. (Note service consumers should be designed with this capability in mind in order to take advantage of it.)
6. Day 2 Operation
Bumblebee Networks provides automated security vulnerability patching and software upgrade without downtime. This improves system availability time and reduces on-going maintenance work on the ops team.
You can run tests on end-to-end bandwidth consumption and round trip delay to quickly identify issues and troubleshoot problems.
Summary
Shutting off Internet access to your OpenAI instance is the best practice to protect your data, as it eliminates the attack surface of your OpenAI instance. Using Bumblebee Global Private Link allows you to achieve that no matter where the source traffic is from, AWS, on-prem or remote developers.
To learn how to setup Global Private Link to access Azure OpenAI privately, check out How to create an App Service for Azure OpenAI