Geek Out Time: Build a Facade API for OpenAI API and Local LLM API

In 2024, OpenAI leads the Generative AI sector, favored for its pioneering, easy-to-use API and the advanced GPT-4. However, factors like cost, data privacy, and the preference for open-source or self-hosted models can make developers look for alternatives. The diversity in API schemas among various LLMs complicates cross-platform support. It will be interesting to build a facade FastAPI server compatible with OpenAI’s API or others.

Step 1: Build a facade API in front of the OpenAI API.

I developed a simulated API that replicates the functionality of OpenAI’s Chat Completion API (/v1/chat/completions). This prototype is built in NodeJS using FastAPI. To validate our mock server, I have used the official NodeJS client library for OpenAI. The goal is to make the library — and by extension, any application leveraging it — believe it’s interacting with OpenAI’s actual server.

const fastify = require ( 'fastify' )({ logger : true }); const OpenAI = require ( 'openai' ); const openai = new OpenAI ({ apiKey : 'sk-xxxxxxxx' // The right approach is to use process.env['OPENAI_API_KEY'], // Here I use the openAI key directly for illustration. }); fastify. post ( '/v1/chat/completions' , async (request, reply) => { const { model, messages } = request. body ; // Basic input validation (expand as needed) if (!model || !messages || ! Array . isArray (messages)) { return reply. status ( 400 ). send ({ error : "Invalid request body" }); } try { const completion = await openai. chat . completions . create ({ model : model || "gpt-3.5-turbo" , // Default model messages : messages. map ( msg => ({ role : msg. role , content : msg. content })), }); // Return OpenAI's response return reply. send (completion. choices [ 0 ]. message . content ); } catch (error) { fastify. log . error (error); // Consider more specific error messages based on error type return reply. status ( 500 ). send ({ error : "Internal Server Error" }); } }); // Start the server const start = async ( ) => { try { await fastify. listen ({ port : 3000 }); fastify. log . info ( `Server listening on ${fastify.server.address().port} ` ); } catch (err) { fastify. log . error (err); process. exit ( 1 ); } } start (); After running the FastAPI server with the command below

This is an excerpt — the full article continues on Medium.