OpenAI Chat Completions API: How to Make a Fine-Tuned GPT-3.5 Model Answer Only What You Want (2025 Guide)

Restricting a fine-tuned language model’s responses means configuring a custom GPT-3.5 model, deployed through OpenAI’s Chat Completions API, to only reply within a predefined topic domain and actively decline or redirect questions that fall outside that scope.

Why You’d Want to Restrict a Fine-Tuned GPT-3.5 Model

Building a focused AI assistant is one of the most practical applications of OpenAI’s fine-tuning system. Whether you’re deploying a customer support bot, a medical triage assistant, or a niche e-commerce helper, you almost certainly don’t want the model going off-script and answering questions about, say, celebrity gossip when it should only discuss your return policy.

The problem is that GPT-3.5, even when fine-tuned, retains knowledge from its base pre-training. That means without guardrails, it will happily answer almost anything. The solution requires a multi-layered strategy that covers prompt design, training data curation, and API parameter tuning.

Layer 1: The System Prompt — Your First Line of Defense

The system role message in the Chat Completions API is the single most important tool for scoping behavior. When calling the API, your request body should always include a firm, explicit system prompt. Here’s a practical example:

{
  "model": "ft:gpt-3.5-turbo:your-org:your-model-id",
  "messages": [
    {
      "role": "system",
      "content": "You are a customer support assistant for AcmeCorp. You ONLY answer questions related to AcmeCorp products, shipping, and returns. If a user asks about anything else, politely decline and redirect them to the relevant topic."
    },
    {
      "role": "user",
      "content": "What is the capital of France?"
    }
  ]
}

A well-written system prompt can deflect the majority of off-topic queries right out of the box — studies from enterprise AI teams suggest prompt-only restriction can handle roughly 70–80% of out-of-scope questions when worded precisely.

Layer 2: Fine-Tuning With Refusal Examples

The real power of fine-tuning for topic restriction lies in training the model with refusal pairs. These are JSONL training examples where the user asks an out-of-scope question and the assistant gives a polite, on-brand refusal.

Your fine-tuning dataset should include three types of examples:

• In-scope Q&A pairs: Questions the model should answer correctly and thoroughly.
• Refusal pairs: Out-of-scope questions paired with a consistent refusal message (e.g., “I’m only able to help with AcmeCorp-related questions. Can I assist you with something else?”).
• Edge-case pairs: Ambiguous questions that partially overlap with your domain, demonstrating the desired nuanced response.

OpenAI recommends a minimum of 50–100 high-quality training examples for meaningful behavior change, but for topic restriction, 200–500 examples — with at least 30% being refusal pairs — tends to produce noticeably more consistent results.

Layer 3: API Parameters That Tighten Control

Once your fine-tuned model is deployed, you can further constrain its output using Chat Completions API parameters:

Temperature

Set temperature to a low value (0.0–0.3) for deterministic, predictable responses. Higher temperatures introduce creativity — which is the last thing you want when you need consistent refusals.

Max Tokens

Use max_tokens to limit response length. A tightly scoped assistant rarely needs to generate 800-word essays. Capping at 150–300 tokens keeps responses focused and reduces the model’s opportunity to drift off-topic.

Stop Sequences

The stop parameter lets you define token sequences where the model must stop generating. While less commonly used for topic restriction, it can prevent runaway completions that veer into unwanted territory.

Layer 4: Post-Response Filtering (The Safety Net)

No prompt or fine-tuning strategy is 100% foolproof. For production systems, implement a lightweight post-response classifier — this can be as simple as a second, cheaper API call (or even a regex/keyword check) that flags or blocks responses containing off-topic content before they reach the user.

Some teams use OpenAI’s Moderation API as a complementary filter, though it’s primarily designed for harmful content rather than topic scope. A custom classifier trained on your specific domain boundaries is more effective.

Practical Workflow: Putting It All Together

1. Define your topic boundaries precisely — write them down as a policy document.
2. Craft a system prompt that enforces those boundaries in plain, direct language.
3. Build a fine-tuning dataset of 200+ examples, including 30–40% refusal pairs.
4. Fine-tune using OpenAI’s API (POST /v1/fine_tuning/jobs) and evaluate against a held-out test set.
5. Deploy with low temperature, capped max tokens, and a post-response filter.
6. Monitor production logs weekly and add new refusal examples for any observed edge cases.

This iterative loop — train, deploy, monitor, retrain — is what separates a hobby project from a reliable production assistant.

Common Mistakes to Avoid

Many developers rely exclusively on the system prompt and skip fine-tuning with refusals. While prompts help, a sufficiently creative user can often bypass them with prompt injection techniques. Fine-tuning the refusal behavior directly into the model weights is far more robust.

Another pitfall is inconsistent refusal phrasing in training data. If your refusal examples say different things (“I can’t help with that,” “That’s outside my expertise,” “Please contact a human agent”), the model learns an inconsistent policy. Pick one or two canonical refusal phrases and stick to them throughout your dataset.

Looking for more tips on ai & digital income? Visit SAVYX

Frequently Asked Questions

Can I restrict a fine-tuned GPT-3.5 model using only the system prompt?

A strong system prompt can handle 70–80% of off-topic queries, but it is not foolproof. For robust restriction, you should combine the system prompt with fine-tuning on refusal examples and a post-response filter for production use.

How many refusal examples do I need in my fine-tuning dataset?

Aim for at least 30–40% of your total training examples to be refusal pairs. If you have 200 total examples, that means 60–80 refusal pairs. More examples generally lead to more consistent out-of-scope behavior.

What temperature setting should I use for a topic-restricted assistant?

Use a low temperature between 0.0 and 0.3. Lower temperatures make the model more deterministic and predictable, which reduces the chance of it generating unexpected or off-topic content.

Will fine-tuning completely prevent the model from answering off-topic questions?

No fine-tuning strategy guarantees 100% restriction. Determined users can sometimes bypass restrictions through prompt injection or creative phrasing. A post-response classifier or moderation layer is recommended as an additional safety net.

Is the OpenAI Moderation API useful for topic restriction?

The OpenAI Moderation API is designed to detect harmful or policy-violating content, not to enforce custom topic boundaries. It is a useful complement for safety, but you will need a custom classifier or keyword filter to enforce your specific domain restrictions.