Laravel AI Spark

A spark driver for the Laravel AI SDK that talks to a self-hosted, OpenAI-compatible vLLM server, such as an NVIDIA DGX Spark. Agents use it like any other provider. Requests go to vLLM's /v1/chat/completions endpoint and can carry extra headers, for example a Cloudflare Access service token when the server sits behind Cloudflare Zero Trust.

Features

• Laravel AI provider – text, streaming, structured output and embeddings through the regular agent API
• Custom headers per connection – e.g. CF-Access-Client-Id / CF-Access-Client-Secret; empty values are dropped
• Optional Bearer key – for vLLM started with --api-key or a key-checking proxy in front of it
• Default request options – chat_template_kwargs and other vLLM extras, overridable per agent
• vLLM-safe payloads – omits tools / tool_choice when an agent has no tools and never sends OpenRouter's structured_outputs flag; vLLM rejects both
• Document attachments – vLLM accepts no file attachments, so text files are inlined and PDFs are sent as their text layer, as page images, or both
• Mixed prompts – images, text files and PDFs in one prompt keep their order, each page image labelled with file and page number
• Server utilities – health, version, served models, token counting, context-window checks and batched prompts via the Spark facade, plus php artisan spark:status
• Clear errors – Cloudflare Access login redirects and 401/403 responses fail with a readable message instead of a JSON parse error
• Multiple servers – every ai.providers.* entry with 'driver' => 'spark' uses this driver

Tech Stack

Requirements

• PHP 8.3 or newer
• Laravel 12 or 13 with laravel/ai installed
• A reachable vLLM server exposing /v1/chat/completions
• For tool calls: vLLM started with --enable-auto-tool-choice and a --tool-call-parser matching the model
• Optional, for PDF text: smalot/pdfparser 2.12+
• Optional, for PDF page images: poppler's pdftoppm (brew install poppler, apt install poppler-utils), or the Imagick extension with Ghostscript, and a vision-capable model
• For the server utilities behind a proxy: /health, /version, /tokenize and /detokenize must be forwarded next to /v1/*

Installation

composer require mindtwo/laravel-ai-spark

The service provider is auto-discovered and registers a spark connection from the package config, so setting the environment variables is enough:

SPARK_URL="https://llm.example.com/v1"
SPARK_MODEL="nvidia/Qwen3.8-27B-NVFP4"
SPARK_CF_ACCESS_CLIENT_ID=
SPARK_CF_ACCESS_CLIENT_SECRET=
SPARK_API_KEY=
SPARK_ENABLE_THINKING=false

To change headers or default options beyond the variables, publish the config:

php artisan vendor:publish --tag="ai-spark-config"

If config/ai.php already defines ai.providers.spark, the package keeps that definition and does not register its own.

Usage

Select the provider per agent with the #[Provider] attribute, or per call:

use Laravel\Ai\Attributes\Provider;
use Laravel\Ai\Attributes\Timeout;
use Laravel\Ai\Contracts\Agent;
use Laravel\Ai\Promptable;

use function Laravel\Ai\agent;

#[Provider('spark')]
#[Timeout(180)]
class InternalSummaryAgent implements Agent
{
    use Promptable;

    public function instructions(): string
    {
        return 'Summarise the given text in three sentences.';
    }
}

$summary = (new InternalSummaryAgent)->prompt($text)->text;

// Ad hoc, with failover to a hosted provider when the server is busy
agent(instructions: 'Be brief.')->prompt('Hello', provider: ['spark', 'anthropic']);

The SDK fails over only on rate limiting (429) and overload, which this driver maps from 502, 503 and 504. A 401/403 (for example a request from outside the allowed network) and connection errors are thrown as they are and do not fall back.

Agent provider options are merged over the configured defaults, so a single agent can turn on reasoning or set sampling parameters vLLM understands:

use Laravel\Ai\Contracts\HasProviderOptions;
use Laravel\Ai\Enums\Lab;

class DeepAnalysisAgent implements Agent, HasProviderOptions
{
    use Promptable;

    public function providerOptions(Lab|string $provider): array
    {
        return $provider === 'spark'
            ? ['chat_template_kwargs' => ['enable_thinking' => true], 'top_k' => 20]
            : [];
    }
}

A second server is an additional entry in config/ai.php:

'providers' => [
    'spark-lab' => [
        'driver' => 'spark',
        'url' => env('SPARK_LAB_URL'),
        'key' => env('SPARK_LAB_API_KEY', ''),
        'headers' => [],
        'models' => ['text' => ['default' => env('SPARK_LAB_MODEL')]],
    ],
],

Attachments

vLLM has no file content part on any of its APIs, so a document sent as a file is rejected with a 400. The driver converts documents on the application server before the request leaves:

Attachments of all kinds can be combined in one prompt. They reach the model in the order they were given, and every page image is preceded by a label such as [report.pdf, page 2 of 5].

PDF modes

text is blind to scans, charts and table layout, and a PDF without a text layer fails. Page images cost image tokens per page and one of the server's parallel slots for longer, which is why images and both stop at max_pages instead of truncating silently.

composer require smalot/pdfparser
brew install poppler

The mode applies per connection (SPARK_PDF_MODE) and can be overridden per agent with an attachments provider option, which the driver removes before the request:

public function providerOptions(Lab|string $provider): array
{
    return $provider === 'spark'
        ? ['attachments' => ['pdf' => 'both', 'max_pages' => 40]]
        : [];
}

When an attachment cannot be sent

Every case that cannot work throws mindtwo\LaravelAiSpark\Exceptions\UnsupportedAttachmentException before any HTTP request, with the reason and the fix in the message: parser or renderer missing, no text layer in text mode, encrypted or damaged PDF, too many pages, unsupported format. A PDF with embedded images in text mode is sent, but a warning is logged that the images were dropped.

try {
    $response = (new InternalSummaryAgent)->prompt('Summarise the report.', [Document::fromStorage('reports/q3.pdf')]);
} catch (UnsupportedAttachmentException $e) {
    report($e); // nothing was sent to the server
}

The exception extends the SDK's AiException and is deliberately not failoverable, so provider: ['spark', 'anthropic'] never sends a document to a hosted provider behind your back.

Beyond page images

Page images cover most documents. For very long documents or heavy table extraction, convert the PDF to Markdown first with Docling (docling-serve runs next to vLLM, also on ARM64) and attach the result as Document::fromString($markdown, 'text/markdown'). OCRmyPDF adds a text layer to scans so that text mode works. A hosted provider with native PDF support, e.g. Anthropic, needs no infrastructure, but the document leaves your network.

Server utilities

The Spark facade talks to the same server with the same URL, key and headers. Use Spark::connection('spark-lab') for another connection.

use mindtwo\LaravelAiSpark\Facades\Spark;

Spark::health();                    // bool, never throws
Spark::version();                   // "0.23.1"
Spark::models();                    // [['id' => 'nvidia/Qwen3.8-27B-NVFP4', 'max_model_len' => 262144]]
Spark::maxModelLength();            // 262144

Spark::countTokens($text);          // tokens of a plain string
Spark::countTokens($messages);      // tokens of a chat, including the chat template
Spark::fits($text, reserve: 4000);  // room left for a 4,000 token answer?
Spark::tokenize($text)->remaining();
Spark::detokenize([9419, 1814]);    // "Hello world"

// Many short, independent prompts in one request, answers keyed like the input
Spark::batch(
    ['a' => 'I love this product.', 'b' => 'Worst purchase ever.'],
    instructions: 'Answer with one word: positive or negative.',
    options: ['max_tokens' => 5],
); // ['a' => 'positive', 'b' => 'negative']

batch() uses vLLM's /v1/chat/completions/batch and has no tools, structured output or streaming; use agents for those. Failures throw SparkServerException, with a hint when a proxy blocks the non-/v1 endpoints.

php artisan spark:status
php artisan spark:status spark-lab

spark:status checks reachability, version, served models and whether the configured model is among them, and which PDF tools are installed. It exits non-zero when the server is down or the model is not served, so it also works as a deploy or CI check.

Timeouts

Self-hosted servers usually have few parallel slots and are slower than hosted APIs. The SDK's default request timeout is 60 seconds; set #[Timeout] on agents that run against Spark.

Architecture

The handlers reuse Prism's OpenRouter implementation because it speaks plain Chat Completions and forwards provider options into the request body. Prism's OpenAI provider is not used since it targets the Responses API. Both laravel/ai and Prism are pre-1.0, so their internals can change between minor versions; the constraints in composer.json are kept narrow on purpose.

Development

Testing

composer test
composer test -- --filter=structured

The tests fake all HTTP traffic with Http::fake(); no server is needed.

Code quality

composer format     # Pint
composer analyse    # Larastan

Releasing

1. Add the changes to CHANGELOG.md under a new version heading.
2. Tag the release: git tag vX.Y.Z && git push --tags.
3. Before widening the laravel/ai or prism-php/prism constraints, run the tests against the new versions.

Conventions

• Semantic Versioning; breaking changes only in major releases
• Code style is enforced by Pint, static analysis by Larastan level 8
• New behaviour needs a Pest test that asserts the request payload sent to vLLM

License

MIT, see LICENSE.

Support

Maintained by the development team at mindtwo GmbH.