dataspires

Overview

AfriLink is a Python SDK that gives you one-line access to A100 GPUs on distributed HPC backends. Authenticate with your DataSpires account, pick a model, pass your dataset, and the SDK handles everything: secure tunnelling, dataset upload, SLURM job submission, status polling, model download, and per-job billing. Install with pip install afrilink-sdk and start finetuning in minutes.

Quick Start

pip install afrilink-sdk

from afrilink import AfriLinkClient

client = AfriLinkClient()
client.authenticate()  # DataSpires + HPC credentials

job = client.finetune(
    model="qwen2.5-0.5b",
    training_mode="low",       # QLoRA, 4-bit, rank 8
    data=your_dataframe,
    gpus=1,
    time_limit="01:00:00",
    backend="cineca",       # HPC backend (default: "cineca")
)
result = job.run(wait=True)

# Download & run inference (only if job succeeded)
if result["status"] == "completed":
    client.download_model(result["job_id"], "./my-model")

    from transformers import AutoModelForCausalLM, AutoTokenizer
    from peft import PeftModel

    base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
    model = PeftModel.from_pretrained(base, "./my-model")
    tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B")

    out = model.generate(**tokenizer("Hello!", return_tensors="pt"), max_new_tokens=64)
    print(tokenizer.decode(out[0], skip_special_tokens=True))

Installation

Install from PyPI:

pip install afrilink-sdk

The package has zero required dependencies — heavy libraries like PyTorch, Transformers, and PEFT are only needed at the point you use them and are pre-installed in most notebook environments.

Authentication

AfriLink uses a two-phase auth flow, both handled inside a single client.authenticate() call:

You can also pass credentials explicitly: client.authenticate(dataspires_email="...", dataspires_password="...")

After authentication you get an SSH certificate valid for ~16 hours, a SLURM job manager, SCP transfer manager, and telemetry tracker — all ready to go.

HPC Backends

The SDK supports multiple HPC backends. Specify the backend parameter in client.finetune() to choose your cluster:

Available Models

The SDK ships with a curated registry of pre-configured models ready for finetuning. Use client.list_available_models() to browse, or filter by size with client.list_available_models(size="tiny").

Training Modes

Choose a training intensity that matches your dataset size and quality requirements:

Dataset Formats

The data parameter in client.finetune() accepts whichever format is most convenient:

Your DataFrame should have a text column with the full prompt + response formatted as a single string (Alpaca-style or chat template).

SDK Reference

AfriLinkClient

FinetuneJob (returned by client.finetune())

run() returns a dict with job_id, status ("completed" or "failed"), output_dir, and model_path. Always check result["status"] before downloading.

Pricing

AfriLink uses pay-as-you-go billing at $2.00 per GPU-hour. You are only charged for actual compute time used. Add credits via card payment on your dashboard, or redeem voucher codes for AfriLink-specific credits. Your balance and usage history are visible on the Billing Dashboard.