Pretty sure people don’t like the misleading title.

Yeah, it was definitely the title. It’s a neat project in of itself, but the title made it seem like you accomplished a frankly massive feat.

Videos like this one where they make absolutely tiny llm models for low power hardware are what I had in mind. Sadly I can’t find the other one that I watched several months back.

This is the model:

{
  "data" : [
    {
      "context_window" : 4096,
      "created" : 1719792000,
      "id" : "apple-foundationmodel",
      "notes" : "Apple on-device model via FoundationModels framework. Unsupported parameters are rejected with 400 when present (except n=1 and logprobs=false). Supported languages: it, zh, fr, en, nb, fr, da, es, es, es, vi, zh, de, pt, tr, en, zh, sv, nl, ja, en, ko, pt",
      "object" : "model",
      "owned_by" : "apple",
      "supported_parameters" : [
        "temperature",
        "max_tokens",
        "seed",
        "stream",
        "tools",
        "tool_choice",
        "response_format",
        "x_context_strategy",
        "x_context_max_turns",
        "x_context_output_reserve"
      ],
      "unsupported_parameters" : [
        "logprobs",
        "n",
        "stop",
        "presence_penalty",
        "frequency_penalty"
      ]
    }
  ],
  "object" : "list"
}

and I’m running it on a MacBook Air M4 16 GB. “Apfel” (the CLI tool) is kinda a wrapper around it and exposes an OpenAI compatible API.