Metadata-Version: 2.4
Name: 3thfloor
Version: 0.1.1
Summary: Local AI inference SDK. pip install, load, chat.
Author-email: Justin Bench <justin.bench@gmail.com>
License: PolyForm Noncommercial License 1.0.0
        
        Required Notice: Copyright Justin Bench, 3th Floor AI (https://3thfloor.com)
        
        Acceptance
        
        In order to get any license under these terms, you must agree to them as
        both strict obligations and conditions to all your licenses.
        
        Copyright License
        
        The licensor grants you a copyright license for the software to do everything
        you might do with the software that would otherwise infringe the licensor's
        copyright in it for any permitted purpose. However, you may only distribute
        the software according to Distribution License and make changes or new works
        based on the software according to Changes and New Works License.
        
        Distribution License
        
        The licensor grants you an additional copyright license to distribute copies
        of the software. Your license to distribute covers distributing the software
        with changes and new works permitted by Changes and New Works License.
        
        Notices
        
        You must ensure that anyone who gets a copy of any part of the software from
        you also gets a copy of these terms or the URL for them above, as well as
        copies of any plain-text lines beginning with Required Notice: that the
        licensor provided with the software. For example:
        
        Required Notice: Copyright Justin Bench, 3th Floor AI (https://3thfloor.com)
        
        Changes and New Works License
        
        The licensor grants you an additional copyright license to make changes and
        new works based on the software for any permitted purpose.
        
        Patent License
        
        The licensor grants you a patent license for the software that covers patent
        claims the licensor can license, or becomes able to license, that you would
        infringe by using the software.
        
        Noncommercial Purposes
        
        Any noncommercial purpose is a permitted purpose.
        
        Personal Uses
        
        Personal use for research, experiment, and testing for the benefit of public
        knowledge, personal study, private entertainment, hobby projects, amateur
        pursuits, or religious observance, without any anticipated commercial
        application, is use for a permitted purpose.
        
        Noncommercial Organizations
        
        Use by any charitable organization, educational institution, public research
        organization, public safety or health organization, environmental protection
        organization, or government institution is use for a permitted purpose
        regardless of the source of funding or obligations resulting from the funding.
        
        Fair Use
        
        You may have "fair use" rights for the software under the law. These terms
        do not limit them.
        
        No Other Rights
        
        These terms do not allow you to sublicense or transfer any of your licenses
        to anyone else, or prevent the licensor from granting licenses to anyone
        else. These terms do not imply any other licenses.
        
        Patent Defense
        
        If you make any written claim that the software infringes or contributes to
        infringement of any patent, your patent license for the software granted
        under these terms ends immediately. If your company makes such a claim, your
        patent license ends immediately for work on behalf of your company.
        
        Violations
        
        The first time you are notified in writing that you have violated any of
        these terms, or done anything with the software not covered by your licenses,
        your licenses can nonetheless continue if you come into full compliance with
        these terms, and take practical steps to correct past violations, within 32
        days of receiving notice. Otherwise, all your licenses end immediately.
        
        No Liability
        
        As far as the law allows, the software comes as is, without any warranty or
        condition, and the licensor will not be liable to you for any damages arising
        out of these terms or the use or nature of the software, under any kind of
        legal claim.
        
        Definitions
        
        The licensor is the individual or entity offering these terms, and the
        software is the software the licensor makes available under these terms.
        
        You refers to the individual or entity agreeing to these terms.
        
        Your company is any legal entity, sole proprietorship, or other kind of
        organization that you work for, plus all organizations that have control
        over, are under the control of, or are under common control with that
        organization. Control means ownership of substantially all the assets of an
        entity, or the power to direct its management and policies by vote, contract,
        or otherwise. Control can be direct or indirect.
        
        Your licenses are all the licenses granted to you for the software under
        these terms.
        
        Use means anything you do with the software requiring one of your licenses.
        
        Commercial Use
        
        For commercial licensing, contact justin@3thfloor.com.
        
Project-URL: Homepage, https://3thfloor.com
Project-URL: Repository, https://github.com/3thfloor/python-engine
Keywords: llm,ai,inference,local,gguf,llama
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: llama-cpp-python>=0.3.0
Provides-Extra: hf
Requires-Dist: huggingface-hub>=0.23.0; extra == "hf"
Provides-Extra: server
Requires-Dist: fastapi>=0.110.0; extra == "server"
Requires-Dist: uvicorn[standard]>=0.27.0; extra == "server"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Dynamic: license-file

# thirthfloor

Local AI inference for Python. One install. No daemon. No API keys.

Your model runs as an object inside your process. Ask a question, get a string back.

```python
from thirthfloor import Engine

engine = Engine()
engine.load("qwen", "/models/qwen3-4b-q4.gguf")
print(engine.chat("qwen", "What is boundary value analysis?"))
```

## Install

```bash
pip install 3thfloor
```

CPU works out of the box. For GPU acceleration, install the matching wheel for your hardware:

**Apple Silicon (Metal)**

```bash
pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal
pip install 3thfloor
```

**NVIDIA (CUDA 12.x)**

```bash
pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124
pip install 3thfloor
```

Same package, same API. The backend picks up your GPU automatically.

## Quick Start

```python
from thirthfloor import Engine

engine = Engine()
engine.load("tester", "/models/3thfloor-tester-q4.gguf")

answer = engine.chat("tester", "Write three test cases for a login form.")
print(answer)
```

`chat()` returns a plain string. Not a dict, not `.choices[0].message.content`. A string.

## Sessions (Conversation History)

```python
session = engine.session("tester", system="You are a senior QA engineer.")

print(session.send("What is exploratory testing?"))
print(session.send("How is that different from what I asked about?"))
```

The session tracks history for you. The second question knows about the first. No message arrays to build, no history to splice.

## Streaming

```python
for token in engine.stream("tester", "Explain risk-based testing in two paragraphs."):
    print(token, end="", flush=True)
```

`stream()` yields token strings as they generate. Print them, pipe them, collect them.

## Multiple Models

```python
engine.load("fast", "/models/qwen3-4b-q4.gguf")
engine.load("smart", "/models/qwen3-32b-q4.gguf")

def ask(question: str) -> str:
    alias = "smart" if len(question) > 200 else "fast"
    return engine.chat(alias, question)
```

Load as many models as your RAM allows. Route between them with the alias.

## Agents

```python
from thirthfloor import tool, run_agent

@tool
def get_build_status(pipeline: str) -> str:
    """Return the latest CI status for a pipeline."""
    return "pipeline main: passing, 312 tests green"

result = run_agent(
    engine, "tester",
    "Is the main pipeline healthy?",
    tools=[get_build_status],
)
print(result)
```

Decorate a function with `@tool`, pass it to `run_agent()`. The model decides when to call it, the engine runs it, you get the final answer as a string. Docstrings and type hints become the tool schema.

## Model Management

```python
engine.models.add("tester", "/models/3thfloor-tester-q4.gguf")

for m in engine.models.list():
    print(m["alias"], m["path"], round(m["size_mb"] / 1024, 1), "GB")

engine.models.download("Qwen/Qwen3-4B-GGUF", filename="qwen3-4b-q4_k_m.gguf")
```

Registered models let you look up the path by alias: `engine.load("tester", engine.models.info("tester")["path"])`.

Downloading from HuggingFace requires the extra:

```bash
pip install "thirthfloor[hf]"
```

## Optional HTTP Server

```python
engine.serve(port=7437)
```

OpenAI-compatible endpoints (`/v1/chat/completions`) for when other tools need HTTP access. Never required. Requires:

```bash
pip install "thirthfloor[server]"
```

## Embed in Your Software

```python
from thirthfloor import Engine

class SupportBot:
    def __init__(self, model_path: str):
        self.engine = Engine()
        self.engine.load("support", model_path)
        self.session = self.engine.session(
            "support",
            system="You answer questions about our test automation product.",
        )

    def reply(self, message: str) -> str:
        return self.session.send(message)

bot = SupportBot("/models/3thfloor-tester-q4.gguf")
print(bot.reply("How do I tag a flaky test?"))
```

The model is an object in your process. No ports to manage, no subprocesses to babysit, no service that has to be running before your app starts. When your process exits, the model is gone. Ship it inside a CLI, a desktop app, a batch job, anywhere Python runs.

---

Built by Justin Bench, [3th Floor AI](https://3thfloor.com).

Free for personal projects, research, experiments, and noncommercial use under the [PolyForm Noncommercial License 1.0.0](./LICENSE). Commercial use requires a license: justin@3thfloor.com.
