feat: per-transformation custom model selection (#978)
* feat: per-transformation custom model selection Fixes #776 * fix: validate model_id exists when creating or updating a transformation The create and update endpoints persisted model_id without checking the referenced model exists, so an invalid reference was stored silently and only surfaced later as a 404 at execution time. Add the same existence check execute_transformation already performs (Model.get -> 404 if missing) to both the create and update paths, so a bad model_id is rejected up front with a clear 404. Update still allows clearing model_id to None. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test: cover model_id existence validation on transformation create/update The create test now mocks Model.get so the new existence check passes and asserts it was awaited with the supplied model_id. The update test expects Model.get to be awaited twice (once validating the update, once at execute) rather than once. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: add CHANGELOG entry for per-transformation model selection (#776) --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Luis Novo <lfnovo@gmail.com>
|
|
@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
- Documented the `ESPERANTO_TTS_TIMEOUT` environment variable (default `300`s) in the environment reference; raise it for slow or self-hosted TTS providers so long podcast segments don't fail with a timeout (#937)
|
||||
- `SECURITY.md` with a coordinated-disclosure policy: how to privately report a vulnerability via GitHub's private vulnerability reporting, supported versions, and response expectations (#943)
|
||||
- LaTeX math rendering (KaTeX) now also applies to source content, source insights, Ask answers, transformation output, and the note editor preview — previously only chat had it (#269)
|
||||
- Per-transformation model selection — each transformation can now be assigned its own language model from the transformation editor, overriding the global transformation default for that transformation only. Runs without an explicit model keep using the system default as before (#776)
|
||||
- "Refresh content" action on web-link sources — re-fetches the URL and re-embeds the source so its content stays current, available from the source card menu once processing has completed (translated across all 14 locales) (#259)
|
||||
|
||||
### Changed
|
||||
|
|
|
|||
92
TRIAGE.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# Triage Rules — Open Notebook
|
||||
|
||||
Project-specific triage rules. The `/triage` loop provides the **mechanism**; this file is the
|
||||
**jurisprudence** for this repo. Keep it the single source of truth for how we label issues.
|
||||
|
||||
## State funnel (outcome of triage)
|
||||
|
||||
Every open issue lands in exactly **one** state. **Untriaged = no state label** — that's what
|
||||
`/triage` scans for.
|
||||
|
||||
| Label | Meaning |
|
||||
|---|---|
|
||||
| *(none)* | Not triaged yet. |
|
||||
| `needs-vision` | Unsure if/how this fits — strategic call for the maintainers. |
|
||||
| `needs-design` | Wanted, but the *how* isn't resolved — needs design/spec before it's ready. |
|
||||
| `needs-info` | Waiting on the reporter to confirm or provide more information. |
|
||||
| `ready` | Fully specified — the dev loop can pick it up. |
|
||||
| **Close** | Use GitHub's native close reasons (duplicate / not planned). Link the canonical issue when duplicate/superseded. |
|
||||
|
||||
## Type
|
||||
|
||||
What kind of work it is. Apply one when it's clear.
|
||||
|
||||
- `bug` · `enhancement` · `documentation`
|
||||
|
||||
> `installation` is an **intake** label applied by the issue-creation workflow (not by triage).
|
||||
> During triage, installation reports are routed to `area: deploy`.
|
||||
|
||||
## Area — apply ALWAYS (one per issue)
|
||||
|
||||
Which part of the system. This is the axis we use to route work and surface contributor-friendly
|
||||
tasks, so every triaged issue should get one area when it clearly belongs to one.
|
||||
|
||||
| Label | What goes here |
|
||||
|---|---|
|
||||
| `area: chat` | Conversation/chat, RAG retrieval, agentic responses, citations |
|
||||
| `area: search` | Full-text and semantic search |
|
||||
| `area: sources` | Source ingestion & processing (URLs, files, content extraction, chunking) |
|
||||
| `area: notebooks` | Notebook features: notes, insights, transformations |
|
||||
| `area: providers` | AI provider integrations + model configuration (OpenAI, Ollama, …) |
|
||||
| `area: embeddings` | Embedding models, vectorization, semantic indexing |
|
||||
| `area: podcast` | Podcast / audio generation |
|
||||
| `area: database` | SurrealDB, persistence, schema, migrations |
|
||||
| `area: ui` | Frontend (Next.js), UX, visual issues |
|
||||
| `area: deploy` | Docker, deployment, k8s, reverse proxy, infra/setup |
|
||||
| `area: offline` | Airgapped/offline operation (tokenizer caching, no network) |
|
||||
| `area: i18n` | Internationalization / localization |
|
||||
|
||||
## Bundling / epics
|
||||
|
||||
- `umbrella` — a tracking issue grouping related work.
|
||||
- `tracked-in-umbrella` — covered by an umbrella; follow the umbrella for progress.
|
||||
- `bundled` — part of a thematic bundle.
|
||||
- `upstream` — root cause lives in one of our libraries (below), not in this repo.
|
||||
|
||||
### Consolidation: one issue vs. umbrella
|
||||
|
||||
When several open issues circle the same topic, pick the model by **how decided the work is** —
|
||||
not just by shared theme. The trap is filing many half-formed issues for a topic that isn't yet
|
||||
decidable work; that inflates the backlog and reads as "90 things to do" when it's ~30 themes.
|
||||
|
||||
- **Pre-vision / pre-design topic** → collapse into **one** issue (`needs-vision` or `needs-design`),
|
||||
capture each request's signal (👍 counts, interested contributors) in its body, and **close the
|
||||
rest as duplicates** pointing to it. A topic isn't N issues — it's one thinking space.
|
||||
- **Already decomposed into real parallel tasks** → use `umbrella` + `tracked-in-umbrella`. Children
|
||||
stay open because each is independently pickable (e.g. the multi-user umbrella #712).
|
||||
|
||||
Rule of thumb: if the issues can't be worked until *we* make a call, they're one issue. If the call
|
||||
is made and the work splits into things a contributor could pick up today, they're an umbrella with
|
||||
children. **Never close an issue that has an active assignee/contributor or open PR** — link it as a
|
||||
phase instead.
|
||||
|
||||
## Ecosystem (our own libraries)
|
||||
|
||||
Mark issues whose real home is an upstream lib:
|
||||
|
||||
- `esperanto` — model abstraction layer (LLM / embedding / TTS / STT).
|
||||
- `content-core` — content extraction.
|
||||
- `podcast-creator` — podcast generation library.
|
||||
|
||||
## Community
|
||||
|
||||
- `good first issue` — small, well-scoped, newcomer-friendly.
|
||||
- `help wanted` — we'd welcome a contributor to take this.
|
||||
|
||||
## Rules
|
||||
|
||||
- Assign **one state**, **one type**, and **one area** where each applies. Multiple
|
||||
bundling/ecosystem labels are fine.
|
||||
- `/triage` output must always carry a state (or be closed). Don't leave issues stateless.
|
||||
- **Don't invent labels** — the set is curated. If something doesn't fit, raise it instead of adding a label.
|
||||
- When closing as duplicate/superseded, link the canonical issue.
|
||||
|
|
@ -112,6 +112,9 @@ class TransformationCreate(BaseModel):
|
|||
apply_default: bool = Field(
|
||||
False, description="Whether to apply this transformation by default"
|
||||
)
|
||||
model_id: Optional[str] = Field(
|
||||
None, description="Model ID to use by default for this transformation"
|
||||
)
|
||||
|
||||
|
||||
class TransformationUpdate(BaseModel):
|
||||
|
|
@ -126,6 +129,9 @@ class TransformationUpdate(BaseModel):
|
|||
apply_default: Optional[bool] = Field(
|
||||
None, description="Whether to apply this transformation by default"
|
||||
)
|
||||
model_id: Optional[str] = Field(
|
||||
None, description="Model ID to use by default for this transformation"
|
||||
)
|
||||
|
||||
|
||||
class TransformationResponse(BaseModel):
|
||||
|
|
@ -135,6 +141,7 @@ class TransformationResponse(BaseModel):
|
|||
description: str
|
||||
prompt: str
|
||||
apply_default: bool
|
||||
model_id: Optional[str] = None
|
||||
created: str
|
||||
updated: str
|
||||
|
||||
|
|
@ -146,7 +153,9 @@ class TransformationExecuteRequest(BaseModel):
|
|||
..., description="ID of the transformation to execute"
|
||||
)
|
||||
input_text: str = Field(..., description="Text to transform")
|
||||
model_id: str = Field(..., description="Model ID to use for the transformation")
|
||||
model_id: Optional[str] = Field(
|
||||
None, description="Model ID to use for this transformation run"
|
||||
)
|
||||
|
||||
|
||||
class TransformationExecuteResponse(BaseModel):
|
||||
|
|
@ -154,7 +163,7 @@ class TransformationExecuteResponse(BaseModel):
|
|||
|
||||
output: str = Field(..., description="Transformed text")
|
||||
transformation_id: str = Field(..., description="ID of the transformation used")
|
||||
model_id: str = Field(..., description="Model ID used")
|
||||
model_id: Optional[str] = Field(None, description="Model ID used")
|
||||
|
||||
|
||||
# Default Prompt API models
|
||||
|
|
|
|||
|
|
@ -20,6 +20,20 @@ from open_notebook.graphs.transformation import graph as transformation_graph
|
|||
router = APIRouter()
|
||||
|
||||
|
||||
def _transformation_response(transformation: Transformation) -> TransformationResponse:
|
||||
return TransformationResponse(
|
||||
id=transformation.id or "",
|
||||
name=transformation.name,
|
||||
title=transformation.title,
|
||||
description=transformation.description,
|
||||
prompt=transformation.prompt,
|
||||
apply_default=transformation.apply_default,
|
||||
model_id=transformation.model_id,
|
||||
created=str(transformation.created),
|
||||
updated=str(transformation.updated),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/transformations", response_model=List[TransformationResponse])
|
||||
async def get_transformations():
|
||||
"""Get all transformations."""
|
||||
|
|
@ -27,16 +41,7 @@ async def get_transformations():
|
|||
transformations = await Transformation.get_all(order_by="name asc")
|
||||
|
||||
return [
|
||||
TransformationResponse(
|
||||
id=transformation.id or "",
|
||||
name=transformation.name,
|
||||
title=transformation.title,
|
||||
description=transformation.description,
|
||||
prompt=transformation.prompt,
|
||||
apply_default=transformation.apply_default,
|
||||
created=str(transformation.created),
|
||||
updated=str(transformation.updated),
|
||||
)
|
||||
_transformation_response(transformation)
|
||||
for transformation in transformations
|
||||
]
|
||||
except Exception as e:
|
||||
|
|
@ -50,25 +55,26 @@ async def get_transformations():
|
|||
async def create_transformation(transformation_data: TransformationCreate):
|
||||
"""Create a new transformation."""
|
||||
try:
|
||||
# Reject unknown model references up front (same check as execute);
|
||||
# otherwise an invalid model_id is stored and only fails at run time.
|
||||
if transformation_data.model_id:
|
||||
model = await Model.get(transformation_data.model_id)
|
||||
if not model:
|
||||
raise HTTPException(status_code=404, detail="Model not found")
|
||||
|
||||
new_transformation = Transformation(
|
||||
name=transformation_data.name,
|
||||
title=transformation_data.title,
|
||||
description=transformation_data.description,
|
||||
prompt=transformation_data.prompt,
|
||||
apply_default=transformation_data.apply_default,
|
||||
model_id=transformation_data.model_id,
|
||||
)
|
||||
await new_transformation.save()
|
||||
|
||||
return TransformationResponse(
|
||||
id=new_transformation.id or "",
|
||||
name=new_transformation.name,
|
||||
title=new_transformation.title,
|
||||
description=new_transformation.description,
|
||||
prompt=new_transformation.prompt,
|
||||
apply_default=new_transformation.apply_default,
|
||||
created=str(new_transformation.created),
|
||||
updated=str(new_transformation.updated),
|
||||
)
|
||||
return _transformation_response(new_transformation)
|
||||
except HTTPException:
|
||||
raise
|
||||
except InvalidInputError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
except Exception as e:
|
||||
|
|
@ -87,10 +93,14 @@ async def execute_transformation(execute_request: TransformationExecuteRequest):
|
|||
if not transformation:
|
||||
raise HTTPException(status_code=404, detail="Transformation not found")
|
||||
|
||||
# Validate model exists
|
||||
model = await Model.get(execute_request.model_id)
|
||||
if not model:
|
||||
raise HTTPException(status_code=404, detail="Model not found")
|
||||
model_id = execute_request.model_id or transformation.model_id
|
||||
|
||||
# Validate explicit or transformation-specific model exists.
|
||||
# None is allowed so the graph can use the configured transformation default.
|
||||
if model_id:
|
||||
model = await Model.get(model_id)
|
||||
if not model:
|
||||
raise HTTPException(status_code=404, detail="Model not found")
|
||||
|
||||
# Execute the transformation
|
||||
result = await transformation_graph.ainvoke(
|
||||
|
|
@ -98,13 +108,13 @@ async def execute_transformation(execute_request: TransformationExecuteRequest):
|
|||
input_text=execute_request.input_text,
|
||||
transformation=transformation,
|
||||
),
|
||||
config=dict(configurable={"model_id": execute_request.model_id}),
|
||||
config=dict(configurable={"model_id": model_id}),
|
||||
)
|
||||
|
||||
return TransformationExecuteResponse(
|
||||
output=result["output"],
|
||||
transformation_id=execute_request.transformation_id,
|
||||
model_id=execute_request.model_id,
|
||||
model_id=model_id,
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
|
|
@ -166,16 +176,7 @@ async def get_transformation(transformation_id: str):
|
|||
if not transformation:
|
||||
raise HTTPException(status_code=404, detail="Transformation not found")
|
||||
|
||||
return TransformationResponse(
|
||||
id=transformation.id or "",
|
||||
name=transformation.name,
|
||||
title=transformation.title,
|
||||
description=transformation.description,
|
||||
prompt=transformation.prompt,
|
||||
apply_default=transformation.apply_default,
|
||||
created=str(transformation.created),
|
||||
updated=str(transformation.updated),
|
||||
)
|
||||
return _transformation_response(transformation)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
|
|
@ -208,19 +209,17 @@ async def update_transformation(
|
|||
transformation.prompt = transformation_update.prompt
|
||||
if transformation_update.apply_default is not None:
|
||||
transformation.apply_default = transformation_update.apply_default
|
||||
if "model_id" in transformation_update.model_fields_set:
|
||||
# Validate a newly supplied model reference (allow clearing to None).
|
||||
if transformation_update.model_id:
|
||||
model = await Model.get(transformation_update.model_id)
|
||||
if not model:
|
||||
raise HTTPException(status_code=404, detail="Model not found")
|
||||
transformation.model_id = transformation_update.model_id
|
||||
|
||||
await transformation.save()
|
||||
|
||||
return TransformationResponse(
|
||||
id=transformation.id or "",
|
||||
name=transformation.name,
|
||||
title=transformation.title,
|
||||
description=transformation.description,
|
||||
prompt=transformation.prompt,
|
||||
apply_default=transformation.apply_default,
|
||||
created=str(transformation.created),
|
||||
updated=str(transformation.updated),
|
||||
)
|
||||
return _transformation_response(transformation)
|
||||
except HTTPException:
|
||||
raise
|
||||
except InvalidInputError as e:
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ Transformations service layer using API.
|
|||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Union
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
from loguru import logger
|
||||
|
||||
|
|
@ -17,39 +17,14 @@ class TransformationsService:
|
|||
def __init__(self):
|
||||
logger.info("Using API for transformations operations")
|
||||
|
||||
def get_all_transformations(self) -> List[Transformation]:
|
||||
"""Get all transformations."""
|
||||
transformations_data = api_client.get_transformations()
|
||||
# Convert API response to Transformation objects
|
||||
transformations = []
|
||||
for trans_data in transformations_data:
|
||||
transformation = Transformation(
|
||||
name=trans_data["name"],
|
||||
title=trans_data["title"],
|
||||
description=trans_data["description"],
|
||||
prompt=trans_data["prompt"],
|
||||
apply_default=trans_data["apply_default"],
|
||||
)
|
||||
transformation.id = trans_data["id"]
|
||||
transformation.created = datetime.fromisoformat(
|
||||
trans_data["created"].replace("Z", "+00:00")
|
||||
)
|
||||
transformation.updated = datetime.fromisoformat(
|
||||
trans_data["updated"].replace("Z", "+00:00")
|
||||
)
|
||||
transformations.append(transformation)
|
||||
return transformations
|
||||
|
||||
def get_transformation(self, transformation_id: str) -> Transformation:
|
||||
"""Get a specific transformation."""
|
||||
response = api_client.get_transformation(transformation_id)
|
||||
trans_data = response if isinstance(response, dict) else response[0]
|
||||
def _build_transformation(self, trans_data: Dict[str, Any]) -> Transformation:
|
||||
transformation = Transformation(
|
||||
name=trans_data["name"],
|
||||
title=trans_data["title"],
|
||||
description=trans_data["description"],
|
||||
prompt=trans_data["prompt"],
|
||||
apply_default=trans_data["apply_default"],
|
||||
model_id=trans_data.get("model_id"),
|
||||
)
|
||||
transformation.id = trans_data["id"]
|
||||
transformation.created = datetime.fromisoformat(
|
||||
|
|
@ -60,6 +35,21 @@ class TransformationsService:
|
|||
)
|
||||
return transformation
|
||||
|
||||
def get_all_transformations(self) -> List[Transformation]:
|
||||
"""Get all transformations."""
|
||||
transformations_data = api_client.get_transformations()
|
||||
# Convert API response to Transformation objects
|
||||
transformations = []
|
||||
for trans_data in transformations_data:
|
||||
transformations.append(self._build_transformation(trans_data))
|
||||
return transformations
|
||||
|
||||
def get_transformation(self, transformation_id: str) -> Transformation:
|
||||
"""Get a specific transformation."""
|
||||
response = api_client.get_transformation(transformation_id)
|
||||
trans_data = response if isinstance(response, dict) else response[0]
|
||||
return self._build_transformation(trans_data)
|
||||
|
||||
def create_transformation(
|
||||
self,
|
||||
name: str,
|
||||
|
|
@ -77,21 +67,7 @@ class TransformationsService:
|
|||
apply_default=apply_default,
|
||||
)
|
||||
trans_data = response if isinstance(response, dict) else response[0]
|
||||
transformation = Transformation(
|
||||
name=trans_data["name"],
|
||||
title=trans_data["title"],
|
||||
description=trans_data["description"],
|
||||
prompt=trans_data["prompt"],
|
||||
apply_default=trans_data["apply_default"],
|
||||
)
|
||||
transformation.id = trans_data["id"]
|
||||
transformation.created = datetime.fromisoformat(
|
||||
trans_data["created"].replace("Z", "+00:00")
|
||||
)
|
||||
transformation.updated = datetime.fromisoformat(
|
||||
trans_data["updated"].replace("Z", "+00:00")
|
||||
)
|
||||
return transformation
|
||||
return self._build_transformation(trans_data)
|
||||
|
||||
def update_transformation(self, transformation: Transformation) -> Transformation:
|
||||
"""Update a transformation."""
|
||||
|
|
@ -104,6 +80,7 @@ class TransformationsService:
|
|||
"description": transformation.description,
|
||||
"prompt": transformation.prompt,
|
||||
"apply_default": transformation.apply_default,
|
||||
"model_id": transformation.model_id,
|
||||
}
|
||||
response = api_client.update_transformation(transformation.id, **updates)
|
||||
trans_data = response if isinstance(response, dict) else response[0]
|
||||
|
|
@ -114,6 +91,7 @@ class TransformationsService:
|
|||
transformation.description = trans_data["description"]
|
||||
transformation.prompt = trans_data["prompt"]
|
||||
transformation.apply_default = trans_data["apply_default"]
|
||||
transformation.model_id = trans_data.get("model_id")
|
||||
transformation.updated = datetime.fromisoformat(
|
||||
trans_data["updated"].replace("Z", "+00:00")
|
||||
)
|
||||
|
|
@ -126,7 +104,7 @@ class TransformationsService:
|
|||
return True
|
||||
|
||||
def execute_transformation(
|
||||
self, transformation_id: str, input_text: str, model_id: str
|
||||
self, transformation_id: str, input_text: str, model_id: Optional[str] = None
|
||||
) -> Union[Dict[Any, Any], List[Dict[Any, Any]]]:
|
||||
"""Execute a transformation on input text."""
|
||||
result = api_client.execute_transformation(
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
'use client'
|
||||
|
||||
import { useEffect, useId } from 'react'
|
||||
import { useEffect, useId, useMemo } from 'react'
|
||||
import { Controller, useForm } from 'react-hook-form'
|
||||
import { zodResolver } from '@hookform/resolvers/zod'
|
||||
import { z } from 'zod'
|
||||
|
|
@ -11,18 +11,33 @@ import { Textarea } from '@/components/ui/textarea'
|
|||
import { Checkbox } from '@/components/ui/checkbox'
|
||||
import { Label } from '@/components/ui/label'
|
||||
import { MarkdownEditor } from '@/components/ui/markdown-editor'
|
||||
import { useCreateTransformation, useUpdateTransformation, useTransformation } from '@/lib/hooks/use-transformations'
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from '@/components/ui/select'
|
||||
import {
|
||||
useCreateTransformation,
|
||||
useUpdateTransformation,
|
||||
useTransformation,
|
||||
} from '@/lib/hooks/use-transformations'
|
||||
import { useModels } from '@/lib/hooks/use-models'
|
||||
import { Transformation } from '@/lib/types/transformations'
|
||||
import { useQueryClient } from '@tanstack/react-query'
|
||||
import { TRANSFORMATION_QUERY_KEYS } from '@/lib/hooks/use-transformations'
|
||||
import { useTranslation } from '@/lib/hooks/use-translation'
|
||||
|
||||
const DEFAULT_MODEL_VALUE = '__default_transformation_model__'
|
||||
|
||||
const transformationSchema = z.object({
|
||||
name: z.string().min(1),
|
||||
title: z.string().min(1),
|
||||
description: z.string().optional(),
|
||||
prompt: z.string().min(1),
|
||||
apply_default: z.boolean().optional(),
|
||||
model_id: z.string().nullable().optional(),
|
||||
})
|
||||
|
||||
type TransformationFormData = z.infer<typeof transformationSchema>
|
||||
|
|
@ -40,10 +55,16 @@ export function TransformationEditorDialog({ open, onOpenChange, transformation
|
|||
const defaultId = useId()
|
||||
const descriptionId = useId()
|
||||
const promptId = useId()
|
||||
const modelId = useId()
|
||||
const isEditing = Boolean(transformation)
|
||||
const { data: fetchedTransformation, isLoading } = useTransformation(transformation?.id ?? '', {
|
||||
enabled: open && Boolean(transformation?.id),
|
||||
})
|
||||
const { data: models = [], isLoading: isLoadingModels } = useModels()
|
||||
const languageModels = useMemo(
|
||||
() => models.filter((model) => model.type === 'language'),
|
||||
[models]
|
||||
)
|
||||
const createTransformation = useCreateTransformation()
|
||||
const updateTransformation = useUpdateTransformation()
|
||||
const queryClient = useQueryClient()
|
||||
|
|
@ -61,12 +82,20 @@ export function TransformationEditorDialog({ open, onOpenChange, transformation
|
|||
description: '',
|
||||
prompt: '',
|
||||
apply_default: false,
|
||||
model_id: null,
|
||||
},
|
||||
})
|
||||
|
||||
useEffect(() => {
|
||||
if (!open) {
|
||||
reset({ name: '', title: '', description: '', prompt: '', apply_default: false })
|
||||
reset({
|
||||
name: '',
|
||||
title: '',
|
||||
description: '',
|
||||
prompt: '',
|
||||
apply_default: false,
|
||||
model_id: null,
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
|
|
@ -77,6 +106,7 @@ export function TransformationEditorDialog({ open, onOpenChange, transformation
|
|||
description: source?.description ?? '',
|
||||
prompt: source?.prompt ?? '',
|
||||
apply_default: source?.apply_default ?? false,
|
||||
model_id: source?.model_id ?? null,
|
||||
})
|
||||
}, [open, transformation, fetchedTransformation, reset])
|
||||
|
||||
|
|
@ -90,6 +120,7 @@ export function TransformationEditorDialog({ open, onOpenChange, transformation
|
|||
description: data.description || undefined,
|
||||
prompt: data.prompt,
|
||||
apply_default: Boolean(data.apply_default),
|
||||
model_id: data.model_id || null,
|
||||
},
|
||||
})
|
||||
queryClient.invalidateQueries({ queryKey: TRANSFORMATION_QUERY_KEYS.transformation(transformation.id) })
|
||||
|
|
@ -100,6 +131,7 @@ export function TransformationEditorDialog({ open, onOpenChange, transformation
|
|||
description: data.description || '',
|
||||
prompt: data.prompt,
|
||||
apply_default: Boolean(data.apply_default),
|
||||
model_id: data.model_id || null,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -170,24 +202,60 @@ export function TransformationEditorDialog({ open, onOpenChange, transformation
|
|||
)}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex items-center gap-2 pt-6 md:pt-8">
|
||||
<div>
|
||||
<Label htmlFor={modelId} className="text-sm font-medium">
|
||||
{t('transformations.model')}
|
||||
</Label>
|
||||
<Controller
|
||||
control={control}
|
||||
name="apply_default"
|
||||
name="model_id"
|
||||
render={({ field }) => (
|
||||
<Checkbox
|
||||
id={defaultId}
|
||||
checked={field.value}
|
||||
onCheckedChange={(checked) => field.onChange(Boolean(checked))}
|
||||
/>
|
||||
<Select
|
||||
name={field.name}
|
||||
value={field.value ?? DEFAULT_MODEL_VALUE}
|
||||
onValueChange={(value) =>
|
||||
field.onChange(
|
||||
value === DEFAULT_MODEL_VALUE ? null : value
|
||||
)
|
||||
}
|
||||
disabled={isLoadingModels}
|
||||
>
|
||||
<SelectTrigger id={modelId} className="w-full">
|
||||
<SelectValue placeholder={t('transformations.selectModel')} />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value={DEFAULT_MODEL_VALUE}>
|
||||
{t('transformations.systemDefault')}
|
||||
</SelectItem>
|
||||
{languageModels.map((model) => (
|
||||
<SelectItem key={model.id} value={model.id}>
|
||||
{model.name}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
)}
|
||||
/>
|
||||
<Label htmlFor={defaultId} className="text-sm">
|
||||
{t('transformations.suggestDefault')}
|
||||
</Label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center gap-2">
|
||||
<Controller
|
||||
control={control}
|
||||
name="apply_default"
|
||||
render={({ field }) => (
|
||||
<Checkbox
|
||||
id={defaultId}
|
||||
checked={field.value}
|
||||
onCheckedChange={(checked) => field.onChange(Boolean(checked))}
|
||||
/>
|
||||
)}
|
||||
/>
|
||||
<Label htmlFor={defaultId} className="text-sm">
|
||||
{t('transformations.suggestDefault')}
|
||||
</Label>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<Label htmlFor={descriptionId} className="text-sm font-medium">
|
||||
{t('notebooks.addDescription').replace('...', '')}
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ export interface Transformation {
|
|||
description: string
|
||||
prompt: string
|
||||
apply_default: boolean
|
||||
model_id: string | null
|
||||
created: string
|
||||
updated: string
|
||||
}
|
||||
|
|
@ -15,6 +16,7 @@ export interface CreateTransformationRequest {
|
|||
description: string
|
||||
prompt: string
|
||||
apply_default?: boolean
|
||||
model_id?: string | null
|
||||
}
|
||||
|
||||
export interface UpdateTransformationRequest {
|
||||
|
|
@ -23,20 +25,21 @@ export interface UpdateTransformationRequest {
|
|||
description?: string
|
||||
prompt?: string
|
||||
apply_default?: boolean
|
||||
model_id?: string | null
|
||||
}
|
||||
|
||||
export interface ExecuteTransformationRequest {
|
||||
transformation_id: string
|
||||
input_text: string
|
||||
model_id: string
|
||||
model_id?: string | null
|
||||
}
|
||||
|
||||
export interface ExecuteTransformationResponse {
|
||||
output: string
|
||||
transformation_id: string
|
||||
model_id: string
|
||||
model_id: string | null
|
||||
}
|
||||
|
||||
export interface DefaultPrompt {
|
||||
transformation_instructions: string
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -124,6 +124,9 @@ class AsyncMigrationManager:
|
|||
AsyncMigration.from_file(
|
||||
"open_notebook/database/migrations/16.surrealql"
|
||||
),
|
||||
AsyncMigration.from_file(
|
||||
"open_notebook/database/migrations/17.surrealql"
|
||||
),
|
||||
]
|
||||
self.down_migrations = [
|
||||
AsyncMigration.from_file(
|
||||
|
|
@ -174,6 +177,9 @@ class AsyncMigrationManager:
|
|||
AsyncMigration.from_file(
|
||||
"open_notebook/database/migrations/16_down.surrealql"
|
||||
),
|
||||
AsyncMigration.from_file(
|
||||
"open_notebook/database/migrations/17_down.surrealql"
|
||||
),
|
||||
]
|
||||
self.runner = AsyncMigrationRunner(
|
||||
up_migrations=self.up_migrations,
|
||||
|
|
|
|||
3
open_notebook/database/migrations/17.surrealql
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
-- Migration 17: Per-transformation model selection
|
||||
|
||||
DEFINE FIELD IF NOT EXISTS model_id ON TABLE transformation TYPE option<record<model>>;
|
||||
3
open_notebook/database/migrations/17_down.surrealql
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
-- Migration 17 rollback: Remove per-transformation model selection
|
||||
|
||||
REMOVE FIELD IF EXISTS model_id ON TABLE transformation;
|
||||
|
|
@ -1,17 +1,26 @@
|
|||
from typing import ClassVar, Optional
|
||||
from typing import Any, ClassVar, Dict, Optional
|
||||
|
||||
from pydantic import Field
|
||||
|
||||
from open_notebook.database.repository import ensure_record_id
|
||||
from open_notebook.domain.base import ObjectModel, RecordModel
|
||||
|
||||
|
||||
class Transformation(ObjectModel):
|
||||
table_name: ClassVar[str] = "transformation"
|
||||
nullable_fields: ClassVar[set[str]] = {"model_id"}
|
||||
name: str
|
||||
title: str
|
||||
description: str
|
||||
prompt: str
|
||||
apply_default: bool
|
||||
model_id: Optional[str] = None
|
||||
|
||||
def _prepare_save_data(self) -> Dict[str, Any]:
|
||||
data = super()._prepare_save_data()
|
||||
if data.get("model_id"):
|
||||
data["model_id"] = ensure_record_id(data["model_id"])
|
||||
return data
|
||||
|
||||
|
||||
class DefaultPrompts(RecordModel):
|
||||
|
|
|
|||
BIN
screenshot-01-initial-load.png
Normal file
|
After Width: | Height: | Size: 49 KiB |
BIN
screenshot-02-notebooks-list.png
Normal file
|
After Width: | Height: | Size: 74 KiB |
BIN
screenshot-03-notebook-detail.png
Normal file
|
After Width: | Height: | Size: 103 KiB |
BIN
screenshot-04-sources-context-menu.png
Normal file
|
After Width: | Height: | Size: 107 KiB |
BIN
screenshot-05-after-insights-only.png
Normal file
|
After Width: | Height: | Size: 103 KiB |
BIN
screenshot-06-source-card-hover.png
Normal file
|
After Width: | Height: | Size: 106 KiB |
BIN
screenshot-07-after-full-content.png
Normal file
|
After Width: | Height: | Size: 104 KiB |
BIN
screenshot-08-after-exclude-sources.png
Normal file
|
After Width: | Height: | Size: 105 KiB |
BIN
screenshot-09-notes-context-menu.png
Normal file
|
After Width: | Height: | Size: 107 KiB |
BIN
screenshot-10-after-exclude-notes.png
Normal file
|
After Width: | Height: | Size: 107 KiB |
BIN
screenshot-11-after-include-notes.png
Normal file
|
After Width: | Height: | Size: 104 KiB |
BIN
screenshot-12-post-delete-notebooks.png
Normal file
|
After Width: | Height: | Size: 70 KiB |
242
tests/test_transformations_api.py
Normal file
|
|
@ -0,0 +1,242 @@
|
|||
from datetime import datetime
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from open_notebook.domain.transformation import Transformation
|
||||
|
||||
|
||||
def _client() -> TestClient:
|
||||
from api.main import app
|
||||
|
||||
return TestClient(app)
|
||||
|
||||
|
||||
def _transformation(model_id: str | None = None) -> Transformation:
|
||||
return Transformation(
|
||||
id="transformation:123",
|
||||
name="summary",
|
||||
title="Summary",
|
||||
description="Summarize the source",
|
||||
prompt="Summarize this",
|
||||
apply_default=False,
|
||||
model_id=model_id,
|
||||
created=datetime(2026, 1, 1, 12, 0, 0),
|
||||
updated=datetime(2026, 1, 1, 12, 0, 0),
|
||||
)
|
||||
|
||||
|
||||
def _create_payload(model_id: str | None = None) -> dict:
|
||||
payload = {
|
||||
"name": "summary",
|
||||
"title": "Summary",
|
||||
"description": "Summarize the source",
|
||||
"prompt": "Summarize this",
|
||||
"apply_default": False,
|
||||
}
|
||||
if model_id is not None:
|
||||
payload["model_id"] = model_id
|
||||
return payload
|
||||
|
||||
|
||||
def test_create_transformation_with_model_id_persists_and_reads_back():
|
||||
client = _client()
|
||||
saved_transformations: list[Transformation] = []
|
||||
|
||||
async def capture_save(transformation: Transformation):
|
||||
saved_transformations.append(transformation)
|
||||
transformation.id = "transformation:created"
|
||||
transformation.created = datetime(2026, 1, 1, 12, 0, 0)
|
||||
transformation.updated = datetime(2026, 1, 1, 12, 0, 0)
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
Transformation, "save", autospec=True, side_effect=capture_save
|
||||
),
|
||||
patch(
|
||||
"api.routers.transformations.Model.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=object(),
|
||||
) as mock_model_get,
|
||||
):
|
||||
response = client.post(
|
||||
"/api/transformations", json=_create_payload("model:local")
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
assert saved_transformations[0].model_id == "model:local"
|
||||
assert response.json()["model_id"] == "model:local"
|
||||
mock_model_get.assert_awaited_once_with("model:local")
|
||||
|
||||
with patch(
|
||||
"api.routers.transformations.Transformation.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=saved_transformations[0],
|
||||
):
|
||||
response = client.get("/api/transformations/transformation:created")
|
||||
|
||||
assert response.status_code == 200
|
||||
assert response.json()["model_id"] == "model:local"
|
||||
|
||||
|
||||
def test_execute_without_request_model_uses_stored_model():
|
||||
client = _client()
|
||||
transformation = _transformation(model_id="model:stored")
|
||||
|
||||
with (
|
||||
patch(
|
||||
"api.routers.transformations.Transformation.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=transformation,
|
||||
),
|
||||
patch(
|
||||
"api.routers.transformations.Model.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=object(),
|
||||
) as mock_model_get,
|
||||
patch(
|
||||
"api.routers.transformations.transformation_graph.ainvoke",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"output": "Stored model output"},
|
||||
) as mock_ainvoke,
|
||||
):
|
||||
response = client.post(
|
||||
"/api/transformations/execute",
|
||||
json={
|
||||
"transformation_id": transformation.id,
|
||||
"input_text": "Input text",
|
||||
},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
assert response.json()["model_id"] == "model:stored"
|
||||
mock_model_get.assert_awaited_once_with("model:stored")
|
||||
assert (
|
||||
mock_ainvoke.call_args.kwargs["config"]["configurable"]["model_id"]
|
||||
== "model:stored"
|
||||
)
|
||||
|
||||
|
||||
def test_execute_without_any_model_uses_default_transformation_model_fallback():
|
||||
client = _client()
|
||||
transformation = _transformation(model_id=None)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"api.routers.transformations.Transformation.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=transformation,
|
||||
),
|
||||
patch(
|
||||
"api.routers.transformations.Model.get",
|
||||
new_callable=AsyncMock,
|
||||
) as mock_model_get,
|
||||
patch(
|
||||
"api.routers.transformations.transformation_graph.ainvoke",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"output": "Default model output"},
|
||||
) as mock_ainvoke,
|
||||
):
|
||||
response = client.post(
|
||||
"/api/transformations/execute",
|
||||
json={
|
||||
"transformation_id": transformation.id,
|
||||
"input_text": "Input text",
|
||||
},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
assert response.json()["model_id"] is None
|
||||
mock_model_get.assert_not_awaited()
|
||||
assert mock_ainvoke.call_args.kwargs["config"]["configurable"]["model_id"] is None
|
||||
|
||||
|
||||
def test_update_transformation_model_id_is_used_by_subsequent_execution():
|
||||
client = _client()
|
||||
transformation = _transformation(model_id="model:old")
|
||||
|
||||
async def save_update(transformation: Transformation):
|
||||
transformation.updated = datetime(2026, 1, 1, 12, 5, 0)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"api.routers.transformations.Transformation.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=transformation,
|
||||
),
|
||||
patch.object(
|
||||
Transformation, "save", autospec=True, side_effect=save_update
|
||||
),
|
||||
patch(
|
||||
"api.routers.transformations.Model.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=object(),
|
||||
) as mock_model_get,
|
||||
patch(
|
||||
"api.routers.transformations.transformation_graph.ainvoke",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"output": "Updated model output"},
|
||||
) as mock_ainvoke,
|
||||
):
|
||||
update_response = client.put(
|
||||
f"/api/transformations/{transformation.id}",
|
||||
json={"model_id": "model:new"},
|
||||
)
|
||||
execute_response = client.post(
|
||||
"/api/transformations/execute",
|
||||
json={
|
||||
"transformation_id": transformation.id,
|
||||
"input_text": "Input text",
|
||||
},
|
||||
)
|
||||
|
||||
assert update_response.status_code == 200
|
||||
assert update_response.json()["model_id"] == "model:new"
|
||||
assert execute_response.status_code == 200
|
||||
assert execute_response.json()["model_id"] == "model:new"
|
||||
# model:new is validated once at update time and looked up again at execute.
|
||||
assert mock_model_get.await_count == 2
|
||||
mock_model_get.assert_awaited_with("model:new")
|
||||
assert (
|
||||
mock_ainvoke.call_args.kwargs["config"]["configurable"]["model_id"]
|
||||
== "model:new"
|
||||
)
|
||||
|
||||
|
||||
def test_execute_request_model_overrides_stored_model():
|
||||
client = _client()
|
||||
transformation = _transformation(model_id="model:stored")
|
||||
|
||||
with (
|
||||
patch(
|
||||
"api.routers.transformations.Transformation.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=transformation,
|
||||
),
|
||||
patch(
|
||||
"api.routers.transformations.Model.get",
|
||||
new_callable=AsyncMock,
|
||||
return_value=object(),
|
||||
) as mock_model_get,
|
||||
patch(
|
||||
"api.routers.transformations.transformation_graph.ainvoke",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"output": "Override output"},
|
||||
) as mock_ainvoke,
|
||||
):
|
||||
response = client.post(
|
||||
"/api/transformations/execute",
|
||||
json={
|
||||
"transformation_id": transformation.id,
|
||||
"input_text": "Input text",
|
||||
"model_id": "model:override",
|
||||
},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
assert response.json()["model_id"] == "model:override"
|
||||
mock_model_get.assert_awaited_once_with("model:override")
|
||||
assert (
|
||||
mock_ainvoke.call_args.kwargs["config"]["configurable"]["model_id"]
|
||||
== "model:override"
|
||||
)
|
||||