Files
MemoryBear/api/app/repositories/ontology_scene_repository.py
Ke Sun 79ab929fb0 Release/v0.2.3 (#355)
* feat(web): add PageEmpty component

* feat(web): add PageTabs component

* feat(web): add PageEmpty component

* feat(web): add PageTabs component

* feat(prompt): add history tracking for prompt releases

* feat(web): add prompt menu

* refactor: The PageScrollList component supports two generic parameters

* feat(web): BodyWrapper compoent update PageLoading

* feat(web): add Ontology menu

* feat(web): memory management add scene

* feat(tasks): add celery task configuration for periodic jobs

- Add ignore_result=True to prevent storing results for periodic tasks
- Set max_retries=0 to skip failed periodic tasks without retry attempts
- Configure acks_late=False for immediate acknowledgment in beat tasks
- Add time_limit and soft_time_limit to regenerate_memory_cache task (3600s/3300s)
- Add time_limit and soft_time_limit to workspace_reflection_task (300s/240s)
- Add time_limit and soft_time_limit to run_forgetting_cycle_task (7200s/7000s)
- Improve task reliability and resource management for scheduled jobs

* feat(sandbox): add Node.js code execution support to sandbox

* Release/v0.2.2 (#260)

* [modify] migration script

* [add] migration script

* fix(web): change form message

* fix(web): the memoryContent field is compatible with numbers and strings

* feat(web): code node hidden

* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.

---------

Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>

* Feature/ontology class clean (#249)

* [add] Complete ontology engineering feature implementation

* [add] Add ontology feature integration and validation utilities

* [add] Add OWL validator and validation utilities

* [fix] Add missing render_ontology_extraction_prompt function

* [fix]Add dependencies, fix functionality

* [add] migration script

* feat(celery): add dedicated periodic tasks worker and queue (#261)

* fix(web): conflict resolve

* Fix/v022 bug (#263)

* [fix]Fix the issue of inconsistent language in explicit and episodic memory.

* [fix]Fix the issue of inconsistent language in explicit and episodic memory.

* [add]Add scene_id

* [fix]Based on the AI review to fix the code

* Fix/develop memory reflex (#265)

* 遗漏的历史映射

* 遗漏的历史映射

* 反思后台报错处理

* [add] migration script

* fix: chat conversation_id add node_start

* feat(web): show code node

* fix(web): Restructure the CustomSelect component, repair the interface that is called multiple times when the form is updated

* feat(web): RadioGroupCard support block mode

* feat(web): create space add icon

* feat(app and model): token consumption statistics

* Add/develop memory (#264)

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 新增长期记忆功能

* 新增长期记忆功能

* 新增长期记忆功能

* 知识库检索多余字段

* 长期

* feat(app and model): token consumption statistics of the cluster

* memory_BUG_fix

* fix(web): prompt history remove pageLoading

* fix(prompt): remove hard-coded import of prompt file paths (#279)

* Fix/develop memory bug (#274)

* 遗漏的历史映射

* 遗漏的历史映射

* fix_timeline_memories

* fix(web): update retrieve_type key

* Fix/develop memory bug (#276)

* 遗漏的历史映射

* 遗漏的历史映射

* fix_timeline_memories

* fix_timeline_memories

* write_gragp/bug_fix

* write_gragp/bug_fix

* write_gragp/bug_fix

* chore(celery): disable periodic task scheduling

* fix(prompt): remove hard-coded import of prompt file paths

---------

Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Ke Sun <kesun5@illinois.edu>

* fix(web): remove delete confirm content

* refactor(workflow): relocate template directory into workflow

* feat(memory): add long-term storage task routing and batching

* fix(web): PageScrollList loading update

* fix(web): PageScrollList loading update

* Ontology v1 bug (#291)

* [changes]Add 'id' as the secondary sorting key, and 'scene_id' now returns a UUID object

* [fix]Fix the "end_user" return to be sorted by update time.

* [fix]Set the default values of the memory configuration model based on the spatial model.

* [fix]Remove the entity extraction check combination model, read the configuration list, and add the return of scene_id

* [fix]Fix the "end_user" return to be sorted by update time.

* [fix]

* fix(memory): add Redis session validation

- Add macOS fork() safety configuration in celery_app.py to prevent initialization issues
- Add null/False checks for Redis session queries in term_memory_save to handle missing sessions gracefully
- Add null/False checks in memory_long_term_storage to prevent processing empty Redis results
- Add null/False checks in aggregate_judgment before format_parsing to avoid errors on missing data
- Initialize redis_messages variable in window_dialogue for consistency
- Add debug logging when no existing session found in Redis for better troubleshooting
- Add TODO comments for magic numbers (scope=6, time=5) to be extracted as constants
- Improve error handling when Redis returns False or empty results instead of crashing

* fix(web): PageScrollList style update

* fix(workflow): fix argument passing in code execution nodes

* fix(web): prompt add disabled

* fix(web): space icon required

* feat(app): modify the key of the token

* fix(fix the key of the app's token):

* fix(workflow): switch code input encoding to base64+URL encoding

* [add]The main project adds multi-API Key load balancing.

* [changes]Attribute security access, secure numerical conversion, unified use of local variables

* fix(web): save add session update

* fix(web): language editor support paste

* [changes]Active status filtering logic, API Key selection strategy

* memory_BUG

* memory_BUG_long_term

* [changes]

* memory_BUG_long_term

* memory_BUG_long_term

* Fix/release memory bug (#306)

* memory_BUG_fix

* memory_BUG

* memory_BUG_long_term

* memory_BUG_long_term

* memory_BUG_long_term

* knowledge_retrieval/bug/fix

* knowledge_retrieval/bug/fix

* knowledge_retrieval/bug/fix

* [fix]1.The "read_all_config" interface returns "scene_name";2.Memory configuration for lightweight query ontology scenarios

* fix(web): replace code editor

* [changes]Modify the description of the time for the recent event

* [changes]Modify the code based on the AI review

* feat(web): update memory config ontology api

* fix(web): ui update

* knowledge_retrieval/bug/fix

* knowledge_retrieval/bug/fix

* knowledge_retrieval/bug/fix

* feat(workflow): add token usage statistics for question classifier and parameter extraction

* feat(web): move prompt menu

* Multiple independent transactions - single transaction

* Multiple independent transactions - single transaction

* Multiple independent transactions - single transaction

* Multiple independent transactions - single transaction

* Write Missing None (#321)

* Write Missing None

* Write Missing None

* Write Missing None

* Apply suggestion from @sourcery-ai[bot]

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Write Missing None

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix/release memory bug (#324)

* Write Missing None

* Write Missing None

* Write Missing None

* Apply suggestion from @sourcery-ai[bot]

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Write Missing None

* redis update

* redis update

* redis update

* redis update

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix/writer memory bug (#326)

* [fix]Fix the bug

* [fix]Fix the bug

* [fix]Correct the direction indication.

* fix(web): markdown table ui update

* Fix/release memory bug (#332)

* Write Missing None

* Write Missing None

* Write Missing None

* Apply suggestion from @sourcery-ai[bot]

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Write Missing None

* redis update

* redis update

* redis update

* redis update

* writer_dup_bug/fix

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix/fact summary (#333)

* [fix]Disable the contents related to fact_summary

* [fix]Disable the contents related to fact_summary

* [fix]Modify the code based on the AI review

* Fix/release memory bug (#335)

* Write Missing None

* Write Missing None

* Write Missing None

* Apply suggestion from @sourcery-ai[bot]

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Write Missing None

* redis update

* redis update

* redis update

* redis update

* writer_dup_bug/fix

* writer_graph_bug/fix

* writer_graph_bug/fix

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Revert "feat(web): move prompt menu"

This reverts commit 9e6e8f50f8.

* fix(web): ui update

* fix(web): update text

* fix(web): ui update

* fix(model): change the "vl" model type of dashscope to "chat"

* fix(model): change the "vl" model type of dashscope to "chat"

---------

Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: Eternity <61316157+myhMARS@users.noreply.github.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
2026-02-06 19:01:57 +08:00

440 lines
14 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# -*- coding: utf-8 -*-
"""本体场景Repository层
本模块提供本体场景的数据访问层实现。
Classes:
OntologySceneRepository: 本体场景数据访问类
"""
import logging
from typing import List, Optional
from uuid import UUID
from sqlalchemy.orm import Session, joinedload
from app.core.logging_config import get_db_logger
from app.models.ontology_scene import OntologyScene
logger = get_db_logger()
class OntologySceneRepository:
"""本体场景Repository
提供本体场景的CRUD操作和权限检查。
Attributes:
db: SQLAlchemy数据库会话
"""
def __init__(self, db: Session):
"""初始化Repository
Args:
db: SQLAlchemy数据库会话
"""
self.db = db
def create(self, scene_data: dict, workspace_id: UUID) -> OntologyScene:
"""创建本体场景
Args:
scene_data: 场景数据字典包含scene_name和scene_description
workspace_id: 所属工作空间ID
Returns:
OntologyScene: 创建的场景对象
Raises:
Exception: 数据库操作失败
Examples:
>>> repo = OntologySceneRepository(db)
>>> scene = repo.create(
... {"scene_name": "医疗场景", "scene_description": "描述"},
... workspace_id
... )
"""
try:
logger.info(
f"Creating ontology scene - "
f"name={scene_data.get('scene_name')}, "
f"workspace_id={workspace_id}"
)
scene = OntologyScene(
scene_name=scene_data.get("scene_name"),
scene_description=scene_data.get("scene_description"),
workspace_id=workspace_id
)
self.db.add(scene)
self.db.flush() # 获取ID但不提交
logger.info(
f"Ontology scene created successfully - "
f"scene_id={scene.scene_id}"
)
return scene
except Exception as e:
logger.error(
f"Failed to create ontology scene: {str(e)}",
exc_info=True
)
raise
def get_by_id(self, scene_id: UUID) -> Optional[OntologyScene]:
"""根据ID获取场景
Args:
scene_id: 场景ID
Returns:
Optional[OntologyScene]: 场景对象不存在则返回None
Examples:
>>> repo = OntologySceneRepository(db)
>>> scene = repo.get_by_id(scene_id)
"""
try:
logger.debug(f"Getting ontology scene by ID: {scene_id}")
scene = self.db.query(OntologyScene).filter(
OntologyScene.scene_id == scene_id
).first()
if scene:
logger.debug(f"Ontology scene found: {scene_id}")
else:
logger.debug(f"Ontology scene not found: {scene_id}")
return scene
except Exception as e:
logger.error(
f"Failed to get ontology scene by ID: {str(e)}",
exc_info=True
)
raise
def get_by_name(self, scene_name: str, workspace_id: UUID) -> Optional[OntologyScene]:
"""根据场景名称和工作空间ID获取场景精确匹配
Args:
scene_name: 场景名称
workspace_id: 工作空间ID
Returns:
Optional[OntologyScene]: 场景对象不存在则返回None
Examples:
>>> repo = OntologySceneRepository(db)
>>> scene = repo.get_by_name("医疗场景", workspace_id)
"""
try:
logger.debug(
f"Getting ontology scene by name - "
f"scene_name={scene_name}, workspace_id={workspace_id}"
)
scene = self.db.query(OntologyScene).options(
joinedload(OntologyScene.classes)
).filter(
OntologyScene.scene_name == scene_name,
OntologyScene.workspace_id == workspace_id
).first()
if scene:
logger.debug(f"Ontology scene found: {scene_name}")
else:
logger.debug(f"Ontology scene not found: {scene_name}")
return scene
except Exception as e:
logger.error(
f"Failed to get ontology scene by name: {str(e)}",
exc_info=True
)
raise
def search_by_name(self, keyword: str, workspace_id: UUID) -> List[OntologyScene]:
"""根据关键词模糊搜索场景
使用 LIKE 进行模糊匹配,支持中文和英文。
Args:
keyword: 搜索关键词
workspace_id: 工作空间ID
Returns:
List[OntologyScene]: 匹配的场景列表
Examples:
>>> repo = OntologySceneRepository(db)
>>> scenes = repo.search_by_name("医疗", workspace_id)
"""
try:
logger.debug(
f"Searching ontology scenes by keyword - "
f"keyword={keyword}, workspace_id={workspace_id}"
)
# 使用 ilike 进行不区分大小写的模糊匹配
scenes = self.db.query(OntologyScene).options(
joinedload(OntologyScene.classes)
).filter(
OntologyScene.scene_name.ilike(f"%{keyword}%"),
OntologyScene.workspace_id == workspace_id
).order_by(
OntologyScene.updated_at.desc()
).all()
logger.info(
f"Found {len(scenes)} ontology scenes matching keyword '{keyword}' "
f"in workspace {workspace_id}"
)
return scenes
except Exception as e:
logger.error(
f"Failed to search ontology scenes by keyword: {str(e)}",
exc_info=True
)
raise
def get_by_workspace(self, workspace_id: UUID, page: Optional[int] = None, page_size: Optional[int] = None) -> tuple:
"""获取工作空间下的所有场景(支持分页)
使用joinedload预加载classes关系以统计数量。
Args:
workspace_id: 工作空间ID
page: 页码可选从1开始
page_size: 每页数量(可选)
Returns:
tuple: (场景列表, 总数量)
Examples:
>>> repo = OntologySceneRepository(db)
>>> scenes, total = repo.get_by_workspace(workspace_id)
>>> scenes, total = repo.get_by_workspace(workspace_id, page=1, page_size=10)
"""
try:
logger.debug(f"Getting ontology scenes by workspace: {workspace_id}, page={page}, page_size={page_size}")
# 构建基础查询
query = self.db.query(OntologyScene).options(
joinedload(OntologyScene.classes)
).filter(
OntologyScene.workspace_id == workspace_id
).order_by(
OntologyScene.updated_at.desc()
)
# 获取总数
total = query.count()
# 如果提供了分页参数,应用分页
if page is not None and page_size is not None:
offset = (page - 1) * page_size
query = query.offset(offset).limit(page_size)
logger.debug(f"Applying pagination: offset={offset}, limit={page_size}")
scenes = query.all()
logger.info(
f"Found {len(scenes)} ontology scenes (total: {total}) in workspace {workspace_id}"
)
return scenes, total
except Exception as e:
logger.error(
f"Failed to get ontology scenes by workspace: {str(e)}",
exc_info=True
)
raise
def update(self, scene_id: UUID, update_data: dict) -> Optional[OntologyScene]:
"""更新场景信息
Args:
scene_id: 场景ID
update_data: 更新数据字典
Returns:
Optional[OntologyScene]: 更新后的场景对象不存在则返回None
Raises:
Exception: 数据库操作失败
Examples:
>>> repo = OntologySceneRepository(db)
>>> scene = repo.update(
... scene_id,
... {"scene_name": "新名称"}
... )
"""
try:
logger.info(f"Updating ontology scene: {scene_id}")
scene = self.get_by_id(scene_id)
if not scene:
logger.warning(f"Ontology scene not found for update: {scene_id}")
return None
# 更新字段
if "scene_name" in update_data and update_data["scene_name"] is not None:
scene.scene_name = update_data["scene_name"]
if "scene_description" in update_data:
scene.scene_description = update_data["scene_description"]
self.db.flush()
logger.info(f"Ontology scene updated successfully: {scene_id}")
return scene
except Exception as e:
logger.error(
f"Failed to update ontology scene: {str(e)}",
exc_info=True
)
raise
def delete(self, scene_id: UUID) -> bool:
"""删除场景(级联删除类型)
依赖数据库级联删除配置ondelete="CASCADE")。
Args:
scene_id: 场景ID
Returns:
bool: 删除成功返回True场景不存在返回False
Raises:
Exception: 数据库操作失败
Examples:
>>> repo = OntologySceneRepository(db)
>>> success = repo.delete(scene_id)
"""
try:
logger.info(f"Deleting ontology scene: {scene_id}")
scene = self.get_by_id(scene_id)
if not scene:
logger.warning(f"Ontology scene not found for delete: {scene_id}")
return False
self.db.delete(scene)
self.db.flush()
logger.info(
f"Ontology scene deleted successfully (cascade): {scene_id}"
)
return True
except Exception as e:
logger.error(
f"Failed to delete ontology scene: {str(e)}",
exc_info=True
)
raise
def check_ownership(self, scene_id: UUID, workspace_id: UUID) -> bool:
"""检查场景是否属于指定工作空间
Args:
scene_id: 场景ID
workspace_id: 工作空间ID
Returns:
bool: 属于返回True否则返回False
Examples:
>>> repo = OntologySceneRepository(db)
>>> is_owner = repo.check_ownership(scene_id, workspace_id)
"""
try:
logger.debug(
f"Checking scene ownership - "
f"scene_id={scene_id}, workspace_id={workspace_id}"
)
count = self.db.query(OntologyScene).filter(
OntologyScene.scene_id == scene_id,
OntologyScene.workspace_id == workspace_id
).count()
is_owner = count > 0
logger.debug(
f"Scene ownership check result: {is_owner} - "
f"scene_id={scene_id}"
)
return is_owner
except Exception as e:
logger.error(
f"Failed to check scene ownership: {str(e)}",
exc_info=True
)
raise
def get_simple_list(self, workspace_id: UUID) -> List[dict]:
"""获取场景简单列表仅包含scene_id和scene_name用于下拉选择
这是一个轻量级查询不加载关联的classes响应速度快。
Args:
workspace_id: 工作空间ID
Returns:
List[dict]: 场景简单列表每项包含scene_id和scene_name
Examples:
>>> repo = OntologySceneRepository(db)
>>> scenes = repo.get_simple_list(workspace_id)
>>> # [{"scene_id": "xxx", "scene_name": "场景1"}, ...]
"""
try:
logger.debug(f"Getting simple scene list for workspace: {workspace_id}")
# 只查询需要的字段,不加载关联数据
results = self.db.query(
OntologyScene.scene_id,
OntologyScene.scene_name
).filter(
OntologyScene.workspace_id == workspace_id
).order_by(
OntologyScene.updated_at.desc()
).all()
scenes = [
{"scene_id": str(r.scene_id), "scene_name": r.scene_name}
for r in results
]
logger.info(f"Found {len(scenes)} scenes (simple list) in workspace {workspace_id}")
return scenes
except Exception as e:
logger.error(
f"Failed to get simple scene list: {str(e)}",
exc_info=True
)
raise