Files
MemoryBear/api/app/repositories/prompt_optimizer_repository.py
Ke Sun 79ab929fb0 Release/v0.2.3 (#355)
* feat(web): add PageEmpty component

* feat(web): add PageTabs component

* feat(web): add PageEmpty component

* feat(web): add PageTabs component

* feat(prompt): add history tracking for prompt releases

* feat(web): add prompt menu

* refactor: The PageScrollList component supports two generic parameters

* feat(web): BodyWrapper compoent update PageLoading

* feat(web): add Ontology menu

* feat(web): memory management add scene

* feat(tasks): add celery task configuration for periodic jobs

- Add ignore_result=True to prevent storing results for periodic tasks
- Set max_retries=0 to skip failed periodic tasks without retry attempts
- Configure acks_late=False for immediate acknowledgment in beat tasks
- Add time_limit and soft_time_limit to regenerate_memory_cache task (3600s/3300s)
- Add time_limit and soft_time_limit to workspace_reflection_task (300s/240s)
- Add time_limit and soft_time_limit to run_forgetting_cycle_task (7200s/7000s)
- Improve task reliability and resource management for scheduled jobs

* feat(sandbox): add Node.js code execution support to sandbox

* Release/v0.2.2 (#260)

* [modify] migration script

* [add] migration script

* fix(web): change form message

* fix(web): the memoryContent field is compatible with numbers and strings

* feat(web): code node hidden

* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.

---------

Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>

* Feature/ontology class clean (#249)

* [add] Complete ontology engineering feature implementation

* [add] Add ontology feature integration and validation utilities

* [add] Add OWL validator and validation utilities

* [fix] Add missing render_ontology_extraction_prompt function

* [fix]Add dependencies, fix functionality

* [add] migration script

* feat(celery): add dedicated periodic tasks worker and queue (#261)

* fix(web): conflict resolve

* Fix/v022 bug (#263)

* [fix]Fix the issue of inconsistent language in explicit and episodic memory.

* [fix]Fix the issue of inconsistent language in explicit and episodic memory.

* [add]Add scene_id

* [fix]Based on the AI review to fix the code

* Fix/develop memory reflex (#265)

* 遗漏的历史映射

* 遗漏的历史映射

* 反思后台报错处理

* [add] migration script

* fix: chat conversation_id add node_start

* feat(web): show code node

* fix(web): Restructure the CustomSelect component, repair the interface that is called multiple times when the form is updated

* feat(web): RadioGroupCard support block mode

* feat(web): create space add icon

* feat(app and model): token consumption statistics

* Add/develop memory (#264)

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 新增长期记忆功能

* 新增长期记忆功能

* 新增长期记忆功能

* 知识库检索多余字段

* 长期

* feat(app and model): token consumption statistics of the cluster

* memory_BUG_fix

* fix(web): prompt history remove pageLoading

* fix(prompt): remove hard-coded import of prompt file paths (#279)

* Fix/develop memory bug (#274)

* 遗漏的历史映射

* 遗漏的历史映射

* fix_timeline_memories

* fix(web): update retrieve_type key

* Fix/develop memory bug (#276)

* 遗漏的历史映射

* 遗漏的历史映射

* fix_timeline_memories

* fix_timeline_memories

* write_gragp/bug_fix

* write_gragp/bug_fix

* write_gragp/bug_fix

* chore(celery): disable periodic task scheduling

* fix(prompt): remove hard-coded import of prompt file paths

---------

Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Ke Sun <kesun5@illinois.edu>

* fix(web): remove delete confirm content

* refactor(workflow): relocate template directory into workflow

* feat(memory): add long-term storage task routing and batching

* fix(web): PageScrollList loading update

* fix(web): PageScrollList loading update

* Ontology v1 bug (#291)

* [changes]Add 'id' as the secondary sorting key, and 'scene_id' now returns a UUID object

* [fix]Fix the "end_user" return to be sorted by update time.

* [fix]Set the default values of the memory configuration model based on the spatial model.

* [fix]Remove the entity extraction check combination model, read the configuration list, and add the return of scene_id

* [fix]Fix the "end_user" return to be sorted by update time.

* [fix]

* fix(memory): add Redis session validation

- Add macOS fork() safety configuration in celery_app.py to prevent initialization issues
- Add null/False checks for Redis session queries in term_memory_save to handle missing sessions gracefully
- Add null/False checks in memory_long_term_storage to prevent processing empty Redis results
- Add null/False checks in aggregate_judgment before format_parsing to avoid errors on missing data
- Initialize redis_messages variable in window_dialogue for consistency
- Add debug logging when no existing session found in Redis for better troubleshooting
- Add TODO comments for magic numbers (scope=6, time=5) to be extracted as constants
- Improve error handling when Redis returns False or empty results instead of crashing

* fix(web): PageScrollList style update

* fix(workflow): fix argument passing in code execution nodes

* fix(web): prompt add disabled

* fix(web): space icon required

* feat(app): modify the key of the token

* fix(fix the key of the app's token):

* fix(workflow): switch code input encoding to base64+URL encoding

* [add]The main project adds multi-API Key load balancing.

* [changes]Attribute security access, secure numerical conversion, unified use of local variables

* fix(web): save add session update

* fix(web): language editor support paste

* [changes]Active status filtering logic, API Key selection strategy

* memory_BUG

* memory_BUG_long_term

* [changes]

* memory_BUG_long_term

* memory_BUG_long_term

* Fix/release memory bug (#306)

* memory_BUG_fix

* memory_BUG

* memory_BUG_long_term

* memory_BUG_long_term

* memory_BUG_long_term

* knowledge_retrieval/bug/fix

* knowledge_retrieval/bug/fix

* knowledge_retrieval/bug/fix

* [fix]1.The "read_all_config" interface returns "scene_name";2.Memory configuration for lightweight query ontology scenarios

* fix(web): replace code editor

* [changes]Modify the description of the time for the recent event

* [changes]Modify the code based on the AI review

* feat(web): update memory config ontology api

* fix(web): ui update

* knowledge_retrieval/bug/fix

* knowledge_retrieval/bug/fix

* knowledge_retrieval/bug/fix

* feat(workflow): add token usage statistics for question classifier and parameter extraction

* feat(web): move prompt menu

* Multiple independent transactions - single transaction

* Multiple independent transactions - single transaction

* Multiple independent transactions - single transaction

* Multiple independent transactions - single transaction

* Write Missing None (#321)

* Write Missing None

* Write Missing None

* Write Missing None

* Apply suggestion from @sourcery-ai[bot]

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Write Missing None

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix/release memory bug (#324)

* Write Missing None

* Write Missing None

* Write Missing None

* Apply suggestion from @sourcery-ai[bot]

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Write Missing None

* redis update

* redis update

* redis update

* redis update

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix/writer memory bug (#326)

* [fix]Fix the bug

* [fix]Fix the bug

* [fix]Correct the direction indication.

* fix(web): markdown table ui update

* Fix/release memory bug (#332)

* Write Missing None

* Write Missing None

* Write Missing None

* Apply suggestion from @sourcery-ai[bot]

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Write Missing None

* redis update

* redis update

* redis update

* redis update

* writer_dup_bug/fix

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Fix/fact summary (#333)

* [fix]Disable the contents related to fact_summary

* [fix]Disable the contents related to fact_summary

* [fix]Modify the code based on the AI review

* Fix/release memory bug (#335)

* Write Missing None

* Write Missing None

* Write Missing None

* Apply suggestion from @sourcery-ai[bot]

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Write Missing None

* redis update

* redis update

* redis update

* redis update

* writer_dup_bug/fix

* writer_graph_bug/fix

* writer_graph_bug/fix

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Revert "feat(web): move prompt menu"

This reverts commit 9e6e8f50f8.

* fix(web): ui update

* fix(web): update text

* fix(web): ui update

* fix(model): change the "vl" model type of dashscope to "chat"

* fix(model): change the "vl" model type of dashscope to "chat"

---------

Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: Eternity <61316157+myhMARS@users.noreply.github.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
2026-02-06 19:01:57 +08:00

322 lines
11 KiB
Python

import uuid
from sqlalchemy.orm import Session
from app.core.logging_config import get_db_logger
from app.models.prompt_optimizer_model import (
PromptOptimizerSession,
PromptOptimizerSessionHistory,
RoleType,
PromptHistory
)
db_logger = get_db_logger()
class PromptOptimizerSessionRepository:
"""Repository for managing prompt optimization sessions and session history."""
def __init__(self, db: Session):
self.db = db
def get_session_by_id(self, session_id: uuid.UUID) -> PromptOptimizerSession | None:
session = self.db.query(PromptOptimizerSession).filter(
PromptOptimizerSession.id == session_id,
).first()
return session
def create_session(
self,
tenant_id: uuid.UUID,
user_id: uuid.UUID
) -> PromptOptimizerSession:
"""
Create a new prompt optimization session for a user and app.
Args:
tenant_id (uuid.UUID): The unique identifier of the tenant.
user_id (uuid.UUID): The unique identifier of the user.
Returns:
PromptOptimizerSession: The newly created session object.
"""
db_logger.debug(f"Create prompt optimization session: tenant_id={tenant_id}, user_id={user_id}")
try:
session = PromptOptimizerSession(
tenant_id=tenant_id,
user_id=user_id,
)
self.db.add(session)
return session
except Exception as e:
db_logger.error(f"Error creating prompt optimization session: - {str(e)}")
raise
def get_session_history(
self,
session_id: uuid.UUID,
user_id: uuid.UUID
) -> list[type[PromptOptimizerSessionHistory]]:
"""
Retrieve all message history of a specific prompt optimization session.
Args:
session_id (uuid.UUID): The unique identifier of the session.
user_id (uuid.UUID): The unique identifier of the user.
Returns:
list[PromptOptimizerSessionHistory]: A list of session history records
ordered by creation time ascending.
"""
db_logger.debug(f"Get prompt optimization session history: "
f"user_id={user_id}, session_id={session_id}")
try:
# First get the internal session ID from the session list table
session = self.db.query(PromptOptimizerSession).filter(
PromptOptimizerSession.id == session_id,
PromptOptimizerSession.user_id == user_id
).first()
if not session:
return []
history = self.db.query(PromptOptimizerSessionHistory).filter(
PromptOptimizerSessionHistory.session_id == session.id,
PromptOptimizerSessionHistory.user_id == user_id
).order_by(PromptOptimizerSessionHistory.created_at.asc()).all()
return history
except Exception as e:
db_logger.error(f"Error retrieving prompt optimization session history: session_id={session_id} - {str(e)}")
raise
def create_message(
self,
tenant_id: uuid.UUID,
session_id: uuid.UUID,
user_id: uuid.UUID,
role: RoleType,
content: str,
) -> PromptOptimizerSessionHistory:
"""
Create a new message in the session history.
This method is a placeholder for future implementation.
"""
try:
# Get the session to ensure it exists and belongs to the user
session = self.db.query(PromptOptimizerSession).filter(
PromptOptimizerSession.id == session_id,
PromptOptimizerSession.user_id == user_id,
PromptOptimizerSession.tenant_id == tenant_id
).first()
if not session:
db_logger.error(f"Session {session_id} not found for user {user_id}")
raise ValueError(f"Session {session_id} not found for user {user_id}")
message = PromptOptimizerSessionHistory(
tenant_id=tenant_id,
session_id=session.id,
user_id=user_id,
role=role.value,
content=content,
)
self.db.add(message)
return message
except Exception as e:
db_logger.error(f"Error creating prompt optimization session history: session_id={session_id} - {str(e)}")
raise
def get_first_user_message(self, session_id: uuid.UUID) -> str | None:
"""
Get the first user message from a session.
Args:
session_id (uuid.UUID): The session ID.
Returns:
str | None: The content of the first user message, or None if not found.
"""
try:
message = self.db.query(PromptOptimizerSessionHistory).filter(
PromptOptimizerSessionHistory.session_id == session_id,
PromptOptimizerSessionHistory.role == RoleType.USER.value
).order_by(
PromptOptimizerSessionHistory.created_at.asc()
).first()
return message.content if message else None
except Exception as e:
db_logger.error(f"Error getting first user message: session_id={session_id} - {str(e)}")
raise
class PromptReleaseRepository:
def __init__(self, db: Session):
self.db = db
def get_prompt_by_session_id(self, session_id: uuid.UUID) -> PromptHistory | None:
prompt_obj = self.db.query(PromptHistory).filter(
PromptHistory.session_id == session_id,
PromptHistory.is_delete.is_(False)
).first()
return prompt_obj
def create_prompt_release(
self,
tenant_id: uuid.UUID,
title: str,
session_id: uuid.UUID,
prompt: str,
) -> PromptHistory:
try:
prompt_obj = PromptHistory(
tenant_id=tenant_id,
title=title,
session_id=session_id,
prompt=prompt,
)
self.db.add(prompt_obj)
return prompt_obj
except Exception as e:
db_logger.error(f"Error creating prompt release: session_id={session_id} - {str(e)}")
raise
def soft_delete_prompt(self, prompt_obj: PromptHistory) -> None:
"""
Soft delete a prompt release by setting is_delete flag to True.
Args:
prompt_obj (PromptHistory): The prompt release object to delete.
"""
try:
prompt_obj.is_delete = True
db_logger.debug(f"Soft deleted prompt release: id={prompt_obj.id}, session_id={prompt_obj.session_id}")
except Exception as e:
db_logger.error(f"Error soft deleting prompt release: id={prompt_obj.id} - {str(e)}")
raise
def get_prompt_by_id(self, prompt_id: uuid.UUID) -> PromptHistory | None:
"""
Get a prompt release by its ID.
Args:
prompt_id (uuid.UUID): The prompt release ID.
Returns:
PromptHistory | None: The prompt release object or None if not found.
"""
try:
prompt_obj = self.db.query(PromptHistory).filter(
PromptHistory.id == prompt_id
).first()
return prompt_obj
except Exception as e:
db_logger.error(f"Error getting prompt release by id: id={prompt_id} - {str(e)}")
raise
def count_prompts(self, tenant_id: uuid.UUID) -> int:
"""
Count total number of non-deleted prompts for a tenant.
Args:
tenant_id (uuid.UUID): The tenant ID.
Returns:
int: Total count of prompts.
"""
try:
count = self.db.query(PromptHistory).filter(
PromptHistory.tenant_id == tenant_id,
PromptHistory.is_delete.is_(False)
).count()
return count
except Exception as e:
db_logger.error(f"Error counting prompts: tenant_id={tenant_id} - {str(e)}")
raise
def get_prompts_paginated(
self,
tenant_id: uuid.UUID,
offset: int,
limit: int
) -> list[PromptHistory]:
"""
Get paginated list of prompt releases for a tenant.
Args:
tenant_id (uuid.UUID): The tenant ID.
offset (int): Number of records to skip.
limit (int): Maximum number of records to return.
Returns:
list[PromptHistory]: List of prompt releases.
"""
try:
prompts = self.db.query(PromptHistory).filter(
PromptHistory.tenant_id == tenant_id,
PromptHistory.is_delete.is_(False)
).order_by(
PromptHistory.created_at.desc()
).offset(offset).limit(limit).all()
return prompts
except Exception as e:
db_logger.error(f"Error getting paginated prompts: tenant_id={tenant_id} - {str(e)}")
raise
def count_prompts_by_keyword(self, tenant_id: uuid.UUID, keyword: str) -> int:
"""
Count total number of non-deleted prompts matching keyword for a tenant.
Args:
tenant_id (uuid.UUID): The tenant ID.
keyword (str): Search keyword for title.
Returns:
int: Total count of matching prompts.
"""
try:
count = self.db.query(PromptHistory).filter(
PromptHistory.tenant_id == tenant_id,
PromptHistory.is_delete.is_(False),
PromptHistory.title.ilike(f"%{keyword}%")
).count()
return count
except Exception as e:
db_logger.error(f"Error counting prompts by keyword: tenant_id={tenant_id}, keyword={keyword} - {str(e)}")
raise
def search_prompts_paginated(
self,
tenant_id: uuid.UUID,
keyword: str,
offset: int,
limit: int
) -> list[PromptHistory]:
"""
Search prompt releases by keyword in title with pagination.
Args:
tenant_id (uuid.UUID): The tenant ID.
keyword (str): Search keyword for title.
offset (int): Number of records to skip.
limit (int): Maximum number of records to return.
Returns:
list[PromptHistory]: List of matching prompt releases.
"""
try:
prompts = self.db.query(PromptHistory).filter(
PromptHistory.tenant_id == tenant_id,
PromptHistory.is_delete.is_(False),
PromptHistory.title.ilike(f"%{keyword}%")
).order_by(
PromptHistory.created_at.desc()
).offset(offset).limit(limit).all()
return prompts
except Exception as e:
db_logger.error(f"Error searching prompts: tenant_id={tenant_id}, keyword={keyword} - {str(e)}")
raise