Commit Graph

2618 Commits

Author SHA1 Message Date
lanceyq
0afc38e7ef refactor(memory): flatten meta_data fields in update_end_user_info response
Align update response with get_end_user_info by extracting profile,
knowledge_tags, and behavioral_hints to top-level keys instead of
returning raw meta_data dict.
2026-04-10 18:45:35 +08:00
lanceyq
627d6a0381 fix : add comments 2026-04-10 10:43:43 +08:00
lanceyq
cd018814fe fix(memory): improve metadata language detection and clean_metadata logic
- Make MetadataExtractor language param optional (default None) to
  support auto-detection fallback when no language is explicitly set
- Refactor clean_metadata from walrus-operator dict comprehension to
  explicit loop for correctness and readability
2026-04-10 00:42:11 +08:00
lanceyq
e0b7e95af6 refactor(memory): remove first-person pronoun replacement and inline metadata utils
- Remove _replace_first_person_with_user from StatementExtractor to preserve
  original user text for downstream metadata/alias extraction
- Delete metadata_utils.py module, inline clean_metadata into Celery task
- Remove unused imports and commented-out collect_user_raw_messages method
- Apply formatting cleanup across metadata models and extraction orchestrator
2026-04-10 00:29:18 +08:00
lanceyq
15a863b41a feat(memory): unify alias extraction into metadata pipeline and deduplicate user entity nodes
- Merge alias add/remove into MetadataExtractionResponse and Celery metadata task,
  removing the separate sync step from extraction_orchestrator
- Replace first-person pronouns ("我") with "用户" in statement extraction to
  preserve identity semantics for downstream metadata/alias extraction
- Update extract_statement.jinja2 prompt to enforce "用户" as subject for user
  statements instead of resolving to real names
- Add alias change instructions (aliases_to_add/aliases_to_remove) to
  extract_user_metadata.jinja2 with incremental merge logic
- Deduplicate special entities ("用户", "AI助手") in graph_saver by reusing
  existing Neo4j node IDs per end_user_id
- Sync final aliases from PgSQL to Neo4j user entity nodes after metadata write
2026-04-09 21:55:59 +08:00
lanceyq
e0546e01ef refactor(memory): delegate metadata merging to LLM instead of code-based merge
- Remove merge_metadata and its helper functions from metadata_utils.py
- Pass existing_metadata to MetadataExtractor.extract_metadata() as LLM context
- Add merge instructions to extract_user_metadata.jinja2 prompt (zh/en)
- Update Celery task to read existing metadata before extraction and overwrite
- Simplify field descriptions in UserMetadataProfile model
- Add _update_timestamps helper to track changed fields
2026-04-09 15:10:29 +08:00
lanceyq
f2d7479229 feat(memory): add async user metadata extraction pipeline
- Add MetadataExtractor to collect user-related statements post-dedup
  and extract profile/behavioral metadata via independent LLM call
- Add Celery task (extract_user_metadata) routed to memory_tasks queue
- Add metadata models (UserMetadata, UserMetadataProfile, etc.)
- Add metadata utility functions (clean, validate, merge with _op support)
- Add Jinja2 prompt template for metadata extraction (zh/en)
- Fix Lucene query parameter naming: rename `q` to `query` across all
  Cypher queries, graph_search functions, and callers
- Escape `/` in Lucene queries to prevent TokenMgrError
- Add `speaker` field to ChunkNode and persist it in Neo4j
- Remove unused imports (argparse, os, UUID) in search.py
- Fix unnecessary db context nesting in interest distribution task
2026-04-09 11:01:56 +08:00
Ke Sun
cfbf83f71e Merge pull request #787 from SuanmoSuanyangTechnology/fix/atomic-update
fix(memory): improve optimistic lock resilience in access history man…
2026-04-07 10:57:20 +08:00
yingzhao
a43e8fa594 Merge pull request #790 from SuanmoSuanyangTechnology/feature/ui_upgrade_zy
Feature/UI upgrade zy
2026-04-03 20:50:05 +08:00
yingzhao
6c8d0d9d64 Merge branch 'develop' into feature/ui_upgrade_zy 2026-04-03 20:49:53 +08:00
zhaoying
bd2a3bd7ef fix(web): VariableSelect & editor 2026-04-03 20:45:37 +08:00
zhaoying
1f72b8aa70 fix(web): Group Variable filter 2026-04-03 20:14:36 +08:00
yingzhao
9bb32888a2 Merge pull request #789 from SuanmoSuanyangTechnology/feature/ui_upgrade_zy
Feature/UI upgrade zy
2026-04-03 19:54:34 +08:00
Mark
caee5d214e Merge pull request #788 from SuanmoSuanyangTechnology/feature/agent-tool_xjn
feat(workflow)
2026-04-03 19:01:17 +08:00
Timebomb2018
38f3455bab feat(workflow):
1. add list operator node for filtering, sorting, limiting, and extracting list items;
2. Increase the session variable to the "file" type
2026-04-03 18:57:28 +08:00
zhaoying
d60cb423a4 feat(web): workflow add opening_statement 2026-04-03 18:53:22 +08:00
zhaoying
b20a65ce29 feat(web): add list-operator node & support file type variables 2026-04-03 18:52:06 +08:00
lanceyq
99862db7a0 refactor(forgetting-engine): replace optimistic locking with APOC atomic operations in access history manager
- Replace version-based optimistic locking and retry loop with apoc.atomic.add/insert for concurrent safety
- Merge duplicate accesses within a batch before updating (access_count_delta)
- Simplify _calculate_update to only compute on new timestamps instead of full history rebuild
- Remove max_retries instance variable (kept as param for backward compat)
- Trim verbose docstrings and inline comments
2026-04-03 18:40:03 +08:00
lanceyq
00a8099857 changes:(api) Change the "jitter" to "tremble". 2026-04-03 16:55:53 +08:00
lanceyq
117e29fbe3 fix(memory): improve optimistic lock resilience in access history manager
- Increase max_retries from 3 to 5 for concurrent conflict recovery
- Add randomized exponential backoff between retries to reduce contention
- Merge duplicate node accesses in batch operations to avoid self-conflicts
- Support access_times parameter for merged batch access counting
- Add Community node label support in atomic update content field map
2026-04-03 16:46:09 +08:00
Mark
32740e8159 Merge pull request #786 from wanxunyang/feat/app-log-wxy
fix(workflow): persist citations in conversation message meta_data
2026-04-03 16:26:02 +08:00
Ke Sun
bc5ea2d421 Merge pull request #784 from SuanmoSuanyangTechnology/fix/aliases-extract
feat(memory): prevent cross-role alias contamination between user and…
2026-04-03 15:26:31 +08:00
Ke Sun
d34bf4bc89 Merge pull request #782 from SuanmoSuanyangTechnology/hotfix/v0.2.9
fix(web): string type language Editor init
2026-04-03 15:21:35 +08:00
lanceyq
c4ff1a325b refactor(memory): harden alias extraction and sync PgSQL with Neo4j deduped aliases
- Strengthen anti-hallucination rules in extract_triplet prompt to
  enforce verbatim-only alias extraction, removing suggestive examples
- Add _extract_deduped_entity_aliases to sync historical aliases from
  Neo4j two-stage dedup into PgSQL end_user_info
- Remove unused _fetch_neo4j_user_aliases; reuse injected connector
  instead of instantiating new Neo4jConnector
- Simplify _would_merge_cross_role and reuse clean_cross_role_aliases
  in _normalize_special_entity_names
- Reuse _USER_PLACEHOLDER_NAMES from dedup module to avoid duplication
2026-04-03 14:38:55 +08:00
wxy
d1f0258065 fix(workflow): persist citations in conversation message meta_data 2026-04-03 14:36:43 +08:00
wxy
5db59bc9cf fix(workflow): persist citations in conversation message meta_data 2026-04-03 14:32:41 +08:00
Mark
a711635694 Merge pull request #785 from wanxunyang/feat/app-log-wxy
feat(workflow): add opening statement and citation support
2026-04-03 13:41:08 +08:00
lanceyq
15b3ce3dd5 refactor(memory): deduplicate assistant alias query and fix case-sensitive placeholder matching
- Extract fetch_neo4j_assistant_aliases() into deduped_and_disamb.py as
  single source of truth, replacing inline Cypher in write_tools and
  extraction_orchestrator
- Normalize USER_PLACEHOLDER_NAMES to lowercase and apply .lower() on
  all comparisons to prevent case-variant names leaking into aliases
2026-04-03 13:15:57 +08:00
lanceyq
9cc19047b4 fix(memory): prevent cross-role alias contamination in entity dedup
- Extract user aliases from raw dialog statements instead of post-dedup
  entities to bypass merge pollution
- Add alias cross-cleaning step in _normalize_special_entity_names to
  strip AI assistant aliases from user entities before dedup
- Call clean_cross_role_aliases after second-layer dedup to handle
  historical dirty data merged from Neo4j
- Fix syntax error in prompt_utils.py (ontology_types variable assignment)
2026-04-03 12:34:04 +08:00
wxy
2e8e63878e feat(workflow): add opening statement and citation support
- Trigger opening statement on new conversation in run/run_stream
- Fix /opening endpoint to support workflow app type
- Fix features field missing in workflow config release snapshot
- Knowledge node returns citations alongside chunks
- Aggregate citations from all knowledge nodes in result builder
- Filter citations based on features.citation.enabled switch
- Fix WorkflowConfigCreate circular import in app_schema
2026-04-03 11:44:07 +08:00
Ke Sun
38955d7d45 Merge pull request #783 from SuanmoSuanyangTechnology/fix/code-none-value
fix(code-node): prevent null errors by adding default value handling
2026-04-03 11:19:58 +08:00
Ke Sun
b6167d4e94 Merge pull request #761 from SuanmoSuanyangTechnology/feature/perceptual-read
feat(memory): add perceptual memory retrieval service with BM25+embedding fusionding fusion
2026-04-03 11:19:33 +08:00
lanceyq
7890970a39 feat(memory): prevent cross-role alias contamination between user and AI entities
- Add speaker context to triplet extraction prompt to distinguish alias ownership
- Add explicit examples and rules in extract_triplet.jinja2 for user vs AI alias attribution
- Introduce cross-role merge protection in dedup (accurate, fuzzy, and LLM stages)
- Normalize special entity names (用户/AI助手) before deduplication
- Add clean_cross_role_aliases() to sanitize aliases before Neo4j write
- Refactor _update_end_user_other_name to merge aliases from PgSQL instead of Neo4j
- Filter AI assistant aliases from user alias extraction in orchestrator
2026-04-03 10:57:30 +08:00
Eternity
203732de1d fix(code-node): prevent null errors by adding default value handling 2026-04-03 10:18:33 +08:00
zhaoying
fa4be10e51 fix(web): string type language Editor init 2026-04-02 17:18:08 +08:00
yingzhao
1b52850526 Merge pull request #780 from SuanmoSuanyangTechnology/feature/ui_upgrade_zy
Feature/UI upgrade zy
2026-04-02 16:11:45 +08:00
zhaoying
1732fc7af5 feat(web): markdown form support submit 2026-04-02 16:07:02 +08:00
yingzhao
a52e2137b7 Merge pull request #779 from SuanmoSuanyangTechnology/develop
Develop
2026-04-02 16:01:08 +08:00
Ke Sun
377f79773d Merge pull request #778 from SuanmoSuanyangTechnology/hotfix/v0.2.9
fix(web): jinja2 editor
2026-04-02 15:39:59 +08:00
Ke Sun
63235de42b Merge pull request #774 from SuanmoSuanyangTechnology/feat/data-transformation
Feat/data transformation
2026-04-02 15:37:50 +08:00
Mark
106a32bc3a Merge pull request #770 from wanxunyang/feat/app-log-wxy
fix: update app_shares.is_active and updated_at when deleting shared app
2026-04-02 15:25:17 +08:00
zhaoying
dcb7b496d3 fix(web): jinja2 editor 2026-04-02 15:16:04 +08:00
Ke Sun
0b47194f12 Merge pull request #776 from SuanmoSuanyangTechnology/hotfix/v0.2.9
Hotfix/v0.2.9
2026-04-02 14:40:20 +08:00
lanceyq
abbd92b74c Interface performance optimization, using only one function 2026-04-02 14:19:27 +08:00
lanceyq
960ee9f2df changes:(services) Modify the query method for user memory to batch processing. 2026-04-02 14:07:51 +08:00
Ke Sun
1c133d3d6c Merge pull request #775 from SuanmoSuanyangTechnology/pref/redis-connections
refactor(tasks, redis_lock): improve Redis connection and lock handling
2026-04-02 14:06:46 +08:00
Eternity
d270d25a99 refactor(tasks, redis_lock): improve Redis connection and lock handling
- Increased max_connections for Redis pool from 10 to 100.
- Extended socket_timeout from 5 to 10 seconds.
- Added retry mechanism with exponential backoff for Redis operations in `RedisFairLock`.
2026-04-02 13:57:22 +08:00
lanceyq
8abd59b26e changes:(controllers & services) The method for calculating general data is extracted and presented as a shared function. 2026-04-02 13:02:21 +08:00
lanceyq
bd48b4fdbe changes:(controllers) Modify the statistical method of the knowledge base 2026-04-02 12:26:20 +08:00
zhaoying
9535545947 fix(web): if-else cases 2026-04-02 12:13:11 +08:00