Commit Graph

625 Commits

Author SHA1 Message Date
Timebomb2018
ccdf7ae81d refactor(model): replace VolcanoChatOpenAI with CompatibleChatOpenAI for unified omni model support 2026-04-16 17:40:30 +08:00
Timebomb2018
7ba0726473 refactor(model): remove mutual exclusion logic between json_output and deep_thinking 2026-04-16 16:36:15 +08:00
Timebomb2018
8c6b65db12 feat(llm): add json_output support for structured LLM responses 2026-04-16 16:27:55 +08:00
Mark
5ce0bdb0f5 Merge pull request #899 from wanxunyang/feature/add-quota-check-decorator
Feature/add quota check decorator
2026-04-16 13:48:40 +08:00
Mark
c7230659e3 Merge branch 'release/v0.3.0' into develop
* release/v0.3.0: (44 commits)
  Revert "fix(web): prompt editor"
  fix(web): prompt editor
  fix(prompt-optimizer): handle escaped quotes in JSON parsing
  fix(custom-tools): remove parameter coercion in custom tool base class
  fix(core): conditionally apply thinking parameters based on model support
  refactor(custom-tools): coerce query and request body parameters to schema types
  fix(prompt-optimizer): support list content type in prompt optimizer
  refactor(memory): unify user placeholder names and harden alias sync logic
  fix(rag): replace semicolon separators with newlines in Excel parser output
  fix(web): Compatible with Windows whitespace
  fix(memory): make PgSQL the single source of truth for user entity aliases
  refactor(rag): simplify Excel parsing logic and remove redundant chunk_token_num assignment
  fix(web): Hide error message when workflow node error message equals empty string
  ci(wechat-notify): add Sourcery summary extraction with Qwen fallback
  fix(http-request,embedding,naive): tighten form-data validation, reduce truncation length to 8000, and disable chunking for Excel
  fix(web): adjust the value of End User Name
  fix(http-request): support array and file variables in form-data files upload
  fix(web): change http body key name
  fix(web): header user name
  fix(web): calculate using the filtered breadcrumbs length
  ...

# Conflicts:
#	web/src/views/UserMemoryDetail/Neo4j.tsx
#	web/src/views/UserMemoryDetail/components/EndUserProfile.tsx
#	web/src/views/UserMemoryDetail/types.ts
2026-04-15 19:31:38 +08:00
wwq
1faa258e23 feat(quota): implement unified quota management system and add community free plan
- Add `default_free_plan.py` to define the configuration for the Community Free Plan.
- Refactor `quota_stub.py` as a unified entry point, delegating checks to `core/quota_manager`.
- Implement core logic in `quota_manager.py` to support retrieving quotas from the premium module or configuration files.
- Update `tenant_subscription_controller` to return Community Free Plan information.
2026-04-15 18:48:09 +08:00
Timebomb2018
cc12ec3fa8 fix(workflow): support direct variable reference in tool parameters to preserve native types 2026-04-15 18:03:39 +08:00
Timebomb2018
3018d186f7 fix(custom-tools): remove parameter coercion in custom tool base class 2026-04-15 13:56:08 +08:00
Timebomb2018
737858731b fix(core): conditionally apply thinking parameters based on model support 2026-04-15 12:24:11 +08:00
Timebomb2018
daaee63bd5 refactor(custom-tools): coerce query and request body parameters to schema types 2026-04-15 12:15:16 +08:00
Ke Sun
29aef4527c Merge pull request #896 from SuanmoSuanyangTechnology/fix/extract-aliases
fix(memory): make PgSQL the single source of truth for user entity al…
2026-04-14 18:40:40 +08:00
lanceyq
49e0801d15 refactor(memory): unify user placeholder names and harden alias sync logic
- Replace hardcoded user placeholder name lists in write_tools and
user_memory_service with shared _USER_PLACEHOLDER_NAMES constant
- Filter user placeholder names during alias merging in _merge_attribute
  to prevent cross-role alias contamination on non-user entities
- Use toLower() in Cypher query for case-insensitive name matching
- Change PgSQL->Neo4j alias sync condition from 'if pg_aliases' to
  'if info is not None' so empty aliases correctly clear stale data
2026-04-14 18:06:56 +08:00
Timebomb2018
441b21774d fix(rag): replace semicolon separators with newlines in Excel parser output 2026-04-14 17:56:30 +08:00
lanceyq
811193dd75 fix(memory): make PgSQL the single source of truth for user entity aliases
- Skip alias merging for user entities during dedup (_merge_attribute and
  _merge_entities_with_aliases) to prevent dirty data from overwriting
  PgSQL authoritative aliases
- Add PgSQL→Neo4j alias sync after Neo4j write in write_tools to
  ensure Neo4j user entities always reflect the PgSQL source
- Remove deduped_aliases (Neo4j history) from alias sync in
  extraction_orchestrator, only append newly extracted aliases to PgSQL
- Guard Neo4j MERGE cypher to preserve existing aliases for user
  entities (name IN ['用户','我','User','I'])
- Fix emotion_analytics_service query to use ExtractedEntity label
  and entity_type property
2026-04-14 17:28:24 +08:00
Timebomb2018
75e95bab01 refactor(rag): simplify Excel parsing logic and remove redundant chunk_token_num assignment 2026-04-14 17:10:52 +08:00
Timebomb2018
e3265e4ba3 fix(http-request,embedding,naive): tighten form-data validation, reduce truncation length to 8000, and disable chunking for Excel
The form-data validation now ensures all items in the list are of type HttpFormData. Truncation length for embedding inputs is reduced from 8191 to 8000 to accommodate tokenizer differences and avoid overflow. Excel parsing now disables chunking by setting chunk_token_num to 0, aligning with intended behavior for structured file ingestion.
2026-04-14 16:14:01 +08:00
Timebomb2018
0965008210 fix(http-request): support array and file variables in form-data files upload
- Updated form-data handling to accept both single FileVariable and ArrayVariable containing FileVariable for file uploads
- Fixed HTTP client redirect handling by enabling follow_redirects=True when downloading remote files
- Adjusted config validation to correctly require list type for form-data fields instead of HttpFormData class
2026-04-14 15:53:16 +08:00
Timebomb2018
2450fe3afe refactor(workflow): move _merge_conv_vars call inside iteration loop for consistent state updates 2026-04-13 19:00:36 +08:00
Timebomb2018
10f1089198 feat(workflow): refactor iteration runtime to support independent subgraph per task
feat(app): support file metadata in chat messages and DSL app overwrite
- Extended chat message file objects with `name`, `size`, and `file_type` fields across app_chat_service and workflow_service
- Added ability to overwrite existing app configurations via DSL import in app_dsl_service, including type validation and config update logic for AgentConfig, MultiAgentConfig, and WorkflowConfig
2026-04-13 18:38:12 +08:00
wxy
72be9f75f9 feat: Add quota check decorator and implement tenant-level API rate limiting
- Add quota check decorator module quota_stub.py, providing community edition stub implementation
- Add quota check decorators to multiple controllers
- Implement tenant-level API call rate limiting
- Remove redundant plan fields from tenant_model.py
- Optimize user permission check logic with added error handling
2026-04-13 11:58:14 +08:00
Ke Sun
b8507a1df6 Merge pull request #843 from SuanmoSuanyangTechnology/feature/openclaw_lm
Feature/openclaw lm
2026-04-10 18:54:09 +08:00
miao
0f28d54c43 fix(tools): add get_required_config_parameters to OpenClawTool
Without this method, the tool status would show as available even when
server_url and api_key are not configured.
2026-04-10 18:47:31 +08:00
Timebomb2018
f01ca51896 Merge branch 'refs/heads/develop' into feature/agent-tool_xjn 2026-04-10 18:30:46 +08:00
Timebomb2018
f4a63f7d55 feat(workflow): support Dify features conversion and file variable migration 2026-04-10 18:30:12 +08:00
Ke Sun
0019f3acfd Merge pull request #860 from SuanmoSuanyangTechnology/hotfix/v0.2.10
Hotfix/v0.2.10
2026-04-10 18:29:38 +08:00
Ke Sun
58d18b476c Merge pull request #851 from SuanmoSuanyangTechnology/feat/extract-metadata
Feat/extract metadata
2026-04-10 18:11:04 +08:00
Timebomb2018
e5e6699168 feat(workflow): support nested variable access and DashScope rerank provider 2026-04-10 16:21:49 +08:00
Timebomb2018
068e2bfb7e fix(workflow): update output pattern to handle standalone curly braces 2026-04-10 15:24:18 +08:00
Timebomb2018
4ce6fede67 fix(workflow): update cycle graph node output type validation 2026-04-10 14:08:51 +08:00
miao
8497c955f9 fix(tools): make image_understand image_url optional and remove unused operation variable
Change image_url from required to optional in both operation_tool.py and
tool_service.py for image_understand operation, avoiding parameter validation
conflict with uploaded_files priority logic.
Remove unused operation variable from OpenClawTool.execute().
2026-04-10 13:31:09 +08:00
Ke Sun
807dee8460 Merge branch 'hotfix/v0.2.10' into develop 2026-04-10 10:16:39 +08:00
lanceyq
cd018814fe fix(memory): improve metadata language detection and clean_metadata logic
- Make MetadataExtractor language param optional (default None) to
  support auto-detection fallback when no language is explicitly set
- Refactor clean_metadata from walrus-operator dict comprehension to
  explicit loop for correctness and readability
2026-04-10 00:42:11 +08:00
lanceyq
e0b7e95af6 refactor(memory): remove first-person pronoun replacement and inline metadata utils
- Remove _replace_first_person_with_user from StatementExtractor to preserve
  original user text for downstream metadata/alias extraction
- Delete metadata_utils.py module, inline clean_metadata into Celery task
- Remove unused imports and commented-out collect_user_raw_messages method
- Apply formatting cleanup across metadata models and extraction orchestrator
2026-04-10 00:29:18 +08:00
lanceyq
15a863b41a feat(memory): unify alias extraction into metadata pipeline and deduplicate user entity nodes
- Merge alias add/remove into MetadataExtractionResponse and Celery metadata task,
  removing the separate sync step from extraction_orchestrator
- Replace first-person pronouns ("我") with "用户" in statement extraction to
  preserve identity semantics for downstream metadata/alias extraction
- Update extract_statement.jinja2 prompt to enforce "用户" as subject for user
  statements instead of resolving to real names
- Add alias change instructions (aliases_to_add/aliases_to_remove) to
  extract_user_metadata.jinja2 with incremental merge logic
- Deduplicate special entities ("用户", "AI助手") in graph_saver by reusing
  existing Neo4j node IDs per end_user_id
- Sync final aliases from PgSQL to Neo4j user entity nodes after metadata write
2026-04-09 21:55:59 +08:00
miao
7842435321 fix(tools): forward set_runtime_context through OperationTool to base_tool
OperationTool wraps builtin tools for multi-operation support but did not
forward set_runtime_context, causing OpenClawTool to miss uploaded_files
and conversation_id when used with operation routing.
2026-04-09 20:01:07 +08:00
Timebomb2018
ca4f7aa65d refactor(rag/nlp): refactor reranking logic to apply post-deduplication and remove debug log 2026-04-09 19:35:43 +08:00
miao
b875626f18 fix(tools): revert CustomTool __init__ to upstream, remove redundant schema parsing
The _parse_openapi_schema() method already handles string-to-dict conversion internally, so the duplicate json.loads in __init__ was unnecessary.
2026-04-09 19:33:27 +08:00
Timebomb2018
130684cac0 refactor(rag/nlp): standardize knowledge graph retrieval to use DocumentChunk and add debug logging
The knowledge graph retrieval logic in `search.py` was updated to consistently return `DocumentChunk` instances instead of raw dictionaries, improving type safety and alignment with the RAG pipeline's expected data structure. Additionally, debug logging was enhanced in `draft_run_service.py` to log the full `retrieve_chunks_result` before extracting page content, aiding troubleshooting.
2026-04-09 19:07:53 +08:00
Timebomb2018
62e0b2730b refactor(workflow/knowledge): update pattern matching to support multiple retrieve types 2026-04-09 18:29:08 +08:00
miao
55b2e05ba8 feat(tools): refactor migrate OpenClaw from custom tool to builtin tool
Create OpenClawTool class inheriting BuiltinTool with dedicated config
Remove all x-openclaw special handling from CustomTool (~270 lines)
Add multi-operation support (print_task, device_query, image_understand, general)
Change ensure_builtin_tools_initialized to incremental mode for auto-provisioning
Fix OperationTool and LangchainAdapter to support OpenClaw operation routing
2026-04-09 18:14:31 +08:00
miao
562ca6c1f1 fix(tools): fix OpenClaw connection test and multimodal format compatibility
- Use safe .get() for server URL to avoid KeyError
- Support both api_key and token in connection test auth
- Add OpenAI/Volcano image format (image_url) support
- Add aiohttp import in _test_openclaw_connection
2026-04-09 18:14:30 +08:00
miao
e298b38de9 feat(tools): add OpenClaw remote agent tool integration
- Detect x-openclaw flag in OpenAPI schema and init dedicated config
- Implement multimodal input/output (image download, compress, base64)
- Add OpenClaw connection test and status validation in tool service
- Fix auth_config token check to support both api_key and bearer_token
- Inject runtime context (user_id, conversation_id, files) in chat services
2026-04-09 18:14:29 +08:00
Timebomb2018
a7b8ba0c66 fix(rag): fix pdfplumber concurrency issue and add debug logging
The pdfplumber parser now uses a global lock to prevent concurrent access issues during PDF image rendering. Additionally, added a warning log to trace knowledge retrieval results for debugging purposes. The syntax fix in knowledge node's match case ensures correct pattern matching behavior.

BREAKING CHANGE: The pdfplumber parser now requires LOCK_KEY_pdfplumber to be defined in sys.modules for thread safety.

Closes #841
2026-04-09 17:48:16 +08:00
Timebomb2018
ea2f5e61c9 fix(tool): strip input_value in datetime_to_timestamp to prevent whitespace-related parsing errors 2026-04-09 15:18:39 +08:00
Timebomb2018
5975d70bf9 feat(tool): add datetime_to_timestamp operation with timezone support 2026-04-09 15:14:15 +08:00
lanceyq
e0546e01ef refactor(memory): delegate metadata merging to LLM instead of code-based merge
- Remove merge_metadata and its helper functions from metadata_utils.py
- Pass existing_metadata to MetadataExtractor.extract_metadata() as LLM context
- Add merge instructions to extract_user_metadata.jinja2 prompt (zh/en)
- Update Celery task to read existing metadata before extraction and overwrite
- Simplify field descriptions in UserMetadataProfile model
- Add _update_timestamps helper to track changed fields
2026-04-09 15:10:29 +08:00
Timebomb2018
70aab94fc3 feat(knowledge): support graph retrieval type with dynamic API key selection 2026-04-09 15:00:49 +08:00
lanceyq
f2d7479229 feat(memory): add async user metadata extraction pipeline
- Add MetadataExtractor to collect user-related statements post-dedup
  and extract profile/behavioral metadata via independent LLM call
- Add Celery task (extract_user_metadata) routed to memory_tasks queue
- Add metadata models (UserMetadata, UserMetadataProfile, etc.)
- Add metadata utility functions (clean, validate, merge with _op support)
- Add Jinja2 prompt template for metadata extraction (zh/en)
- Fix Lucene query parameter naming: rename `q` to `query` across all
  Cypher queries, graph_search functions, and callers
- Escape `/` in Lucene queries to prevent TokenMgrError
- Add `speaker` field to ChunkNode and persist it in Neo4j
- Remove unused imports (argparse, os, UUID) in search.py
- Fix unnecessary db context nesting in interest distribution task
2026-04-09 11:01:56 +08:00
Ke Sun
ae1909b7e9 Merge pull request #833 from SuanmoSuanyangTechnology/release/v0.2.10
Release/v0.2.10
2026-04-08 21:45:35 +08:00
Timebomb2018
e3d50c5c55 fix(workflow): unify token usage metadata handling across LLM-related nodes 2026-04-08 20:44:02 +08:00