Release/v0.2.2 (#258)

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* Fix/interface home (#182)

* [fix]Fix the interface for statistics of recent activities and applications

* [changes]Modify the code based on the AI review
1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter.
2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls.

* [fix]Fix the interface for statistics of recent activities and applications

* [changes]Modify the code based on the AI review
1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter.
2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls.

* Fix/optimize inerface (#183)

* [changes]Optimize the time consumption of the "/end_users" interface

* [fix]Optimize the time consumption of the "/hot_memory_tags" interface

* [changes]Optimize the time consumption of the "/end_users" interface

* [fix]Optimize the time consumption of the "/hot_memory_tags" interface

* [changes]Improve the code based on AI review

* Fix/memory mcp2 1 (#184)

* 优化快速检索的回复内容

* 优化快速检索的回复内容

* Fix/memory mcp2 1 (#185)

* 优化快速检索的回复内容

* 优化快速检索的回复内容

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* Fix/memory mcp2 1 (#188)

* 优化快速检索的回复内容

* 优化快速检索的回复内容

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* 解决冲突

* 解决冲突

* feat(home page): version description update

* Fix/memory mcp2 1 (#190)

* 优化快速检索的回复内容

* 优化快速检索的回复内容

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* feat(web): memory related interface parameter transfer adjustment

* 感知meta_data字段BUG修复

* Fix/memory bug fix (#171)

* feat(sandbox): add Python 3 code execution sandbox support

* feat(workflow): emit SSE events for node exception output

* perf(sandbox): optimize code encryption handling

* perf(workflow): update standard node output structure

* [add] migration script

* [modify] migration script

* feat(web): add workflow runtime info

* fix(web):  handleSSE bugfix

* fix(sandbox): prevent imports from being blocked when network is disabled

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* Fix/memory bug fix (#199)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* user_id->显示为config_id_old传输

* feat(web): update read_all_config select valueKey

* user_id->显示为config_id_old传输

* feat(workflow): Add a new node for executing code

* fix(web): KnowledgeConfigModal bugfix

* fix(web): iteration's variable add parameter-extractor  node

* fix(sandbox): treat non-zero exit codes as errors instead of relying only on stderr

* Fix/memory bug fix (#200)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* Refactor/benchmark test (#196)

* [changes]refactor locomo_test

* [fix]Fix the circular import of ModelParameters

* [changes]The benchmark test can run stably.

* [fix]Complete end-to-end LoCoMo repair

* [fix]Complete the end-to-end longmemeval and memsciqa fixes

* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.

* [changes]refactor locomo_test

* [fix]Fix the circular import of ModelParameters

* [changes]The benchmark test can run stably.

* [fix]Complete end-to-end LoCoMo repair

* [fix]Complete the end-to-end longmemeval and memsciqa fixes

* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.

* [changes]Benchmark test adaptation for end_user_id

* [changes]refactor locomo_test

* [fix]Fix the circular import of ModelParameters

* [changes]The benchmark test can run stably.

* [fix]Complete end-to-end LoCoMo repair

* [fix]Complete the end-to-end longmemeval and memsciqa fixes

* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.

* [fix]Complete the end-to-end longmemeval and memsciqa fixes

* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.

* [changes]Benchmark test adaptation for end_user_id

* [modify] migration script

* delete benchmark-test (#204)

* Refactor: Move evaluation folder to redbear-mem-benchmark submodule

* [changes]Restore .gitmodules

* feat(web): workflow add code node

* 检查需要更改的格式问题

* Fix/redbear benchmark (#205)

* Refactor: Move evaluation folder to redbear-mem-benchmark submodule

* [changes]Update submodule reference

* Refactor: Move evaluation folder to redbear-mem-benchmark submodule

* [changes]Update submodule reference

* Remove duplicate evaluation submodule, use redbear-mem-benchmark instead

* Fix/memory bug fix (#207)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* fix(web): remove URI decode and encode

* [add] plugin system and base sso module

* 修复宿主列表获取memory_config_idBUG

* Fix/memory bug fix (#209)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* [modify] file local server url

* [add] migration script

* fix(workflow): fix activation and branch control issues in streaming output

* fix(workflow): fix function cache not taking effect and potential list index overflow

* style(workflow): enforce PEP8 style and remove redundant imports

* fix(workflow): fix streaming output error when variable is not a string

* [fix]remove aspose-slides

* perf(workflow): enhance streaming output node activation performance

* feat(workflow): store token usage in message table

* feat(web): add PageEmpty component

* feat(web): add PageTabs component

* perf(workflow): make memory configuration backward compatible

* feat(web): update model management

* config_id做映射

* config_id做映射

* Fix/memory bug fix (#211)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

* config_id做映射

* config_id做映射

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* feat(web): getModelListUrl add is_active param

* config_id做映射+1

* config_id做映射+1

* config_id做映射+1

* feat(web): remove file url replace

* Fix/memory bug fix (#212)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

* config_id做映射

* config_id做映射

* config_id做映射+1

* config_id做映射+1

* config_id做映射+1

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics

* feat(web): model logo update

* 应用层memory_content->memory_config

* fix(web): correct spelling

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

* Fix/memory bug fix (#215)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

* config_id做映射

* config_id做映射

* config_id做映射+1

* config_id做映射+1

* config_id做映射+1

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics

* fix(web): model loading update

* 统一字段为config_id_old

* 统一字段为config_id_old

* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics

* 统一字段为config_id_old

* 统一字段为config_id_old

* memory_content暂时不修改

* memory_content暂时不修改

* Fix/memory bug fix (#217)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

* config_id做映射

* config_id做映射

* config_id做映射+1

* config_id做映射+1

* config_id做映射+1

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

* 统一字段为config_id_old

* 统一字段为config_id_old

* 统一字段为config_id_old

* 统一字段为config_id_old

* memory_content暂时不修改

* memory_content暂时不修改

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* feat(web): add app statistics

* fix(workflow): fix streaming output issues with multi-output End nodes

End nodes with multiple output segments could cause cursor errors or leave some
segments inactive, resulting in incorrect final outputs.
Unified _emit_active_chunks and _update_scope_activate to ensure all segments
are activated in order and streamed correctly.

* feat(web): add apps statistics api

* fix(web): agent's knowledge_bases bugfix

* Revert "feat(web): update read_all_config select valueKey"

This reverts commit 46f0f3cee9.

* [add] migrations script

* perf(workflow): make memory write node backward-compatible and defer config validation

* 旧数据兼容

* 旧数据兼容

* 旧数据兼容

* 旧数据兼容

* fix(web): model bugfix

* fix(web): model bugfix

* 提交遗漏 (#228)

* [fix] chat api for workflow

* [fix] web search set for v1 api

* fix(web): model bugfix

* fix(web): model list remove is_active

* fix(model): bug fix

* [add]migration script

* [fix] api

* [fix] api

* fix(web): model bugfix

* fix(model): the model type does not allow modification,  delete tts and speech2text type

* fix(model): bug fix

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* Add/develop memory (#239)

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* feat(web): model ui update

* feat(web): model ui update

* Add/develop memory (#243)

* 遗漏的历史映射

* 遗漏的历史映射

* fix(model): bug fix

* feat(web): model ui update

* Add/develop memory (#247)

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* [modify] migration script

* [add] migration script

* fix(web): change form message

* fix(web): the memoryContent field is compatible with numbers and strings

* feat(web): code node hidden

* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.

---------

Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: yujiangping <yujiangping@taofen8.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: lixiangcheng1 <lixiangcheng1@wanda.cn>
This commit is contained in:
Ke Sun
2026-01-30 14:51:34 +08:00
committed by GitHub
parent 988a41f5e4
commit 0159fdf149
320 changed files with 11769 additions and 11942 deletions

134
sandbox/app/config.py Normal file
View File

@@ -0,0 +1,134 @@
"""Configuration management"""
import os
from typing import List, Optional
from pydantic import BaseModel, Field
import yaml
SANDBOX_USER_ID = 1000
SANDBOX_GROUP_ID = 1000
DEFAULT_PYTHON_LIB_REQUIREMENTS_AMD = [
"/usr/local/lib/python3.12",
"/usr/lib/python3",
"/usr/lib/x86_64-linux-gnu",
"/etc/ssl/certs/ca-certificates.crt",
"/etc/nsswitch.conf",
"/etc/hosts",
"/etc/resolv.conf",
"/run/systemd/resolve/stub-resolv.conf",
"/run/resolvconf/resolv.conf",
"/etc/localtime",
"/usr/share/zoneinfo",
"/etc/timezone",
]
class AppConfig(BaseModel):
"""Application configuration"""
port: int = 8194
debug: bool = True
key: str = "redbear-sandbox"
class ProxyConfig(BaseModel):
"""Proxy configuration"""
socks5: str = ""
http: str = ""
https: str = ""
class Config(BaseModel):
"""Global configuration"""
app: AppConfig = Field(default_factory=AppConfig)
max_workers: int = 4
max_requests: int = 50
worker_timeout: int = 30
nodejs_path: str = "node"
enable_network: bool = True
enable_preload: bool = False
python_path: str = ""
python_lib_paths: list = Field(default=DEFAULT_PYTHON_LIB_REQUIREMENTS_AMD)
python_deps_update_interval: str = "30m"
allowed_syscalls: List[int] = Field(default_factory=list)
proxy: ProxyConfig = Field(default_factory=ProxyConfig)
# Global configuration instance
_config: Optional[Config] = None
def load_config(config_path: str) -> Config:
"""Load configuration from YAML file"""
global _config
# Load from file
if os.path.exists(config_path):
with open(config_path, 'r') as f:
data = yaml.safe_load(f)
_config = Config(**data)
else:
_config = Config()
# Override with environment variables
if os.getenv("DEBUG"):
_config.app.debug = os.getenv("DEBUG").lower() in ("true", "1", "yes")
if os.getenv("MAX_WORKERS"):
_config.max_workers = int(os.getenv("MAX_WORKERS"))
if os.getenv("MAX_REQUESTS"):
_config.max_requests = int(os.getenv("MAX_REQUESTS"))
if os.getenv("SANDBOX_PORT"):
_config.app.port = int(os.getenv("SANDBOX_PORT"))
if os.getenv("WORKER_TIMEOUT"):
_config.worker_timeout = int(os.getenv("WORKER_TIMEOUT"))
if os.getenv("API_KEY"):
_config.app.key = os.getenv("API_KEY")
if os.getenv("NODEJS_PATH"):
_config.nodejs_path = os.getenv("NODEJS_PATH")
if os.getenv("ENABLE_NETWORK"):
_config.enable_network = os.getenv("ENABLE_NETWORK").lower() in ("true", "1", "yes")
if os.getenv("ENABLE_PRELOAD"):
_config.enable_preload = os.getenv("ENABLE_PRELOAD").lower() in ("true", "1", "yes")
if os.getenv("ALLOWED_SYSCALLS"):
_config.allowed_syscalls = [int(x) for x in os.getenv("ALLOWED_SYSCALLS").split(",")]
if os.getenv("SOCKS5_PROXY"):
_config.proxy.socks5 = os.getenv("SOCKS5_PROXY")
if os.getenv("HTTP_PROXY"):
_config.proxy.http = os.getenv("HTTP_PROXY")
if os.getenv("HTTPS_PROXY"):
_config.proxy.https = os.getenv("HTTPS_PROXY")
# python
if os.getenv("PYTHON_PATH"):
_config.python_path = os.getenv("PYTHON_PATH")
if os.getenv("PYTHON_LIB_PATH"):
_config.python_lib_paths = os.getenv("PYTHON_LIB_PATH").split(',')
if os.getenv("PYTHON_DEPS_UPDATE_INTERVAL"):
_config.python_deps_update_interval = os.getenv("PYTHON_DEPS_UPDATE_INTERVAL")
return _config
config_path = os.getenv("CONFIG_PATH", "config.yaml")
load_config(config_path)
def get_config() -> Config:
"""Get global configuration"""
if _config is None:
raise RuntimeError("Configuration not loaded. Call load_config() first.")
return _config

View File

@@ -0,0 +1,8 @@
from fastapi import APIRouter
from . import health_controller, sandbox_controller
manager_router = APIRouter()
manager_router.include_router(health_controller.router)
manager_router.include_router(sandbox_controller.router)

View File

@@ -0,0 +1,12 @@
"""Health check endpoint"""
from fastapi import APIRouter
from app.models import HealthResponse
router = APIRouter()
@router.get("/health", response_model=HealthResponse)
async def health_check():
"""Health check endpoint"""
return HealthResponse(status="healthy", version="2.0.0")

View File

@@ -0,0 +1,59 @@
"""Sandbox API endpoints"""
from fastapi import APIRouter, Depends
from app.middleware.auth import verify_api_key
from app.middleware.concurrency import check_max_requests, acquire_worker
from app.models import (
RunCodeRequest,
ApiResponse,
UpdateDependencyRequest,
error_response
)
from app.services.python_service import (
run_python_code,
list_python_dependencies,
update_python_dependencies
)
router = APIRouter(
prefix="/v1/sandbox",
tags=["sandbox"],
dependencies=[Depends(verify_api_key)]
)
@router.post(
"/run",
response_model=ApiResponse,
dependencies=[Depends(check_max_requests),
Depends(acquire_worker)]
)
async def run_code(request: RunCodeRequest):
"""Execute code in sandbox"""
if request.language == "python3":
return await run_python_code(request.code, request.preload, request.options)
elif request.language == "nodejs":
# TODO
return error_response(-400, "TODO")
else:
return error_response(-400, "unsupported language")
@router.get("/dependencies", response_model=ApiResponse)
async def get_dependencies(language: str):
"""Get installed dependencies"""
if language == "python3":
return await list_python_dependencies()
else:
return error_response(-400, "unsupported language")
@router.post("/dependencies/update", response_model=ApiResponse)
async def update_dependencies(request: UpdateDependencyRequest):
"""Update dependencies"""
if request.language == "python3":
return await update_python_dependencies()
else:
return error_response(-400, "unsupported language")

View File

@@ -0,0 +1 @@
"""Core functionality package"""

View File

@@ -0,0 +1,33 @@
"""Code encryption utilities"""
import base64
def encrypt_code(code: bytes, key: bytes) -> str:
"""Encrypt code using XOR cipher with base64 encoding
Args:
code: Plain code string
key: Encryption key bytes
Returns:
Base64 encoded encrypted code
"""
key_length = len(key)
encrypted_code = bytearray(len(code))
for i in range(len(code)):
encrypted_code[i] = code[i] ^ key[i % key_length]
encoded_code = base64.b64encode(encrypted_code).decode("utf-8")
return encoded_code
def generate_key(length: int = 64) -> bytes:
"""Generate random encryption key
Args:
length: Key length in bytes (default 64 for 512 bits)
Returns:
Random key bytes
"""
import secrets
return secrets.token_bytes(length)

View File

@@ -0,0 +1,47 @@
"""Code execution engine"""
import os
from typing import Optional
from abc import ABC, abstractmethod
from app.config import get_config
from app.logger import get_logger
from app.models import RunnerOptions
class ExecutionResult:
"""Result of code execution"""
def __init__(self, stdout: str = "", stderr: str = "", exit_code: int = 0, error: Optional[str] = None):
self.stdout = stdout
self.stderr = stderr
self.exit_code = exit_code
class CodeExecutor(ABC):
"""Base code executor"""
def __init__(self):
self.logger = get_logger()
self.config = get_config()
@abstractmethod
async def run(
self,
code: str,
options: RunnerOptions,
preload: str = "",
timeout: Optional[int] = None
) -> ExecutionResult:
pass
def cleanup_temp_file(self, file_path: str) -> None:
"""Remove temporary file
Args:
file_path: Path to file to remove
"""
try:
if os.path.exists(file_path):
os.remove(file_path)
except Exception as e:
self.logger.warning(f"Failed to cleanup temp file {file_path}: {e}")

View File

@@ -0,0 +1 @@
"""Code runners package"""

View File

@@ -0,0 +1,4 @@
# -*- coding: UTF-8 -*-
# Author: Eternity
# @Email: 1533512157@qq.com
# @Time : 2026/1/23 11:27

View File

@@ -0,0 +1,50 @@
import asyncio
import tempfile
import stat
from pathlib import Path
from app.config import get_config
from app.core.runners.python.settings import LIB_PATH
from app.logger import get_logger
logger = get_logger()
async def prepare_python_dependencies_env():
config = get_config()
with tempfile.TemporaryDirectory(dir="/") as root_path:
root = Path(root_path)
env_sh = root / "env.sh"
with open("script/env.sh") as f:
env_sh.write_text(f.read())
env_sh.chmod(env_sh.stat().st_mode | stat.S_IXUSR)
for lib_path in config.python_lib_paths:
lib_path = Path(lib_path)
if not lib_path.exists():
logger.warning("python lib path %s is not available", lib_path)
continue
cmd = [
"bash",
str(env_sh),
str(lib_path),
str(LIB_PATH),
]
process = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE
)
stdout, stderr = await process.communicate()
retcode = process.returncode
if retcode != 0:
logger.error(
f"create env error for file {lib_path}: retcode={retcode}, stderr={stderr.decode()}"
)

View File

@@ -0,0 +1,56 @@
import ctypes
import os
import sys
import traceback
from base64 import b64decode
# Setup exception hook
def excepthook(etype, value, tb):
sys.stderr.write("".join(traceback.format_exception(etype, value, tb)))
sys.stderr.flush()
sys.exit(-1)
sys.excepthook = excepthook
# Load security library if available
lib = ctypes.CDLL("./libpython.so")
lib.init_seccomp.argtypes = [ctypes.c_uint32, ctypes.c_uint32, ctypes.c_bool]
lib.init_seccomp.restype = None # TODO: raise error info
# Get running path
running_path = sys.argv[1]
if not running_path:
exit(-1)
# Get decrypt key
key = sys.argv[2]
if not key:
exit(-1)
key = b64decode(key)
os.chdir(running_path)
# Preload code
{{preload}}
# Apply security if library is available
lib.init_seccomp({{uid}}, {{gid}}, {{enable_network}})
# Decrypt and execute code
code = b64decode("{{code}}")
def decrypt(code, key):
key_len = len(key)
code_len = len(code)
code = bytearray(code)
for i in range(code_len):
code[i] = code[i] ^ key[i % key_len]
return bytes(code)
code = decrypt(code, key)
exec(code)

View File

@@ -0,0 +1,154 @@
"""Python code runner"""
import asyncio
import base64
import os
import uuid
from typing import Optional
from app.config import SANDBOX_USER_ID, SANDBOX_GROUP_ID, get_config
from app.core.encryption import generate_key, encrypt_code
from app.core.executor import CodeExecutor, ExecutionResult
from app.core.runners.python.settings import check_lib_avaiable, release_lib_binary, LIB_PATH
from app.logger import get_logger
from app.models import RunnerOptions
# Python sandbox prescript template
with open("app/core/runners/python/prescript.py") as f:
PYTHON_PRESCRIPT = f.read()
logger = get_logger()
class PythonRunner(CodeExecutor):
"""Python code runner with security isolation"""
def __init__(self):
super().__init__()
@staticmethod
def init_enviroment(code: bytes, preload, options: RunnerOptions) -> tuple[str, str]:
if not check_lib_avaiable():
release_lib_binary(False)
config = get_config()
code_file_name = uuid.uuid4().hex.replace("-", "_")
script = PYTHON_PRESCRIPT.replace("{{uid}}", str(SANDBOX_USER_ID), 1)
script = script.replace("{{gid}}", str(SANDBOX_GROUP_ID), 1)
script = script.replace(
"{{enable_network}}",
str(int(options.enable_network and config.enable_network)
),
1
)
script = script.replace("{{preload}}", f"{preload}\n", 1)
key = generate_key(64)
encoded_code = encrypt_code(code, key)
encoded_key = base64.b64encode(key).decode("utf-8")
script = script.replace("{{code}}", encoded_code, 1)
code_path = f"{LIB_PATH}/tmp/{code_file_name}.py"
try:
os.makedirs(os.path.dirname(code_path), mode=0o755, exist_ok=True)
with open(code_path, "w", encoding="utf-8") as f:
f.write(script)
os.chmod(code_path, 0o755)
except OSError as e:
raise RuntimeError(f"Failed to write {code_path}") from e
return code_path, encoded_key
async def run(
self,
code: str,
options: RunnerOptions,
preload: str = "",
timeout: Optional[int] = None
) -> ExecutionResult:
"""Run Python code in sandbox
Args:
options:
code: Base64 encoded encrypted code
preload: Preload code to execute before main code
timeout: Execution timeout in seconds
Returns:
ExecutionResult with stdout, stderr, and exit code
"""
config = self.config
if timeout is None:
timeout = config.worker_timeout
# Check if preload is allowed
if not config.enable_preload:
preload = ""
code = base64.b64decode(code)
script_path, encoded_key = self.init_enviroment(code, preload, options=options)
try:
# Setup environment
env = {}
# Add proxy settings if configured
if config.proxy.socks5:
env["HTTPS_PROXY"] = config.proxy.socks5
env["HTTP_PROXY"] = config.proxy.socks5
elif config.proxy.https or config.proxy.http:
if config.proxy.https:
env["HTTPS_PROXY"] = config.proxy.https
if config.proxy.http:
env["HTTP_PROXY"] = config.proxy.http
# Add allowed syscalls if configured
if config.allowed_syscalls:
env["ALLOWED_SYSCALLS"] = ",".join(map(str, config.allowed_syscalls))
# Execute with Python interpreter
logger.info(encoded_key)
process = await asyncio.create_subprocess_exec(
config.python_path,
script_path,
LIB_PATH,
encoded_key,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=env,
cwd=LIB_PATH
)
# Wait for completion with timeout
try:
stdout, stderr = await asyncio.wait_for(
process.communicate(),
timeout=timeout
)
return ExecutionResult(
stdout=stdout.decode('utf-8', errors='replace'),
stderr=stderr.decode('utf-8', errors='replace'),
exit_code=process.returncode
)
except asyncio.TimeoutError:
# Kill process on timeout
try:
process.kill()
await process.wait()
except:
pass
return ExecutionResult(
stdout="",
stderr="Execution timeout",
exit_code=-1,
)
finally:
# Cleanup temporary file
self.cleanup_temp_file(script_path)

View File

@@ -0,0 +1,62 @@
import os
from app.logger import get_logger
logger = get_logger()
RELEASE_LIB_PATH = "./lib/seccomp_python/target/release/libpython.so"
LIB_PATH = "/var/sandbox/sandbox-python"
LIB_NAME = "libpython.so"
try:
with open(RELEASE_LIB_PATH, "rb") as f:
_PYTHON_LIB = f.read()
except:
logger.critical("failed to load python lib")
raise
def check_lib_avaiable():
return os.path.exists(os.path.join(LIB_PATH, LIB_NAME))
def release_lib_binary(force_remove: bool):
logger.info("init runtime enviroment")
lib_file = os.path.join(LIB_PATH, LIB_NAME)
if os.path.exists(lib_file):
if force_remove:
try:
os.remove(lib_file)
except OSError:
logger.critical(f"failed to remove {os.path.join(LIB_PATH, LIB_NAME)}")
raise
try:
os.makedirs(LIB_PATH, mode=0o755, exist_ok=True)
except OSError:
logger.critical(f"failed to create {LIB_PATH}")
raise
try:
with open(lib_file, "wb") as f:
f.write(_PYTHON_LIB)
os.chmod(lib_file, 0o755)
except OSError:
logger.critical(f"failed to write {lib_file}")
raise
else:
try:
os.makedirs(LIB_PATH, mode=0o755, exist_ok=True)
except OSError:
logger.critical(f"failed to create {LIB_PATH}")
raise
try:
with open(lib_file, "wb") as f:
f.write(_PYTHON_LIB)
os.chmod(lib_file, 0o755)
except OSError:
logger.critical(f"failed to write {lib_file}")
raise
logger.info("python runner environment initialized")

161
sandbox/app/dependencies.py Normal file
View File

@@ -0,0 +1,161 @@
"""Dependency management"""
import asyncio
from pathlib import Path
from typing import List, Dict
from app.config import get_config
from app.core.runners.python.env import prepare_python_dependencies_env
from app.logger import get_logger
async def setup_dependencies():
"""Setup initial dependencies"""
logger = get_logger()
try:
logger.info("Installing Python dependencies...")
await install_python_dependencies()
logger.info("Python dependencies installed")
logger.info("Preparing Python dependencies environment...")
await prepare_python_dependencies_env()
logger.info("Python dependencies environment ready")
except Exception as e:
logger.error(f"Failed to setup dependencies: {e}")
async def update_dependencies():
# TODO
return
async def install_python_dependencies():
"""Install Python dependencies from requirements file"""
logger = get_logger()
config = get_config()
# Check if requirements file exists
req_file = Path("dependencies/python-requirements.txt")
if not req_file.exists():
logger.warning("Python requirements file not found, skipping installation")
return
# Read requirements
requirements = req_file.read_text().strip()
if not requirements:
logger.info("No Python requirements to install")
return
# Install using pip
cmd = [
config.python_path,
"-m",
"pip",
"install",
"--upgrade"
]
# Add packages from requirements
for line in requirements.split("\n"):
line = line.strip()
if line and not line.startswith("#"):
cmd.append(line)
try:
process = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE
)
stdout, stderr = await process.communicate()
if process.returncode != 0:
logger.error(f"Failed to install Python dependencies: {stderr.decode()}")
else:
logger.info("Python dependencies installed successfully")
except Exception as e:
logger.error(f"Error installing Python dependencies: {e}")
async def list_dependencies(language: str) -> List[Dict[str, str]]:
"""List installed dependencies
Args:
language: Language (python or Node.js)
Returns:
List of dependencies with name and version
"""
if language == "python":
return await list_python_packages()
else:
return []
async def list_python_packages() -> List[Dict[str, str]]:
"""List installed Python packages"""
config = get_config()
try:
process = await asyncio.create_subprocess_exec(
config.python_path,
"-m",
"pip",
"list",
"--format=freeze",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE
)
stdout, stderr = await process.communicate()
if process.returncode != 0:
return []
# Parse output
packages = []
for line in stdout.decode().split("\n"):
line = line.strip()
if line and "==" in line:
name, version = line.split("==", 1)
packages.append({"name": name, "version": version})
return packages
except Exception as e:
get_logger().error(f"Failed to list Python packages: {e}")
return []
async def update_dependencies_periodically():
"""Periodically update dependencies"""
logger = get_logger()
config = get_config()
# Parse interval
interval_str = config.python_deps_update_interval
# Convert to seconds
if interval_str.endswith("m"):
interval = int(interval_str[:-1]) * 60
elif interval_str.endswith("h"):
interval = int(interval_str[:-1]) * 3600
elif interval_str.endswith("s"):
interval = int(interval_str[:-1])
else:
interval = 1800 # Default 30 minutes
logger.info(f"Starting periodic dependency updates every {interval} seconds")
while True:
await asyncio.sleep(interval)
try:
logger.info("Updating Python dependencies...")
# TODO: await update_dependencies("python")
logger.info("Python dependencies updated successfully")
except Exception as e:
logger.error(f"Failed to update Python dependencies: {e}")

42
sandbox/app/logger.py Normal file
View File

@@ -0,0 +1,42 @@
"""Logging configuration"""
import logging
import sys
from typing import Optional
from app.config import get_config
_logger: Optional[logging.Logger] = None
def setup_logger() -> logging.Logger:
"""Setup application logger"""
global _logger
config = get_config()
# Create logger
_logger = logging.getLogger("sandbox")
_logger.setLevel(logging.DEBUG if config.app.debug else logging.INFO)
# Create console handler
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG if config.app.debug else logging.INFO)
# Create formatter
formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
handler.setFormatter(formatter)
# Add handler to logger
_logger.addHandler(handler)
return _logger
def get_logger() -> logging.Logger:
"""Get application logger"""
if _logger is None:
return setup_logger()
return _logger

View File

@@ -0,0 +1 @@
"""Middleware package"""

View File

@@ -0,0 +1,15 @@
"""Authentication middleware"""
from fastapi import Header, HTTPException, status
from app.config import get_config
async def verify_api_key(x_api_key: str = Header(..., alias="X-Api-Key")):
"""Verify API key from request header"""
config = get_config()
if x_api_key != config.app.key:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid API key"
)
return x_api_key

View File

@@ -0,0 +1,48 @@
"""Concurrency control middleware"""
import asyncio
from fastapi import HTTPException, status
from app.config import get_config
from app.models import error_response
# Global semaphores
_worker_semaphore: None | asyncio.Semaphore = None
_request_counter = 0
_request_lock = asyncio.Lock()
def init_concurrency_control():
"""Initialize concurrency control"""
global _worker_semaphore
config = get_config()
_worker_semaphore = asyncio.Semaphore(config.max_workers)
async def check_max_requests():
"""Check if max requests limit is reached"""
global _request_counter
config = get_config()
async with _request_lock:
if _request_counter >= config.max_requests:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=error_response(-503, "Too many requests")
)
_request_counter += 1
try:
yield
finally:
async with _request_lock:
_request_counter -= 1
async def acquire_worker():
"""Acquire a worker slot"""
if _worker_semaphore is None:
init_concurrency_control()
async with _worker_semaphore:
yield

80
sandbox/app/models.py Normal file
View File

@@ -0,0 +1,80 @@
"""Data models"""
from typing import Optional, Any
from pydantic import BaseModel, Field
class RunnerOptions(BaseModel):
enable_network: bool = Field(default=False, description="Sandbox network flag")
class RunCodeRequest(BaseModel):
"""Request model for code execution"""
language: str = Field(..., description="Programming language (python3 or nodejs)")
code: str = Field(..., description="Base64 encoded encrypted code")
preload: Optional[str] = Field(default="", description="Preload code")
options: RunnerOptions = Field(default_factory=RunnerOptions, description="Enable network access")
class RunCodeResponse(BaseModel):
"""Response model for code execution"""
stdout: str = Field(default="", description="Standard output")
stderr: str = Field(default="", description="Standard error")
class DependencyRequest(BaseModel):
"""Request model for dependency operations"""
language: str = Field(..., description="Programming language")
class UpdateDependencyRequest(BaseModel):
"""Request model for updating dependencies"""
language: str = Field(..., description="Programming language")
packages: list[str] = Field(default_factory=list, description="Packages to install")
class Dependency(BaseModel):
"""Dependency information"""
name: str
version: str
class ListDependenciesResponse(BaseModel):
"""Response model for listing dependencies"""
dependencies: list[Dependency] = Field(default_factory=list)
class RefreshDependenciesResponse(BaseModel):
"""Response model for refreshing dependencies"""
dependencies: list[Dependency] = Field(default_factory=list)
class UpdateDependenciesResponse(BaseModel):
"""Response model for updating dependencies"""
success: bool = True
installed: list[str] = Field(default_factory=list)
class HealthResponse(BaseModel):
"""Health check response"""
status: str = "healthy"
version: str = "2.0.0"
class ApiResponse(BaseModel):
"""Standard API response wrapper"""
code: int = Field(default=0, description="Response code (0 for success, negative for error)")
message: str = Field(default="success", description="Response message")
data: Optional[Any] = Field(default=None, description="Response data")
def success_response(data: Any) -> ApiResponse:
"""Create success response"""
return ApiResponse(code=0, message="success", data=data)
def error_response(code: int, message: str) -> ApiResponse:
"""Create error response"""
if code >= 0:
code = -1
return ApiResponse(code=code, message=message, data=None)

View File

@@ -0,0 +1 @@
"""Services package"""

View File

@@ -0,0 +1,80 @@
"""Python execution service"""
import signal
from app.core.runners.python.python_runner import PythonRunner
from app.dependencies import (
list_dependencies as list_deps,
update_dependencies as update_deps
)
from app.logger import get_logger
from app.models import (
success_response,
error_response,
RunCodeResponse,
ListDependenciesResponse,
UpdateDependenciesResponse,
Dependency,
RunnerOptions
)
async def run_python_code(code: str, preload: str, options: RunnerOptions):
"""Execute Python code in sandbox
Args:
options:
code: Base64 encoded encrypted code
preload: Preload code
Returns:
API response with execution result
"""
logger = get_logger()
try:
runner = PythonRunner()
result = await runner.run(code, options, preload)
if result.exit_code == -signal.SIGSYS:
return error_response(31, "sandbox security policy violation")
if result.stderr and result.exit_code != 0:
return error_response(500, result.stderr)
return success_response(RunCodeResponse(
stdout=result.stdout,
stderr=result.stderr
))
except Exception as e:
logger.error(f"Python execution failed: {e}", exc_info=True)
return error_response(-500, str(e))
async def list_python_dependencies():
"""List installed Python dependencies
Returns:
API response with dependency list
"""
try:
deps = await list_deps("python")
dependencies = [
Dependency(name=dep["name"], version=dep["version"])
for dep in deps
]
return success_response(ListDependenciesResponse(dependencies=dependencies))
except Exception as e:
return error_response(500, str(e))
async def update_python_dependencies():
"""Update Python dependencies
Returns:
API response with update result
"""
try:
await update_deps()
return success_response(UpdateDependenciesResponse(success=True))
except Exception as e:
return error_response(500, str(e))