mirror of
https://github.com/TeamWiseFlow/wiseflow.git
synced 2025-02-02 18:28:46 +08:00
55 lines
1.7 KiB
Markdown
55 lines
1.7 KiB
Markdown
# For Developer Only
|
||
|
||
```bash
|
||
conda create -n wiseflow python=3.10
|
||
conda activate wiseflow
|
||
cd core
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
- tasks.py background task circle process
|
||
- backend.py main process pipeline service (based on fastapi)
|
||
|
||
### WiseFlow fastapi detail
|
||
|
||
- api address http://127.0.0.1:8077/feed
|
||
- request method : post
|
||
- body :
|
||
|
||
```python
|
||
{'user_id': str, 'type': str, 'content':str, 'addition': Optional[str]}
|
||
# Type is one of "text", "publicMsg", "site" and "url"；
|
||
# user_id: str
|
||
type: Literal["text", "publicMsg", "file", "image", "video", "location", "chathistory", "site", "attachment", "url"]
|
||
content: str
|
||
addition: Optional[str] = None
|
||
```
|
||
|
||
see more (when backend started) http://127.0.0.1:8077/docs
|
||
|
||
### WiseFlow Repo File Structure
|
||
|
||
```
|
||
wiseflow
|
||
|- dockerfiles
|
||
|- ...
|
||
|- core
|
||
|- tasks.py
|
||
|- backend.py
|
||
|- insights
|
||
|- __init__.py # main process
|
||
|- get_info.py # module use llm to get a summary of information and match tags
|
||
|- llms # llm service wrapper
|
||
|- pb # pocketbase filefolder
|
||
|- scrapers
|
||
|- __init__.py # You can register a proprietary site scraper here
|
||
|- general_scraper.py # module to get all possible article urls for general site
|
||
|- general_crawler.py # module for general article sites
|
||
|- mp_crawler.py # module for mp article (weixin public account) sites
|
||
|- utils # tools
|
||
```
|
||
|
||
Although the general_scraper included in wiseflow can be applied to the parsing of most static pages, for actual business, we still recommend that customers to write their own crawlers aiming the actual info source.
|
||
|
||
See core/scrapers/README.md for integration instructions for proprietary crawlers
|