Compare commits

..

3 Commits

Author SHA1 Message Date
be0954828c 2025-12-2genxin 2025-12-02 14:58:52 +08:00
4fef65bd93 新版可用 2025-11-27 18:40:08 +08:00
46de43ce72 上传 frontend 文件夹 2025-11-27 18:32:24 +08:00
48 changed files with 44789 additions and 612 deletions

27
.gitignore vendored
View File

@@ -1,27 +0,0 @@
# ---> Go
# If you prefer the allow list template instead of the deny list, see community template:
# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
#
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Dependency directories (remove the comment below to include it)
# vendor/
# Go workspace file
go.work
go.work.sum
# env file
.env

211
README.md
View File

@@ -1,3 +1,210 @@
# yixiaogao # 🔍 易搜高 - 微信公众号文章爬虫系统
易小稿网站微信公众号文章爬 一个功能完整的微信公众号文章爬虫与自媒体监控系统,支持文章采集、数据分析、用户管理等功能。
![License](https://img.shields.io/badge/license-MIT-blue.svg)
![Go Version](https://img.shields.io/badge/go-1.24.0-blue.svg)
![Status](https://img.shields.io/badge/status-active-success.svg)
## ✨ 功能特性
### 📱 自媒体监控系统
- **文章监控**:实时追踪公众号、头条号、百家号等平台的最新文章
- **数据分析**:统计阅读量、点赞数、评论数、分享数等关键指标
- **历史文章**:浏览和管理已采集的文章列表
- **文章详情**:查看文章完整内容、统计数据和元信息
- **用户中心**:个人信息管理、收藏夹、浏览历史
### 🚀 爬虫系统
- **提取公众号主页**:从文章链接获取公众号主页
- **获取文章列表**:批量获取公众号所有历史文章
- **批量下载文章**:下载文章内容和图片资源
- **获取文章详情**:获取阅读量、点赞数、评论等详细数据
### 👥 用户管理
- **用户注册/登录**:完整的用户认证系统
- **登录状态同步**:所有页面统一的登录状态管理
- **安全认证**:基于 Token 的安全认证机制
## 🏗️ 项目架构
```
Access_wechat_article/
├── backend/ # 后端服务Go
│ ├── api/ # API 接口
│ ├── cmd/ # 命令行工具
│ ├── configs/ # 配置文件
│ ├── pkg/ # 核心包
│ └── tools/ # 工具函数
├── frontend/ # 前端页面HTML/CSS/JS
│ ├── css/ # 样式文件
│ ├── js/ # JavaScript 文件
│ ├── frontend.html # 自媒体监控首页
│ ├── index.html # 爬虫系统页面
│ ├── login.html # 登录页面
│ ├── register.html # 注册页面
│ ├── user-center.html # 用户中心
│ ├── history-articles.html # 历史文章列表
│ └── article-detail.html # 文章详情页
├── database/ # 数据库文件
└── data/ # 数据存储目录
```
## 🚀 快速开始
### 环境要求
- **Go**: 1.24.0 或更高版本
- **Python**: 3.x用于前端服务器
- **浏览器**: Chrome、Firefox、Edge 等现代浏览器
### 安装步骤
1. **克隆项目**
```bash
git clone <repository-url>
cd Access_wechat_article
```
2. **安装后端依赖**
```bash
cd backend
go mod download
```
3. **启动系统**
**方式一:使用启动脚本(推荐)**
```bash
# Windows 系统
双击运行 "启动Web系统.bat"
```
**方式二:手动启动**
```bash
# 启动 API 服务器
cd backend/api
go run start_api.go
# 新开终端,启动前端服务器
cd frontend
python -m http.server 8000
```
4. **访问系统**
- 前端界面http://localhost:8000
- API 服务器http://localhost:8080
## 📖 使用说明
### 1. 自媒体监控系统
访问 `http://localhost:8000/frontend.html`
- **首页**:查看最新文章列表和热门内容
- **历史文章**:浏览和搜索已采集的文章
- **文章详情**:查看文章完整内容和数据分析
- **用户中心**:管理个人信息和收藏
### 2. 爬虫系统
访问 `http://localhost:8000/index.html`
#### 提取公众号主页
1. 输入公众号下任意一篇文章链接
2. 点击"提取主页链接"
3. 获取公众号主页 URL
#### 获取文章列表
1. 使用 Fiddler 抓取包含认证信息的 URL
2. 粘贴完整的 Access Token URL
3. 设置获取页数(留空表示全部)
4. 点击"开始获取"
#### 批量下载文章
1. 输入公众号名称或文章链接
2. 选择是否保存图片和内容
3. 点击"开始批量下载"
#### 获取文章详情
1. 粘贴 Access Token URL
2. 设置获取页数
3. 点击"开始获取"
4. 数据保存在 `data/公众号名称/文章详细/` 目录
### 3. 用户系统
#### 注册账号
1. 访问注册页面:`http://localhost:8000/register.html`
2. 填写用户名、邮箱、密码
3. 点击"立即注册"
#### 登录
1. 访问登录页面:`http://localhost:8000/login.html`
2. 输入用户名和密码
3. 选择"记住我"可保持登录状态
4. 登录成功后自动跳转到首页
## 🛠️ 技术栈
### 后端
- **语言**: Go 1.24.0
- **HTTP 客户端**: go-resty/resty v2.17.0
- **数据库**: SQLite (modernc.org/sqlite v1.40.1)
### 前端
- **HTML5/CSS3**: 响应式布局
- **JavaScript**: 原生 JS + jQuery 3.6.0
- **UI 设计**: 渐变配色、卡片式布局、动画效果
## 📊 API 接口
### 用户相关
- `POST /api/user/register` - 用户注册
- `POST /api/user/login` - 用户登录
- `POST /api/user/logout` - 用户登出
- `GET /api/user/info` - 获取用户信息
### 文章相关
- `GET /api/articles` - 获取文章列表
- `GET /api/articles/:id` - 获取文章详情
- `POST /api/articles/collect` - 收藏文章
### 爬虫相关
- `POST /api/crawler/homepage` - 提取公众号主页
- `POST /api/crawler/articles` - 获取文章列表
- `POST /api/crawler/batch` - 批量下载文章
- `POST /api/crawler/detail` - 获取文章详情
## 📝 注意事项
1. **网络要求**:需要稳定的网络连接访问微信公众号平台
2. **认证信息**:部分功能需要有效的 Access Token
3. **数据存储**:采集的数据保存在 `data/` 目录下
4. **浏览器兼容**:建议使用 Chrome、Firefox、Edge 等现代浏览器
5. **服务器端口**:确保 8000 和 8080 端口未被占用
## 🔒 安全说明
- 用户密码经过加密存储
- 使用 Token 认证机制保护 API
- 所有敏感操作需要登录验证
- 支持会话管理和自动过期
## 📄 许可证
本项目仅供学习研究使用,请勿用于商业用途。
## 🤝 贡献
欢迎提交 Issue 和 Pull Request
## 📧 联系方式
如有问题或建议,请通过以下方式联系:
- 提交 Issue
- 发送邮件
---
**⚠️ 免责声明**:本工具仅供学习交流使用,使用者需遵守相关法律法规和平台规则,不得用于非法用途。使用本工具产生的任何后果由使用者自行承担。

View File

@@ -1,73 +0,0 @@
# 微信公众号文章爬取工具Go版本
这是一个基于Go语言开发的微信公众号文章爬取工具可以自动获取指定公众号的所有文章列表和详细内容。
## 功能特性
- 获取公众号所有文章列表
- 获取每篇文章的详细内容
- 获取文章的阅读量、点赞数、转发数等统计信息
- 支持获取文章评论
- 自动保存文章列表和详细内容
## 环境要求
- Go 1.20 或更高版本
- Windows 操作系统脚本已针对Windows优化
## 安装使用
### 1. 配置Cookie
-`cookie.txt.example` 重命名为 `cookie.txt`
- 按照文件中的说明获取微信公众平台的Cookie
- 将Cookie信息粘贴到 `cookie.txt` 文件中
### 2. 运行程序
直接双击 `run.bat` 脚本文件,程序会自动:
- 下载所需依赖
- 编译Go程序
- 运行爬取工具
## 项目结构
```
backend/
├── cmd/
│ └── main.go # 主程序入口
├── configs/
│ └── config.go # 配置管理
├── pkg/
│ ├── utils/ # 工具函数
│ │ └── utils.go
│ └── wechat/ # 微信相关功能实现
│ └── access_articles.go
├── data/ # 数据存储目录
├── cookie.txt # Cookie文件需要手动创建
├── go.mod # Go模块定义
├── run.bat # Windows启动脚本
└── README.md # 使用说明
```
## 注意事项
1. 使用本工具前,请确保您已获得相关公众号的访问权限
2. 请遵守相关法律法规,合理使用本工具
3. 频繁请求可能会触发微信的反爬虫机制,请控制爬取频率
4. 由于微信接口可能会变化,工具可能需要相应调整
## 常见问题
### Q: 获取Cookie失败怎么办
A: 请确保您已登录微信公众平台并且在开发者工具中正确复制了完整的Cookie信息。
### Q: 爬取过程中出现网络错误怎么办?
A: 工具会自动处理简单的网络错误,请确保网络连接正常。如果持续失败,可能是微信接口发生了变化。
### Q: 如何修改爬取的公众号?
A: 工具会自动从Cookie中获取当前登录用户可访问的公众号信息。如果需要爬取不同的公众号请在微信公众平台中切换账号后重新获取Cookie。
## 许可证
本项目仅供学习和研究使用。

BIN
backend/api/api-server.exe Normal file

Binary file not shown.

BIN
backend/api/api-server.exe~ Normal file

Binary file not shown.

26
backend/api/build.bat Normal file
View File

@@ -0,0 +1,26 @@
@echo off
chcp 65001 >nul
echo ===============================================
echo 📦 编译 API 服务器
echo ===============================================
echo.
echo 🔨 正在编译 api-server.exe...
go build -o api-server.exe server.go
if %errorlevel% neq 0 (
echo.
echo ❌ 编译失败!
echo.
pause
exit /b 1
)
echo.
echo ✅ 编译成功!
echo 📁 输出文件: api-server.exe
echo.
echo ===============================================
echo 编译完成
echo ===============================================
pause

File diff suppressed because one or more lines are too long

1225
backend/api/server.go Normal file

File diff suppressed because it is too large Load Diff

23
backend/api/start_api.bat Normal file
View File

@@ -0,0 +1,23 @@
@echo off
chcp 65001 >nul
title 微信公众号文章爬虫 - API服务器
:: 检查api-server.exe是否存在
if not exist "api-server.exe" (
echo ===============================================
echo ⚠️ API服务器未编译
echo ===============================================
echo.
echo 正在编译 API 服务器...
echo.
call build.bat
if %errorlevel% neq 0 (
echo 编译失败,无法启动服务器
pause
exit /b 1
)
)
:: 启动API服务器
cls
api-server.exe

View File

@@ -0,0 +1,11 @@
序号,创建时间,标题,链接
1,0,专家分析2026年考研报名人数,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500657&idx=1&sn=81eae7df4bfa2fdfc8bca69389489c52&chksm=ea981e22044aca6bbe5633849bfcd4903cb6f491646cd2ccf9321f4d9c852c64fe036f033c14&scene=27#wechat_redirect
2,0,教育部2026年全国硕士研究生报名人数为343万,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500650&idx=1&sn=9f230bbfefb24d98c18e42bd3651ad53&chksm=eac72972d56ff9b66f3658f0c3b1e6e363e56ddf879d56aba9c9c8f587b53ef00bcabe7992ff&scene=27#wechat_redirect
3,0,【小研来了】“务必再坚持坚持”,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500645&idx=1&sn=8e1d5921861dc4e3647f7bf8adaada81&chksm=ea26b17ce2f7255aacd9d1d6358c9aeb8d4e043c692efb8b4d8183cfc8363b3068be79d585c2&scene=27#wechat_redirect
4,0,学累了不点进来看看这4个“续航”方法,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500631&idx=1&sn=b640b0e43378e368166e50a7f46735f2&chksm=ea71f10a83b7811e1896cd9704eac5d064b763f3e020b5b37c72727c55bb1b0862a92e9c4cf0&scene=27#wechat_redirect
5,0,教育部:在“双一流”建设高校开展科技教育硕士培养,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500589&idx=1&sn=539d1229c9475ba5a2371698a362e9a7&chksm=ea4f97d3831139a276e50050f2f3307868b9c6ec7eb115bb9e288312f08572c47128a8016dce&scene=27#wechat_redirect
6,0,“研味儿”正浓,冲刺在即!请你一定别放弃,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500584&idx=1&sn=294b6ba8d12f0948913abf04af8cb188&chksm=ea4cfb5b16684bdd12634b6e46d8d8f3ab72ca9108be0d4d7f83dfded09c6ecb9f31b1531e31&scene=27#wechat_redirect
7,0,4个思维升级让我找回了读研的掌控感,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500579&idx=1&sn=fa00084c8711e3009ff7e31fe0b3bc51&chksm=eaff1ec212ddbb738d20542a965bbd1b79ae3a9d2e5af5704ddcf41de3a8b8d658e562771f0c&scene=27#wechat_redirect
8,0,研考网上确认成功后,需重点关注四件事,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500569&idx=1&sn=7707b698932ff6847de39d7351d3ac98&chksm=ea402eec6a96125a5bb02600aff24c3c1211eb5aaf5347080bbfe1f5861e9ca97fe9c400df21&scene=27#wechat_redirect
9,0,,
10,0,【小研来了】“小研,没有准考证照片怎么办?”,http://mp.weixin.qq.com/s?__biz=MzI3NzQzODQ5OA==&mid=2247500553&idx=1&sn=4fc6fd69684f02222e72d457c1004a81&chksm=eafc91ea346080790f9b641495fc3d9e31302ee5c2c9957eb4fa2bc9a139eda78163899b9219&scene=27#wechat_redirect

View File

@@ -4,6 +4,7 @@ import (
"fmt" "fmt"
"io/ioutil" "io/ioutil"
"log" "log"
"net/url"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
@@ -204,7 +205,6 @@ func startCrawling(cfg *configs.Config) {
ReadCount: stats["read_num"], ReadCount: stats["read_num"],
LikeCount: stats["old_like_num"], LikeCount: stats["old_like_num"],
ShareCount: stats["share_num"], ShareCount: stats["share_num"],
ShowRead: stats["show_read"],
Comments: comments, Comments: comments,
CommentLikes: commentLikes, CommentLikes: commentLikes,
CommentID: commentID, CommentID: commentID,
@@ -600,21 +600,48 @@ func parseAccessTokenParams(accessToken string) (string, string, string, string,
if err != nil { if err != nil {
return "", "", "", "", fmt.Errorf("未找到__biz参数") return "", "", "", "", fmt.Errorf("未找到__biz参数")
} }
// URL解码biz参数
biz, err = url.QueryUnescape(biz)
if err != nil {
fmt.Printf("警告: URL解码__biz失败: %v使用原始值\n", err)
}
uin, err := utils.ExtractFromRegex(accessToken, "uin=([^&]*)") uin, err := utils.ExtractFromRegex(accessToken, "uin=([^&]*)")
if err != nil { if err != nil {
return "", "", "", "", fmt.Errorf("未找到uin参数") return "", "", "", "", fmt.Errorf("未找到uin参数")
} }
// URL解码uin参数
uin, err = url.QueryUnescape(uin)
if err != nil {
fmt.Printf("警告: URL解码uin失败: %v使用原始值\n", err)
}
key, err := utils.ExtractFromRegex(accessToken, "key=([^&]*)") key, err := utils.ExtractFromRegex(accessToken, "key=([^&]*)")
if err != nil { if err != nil {
return "", "", "", "", fmt.Errorf("未找到key参数") return "", "", "", "", fmt.Errorf("未找到key参数")
} }
// URL解码key参数
key, err = url.QueryUnescape(key)
if err != nil {
fmt.Printf("警告: URL解码key失败: %v使用原始值\n", err)
}
passTicket, err := utils.ExtractFromRegex(accessToken, "pass_ticket=([^&]*)") passTicket, err := utils.ExtractFromRegex(accessToken, "pass_ticket=([^&]*)")
if err != nil { if err != nil {
return "", "", "", "", fmt.Errorf("未找到pass_ticket参数") return "", "", "", "", fmt.Errorf("未找到pass_ticket参数")
} }
// URL解码pass_ticket参数
passTicket, err = url.QueryUnescape(passTicket)
if err != nil {
fmt.Printf("警告: URL解码pass_ticket失败: %v使用原始值\n", err)
}
// 打印解码后的参数用于调试
fmt.Printf("\n提取到的参数已解码\n")
fmt.Printf(" __biz: %s\n", biz)
fmt.Printf(" uin: %s\n", uin)
fmt.Printf(" key长度: %d 字符\n", len(key))
fmt.Printf(" pass_ticket长度: %d 字符\n", len(passTicket))
return biz, uin, key, passTicket, nil return biz, uin, key, passTicket, nil
} }

View File

@@ -1,27 +0,0 @@
package main
import (
"fmt"
"os"
"github.com/wechat-crawler/pkg/wechat"
)
func main() {
fmt.Println("开始测试文章内容提取功能...")
// 创建一个简单的爬虫实例
crawler := wechat.NewSimpleCrawler()
// 设置公众号名称(根据实际情况修改)
officialAccountName := "验证"
// 调用GetListArticleFromFile函数测试
err := crawler.GetListArticleFromFile(officialAccountName, false, true)
if err != nil {
fmt.Printf("测试失败: %v\n", err)
os.Exit(1)
}
fmt.Println("测试完成!请检查文章内容是否已正确提取。")
}

View File

@@ -1 +0,0 @@
__biz=MzUxMjA4MTI0MjA1; uin=MTIzNDU2Nzg5; key=abcdef1234567890abcdef1234567890; pass_ticket=abcdefghijklmnopqrstuvwxyz1234567890; version=63090b13; wxtype=1; pass_ticket=abcdefghijklmnopqrstuvwxyz1234567890;

View File

@@ -1,12 +0,0 @@
请将此文件重命名为cookie.txt并填入微信公众平台的cookie信息
如何获取cookie:
1. 打开浏览器,登录微信公众平台
2. 按F12打开开发者工具
3. 切换到Network标签
4. 刷新页面或访问任意页面
5. 选择一个请求查看Headers中的Cookie
6. 复制完整的Cookie到本文件中
Cookie格式示例:
__biz=MzUxMjA4MTI0MjA1; uin=MTIzNDU2Nzg5; key=abcdef1234567890abcdef1234567890; pass_ticket=abcdefghijklmnopqrstuvwxyz1234567890; version=63090b13; wxtype=1; pass_ticket=abcdefghijklmnopqrstuvwxyz1234567890;

View File

@@ -0,0 +1,246 @@
package main
import (
"encoding/json"
"fmt"
"log"
"github.com/wechat-crawler/pkg/database"
)
func main() {
fmt.Println("==============================================")
fmt.Println(" 微信公众号文章数据库管理系统示例")
fmt.Println("==============================================\n")
// 1. 初始化数据库
db, err := database.InitDB("../data/wechat_articles.db")
if err != nil {
log.Fatal("数据库初始化失败:", err)
}
defer db.Close()
// 2. 创建仓库实例
officialRepo := database.NewOfficialAccountRepository(db)
articleRepo := database.NewArticleRepository(db)
contentRepo := database.NewArticleContentRepository(db)
// 3. 示例:添加公众号
fmt.Println("📝 示例1: 添加公众号信息")
official := &database.OfficialAccount{
Biz: "MzI1NjEwMTM4OA==",
Nickname: "研招网资讯",
Homepage: "https://mp.weixin.qq.com/mp/profile_ext?action=home&__biz=MzI1NjEwMTM4OA==&scene=124",
Description: "中国研究生招生信息网官方公众号",
}
// 检查是否已存在
existing, err := officialRepo.GetByBiz(official.Biz)
if err != nil {
log.Fatal("查询公众号失败:", err)
}
var officialID int64
if existing == nil {
// 不存在,创建新记录
officialID, err = officialRepo.Create(official)
if err != nil {
log.Fatal("创建公众号失败:", err)
}
fmt.Printf("✅ 成功创建公众号: %s (ID: %d)\n\n", official.Nickname, officialID)
} else {
// 已存在
officialID = existing.ID
fmt.Printf(" 公众号已存在: %s (ID: %d)\n\n", existing.Nickname, officialID)
}
// 4. 示例:添加文章
fmt.Println("📝 示例2: 添加文章信息")
article := &database.Article{
OfficialID: officialID,
Title: "专家分析2026年考研报名人数",
Author: "研招网资讯",
Link: "https://mp.weixin.qq.com/s?__biz=MzI1NjEwMTM4OA==&mid=2651232405&idx=1",
PublishTime: "2024-11-27 10:00:00",
CreateTime: "2024-11-27 15:30:00",
CommentID: "2247491372",
ReadNum: 15234,
LikeNum: 456,
ShareNum: 123,
ContentPreview: "根据最新统计数据显示2026年全国硕士研究生报名人数预计将达到新高...",
ParagraphCount: 15,
}
// 检查文章是否已存在
existingArticle, err := articleRepo.GetByLink(article.Link)
if err != nil {
log.Fatal("查询文章失败:", err)
}
var articleID int64
if existingArticle == nil {
articleID, err = articleRepo.Create(article)
if err != nil {
log.Fatal("创建文章失败:", err)
}
fmt.Printf("✅ 成功创建文章: %s (ID: %d)\n\n", article.Title, articleID)
} else {
articleID = existingArticle.ID
fmt.Printf(" 文章已存在: %s (ID: %d)\n\n", existingArticle.Title, articleID)
}
// 5. 示例:添加文章内容
fmt.Println("📝 示例3: 添加文章详细内容")
paragraphs := []string{
"根据最新统计数据显示2026年全国硕士研究生报名人数预计将达到新高。",
"教育部相关负责人表示,随着社会对高层次人才需求的增加,考研热度持续上升。",
"专家建议考生理性选择,注重提升自身综合素质。",
}
images := []string{
"https://mmbiz.qpic.cn/mmbiz_jpg/xxx1.jpg",
"https://mmbiz.qpic.cn/mmbiz_jpg/xxx2.jpg",
}
content := &database.ArticleContent{
ArticleID: articleID,
HtmlContent: "<div>文章HTML内容</div>",
TextContent: "文章纯文本内容...",
Paragraphs: database.StringsToJSON(paragraphs),
Images: database.StringsToJSON(images),
}
// 检查内容是否已存在
existingContent, err := contentRepo.GetByArticleID(articleID)
if err != nil {
log.Fatal("查询文章内容失败:", err)
}
if existingContent == nil {
contentID, err := contentRepo.Create(content)
if err != nil {
log.Fatal("创建文章内容失败:", err)
}
fmt.Printf("✅ 成功添加文章内容 (ID: %d)\n\n", contentID)
} else {
fmt.Printf(" 文章内容已存在 (ID: %d)\n\n", existingContent.ID)
}
// 6. 示例:查询文章列表
fmt.Println("📋 示例4: 查询文章列表")
articles, total, err := articleRepo.List(officialID, 1, 10)
if err != nil {
log.Fatal("查询文章列表失败:", err)
}
fmt.Printf("共找到 %d 篇文章:\n", total)
for i, item := range articles {
fmt.Printf("%d. %s (👁️ %d | 👍 %d)\n", i+1, item.Title, item.ReadNum, item.LikeNum)
}
fmt.Println()
// 7. 示例:获取文章详情
fmt.Println("📖 示例5: 获取文章详情")
detail, err := contentRepo.GetArticleDetail(articleID)
if err != nil {
log.Fatal("获取文章详情失败:", err)
}
if detail != nil {
fmt.Printf("标题: %s\n", detail.Title)
fmt.Printf("作者: %s\n", detail.Author)
fmt.Printf("公众号: %s\n", detail.OfficialName)
fmt.Printf("发布时间: %s\n", detail.PublishTime)
fmt.Printf("阅读数: %d | 点赞数: %d\n", detail.ReadNum, detail.LikeNum)
fmt.Printf("段落数: %d\n", len(detail.Paragraphs))
fmt.Printf("图片数: %d\n", len(detail.Images))
if len(detail.Paragraphs) > 0 {
fmt.Printf("第一段: %s\n", detail.Paragraphs[0])
}
}
fmt.Println()
// 8. 示例:搜索文章
fmt.Println("🔍 示例6: 搜索文章")
searchResults, searchTotal, err := articleRepo.Search("考研", 1, 10)
if err != nil {
log.Fatal("搜索文章失败:", err)
}
fmt.Printf("搜索\"考研\"找到 %d 篇文章:\n", searchTotal)
for i, item := range searchResults {
fmt.Printf("%d. %s\n", i+1, item.Title)
}
fmt.Println()
// 9. 示例:获取统计信息
fmt.Println("📊 示例7: 获取统计信息")
stats, err := db.GetStatistics()
if err != nil {
log.Fatal("获取统计信息失败:", err)
}
fmt.Printf("公众号总数: %d\n", stats.TotalOfficials)
fmt.Printf("文章总数: %d\n", stats.TotalArticles)
fmt.Printf("总阅读数: %d\n", stats.TotalReadNum)
fmt.Printf("总点赞数: %d\n", stats.TotalLikeNum)
fmt.Println()
// 10. 示例:批量插入文章
fmt.Println("📦 示例8: 批量插入文章")
batchArticles := []*database.Article{
{
OfficialID: officialID,
Title: "教育部2026年全国硕士研究生报名人数为343万",
Author: "研招网资讯",
Link: "https://mp.weixin.qq.com/s?__biz=MzI1NjEwMTM4OA==&mid=2651232406",
PublishTime: "2024-11-26 09:00:00",
ReadNum: 8965,
LikeNum: 234,
ContentPreview: "教育部公布2026年研究生招生数据...",
ParagraphCount: 12,
},
{
OfficialID: officialID,
Title: "研考网上确认成功后,需重点关注四件事",
Author: "研招网资讯",
Link: "https://mp.weixin.qq.com/s?__biz=MzI1NjEwMTM4OA==&mid=2651232407",
PublishTime: "2024-11-25 15:30:00",
ReadNum: 6543,
LikeNum: 189,
ContentPreview: "网上确认通过后,考生还需要注意以下事项...",
ParagraphCount: 8,
},
}
err = articleRepo.BatchInsertArticles(batchArticles)
if err != nil {
log.Fatal("批量插入文章失败:", err)
}
fmt.Printf("✅ 成功批量插入 %d 篇文章\n\n", len(batchArticles))
// 11. 示例导出JSON数据
fmt.Println("💾 示例9: 导出文章列表为JSON")
allArticles, _, err := articleRepo.List(0, 1, 100)
if err != nil {
log.Fatal("查询文章列表失败:", err)
}
jsonData, err := json.MarshalIndent(allArticles, "", " ")
if err != nil {
log.Fatal("JSON序列化失败:", err)
}
fmt.Println("文章列表JSON (前200字符):")
if len(jsonData) > 200 {
fmt.Println(string(jsonData[:200]) + "...")
} else {
fmt.Println(string(jsonData))
}
fmt.Println()
fmt.Println("==============================================")
fmt.Println(" 数据库操作示例演示完成!")
fmt.Println("==============================================")
}

View File

@@ -1,7 +1,22 @@
module github.com/wechat-crawler module github.com/wechat-crawler
go 1.20 go 1.24.0
require github.com/go-resty/resty/v2 v2.10.0 require (
github.com/go-resty/resty/v2 v2.17.0
modernc.org/sqlite v1.40.1
)
require golang.org/x/net v0.17.0 // indirect require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
golang.org/x/net v0.43.0 // indirect
golang.org/x/sys v0.36.0 // indirect
modernc.org/libc v1.66.10 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)

View File

@@ -1,44 +1,57 @@
github.com/go-resty/resty/v2 v2.10.0 h1:Qla4W/+TMmv0fOeeRqzEpXPLfTUnR5HZ1+lGs+CkiCo= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/go-resty/resty/v2 v2.10.0/go.mod h1:iiP/OpA0CkcL3IGt1O0+/SIItFUbkkyw5BGXiVdTu+A= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= github.com/go-resty/resty/v2 v2.17.0 h1:pW9DeXcaL4Rrym4EZ8v7L19zZiIlWPg5YXAcVmt+gN0=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= github.com/go-resty/resty/v2 v2.17.0/go.mod h1:kCKZ3wWmwJaNc7S29BRtUhJwy7iqmn+2mLtQrOyQlVA=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4= github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg= github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b h1:M2rDM6z3Fhozi9O7NWsxAkg/yqS/lQJ6PmkyIV3YP+o=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b/go.mod h1:3//PLf8L/X+8b4vuAfHzxeRUl04Adcb341+IGKfnqS8=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/mod v0.27.0 h1:kb+q2PyFnEADO2IEF935ehFUXlWiNjJWtRNgBLSfbxQ=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/mod v0.27.0/go.mod h1:rWI627Fq0DEoudcK+MBkNkCe0EetEaDSwJJkCcjpazc=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo= golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U= golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= modernc.org/cc/v4 v4.26.5 h1:xM3bX7Mve6G8K8b+T11ReenJOT+BmVqQj0FY5T4+5Y4=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= modernc.org/cc/v4 v4.26.5/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= modernc.org/ccgo/v4 v4.28.1 h1:wPKYn5EC/mYTqBO373jKjvX2n+3+aK7+sICCv4Fjy1A=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= modernc.org/ccgo/v4 v4.28.1/go.mod h1:uD+4RnfrVgE6ec9NGguUNdhqzNIeeomeXf6CL0GTE5Q=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= modernc.org/fileutil v1.3.40 h1:ZGMswMNc9JOCrcrakF1HrvmergNLAmxOPjizirpfqBA=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE= modernc.org/fileutil v1.3.40/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4= modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= modernc.org/libc v1.66.10 h1:yZkb3YeLx4oynyR+iUsXsybsX4Ubx7MQlSYEw4yj59A=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= modernc.org/libc v1.66.10/go.mod h1:8vGSEwvoUoltr4dlywvHqjtAqHBaw0j1jI7iFBTAr2I=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.40.1 h1:VfuXcxcUWWKRBuP8+BR9L7VnmusMgBNNnBYGEe9w/iY=
modernc.org/sqlite v1.40.1/go.mod h1:9fjQZ0mB1LLP0GYrp39oOJXx/I2sxEnZtzCmEQIKvGE=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=

Binary file not shown.

117
backend/pkg/database/db.go Normal file
View File

@@ -0,0 +1,117 @@
package database
import (
"database/sql"
"fmt"
"os"
"path/filepath"
_ "modernc.org/sqlite"
)
// DB 数据库实例
type DB struct {
*sql.DB
}
// InitDB 初始化数据库
func InitDB(dbPath string) (*DB, error) {
// 确保数据库目录存在
dbDir := filepath.Dir(dbPath)
if err := os.MkdirAll(dbDir, 0755); err != nil {
return nil, fmt.Errorf("创建数据库目录失败: %v", err)
}
// 打开数据库连接使用modernc.org/sqlite驱动
db, err := sql.Open("sqlite", dbPath)
if err != nil {
return nil, fmt.Errorf("打开数据库失败: %v", err)
}
// 测试连接
if err := db.Ping(); err != nil {
return nil, fmt.Errorf("数据库连接测试失败: %v", err)
}
// 创建表
if err := createTables(db); err != nil {
return nil, fmt.Errorf("创建数据表失败: %v", err)
}
fmt.Println("✅ 数据库初始化成功:", dbPath)
return &DB{db}, nil
}
// createTables 创建数据表
func createTables(db *sql.DB) error {
// 公众号表
officialAccountTable := `
CREATE TABLE IF NOT EXISTS official_accounts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
biz TEXT NOT NULL UNIQUE,
nickname TEXT NOT NULL,
homepage TEXT,
description TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_biz ON official_accounts(biz);
CREATE INDEX IF NOT EXISTS idx_nickname ON official_accounts(nickname);
`
// 文章表
articleTable := `
CREATE TABLE IF NOT EXISTS articles (
id INTEGER PRIMARY KEY AUTOINCREMENT,
official_id INTEGER NOT NULL,
title TEXT NOT NULL,
author TEXT,
link TEXT UNIQUE,
publish_time TEXT,
create_time TEXT,
comment_id TEXT,
read_num INTEGER DEFAULT 0,
like_num INTEGER DEFAULT 0,
share_num INTEGER DEFAULT 0,
content_preview TEXT,
paragraph_count INTEGER DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (official_id) REFERENCES official_accounts(id)
);
CREATE INDEX IF NOT EXISTS idx_official_id ON articles(official_id);
CREATE INDEX IF NOT EXISTS idx_title ON articles(title);
CREATE INDEX IF NOT EXISTS idx_publish_time ON articles(publish_time);
CREATE INDEX IF NOT EXISTS idx_link ON articles(link);
`
// 文章内容表
articleContentTable := `
CREATE TABLE IF NOT EXISTS article_contents (
id INTEGER PRIMARY KEY AUTOINCREMENT,
article_id INTEGER NOT NULL UNIQUE,
html_content TEXT,
text_content TEXT,
paragraphs TEXT,
images TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (article_id) REFERENCES articles(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_article_id ON article_contents(article_id);
`
// 执行创建表语句
tables := []string{officialAccountTable, articleTable, articleContentTable}
for _, table := range tables {
if _, err := db.Exec(table); err != nil {
return err
}
}
return nil
}
// Close 关闭数据库连接
func (db *DB) Close() error {
return db.DB.Close()
}

View File

@@ -0,0 +1,76 @@
package database
import (
"time"
)
// OfficialAccount 公众号信息
type OfficialAccount struct {
ID int64 `json:"id"`
Biz string `json:"biz"` // 公众号唯一标识
Nickname string `json:"nickname"` // 公众号名称
Homepage string `json:"homepage"` // 公众号主页链接
Description string `json:"description"` // 公众号描述
CreatedAt time.Time `json:"created_at"` // 创建时间
UpdatedAt time.Time `json:"updated_at"` // 更新时间
}
// Article 文章信息
type Article struct {
ID int64 `json:"id"`
OfficialID int64 `json:"official_id"` // 关联的公众号ID
Title string `json:"title"` // 文章标题
Author string `json:"author"` // 作者
Link string `json:"link"` // 文章链接
PublishTime string `json:"publish_time"` // 发布时间
CreateTime string `json:"create_time"` // 创建时间(抓取时间)
CommentID string `json:"comment_id"` // 评论ID
ReadNum int `json:"read_num"` // 阅读数
LikeNum int `json:"like_num"` // 点赞数
ShareNum int `json:"share_num"` // 分享数
ContentPreview string `json:"content_preview"` // 内容预览前200字
ParagraphCount int `json:"paragraph_count"` // 段落数
CreatedAt time.Time `json:"created_at"` // 数据库创建时间
UpdatedAt time.Time `json:"updated_at"` // 数据库更新时间
}
// ArticleContent 文章详细内容
type ArticleContent struct {
ID int64 `json:"id"`
ArticleID int64 `json:"article_id"` // 关联的文章ID
HtmlContent string `json:"html_content"` // HTML原始内容
TextContent string `json:"text_content"` // 纯文本内容
Paragraphs string `json:"paragraphs"` // 段落内容JSON数组
Images string `json:"images"` // 图片链接JSON数组
CreatedAt time.Time `json:"created_at"` // 创建时间
}
// ArticleListItem 文章列表项用于API返回
type ArticleListItem struct {
ID int64 `json:"id"`
Title string `json:"title"`
Author string `json:"author"`
PublishTime string `json:"publish_time"`
ReadNum int `json:"read_num"`
LikeNum int `json:"like_num"`
OfficialName string `json:"official_name"`
ContentPreview string `json:"content_preview"`
}
// ArticleDetail 文章详情用于API返回
type ArticleDetail struct {
Article
OfficialName string `json:"official_name"`
HtmlContent string `json:"html_content"`
TextContent string `json:"text_content"`
Paragraphs []string `json:"paragraphs"`
Images []string `json:"images"`
}
// Statistics 统计信息
type Statistics struct {
TotalOfficials int `json:"total_officials"` // 公众号总数
TotalArticles int `json:"total_articles"` // 文章总数
TotalReadNum int `json:"total_read_num"` // 总阅读数
TotalLikeNum int `json:"total_like_num"` // 总点赞数
}

View File

@@ -0,0 +1,455 @@
package database
import (
"database/sql"
"encoding/json"
"fmt"
"strings"
)
// OfficialAccountRepository 公众号数据仓库
type OfficialAccountRepository struct {
db *DB
}
// NewOfficialAccountRepository 创建公众号仓库
func NewOfficialAccountRepository(db *DB) *OfficialAccountRepository {
return &OfficialAccountRepository{db: db}
}
// Create 创建公众号
func (r *OfficialAccountRepository) Create(account *OfficialAccount) (int64, error) {
result, err := r.db.Exec(`
INSERT INTO official_accounts (biz, nickname, homepage, description)
VALUES (?, ?, ?, ?)
`, account.Biz, account.Nickname, account.Homepage, account.Description)
if err != nil {
return 0, err
}
return result.LastInsertId()
}
// GetByBiz 根据Biz获取公众号
func (r *OfficialAccountRepository) GetByBiz(biz string) (*OfficialAccount, error) {
account := &OfficialAccount{}
err := r.db.QueryRow(`
SELECT id, biz, nickname, homepage, description, created_at, updated_at
FROM official_accounts WHERE biz = ?
`, biz).Scan(&account.ID, &account.Biz, &account.Nickname, &account.Homepage,
&account.Description, &account.CreatedAt, &account.UpdatedAt)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return account, nil
}
// GetByID 根据ID获取公众号
func (r *OfficialAccountRepository) GetByID(id int64) (*OfficialAccount, error) {
account := &OfficialAccount{}
err := r.db.QueryRow(`
SELECT id, biz, nickname, homepage, description, created_at, updated_at
FROM official_accounts WHERE id = ?
`, id).Scan(&account.ID, &account.Biz, &account.Nickname, &account.Homepage,
&account.Description, &account.CreatedAt, &account.UpdatedAt)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return account, nil
}
// List 获取所有公众号列表
func (r *OfficialAccountRepository) List() ([]*OfficialAccount, error) {
rows, err := r.db.Query(`
SELECT id, biz, nickname, homepage, description, created_at, updated_at
FROM official_accounts ORDER BY created_at DESC
`)
if err != nil {
return nil, err
}
defer rows.Close()
var accounts []*OfficialAccount
for rows.Next() {
account := &OfficialAccount{}
err := rows.Scan(&account.ID, &account.Biz, &account.Nickname, &account.Homepage,
&account.Description, &account.CreatedAt, &account.UpdatedAt)
if err != nil {
return nil, err
}
accounts = append(accounts, account)
}
return accounts, nil
}
// Update 更新公众号信息
func (r *OfficialAccountRepository) Update(account *OfficialAccount) error {
_, err := r.db.Exec(`
UPDATE official_accounts
SET nickname = ?, homepage = ?, description = ?, updated_at = CURRENT_TIMESTAMP
WHERE id = ?
`, account.Nickname, account.Homepage, account.Description, account.ID)
return err
}
// ArticleRepository 文章数据仓库
type ArticleRepository struct {
db *DB
}
// NewArticleRepository 创建文章仓库
func NewArticleRepository(db *DB) *ArticleRepository {
return &ArticleRepository{db: db}
}
// Create 创建文章
func (r *ArticleRepository) Create(article *Article) (int64, error) {
result, err := r.db.Exec(`
INSERT INTO articles (
official_id, title, author, link, publish_time, create_time,
comment_id, read_num, like_num, share_num, content_preview, paragraph_count
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`, article.OfficialID, article.Title, article.Author, article.Link,
article.PublishTime, article.CreateTime, article.CommentID,
article.ReadNum, article.LikeNum, article.ShareNum,
article.ContentPreview, article.ParagraphCount)
if err != nil {
return 0, err
}
return result.LastInsertId()
}
// GetByID 根据ID获取文章
func (r *ArticleRepository) GetByID(id int64) (*Article, error) {
article := &Article{}
err := r.db.QueryRow(`
SELECT id, official_id, title, author, link, publish_time, create_time,
comment_id, read_num, like_num, share_num, content_preview,
paragraph_count, created_at, updated_at
FROM articles WHERE id = ?
`, id).Scan(&article.ID, &article.OfficialID, &article.Title, &article.Author,
&article.Link, &article.PublishTime, &article.CreateTime, &article.CommentID,
&article.ReadNum, &article.LikeNum, &article.ShareNum, &article.ContentPreview,
&article.ParagraphCount, &article.CreatedAt, &article.UpdatedAt)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return article, nil
}
// GetByLink 根据链接获取文章
func (r *ArticleRepository) GetByLink(link string) (*Article, error) {
article := &Article{}
err := r.db.QueryRow(`
SELECT id, official_id, title, author, link, publish_time, create_time,
comment_id, read_num, like_num, share_num, content_preview,
paragraph_count, created_at, updated_at
FROM articles WHERE link = ?
`, link).Scan(&article.ID, &article.OfficialID, &article.Title, &article.Author,
&article.Link, &article.PublishTime, &article.CreateTime, &article.CommentID,
&article.ReadNum, &article.LikeNum, &article.ShareNum, &article.ContentPreview,
&article.ParagraphCount, &article.CreatedAt, &article.UpdatedAt)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return article, nil
}
// List 获取文章列表(分页)
func (r *ArticleRepository) List(officialID int64, page, pageSize int) ([]*ArticleListItem, int, error) {
// 构建查询条件
whereClause := ""
args := []interface{}{}
if officialID > 0 {
whereClause = "WHERE a.official_id = ?"
args = append(args, officialID)
}
// 获取总数
countQuery := fmt.Sprintf("SELECT COUNT(*) FROM articles a %s", whereClause)
var total int
err := r.db.QueryRow(countQuery, args...).Scan(&total)
if err != nil {
return nil, 0, err
}
// 获取列表
offset := (page - 1) * pageSize
listQuery := fmt.Sprintf(`
SELECT a.id, a.title, a.author, a.publish_time, a.read_num, a.like_num,
a.content_preview, o.nickname
FROM articles a
LEFT JOIN official_accounts o ON a.official_id = o.id
%s
ORDER BY a.publish_time DESC
LIMIT ? OFFSET ?
`, whereClause)
args = append(args, pageSize, offset)
rows, err := r.db.Query(listQuery, args...)
if err != nil {
return nil, 0, err
}
defer rows.Close()
var items []*ArticleListItem
for rows.Next() {
item := &ArticleListItem{}
err := rows.Scan(&item.ID, &item.Title, &item.Author, &item.PublishTime,
&item.ReadNum, &item.LikeNum, &item.ContentPreview, &item.OfficialName)
if err != nil {
return nil, 0, err
}
items = append(items, item)
}
return items, total, nil
}
// Search 搜索文章
func (r *ArticleRepository) Search(keyword string, page, pageSize int) ([]*ArticleListItem, int, error) {
keyword = "%" + keyword + "%"
// 获取总数
var total int
err := r.db.QueryRow(`
SELECT COUNT(*) FROM articles WHERE title LIKE ? OR author LIKE ?
`, keyword, keyword).Scan(&total)
if err != nil {
return nil, 0, err
}
// 获取列表
offset := (page - 1) * pageSize
rows, err := r.db.Query(`
SELECT a.id, a.title, a.author, a.publish_time, a.read_num, a.like_num,
a.content_preview, o.nickname
FROM articles a
LEFT JOIN official_accounts o ON a.official_id = o.id
WHERE a.title LIKE ? OR a.author LIKE ?
ORDER BY a.publish_time DESC
LIMIT ? OFFSET ?
`, keyword, keyword, pageSize, offset)
if err != nil {
return nil, 0, err
}
defer rows.Close()
var items []*ArticleListItem
for rows.Next() {
item := &ArticleListItem{}
err := rows.Scan(&item.ID, &item.Title, &item.Author, &item.PublishTime,
&item.ReadNum, &item.LikeNum, &item.ContentPreview, &item.OfficialName)
if err != nil {
return nil, 0, err
}
items = append(items, item)
}
return items, total, nil
}
// Update 更新文章信息
func (r *ArticleRepository) Update(article *Article) error {
_, err := r.db.Exec(`
UPDATE articles
SET read_num = ?, like_num = ?, share_num = ?, updated_at = CURRENT_TIMESTAMP
WHERE id = ?
`, article.ReadNum, article.LikeNum, article.ShareNum, article.ID)
return err
}
// ArticleContentRepository 文章内容数据仓库
type ArticleContentRepository struct {
db *DB
}
// NewArticleContentRepository 创建文章内容仓库
func NewArticleContentRepository(db *DB) *ArticleContentRepository {
return &ArticleContentRepository{db: db}
}
// Create 创建文章内容
func (r *ArticleContentRepository) Create(content *ArticleContent) (int64, error) {
result, err := r.db.Exec(`
INSERT INTO article_contents (article_id, html_content, text_content, paragraphs, images)
VALUES (?, ?, ?, ?, ?)
`, content.ArticleID, content.HtmlContent, content.TextContent,
content.Paragraphs, content.Images)
if err != nil {
return 0, err
}
return result.LastInsertId()
}
// GetByArticleID 根据文章ID获取内容
func (r *ArticleContentRepository) GetByArticleID(articleID int64) (*ArticleContent, error) {
content := &ArticleContent{}
err := r.db.QueryRow(`
SELECT id, article_id, html_content, text_content, paragraphs, images, created_at
FROM article_contents WHERE article_id = ?
`, articleID).Scan(&content.ID, &content.ArticleID, &content.HtmlContent,
&content.TextContent, &content.Paragraphs, &content.Images, &content.CreatedAt)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return content, nil
}
// GetArticleDetail 获取文章详情(包含内容)
func (r *ArticleContentRepository) GetArticleDetail(articleID int64) (*ArticleDetail, error) {
detail := &ArticleDetail{}
var paragraphsJSON, imagesJSON string
err := r.db.QueryRow(`
SELECT a.id, a.official_id, a.title, a.author, a.link, a.publish_time,
a.create_time, a.comment_id, a.read_num, a.like_num, a.share_num,
a.content_preview, a.paragraph_count, a.created_at, a.updated_at,
o.nickname, c.html_content, c.text_content, c.paragraphs, c.images
FROM articles a
LEFT JOIN official_accounts o ON a.official_id = o.id
LEFT JOIN article_contents c ON a.id = c.article_id
WHERE a.id = ?
`, articleID).Scan(
&detail.ID, &detail.OfficialID, &detail.Title, &detail.Author,
&detail.Link, &detail.PublishTime, &detail.CreateTime, &detail.CommentID,
&detail.ReadNum, &detail.LikeNum, &detail.ShareNum, &detail.ContentPreview,
&detail.ParagraphCount, &detail.CreatedAt, &detail.UpdatedAt,
&detail.OfficialName, &detail.HtmlContent, &detail.TextContent,
&paragraphsJSON, &imagesJSON,
)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
// 解析JSON数组
if paragraphsJSON != "" {
json.Unmarshal([]byte(paragraphsJSON), &detail.Paragraphs)
}
if imagesJSON != "" {
json.Unmarshal([]byte(imagesJSON), &detail.Images)
}
return detail, nil
}
// GetStatistics 获取统计信息
func (db *DB) GetStatistics() (*Statistics, error) {
stats := &Statistics{}
err := db.QueryRow(`
SELECT
(SELECT COUNT(*) FROM official_accounts) as total_officials,
(SELECT COUNT(*) FROM articles) as total_articles,
(SELECT COALESCE(SUM(read_num), 0) FROM articles) as total_read_num,
(SELECT COALESCE(SUM(like_num), 0) FROM articles) as total_like_num
`).Scan(&stats.TotalOfficials, &stats.TotalArticles, &stats.TotalReadNum, &stats.TotalLikeNum)
if err != nil {
return nil, err
}
return stats, nil
}
// BatchInsertArticles 批量插入文章
func (r *ArticleRepository) BatchInsertArticles(articles []*Article) error {
if len(articles) == 0 {
return nil
}
// 开始事务
tx, err := r.db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
stmt, err := tx.Prepare(`
INSERT OR IGNORE INTO articles (
official_id, title, author, link, publish_time, create_time,
comment_id, read_num, like_num, share_num, content_preview, paragraph_count
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`)
if err != nil {
return err
}
defer stmt.Close()
for _, article := range articles {
_, err = stmt.Exec(
article.OfficialID, article.Title, article.Author, article.Link,
article.PublishTime, article.CreateTime, article.CommentID,
article.ReadNum, article.LikeNum, article.ShareNum,
article.ContentPreview, article.ParagraphCount,
)
if err != nil {
return err
}
}
return tx.Commit()
}
// Helper function: 将字符串数组转换为JSON字符串
func StringsToJSON(strs []string) string {
if len(strs) == 0 {
return "[]"
}
data, _ := json.Marshal(strs)
return string(data)
}
// Helper function: 生成内容预览
func GeneratePreview(content string, maxLen int) string {
if len(content) <= maxLen {
return content
}
// 移除换行符和多余空格
content = strings.ReplaceAll(content, "\n", " ")
content = strings.ReplaceAll(content, "\r", "")
content = strings.Join(strings.Fields(content), " ")
if len(content) <= maxLen {
return content
}
return content[:maxLen] + "..."
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,48 +0,0 @@
@echo off
echo WeChat Public Article Crawler Startup Script
echo =================================
REM Check if cookie.txt file exists
if not exist "cookie.txt" (
echo Error: cookie.txt file not found!
echo Please create cookie.txt file in backend directory and add WeChat public platform cookie information.
echo.
echo cookie.txt format example:
echo __biz=xxx; uin=xxx; key=xxx; pass_ticket=xxx;
echo.
pause
exit /b 1
)
REM Set Go environment variables (if needed)
REM set GOPATH=%USERPROFILE%\go
REM set GOROOT=C:\Go
REM set PATH=%PATH%;%GOROOT%\bin;%GOPATH%\bin
echo Downloading dependencies...
go mod tidy
if %errorlevel% neq 0 (
echo Failed to download dependencies!
pause
exit /b 1
)
echo Compiling program...
go build -o output\wechat-crawler.exe cmd\main.go
if %errorlevel% neq 0 (
echo Compilation failed!
pause
exit /b 1
)
echo Compilation successful! Starting program...
echo.
REM Ensure data directory exists
if not exist "data" mkdir data
REM Run the program
output\wechat-crawler.exe
pause

View File

@@ -1,57 +0,0 @@
@echo off
rem WeChat Official Account Article Crawler - Script for crawling via article link
setlocal enabledelayedexpansion
REM 检查是否有命令行参数传入
if "%1" neq "" (
REM 如果有参数,直接将其作为文章链接传入程序
echo.
echo Compiling and running...
go run "cmd/main.go" "%1"
if errorlevel 1 (
echo.
echo Failed to run, please check error messages above
pause
exit /b 1
)
echo.
echo Crawling completed successfully!
pause
exit /b 0
) else (
REM 如果没有参数,运行交互式模式
:input_loop
cls
echo ========================================
echo WeChat Official Account Article Crawler
echo ========================================
echo.
echo Please enter WeChat article link:
echo Example: https://mp.weixin.qq.com/s/4r_LKJu0mOeUc70ZZXK9LA
set /p ARTICLE_LINK=
if "%ARTICLE_LINK%"=="" (
echo.
echo Error: Article link cannot be empty!
pause
goto input_loop
)
echo.
echo Compiling and running...
go run "cmd/main.go" "%ARTICLE_LINK%"
if errorlevel 1 (
echo.
echo Failed to run, please check error messages above
pause
exit /b 1
)
echo.
echo Crawling completed successfully!
pause
)

21
backend/tools/view_db.bat Normal file
View File

@@ -0,0 +1,21 @@
@echo off
chcp 65001 >nul
cls
echo ===============================================
echo 📊 数据库内容查看工具
echo ===============================================
echo.
cd /d "%~dp0"
echo 正在查询数据库...
echo.
go run view_db.go
echo.
echo ===============================================
echo 查询完成!
echo ===============================================
pause

231
backend/tools/view_db.go Normal file
View File

@@ -0,0 +1,231 @@
package main
import (
"database/sql"
"encoding/json"
"fmt"
"log"
_ "modernc.org/sqlite"
)
func main() {
// 打开数据库
db, err := sql.Open("sqlite", "../../data/wechat_articles.db")
if err != nil {
log.Fatal("打开数据库失败:", err)
}
defer db.Close()
fmt.Println("=" + repeatStr("=", 80))
fmt.Println("📊 微信公众号文章数据库内容查看")
fmt.Println("=" + repeatStr("=", 80))
// 查询公众号
fmt.Println("\n📢 【公众号列表】")
fmt.Println(repeatStr("-", 80))
queryOfficialAccounts(db)
// 查询文章
fmt.Println("\n📝 【文章列表】")
fmt.Println(repeatStr("-", 80))
queryArticles(db)
// 查询文章内容
fmt.Println("\n📄 【文章详细内容】")
fmt.Println(repeatStr("-", 80))
queryArticleContents(db)
fmt.Println("\n" + repeatStr("=", 80))
}
func queryOfficialAccounts(db *sql.DB) {
rows, err := db.Query(`
SELECT id, biz, nickname, homepage, description, created_at, updated_at
FROM official_accounts
ORDER BY id
`)
if err != nil {
log.Printf("查询公众号失败: %v\n", err)
return
}
defer rows.Close()
count := 0
for rows.Next() {
var id int
var biz, nickname, homepage, description, createdAt, updatedAt string
err := rows.Scan(&id, &biz, &nickname, &homepage, &description, &createdAt, &updatedAt)
if err != nil {
log.Printf("读取数据失败: %v\n", err)
continue
}
count++
fmt.Printf("\n🔹 公众号 #%d\n", id)
fmt.Printf(" 名称: %s\n", nickname)
fmt.Printf(" BIZ: %s\n", biz)
fmt.Printf(" 主页: %s\n", homepage)
fmt.Printf(" 简介: %s\n", description)
fmt.Printf(" 创建时间: %s\n", createdAt)
fmt.Printf(" 更新时间: %s\n", updatedAt)
}
if count == 0 {
fmt.Println(" 暂无数据")
} else {
fmt.Printf("\n总计: %d 个公众号\n", count)
}
}
func queryArticles(db *sql.DB) {
rows, err := db.Query(`
SELECT a.id, a.official_id, a.title, a.author, a.link, a.publish_time,
a.read_num, a.like_num, a.share_num, a.paragraph_count,
a.content_preview, a.created_at, oa.nickname
FROM articles a
LEFT JOIN official_accounts oa ON a.official_id = oa.id
ORDER BY a.id
`)
if err != nil {
log.Printf("查询文章失败: %v\n", err)
return
}
defer rows.Close()
count := 0
for rows.Next() {
var id, officialID, readNum, likeNum, shareNum, paragraphCount int
var title, author, link, publishTime, contentPreview, createdAt, officialName sql.NullString
err := rows.Scan(&id, &officialID, &title, &author, &link, &publishTime,
&readNum, &likeNum, &shareNum, &paragraphCount, &contentPreview, &createdAt, &officialName)
if err != nil {
log.Printf("读取数据失败: %v\n", err)
continue
}
count++
fmt.Printf("\n🔹 文章 #%d\n", id)
fmt.Printf(" 标题: %s\n", getStringValue(title))
if officialName.Valid {
fmt.Printf(" 公众号: %s\n", officialName.String)
}
fmt.Printf(" 作者: %s\n", getStringValue(author))
fmt.Printf(" 链接: %s\n", getStringValue(link))
fmt.Printf(" 发布时间: %s\n", getStringValue(publishTime))
fmt.Printf(" 阅读数: %d | 点赞数: %d | 分享数: %d\n", readNum, likeNum, shareNum)
fmt.Printf(" 段落数: %d\n", paragraphCount)
if contentPreview.Valid && contentPreview.String != "" {
preview := contentPreview.String
if len(preview) > 100 {
preview = preview[:100] + "..."
}
fmt.Printf(" 内容预览: %s\n", preview)
}
fmt.Printf(" 抓取时间: %s\n", getStringValue(createdAt))
}
if count == 0 {
fmt.Println(" 暂无数据")
} else {
fmt.Printf("\n总计: %d 篇文章\n", count)
}
}
func queryArticleContents(db *sql.DB) {
rows, err := db.Query(`
SELECT ac.id, ac.article_id, ac.html_content, ac.text_content,
ac.paragraphs, ac.images, ac.created_at, a.title
FROM article_contents ac
LEFT JOIN articles a ON ac.article_id = a.id
ORDER BY ac.id
`)
if err != nil {
log.Printf("查询文章内容失败: %v\n", err)
return
}
defer rows.Close()
count := 0
for rows.Next() {
var id, articleID int
var htmlContent, textContent, paragraphs, images, createdAt, title sql.NullString
err := rows.Scan(&id, &articleID, &htmlContent, &textContent,
&paragraphs, &images, &createdAt, &title)
if err != nil {
log.Printf("读取数据失败: %v\n", err)
continue
}
count++
fmt.Printf("\n🔹 内容 #%d (文章ID: %d)\n", id, articleID)
if title.Valid {
fmt.Printf(" 文章标题: %s\n", title.String)
}
// HTML内容长度
htmlLen := 0
if htmlContent.Valid {
htmlLen = len(htmlContent.String)
}
fmt.Printf(" HTML内容长度: %d 字符\n", htmlLen)
// 文本内容
if textContent.Valid && textContent.String != "" {
text := textContent.String
if len(text) > 200 {
text = text[:200] + "..."
}
fmt.Printf(" 文本内容: %s\n", text)
}
// 段落信息
if paragraphs.Valid && paragraphs.String != "" {
var paragraphList []interface{}
if err := json.Unmarshal([]byte(paragraphs.String), &paragraphList); err == nil {
fmt.Printf(" 段落数量: %d\n", len(paragraphList))
}
}
// 图片信息
if images.Valid && images.String != "" {
var imageList []interface{}
if err := json.Unmarshal([]byte(images.String), &imageList); err == nil {
fmt.Printf(" 图片数量: %d\n", len(imageList))
if len(imageList) > 0 {
fmt.Printf(" 图片URL:\n")
for i, img := range imageList {
if i >= 3 {
fmt.Printf(" ... 还有 %d 张图片\n", len(imageList)-3)
break
}
fmt.Printf(" %d. %v\n", i+1, img)
}
}
}
}
fmt.Printf(" 存储时间: %s\n", getStringValue(createdAt))
}
if count == 0 {
fmt.Println(" 暂无数据")
} else {
fmt.Printf("\n总计: %d 条详细内容\n", count)
}
}
func getStringValue(s sql.NullString) string {
if s.Valid {
return s.String
}
return ""
}
func repeatStr(s string, n int) string {
result := ""
for i := 0; i < n; i++ {
result += s
}
return result
}

BIN
data/wechat_articles.db Normal file

Binary file not shown.

Binary file not shown.

94
database/test_user_db.py Normal file
View File

@@ -0,0 +1,94 @@
import os
import sys
from user_database import UserDatabase
def test_user_database():
# 获取数据库路径
db_path = os.path.join(os.path.dirname(__file__), 'users.db')
print(f"测试数据库路径: {db_path}")
try:
# 初始化数据库
db = UserDatabase(db_path)
print("✅ 数据库初始化成功")
# 测试用户创建
test_username = "testuser"
test_email = "test@example.com"
test_password = "Test123456"
# 先尝试删除测试用户(如果存在)
try:
user = db.get_user_by_username(test_username)
if user:
print(f"⚠️ 测试用户 '{test_username}' 已存在,尝试创建新的测试用户")
test_username = "testuser_new"
test_email = "test_new@example.com"
except Exception as e:
print(f" 检查用户时发生错误: {e}")
# 创建新用户
print(f"\n🔄 创建测试用户: {test_username}")
user_id = db.create_user(test_username, test_password, test_email)
print(f"✅ 用户创建成功! 用户ID: {user_id}")
# 测试获取用户信息
print(f"\n🔄 测试获取用户信息")
user = db.get_user_by_username(test_username)
if user:
print(f"✅ 用户信息获取成功!")
print(f" - 用户ID: {user['user_id']}")
print(f" - 用户名: {user['username']}")
print(f" - 邮箱: {user['email']}")
print(f" - 创建时间: {user['created_at']}")
else:
print(f"❌ 无法获取用户信息")
# 测试密码验证
print(f"\n🔄 测试密码验证")
# 正确密码
valid_user = db.verify_password(test_username, test_password)
if valid_user:
print(f"✅ 正确密码验证成功")
else:
print(f"❌ 正确密码验证失败")
# 错误密码
invalid_user = db.verify_password(test_username, "WrongPassword")
if not invalid_user:
print(f"✅ 错误密码验证正确返回None")
else:
print(f"❌ 错误密码验证失败应返回None")
# 测试更新登录时间
print(f"\n🔄 测试更新登录时间")
if db.update_login_time(user_id):
print(f"✅ 登录时间更新成功")
# 验证更新是否成功
updated_user = db.get_user_by_id(user_id)
print(f" - 更新后的最后登录时间: {updated_user['last_login_at']}")
else:
print(f"❌ 登录时间更新失败")
print("\n🎉 所有测试完成!")
return True
except Exception as e:
print(f"❌ 测试失败: {str(e)}")
import traceback
traceback.print_exc()
return False
finally:
if 'db' in locals():
try:
db.close()
print(" 数据库连接已关闭")
except:
pass
if __name__ == "__main__":
print("====================================")
print("用户数据库功能测试")
print("====================================")
success = test_user_database()
sys.exit(0 if success else 1)

145
database/user_cli.py Normal file
View File

@@ -0,0 +1,145 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
用户数据库命令行接口
用于Go后端调用执行用户操作
"""
import sys
import json
from user_database import UserDatabase
def main():
if len(sys.argv) < 2:
print(json.dumps({"error": "缺少命令参数"}))
sys.exit(1)
command = sys.argv[1]
db = UserDatabase()
try:
if command == "create":
# 创建用户: python user_cli.py create <username> <password> <email>
if len(sys.argv) < 5:
print(json.dumps({"error": "参数不足: 需要用户名、密码和邮箱"}))
sys.exit(1)
username = sys.argv[2]
password = sys.argv[3]
email = sys.argv[4]
user_id = db.create_user(username, password, email)
print(json.dumps({
"success": True,
"user_id": user_id,
"username": username
}))
elif command == "verify":
# 验证密码: python user_cli.py verify <username> <password>
if len(sys.argv) < 4:
print(json.dumps({"error": "参数不足: 需要用户名和密码"}))
sys.exit(1)
username = sys.argv[2]
password = sys.argv[3]
user = db.verify_password(username, password)
if user:
# 删除密码哈希
user_data = {k: v for k, v in user.items() if k != 'password_hash'}
# 转换datetime为字符串
if 'created_at' in user_data and user_data['created_at']:
user_data['created_at'] = str(user_data['created_at'])
if 'last_login_at' in user_data and user_data['last_login_at']:
user_data['last_login_at'] = str(user_data['last_login_at'])
# 更新登录时间
db.update_login_time(user['user_id'])
print(json.dumps({
"success": True,
**user_data
}))
else:
print(json.dumps({"error": "用户名或密码错误"}))
sys.exit(1)
elif command == "get":
# 获取用户信息: python user_cli.py get <user_id>
if len(sys.argv) < 3:
print(json.dumps({"error": "参数不足: 需要用户ID"}))
sys.exit(1)
user_id = int(sys.argv[2])
user = db.get_user_by_id(user_id)
if user:
# 删除密码哈希
user_data = {k: v for k, v in user.items() if k != 'password_hash'}
# 转换datetime为字符串
if 'created_at' in user_data and user_data['created_at']:
user_data['created_at'] = str(user_data['created_at'])
if 'last_login_at' in user_data and user_data['last_login_at']:
user_data['last_login_at'] = str(user_data['last_login_at'])
print(json.dumps(user_data))
else:
print(json.dumps({"error": "用户不存在"}))
sys.exit(1)
elif command == "update":
# 更新用户信息: python user_cli.py update <user_id> [--email <email>] [--bio <bio>]
if len(sys.argv) < 3:
print(json.dumps({"error": "参数不足: 需要用户ID"}))
sys.exit(1)
user_id = int(sys.argv[2])
kwargs = {}
# 解析命令行参数
i = 3
while i < len(sys.argv):
if sys.argv[i] == "--email" and i + 1 < len(sys.argv):
kwargs['email'] = sys.argv[i + 1]
i += 2
elif sys.argv[i] == "--bio" and i + 1 < len(sys.argv):
kwargs['bio'] = sys.argv[i + 1]
i += 2
elif sys.argv[i] == "--password" and i + 1 < len(sys.argv):
kwargs['password'] = sys.argv[i + 1]
i += 2
else:
i += 1
if not kwargs:
print(json.dumps({"error": "没有提供需要更新的字段"}))
sys.exit(1)
success = db.update_user(user_id, **kwargs)
if success:
print(json.dumps({"success": True, "message": "更新成功"}))
else:
print(json.dumps({"error": "更新失败"}))
sys.exit(1)
else:
print(json.dumps({"error": f"未知命令: {command}"}))
sys.exit(1)
except db.UserExistsError as e:
print(json.dumps({"error": str(e), "type": "UserExistsError"}))
sys.exit(1)
except db.ValidationError as e:
print(json.dumps({"error": str(e), "type": "ValidationError"}))
sys.exit(1)
except db.UserNotFoundError as e:
print(json.dumps({"error": str(e), "type": "UserNotFoundError"}))
sys.exit(1)
except Exception as e:
print(json.dumps({"error": str(e), "type": "Exception"}))
sys.exit(1)
if __name__ == "__main__":
main()

556
database/user_database.py Normal file
View File

@@ -0,0 +1,556 @@
import sqlite3
import datetime
import os
import bcrypt
import re
import logging
# 配置日志
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
filename=os.path.join(os.path.dirname(__file__), 'user_db.log')
)
logger = logging.getLogger('UserDatabase')
class UserDatabase:
def __init__(self, db_path='users.db'):
"""
初始化数据库配置
:param db_path: 数据库文件路径
"""
self.db_path = os.path.join(os.path.dirname(__file__), db_path)
# 初始化时只创建表,不保持连接
self._setup_adapters() # 设置全局适配器
self._create_tables_at_init() # 初始化时创建表
def _setup_adapters(self):
"""设置SQLite适配器和转换器全局设置"""
# 添加自定义datetime适配器以解决Python 3.12弃用警告
def adapt_datetime(dt):
return dt.isoformat()
# 注册适配器
sqlite3.register_adapter(datetime.datetime, adapt_datetime)
# 添加转换器从字符串转换回datetime
def convert_datetime(val):
try:
return datetime.datetime.fromisoformat(val.decode())
except (ValueError, AttributeError):
return val
# 注册转换器
sqlite3.register_converter('TIMESTAMP', convert_datetime)
def _get_connection(self):
"""
获取新的数据库连接(线程安全)
:return: 数据库连接对象
"""
try:
conn = sqlite3.connect(self.db_path)
conn.row_factory = sqlite3.Row # 允许通过列名访问结果
return conn
except sqlite3.Error as e:
print(f"数据库连接错误: {e}")
raise
def _create_tables_at_init(self):
"""
初始化时创建用户表
"""
conn = self._get_connection()
try:
cursor = conn.cursor()
# 创建用户表,添加适当的主键和索引
cursor.execute('''
CREATE TABLE IF NOT EXISTS users (
user_id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT NOT NULL UNIQUE,
email TEXT NOT NULL UNIQUE,
password_hash TEXT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
last_login_at TIMESTAMP,
status INTEGER DEFAULT 1,
profile_image TEXT,
bio TEXT,
UNIQUE(username),
UNIQUE(email)
)
''')
# 创建索引以提高查询效率
cursor.execute('CREATE INDEX IF NOT EXISTS idx_username ON users(username)')
cursor.execute('CREATE INDEX IF NOT EXISTS idx_email ON users(email)')
cursor.execute('CREATE INDEX IF NOT EXISTS idx_status ON users(status)')
conn.commit()
except sqlite3.Error as e:
conn.rollback()
print(f"创建表错误: {e}")
raise
finally:
conn.close()
def _hash_password(self, password):
"""
对密码进行哈希加密
:param password: 原始密码
:return: 加密后的密码哈希
"""
# 使用bcrypt进行哈希更安全的密码存储方式
salt = bcrypt.gensalt()
return bcrypt.hashpw(password.encode(), salt).decode()
# 定义自定义异常类
class DatabaseError(Exception):
"""数据库操作异常基类"""
pass
class UserExistsError(DatabaseError):
"""用户已存在异常"""
pass
class UserNotFoundError(DatabaseError):
"""用户不存在异常"""
pass
class ValidationError(DatabaseError):
"""数据验证异常"""
pass
def _validate_input(self, username, email, password=None):
"""
验证输入数据的有效性
:param username: 用户名
:param email: 邮箱
:param password: 密码(可选)
:return: 验证通过返回True否则抛出异常
"""
# 验证用户名
if not username or not isinstance(username, str):
raise self.ValidationError("用户名不能为空")
if len(username) < 3:
raise self.ValidationError("用户名必须至少包含3个字符")
if len(username) > 50:
raise self.ValidationError("用户名不能超过50个字符")
if not re.match(r'^[a-zA-Z0-9_]+$', username):
raise self.ValidationError("用户名只能包含字母、数字和下划线")
# 验证邮箱 - 使用更严格的正则表达式
if not email or not isinstance(email, str):
raise self.ValidationError("邮箱不能为空")
email_pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
if not re.match(email_pattern, email):
raise self.ValidationError("邮箱格式不正确")
if len(email) > 255:
raise self.ValidationError("邮箱不能超过255个字符")
# 验证密码
if password is not None:
if not password:
raise self.ValidationError("密码不能为空")
if len(password) < 6:
raise self.ValidationError("密码必须至少包含6个字符")
if len(password) > 128:
raise self.ValidationError("密码不能超过128个字符")
return True
def create_user(self, username, password, email, nickname=''):
"""
创建新用户
:param username: 用户名
:param password: 密码
:param email: 邮箱
:param nickname: 昵称(可选)
:return: 创建成功返回用户ID否则抛出异常
"""
conn = self._get_connection()
try:
cursor = conn.cursor()
# 验证输入
self._validate_input(username, email, password)
# 检查用户名是否已存在
cursor.execute("SELECT user_id FROM users WHERE username = ?", (username,))
if cursor.fetchone():
raise self.UserExistsError("用户名已存在")
# 检查邮箱是否已存在
cursor.execute("SELECT user_id FROM users WHERE email = ?", (email,))
if cursor.fetchone():
raise self.UserExistsError("邮箱已被注册")
# 哈希密码
password_hash = self._hash_password(password)
# 插入用户记录
cursor.execute('''
INSERT INTO users (username, email, password_hash, created_at)
VALUES (?, ?, ?, ?)
''', (username, email, password_hash, datetime.datetime.now()))
conn.commit()
user_id = cursor.lastrowid
logger.info(f"创建用户成功ID: {user_id}, 用户名: {username}")
return user_id
except self.DatabaseError:
raise
except sqlite3.Error as e:
conn.rollback()
error_msg = f"创建用户失败: {str(e)}"
logger.error(f"创建用户错误: {e}, 用户名: {username}, 邮箱: {email}")
raise self.DatabaseError(error_msg)
def get_user_by_id(self, user_id):
"""
根据用户ID获取用户信息
:param user_id: 用户ID
:return: 用户信息字典如果不存在返回None
"""
conn = self._get_connection()
try:
cursor = conn.cursor()
cursor.execute("SELECT * FROM users WHERE user_id = ?", (user_id,))
row = cursor.fetchone()
if row:
return dict(row)
return None
except sqlite3.Error as e:
print(f"查询用户错误: {e}")
return None
finally:
conn.close()
def get_user_by_username(self, username):
"""
根据用户名获取用户信息
:param username: 用户名
:return: 用户信息字典如果不存在返回None
"""
conn = self._get_connection()
try:
cursor = conn.cursor()
cursor.execute("SELECT * FROM users WHERE username = ?", (username,))
row = cursor.fetchone()
if row:
return dict(row)
return None
except sqlite3.Error as e:
print(f"查询用户错误: {e}")
return None
finally:
conn.close()
def get_user_by_email(self, email):
"""
根据邮箱获取用户信息
:param email: 邮箱
:return: 用户信息字典如果不存在返回None
"""
conn = self._get_connection()
try:
cursor = conn.cursor()
cursor.execute("SELECT * FROM users WHERE email = ?", (email,))
row = cursor.fetchone()
if row:
return dict(row)
return None
except sqlite3.Error as e:
print(f"查询用户错误: {e}")
return None
finally:
conn.close()
def verify_password(self, username, password):
"""
验证用户密码
:param username: 用户名
:param password: 密码
:return: 验证成功返回用户信息字典失败返回None
"""
try:
# 获取用户信息get_user_by_username已经使用独立连接
user = self.get_user_by_username(username)
if not user:
logger.warning(f"密码验证失败: 用户不存在 - {username}")
return None
# 检查账号状态
if user['status'] != 1:
logger.warning(f"密码验证失败: 账号状态异常 - 用户名: {username}, 状态: {user['status']}")
return None
# 验证密码 - 使用bcrypt的checkpw方法
is_valid = bcrypt.checkpw(password.encode('utf-8'), user['password_hash'].encode('utf-8'))
if is_valid:
logger.info(f"密码验证成功: {username}")
return user # 验证成功返回用户信息
else:
logger.warning(f"密码验证失败: 密码错误 - {username}")
return None
except Exception as e:
logger.error(f"密码验证错误: {e}, 用户名: {username}")
return None
def update_login_time(self, user_id):
"""
更新用户最后登录时间
:param user_id: 用户ID
:return: 更新成功返回True否则返回False
"""
conn = self._get_connection()
try:
cursor = conn.cursor()
cursor.execute(
"UPDATE users SET last_login_at = ? WHERE user_id = ?",
(datetime.datetime.now(), user_id)
)
conn.commit()
return cursor.rowcount > 0
except sqlite3.Error as e:
conn.rollback()
print(f"更新登录时间错误: {e}")
return False
finally:
conn.close()
def update_user(self, user_id, **kwargs):
"""
更新用户信息
:param user_id: 用户ID
:param kwargs: 要更新的字段
:return: 更新成功返回True否则返回False
"""
conn = self._get_connection()
try:
cursor = conn.cursor()
# 检查用户是否存在
if not self.get_user_by_id(user_id):
raise self.UserNotFoundError("用户不存在")
# 准备更新字段
update_fields = []
update_values = []
# 记录要更新的字段用于日志
updated_fields_log = []
if 'username' in kwargs:
update_fields.append("username = ?")
update_values.append(kwargs['username'])
updated_fields_log.append('username')
# 验证用户名
self._validate_input(kwargs['username'], "dummy@example.com")
# 检查用户名是否被其他用户使用
cursor.execute(
"SELECT user_id FROM users WHERE username = ? AND user_id != ?",
(kwargs['username'], user_id)
)
if cursor.fetchone():
raise self.UserExistsError("用户名已存在")
if 'email' in kwargs:
update_fields.append("email = ?")
update_values.append(kwargs['email'])
updated_fields_log.append('email')
# 验证邮箱
self._validate_input("dummy", kwargs['email'])
# 检查邮箱是否被其他用户使用
cursor.execute(
"SELECT user_id FROM users WHERE email = ? AND user_id != ?",
(kwargs['email'], user_id)
)
if cursor.fetchone():
raise self.UserExistsError("邮箱已被注册")
if 'password' in kwargs:
# 验证密码
self._validate_input("dummy", "dummy@example.com", kwargs['password'])
# 哈希密码
password_hash = self._hash_password(kwargs['password'])
update_fields.append("password_hash = ?")
update_values.append(password_hash)
updated_fields_log.append('password')
if 'status' in kwargs:
# 验证状态值
if not isinstance(kwargs['status'], int) or kwargs['status'] not in (0, 1, 2):
raise self.ValidationError("无效的用户状态值")
update_fields.append("status = ?")
update_values.append(kwargs['status'])
updated_fields_log.append('status')
if 'profile_image' in kwargs:
# 验证头像路径
if kwargs['profile_image'] and len(kwargs['profile_image']) > 255:
raise self.ValidationError("头像路径过长")
update_fields.append("profile_image = ?")
update_values.append(kwargs['profile_image'])
updated_fields_log.append('profile_image')
if 'bio' in kwargs:
# 验证个人简介
if kwargs['bio'] and len(kwargs['bio']) > 500:
raise self.ValidationError("个人简介不能超过500个字符")
update_fields.append("bio = ?")
update_values.append(kwargs['bio'])
updated_fields_log.append('bio')
# 执行更新
if update_fields:
update_values.append(user_id)
update_sql = f"UPDATE users SET {', '.join(update_fields)} WHERE user_id = ?"
cursor.execute(update_sql, update_values)
conn.commit()
logger.info(f"更新用户信息成功ID: {user_id}, 更新字段: {', '.join(updated_fields_log)}")
return True
else:
# 没有要更新的字段
logger.warning(f"没有更新字段提供用户ID: {user_id}")
return True
except self.DatabaseError:
raise
except sqlite3.Error as e:
conn.rollback()
logger.error(f"更新用户信息错误: {e}, 用户ID: {user_id}")
raise self.DatabaseError(f"更新用户信息失败: {str(e)}")
finally:
conn.close()
def delete_user(self, user_id):
"""
删除用户
:param user_id: 用户ID
:return: 删除成功返回True否则返回False
"""
try:
# 检查用户是否存在
user = self.get_user_by_id(user_id)
if not user:
raise self.UserNotFoundError("用户不存在")
# 开始事务
self.cursor.execute("DELETE FROM users WHERE user_id = ?", (user_id,))
self.conn.commit()
if self.cursor.rowcount > 0:
logger.info(f"删除用户成功ID: {user_id}, 用户名: {user['username']}")
return True
return False
except self.DatabaseError:
raise
except sqlite3.Error as e:
self.conn.rollback()
error_msg = f"删除用户失败: {str(e)}"
logger.error(f"删除用户错误: {e}, 用户ID: {user_id}")
raise self.DatabaseError(error_msg)
def list_users(self, limit=100, offset=0):
"""
列出用户(支持分页)
:param limit: 每页数量
:param offset: 偏移量
:return: 用户列表
"""
conn = self._get_connection()
try:
cursor = conn.cursor()
cursor.execute(
"SELECT * FROM users ORDER BY created_at DESC LIMIT ? OFFSET ?",
(limit, offset)
)
users = []
for row in cursor.fetchall():
users.append(dict(row))
return users
except sqlite3.Error as e:
print(f"列出用户错误: {e}")
return []
finally:
conn.close()
def close(self):
"""
关闭数据库连接(保持兼容性)
"""
# 现在每个操作都使用独立连接,不需要关闭持久连接
print("数据库连接管理已更新,每个操作使用独立连接")
# 示例用法
if __name__ == "__main__":
print("=== 用户数据库示例程序 ===")
# 创建数据库实例
db = UserDatabase()
try:
print("\n1. 创建测试用户")
# 尝试创建已存在的用户
try:
user_id = db.create_user(
username="demo_user",
email="demo@example.com",
password="SecurePass123"
)
print(f"✅ 创建用户成功ID: {user_id}")
except db.UserExistsError as e:
print(f" {e},继续使用现有用户")
user = db.get_user_by_username("demo_user")
if user:
user_id = user['user_id']
print("\n2. 查询用户信息")
user = db.get_user_by_id(user_id)
if user:
# 不打印密码哈希
safe_user = {k: v for k, v in user.items() if k != 'password_hash'}
print(f"✅ 查询用户成功: {safe_user}")
print("\n3. 验证密码")
is_valid = db.verify_password("demo_user", "SecurePass123")
print(f"✅ 密码验证: {'通过' if is_valid else '失败'}")
# 测试密码验证失败的情况
is_valid_wrong = db.verify_password("demo_user", "WrongPassword")
print(f"✅ 错误密码验证: {'通过' if is_valid_wrong else '失败'}")
print("\n4. 更新用户信息")
success = db.update_user(user_id,
bio="这是一个测试用户账号",
status=1)
if success:
updated_user = db.get_user_by_id(user_id)
safe_updated = {k: v for k, v in updated_user.items() if k != 'password_hash'}
print(f"✅ 更新用户成功: {safe_updated}")
print("\n5. 列出用户")
users = db.list_users(limit=10)
safe_users = [{k: v for k, v in u.items() if k != 'password_hash'} for u in users]
print(f"✅ 共找到 {len(safe_users)} 个用户")
for u in safe_users:
print(f" - {u['username']} ({u['email']})")
print("\n6. 测试数据验证")
try:
db.create_user(username="ab", email="invalid-email", password="123")
except db.ValidationError as e:
print(f"✅ 验证错误捕获成功: {e}")
print("\n7. 更新登录时间")
if db.update_login_time(user_id):
print("✅ 登录时间更新成功")
print("\n=== 测试完成 ===")
except Exception as e:
print(f"❌ 发生错误: {e}")
finally:
# 关闭数据库连接
print("\n关闭数据库连接...")
db.close()
print("数据库连接已关闭")

20
database/user_db.log Normal file
View File

@@ -0,0 +1,20 @@
2025-12-02 13:50:34,469 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD>û<EFBFBD><C3BB>ɹ<EFBFBD><C9B9><EFBFBD>ID: 6, <20>û<EFBFBD><C3BB><EFBFBD>: testuser123
2025-12-02 13:50:40,670 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: testuser123
2025-12-02 13:52:24,597 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD>û<EFBFBD><C3BB>ɹ<EFBFBD><C9B9><EFBFBD>ID: 7, <20>û<EFBFBD><C3BB><EFBFBD>: testuser3
2025-12-02 13:52:31,767 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: testuser3
2025-12-02 13:54:04,070 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD>û<EFBFBD><C3BB>ɹ<EFBFBD><C9B9><EFBFBD>ID: 8, <20>û<EFBFBD><C3BB><EFBFBD>: test
2025-12-02 13:54:55,089 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: testuser3
2025-12-02 13:56:44,653 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: testuser3
2025-12-02 13:56:51,923 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: testuser3
2025-12-02 13:57:52,850 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: testuser3
2025-12-02 13:58:03,952 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: testuser3
2025-12-02 13:58:30,809 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: testuser3
2025-12-02 13:59:00,866 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: test
2025-12-02 14:02:56,791 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD>û<EFBFBD><C3BB><EFBFBD>Ϣ<EFBFBD>ɹ<EFBFBD><C9B9><EFBFBD>ID: 7, <20><><EFBFBD><EFBFBD><EFBFBD>ֶ<EFBFBD>: email, bio
2025-12-02 14:06:03,849 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: test
2025-12-02 14:12:21,407 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: test
2025-12-02 14:14:03,743 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: test
2025-12-02 14:16:30,941 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: test
2025-12-02 14:18:21,143 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: test
2025-12-02 14:23:19,324 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: test
2025-12-02 14:51:35,109 - UserDatabase - INFO - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD>ɹ<EFBFBD>: test

BIN
database/users.db Normal file

Binary file not shown.

View File

@@ -0,0 +1,67 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
查看所有用户信息脚本
"""
import os
import sys
# 添加项目根目录到Python路径
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from database.user_database import UserDatabase
def main():
"""
主函数:查看所有用户信息
"""
print("=== 查看所有用户信息 ===")
try:
# 创建数据库实例 - 指定正确的数据库路径
db_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'backend', 'users.db')
print(f"使用数据库路径: {db_path}")
db = UserDatabase(db_path=db_path)
# 获取所有用户
print("\n正在查询用户信息...")
users = db.list_users(limit=1000)
if not users:
print("\n❌ 未找到任何用户")
return
print(f"\n✅ 共找到 {len(users)} 个用户:")
print("-" * 100)
print(f"{'用户ID':<10} | {'用户名':<20} | {'邮箱':<30} | {'状态':<10} | {'创建时间':<20}")
print("-" * 100)
# 显示用户信息(不包含密码哈希)
for user in users:
# 创建不包含密码哈希的安全用户信息
safe_user = {k: v for k, v in user.items() if k != 'password_hash'}
# 格式化输出
user_id = safe_user.get('user_id', '')
username = safe_user.get('username', '')
email = safe_user.get('email', '')
status = '正常' if safe_user.get('status') == 1 else '禁用' if safe_user.get('status') == 0 else '异常'
created_at = safe_user.get('created_at', '')
print(f"{user_id:<10} | {username:<20} | {email:<30} | {status:<10} | {created_at:<20}")
print("-" * 100)
print("\n✅ 用户信息查看完成")
except Exception as e:
print(f"\n❌ 发生错误: {e}")
import traceback
traceback.print_exc()
finally:
# 关闭数据库连接
if 'db' in locals():
print("\n关闭数据库连接...")
db.close()
if __name__ == "__main__":
main()

View File

@@ -393,9 +393,9 @@
<a href="#" class="nav-link">监控中心</a> <a href="#" class="nav-link">监控中心</a>
<a href="#" class="nav-link">数据分析</a> <a href="#" class="nav-link">数据分析</a>
<a href="#" class="nav-link">帮助文档</a> <a href="#" class="nav-link">帮助文档</a>
<a href="user-center.html" class="nav-link" id="userMenu" style="display: none;">👤 用户中心</a> <a href="user-center.html" class="nav-link" id="userCenterLink">👤 用户中心</a>
<a href="login.html" class="nav-link" id="loginLink">🔐 登录</a> <a href="login.html" class="nav-link" id="loginLink">🔐 登录</a>
<a href="#" class="nav-link" id="logoutLink" style="display: none;" onclick="logout()">🚪 退出</a> <div id="userMenu" class="user-info"></div>
</nav> </nav>
</div> </div>
</header> </header>
@@ -736,11 +736,18 @@ ${article.content.replace(/<[^>]*>/g, '')}
// 登录状态管理 // 登录状态管理
function checkLoginStatus() { function checkLoginStatus() {
const isLoggedIn = localStorage.getItem('isLoggedIn') === 'true'; // 检查认证数据
const username = localStorage.getItem('username'); const authData = localStorage.getItem('authData') || sessionStorage.getItem('authData');
if (isLoggedIn && username) { if (authData) {
showLoggedInState(username); try {
const auth = JSON.parse(authData);
const username = auth.user_info && auth.user_info.username ? auth.user_info.username : '用户';
showLoggedInState(username);
} catch (e) {
console.error('解析登录数据失败:', e);
showLoggedOutState();
}
} else { } else {
showLoggedOutState(); showLoggedOutState();
} }
@@ -748,38 +755,87 @@ ${article.content.replace(/<[^>]*>/g, '')}
// 显示已登录状态 // 显示已登录状态
function showLoggedInState(username) { function showLoggedInState(username) {
document.getElementById('loginLink').style.display = 'none'; const loginLink = document.getElementById('loginLink');
document.getElementById('userMenu').style.display = 'block'; const userMenu = document.getElementById('userMenu');
document.getElementById('logoutLink').style.display = 'block';
// 为用户中心菜单添加点击事件 if (loginLink) loginLink.style.display = 'none';
document.getElementById('userMenu').onclick = function() { if (userMenu) {
window.location.href = 'user-center.html'; userMenu.innerHTML = `
}; <span style="margin-right: 10px;">👋 ${username}</span>
<button onclick="logout()" style="
background: rgba(255, 255, 255, 0.2);
color: white;
border: 1px solid rgba(255, 255, 255, 0.3);
padding: 6px 12px;
border-radius: 4px;
cursor: pointer;
font-size: 13px;
transition: all 0.3s;
" onmouseover="this.style.background='rgba(255, 255, 255, 0.3)'"
onmouseout="this.style.background='rgba(255, 255, 255, 0.2)'">
退出登录
</button>
`;
userMenu.style.display = 'flex';
}
} }
// 显示未登录状态 // 显示未登录状态
function showLoggedOutState() { function showLoggedOutState() {
document.getElementById('loginLink').style.display = 'block'; const loginLink = document.getElementById('loginLink');
document.getElementById('userMenu').style.display = 'none'; const userMenu = document.getElementById('userMenu');
document.getElementById('logoutLink').style.display = 'none';
if (loginLink) loginLink.style.display = 'inline';
if (userMenu) userMenu.innerHTML = '';
} }
// 登出功能 // 登出功能
function logout() { function logout() {
if (confirm('确定要退出登录吗?')) { if (confirm('确定要退出登录吗?')) {
localStorage.removeItem('isLoggedIn'); // 获取认证数据
localStorage.removeItem('username'); const authData = localStorage.getItem('authData') || sessionStorage.getItem('authData');
localStorage.removeItem('token'); let token = '';
sessionStorage.removeItem('userSession');
showLoggedOutState(); if (authData) {
alert('已成功退出登录!'); try {
const auth = JSON.parse(authData);
// 如果在用户中心页面,跳转到首页 token = auth.token || '';
if (window.location.pathname.includes('user-center.html')) { } catch (e) {
window.location.href = 'frontend.html'; console.error('解析token失败:', e);
}
} }
// 调用后端登出API
fetch('http://localhost:8080/api/user/logout', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': token ? `Bearer ${token}` : ''
}
}).then(response => {
// 无论API调用成功与否都清除本地数据
localStorage.removeItem('authData');
localStorage.removeItem('isLoggedIn');
localStorage.removeItem('username');
localStorage.removeItem('token');
sessionStorage.removeItem('authData');
sessionStorage.removeItem('userSession');
alert('已成功退出登录!');
window.location.href = 'login.html';
}).catch(error => {
console.error('登出请求失败:', error);
// 即使API调用失败也清除本地数据
localStorage.removeItem('authData');
localStorage.removeItem('isLoggedIn');
localStorage.removeItem('username');
localStorage.removeItem('token');
sessionStorage.removeItem('authData');
sessionStorage.removeItem('userSession');
alert('已成功退出登录!');
window.location.href = 'login.html';
});
} }
} }

521
frontend/css/style.css Normal file
View File

@@ -0,0 +1,521 @@
/* 全局样式 */
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
line-height: 1.6;
color: #333;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
min-height: 100vh;
}
.container {
max-width: 1200px;
margin: 0 auto;
padding: 20px;
}
/* 头部样式 */
.header {
text-align: center;
color: white;
margin-bottom: 30px;
}
.header h1 {
font-size: 2.5em;
margin-bottom: 10px;
text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3);
}
.subtitle {
font-size: 1.1em;
opacity: 0.9;
}
/* 主内容区域 */
.main-content {
background: white;
border-radius: 15px;
padding: 30px;
box-shadow: 0 10px 30px rgba(0, 0, 0, 0.2);
min-height: 500px;
}
/* 功能卡片网格 */
.feature-cards {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
gap: 25px;
margin-bottom: 30px;
}
.card {
background: linear-gradient(135deg, #f5f7fa 0%, #c3cfe2 100%);
border-radius: 12px;
padding: 25px;
text-align: center;
transition: all 0.3s ease;
border: 2px solid transparent;
cursor: pointer;
}
.card:hover {
transform: translateY(-5px);
box-shadow: 0 15px 35px rgba(0, 0, 0, 0.1);
border-color: #667eea;
}
.card-icon {
font-size: 3em;
margin-bottom: 15px;
}
.card h3 {
font-size: 1.3em;
margin-bottom: 10px;
color: #2c3e50;
}
.card p {
color: #7f8c8d;
margin-bottom: 20px;
}
/* 按钮样式 */
.btn {
padding: 12px 24px;
border: none;
border-radius: 8px;
cursor: pointer;
font-size: 1em;
font-weight: 600;
transition: all 0.3s ease;
text-decoration: none;
display: inline-block;
min-width: 100px;
}
.btn-primary {
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
color: white;
}
.btn-primary:hover {
transform: translateY(-2px);
box-shadow: 0 8px 25px rgba(102, 126, 234, 0.4);
}
.btn-success {
background: linear-gradient(135deg, #56ab2f 0%, #a8e6cf 100%);
color: white;
}
.btn-success:hover {
transform: translateY(-2px);
box-shadow: 0 8px 25px rgba(86, 171, 47, 0.4);
}
.btn-info {
background: linear-gradient(135deg, #3498db 0%, #74b9ff 100%);
color: white;
}
.btn-info:hover {
transform: translateY(-2px);
box-shadow: 0 8px 25px rgba(52, 152, 219, 0.4);
}
.btn-warning {
background: linear-gradient(135deg, #f39c12 0%, #fdcb6e 100%);
color: white;
}
.btn-warning:hover {
transform: translateY(-2px);
box-shadow: 0 8px 25px rgba(243, 156, 18, 0.4);
}
.btn-secondary {
background: linear-gradient(135deg, #95a5a6 0%, #bdc3c7 100%);
color: white;
}
.btn-secondary:hover {
transform: translateY(-2px);
box-shadow: 0 8px 25px rgba(149, 165, 166, 0.4);
}
/* 按钮禁用状态 */
.btn:disabled,
.btn.disabled {
opacity: 0.6;
cursor: not-allowed;
transform: none !important;
box-shadow: none !important;
}
.btn:disabled:hover,
.btn.disabled:hover {
transform: none !important;
box-shadow: none !important;
}
/* 区域样式 */
.section {
animation: fadeIn 0.5s ease-in;
}
@keyframes fadeIn {
from { opacity: 0; transform: translateY(20px); }
to { opacity: 1; transform: translateY(0); }
}
.section-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 30px;
padding-bottom: 15px;
border-bottom: 2px solid #ecf0f1;
}
.section-header h2 {
color: #2c3e50;
font-size: 1.8em;
}
/* 表单样式 */
.form-group {
margin-bottom: 25px;
}
.form-group label {
display: block;
margin-bottom: 8px;
font-weight: 600;
color: #2c3e50;
}
.form-group input,
.form-group textarea,
.form-group select {
width: 100%;
padding: 12px 15px;
border: 2px solid #ddd;
border-radius: 8px;
font-size: 1em;
transition: border-color 0.3s ease;
font-family: inherit;
}
.form-group input:focus,
.form-group textarea:focus,
.form-group select:focus {
outline: none;
border-color: #667eea;
box-shadow: 0 0 0 3px rgba(102, 126, 234, 0.1);
}
.form-group small {
display: block;
margin-top: 5px;
color: #7f8c8d;
font-size: 0.9em;
}
.checkbox-label {
display: inline-flex !important;
align-items: center;
margin-right: 20px;
margin-bottom: 10px;
font-weight: normal !important;
}
.checkbox-label input[type="checkbox"] {
width: auto !important;
margin-right: 8px;
}
.form-actions {
display: flex;
gap: 15px;
flex-wrap: wrap;
margin-top: 25px;
}
/* 结果显示区域 */
.result {
margin-top: 25px;
padding: 20px;
border-radius: 8px;
display: none;
}
.result.success {
background-color: #d4edda;
color: #155724;
border: 1px solid #c3e6cb;
}
.result.error {
background-color: #f8d7da;
color: #721c24;
border: 1px solid #f5c6cb;
}
.result.info {
background-color: #d1ecf1;
color: #0c5460;
border: 1px solid #bee5eb;
}
.result.loading {
background-color: #fff3cd;
color: #856404;
border: 1px solid #ffeaa7;
}
/* 数据列表样式 */
.data-list {
margin-top: 25px;
}
.data-item {
background: #f8f9fa;
border: 1px solid #dee2e6;
border-radius: 8px;
padding: 20px;
margin-bottom: 15px;
transition: all 0.3s ease;
}
.data-item:hover {
transform: translateX(5px);
box-shadow: 0 5px 15px rgba(0, 0, 0, 0.1);
}
.data-item h4 {
color: #2c3e50;
margin-bottom: 10px;
font-size: 1.2em;
}
.data-item-info {
display: flex;
justify-content: space-between;
align-items: center;
flex-wrap: wrap;
gap: 10px;
}
.data-item-stats {
display: flex;
gap: 20px;
align-items: center;
}
.stat-item {
display: flex;
align-items: center;
gap: 5px;
font-size: 0.9em;
color: #7f8c8d;
}
.data-item-actions {
display: flex;
gap: 10px;
}
/* 进度条样式 */
.progress-container {
margin-top: 20px;
}
.progress-bar {
width: 100%;
height: 8px;
background-color: #ecf0f1;
border-radius: 4px;
overflow: hidden;
}
.progress-fill {
height: 100%;
background: linear-gradient(90deg, #667eea, #764ba2);
width: 0%;
transition: width 0.3s ease;
animation: pulse 1.5s infinite;
}
@keyframes pulse {
0%, 100% { opacity: 1; }
50% { opacity: 0.7; }
}
.progress-text {
margin-top: 10px;
text-align: center;
font-size: 0.9em;
color: #7f8c8d;
}
/* 状态指示器 */
.status-indicator {
display: inline-block;
width: 10px;
height: 10px;
border-radius: 50%;
margin-right: 8px;
}
.status-success { background-color: #27ae60; }
.status-error { background-color: #e74c3c; }
.status-warning { background-color: #f39c12; }
.status-info { background-color: #3498db; }
/* 底部样式 */
.footer {
text-align: center;
margin-top: 30px;
padding: 20px;
color: white;
opacity: 0.8;
}
/* 工具提示 */
.tooltip {
position: relative;
cursor: help;
}
.tooltip:hover::after {
content: attr(data-tooltip);
position: absolute;
bottom: 100%;
left: 50%;
transform: translateX(-50%);
background: #2c3e50;
color: white;
padding: 8px 12px;
border-radius: 4px;
font-size: 0.9em;
white-space: nowrap;
z-index: 1000;
}
/* 响应式设计 */
@media (max-width: 768px) {
.container {
padding: 10px;
}
.header h1 {
font-size: 2em;
}
.main-content {
padding: 20px;
}
.feature-cards {
grid-template-columns: 1fr;
gap: 15px;
}
.section-header {
flex-direction: column;
align-items: flex-start;
gap: 15px;
}
.form-actions {
flex-direction: column;
}
.data-item-info {
flex-direction: column;
align-items: flex-start;
}
.data-item-actions {
width: 100%;
justify-content: flex-start;
}
}
@media (max-width: 480px) {
.header h1 {
font-size: 1.8em;
}
.card {
padding: 20px;
}
.btn {
padding: 10px 20px;
font-size: 0.9em;
}
}
/* 加载动画 */
.loading-spinner {
display: inline-block;
width: 20px;
height: 20px;
border: 2px solid #f3f3f3;
border-top: 2px solid #667eea;
border-radius: 50%;
animation: spin 1s linear infinite;
margin-right: 10px;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
/* 文本对齐工具类 */
.text-center { text-align: center; }
.text-left { text-align: left; }
.text-right { text-align: right; }
/* 间距工具类 */
.mt-10 { margin-top: 10px; }
.mt-20 { margin-top: 20px; }
.mb-10 { margin-bottom: 10px; }
.mb-20 { margin-bottom: 20px; }
/* 显示/隐藏工具类 */
.hidden { display: none !important; }
.visible { display: block !important; }
/* 帮助文本样式 */
.help-text {
background: #f8f9fa;
padding: 20px;
border-radius: 8px;
margin-bottom: 25px;
border-left: 4px solid #667eea;
}
.help-text p {
margin: 10px 0;
color: #2c3e50;
}
.help-text ul {
margin: 10px 0 10px 20px;
color: #555;
}
.help-text li {
margin: 8px 0;
line-height: 1.6;
}

View File

@@ -1053,9 +1053,9 @@
<!-- 头部 --> <!-- 头部 -->
<header class="header"> <header class="header">
<div class="header-content"> <div class="header-content">
<a href="#" class="logo">🔍 易搜高</a> <a href="frontend.html" class="logo">🔍 易搜高</a>
<nav class="nav-menu"> <nav class="nav-menu">
<a href="#" class="nav-link active">首页</a> <a href="frontend.html" class="nav-link active">首页</a>
<a href="#" class="nav-link">监控中心</a> <a href="#" class="nav-link">监控中心</a>
<a href="#" class="nav-link">数据分析</a> <a href="#" class="nav-link">数据分析</a>
<a href="#" class="nav-link">帮助文档</a> <a href="#" class="nav-link">帮助文档</a>
@@ -1073,6 +1073,7 @@
<div class="nav-tabs"> <div class="nav-tabs">
<button class="nav-tab active" data-tab="library">📚 自媒体库</button> <button class="nav-tab active" data-tab="library">📚 自媒体库</button>
<button class="nav-tab" data-tab="monitor">👁️ 监控账号</button> <button class="nav-tab" data-tab="monitor">👁️ 监控账号</button>
<button class="nav-tab" onclick="window.location.href='index.html'" style="background: linear-gradient(135deg, #28a745 0%, #20c997 100%); color: white; margin-top: 10px;">🚀 爬虫系统</button>
</div> </div>
</div> </div>

View File

@@ -310,9 +310,9 @@
<a href="#" class="nav-link">监控中心</a> <a href="#" class="nav-link">监控中心</a>
<a href="#" class="nav-link">数据分析</a> <a href="#" class="nav-link">数据分析</a>
<a href="#" class="nav-link">帮助文档</a> <a href="#" class="nav-link">帮助文档</a>
<a href="user-center.html" class="nav-link" id="userMenu" style="display: none;">👤 用户中心</a> <a href="user-center.html" class="nav-link" id="userCenterLink">👤 用户中心</a>
<a href="login.html" class="nav-link" id="loginLink">🔐 登录</a> <a href="login.html" class="nav-link" id="loginLink">🔐 登录</a>
<a href="#" class="nav-link" id="logoutLink" style="display: none;" onclick="logout()">🚪 退出</a> <div id="userMenu" class="user-info"></div>
</nav> </nav>
</div> </div>
<h1>历史文章</h1> <h1>历史文章</h1>
@@ -629,11 +629,18 @@
// 登录状态管理 // 登录状态管理
function checkLoginStatus() { function checkLoginStatus() {
const isLoggedIn = localStorage.getItem('isLoggedIn') === 'true'; // 检查认证数据
const username = localStorage.getItem('username'); const authData = localStorage.getItem('authData') || sessionStorage.getItem('authData');
if (isLoggedIn && username) { if (authData) {
showLoggedInState(username); try {
const auth = JSON.parse(authData);
const username = auth.user_info && auth.user_info.username ? auth.user_info.username : '用户';
showLoggedInState(username);
} catch (e) {
console.error('解析登录数据失败:', e);
showLoggedOutState();
}
} else { } else {
showLoggedOutState(); showLoggedOutState();
} }
@@ -641,38 +648,68 @@
// 显示已登录状态 // 显示已登录状态
function showLoggedInState(username) { function showLoggedInState(username) {
document.getElementById('loginLink').style.display = 'none'; const loginLink = document.getElementById('loginLink');
document.getElementById('userMenu').style.display = 'block'; const userMenu = document.getElementById('userMenu');
document.getElementById('logoutLink').style.display = 'block';
// 为用户中心菜单添加点击事件 if (loginLink) loginLink.style.display = 'none';
document.getElementById('userMenu').onclick = function() { if (userMenu) {
window.location.href = 'user-center.html'; userMenu.innerHTML = `
}; <div class="user-info">
<span class="user-name">${username}</span>
<button class="logout-btn" onclick="logout()" style="background: rgba(255,255,255,0.2); color: white; border: 1px solid rgba(255,255,255,0.3); padding: 6px 16px; border-radius: 6px; cursor: pointer; font-size: 14px; transition: all 0.3s;">退出登录</button>
</div>
`;
userMenu.style.display = 'flex';
}
} }
// 显示未登录状态 // 显示未登录状态
function showLoggedOutState() { function showLoggedOutState() {
document.getElementById('loginLink').style.display = 'block'; const loginLink = document.getElementById('loginLink');
document.getElementById('userMenu').style.display = 'none'; const userMenu = document.getElementById('userMenu');
document.getElementById('logoutLink').style.display = 'none';
if (loginLink) loginLink.style.display = 'inline';
if (userMenu) userMenu.innerHTML = '';
} }
// 登出功能 // 登出功能
function logout() { function logout() {
if (confirm('确定要退出登录吗?')) { if (confirm('确定要退出登录吗?')) {
localStorage.removeItem('isLoggedIn'); // 获取认证数据
localStorage.removeItem('username'); const authData = localStorage.getItem('authData') || sessionStorage.getItem('authData');
localStorage.removeItem('token'); let token = '';
sessionStorage.removeItem('userSession');
showLoggedOutState(); if (authData) {
alert('已成功退出登录!'); try {
const auth = JSON.parse(authData);
// 如果在用户中心页面,跳转到首页 token = auth.token || '';
if (window.location.pathname.includes('user-center.html')) { } catch (e) {
window.location.href = 'frontend.html'; console.error('解析token失败:', e);
}
} }
// 调用后端API登出
fetch('http://localhost:8080/api/user/logout', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': token
},
body: JSON.stringify({ token: token })
})
.finally(() => {
// 清除所有认证信息
localStorage.removeItem('authData');
sessionStorage.removeItem('authData');
localStorage.removeItem('isLoggedIn');
localStorage.removeItem('username');
localStorage.removeItem('token');
sessionStorage.removeItem('userSession');
showLoggedOutState();
alert('已成功退出登录!');
window.location.href = 'login.html';
});
} }
} }

286
frontend/index.html Normal file
View File

@@ -0,0 +1,286 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>微信公众号文章爬虫系统</title>
<link rel="stylesheet" href="css/style.css">
</head>
<body>
<!-- 头部导航 -->
<header class="top-navbar" style="background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); color: white; padding: 15px 0; box-shadow: 0 2px 10px rgba(0,0,0,0.1); position: sticky; top: 0; z-index: 100;">
<div class="navbar-content" style="max-width: 1200px; margin: 0 auto; padding: 0 20px; display: flex; align-items: center; justify-content: space-between;">
<a href="frontend.html" class="logo" style="font-size: 24px; font-weight: bold; text-decoration: none; color: white;">🔍 易搜高</a>
<nav class="nav-menu" style="display: flex; gap: 15px; align-items: center;">
<a href="frontend.html" class="nav-link" style="color: white; text-decoration: none; font-size: 14px; font-weight: 500; padding: 8px 16px; border-radius: 6px; transition: all 0.3s; background: rgba(255, 255, 255, 0.1); border: 1px solid rgba(255, 255, 255, 0.2);" onmouseover="this.style.background='rgba(255, 255, 255, 0.2)'" onmouseout="this.style.background='rgba(255, 255, 255, 0.1)'">首页</a>
<a href="#" class="nav-link" style="color: white; text-decoration: none; font-size: 14px; font-weight: 500; padding: 8px 16px; border-radius: 6px; transition: all 0.3s; background: rgba(255, 255, 255, 0.1); border: 1px solid rgba(255, 255, 255, 0.2);" onmouseover="this.style.background='rgba(255, 255, 255, 0.2)'" onmouseout="this.style.background='rgba(255, 255, 255, 0.1)'">监控中心</a>
<a href="#" class="nav-link" style="color: white; text-decoration: none; font-size: 14px; font-weight: 500; padding: 8px 16px; border-radius: 6px; transition: all 0.3s; background: rgba(255, 255, 255, 0.1); border: 1px solid rgba(255, 255, 255, 0.2);" onmouseover="this.style.background='rgba(255, 255, 255, 0.2)'" onmouseout="this.style.background='rgba(255, 255, 255, 0.1)'">数据分析</a>
<a href="#" class="nav-link" style="color: white; text-decoration: none; font-size: 14px; font-weight: 500; padding: 8px 16px; border-radius: 6px; transition: all 0.3s; background: rgba(255, 255, 255, 0.1); border: 1px solid rgba(255, 255, 255, 0.2);" onmouseover="this.style.background='rgba(255, 255, 255, 0.2)'" onmouseout="this.style.background='rgba(255, 255, 255, 0.1)'">帮助文档</a>
<a href="user-center.html" class="nav-link" id="userCenterLink" style="color: white; text-decoration: none; font-size: 14px; font-weight: 500; padding: 8px 16px; border-radius: 6px; transition: all 0.3s; background: rgba(255, 255, 255, 0.1); border: 1px solid rgba(255, 255, 255, 0.2);" onmouseover="this.style.background='rgba(255, 255, 255, 0.2)'" onmouseout="this.style.background='rgba(255, 255, 255, 0.1)'">👤 用户中心</a>
<a href="login.html" class="nav-link" id="loginLink" style="color: white; text-decoration: none; font-size: 14px; font-weight: 500; padding: 8px 16px; border-radius: 6px; transition: all 0.3s; background: rgba(255, 255, 255, 0.1); border: 1px solid rgba(255, 255, 255, 0.2);" onmouseover="this.style.background='rgba(255, 255, 255, 0.2)'" onmouseout="this.style.background='rgba(255, 255, 255, 0.1)'">🔐 登录</a>
<div id="userMenu" class="user-info" style="display: flex; align-items: center; gap: 10px;"></div>
</nav>
</div>
</header>
<div class="container">
<header class="header">
<h1>🚀 微信公众号文章爬虫系统</h1>
<p class="subtitle">Wechat Official Account Article Crawler</p>
</header>
<div class="main-content">
<!-- 功能选择卡片 -->
<div class="feature-cards">
<div class="card" id="card-homepage">
<div class="card-icon">🏠</div>
<h3>提取公众号主页</h3>
<p>输入文章链接获取公众号主页链接</p>
<button class="btn btn-primary" onclick="showSection('homepage')">进入</button>
</div>
<div class="card" id="card-list">
<div class="card-icon">📋</div>
<h3>获取文章列表</h3>
<p>获取公众号所有文章列表</p>
<button class="btn btn-primary" onclick="showSection('list')">进入</button>
</div>
<div class="card" id="card-batch">
<div class="card-icon">📦</div>
<h3>批量下载文章</h3>
<p>批量下载文章详细内容</p>
<button class="btn btn-primary" onclick="showSection('batch')">进入</button>
</div>
<div class="card" id="card-detail">
<div class="card-icon">📊</div>
<h3>获取文章详情</h3>
<p>获取文章阅读量、点赞数、评论等详细信息</p>
<button class="btn btn-primary" onclick="showSection('detail')">进入</button>
</div>
</div>
<!-- 提取公众号主页区域 -->
<div class="section" id="section-homepage" style="display:none;">
<div class="section-header">
<h2>🏠 提取公众号主页</h2>
<button class="btn btn-secondary" onclick="showSection('home')">返回</button>
</div>
<div class="form-group">
<label>公众号文章链接:</label>
<input type="text" id="homepage-url" placeholder="请输入公众号下任意一篇已发布的文章链接...">
<small>支持公众号文章完整URL无需Cookie即可获取公众号主页链接</small>
</div>
<div class="form-actions">
<button class="btn btn-success" onclick="extractHomepage()">提取主页链接</button>
<button class="btn btn-info" onclick="loadExampleUrl()">查看示例</button>
</div>
<div class="result" id="homepage-result"></div>
</div>
<!-- 获取文章列表区域 -->
<div class="section" id="section-list" style="display:none;">
<div class="section-header">
<h2>📋 获取文章列表</h2>
<button class="btn btn-secondary" onclick="showSection('home')">返回</button>
</div>
<div class="form-group">
<label>Access Token URL</label>
<textarea id="access-token" placeholder="请粘贴从Fiddler获取的完整URL..." rows="4"></textarea>
<small>包含 __biz, uin, key, pass_ticket 等参数的完整URL</small>
</div>
<div class="form-group">
<label>获取页数:</label>
<input type="number" id="pages" value="1" min="1" max="999">
<small>留空表示获取全部</small>
</div>
<div class="form-actions">
<button class="btn btn-success" onclick="getArticleList()">开始获取</button>
</div>
<div class="result" id="list-result"></div>
</div>
<!-- 批量下载区域 -->
<div class="section" id="section-batch" style="display:none;">
<div class="section-header">
<h2>📦 批量下载文章</h2>
<button class="btn btn-secondary" onclick="showSection('home')">返回</button>
</div>
<div class="form-group">
<label>公众号名称或文章链接:</label>
<input type="text" id="official-account" placeholder="请输入公众号名称或任意一篇文章链接...">
</div>
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="batch-save-image"> 保存图片
</label>
<label class="checkbox-label">
<input type="checkbox" id="batch-save-content" checked> 保存内容
</label>
</div>
<div class="form-actions">
<button class="btn btn-success" onclick="batchDownload()">开始批量下载</button>
</div>
<div class="result" id="batch-result"></div>
</div>
<!-- 获取文章详情区域 -->
<div class="section" id="section-detail" style="display:none;">
<div class="section-header">
<h2>📊 获取文章详情</h2>
<button class="btn btn-secondary" onclick="showSection('home')">返回</button>
</div>
<div class="help-text">
<p><strong>💡 功能说明:</strong></p>
<ul>
<li>📈 自动获取公众号所有文章列表</li>
<li>📊 下载每篇文章的阅读量、点赞数、分享数等统计数据</li>
<li>💬 获取文章评论信息(如果有)</li>
<li>📁 保存为TXT格式便于查看和分析</li>
</ul>
<p><strong>⚠️ 注意事项:</strong></p>
<ul>
<li>需要提供有效的Access Token URL包含登录凭证</li>
<li>处理时间较长,请耐心等待</li>
<li>数据将保存在 data/公众号名称/文章详细 目录下</li>
</ul>
</div>
<div class="form-group">
<label>Access Token URL</label>
<textarea id="detail-access-token" placeholder="请粘贴从Fiddler获取的完整URL...\n\n步骤\n1. 在浏览器中登录微信公众号平台\n2. 访问目标公众号主页\n3. 向下滚动加载文章列表\n4. 在Fiddler中找到profile_ext?action=getmsg请求\n5. 复制完整的URL包含__biz、uin、key、pass_ticket等参数" rows="6"></textarea>
<small>示例https://mp.weixin.qq.com/mp/profile_ext?action=getmsg&__biz=...&uin=...&key=...&pass_ticket=...</small>
</div>
<div class="form-group">
<label>获取页数:</label>
<input type="number" id="detail-pages" value="0" min="0" max="999" placeholder="留空或0表示获取全部">
<small>每页约10篇文章留空或输入0表示获取全部文章</small>
</div>
<div class="form-actions">
<button class="btn btn-success" onclick="getArticleDetail()">开始获取</button>
<button class="btn btn-info" onclick="loadDetailExample()">查看示例</button>
</div>
<div class="result" id="detail-result"></div>
</div>
</div>
<footer class="footer">
<p>© 2025 微信公众号文章爬虫系统 | 仅供学习使用</p>
</footer>
</div>
<script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
<script src="js/app.js"></script>
<script>
// 登录状态管理
function checkLoginStatus() {
// 检查认证数据
const authData = localStorage.getItem('authData') || sessionStorage.getItem('authData');
if (authData) {
try {
const auth = JSON.parse(authData);
const username = auth.user_info && auth.user_info.username ? auth.user_info.username : '用户';
showLoggedInState(username);
} catch (e) {
console.error('解析登录数据失败:', e);
showLoggedOutState();
}
} else {
showLoggedOutState();
}
}
// 显示已登录状态
function showLoggedInState(username) {
const loginLink = document.getElementById('loginLink');
const userMenu = document.getElementById('userMenu');
if (loginLink) loginLink.style.display = 'none';
if (userMenu) {
userMenu.innerHTML = `
<span style="margin-right: 10px;">👋 ${username}</span>
<button onclick="logout()" style="
background: rgba(255, 255, 255, 0.2);
color: white;
border: 1px solid rgba(255, 255, 255, 0.3);
padding: 6px 12px;
border-radius: 4px;
cursor: pointer;
font-size: 13px;
transition: all 0.3s;
" onmouseover="this.style.background='rgba(255, 255, 255, 0.3)'"
onmouseout="this.style.background='rgba(255, 255, 255, 0.2)'">
退出登录
</button>
`;
userMenu.style.display = 'flex';
}
}
// 显示未登录状态
function showLoggedOutState() {
const loginLink = document.getElementById('loginLink');
const userMenu = document.getElementById('userMenu');
if (loginLink) loginLink.style.display = 'inline';
if (userMenu) userMenu.innerHTML = '';
}
// 登出功能
function logout() {
if (confirm('确定要退出登录吗?')) {
// 获取认证数据
const authData = localStorage.getItem('authData') || sessionStorage.getItem('authData');
let token = '';
if (authData) {
try {
const auth = JSON.parse(authData);
token = auth.token || '';
} catch (e) {
console.error('解析token失败:', e);
}
}
// 调用后端登出API
fetch('http://localhost:8080/api/user/logout', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': token ? `Bearer ${token}` : ''
}
}).then(response => {
// 无论API调用成功与否都清除本地数据
localStorage.removeItem('authData');
localStorage.removeItem('isLoggedIn');
localStorage.removeItem('username');
localStorage.removeItem('token');
sessionStorage.removeItem('authData');
sessionStorage.removeItem('userSession');
alert('已成功退出登录!');
window.location.href = 'login.html';
}).catch(error => {
console.error('登出请求失败:', error);
// 即使API调用失败也清除本地数据
localStorage.removeItem('authData');
localStorage.removeItem('isLoggedIn');
localStorage.removeItem('username');
localStorage.removeItem('token');
sessionStorage.removeItem('authData');
sessionStorage.removeItem('userSession');
alert('已成功退出登录!');
window.location.href = 'login.html';
});
}
}
// 页面加载时检查登录状态
document.addEventListener('DOMContentLoaded', function() {
checkLoginStatus();
});
</script>
</body>
</html>

602
frontend/js/app.js Normal file
View File

@@ -0,0 +1,602 @@
// 全局变量
let isTaskRunning = false;
let currentSection = 'home';
let taskCheckInterval;
const API_BASE_URL = 'http://localhost:8080/api'; // API基础地址
// DOM加载完成后初始化
$(document).ready(function() {
showSection('home');
console.log('✅ 微信公众号文章爬虫系统已加载');
});
// 显示指定区域
function showSection(sectionName) {
// 隐藏所有区域
$('.section').hide();
$('.feature-cards').hide();
if (sectionName === 'home') {
$('.feature-cards').show();
currentSection = 'home';
} else {
$('#section-' + sectionName).show();
currentSection = sectionName;
}
}
// 提取公众号主页相关函数
function extractHomepage() {
const articleUrl = $('#homepage-url').val().trim();
if (!articleUrl) {
showResult('homepage', 'error', '请输入文章链接');
return;
}
if (!articleUrl.includes('mp.weixin.qq.com')) {
showResult('homepage', 'error', '请输入有效的微信公众号文章链接');
return;
}
if (isTaskRunning) {
showResult('homepage', 'error', '有任务正在执行,请稍候...');
return;
}
isTaskRunning = true;
showResult('homepage', 'loading', '正在提取公众号主页链接...');
// 调用后端API
$.ajax({
url: `${API_BASE_URL}/homepage/extract`,
method: 'POST',
contentType: 'application/json',
data: JSON.stringify({ url: articleUrl }),
success: function(response) {
isTaskRunning = false;
if (response.success && response.data && response.data.homepage) {
const homepageUrl = response.data.homepage;
const safeUrl = homepageUrl.replace(/'/g, "\\'");
const resultHtml = `
<div style="background: #f8f9fa; padding: 15px; border-radius: 8px; margin-top: 10px;">
<h4 style="color: #28a745; margin-bottom: 10px;">✅ 提取成功</h4>
<p><strong>公众号主页链接:</strong></p>
<div style="background: white; padding: 10px; border: 1px solid #ddd; border-radius: 4px; word-break: break-all; font-family: monospace; font-size: 0.9em;">
${homepageUrl}
</div>
<div style="margin-top: 15px;">
<button class="btn btn-info" onclick="copyToClipboard('${safeUrl}')" style="margin-right: 10px;">📋 复制链接</button>
<button class="btn btn-warning" onclick="openInNewTab('${safeUrl}')">🔗 打开主页</button>
</div>
</div>
`;
showResult('homepage', 'success', resultHtml);
} else {
showResult('homepage', 'error', response.message || '提取失败');
}
},
error: function(xhr, status, error) {
isTaskRunning = false;
let errorMsg = '请求失败:' + error;
if (xhr.status === 0) {
errorMsg = '无法连接到后端服务器,请确保 API 服务器已启动(运行 api_server.exe';
}
showResult('homepage', 'error', errorMsg);
}
});
}
// 生成模拟的公众号主页链接
function generateMockHomepageUrl(articleUrl) {
// 从文章链接中提取__biz参数来模拟真实的主页链接
const bizMatch = articleUrl.match(/__biz=([^&]+)/);
if (bizMatch) {
const biz = bizMatch[1];
return `https://mp.weixin.qq.com/mp/profile_ext?action=home&__biz=${biz}&scene=124`;
}
// 如果无法提取,返回示例链接
return 'https://mp.weixin.qq.com/mp/profile_ext?action=home&__biz=MzI1NjEwMTM4OA==&scene=124';
}
function loadExampleUrl() {
const exampleUrl = 'https://mp.weixin.qq.com/s?__biz=MzI1NjEwMTM4OA==&mid=2651232405&idx=1&sn=7c8f5b2e3d4a1b9c8e7f6a5b4c3d2e1f&chksm=f1d7e8c4c6a061d2b9e8f7a6b5c4d3e2f1a0b9c8d7e6f5a4b3c2d1e0f9a8b7c6d5e4f3a2b1c0&scene=27';
$('#homepage-url').val(exampleUrl);
showResult('homepage', 'info', '已加载文章链接示例,点击"提取主页链接"开始处理');
}
// 打开链接的辅助函数
function openInNewTab(url) {
window.open(url, '_blank');
}
// 下载单篇文章
// 获取文章列表
function getArticleList() {
const accessToken = $('#access-token').val().trim();
const pages = parseInt($('#pages').val()) || 0;
if (!accessToken) {
showResult('list', 'error', '请输入Access Token URL');
return;
}
if (isTaskRunning) {
showResult('list', 'error', '有任务正在执行,请稍候...');
return;
}
isTaskRunning = true;
showResult('list', 'loading', '正在获取文章列表,请稍候...');
// 调用后端API同步等待
$.ajax({
url: `${API_BASE_URL}/article/list`,
method: 'POST',
contentType: 'application/json',
data: JSON.stringify({ access_token: accessToken, pages: pages }),
success: function(response) {
isTaskRunning = false;
if (response.success && response.data) {
const data = response.data;
const fileExt = data.filename.endsWith('.txt') ? 'TXT文件' : 'Excel文件';
const resultHtml = `
<div style="background: #f8f9fa; padding: 15px; border-radius: 8px; margin-top: 10px;">
<h4 style="color: #28a745; margin-bottom: 10px;">✅ 获取成功</h4>
<p><strong>公众号:</strong>${data.account}</p>
<p><strong>文件:</strong>${data.filename}</p>
<div style="margin-top: 15px;">
<a href="${API_BASE_URL}${data.download}" class="btn btn-success" download>📥 下载${fileExt}</a>
</div>
</div>
`;
showResult('list', 'success', resultHtml);
// 自动触发下载
window.location.href = `${API_BASE_URL}${data.download}`;
} else {
showResult('list', 'error', response.message || '获取失败');
}
},
error: function(xhr, status, error) {
isTaskRunning = false;
let errorMsg = '请求失败:' + error;
if (xhr.status === 0) {
errorMsg = '无法连接到后端服务器,请确保 API 服务器已启动';
} else if (xhr.responseJSON && xhr.responseJSON.message) {
errorMsg = xhr.responseJSON.message;
}
showResult('list', 'error', errorMsg);
},
timeout: 120000 // 2分钟超时
});
}
// 批量下载文章
function batchDownload() {
const officialAccount = $('#official-account').val().trim();
const saveImage = $('#batch-save-image').is(':checked');
const saveContent = $('#batch-save-content').is(':checked');
if (!officialAccount) {
showResult('batch', 'error', '请输入公众号名称或文章链接');
return;
}
if (isTaskRunning) {
showResult('batch', 'error', '有任务正在执行,请稍候...');
return;
}
isTaskRunning = true;
showResult('batch', 'loading', '正在批量下载文章,请稍候...');
// 调用后端 API同步等待
$.ajax({
url: `${API_BASE_URL}/article/batch`,
method: 'POST',
contentType: 'application/json',
data: JSON.stringify({
official_account: officialAccount,
save_image: saveImage,
save_content: saveContent
}),
success: function(response) {
isTaskRunning = false;
if (response.success && response.data) {
const data = response.data;
const resultHtml = `
<div style="background: #f8f9fa; padding: 15px; border-radius: 8px; margin-top: 10px;">
<h4 style="color: #28a745; margin-bottom: 10px;">✅ ${response.message}</h4>
<p><strong>公众号:</strong>${data.account}</p>
<p><strong>文章数量:</strong>${data.articleCount} 篇</p>
<p><strong>保存路径:</strong>${data.path}</p>
</div>
`;
showResult('batch', 'success', resultHtml);
} else {
showResult('batch', 'error', response.message || '批量下载失败');
}
},
error: function(xhr, status, error) {
isTaskRunning = false;
let errorMsg = '请求失败:' + error;
if (xhr.status === 0) {
errorMsg = '无法连接到后端服务器,请确保 API 服务器已启动';
} else if (xhr.responseJSON && xhr.responseJSON.message) {
errorMsg = xhr.responseJSON.message;
}
showResult('batch', 'error', errorMsg);
},
timeout: 300000 // 5分钟超时
});
}
// 获取文章详情功能4
function getArticleDetail() {
const accessToken = $('#detail-access-token').val().trim();
const pages = parseInt($('#detail-pages').val()) || 0;
const submitBtn = $('#section-detail .btn-success');
// 1. 验证输入
if (!accessToken) {
showResult('detail', 'error', '⚠️ 请输入Access Token URL');
// 高亮输入框
$('#detail-access-token').css('border-color', '#e74c3c').focus();
setTimeout(() => {
$('#detail-access-token').css('border-color', '');
}, 2000);
return;
}
// 2. 检查是否有任务正在执行
if (isTaskRunning) {
showResult('detail', 'error', '⚠️ 有任务正在执行,请稍候...');
return;
}
// 3. 设置任务状态并禁用按钮
isTaskRunning = true;
submitBtn.prop('disabled', true)
.addClass('disabled')
.html('⏳ 处理中...');
// 4. 构建提示信息
const pagesInfo = pages > 0 ? `${pages}页(约${pages * 10}篇文章)` : '全部文章';
// 5. 显示加载状态
showResult('detail', 'loading', `
<div style="text-align: center;">
<div style="font-size: 1.2em; margin-bottom: 10px;">⏳ 正在获取文章详情,请耐心等待...</div>
<div style="background: #e3f2fd; padding: 12px; border-radius: 5px; margin: 15px 0; border-left: 4px solid #2196f3;">
<p style="margin: 5px 0; color: #1565c0;"><strong>📌 本次获取范围:</strong>${pagesInfo}</p>
</div>
<div style="background: #fff3cd; padding: 12px; border-radius: 5px; margin: 15px 0;">
<p style="margin: 5px 0; color: #856404;"><strong>📝 执行步骤:</strong></p>
<ol style="text-align: left; margin: 10px 0; padding-left: 30px; color: #856404;">
<li>正在获取公众号文章列表...</li>
<li>正在下载每篇文章的详细数据...</li>
<li>正在保存为TXT文件...</li>
</ol>
<p style="margin: 10px 0 0 0; color: #d9534f;"><strong>⚠️ 此过程可能需要几分钟,系统会自动延时避免被封禁</strong></p>
<p style="margin: 5px 0 0 0; color: #856404; font-size: 0.9em;">💡 提示:请不要关闭页面或刷新浏览器</p>
</div>
</div>
`);
console.log('🚀 开始获取文章详情...');
console.log('📄 获取页数:', pages === 0 ? '全部' : pages);
// 6. 调用后端 API
$.ajax({
url: `${API_BASE_URL}/article/detail`,
method: 'POST',
contentType: 'application/json',
data: JSON.stringify({
access_token: accessToken,
pages: pages
}),
beforeSend: function() {
console.log('📡 发送请求到后端 API...');
},
success: function(response) {
console.log('✅ 收到服务器响应:', response);
// 恢复任务状态和按钮
isTaskRunning = false;
submitBtn.prop('disabled', false)
.removeClass('disabled')
.html('开始获取');
if (response.success && response.data) {
const data = response.data;
const resultHtml = `
<div style="background: #f8f9fa; padding: 20px; border-radius: 8px; margin-top: 10px; border: 2px solid #28a745;">
<h4 style="color: #28a745; margin-bottom: 15px; font-size: 1.3em;">✅ 获取成功!</h4>
<div style="background: white; padding: 15px; border-radius: 5px; margin: 10px 0;">
<p style="margin: 8px 0;"><strong>📱 公众号:</strong><span style="color: #667eea;">${data.account}</span></p>
<p style="margin: 8px 0;"><strong>📊 文章数量:</strong><span style="color: #667eea;">${data.articleCount} 篇</span></p>
<p style="margin: 8px 0;"><strong>📁 保存路径:</strong><span style="color: #667eea; font-size: 0.9em;">${data.path}</span></p>
</div>
<div style="background: #e7f3ff; padding: 15px; border-radius: 5px; margin: 15px 0; border-left: 4px solid #3498db;">
<p style="margin: 5px 0; color: #0066cc; font-weight: bold;">📊 数据包含:</p>
<ul style="margin: 10px 0; padding-left: 25px; color: #0066cc;">
<li style="margin: 5px 0;">📄 文章文本内容</li>
<li style="margin: 5px 0;">👁️ 阅读量、点赞数、转发数、在看数</li>
<li style="margin: 5px 0;">💬 评论内容及评论点赞数</li>
</ul>
</div>
<p style="margin-top: 15px; text-align: center; color: #7f8c8d; font-size: 0.9em;">💡 数据已保存在 data/${data.account}/文章详细 目录下</p>
</div>
`;
showResult('detail', 'success', resultHtml);
} else {
showResult('detail', 'error', `
<div style="padding: 10px;">
<h4 style="color: #e74c3c; margin-bottom: 10px;">❌ 获取失败</h4>
<p style="margin: 10px 0;">${response.message || '未知错误,请稍后重试'}</p>
<div style="background: #fff3cd; padding: 12px; border-radius: 5px; margin-top: 15px;">
<p style="margin: 5px 0; color: #856404;"><strong>💡 可能的原因:</strong></p>
<ul style="margin: 10px 0; padding-left: 25px; color: #856404;">
<li>Access Token 已过期</li>
<li>公众号权限不足</li>
<li>网络连接问题</li>
</ul>
<p style="margin: 10px 0 0 0; color: #856404;">请重新从 Fiddler 获取 Access Token URL 并重试</p>
</div>
</div>
`);
}
},
error: function(xhr, status, error) {
console.error('❌ 请求失败:', { xhr, status, error });
// 恢复任务状态和按钮
isTaskRunning = false;
submitBtn.prop('disabled', false)
.removeClass('disabled')
.html('开始获取');
let errorMsg = '请求失败';
let errorDetail = '';
if (xhr.status === 0) {
errorMsg = '无法连接到后端服务器';
errorDetail = `
<div style="background: #f8d7da; padding: 12px; border-radius: 5px; margin-top: 15px; border-left: 4px solid #e74c3c;">
<p style="margin: 5px 0; color: #721c24;"><strong>🔧 解决方法:</strong></p>
<ol style="margin: 10px 0; padding-left: 25px; color: #721c24;">
<li>确认后端 API 服务器已启动</li>
<li>检查服务器地址http://localhost:8080</li>
<li>查看服务器控制台是否有错误信息</li>
</ol>
</div>
`;
} else if (xhr.status === 404) {
errorMsg = '接口不存在';
errorDetail = `<p style="margin-top: 10px; color: #721c24;">请检查API路径配置是否正确</p>`;
} else if (xhr.status === 500) {
errorMsg = '服务器内部错误';
errorDetail = `<p style="margin-top: 10px; color: #721c24;">请查看服务器日志获取详细错误信息</p>`;
} else if (xhr.responseJSON && xhr.responseJSON.message) {
errorMsg = xhr.responseJSON.message;
} else if (status === 'timeout') {
errorMsg = '请求超时';
errorDetail = `
<p style="margin-top: 10px; color: #721c24;">
文章数量较多处理时间超过10分钟。<br>
建议:减少文章数量或稍后查看服务器是否已生成文件。
</p>
`;
} else {
errorMsg += ': ' + (error || '未知错误');
}
showResult('detail', 'error', `
<div style="padding: 10px;">
<h4 style="color: #e74c3c; margin-bottom: 10px;">❌ ${errorMsg}</h4>
${errorDetail}
<p style="margin-top: 15px; color: #7f8c8d; font-size: 0.9em;">
错误状态码: ${xhr.status || 'N/A'}<br>
错误类型: ${status || 'N/A'}
</p>
</div>
`);
},
complete: function() {
console.log('🏁 请求完成');
},
timeout: 600000 // 10分钟超时因为功能4需要较长时间
});
}
// 加载示例 Access Token
function loadDetailExample() {
const exampleToken = 'https://mp.weixin.qq.com/mp/profile_ext?action=getmsg&__biz=MzI1NjEwMTM4OA==&uin=MTIzNDU2Nzg5&key=abcdef123456&pass_ticket=xyz789&...';
$('#detail-access-token').val(exampleToken);
showResult('detail', 'info', '已加载 Access Token 示例。<br><br><strong>注意:</strong>请从 Fiddler 中获取实际的 Access Token URL示例仅供参考。<br><br><strong>获取步骤:</strong><br>1. 在微信客户端打开公众号主页<br>2. 在 Fiddler 中查找 URL 为 /mp/profile_ext?action=getmsg 的请求<br>3. 按 Ctrl+U 复制完整 URL<br>4. 粘贴到此处');
}
// 任务管理函数
function startTask(section, message) {
isTaskRunning = true;
showResult(section, 'loading', message);
// 显示进度条
const resultDiv = $(`#${section}-result`);
resultDiv.append(`
<div class="progress-container">
<div class="progress-bar">
<div class="progress-fill"></div>
</div>
<div class="progress-text">0%</div>
</div>
`);
// 禁用相关按钮
disableButtons();
}
function updateTaskProgress(percent, message) {
const progressFill = $('.progress-fill');
const progressText = $('.progress-text');
progressFill.css('width', percent + '%');
progressText.text(Math.floor(percent) + '% - ' + message);
}
function endTask(section, type, message) {
isTaskRunning = false;
// 移除进度条
$('.progress-container').remove();
showResult(section, type, message);
enableButtons();
}
function disableButtons() {
$('.btn').prop('disabled', true).addClass('disabled');
}
function enableButtons() {
$('.btn').prop('disabled', false).removeClass('disabled');
}
// 结果显示函数
function showResult(section, type, message) {
const resultDiv = $(`#${section}-result`);
resultDiv.removeClass('success error info loading')
.addClass(type)
.html(getResultIcon(type) + message)
.show();
// 自动滚动到结果区域
resultDiv[0].scrollIntoView({ behavior: 'smooth' });
}
function hideResult(section) {
$(`#${section}-result`).hide();
}
function getResultIcon(type) {
switch (type) {
case 'success': return '<span class="loading-spinner" style="display:none;"></span>✅ ';
case 'error': return '<span class="loading-spinner" style="display:none;"></span>❌ ';
case 'info': return '<span class="loading-spinner" style="display:none;"></span> ';
case 'loading': return '<span class="loading-spinner"></span>';
default: return '';
}
}
// 表单验证函数
function validateUrl(url) {
try {
new URL(url);
return true;
} catch {
return false;
}
}
function validateInput(value, type) {
switch (type) {
case 'url':
return validateUrl(value);
case 'notEmpty':
return value.trim().length > 0;
case 'number':
return !isNaN(value) && parseInt(value) > 0;
default:
return true;
}
}
// 工具函数
function formatFileSize(bytes) {
if (bytes === 0) return '0 Bytes';
const k = 1024;
const sizes = ['Bytes', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i];
}
function formatDate(dateString) {
const date = new Date(dateString);
return date.getFullYear() + '-' +
String(date.getMonth() + 1).padStart(2, '0') + '-' +
String(date.getDate()).padStart(2, '0');
}
function copyToClipboard(text) {
navigator.clipboard.writeText(text).then(() => {
alert('已复制到剪贴板');
}).catch(() => {
alert('复制失败,请手动复制');
});
}
// 快捷键支持
$(document).keydown(function(e) {
// ESC键返回首页
if (e.keyCode === 27 && currentSection !== 'home') {
showSection('home');
}
// Ctrl+Enter 执行当前页面的主要操作
if (e.ctrlKey && e.keyCode === 13) {
switch (currentSection) {
case 'homepage':
extractHomepage();
break;
case 'single':
downloadSingleArticle();
break;
case 'list':
getArticleList();
break;
case 'batch':
batchDownload();
break;
case 'detail':
getArticleDetail();
break;
case 'data':
loadDataList();
break;
}
}
});
// 页面可见性变化时的处理
document.addEventListener('visibilitychange', function() {
if (document.hidden) {
console.log('页面已隐藏');
} else {
console.log('页面已显示');
// 可以在这里刷新任务状态
}
});
// 错误处理
window.onerror = function(message, source, lineno, colno, error) {
console.error('页面错误:', message, '位置:', source + ':' + lineno);
return false;
};
// 控制台欢迎信息
console.log(`
🚀 微信公众号文章爬虫系统 Web界面
====================================
版本: 1.0.0
开发者: AI Assistant
更新时间: 2025-11-27
====================================
💡 提示:
- 按 ESC 键返回首页
- 按 Ctrl+Enter 执行当前操作
- 所有操作都会显示详细进度
====================================
`);

View File

@@ -568,7 +568,7 @@
}; };
// 调用后端登录API // 调用后端登录API
fetch('http://localhost:8000/api/user/login', { fetch('http://localhost:8080/api/user/login', {
method: 'POST', method: 'POST',
headers: { headers: {
'Content-Type': 'application/json', 'Content-Type': 'application/json',
@@ -576,17 +576,18 @@
body: JSON.stringify(loginData) body: JSON.stringify(loginData)
}) })
.then(response => { .then(response => {
console.log('✅ 收到响应,状态码:', response.status);
if (!response.ok) { if (!response.ok) {
// 处理HTTP错误状态码 // 处理HTTP错误状态码
if (response.status === 500) { if (response.status === 500) {
throw new Error('服务器内部错误'); throw new Error('服务器内部错误');
} else if (response.status === 401) { } else if (response.status === 401) {
return response.json().then(data => { return response.json().then(data => {
throw new Error(data.msg || '用户名或密码错误'); throw new Error(data.message || '用户名或密码错误');
}); });
} else if (response.status === 400) { } else if (response.status === 400) {
return response.json().then(data => { return response.json().then(data => {
throw new Error(data.msg || '请求参数错误'); throw new Error(data.message || '请求参数错误');
}); });
} else { } else {
throw new Error('网络响应异常'); throw new Error('网络响应异常');
@@ -595,13 +596,14 @@
return response.json(); return response.json();
}) })
.then(data => { .then(data => {
console.log('✅ 收到数据:', data);
// 检查响应状态 // 检查响应状态
if (data.code === 200) { if (data.success || data.code === 200) {
// 登录成功,存储认证信息 // 登录成功,存储认证信息
const authData = { const authData = {
token: data.token, token: data.data.token,
user_id: data.user_id, user_id: data.data.user_id,
user_info: data.user_info, user_info: data.data.user_info,
loginTime: new Date().toISOString(), loginTime: new Date().toISOString(),
remember: remember remember: remember
}; };
@@ -635,19 +637,19 @@
// 根据错误码显示不同的错误信息 // 根据错误码显示不同的错误信息
switch(data.code) { switch(data.code) {
case 401: case 401:
errorMessage = data.msg || '用户名或密码错误'; errorMessage = data.message || '用户名或密码错误';
break; break;
case 400: case 400:
errorMessage = data.msg || '请求参数错误,请检查输入'; errorMessage = data.message || '请求参数错误,请检查输入';
break; break;
case 404: case 404:
errorMessage = data.msg || '账号不存在'; errorMessage = data.message || '账号不存在';
break; break;
case 500: case 500:
errorMessage = data.msg || '服务器内部错误,请稍后再试'; errorMessage = data.message || '服务器内部错误,请稍后再试';
break; break;
default: default:
errorMessage = data.msg || '登录失败,请检查用户名和密码'; errorMessage = data.message || '登录失败,请检查用户名和密码';
} }
showError(errorMessage); showError(errorMessage);
@@ -657,12 +659,17 @@
}) })
.catch(error => { .catch(error => {
// 捕获网络或其他错误 // 捕获网络或其他错误
console.error('登录请求失败:', error); console.error('登录请求失败:', error);
console.error('错误类型:', error.name);
console.error('错误消息:', error.message);
// 根据错误信息显示具体的错误提示 // 根据错误信息显示具体的错误提示
let errorMessage = '登录失败,请稍后再试'; let errorMessage = '登录失败,请稍后再试';
if (error.message.includes('账号不存在')) { // 检查是否是网络错误
if (error.name === 'TypeError' && error.message.includes('Failed to fetch')) {
errorMessage = '无法连接到服务器,请确保:\n1. API服务器正在运行 (http://localhost:8080)\n2. 检查网络连接';
} else if (error.message.includes('账号不存在')) {
errorMessage = '账号不存在,请先注册'; errorMessage = '账号不存在,请先注册';
} else if (error.message.includes('密码错误')) { } else if (error.message.includes('密码错误')) {
errorMessage = '密码错误,请重新输入'; errorMessage = '密码错误,请重新输入';
@@ -701,15 +708,25 @@
window.addEventListener('load', function() { window.addEventListener('load', function() {
const authData = localStorage.getItem('authData') || sessionStorage.getItem('authData'); const authData = localStorage.getItem('authData') || sessionStorage.getItem('authData');
if (authData) { if (authData) {
const auth = JSON.parse(authData); try {
showSuccess(`欢迎回来,${auth.user_info.username}`); const auth = JSON.parse(authData);
// 检查user_info是否存在且有username
// 如果已经登录,可以选择直接跳转 if (auth.user_info && auth.user_info.username) {
setTimeout(() => { showSuccess(`欢迎回来,${auth.user_info.username}`);
if (confirm('检测到您已登录,是否跳转到首页?')) {
window.location.href = 'frontend.html'; // 如果已经登录,可以选择直接跳转
setTimeout(() => {
if (confirm('检测到您已登录,是否跳转到首页?')) {
window.location.href = 'frontend.html';
}
}, 1000);
} }
}, 1000); } catch (e) {
console.error('解析登录数据失败:', e);
// 清除无效的登录数据
localStorage.removeItem('authData');
sessionStorage.removeItem('authData');
}
} }
}); });

View File

@@ -639,7 +639,7 @@
}; };
// 调用后端注册API // 调用后端注册API
fetch('http://localhost:8000/api/user/register', { fetch('http://localhost:8080/api/user/register', {
method: 'POST', method: 'POST',
headers: { headers: {
'Content-Type': 'application/json', 'Content-Type': 'application/json',
@@ -654,7 +654,7 @@
}) })
.then(data => { .then(data => {
// 检查响应状态 // 检查响应状态
if (data.code === 200) { if (data.success || data.code === 200) {
// 注册成功 // 注册成功
showSuccess('注册成功!正在跳转到登录页面...'); showSuccess('注册成功!正在跳转到登录页面...');
@@ -664,7 +664,7 @@
}, 1500); }, 1500);
} else { } else {
// 注册失败,显示错误信息 // 注册失败,显示错误信息
showError(data.msg || '注册失败,请稍后重试'); showError(data.message || '注册失败,请稍后重试');
registerBtn.textContent = '注册账号'; registerBtn.textContent = '注册账号';
registerBtn.disabled = false; registerBtn.disabled = false;
} }

91
frontend/start_web.bat Normal file
View File

@@ -0,0 +1,91 @@
@echo off
chcp 65001 >nul
title 微信公众号文章爬虫系统 - Web界面
echo.
echo ===============================================
echo 🚀 微信公众号文章爬虫系统
echo Web界面启动
echo ===============================================
echo.
:: 检查Python是否安装
python --version >nul 2>&1
if errorlevel 1 (
echo ❌ 未检测到Python正在尝试其他方法...
goto :use_powershell
) else (
echo ✅ 检测到Python环境
goto :use_python
)
:use_python
echo 📱 使用Python启动Web服务器...
echo 🌐 服务地址: http://localhost:8000
echo ⏰ 启动时间: %date% %time%
echo.
echo 💡 提示: 按 Ctrl+C 停止服务器
echo ===============================================
echo.
echo 🌐 正在打开浏览器...
start http://localhost:8000/frontend.html
echo.
cd /d "%~dp0"
python -m http.server 8000
goto :end
:use_powershell
echo 📱 使用PowerShell启动Web服务器...
echo 🌐 服务地址: http://localhost:8080
echo ⏰ 启动时间: %date% %time%
echo.
echo 💡 提示: 按 Ctrl+C 停止服务器
echo ===============================================
echo.
cd /d "%~dp0"
powershell -Command "
$listener = New-Object System.Net.HttpListener
$listener.Prefixes.Add('http://localhost:8080/')
$listener.Start()
Write-Host '✅ Web服务器已启动: http://localhost:8080'
Write-Host '🌐 正在打开浏览器...'
Start-Process 'http://localhost:8080/frontend.html'
while ($listener.IsListening) {
$context = $listener.GetContext()
$request = $context.Request
$response = $context.Response
$path = $request.Url.LocalPath
if ($path -eq '/') { $path = '/frontend.html' }
$filePath = Join-Path (Get-Location) $path.TrimStart('/')
if (Test-Path $filePath) {
$content = [System.IO.File]::ReadAllBytes($filePath)
$response.ContentType = switch ([System.IO.Path]::GetExtension($filePath).ToLower()) {
'.html' { 'text/html; charset=utf-8' }
'.css' { 'text/css; charset=utf-8' }
'.js' { 'application/javascript; charset=utf-8' }
'.json' { 'application/json; charset=utf-8' }
default { 'text/plain; charset=utf-8' }
}
$response.ContentLength64 = $content.Length
$response.OutputStream.Write($content, 0, $content.Length)
} else {
$response.StatusCode = 404
$response.StatusDescription = 'Not Found'
}
$response.OutputStream.Close()
}
"
:end
echo.
echo ===============================================
echo 服务器已停止运行
echo ===============================================
pause

1855
frontend/user-center.html Normal file

File diff suppressed because it is too large Load Diff

49
启动Web系统.bat Normal file
View File

@@ -0,0 +1,49 @@
@echo off
chcp 65001 >nul
cls
echo ===============================================
echo 🚀 微信公众号文章爬虫 - Web系统启动器
echo ===============================================
echo.
echo 正在启动系统,请稍候...
echo.
:: 启动API服务器后台运行
echo [1/2] 启动 API 服务器...
cd backend\api
start "微信爬虫-API服务器" cmd /c "start_api.bat"
cd ..\..
timeout /t 2 /nobreak >nul
:: 启动前端服务器
echo [2/2] 启动 前端服务器...
cd frontend
start "微信爬虫-前端服务器" cmd /c "start_web.bat"
cd ..
echo.
echo ===============================================
echo ✅ 系统启动完成!
echo ===============================================
echo.
echo 📝 重要提示:
echo.
echo 1⃣ API服务器: http://localhost:8080
echo - 提供后端接口服务
echo - 窗口标题: "微信爬虫-API服务器"
echo.
echo 2⃣ 前端界面: http://localhost:8000
echo - Web操作界面
echo - 窗口标题: "微信爬虫-前端服务器"
echo.
echo ⚠️ 请不要关闭这两个窗口!
echo.
echo 💡 使用说明:
echo - 浏览器会自动打开前端界面
echo - 如未自动打开,请手动访问 http://localhost:8000
echo - 使用完毕后,关闭两个服务器窗口即可
echo.
echo ===============================================
pause