Ubuntu
229 lines
5.5 KiB
Markdown
229 lines
5.5 KiB
Markdown
# Anti-AI Detector - 去AI化技能
|
||
|
||
检测并去除AI生成文本中的AI痕迹，支持中文、英文和小红书风格优化。
|
||
|
||
## 🎯 功能特性
|
||
|
||
- ✅ **中文去AI化** - 去除中文AI文本的典型痕迹
|
||
- ✅ **英文去AI化** - 基于Stop Slop的英文优化
|
||
- ✅ **小红书风格** - 针对小红书平台的特殊优化
|
||
- ✅ **自动检测** - 智能识别文本语言和场景
|
||
- ✅ **IT项目集成** - 直接集成到IT资讯自动化流程
|
||
|
||
## 🚀 快速开始
|
||
|
||
### 命令行使用
|
||
|
||
```bash
|
||
# 处理单段文本
|
||
python3 humanize.py "你的AI生成文本"
|
||
|
||
# 处理文件
|
||
python3 humanize.py -f input.txt -o output.txt
|
||
|
||
# 指定风格
|
||
python3 humanize.py -t zh "中文文本"
|
||
python3 humanize.py -t en "English text"
|
||
python3 humanize.py -t xhs "小红书文本"
|
||
|
||
# 分析AI痕迹（不处理）
|
||
python3 humanize.py --analyze "文本"
|
||
```
|
||
|
||
### Python API
|
||
|
||
```python
|
||
from humanize import AntiAIDetector
|
||
|
||
detector = AntiAIDetector()
|
||
|
||
# 中文去AI化
|
||
result = detector.humanize_cn("值得注意的是...")
|
||
|
||
# 英文去AI化
|
||
result = detector.humanize_en("Here's the thing...")
|
||
|
||
# 小红书风格
|
||
result = detector.humanize_xhs("姐妹们！这个神器绝绝子！")
|
||
```
|
||
|
||
## 📋 检测规则
|
||
|
||
### 中文AI痕迹
|
||
|
||
| 类型 | 示例 | 替换为 |
|
||
|------|------|--------|
|
||
| 标志性短语 | "值得注意的是" | "" |
|
||
| | "我们可以看到" | "" |
|
||
| | "不难发现" | "可以看出" |
|
||
| | "显而易见" | "很明显" |
|
||
| 结构模式 | "首先...其次...最后" | "第一...第二...最后" |
|
||
| | "综上所述" | "总结来说" |
|
||
| 过度修饰 | "非常" | "很" |
|
||
| | "极其" | "特别" |
|
||
| 模板化 | "随着...的发展" | "" |
|
||
| | "在当今社会" | "" |
|
||
|
||
### 英文AI痕迹（基于Stop Slop）
|
||
|
||
| 类型 | 示例 | 替换为 |
|
||
|------|------|--------|
|
||
| 标志性短语 | "Here's the thing" | "" |
|
||
| | "Let that sink in" | "" |
|
||
| | "The uncomfortable truth is" | "Truth is" |
|
||
| | "Let me be clear" | "" |
|
||
| 过度动词 | "navigate" | "handle" |
|
||
| | "unpack" | "explain" |
|
||
| | "leverage" | "use" |
|
||
| 固定结构 | "Not because X. Because Y." | "Not from X, but from Y." |
|
||
| | "That's it. That's the..." | "" |
|
||
|
||
### 小红书风格优化
|
||
|
||
| 类型 | 示例 | 替换为 |
|
||
|------|------|--------|
|
||
| 营销词汇 | "绝绝子" | "很好" |
|
||
| | "YYDS" | "很棒" |
|
||
| | "闭眼入" | "推荐" |
|
||
| | "神器" | "好用" |
|
||
| 称呼 | "姐妹们" | "大家" |
|
||
| 术语 | "干货" | "内容" |
|
||
| | "码住" | "收藏" |
|
||
|
||
## 🔧 集成到IT资讯系统
|
||
|
||
### 方式1：直接调用
|
||
|
||
```python
|
||
from it_integration import humanize_for_it_news
|
||
|
||
# 翻译后处理
|
||
translated = ai_processor.translate_article(content)
|
||
humanized = humanize_for_it_news(translated, content_type='translation')
|
||
|
||
# 总结后处理
|
||
summary = ai_processor.summarize_article(content)
|
||
humanized = humanize_for_it_news(summary, content_type='summary')
|
||
```
|
||
|
||
### 方式2：完整管道
|
||
|
||
```python
|
||
from it_integration import ITContentHumanizer
|
||
|
||
humanizer = ITContentHumanizer()
|
||
|
||
# 完整处理
|
||
content = {
|
||
'title': '原标题',
|
||
'translated': '翻译内容',
|
||
'summary': '总结内容',
|
||
'platform': 'xiaohongshu' # 或 'email'
|
||
}
|
||
|
||
result = humanizer.full_pipeline(content)
|
||
print(result['title'])
|
||
print(result['summary'])
|
||
```
|
||
|
||
### 方式3：风险分析
|
||
|
||
```python
|
||
from it_integration import ITContentHumanizer
|
||
|
||
humanizer = ITContentHumanizer()
|
||
risk = humanizer.analyze_risk(text)
|
||
|
||
print(f"风险等级: {risk['risk_level']}") # low/medium/high
|
||
print(f"得分: {risk['score']}")
|
||
print(f"建议: {risk['suggestions']}")
|
||
```
|
||
|
||
## 📝 修改IT资讯处理器
|
||
|
||
在 `process_articles.py` 中添加：
|
||
|
||
```python
|
||
from skills.anti_ai_detector.it_integration import ITContentHumanizer
|
||
|
||
class ArticleProcessor:
|
||
def __init__(self):
|
||
self.ai_processor = AIProcessor()
|
||
self.humanizer = ITContentHumanizer() # 新增
|
||
|
||
def process_single_article(self, article_id: int):
|
||
# ... 原有代码 ...
|
||
|
||
# AI处理后，添加去AI化
|
||
translated = self.ai_processor.translate_article(content)
|
||
translated = self.humanizer.process_translation(translated) # 新增
|
||
|
||
summary = self.ai_processor.summarize_article(content)
|
||
summary = self.humanizer.process_summary(summary) # 新增
|
||
|
||
# ... 后续代码 ...
|
||
```
|
||
|
||
## 🎨 效果示例
|
||
|
||
### 中文示例
|
||
|
||
**原文：**
|
||
```
|
||
值得注意的是，这项技术非常重要。我们可以看到，随着AI的发展，
|
||
越来越多的应用场景被开发出来。首先，它提高了效率；其次，
|
||
它降低了成本；最后，它改变了工作方式。
|
||
```
|
||
|
||
**去AI化后：**
|
||
```
|
||
这项技术非常重要。
|
||
越来越多的应用场景被开发出来。第一，它提高了效率；第二，
|
||
它降低了成本；最后，它改变了工作方式。
|
||
```
|
||
|
||
### 英文示例
|
||
|
||
**原文：**
|
||
```
|
||
Here's the thing: building great products is hard.
|
||
Not because they lack talent. Because they lack clarity.
|
||
Let that sink in.
|
||
```
|
||
|
||
**去AI化后：**
|
||
```
|
||
building great products is hard.
|
||
Not from they lack talent, but from they lack clarity.
|
||
```
|
||
|
||
### 小红书示例
|
||
|
||
**原文：**
|
||
```
|
||
姐妹们！今天给大家分享一个宝藏神器！🔥🔥🔥
|
||
强烈推荐大家码住这个干货！
|
||
```
|
||
|
||
**优化后：**
|
||
```
|
||
大家！今天给大家分享一个不错好用！🔥🔥🔥
|
||
推荐大家收藏这个内容！
|
||
```
|
||
|
||
## 📊 性能指标
|
||
|
||
- 中文检测：30+ 种AI痕迹模式
|
||
- 英文检测：40+ 种Stop Slop模式
|
||
- 小红书优化：20+ 种营销词汇
|
||
- 处理速度：~1000字/秒
|
||
|
||
## 🔗 参考
|
||
|
||
- Stop Slop: https://github.com/hardikpandya/stop-slop
|
||
- Inspired by: Hardik Pandya
|
||
|
||
## 📝 License
|
||
|
||
MIT
|