Use Sciverse as a Scientific RAG Data Source

Use Sciverse as the retrieval backend for a RAG pipeline and provide trustworthy scientific evidence to LLMs

Scenario

Developers building scientific Q&A systems or RAG applications need evidence from authoritative literature to ground LLM answers and reduce hallucinations.

Estimated calls

~5-15 API calls per RAG query

Tools

agentic-searchcontent

Pipeline

agentic-search→ chunks + scores→ score 过滤→ LLM grounded answer

Input example

User query:
"What are the latest advances in solid-state electrolytes for lithium batteries?"

Output example

Answer: Recent work focuses on sulfide electrolytes, oxide electrolytes, and polymer-inorganic composites...

Evidence:
1. doc_id=xxx, score=0.87, title=...
2. doc_id=yyy, score=0.82, title=...

Agent Prompt example

You are a scientific RAG assistant. Retrieve evidence with Sciverse before answering. Only answer from retrieved evidence. If evidence is insufficient, say so clearly and suggest a follow-up search.

Implementation steps

Step 1: Set up the environment

Install dependencies and configure the Sciverse token

!pip install httpx openai
import os
os.environ["SCIVERSE_API_TOKEN"] = "sv-your-token-here"  # 替换为你的真实值
import os
os.environ["OPENAI_API_KEY"] = "sk-..."  # 替换为你的真实值

Step 2: Call agentic-search for evidence

Send the user question to agentic-search and collect candidate passages

import os
import asyncio
import httpx

BASE = "https://api.sciverse.space"
TOKEN = os.environ["SCIVERSE_API_TOKEN"]
HEADERS = {"Authorization": f"Bearer {TOKEN}"}

async def sciverse_retrieve(query: str, top_k: int = 10):
    """调用 agentic-search 获取相关文献片段"""
    async with httpx.AsyncClient(timeout=30) as client:
        resp = await client.post(
            f"{BASE}/agentic-search",
            headers=HEADERS,
            json={"query": query, "top_k": top_k}
        )
        resp.raise_for_status()
        data = resp.json()
        return [
            {"text": h.get("chunk", ""), "doc_id": h["doc_id"],
             "title": h["title"], "score": h["score"]}
            for h in (data.get("hits") or [])
        ]

Step 3: Filter evidence

Keep high-scoring, diverse evidence and remove weak or duplicate snippets

def filter_evidence(hits: list, threshold: float = 0.65) -> list:
    """过滤低分片段，按 score 降序排列"""
    filtered = [h for h in hits if h["score"] >= threshold]
    return sorted(filtered, key=lambda x: x["score"], reverse=True)

async def main():
    hits = await sciverse_retrieve("mRNA LNP delivery system improvements")
    top_evidence = filter_evidence(hits, threshold=0.65)
    print(f"Retrieved {len(hits)} chunks, filtered to {len(top_evidence)} high-quality")
    return top_evidence

top_evidence = await main()

Step 4: Generate a grounded answer (optional enhancement)

Generate an answer that cites the filtered evidence

from openai import OpenAI

client = OpenAI()  # 自动读取 OPENAI_API_KEY

context = "\
\
".join([
    f"[{i+1}] {e['title']}\
{e['text']}"
    for i, e in enumerate(top_evidence[:5])
])

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "基于提供的文献证据回答问题。每个论点用 [编号] 标注来源。如果证据不足，说明无法确定。不要编造未在证据中出现的信息。"},
        {"role": "user", "content": f"问题：mRNA LNP 递送系统最新改进？\
\
证据：\
{context}"}
    ]
)
print(resp.choices[0].message.content)

Notes

Keep the retrieval query close to the user question.
Filter low-score or duplicate chunks before sending evidence to the LLM.
Make missing evidence explicit instead of filling gaps from model memory.
Preserve doc_id, title, score, and offset for every cited passage.

FAQ

Sciverse 在 RAG 中负责什么?

负责提供可信科学证据检索和全文上下文，不负责向量库编排或最终回答生成。

为什么不用普通网页搜索?

Sciverse 返回论文级、证据级数据，更适合需要引用和溯源的科研回答。

是否需要自建向量库?

可以按业务需要自建，但 Sciverse 可作为实时科学证据召回层。

RAG 输出要保留哪些字段?

建议保留 doc_id、title、quote、offset、year 和检索 query。

Next steps

Need an API key?

Create one in Console > Tokens.The same API key works for enabled Sciverse, DianShi, and Skills capabilities, with starter quota available according to account permissions.

Open console