/nai-paper-ingest — filing a GitHub-linked paper into the research library

You type /nai-paper-ingest <issue-number-or-url>. The command will “Read one GitHub issue on this repo, check whether it links to a research paper (or to a web page on a short list of sources worth keeping as they are), and if so file that document into the personal research library for the next library-wiki batch ingest to pick up.” That is the whole job: “This command never summarizes or tags the paper itself — that stays library-wiki's own job, run separately.” The orchestrator does the cheap detection itself; everything that touches the network or the library happens in paper-ingester, which “Resolves a candidate link from a GitHub issue to its actual PDF (direct link, arXiv abstract-page transform, or one hop via a landing page), or renders a listed web page to PDF with a headless browser, downloads it, and files it into the right work/<Topic>/ folder of the personal research library via the Google Drive API. Never writes to the wiki itself — that stays library-wiki's own batch-ingestion job.”

%%{init: {'theme':'neutral', 'flowchart':{'htmlLabels':false,'nodeSpacing':45,'rankSpacing':55}}}%%
flowchart TD
    H["Human types<br/>/nai-paper-ingest &lt;issue-number-or-url&gt;"] --> C

    subgraph L1 ["1 - Orchestrator (.claude/commands)"]
        C["/nai-paper-ingest<br/>resolve the number, gh issue view,<br/>detect candidates (paper class before web-source class)"]
    end

    C -->|"no number given / issue not found"| E1["STOP — ask the user"]
    C -->|"no candidate of either class"| E2["STOP — no subagent dispatched,<br/>no comment posted"]
    C -->|"paper URLs + web-source URLs"| A

    subgraph L2 ["2 - Subagent (.claude/agents)"]
        A["paper-ingester<br/>topics once, then per URL:<br/>resolve or render &rarr; download &rarr; read &rarr; route &rarr; dedupe &rarr; upload"]
    end

    A --> T1
    A --> T2
    A --> T3
    subgraph L3 ["3 - Tools (the only route out)"]
        T1["Bash: curl download, render.mjs,<br/>drive.mjs, temp-file cleanup"]
        T2["WebFetch: one landing page's<br/>PDF link, nothing broader"]
        T3["Read: pages 1-2 of the PDF"]
    end

    T1 ==> PLACE["work/&lt;Topic&gt;/&lt;stem&gt;.pdf<br/>Google Drive API, outside this repo"]

    A -.->|"one line per URL"| C
    A -.->|"a single blocked: line"| CH["relayed in chat only,<br/>never posted to the issue"]
    C ==> CMT["ONE gh issue comment<br/>never closes, labels, or edits the issue"]
    C -.->|"last, never blocking"| LED["record-run.mjs<br/>one run instance in nai-onto"]

    classDef orch fill:#dcefe9,stroke:#0f6f63,stroke-width:2px
    classDef tool fill:#f4f1ea,stroke:#8a7a5a
    classDef human fill:#fdf3e0,stroke:#8a5a00,stroke-width:2px
    classDef agent fill:#fff,stroke:#555
    class C orch
    class T1,T2,T3 tool
    class H,E1,E2,CH,CMT,PLACE,LED human
    class A agent

Detecting a candidate, before anything is dispatched

Step 0 takes the argument in either form: “$ARGUMENTS is a bare issue number or a pasted issue URL. Extract the number with /issues/(\d+) if a URL was given. Neither form present → STOP and ask the user for an issue number or URL.” Step 1 fetches it with gh issue view <n> --json number,title,body,url,state, and “Issue not found → STOP and tell the user the number didn't resolve; ask them to check it.”

Step 2 scans the body for URLs. “Two classes of candidate exist; check them in this order, so a direct PDF on a web-source host is a paper candidate, never a web source.”

The web-source list is only half of that gate: “A web-source host must also be allowed in the cloud environment's network access before a scheduled run can reach it (docs/machine-setup.md, research library section); the list here is the second gate, so adding a host means both.” And when the scan comes up empty: “No candidate of either class found → STOP. Tell the user "no paper link found in issue #<n>" and go no further — no subagent dispatched, no comment posted.”

The one subagent

The tools: column is copied exactly from the agent file's frontmatter. The command file's step 3 names it as the dispatch: “Invoke the Agent tool for paper-ingester with the issue number, its title, the list of paper candidate URLs, and the list of web-source candidate URLs found in step 2 (either list may be empty, not both).” It is told to “resolve, download or render, and file each one independently; it does not stop early just because one candidate fails.”

Each of the three tools is granted for a named purpose and nothing else. Bash “is granted for exactly four purposes: curl-downloading a resolved PDF URL to a system temp path, running the render helper node scripts/paper-upload/render.mjs (below) for a web-source candidate, running the Drive helper node scripts/paper-upload/drive.mjs (below), and cleanup of the temp file afterward.” WebFetch “is granted for exactly one purpose: fetching a candidate landing page to look for the PDF link it contains. Not for general research, not for verifying claims, not for anything beyond finding that one link.” Read opens the downloaded file “with pages: "1-2" — the title, authors, and abstract are normally on the first page or two; you don't need the whole document.” The issue's own title travels along as “context only — used for the topic-inference fallback in step 4 below, never as a source of the PDF itself”.

The two helpers: the only way into the library

The library “lives in a Google Drive shared drive (the same one the H:\Geteilte Ablagen\knowledge\ mount on the user's own machine points at); reach it over the Drive API, not the filesystem. Its work/ folder has Drive file id 1C_wUiLmoWguZaI0f56MjaaJNINoSiZyG — that is where you place files, by parentId.” The agent reaches it through two Node scripts, each of which prints one line and always exits 0.

The render helper “is your only way of turning a web page into a PDF. It fetches the page once (and refuses anything that is not a 2xx HTML answer, so a 403 or 404 page is never printed as if it were the source), then prints it with a headless Chromium-family browser to the temp path you name.” The Drive helper “is your only way into the library. It authenticates with the GDRIVE_SA_JSON service account, streams the PDF from disk (so file size does not matter, and the bytes stay out of your own output), and prints one line, always exit 0”.

How one candidate is filed

Before any download, the environment is checked once. If “The helper's topics call prints ERROR: ... (for example GDRIVE_SA_JSON must be set, a token failure, or a Drive permission error) → STOP entirely and report blocked: work folder (Drive id 1C_wUiLmoWguZaI0f56MjaaJNINoSiZyG) not reachable via Google Drive API: <the ERROR text>.” The order is deliberate: “Make the topics call once, before the first download, so a broken environment costs no downloads.”

The guardrails, each from the file that enforces it

What the output looks like

Inside this repo, nothing. The agent file's output section is one sentence long on that point: “Nothing written inside this repo. The only change is the one .pdf uploaded into the matching Topic subfolder of work/ (Drive id 1C_wUiLmoWguZaI0f56MjaaJNINoSiZyG) per successfully filed candidate — plus the local temp file, which is always deleted before you return, win or lose.”

Back to the orchestrator goes “One line per candidate URL, in the exact vocabulary used above:” filed: work/<Topic>/<stem>.pdf | filed (web page): work/<Topic>/<stem>.pdf | duplicate: <existing filename> — <url> | unresolved: no PDF link on landing page — <url> | download failed: <url> | download failed: <message> — <url> | unfiled: no topic match — <url> | upload failed: <message> — <url>. The one exception: “If you hit the blocked: case in Missing input instead (the work/ folder itself unreachable), that one stops the whole run: return only that line.”

The orchestrator turns those lines into one gh issue comment <n> --body "<summary>". The command file's example starts “Paper ingest:” and lists “- filed: work/AI/Smith_Attention_2401.12345.pdf”; a filed web page carries its linked PDFs beneath it, and “The also linked on the page list is informational: those PDFs were not downloaded, and the user decides which of them deserve an issue of their own.” The chat report is the same text: “Tell the user the same summary shown in the comment, plus the comment's URL.”

Last, and never blocking: “After the report, record this run in the NAI OS ontology (udopnink/nai-onto, ontologies/nai-os, class run) with one Bash call:” node scripts/onto-sync/record-run.mjs nai-paper-ingest "<issue-number>" {outcome}. See the research library concept page for the library this system writes into.