NanoDAPFAM

Overview

NanoDAPFAM is the compact Nano set for DAPFAM, a domain-aware patent-family retrieval benchmark. It evaluates citation-linked prior-art retrieval at the patent-family level. The group contains eighteen variants formed by three domain conditions (All, In, and Out), two query representations (title-abstract or title-abstract-claims), and three target representations (title-abstract, title-abstract-claims, or full text).

The group is useful because it separates ordinary lexical patent similarity from cross-domain prior-art retrieval. Same-domain and all-domain variants give retrievers many technical anchors: components, materials, methods, and claim phrases. OUT-domain variants remove shared IPC3 technical classes, so the model must retrieve cited families related by transferable mechanisms or problem-solution patterns rather than by the same surface vocabulary. BM25, dense retrieval, and reranking_hybrid all reveal different parts of that domain gap.

What This Group Measures

DAPFAM: A Domain-Aware Family-level Dataset to benchmark cross domain patent retrieval introduces a patent-family retrieval benchmark with citation-based relevance judgments and explicit domain partitions. The source benchmark aggregates patents at family level to reduce international duplicate publications and uses IPC3 overlap to distinguish same-domain from cross-domain retrieval.

NanoDAPFAM preserves the DAPFAM design in compact 200-query splits. Each split uses the same high-level prior-art retrieval task but changes the domain condition and patent text fields. This makes the group a controlled probe of three factors: whether positives are same-domain or cross-domain, whether the query includes claims, and whether the target is a compact summary or a very long patent text.

Task Families

• All-domain prior-art retrieval: All variants include both same-IPC3 and cross-IPC3 positives, measuring overall citation-linked patent-family retrieval.
• In-domain prior-art retrieval: In variants keep same-domain positives, where shared technical vocabulary is often available.
• Out-domain prior-art retrieval: Out variants keep cross-domain positives, where relevance depends more on transferred mechanisms, materials, or problem-solution analogies.
• Representation comparisons: each domain condition is evaluated with title-abstract and title-abstract-claims queries against title-abstract, title-abstract-claims, and full-text targets.

Dataset Shape

NanoDAPFAM contains 18 task pages, 3,600 queries, 180,000 split-local documents, and 49,879 positive qrel rows. Each split has 200 queries and 10,000 candidate documents. All variants are multi-positive, but density changes by domain: All-domain variants average about 20 positives per query, In-domain variants about 15, and Out-domain variants about 6.

Text length is a core variable. Title-abstract queries average under 800 characters, while title-abstract-claims queries range from about 8,300 to 9,300 characters. Target documents range from short title-abstract records around 778 characters to full-text patent-family documents around 69,000 to 72,000 characters. The group should therefore be read as a matrix of domain difficulty and representation length, not as eighteen independent tasks.

Retrieval Behavior

BM25 Profile

BM25 is much stronger on All and In variants than on Out variants. In the same-domain setting, shared IPC3 areas provide repeated technical terms, components, material names, and claim language. In OUT-domain retrieval, those anchors are weaker or absent, so BM25 must rely on partial mechanism overlap.

Representation also changes sparse behavior. Claim-bearing targets often expose more components and operations than title-abstract targets. Full text gives even more vocabulary but adds large amounts of boilerplate and unrelated legal or descriptive context. A higher BM25 score on long targets does not always mean better semantic retrieval; it may mean more lexical chances.

Dense Profile

Dense retrieval is the best profile for nearly every NanoDAPFAM variant in the current metadata. It improves most clearly in OUT-domain tasks, where the positive may share an abstract mechanism with the query even when patent-field vocabulary differs. Dense retrieval also benefits title-abstract targets, where there is less text for exact matching.

Dense performance should still be interpreted carefully. Patent retrieval depends on exact technical distinctions, claim scope, and family-level invention identity. A semantically related patent can be wrong if it solves a different problem or lacks the cited technical relation.

Reranking Hybrid Profile

reranking_hybrid often falls between BM25 and dense in nDCG@10, but it is useful for candidate generation. Patent retrieval needs both exact technical anchors and broader mechanism matching. In several title-abstract and full-text variants, the hybrid pool can preserve candidates found by either signal.

For reranker experiments, OUT-domain variants are the most important stress test. If the first-stage candidate pool misses cross-domain positives, reranking cannot recover the analogy or mechanism relation.

Task Summary

Interpretation Notes for Model Researchers

NanoDAPFAM is best interpreted by domain condition first. All and In variants measure patent-family retrieval when the target is usually in or near the same technical area. Out variants measure harder cross-domain prior-art retrieval, where models need analogy and mechanism transfer. A strong model should reduce the Out-domain gap without merely memorizing patent families.

Representation effects should be interpreted second. Claims add legal and component detail; full text adds enormous context. Better performance on full text may reflect useful mechanism evidence, but it may also reflect more opportunities for lexical overlap. Comparing title-abstract targets with claim-bearing and full-text targets helps separate concise semantic matching from long-document term coverage.

Training and Leakage Notes

Useful training data includes patent-family citation retrieval, prior-art search pairs, patent semantic similarity, cross-IPC citation prediction, patent analogy retrieval, and field-aware training over titles, abstracts, claims, and full descriptions. Hard negatives should include same-IPC patents that share terminology but are not cited, plus cross-domain patents that share surface terms without the relevant mechanism.

Exclude NanoDAPFAM evaluation family IDs, qrels, positive target families, same-family duplicate publications, and near-duplicate patent publications from other jurisdictions. Family-level aggregation is important: using another member of the same patent family can leak the invention.

Source Reference Table

Metadata Summary

Task Metadata Summary