Civitai Feed Deduplicator

Removes duplicate images/videos (same author, file name, dimensions, file size and blurhash) from Civitai feeds before they are rendered, so the masonry grid stays intact. Tested mainly on user profile pages.

Installa questo script ?

Fai una domanda, pubblica una recensione, oppure segnala lo script.

Autore

V_01

Installazioni giornaliere

Installazioni totali

Valutazione

0 0 0

Versione

1.3.1

Creato il

07/10/2026

Aggiornato il

07/10/2026

Dimensione

12,5 KB

Licenza

MIT

Applica a

Description

Code written by Claude (Anthropic), tested by me.

Please report spam: this script only hides duplicates on your side. If you see an account flooding feeds with identical uploads, please report it to Civitai moderation.

How it works

Civitai loads feeds through tRPC (image.getInfinite). The response is { result: { data: <payload> } } where is written by one of two serializers, depending on which server pool answered (the site migrates superjson -> devalue pool by pool, and a single response can fall back to superjson):

devalue : a STRING holding a flat array. arr[0] is the root object, every value inside objects is an INDEX into the same array, -1 means undefined.
superjson: an OBJECT { json, meta }. Dates etc. are described by paths in meta ("items.3.createdAt"), so removing items requires re-numbering those paths.

The script wraps window.fetch, removes duplicate items from the items list of either format and hands the page a normal response. Removing items before the page renders them (instead of hiding them with CSS) keeps the virtualized masonry grid from leaving empty gaps.

Two items are duplicates when author id, file name, width, height and blurhash are all identical. The first copy in feed order is kept.

Pagination: every response carries nextCursor. A later request that contains that cursor (in the URL or in a POST body) continues the same "chain" and shares its memory of what was already shown. A request without a known cursor starts a fresh chain. This does not depend on how the request input is encoded, and keeps different feeds on the same page independent.

Caveats

Tested primarily on user profile pages. Other feeds (Most Reactions, model pages, collections) may differ and are not verified.
Only the image.getInfinite procedure is processed. Feeds served by other procedures are left untouched.
It relies on Civitai's internal formats (not a public API). If the site changes them, the script silently does nothing. To debug, set DEBUG = true at the top of the script and look for "[dedupe]" lines in the browser console.
The superjson branch and the batch skip were tested only on synthetic data; so far only devalue responses have been observed on the live site.
Batched requests (?batch=1, used by the site for some logged-in cohorts) come back as a stream and are NOT processed.
Pages whose first batch of items is embedded in the HTML by the server (SSR hydration) may never go through fetch, so that first batch would not be processed. Items loaded afterwards are.
It suppresses any repeated upload from the same author, not only spam. If someone deliberately posts the same file twice, only the first one is shown.

Privacy

The script makes no network requests of its own and sends nothing anywhere. It only filters Civitai's own API responses inside your browser.