How the data is built.

Censorship measurement

Voidly brings together its own observations and external measurement sources including OONI, CensoredPlanet, and IODA. Their coverage and methods differ. A public country-index entry is not proof that a Voidly-operated probe sits in that country.

Evidence records distinguish network observations, suspected interference, and corroborated findings. Use the source links, measurement window, and confidence information for the specific record. A connectivity anomaly alone does not establish deliberate censorship.

The current Voidly methodology describes the source mix, model evaluation, and known limitations. Forecast research is published with evaluation caveats; it should not be presented as a validated prediction of new shutdowns. Data licenses vary between Voidly-authored annotations and upstream material; consult the data and reuse documentation.

Coverage definitions explain the public index and historical archive. Earlier technical articles remain dated records of their original designs, rather than a statement of current production behavior.

Federal Regulatory Data Hub — ingest methodology

The 229 catalogued datasets in the Federal Regulatory Data Hub share a common ingest pattern across government sources, regulators/SROs, and derived indexes. Each catalog entry identifies its publisher or documented source system, and derived indexes are labeled. Routes retain provenance fields where their source and response schema provide them.

1. Documented source fetch. Data is pulled from documented government, regulator, and SRO endpoints such as EDGAR, FINRA, openFDA, OFAC, EPA ECHO, SAM.gov, USAspending, FinCEN, FDIC BankFind, CMS, NIST NVD, and CISA KEV.
2. Source-specific scheduled refresh. Enforcement feeds, periodic registries, and historical archives follow different cadences based on how their publishers release data.
3. Normalization. Records are normalized into per-vertical SQLite tables on Cloudflare D1. Field names are standardized; dates are parsed to ISO 8601; dollar amounts are stored as integers (cents); entity identifiers (CIK, UEI, NPI, DUNS, LEI, ticker) are extracted into indexed columns.
4. Entity bridge. An entity_master table joins supported records by CIK, ticker, UEI, LEI, DUNS, and NPI. This is the cross-agency join layer for the source systems wired into each entity route.
5. Machine-readable publication. Dataset landing pages expose metadata, samples, and query routes. Selected record routes also publish HTML, Markdown, and Schema.org JSON-LD.

Reuse terms vary by source. US federal employee-authored works are generally public domain under 17 U.S.C. § 105, while non-federal and SRO source material retains its source terms. AI Analytics-authored metadata and normalization are CC0 only where explicitly stated and applicable. See api.ai-analytics.org/coverage for live per-dataset indexed row counts. For a full technical write-up of the D1 schema, ingest architecture, and vertical sharding strategy, see Building the Federal Regulatory Data Hub on Cloudflare D1 →

Accountability datasets — common method

The sixteen Voidly accountability datasets share one documentary method. Automated source pipelines and curated records have dataset-specific build checks:

1. Primary public sources only. Government files (ICE detention statistics, USDA AFIDA, Department of Education Section 117, EIA-860, OFAC program pages, official legal texts), regulators' designation feeds, or the subject entity's own filings. The censorship index is the one exception by design: it is direct network measurement, documented above.
2. Shared publication data. Automated pipelines rebuild their source records with checked-in scripts. Curated collections such as SpyLedger procurement use reviewed records with source receipts. Published JSON, pages, and manifest counts derive from the same collection in one build.
3. Schema gates. Automated ingestion uses dataset-specific source-column and privacy checks. Curated records validate approved fields, source references, dates, and publication parity; their wording still requires editorial review. The public accountability collections cover institutions and aggregates, with each collection's checks documented in its methodology.
4. The evidence ladder. An entity is named only where a government record or the entity's own filing names it, and every attribution carries its evidence tier and source link. Where the record stops, the dataset says so instead of guessing.
5. Machine parity. Every dataset ships as keyless static JSON, enumerated in one manifest at /voidly/datasets.json; counts on the pages and in the JSON come from the same build.

The rules themselves — zero personal data, records-not-allegations, corrections and right of reply — are stated in full at /standards/. Each dataset's landing page carries its own Method & caveats section with the source list, build date, and the dataset-specific gates.